Port New Model to TensorRT - wom-ai/inference_results_v1.0 GitHub Wiki

Analysis on Reference Code

code/ssd-mobilenet/tensorrt/SSDMobileNet.py

# Do graph surgery on pb graph and convert to UFF.                                                                                                                                                                                                                          
uff_model = uff.from_tensorflow_frozen_model(self.model_path, preprocessor="code/ssd-mobilenet/tensorrt/SSDMobileNet.py") 

• After calling preprocess() uff.from_tensorflow_frozen_model() shows the following prints

NOTE: UFF has been tested with TensorFlow 1.15.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.9
=== Automatically deduced input nodes ===
[name: "Input"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 1
      }
      dim {
        size: 3
      }
      dim {
        size: 300
      }
      dim {
        size: 300
      }
    }
  }
}
]
=========================================

=== Automatically deduced output nodes ===
[name: "Postprocessor"
op: "NMS_OPT_TRT"
input: "BoxPredictor_0/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_1/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_2/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_3/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_4/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_5/BoxEncodingPredictor/BiasAdd"
input: "concat_priorbox"
input: "BoxPredictor_0/ClassPredictor/BiasAdd"
input: "BoxPredictor_1/ClassPredictor/BiasAdd"
input: "BoxPredictor_2/ClassPredictor/BiasAdd"
input: "BoxPredictor_3/ClassPredictor/BiasAdd"
input: "BoxPredictor_4/ClassPredictor/BiasAdd"
input: "BoxPredictor_5/ClassPredictor/BiasAdd"
attr {
  key: "backgroundLabelId_u_int"
  value {
    i: 0
  }
}
attr {
  key: "confSigmoid_u_int"
  value {
    i: 1
  }
}
attr {
  key: "confSoftmax_u_int"
  value {
    i: 0
  }
}
attr {
  key: "confidenceThreshold_u_float"
  value {
    f: 0.30000001192092896
  }
}
attr {
  key: "inputOrder_u_ilist"
  value {
    list {
      i: 0
      i: 7
      i: 6
    }
  }
}
attr {
  key: "isNormalized_u_int"
  value {
    i: 1
  }
}
attr {
  key: "keepTopK_u_int"
  value {
    i: 100
  }
}
attr {
  key: "nmsThreshold_u_float"
  value {
    f: 0.6000000238418579
  }
}
attr {
  key: "numClasses_u_int"
  value {
    i: 91
  }
}
attr {
  key: "numLayers_u_int"
  value {
    i: 6
  }
}
attr {
  key: "shareLocation_u_int"
  value {
    i: 1
  }
}
attr {
  key: "topK_u_int"
  value {
    i: 100
  }
}
attr {
  key: "varianceEncodedInTarget_u_int"
  value {
    i: 0
  }
}
]
==========================================

Using output node Postprocessor
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_OPT_TRT yet.
Converting Postprocessor as custom op: NMS_OPT_TRT
Warning: No conversion function registered for layer: GridAnchor_TRT yet.
Converting MultipleGridAnchorGenerator as custom op: GridAnchor_TRT
DEBUG [/usr/local/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:143] Marking ['Postprocessor'] as outputs
No. nodes: 263

Input Order Match

• Postprocessor's inputOrder should be matched to BoxPredictor_0/BoxEncodingPredictor, BoxPredictor_0/ClassPredictor, concat_priorbox

Postprocessor = gs.create_plugin_node(name="Postprocessor", op="NMS_OPT_TRT",
                                      shareLocation=1,
                                      varianceEncodedInTarget=0,
                                      backgroundLabelId=0,
                                      confidenceThreshold=0.3,
                                      nmsThreshold=0.6,
                                      topK=100,
                                      keepTopK=100,
                                      numClasses=91,
                                      inputOrder=[0, 7, 6],
                                      confSigmoid=1,
                                      confSoftmax=0,
                                      isNormalized=1,
                                      numLayers=6)

[name: "Postprocessor"
op: "NMS_OPT_TRT"
input: "BoxPredictor_0/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_1/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_2/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_3/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_4/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_5/BoxEncodingPredictor/BiasAdd"
input: "concat_priorbox"
input: "BoxPredictor_0/ClassPredictor/BiasAdd"
input: "BoxPredictor_1/ClassPredictor/BiasAdd"
input: "BoxPredictor_2/ClassPredictor/BiasAdd"
input: "BoxPredictor_3/ClassPredictor/BiasAdd"
input: "BoxPredictor_4/ClassPredictor/BiasAdd"
input: "BoxPredictor_5/ClassPredictor/BiasAdd"

• NMS input organization check the number 6 in range(6)and 7 in nms_layer.set_input(i + 7, tensor)

    def postprocess(self, replace_relu6=False):

        ...

        # Connect NMS input to manually merged convolution layer
        for i in range(0, 6):
            tensor = mergeLocConfConv(self.network, i)
            nms_layer.set_input(i, tensor)
            nms_layer.set_input(i + 7, tensor)

code/ssd-mobilenet/tensorrt/SSDMobileNet_for_ssd_mobilenet_v2_coco_2018_03_29.py

• Postprocessor's inputOrder is BoxPredictor_0/BoxEncodingPredictor=6, BoxPredictor_0/ClassPredictor=0, concat_priorbox=12

[name: "Postprocessor"
op: "NMS_OPT_TRT"
input: "BoxPredictor_0/ClassPredictor/BiasAdd"
input: "BoxPredictor_1/ClassPredictor/BiasAdd"
input: "BoxPredictor_2/ClassPredictor/BiasAdd"
input: "BoxPredictor_3/ClassPredictor/BiasAdd"
input: "BoxPredictor_4/ClassPredictor/BiasAdd"
input: "BoxPredictor_5/ClassPredictor/BiasAdd"
input: "BoxPredictor_0/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_1/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_2/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_3/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_4/BoxEncodingPredictor/BiasAdd"
input: "BoxPredictor_5/BoxEncodingPredictor/BiasAdd"
input: "concat_priorbox"