• 最新的commit的需要torch的版本是2.3~2.4，而且还是要带cuda的torch，英伟达官方最多只提供到2.2，所以你要我到哪里找？
• 2.2版本的torch-tensorrt，pip --no-deps能装上，但是import后就有未识别的符号，而且google不到解决方案，另外2.x的版本明确说了，别人支持的是cuda-12.x，而不是我现在仅有的cuda-11.4

所以只能用1.4版本的了

Installation — Torch-TensorRT documentation (pytorch.org)

github的repo和pytorch的网页都有编译命令，选了一个靠谱的就行，主要是修改WORKSPACE的内容。

TensorRT/docsrc/user_guide/saving_models.rst at main · pytorch/TensorRT (github.com)

In Torch-TensorRT 1.X versions, the primary way to compile and run inference with Torch-TensorRT is using Torchscript IR. For ir=ts, this behavior stays the same in 2.X versions as well.

Dynamo IR

The output type of ir=dynamo compilation of Torch-TensorRT is torch.export.ExportedProgram object by default. In addition, we provide a new parameter output_format in the CompilationSetting object provided before compilation. The output_format can take the following options

• exported_program (or) ep : This is the default. Returns an ExportedProgram
• torchscript (or) ts : This returns a TorchScript module
• graph_module (or) fx : This returns a torch.fx.GraphModule which can be traced into Torchscript to save to disk.

export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
./build.sh --config Release --build_shared_lib --parallel --skip_tests --use_cuda --cuda_home /usr/local/cuda --cudnn_home /usr/local/cuda --use_tensorrt --tensorrt_home /usr/local/cuda --cmake_extra_defines onnxruntime_DISABLE_FLOAT8_TYPES=ON

/media/loongson/phd19/home/zhou/graduate9/work/update/onnxruntime/onnxruntime/contrib_ops/cuda/quantization/matmul_bnb4.cu(78): error: dynamic initialization is not supported for a function-scope static __shared__ variable within a __device__/__global__ function
# 什么鬼嘛

diff --git a/onnxruntime/contrib_ops/cuda/quantization/matmul_bnb4.cu b/onnxruntime/contrib_ops/cuda/quantization/matmul_bnb4.cu
index 098e361..98290db 100644
--- a/onnxruntime/contrib_ops/cuda/quantization/matmul_bnb4.cu
+++ b/onnxruntime/contrib_ops/cuda/quantization/matmul_bnb4.cu
@@ -75,7 +75,7 @@ __global__ void kgemm_4bit_inference_naive(
   uint8_t local_B_4bit[num_values_8bit];
   T local_B[num_values_4bit / 4];
   T local_A[num_values_4bit / 4];
-  __shared__ T quant_map[16];
+  T quant_map[16];
   T local_absmax = T(0.0f);
 
   for (int i = threadIdx.x; i < 16; i++) quant_map[i] = T(datatype[i]);