Fail to get INT8 engine with Deepstream7.1

[SiewKee-Lim]

Hi, I tried to get yolov8 INT8 engine file by following steps from https://github.com/marcoslucianops/DeepStream-Yolo/blob/master/docs/INT8Calibration.md.
However, I getting this error "ERROR: [TRT]: [checkSanity.cpp::checkLinks::218] Error Code 2: Internal Error (Assertion item.second != nullptr failed. region should have been removed from Graph::regions) Segmentation fault (core dumped)" when running command "deepstream-app -c deepstream_app_config.txt". Could you help me on this issue?

Here are the configuration that I have run with:

1. Deepstream 7.1
2. TensorRT 10.3
3. CUDA 12.6

[RX-02333]

same issue,and I found ultralytics/ultralytics#15806 says that downgrade TensorRT to 8.6.1.6 works,so maybe yolo int8 models are incompatible with TensorRT 10.X...

[marcoslucianops]

I didn't get issues running INT8 calibration with DeepStream 7.1 here

[ietamaher]

same issue, Building the TensorRT Engine

ERROR: [TRT]: [checkSanity.cpp::checkLinks::218] Error Code 2: Internal Error (Assertion item.second != nullptr failed. region should have been removed from Graph::regions)

same config as above.
Deepstream 7.1
TensorRT 10.3
CUDA 12.6
Runing on jetson AGX Orin
Opencv 4.10

Ps
I tested few weeks ago on jetson xavier nx with no issues

[marcoslucianops]

Which model are you using? Can you send the full log?

[ietamaher]

I'm running Yolov8. the issue is present only for INT8 Calibration.

rapit@ubuntu:~/DeepStream-Yolo$ deepstream-app -c deepstream_app_config.txt
Setting min object dimensions as 16x16 instead of 1x1 to support VIC compute mode.
WARNING: Deserialize engine failed because file path: /home/rapit/DeepStream-Yolo/model_b1_gpu0_int8.engine open error
0:00:00.178367820 2848 0xaaaafd225c70 WARN nvinfer gstnvinfer.cpp:681:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:2080> [UID = 1]: deserialize engine from file :/home/rapit/DeepStream-Yolo/model_b1_gpu0_int8.engine failed
0:00:00.178429612 2848 0xaaaafd225c70 WARN nvinfer gstnvinfer.cpp:681:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2185> [UID = 1]: deserialize backend context from engine from file :/home/rapit/DeepStream-Yolo/model_b1_gpu0_int8.engine failed, try rebuild
0:00:00.178449676 2848 0xaaaafd225c70 INFO nvinfer gstnvinfer.cpp:684:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2106> [UID = 1]: Trying to create engine from model files
WARNING: INT8 calibration file not specified/accessible. INT8 calibration can be done through setDynamicRange API in 'NvDsInferCreateNetwork' implementation

Building the TensorRT Engine

File does not exist: /home/rapit/DeepStream-Yolo/calib.table
ERROR: [TRT]: [checkSanity.cpp::checkLinks::218] Error Code 2: Internal Error (Assertion item.second != nullptr failed. region should have been removed from Graph::regions)
Segmentation fault (core dumped)

---

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV8.txt

---

GNU nano 6.2 config_infer_primary_yoloV8.txt
[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-color-format=0
onnx-file=yolov8m.pt.onnx
model-engine-file=model_b1_gpu0_int8.engine
int8-calib-file=calib.table
labelfile-path=labels.txt
batch-size=1
network-mode=1
num-detected-classes=80
interval=0
gie-unique-id=1
process-mode=1
network-type=0
cluster-mode=2
maintain-aspect-ratio=1
symmetric-padding=1
#workspace-size=2000
parse-bbox-func-name=NvDsInferParseYolo
#parse-bbox-func-name=NvDsInferParseYoloCuda
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.45
pre-cluster-threshold=0.25
topk=300

[marcoslucianops]

Looks like it's a issue with the TRT 10.3 for Jetson boards. I don't have a Orin to debug, so it's hard to check this issue.