Making Clinical Video Evaluation 10x Faster¶
The initial X3D evaluation loop processed long clinical videos at roughly six video frames per second. With videos exceeding 20 minutes and an archive spanning 20 years, evaluation throughput controlled research iteration time.
Two Bottlenecks, One Loop¶
The pipeline used OpenCV for decoding and PyTorch for X3D-M inference. Both stages changed:
- PyAV replaced OpenCV decoding.
- X3D-M was exported to TensorRT for the forward pass.
The optimized path ran on an RTX 3090 with batch size 128 and was used in the real evaluation loop, not only in an isolated microbenchmark.
The Result¶
End-to-end evaluation became approximately 10x faster.
That is deliberately reported as a pipeline result. The experiment did not separately benchmark pure TensorRT forward latency against pure PyTorch forward latency. Assigning the full gain to TensorRT would therefore claim more than the measurement supports.
Why the Boundary Matters¶
Changing two stages in parallel is reasonable when research iteration is the target. It is not sufficient when the target is attribution. The benchmark answers:
How much sooner does this evaluation run finish?
It does not answer:
How much faster is TensorRT than PyTorch for this model?
Both questions are useful, but they require different experiments.
Operational Context¶
The same project had already moved to chunked decoding because complete long videos exceeded RAM. Performance work followed correctness under the actual constraint: process arbitrarily long video first, then reduce wall-clock evaluation time.
That ordering kept the optimization tied to the research workflow instead of producing a fast component that could not process the archive.