BrowserPose¶
Completed
Overview¶
BrowserPose is an experiment in running action recognition entirely in a web browser, without uploading video to a server.
The project originated from my research on epileptic seizure detection. I wanted to investigate whether an X3D-M model could run directly on a user's device, keeping potentially sensitive video data local.
Implementation¶
I exported X3D-M from PyTorch to ONNX and implemented a browser inference pipeline using JavaScript and ONNX Runtime Web.
The application collects camera frames, maintains a 16-frame buffer, and converts the frames into the [1, 3, 16, 256, 256] tensor expected by the model.
Inference runs through the WebAssembly execution provider. A softmax operation and a label mapping convert the model outputs into readable predictions.
No inference backend is required.
Results¶
On Firefox 150 (Linux x86_64, 16 CPU cores), the application achieved:
- 0.76 inference windows per second, with 16 frames per window.
- 1,304 ms mean inference latency per window.
- 1,377 ms p95 total processing time.
- Approximately
1e-6maximum absolute difference between PyTorch and ONNX Runtime outputs.
The browser implementation was slower than native CPU inference, but demonstrated that X3D-M could execute locally without uploading video.
The project is completed as an experimental implementation. Its latency remains a limitation for applications requiring frequent predictions.
Technical Notes¶
Can Action Recognition Run Entirely in a Browser? — A closer look at numerical validation, inference performance, and why effective FPS can be misleading for temporal models.