EchoPulse-CPP C++/CUDA Inference Engine for Echo State Networks¶
Completed
Overview¶
EchoPulse-CPP is an inference engine I developed to run Echo State Networks (ESNs) in C++ and CUDA without requiring PyTorch at runtime.
A colleague introduced me to Echo State Networks, and I wanted to experiment with the architecture. I decided to separate training in Python from inference in C++, using the project to explore GPU acceleration and sparse matrix computation.
Implementation¶
I separated the project into two components.
The Python component handles ESN initialization, training, and weight export. Trained parameters are serialized into binary files that can be loaded by the C++ engine.
The inference engine is implemented in C++17 and supports CPU and CUDA execution.
For the reservoir computations, I implemented a custom Compressed Sparse Row (CSR) representation to store sparse weight matrices. I also developed CUDA kernels for GPU execution rather than relying on PyTorch for these operations.
The engine integrates OpenCV to process video files and webcam streams.
I included a command-line benchmarking tool to compare corresponding CPU and GPU operations and inspect their execution times.
Results¶
The engine successfully runs ESN inference without requiring PyTorch at runtime.
During benchmarking, GPU execution was faster than CPU execution for the tested operations. Using CSR for sparse reservoir computations also improved performance compared with the representation I initially used.
These improvements were observed experimentally, but I no longer have the exact benchmark figures available, so I cannot report a reliable speedup.
The project was completed as an experimental inference engine. Its performance was not validated on dedicated edge hardware.