OpenCV GSoC 2021 Proposal - huningxin/opencv GitHub Wiki

Hardware-accelerated OpenCV.js DNN module through WebNN

• Description: OpenCV.js exposes JavaScript API of dnn module that allows web apps to do deep learning model inference in web browsers. This capability enables web developers to create intelligent usages, like image classification, object detection, segmentation and style transfer, with rich web contents, like images, videos and camera streams. Although OpenCV.js dnn module is using WebAssembly with multi-threading and SIMD128 optimization (the GSoC 2017 project), the OpenCV.js dnn module is still 4-10X slower than the native OpenCV. This is because native OpenCV can leverage more advanced hardware features and even ML-specific accelerators via native instructions and ML APIs, such as AVX2, cuDNN, OpenVINO etc.,. Web Neural Network API (WebNN) is an emerging web standard that allows web apps to access the AI hardware accelerations. WebNN current spec editors are Intel and Microsoft with strong support from Google among others. Our early empirical results indicate that WebNN gets near-native performance on a variety of Intel-based platforms through the open source module oneDNN. We propose to start optimizing OpenCV.js dnn module with WebNN and demonstrate the performance gain.

• Expected Outcomes:

  • Create the WebNN backend prototype for OpenCV.js dnn module in C++
  • Create the WebNN implementation of selected compute intensive operations, like convolution and matrix multiplication in C++
  • Compile the OpenCV.js dnn WebNN backend implementation into WebAssembly
  • Create the performance test of OpenCV.js dnn module, collect and analyze the performance numbers of different implementations including WebAssembly, WebNN and native.