• Markdown Syntax
• OpenCV
  • OpenCV Home Page
  • OpenCV Developer Site
  • OpenCV Meeting Notes Home

Meeting_notes

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Name
  • todo

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Last progress before 2-week end-of-year break

• Intel team
  • OpenCV 4.5.1 and 3.4.13 have been released.
  • answers.opencv.org is read-only now. Activity on forum.opencv.org is where Q&A happens now.
  • The team has reviewed Euler Angles PR, left some comments (https://github.com/opencv/opencv/pull/19098)
  • Putting RGBD parts to 3D module is in progress
• Org: Done:
  • Anastasia: fixed issue #18878
  • Amir: worked on amendments to Voronoi diagram construction with respect to new topology (i.e. Delaunay triangulation) defined by the new predicates. PR #19126 is being reviewed.
  • Alexander: Implemented full RISC-V testing pipeline for OpenCV CN CI with QEMU emulator. Prepared test suppression list for major modules.
    • Work on multi-core & multi-theading solution for RISC-V is in progress.
    • Implemented pre-commit pipeline for PR testing on Ubuntu 18.04. Work on ARM analogue is in progress.
    • Resolved several networking and security issues and exposed CI server to the Internet. CI master is available at https://build.opencv.org.cn/
    • Analyzed failed tests on CI, filed relevant bugs to Github. All issues are tracked in spreadsheet.
• In progress:
  • Anastasia: working on #19183
  • Amir: started with issue #4775
  • Igor: Working on opencv-python packages build speed up
  • Alexander: CI job for PR testing on ARM
    • Read-only access for subset of jobs on CI: pre-commit, nightly.
    • General infrastructure-related tasks for China CI: notifications, backup, etc
• Vadim in Shenzhen

In case we skip the meeting next week: Merry Christmas and Happy New Year everyone! Hope 2021 will be a beautiful and uneventful year unlike 2020 😄

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• OpenCV 4.5.1 / 3.4.13 release.
  • target release date: Dec 20-25
  • wiki: changelog
• GSOC blogposts
• OpenCV fails tests on ARM64, module : GAPI. Issues ## 19117..19124.
• ARM tests tolerance check adjustment (put under #if condition)
• China CI integration

• Release expected on time. AI to all: add notes to changelog.
• AI to all GSoC 2020 maintainers: ping students. Blog post about GSoC project is required. Lydia will contact to RISC-V case author and Depth Fusion
• G-API tests fail on ARM64. Alexander Alekhin reproduced a significant part of tests. The issue is treated as a non-accurate test implementation, but not a library issue. Alexander Smorkalov disables G-API on ARM64.
• Alexander Smorkalov works on OpenCV China CI and stabilizes base builds. A soon as they are stable, Alexander will work on

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • got back to China successfully. On the quarantine currently.
  • working on OE34 (named parameters), since it may affect a lot of OpenCV 5.x API. Stub solution for Python, Java, JS is ready. Now working on a good solution for Python.
  • Prepared PR 19082 with RISC-V copyright info, but it causes some confusion. It's now being resolved and the updated patch will be prepared shortly.
• Anna:
  • new person (Amir) joined the team. Working on fixing Delaunay triangulation in one corner case (#19126)
  • Alexander is working on a number of PRs. ARM64 issue has been filed.
  • He is also working on OpenCV China CI.
  • Nearest-neighbor resize alignment is another topic.
  • Python bindings is in progress.
  • RISC-V build preparation is in progress.
• Gary:
  • AprilTag: did not have meeting past week. Sent a bunch of datasets to the student for proper testing.
  • Want to take a look at color calibration, contributed by H inc.
  • OpenCV.org is unrolling; OpenCV trademark etc.
  • Negotiating possibility to "hire" expert to implement state-of-art bundle adjustment for OpenCV; self-calibration is another possible topic.
  • OpenCV board meetings need to be resumed. The nearest one is planned at the end of 2020.
• Alexander:
  • A. Alekhin is working on adding versioning to videoio plugings (ffmpeg etc.)
  • Closed 300 invalid/obsolete issues.
  • Finished documentation refactoring. Better look'n'feel: https://pullrequest.opencv.org/buildbot/export/pr/18712/docs/d9/df8/tutorial_root.html
  • Rostislav is moving some functionality from rgbd to 3d; reviewing PR with dual quaternions.
  • Text recognition GSoC project results have been merged
• Shiqi:
  • helped to resolve issue with RISC-V related copyrights
  • installed fresh NVidia drivers on OpenCV China CI build nodes
  • SSL certificates uploaded
  • network has been tested
• Vincent:
  • filed several issues in the bug tracker about problems found in 4.5.0 and 3.4.12.
• Stefano:
  • participating in TF project; SiFive does excellent job on RISC-V backend

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • calib3d is split into 3 modules; pull request has been merged (#18940). We can now add more stuff from opencv_contrib.
  • risc-v was tested and merged. Currently, 17 tests out of 11K fail, but this is very good result for the first drop.
  • discussed with Alexander from xperience.ai requirements and plans on extra OpenCV in China. It will focus on ARM & RISC-V testing.
  • reviewed patch and blog post on using Orbbec's Astra cameras; excellent job!
  • now updating evolution proposals in OpenCV wiki; put OpenCV 5-related proposals there
• Anna:
  • Alexander: Deployed Jenkins-CI server and agents (Ubuntu, Ubuntu ARM, Windows 10) for OpenCV China. Prepared Docker images for CI build and test (Ubuntu 18.04, Ubuntu 20.04, Ubuntu 18.04 ARM). Work on CI pipelines is in progress.
  • Alexander: Introduced tests for cv::CAP_PROP_FRAME_MSEC in cv::VideoCapture (PR #18968), helped to finalize related PR #18966.
  • Alexander: Reviewed, tested, helped to resolve issues with PRs #18955, #18971, 18948
  • Sergei: Changed current implementation of downloading scripts in samples/dnn. Extended current Python bindings to reuse existing code from filesystem module.
  • Sergei: Fixed the problem with DNN module build with Vulkan enabled.
  • Igor: Made some improvements to the tutorial about using Astra cameras in OpenCV.
  • Igor: Blog post about Orbbec Astra cameras is ready.
  • (WIP) Sergei: Implementing Conv1d/Pool1d layers for CUDA backend in DNN Module
  • (WIP) Anastasia: Fixed bilinear upsampling layer shift problem (#18721), working on code refactoring
  • (WIP) Igor: Investigating opencv-python packages build issues.
• Gary:
  • AprilTag detector project continues; all-deep-learning solution looks quite promising.
  • Probably ask one experienced guy to develop state-of-art bundle adjustment for OpenCV's 3D module (for multi-camera multi-board calibration setup).
  • OpenCV now has official CLA.
  • Sent CLA to Chinese company who is going to contribute state-of-art QR detector.
  • Macbeth detector has been improved; need to test it.
• Alexander:
  • OpenCV 5.0: OpenCV 4.x latest snapshot merged to "next"
  • mac CI machine was updated with Big Sur and Xcode 12.2
  • github infrastructure is cleaned up.
  • GSoC project: tutorial on PyTorch, TF => OpenCV was merged
  • GSoC project on text detection is still under review
  • Idea to put OpenCV history page to opencv.org and/or wiki (everybody liked it!)

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Started to refactor calib3d module: https://github.com/opencv/opencv/pull/18915. It's split into 3 modules.
  • Continue testing of RISC-V patches; will hopefully be merged this week
• Shiqi:
  • Anna will give "AI on Edge devices" talk; it continues the serie of OpenCV China webinars.
• Anna:
  • 1D support is finished for CPU and OpenCL, merged.
  • Tutorial on Astra cameras is finished, merged.
  • Changed script for downloading models
  • 16-bit FP support in DNN has some problems; investigating it.
  • blog post for opencv blog is finished.
  • started 2 big activities:
    1. official OpenCV Python module (pip); took it from the previous maintainer; will fix some long-standing bugs like imshow
    2. setting up CI in China
• Gary:
  • created CLA for OpenCV contributions (based on Apache CLA). It's under review now.
• Alexander:
  • text recognition GSoC project. ObjC bindings issues have been found and fixed.
  • libjpeg-turbo has been upgraded from 2.0.5 to 2.0.6.
  • Rostislav is working on OpenCL implementation of Hashed SDF.
  • Color calibration algorithm has been megred.
  • support for XCFramework by Apple is added
  • merged the patch with quaternions
  • audio capturing functionality (mic support) is in progress.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Choosing a name for 3D module: Alexander Smorkalov will make experiments with a fake module and name with “_”, he will provide the feedback till the end of next week. Issue will be converted to ticket, voting will be closed.
• Xperience AI team takes ownership on Python packages for Pypi: Alexander Smorkalov will collect the list with the necessary accesses and share it with Alexander Alekhin.
• The new patch integration policy will be discussed until December, 1
• PRs and Issues:
  • OpenCL #18465: Sergei will finish current PRs and make a basic analysis of bugs; Xperience team will fix some of the bugs themselves; if necessary the discussion with the core team will be started by email

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Truetype patch is finally merged. Took just 2 months.
  • Wrote the draft version of new merge policy that should accelerate patch integration.
  • Tested one of the two RISC-V patches — compiles well and runs surprisingly fast on QEMU. If OpenCV unit tests are run in multiple processes, it can be quite realistic to test the whole OpenCV for RISC-V using QEMU. Previously, I only hoped to test a small subset of OpenCV.
  • Voting on the 3d module name continues. The current leader is opencv_3d (cv::_3d) and opencv_xyz (cv::xyz). This Friday we will hopefully finalize the name and start forming 3d module in the "next" branch.
• Shiqi:
  • Talked to one company in China who are willing contribute very good QR code detector into OpenCV. The negotiations are in the advanced state.
• Anna:
  • Finalized support for 1D conv in DNN; merged to OpenCV
  • Parameterized input/output is also ready;
  • Working on 1D pooling
  • Wrote tutorial about using Orbbec's Astra cameras.
  • 100+ => 89 open pull requests.
• Gary:
  • AprilTag detector is still in progress. It's pretty solid now. Autorotation on Android brings in some difficulties.
• Alexander:
  • 2 trackers: GoTurn & MIL have been moved to the main repository; KCF is not moved yet because of the big binary blob used (>3.5Mb)
  • dnnModel class has been refactored
  • the team is preparing OpenCV 4.5.1 and 3.4.13 released.
  • one bug in core, reported by Vincent, has been fixed.
• Vincent:
  • 2 efforts: C API => C++ API, updating OpenCV 2.x to 4.x. k-means API has changed between 2.x and 4.x and it was the only API that required changes.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• 1. Need to choose the 3D-module name. The deadline is next Friday.
• 2. DNN module:
  • Conv1d support: OpenCL has low priority
  • Float16 support: Sergei will fix just the current issue
• 3. PR’s
  • a. Ordinary quaternion #18335: choosing a “next” branch, module name will be chosen till the end of next week
  • b. Tracking API target location:
    • i. 4x: module "video" remains in, "tracking" is adding
    • ii. 5x: module "video" will be removed
  • c. Merge fisheye camera into default camera calibration API #11885- "int cameraModel" ==> "class CameraModel", possible target is 5.x (Alexander Alekhin)
  • d. CMake: make OPENCV_TEST_DATA_PATH cached, add warning if tests are built #18703 - will be close (Alexander Alekhin)
  • e. Python scripts for file downloading in samples/dnn #18591 - annotation should be more functional (Sergei)
• 4. New forum address: https://forum.opencv.org/, Alexander Voron will text about it to all team, including Satya Mallick and Gary Bradski.
• 5. The new patch integration process: Vadim Pisarevsky will write a draft document for the OpenCV mergemen. It will take effect after all team members approve it.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Lapack patch was finally merged. Truetype patch is still under review (took nearly 2 months already and still counting)
  • Briefly reviewed the pull request with automatic model downloading from Python. Still lacks good usage example, but the functionality seems to be there already.
  • Return to China is postponed till the next year. Will continue to work remotely.
• Shiqi:
  • ARM server is installed and the network is configured. It includes 96 cores and 128 gigs of RAM. Finally, OpenCV testing on ARM will simply rock!
  • mac mini (Intel-based) is ordered; that will round the build farm in China. Just announced ARM-based mac mini will be added to the farm next year perhaps.
  • Got many proposals from Chinese universities for collaboration. They are ready to contribute deep learning models to OpenCV.
  • Need higher-level app-specific API on top of DNN in separate modules.
  • One of the students is working on free alternative of Wider to train absolutely free face detection model.
• Anna:
  • 1D support in DNN
  • working on the usage example for model downloading from Python
  • Orbbec cameras usage tutorial is in progress
  • Another article for OpenCV blog is almost ready; it's about the image retrieval using deep learning
  • Another tutorial is in progress: how to build very compact version of OpenCV.
  • Ideas for other tutorials are welcome. Input from Gary: any tutorials about using DNN are welcome
  • OpenCV python releases — whether it makes sense for OpenCV team to distribute them.
• Gary:
  • The student continues working on AprilTag detector. Now trying to compare efficiency of checkerboard vs apriltag.
  • Still thinking of the calibration toolbox
• Alexander:
  • the team is refactoring tracking API; migrating several trackers to the main repository.
  • GSoC text detection API is being refined.
  • opencv is migrating to the new forum. Probably, could use "Discussions" on github.
  • OpenCV team at Intel is preparing several seminars for Intel academy. In particular, 3D vision talks are prepared by Rostislav.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Xperience team will do tutorials; it might be guidelines at the wiki and also blog posts as well. Also, Vadim Pisarevsky will also discuss specific requests for this task with Shiki
• Quarterly Roadmap meetings will be introduced on Wednesdays since November. The first meeting will be held on November, 18.
• Image reading: need to decide what types of data and what formats should be supported (at least JPEG). Anna will tell potential partners about this existing opportunity. Xperience team will take this task to work.
• Quaternion normalization - need to decide about the module’s name
• #17570 Vadim Pisarevsky will add comments about the model
• #12186 will be closed (Alexander Alekhin)

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Still working on Lapack. Reduced the patch size to very minimum, not submitted yet. Hopefully, it will be integrated soon.
  • Another round with truetype font suppport. Made use of STB_truetype (tiny, 4Kloc single-file ttf parser and renderer), patched it to support variable fonts, almost refactored previous patch to use STB_truetype instead of freetype. Patch size went down from several megabytes to ~300Kb.
  • Tested network access from Russia. Works well.
• Anna:
  • finished script to download models
  • finalized GSoC video, published blog post
  • 1D convolution in DNN module; initial version is ready.
  • reviewing some pull requests, working on the Orbbec camera tutorial
  • reviewing Tony Ng article for opencv blog
• Gary:
  • 2 more people are allocated for OpenCV. They will work on the better calibration stack.
  • Thinking of organizing OpenCV summer of code 2021.
  • Student is advancing with good-quality AprilTag detector
  • OpenCV 5 needs to be properly advertised. New edition of OpenCV Book? Or maybe even serie of books?
• Alexander:
  • Support for 1D mat continues.
  • PoC on drawing module is in progress.
  • Tutorial layout is being revised.
  • QR code decoder/encoder (results from OSPP) have been merged.
  • mp3 support (as a part of audio support) has been added.
• Shiqi:
  • Huawei will help to setup CI network.
  • ARM server will be added to OpenCV CI this week; it includes 64 cores, and also includes NPU.
  • The 2nd video on OpenCV 20th-year anniversary has been translated to Chinese.
  • Conversation on RISC-V project continues.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • working on Lapack.
  • got updates for color calibration, will suggest new API & new tests
  • got updates from text detection & recognition team; will work on improving text detection functionality even further.
  • reviewed model downloading Python script, suggested some improvements.
• Anna:
  • reviewing pull requests: file storage, quaternions & gstreamer.
  • DNN side: changing the script to download models. conv1d layer is in progress.
  • Started playing with Orbbec cameras.
  • Working on the final video for GSoC.
• Vincent:
  • Migration guide for OpenCV 5.0 would be useful.
  • OpenCV 4.5.0 is started to be used internally. There should be way to disable exception globally. setBreakOnError() helps. In generall, the integration goes smoothly.
• Gary:
  • Got extra funding from FW. Can now hire some more developers. The main area is autonomous driving.
  • It can be some cooperation/intergration with ROS 2.
  • More comprehensive calibration, possibly including multi-sensor joint calibration/registration, self-calibration etc. This calibration may have a substantial footprint, so it might be a dedicated github.com/opencv/calibration-toolbox or something like that.
• Alexander:
  • OpenCV 4.5.0 release is announced now.
  • OpenCV 5.0:
    • reviewed quaternions
    • 1D mat support in OpenCV DNN is going on
    • the new hash TSDF implementation has been integrated; better performance and memory footprint
    • GSoC: webasm was integrated; bit-exact version of SIFT is still in progress.
• Shiqi:
  • network for CI is ready in Shenzhen; can be accessed worldwide.
  • ARM server is received. Now need to find mac mini or something like that for testing OpenCV on Mac.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Prepared and submitted patch hat adds Lapack to OpenCV. Passes all the tests. Now the new RANSAC can be safely compiled on any platform. The old RANSAC (old branches in findFundamentalMat, findEssentialMat, findHomography (except for RHO algorithm)) can be removed from OpenCV 5.
  • Reviewed 2 follow-up patches from my GSoC students: on RANSAC and on Julia bindings
  • Started looking into RISC-V patches; trying to run it under patched QEMU.
• Anna:
  • dynamic shapes in DNN are now supported.
  • downloading data in Python samples was implemented.
  • 2 more videos (including interview with Andrew Ng) related to 20th anniversary have been prepared and published at opencv.org.
• Gary:
  • Trying April Tag calibration rig now. The problem is that deep network used finds corner locations, but the accuracy is not good enough.
  • OpenCV dev team will hopefully be extended by 2 more engineers soon. They will likely work on improving calibration, better support for Robotics etc.
  • Still need to test color calibration functionality
• Alexander:
  • On Monday OpenCV 4.5.0 has been finally released.
  • "next" branch was created.
  • Installation tutorials have been updated
  • GSoC 2020 results on "large scale depth fusion" have been reviewed and merged.
  • OSPP: QR decoder/encoder code is still pending, some things need to be done yet.
• Shiqi:
  • ARM server for OpenCV testing will be delivered this month.
  • A talk about RISC-V project was given by GSoC 2020 student in China (organized by OpenCV China)
  • Text recognition webinar will be given next
  • There are some complains that OpenCV QR code detector is not robust enough or accurate enough. Need to improve it.
• Stefano:
  • In the DNN compiler stack there is work to define/standard high-level ops; the effort is called TOSA. Proposed by ARM (Linaro project). A part of MLplatform.org project (MLIR?)

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Finished the patch with truetype support
  • Started working on adding Lapack back to OpenCV
• Anna:
  • DNN support for models with dynamic shapes - almost finished. preparing tests. resize layers etc. have been updated
  • OpenCV blog — Lydia is working on some document with guidelines for blog authors.
  • document to describe contribution and collaboration process - also in progress, discussed with Intel team.
• Gary:
  • still have not got to the final Macbeth detector review
  • high-speed autonomous car racing — the negotiations on collaboration are still going on
• Alexander:
  • OpenVINO has been released this week.
  • OpenCV 4.5.0 will be released next week
  • OpenCV 5.0 CI (next branch) will be completed this week
  • OSPP project is finished, student still working on polishing the code
  • Audio project continues
  • MS Photos app appears to use OpenCV
• Vincent:
  • Difficult process of migration from OpenCV 2.x and 3.x to OpenCV 4.x.
• Edgar:
  • Small workshop on PyTorch next week, will give talk on Kornia.
  • Open community meeting tomorrow
  • 0.4.1 release next week
  • Work on differentiable RANSAC is going on.
  • Morphology functionality is also been prepared.
• Stefano:
  • Looking at compiler pipeline in TF
  • Interesting progress on ARM CPU support & CUDA

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • The patch with truetype support now passes all tests (except for the patch size warning). More advanced version of the patch is now prepared that will include some new features, provide much better performance and will also include Harfbuzz-based text shaping (correct rendering for some languages).
  • On Monday gave talk at the online ML/AI workshop, organized by H. Presented OpenCV status and plans for OpenCV 5.
  • Composed initial spreadsheet with OpenCV 5-specific tasks on 3D module, Python bindings and other components.
• Shiqi:
  • OpenCV CI in Shenzhen is being setup by H. Need to wait till after mid-Autumn holiday to complete that.
  • 2 students are now studying and trained to contribute to OpenCV. The main interest is 3D module.
  • Need to update face detection in OpenCV.
• Anna:
  • Engineers on vacation this week.
  • Parameterized-input networks with dynamic shapes is now supported better, need to test it more thoroughtly.
  • 2 blog posts sponsored by Intel are now prepared, 1 is published already (related to OpenVINO)
  • Another blog post by Always-AI is prepared; will be published soon.
• Gary:
  • Macbeth chart detector is being updated.
  • Received updated code for color calibration, have not tested yet.
  • Could hopefully expand OpenCV team soon (by 1-2 people)
  • Initial negotiations about collaboration with some teams in the new autonomous cars competitions. One thing that can be done within this project is better camera calibration toolbox. Can also help with OpenCV promotion.
• Alexander:
  • OpenVINO new version is around the door; new OpenCV release (4.5) is expected in two weeks from now.
  • Alex A is working on CI for "next" branch.
  • OpenCV build with Intel VA is fixed.
  • DNN OpenCL segfault has been fixed.
  • Minor improvements in Video I/O in OpenCV.js
  • OSPP program on QR code decoder/encoder is finished successfully.
  • lena.jpg in OpenCV was removed from Debian/Fedora.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Almost finished patch with truetype font support. It will replace ~20 years-old Hershey fonts in OpenCV. Many languages are supported, including CJK. The pull request will be submitted this week.
  • Received patch from Maksym (GSoC 2020 student) with further improvements in RANSAC; under review
  • Received updated patch from H about color calibration; under review
  • Just received patch from T-head with RISC-V intrinsics.
  • H is setting up network for alternative OpenCV CI (where we will test ARM, RISC-V etc.)
• Shiqi:
  • CI infrastructure almost finished
  • Need to update installation guides
• Anna:
  • Updated wiki page for DNN module
  • Working on adding support for models with dynamic shapes (ONNX); Bringing TorchVision models (RCNN in particular) support to OpenCV.
  • Some update Youtube channel; OpenVINO blog posts for opencv.org is being prepared.
  • Working on 2 OpenCV anniversary videos
  • More blog posts are in the queue
• Gary:
  • Color calibration: yet need to download and try how well it works.
  • Separate bank account for OpenCV created.
  • Interesting interview with some Computer vision gurus.
• Vincent:
  • Upgrading from OpenCV 1.x or 1.x-pre to 4.x may take substantial time
  • ConstMat may be useful to have
  • partial imread()
• Alexander:
  • OpenCV team has participated in forum at NNSU.
  • OpenCV 4.5 should be released on WW40-41 (right after OpenVINO); some critical PRs will be merged before it.
  • OSPP program: student has finished coding and implemented some unit tests.
  • HashedSDF (rgbd) memory optimization has been done.
  • Audio support (mp3 format support)
• Edgar:
  • Kornia v0.4.1 is released.
  • The paper about Kornia has been published https://arxiv.org/abs/2009.10521
  • Promotion activities via "OpenCommunity" have been started.
• Stefano:
  • TF activities.
  • ffmpeg can now execute dnn networks using OpenVINO (is this just for video transformation, not video analysis?)

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • 9/14 videos are collected. We are mostly covered: https://docs.google.com/spreadsheets/d/1XFK4PfBZ807nAZit7L6yC7U7JfJJwoewVjiobKRRHlA
  • There is some good progress on the functional language compiler
  • Archit Rungta has prepared another patch that extends Julia bindings with 3dcalib module support.
• Anna:
  • Investigation of Orbbec 3D support
  • Activity about OpenCV DNN: Segmentation networks, RCNN support
  • "Always AI" company has prepared blog post for opencv.org
  • New video for OpenCV anniversary (interview with Bill Freeman) was prepared.
• Gary:
  • Color correction patch was received and briefly reviewed, but have not tried it yet.
  • Student who did Macbeth chart detector continued with AprilTag detector. Need to have more accurate corner localization.
  • (Edgar has some functionality for sub-pixel feature localization in Kornia)
• Alexander:
  • CI for OpenCV 5.0: Alexander A. has prepared patch that adds next branch
  • OSPP (QR detector) patch is being reviewed
  • TSDF volume integration is almost finished by Rostislav
  • SLAM implementation has started (based ORB-SLAM); document with the implementation plan is ready.
• Rostislav:
  • discussion of 3D module properties:
    • bridge to Open3D should include some data structure conversion functions
    • visualiazion functionality is using vtk currently; need to rewrite it w/o using vtk.
    • (Edgar: GradSLAM is deep learning community take on SLAM).

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• All GSoC 2020 projects finished successfully. TODO (Vadim action item): Need to add the links to videos into the spreadsheet. Maybe send the final "thank you" to all the students and mentors.

• Gary

  • Can do the final GSoC video.
  • 2 pull requests about color calibration in opencv_contrib. Will review it shortly (Vadim do technical review w.r.t. coding style)
  • color calibration algorithms may do incorrect things when complex model is used with 3-row color chart
  • AprilTag part of the project advanced quite a bit. It can be a good component for SLAM etc.

• Alexander

  • OpenCV current version (master) updated to 4.5-pre. "next" branch is not created yet (work in progress).
  • 3D proposal is updated by Rostislav
  • answers.opencv.org to migrate to the new repository
  • audio support is implemented (Linux & Windows are supported).

• Anna

  • DNN module updates
    • proposal has been composed, will discuss it on Friday
    • Support for more topologies.
    • ONNX graph simplification
    • Updates to model importer
  • RISC-V GSoC project finished successfully, but it does not run smoothly under QEMU
  • GSoC DNN tutorial project. The text and code are still under review; student will continue work on the code in September
  • New article for opencv.org blog (AI related topic) is being prepaired
  • The article about Kornia has been already published

• Shiqi

  • Partnership activities. 2 more companies are going to join the partnership program.

• Edgar

  • Kornia is going to become some legal entity to start collecting contributions, maybe hire extra developers
  • Stefano suggested a way to collect feedback from community, specifically regarding AI functionality. The forum will run starting next month.

Action items:

• Gary: answers.opencv.org/opencv.org payment?

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim is on vacation in his secret lair

• Last week of GSoC mentors need to get evaluations in!

  • Mentors must get in their final reviews before Sept 7th
    • Students should have a merged or substantial pull request in
    • Video youtube summary/demo -- can be simple, some students do a kind of couple minute "presentation" showing slides and demo. Up to you/them
  • If you know your student has passed, then pass them NOW. We admins hate last day stress because if we don't get our evaluations in, Google will not let us in next year.
    • It takes literally 2 or 3 minutes. Go to
      • https://summerofcode.withgoogle.com/dashboard/
    • and fill out the evaluation.
  • Thank you all for your work through the summer! This is probably our best run/best results year on record!

• Gary

  • Macbeth is already a merged pull request
  • Had student submit a new patch that allows
    • access to which squares were actually found/quality measure
    • a method that draws the squares to help with debug
    • This is a pull request waiting final review, but the project is easily a "pass"
  • Have also done major work on AprilTag detection via deepnets.
    • I think this is all working, but may need more refined training
    • Student wants to continue after to get this merged

• Alexander

  • All projects on SIFT, Gaussian Blur, Ransac on track
  • Performance Object tracker merged
  • Large scale depth fusion, depth fusion refactoring
  • QR codes, working on Encoder part OK
  • Other activities
    • 3D model proposal for opencv 5.0
      • See proposal page: https://github.com/opencv/opencv/wiki/OE-33.-3D-Module
    • Fixing some issues

• Anna

  • GSoC
    • Risk-V Difficult problems to resolve (toolchain is not complete), but good job with existing tools. Compilier bug being resolve
      • Couple of pull requests. It's a pass but Risk-V still has to mature
  • Tensorflow and Pytorch
    • Easy load and convert and compare model with DNN
    • Classification and detection results
    • Almost all is in submitted pull request
  • Issues with opencheck integration fixed, parameters so file size is reduced
  • Torch Fusion to DNN prepared list of unsupported features
    • Analyzing models to see what to add to support that
  • Interviews going to to Anniversary site opencv/anniversary
  • Blog with Kornia prep
  • Shiqi video being added

• Shiqi

  • GSoC
    • OCR project quality text detection and recognition doing well
    • Pull requests submitted
    • Video demo created -- all on track
      • Post GSoC, we'll create some application
  • OpenCV China would like to translate the OpenCV videos (subtitles)
    • Working on others

• Kornia

  • Working on article for blog
  • Working on data augmentation for medical imaging
  • Invited to give a short talk on NVidia CVGP (lightning also invited)
    • https://montrealrobotics.ca/diffcvgp/
  • NIPS workshop on differential computer vision next year
  • Writing small abstract on API for data augmentation

• Gary
  • Send out final reminder to mentors to get evaluations in

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

1. Vadim:
   • GSoC: the final evaluation for students has started. Need to ask the students to submit the final evaluation form.
   • GSoC: Julia bindings project is 100% finished; the final video was received. Article for the blog is in progress.
   • GSoC: RANSAC project is 100% finished. The final video was received. Article for the blog is in progress.
   • Discussed DNN project evolution for OpenCV 5. There probably be no radical refactoring of the module in OpenCV 5 (e.g. towards NGraph-like architecture), however there are going to be some important improvements:
     • reworked importers, especially ONNX importer.
     • an external utility that will do careful network diagnostic and probably some transformations to simplify its use within OpenCV.
     • probably an updated layer fusion part that will do multi-pass fusion.
     • python samples could automatically download and store in some "cache" missing deep learning models.
     • even better CUDA/cuDNN support, maybe via dynamically loaded "plugin", similar to opencv_ffmpeg, dynamically loaded FFMPEG wrapper plugin on Windows.
   • Continued discussion about 3D module. Rostislav from Intel team will transform 3D module evolution proposal and will put it to OpenCV wiki
2. Anna:
   • travelling
3. Gary:
   • Asked Ajit to prepare the final request to expose some internals of Macbeth detector. Here are some results: https://github.com/opencv/opencv_contrib/pull/2644.
   • April tag detector is in progress. Will it be finished? Gary suggested to re-use some of existing implementations.
4. Alexander:
   • initial support for SYCL (OpenCL One API). It will be put to OpenCV 4.5.0
   • "next" branch for 5.0 will be created.
   • function to draw OpenCV logo has been added
   • audio input/video support is in progress. Windows & linux are covered.
   • GSoC:
     • SIFT PR is under review. video report has been prepared.
     • WASM optimization is done. Performance data is being collected.
     • Depth fusion is on track. Students is polishing the final PR.
     • Object tracking: PR has been merged.
   • OSPP:
     • QR encoder/decoder is generally working pretty good. The student will spend the rest of time improving QR encoder.
5. Shiqi:
   • GSoC: text detection & recognition, Wenqing submitted updated PRs with faster networks. Video is still to be improved.
   • Found 2 potential developers for OpenCV, who can ready contribute to OpenCV part time.
   • Found some more potential members of OpenCV development partnership program. Negotiations are still going on. This year we might get up to 3-4 partners in total.
   • OpenCV 5.0 proposal: one company that produces web cameras with depth sensors; they are interested to participate in 3D module improvements. Ethernet cameras support is also very important: actively used in autonomous driving and in medical areas.
   • Continue work to find more partners for OpenCV, in particular, hardware companies.
6. Stefano:
   • some more discussion about use of some standard compiler stack for OpenCV DNN
   • object tracking pull request is merged into 3.4. Blog post can be prepared.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

1. Vadim:
   • GSoC: Julia bindings project is 99% finished; all the pull requests are merged; tutorial is finished and merged. Video is in progress.
   • GSoC: RANSAC project is finished and merged (together with a short tutorial); Draft video is created, it's being improved now.
   • 2 OSPP projects have passed the first round (1 is QR code encoder/decoder, mentored by Intel team; another one is 3D point cloud simplification mentored by Mihai Bujanca (waiting for confirmation))
   • Extended 3D module proposal draft, received comments from several people. Now need to publish it as OpenCV evolution proposal.
   • OpenCV has finally migrated to Apache 2 license. Pull request with the new license has been merged into master branch.
2. Anna:
   • Reviewed, helped to resolve issues with ## 18037, 18105, 18129, 18117, 18056, 18110
   • Investigated issue 17914. Seems that OpenCV loader for ONNX networks does not support any model with dynamic axes (feature of onnx converter).
   • Finalizing the document on unsupported features/layers of DNN module (mainly for Detection/Segmentation networks).
   • Have some problems with finalizing solutions for issues 17516, 17531, 17588: different problems with OpenCL tests.
   • GSoC: the student who works on the PyTorch/TF+OpenCV tutorials is finalizing everything. Expect to have the final pull request in the middle of the next week, and the blog post by the end of the week.
   • GSoC: RISC-V project is blocked by the tooling which is not there yet. The student waits for his university team to finalize the missing tools. We are happy with his performance though.
   • Org: the anniversary page is live! Some people send us wishes in social media.
   • Org: moved the gold sponsorship page to a public menu - now we can promote it.
3. Gary:
   • Ajit is working on AprilTag detector
   • some financial activities related to opencv.org
   • thinking of going to Patreon
   • SuperAnnotate
4. Shiqi:
   • text recognition (Wenqing Zhang) – the final pull request is submitted. It needs to be reviewed and merged.
   • working on an article about organizing a summer internship program in China, somewhat resembling GSoC. One possible project is data augmentation
5. Vladimir:
   • GSoC projects are going well.
     • The project on object tracking appeared to be more complex than expected.
     • 3D: 1st PR is ready to be merged; the final PR is in progress
     • OpenCV.js (WASM) is in track
     • SIFT implementation is on track as well
   • Working on audio support: Linux backend is almost ready, Windows backend is in progress.
     • Sample should run one of speech-related DNN models
   • OSPP project: the produced code is not quite clean, need to refactor it.
   • "next" branch for OpenCV 5.0 should be created in August.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

1. Vadim:
   • GSoC: RANSAC project contents is finished; tutorial is in progress.
   • GSoC: Julia bindings project is finished; tutorial is in progress
   • GSoC: In each OpenCV GSoC 2020 project at least one PR has been submitted.
   • GSoC: Canonical object (plane, sphere, cylinder) fitting into 3D point cloud. The student cannot proceed due to family reasons. Still, there is a plenty of code written.
   • Rostislav has submitted draft proposal about 3D module in OpenCV (at Google drive). The link will be sent separately. After that, it will be published as OpenCV evolution proposal.
   • PR with the license change is submitted.
2. Gary:
   • GSoC: Macbeth chart; want to expose some extra functionality in API.
3. Shiqi:
   • GSoC: text detection and recognition project is also basically finished; excellent results

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

1. License:
   • Vadim submits a new license
   • Alexander S. updates pr templates, etc
   • Anna updates logo on Wikipedia
   • Anna will update the license on Wikipedia after the first release with new license
   • SVG logo goes to docs
2. Alexander A will create branch “next” and tune CI after license related changes
3. OpenCV blog:
   • New article published in OpenCV blog, link
   • Vadim will send letters to GSoC for mentors about student’s articles
4. C++14 is approved as default C++ configuration for 5.0. Major concern: OpenVINO and CentOS 7.x. The team will accommodate in case of issues.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

[most of the meeting was spent discussing strategy, org structure, partnership etc.]

1. Vadim:
   • 3rd PR with automatic Julia bindings generator has been submitted.
   • Improved RANSAC: build & test failures have been resolved. Should be merged soon.
   • Two GSoC projects on text detection & recognition have been reviewed; provid very good quality and very good performance
2. Anna:
   • OpenCV hardware program is in progress; Alex from x.ai continues working on h/w & s/w infrastructure
   • 13 PRs have been reviewed
   • Alex is working on RISC-V build infrastructure; docker image is ready, OpenCV builds and runs under Qemu.
   • Some preliminary work on DNN module
   • OpenCV blog: another article is published. Some more abstracts have been reviewed.
   • 2nd part of OpenCV video is in progress. 2 videos are awaited.
   • OpenCV development partnership details have been published
3. Gary:
   • PR on Macbeth chart detector has been merged
   • Deepnet AprilTag detector is in progress. The goal is get the similar quality that Gary has in some production system.
   • Some financial activities on OpenCV.org
4. Edgar:
   • Kornia 0.4 should be released by the end of the week
   • Q1: what would be the motivation to write blog posts for opencv.org? How big is the audience? A. 20K subscribers so far. With some more effort the audience could be increased significantly. Q2. Any OpenCV schwag for OpenCV blog authors? A. definitely good idea, will consider various options how to implement it.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

1. The second round of GSoC evaluations is successfully completed.
2. The first-anniversary video will be available on Thursday. Working on the second video in progress.
3. Pau Rodriguez’s article will be published today or this weekend. Lydia will send the link to the team. Also Lydia, Anna and Vadim will meet on Monday to discuss suggesting abstracts.
4. DNN module changes:
   • Change graph organization logic. Nodes -- operations, connection --memory buffers.
   • Make back-ends independent. Make fusion operations independent from the back-end. Make multi-stage fusion.
   • CUDA interface for DNN
   • Non FP32 networks: FP16
   • Multi-stage model loader. Report all issues like unsupported operations for all nodes, but not stop on the first failure.
   • The features and changes will be discussed internally and then shared with the community.
   • Add functionality to Python bindings to download DNN models and cache them
5. Enhancement: unicode/UTF-8 support with InitFont & PutText #17980 --Vadim already commented on the proposal. There is a plan to add FreeType and modern fonts support to 5.0. The ticket targeted to milestone 5.0.
6. OpenCV 2.4 patch -- OpenCV 2.4 branch is closed for changes. Only security fixes are applicable.
7. 3D vision module will be included in OpenCV 5.0; list of functions and the interface is under discussion.
8. Required C++ standard for OpenCV 5.0 will be discussed next week. It's suggested to raise it to C++ 14.
   • Draft of evolution proposal to make OpenCV API easily extendible is composed: https://github.com/opencv/opencv/issues/17997

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

1. Vadim:
   • GSoC: the 2nd round of evaluations. 10 evaluations are not completed yet.
   • GSoC: RANSAC, Julia bindings.
   • OpenCV 5 evolution proposal
2. Gary:
   • the student will address the comments in order to merge Macbeth chart detector in
3. Vincent:
   • trying to remove some very old pieces of OpenCV (pre 1.0) in the internal codebase.
4. Anna:
   • tested a bunch of PRs this week
   • RISC-V project: finally managed to build RISC-V version of OpenCV and run the tests on QEMU (some unofficial branch of it) with RVV enabled.
   • some patches for OpenCV DNN are coming
   • video for OpenCV film finished. uploaded to Youtube, will be unveiled next week.
   • finishing anniversary site
   • new logo has been created. PR is created. opencv.org/resources/mediakit
   • new article for OpenCV blog is basically ready, waiting for the review
5. Stefano:
   • the tracking project is going well. The project required some serious refactoring of OpenCV DNN. Instead, the core team did this refactoring. The scope of the project needed some changes. Hopefully, next time during GSoC we will avoid such situation
   • need to refactor ONNX importer (and maybe other) to simplify importing of 3rd-party models
6. Shiqi:
   • text detection & recognition project goes well. The student is now working on improving LSTM inside OpenCV DNN.
   • development partnership is evolving. have 2 partners already, more is coming. Need to publish the program finally.
   • two 3d sensors have been sent to Anna
7. Edgar:
   • PyTorch 1.6 is out. Kornia is being updated to support this version.
8. Vladimir:
   • GSoC projects & OSPP projects are under control. All GSoC students have submitted PRs.
   • ORB SLAM investigation is going on
   • Dynamic fusion algorithm is being developed
   • Support for Audio is being added

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

1. The first anniversary video can be published on Tuesday. Current version is sent to Vadim, Satia, Victor for approval. Anna, Lydia and Gary work on regular video interviews about OpenCV, computer vision in general and related topics. Schedule: weekly or so. The interviews can be interleaved with screencasts or other non-interview formats. + podcasts format

2. Alexander is now working on installing all the necessary software stack (Jenkins-based) to the local master machine and then to worker machines. The primary goal is to test RISC-V build of OpenCV (using RISC-V emulator). Other important targets are “always-enabled” ARM build, probably Android build. It would be nice to have powerful ARM server, in this case we could use Docker to test Linux-on-ARM and Android-on-ARM much more efficiently.

3. New OpenCV logo has been designed and approved. The site and OpenCV repository will be updated shortly.

4. Vadim:

   • Had some conversations with OpenCV DNN folks. The agreement is to make a series of evolution proposals on how to refactor OpenCV DNN to make it more efficient and more convenient to use.
   • Macbeth color checker detector (GSoC project) has been accelerated by more than 10x. Both traditional vision and deep learning algorithms now work reasonably fast.
   • Experimented with Python modules. The idea is to convert OpenCV Python bindings from a single-file binary-only representation (cv2.so/.pyd) to a directory:

   cv2/
       cv2_bin{.so|.pyd}
       __init__.py # will contains `from .cv2_bin import *`
       <some more python scripts>
   

   • that is, some bits of Python code can be used in the bindings and that will make some API (like FileStorage etc.) more “Pythonic”. The main motivation of this change is to make the code that automatically downloads DNN models a part of OpenCV Python bindings.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vincent:

  • OSS-Fuzz was run on OpenCV 4.4, just 1 issue was found. It has been reported. https://oss-fuzz.com/testcases?open=yes&project=opencv

• Vadim:

  • GSoC: Julia bindings are in progress (3rd phase)

  • GSoC: color checker detector accelerated.

  • GSoC: RANSAC

  • Deep Learning: discussed with D. K how to improve OpenCV DNN:

    • add functionality to OpenCV Python bindings to simplify the use of deep learning models
    • ONNX importing issues
    • Performance of MobileNet SSD was improved (by adding depth-wise convolution)

• Stefano:

  • It's important to have more comprehensive analysis of imported topologies in OpenCV DNN. Not to fail as soon as possible.
  • It would be useful to have information about used topologies how it was trained. Because the models can be updated.
  • Explore the use of IE IR for OpenCV: https://github.com/openvinotoolkit/openvino/blob/master/docs/MO_DG/IR_and_opsets.md

• Anna:

  • working on various OpenCV PRs (9 PRs). Sergey from xperience.ai started to look at DNN-related issues.
  • Alexander Sm is now deploying Jenkins, setting up building farm for OpenCV H/W program.
  • The draft of the 1st episode of OpenCV film is ready.
  • Anniversary site is almost ready, collecting final materials for it.
  • More abstracts for OpenCV blog posts have been received and are now reviewed
  • The post about G-API has been just published

• Stefano & Anna:

  • Maybe it makes sense to provide Docker image for OpenCV.

• Shiqi:

  • 3D functionality in OpenCV 5?
    • Yes. 2 parts: 1) data acquisition from depth sensors/lidars, 2) point cloud processing.
    • probably, some 3D sensors will be shipped to Russia to test them.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• OpenCV 4.4.0 release status -- is in progress right now. Will be published soon.
• Branch “next” will be created after release.
• OpenCV license migration status:
  • Vadim Pisarevsky will move old license to docs or other place and switch oto Apache 2.0 license everywhere.
• Anniversary video in progress
• OpenCV 5.0
  • AA: milestone "5.0" with "next" branch
  • review issues with "future" label
• New OpenCV logo discussion started on

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Is now out on Kickstarter: OpenCV AI Kit

• We are starting discussions on the using graph API (GAPI). The reason for interest is this is probably the right way to go since it can allow optimization backends to hardware.
  • Specific example: G-API would allow us to express opencv functions in an optimal way on the Movidius chips in the OpenCV AI Kit vastly expanding the programs that can be written and expressed on the device
• OpenCV.org Tax day today! We are a non-profit org, so our tax bill is $0, but we still have to file
• Macbeth project -- pull requests has some changes requested https://github.com/opencv/opencv_contrib/pull/2532
  • Ajit is working on answering the change requests, and documenting the winer_filter license. will get in tonight
  • We also have a preliminary working April Tag detector working, pull request in a week or two
• Anna: Technical side -- pull requests
  • DNN pull requests talked to Dimitri to gather thoughts on how to improve DNN
    • DNN issues focus to move it along
    • Refactor DNN cpp file -- in order to help generalize optimizations over different platforms
• Blog authors -- 3 candidates
  • Video production takes 2 days
• 20th Anniversary
  • prototype site is up
  • opencv[not yet]/anniversary
  • We have about 20 vlogs in the queue
• OpenVino 2020.4 (includes OpenCV 4.4.0) is out!
  • https://github.com/openvinotoolkit
• Pull requests on Ransac has been updated based on
• ObjectiveC and swift bindings progressing
• OpenCV 5 comments on the internal drive -- where we want to library to evolve to
  • OpenCV DNN convolution accelerations
• Vincent back from holiday
  • 4.4 running sanitizers now in clang, good to use
  • clang sanitizers
  • OpenCV SW issues going down over time https://www.cvedetails.com/product/36994/Opencv-Opencv.html?vendor_id=16327
• Release 4.5 will come out (long term) switching to Apache 2.
• OpenCV 5.0 will be completely under the Apache 2 license for better patent protection
• Edgar Lydia cover letter for authors blog

• Name
  • todo

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Julia bindings: 2nd PR has been merged
  • RANSAC: 1st PR is being reviewed
  • OSPP: 2 projects: point cloud simplification & QR encoder/decoder.
• Anna:
  • several PRs on CUDA support are being reviewed
  • prepared guidelines for OpenCV blog authors.
  • videos for OpenCV film: 2/10 will be prepared by xperience.ai (hopefully, will be finalized next week).
  • scheduling meeting about model storage solution
• Gary:
  • macbeth color chart. Gary has tested the submitted code. Student is switching to AprilTag part.
  • some opencv.org financial activities are going on
• Vladimir:
  • 4 GSoC projects are on track.
  • SIFT project mentor (Vitaly) was on sick leave, now he is back
  • YOLOv4 – found some issues with OpenCL. Probably, need to disable OpenCL on iGPU.
• Edgar: *
• Stefano:
  • GSoC Object tracking project. There are some problems with exporting model to ONNX format compatible with OpenCV. The version from PyTorch Vision should be compatible with ONNX, trying to use it.
  • PlaidML started to integrate OpenVINO; there is also some experimental Vulkan support.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Vadim:
  • Julia bindings: 2nd PR has been submitted
  • RANSAC: 1st PR has been submitted
• Anna:
  • GSoC student has submitted PR on PyTorch/TF+OpenCV tutorial
  • RISC-V GSoC student did not submit PR, but he is doing well
  • xp.ai is working on PRs
  • xp.ai is working on practical aspect of external build farm for h/w partnership program
    • Jenkins will be used as master machine with externally visible IP, and it will talk to build slaves for CI
    • It's under question how it will connect to build slaves in China because of firewall
  • Film raw materials are almost prepared, film preparation is in progress
  • Web site for 20th OpenCV anniversary should be ready by the end of the week
  • Blog post for OpenCV.org is ready and published
• Gary:
  • Macbeth GSoC project PR has been submitted. Gary is now testing it.
  • Current implementation (in C++) is not very fast, ~1fps. Gary's prototype implementation in Python takes ~same time.
  • Hybrid implementation (deep nets + traditional CV for checking pattern) is much faster.
  • Sample can assume that the model is downloaded and put into the current working directory.
• Edgar:
  • Kornia 3.2 is going to be released soon
• Stefano:
  • ONNX smooth import is still not solved problem
• Need to find some hosting for OpenCV models (GSoC results, ...)
  • In general, the problem of using deep-learning based samples/API in OpenCV has 3 sides:
    • Hosting for the models <== need to find good hosting
    • API to download the model and cache it <== need to provide standard API for that (at least for Python, there are some bits of code which we can re-use)
    • Better diagnostic if something goes wrong with the model (it's too complex to be loaded) (need to modify ONNX and other importers in DNN to give better "global" diagnostic)

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Gary: PR on color calibration submitted (https://github.com/opencv/opencv_contrib/pull/2532). Need to review it.
• Macbeth detector is in 2 variants: purely classical method and hybrid, where deepnet-based is used to localize macbeth quadrangle with further verification by classical code.
• Stefano is working on plugin for VSCode to let some PR's (for OpenCV or TF) to be tested automatically from within VSCode (CI testing).
• PRs on OpenCV.js have not been submitted yet, the work is organized in H.N. fork of OpenCV.
• the second PR on Julia is about to be submitted. Some merge conflicts now need to be resolved by because of its overlap with 1st stage PR (that's already merged).
• Internal PR on RANSAC is submitted to Maksym repository, still being reviewed, looks good. Will be submitted to opencv_contrib soon.
• In OSPP submission period is over, now the proposals are reviewed. Several proposals on QR code encoder/decoder, one on 3D. Need to decide by June 30th.
• Tutorial on PyTorch & TF -> OpenCV, 4 stages: segmentation, classification, detection, blog post. PR on segmentation is submitted.
• RISC-V project: PR is difficult to submit at this stage, but student is working well.
• Videos on OpenCV film are prepared. Several interviews are already done.
• Work on PRs continues.
• Blog post for OpenCV.org (on saliency) is finished (content), now formatted to be published.
• Finalizing the anniversary site. When 1st video is ready, it will be released: Vadim is to send video & photo.
• There are some requests from young people (novices in OpenCV and CV) to participate in OpenCV. Need to find some relatively easy tasks to get started.
• Vladimir: several major directions:
  1. OpenVINO release is being prepared.
  2. some Intel-related things like fixing VTune instrumentation.
  3. GSoC:
  • PR on Visual trackers has been created.
  • WASM (OpenCV.js) PR is created
  • SIFT PR is submitted
  • no PR on 3D is submitted yet
  4. several other activities:
  • QRcode (improved detector
  • which supports distorted images etc.)
  • ORB-SLAM
  • YOLO v4 support (done)
  • FlowNet v2 support
  • Adding audio support is on-going
• After OpenCV 4.5 release master branch is going to be used for pre-5.0.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• OpenCV 4.4.0 release status
  • Next releases: 4.4.1 or 4.5.0?
• OpenCV video: at the moment, questions are being sent to speakers and video recordings are being collected.
• OpenCV blog: new article will be published next monday
  PRs & Issues:
  • FFmpeg / libav support PR #17489
  • Added a copy constructor and an overloaded assignment operator to FileStorage #17458 - "deep copy" make sense or keep a smart class (smart ptr behavior)

• OpenCV 4.4.0
  • bump 4.4.0-pre version
  • expected release date: July 6-10
  • Changelog placeholder is on Wiki
• The next release -- 4.5.0
  • Master will be used for 5.0 preparation (alternative: rename "master" to "5.x")
  • Alexander Alekihin will create branch 4.x for 4.5 and further bug-fix releases
  • GSoC work will go to master
  • Some old dependencies will be dropped. Decided to document baseline on Wiki
• Anna will update about page on OpenCV.org
• Lydia will collect a list of team members with photos and publish slides about OpenCV
• FFmpeg / libav support PR #17489 -- modify test to check if autorotate option is applied correctly.
• Added a copy constructor and an overloaded assignment operator to FileStorage #17458 - decided to keep a smart class (smart ptr behavior).

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• C-API is not documented anymore and will go with 5.0

• GSoC

  • Web GPU -- contact
  • Matrix GPU https://matrix.to/#/%23WebGPU:matrix.org
  • Pull requests of Julia under review
  • Ransac prepared draft pull requests internally
  • Tracker -- no pull request yet, but coming
  • Macbeth
    • Color correction guy in China to sync
  • Text detection results
  • Javascript progress going well
  • Tutorials ?
  • Risc-V on track
  • Sift?
  • Plane and geometric shape fitting?
  • OpenCV js
  • Depth fusion?
  • Not sure on
    • js
    • sift
    • text detect
    • 3D depth fusion

• China summer of code OSVP

  • http://www.opencv.org.cn/?page_id=411 (information site, ideas list)
  • https://isrc.iscas.ac.cn/summer2020/#/index (Application site)
  • 3 days left to apply
    • Barcode detection projects
    • Color

• Monday tutorial on features and ransac modification -- mentions improvements to Ransac in OpenCV coming

  • https://local-features-tutorial.github.io/

• Reviewed pull requests

  • Risc-V
  • OpenCV-Contrib
  • Macbeth in (Vadim will check)

• Anniversary preparations

  • History reviewed by Vadim
  • Anniversary site ready, currently hidden
  • Prepping questions for video interviews
  • First blog due soon
  • Prepped blog format site

• Who is part of the technical committee

• Release OpenVino 2020.4

  • OpenCV releases now aligned in time

• Ficus

  • 5 programs now compiling with correct results

• API changes over time for open source projects

• Name
  • todo

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Anniversary video
  • Now more than 23 people have agreed to do interviews
  • preparing video instructions now
  • New (nice) site will soon join opencv.org site
    • Will release when we
• MLPerf had talks about how we might work together
• GSoC
  • Macbeth project moving along -- was having trouble on getting the DNN approach to work with the network trained in PyTorch, so switching to TensorFlow training right now
  • The classical approach works, can use the above as a mask, and will have it's own API
  • Risc-V
    • Ran full set of tests on Risk-V and they pass, so
  • Models => DNN
    • Converter classification, detection and segmentation is the focus
    • MLPerf can connect
      • OpenVino is in contact
      • TinyML?
  • Julia bindings pull request in
    • Mostly ready, but unlikely make 4.4.0 release
    • 3 stage project
      • 1 drop of bindings with cmake scripts etc. (PR submitted), DNN support (mostly ready), automatic generation of bindings
  • RANSAC
    • Refactoring to bring into OpenCV Coding style
    • Planned slow down for student exams, but already ahead of schedule
  • Text detection
    • Model already prepared, running in DNN
    • Adding functionality for post processing
  • Support audio component (audio classification)
    • Audio from video
• OpenCV 4.3.0 Just released with OpenVino
  • Next release 4.4.0 ~about a month from now
  • 2020.2 new components
• License (evolution proposal) going to Apache 2.0
  • Particular evolution proposal is here
• Chinese "GSoC"
  • 2 applications QR code
  • Macbeth project -- connect them
• Ficus -- compiler ran for the first time
• Webinar "OpenCV Overview" organized by opencv.org.cn
  • Plan to have a series of webinars
  • Also wants to be on OpenCV youtube channel
• OpenAI lab (Shenzhen)
  • Tengine (focus on performance on ARM -- multi threaded)
  • C++ to C for better performance
• Arm servers needed
  • They are expensive
• Risc-V servers
  • SiFive cores
  • Contact when we have working code
  • NGraph working on common infrastructure

• Gary
  • CC Anna to ML Perf people
  • Connect Chinese Macbeth

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• License migration process
• Hackathon status.
• Correct 1D Mat support in OpenCV (audio data and others libs)
• OpenCV video script: work in progress, shooting will launch next week
• OpenCV passes all tests on RISC-V 64 with Qemu, decides exception “not implemented” in g-api tests #17470
• PRs & Issues: CUDA: drop support of old CC #17478
• OpenCV analytics

• License migration process
  • Open issue about license migration
  • Announcement on opencv.org
  • After 4.4.0 release: update PR template, update licensing documents
  • Affected GSoC PRs should be opened under Apache 2 license in the PR's description
• Hackathon:
  • Not so many contributions, there is no sense to make blog post about status.
  • AI: remove link to hackathon announcement.
• 1D Mat
  • 1d mat trieted differently in different functions: row or call.
  • OutputArray and Mat treat it differently.
  • Std::Vector good choice for 1d data.
  • Dnn uses cv::Mat as container and we need 1d Mat for it.
  • Vadim recommend to create Evaluation Proposal for it.
• Drop old CUDA CC:
  • The PR removes just cmake part, but does not introduce changes in code itself
  • It’s reasonable to change hardware requirements in major release: 5.0.
  • Alexander Smorkalov proposed not to remove deprecated hardware configurations

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Can add donate button to OpenCV by a text file
  • https://github.com/qubvel/segmentation_models.pytorch
• LICENSE CHANGE!
  • OpenCV is going to change from BSD-3 to Apache-2
  • Vadim will send an announcement
  • Will wait until after hackathon announcement
• GSoC
  • Julia bindings
    • first pull request in already works well on Linux and Mac.
    • Windows coming
    • Deep learning being added now
  • Ransac
    • First code review
    • Lot of technical work to do, but most dependencies eliminated, only Eigen dependency
  • Risc-V support
    • In progress
  • SIFT, Trackers in progress, 3D -- first version of algorithm implemented
  • Remind students that first pull requests need to get in in June (even if it's just a stub)
• Hackathon this week
  • Lighter work than last time
  • 11 pull requests were merged
• Script for interview films being finalized
  • Interview best stories about OpenCV
• Intel side 2020.3 4.3 OpenCV release out
  • June 9 release aligned with next OpenVino release (which is now also open sourced)
• Annotation tools
  • We have a company that wants a repository
    • Verses partnership program -- to hold to that or not
    • Policy (startups often fail -- code needs
  • Discuss with Nikita who runs Cvat? (Cvat annotation tool)-- free for any kind of use)
  • Joint announcement post on OpenCV.org?
  • Fall under collaboration program
  • Roadmap -- with OpenVino
  • Partnership program
    • Feature comparison table vs Cvat
    • Justification to put in
    • And partnership

• Gary
  • finish interview question review
  • send out requests

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Patent check on OpenCV code
• OpenCV blog:
  • Paper’s draft
  • Potential blog’s authors review
• OpenCV video script
• PRs and Issues:
  • ObjC integration: proposal is to keep current package for 4.4.0 release
  • Mixed raw pointers and Ptr<> in features2d java wrappers #11268
  • Create separate section and table of contents for contours articles #17338
  • Linking Tutorial Pages to Their Github Source Page #17342
  • OpenCV VideoCapture::read reads an empty matrix mid video #15352
  • Many documentation PRs

• OpenCV 4.4.0 release schedule: "Pre" version on June 9-10, final release: beginning of July
• Matlab bindings are out-of-date. Could be removed from OpenCV contrib. AI to Alexander: prepare wiki page with information about language bindings and distributions. We can add links to 3rd party bindings with “non-official” labels
• OpenCV anniversary:
  • Lydia works on script for videos
  • Anna works on web site for anniversary. Milestone: June 7.
• ObjC
  • Vadim: Integration looks good in general.
  • Alexander Alekhin: The patch changes framework name or something else and the generated framework is not compatible with previous releases. Proposal: release with old name in 4.4.0 and change name for 5.0.
• Mixed raw pointers and Ptr<> in features2d java wrappers #11268 -- Rewrite constructors to generate Ptr.
• Create separate section and table of contents for contours articles #17338 -- version switcher should work as doxygen id is the same for the pages. Alexander Alekhin is ok with patch.
• OpenCV VideoCapture::read reads an empty matrix mid video #15352. The attached videos contain duplicated frames. OpenCV works with codec directly and does not handle this (muxer should do it). AI to Alexander Alekhin post findings to the ticket.
• Many documentation PRs - no decision made

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• One membership to connect, and revamp the systems
  • Build up Answers forum
    • Move to discourse https://www.discourse.org/about
    • Maxim
  • Build up email list
  • Seat for feature priority
  • Certification for hardware
    • But be careful about legal implications -- tested/compliant/OpenCV/Capsuls/registered
• Hardware program => "Membership program"
  • Gary to start off suggestions, carry on to Friday meeting
    • Priority on direction, bug fix
    • Kickstarter type things, or contribute and set priority such as ROS does
  • https://docs.google.com/document/d/1Da1YtKkaBOxr3TNW8j38PiZ07NmJQb5sYCgxEsAARSA/edit
• GSoC
  • Download models from URL & cashable
  • There is a script for downloading deepnet models
  • OpenCV hosts, but people can pay (github and google drive)
    • Cloudflare in front of it for free
    • Create downloads
  • Macbeth moving well
  • Julia bindings -- student nearly finished pull requests.
    • Mac, Linux, not yet Windows
  • Ransac -- meeting model fitting in two point clouds
    • useful for 3D cloud too
    • got rid of all dependencies except Eigen
  • Tutorial conversion pytorch to dnn. Planning in details
• OpenCV hackathon this week
  • Working on pull requests
• Blog posts by a CVPR author
• Interviews getting sent out. Need help setting up format
• OpenCV.org is now registered as a non-profit, thanks Satya!
  • Unlimited accounts
  • OSVF as an alias?

• Satya
  • Talk to Maxim about Answers
  • See if you can get aliases for OSVF in opencv
• Gary
  • Put together membership program proposal
  • Maybe add Takeo to my request
• Vadim
  • Takeo request

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• OpenCV license: BSD => Apache 2
• GSoC update
• Hackathon
• PRs & issues for discussion:
  • Linking Tutorial Pages to Their Github Source Pag#17342
  • Many documentation PRs
  • Add eigen tensor conversions #17320
  • Objc binding #17165 -- no review progress
  • Flush to zero Convolution denormal weights #17295
  • Create multiscale_sharpen #17274
  • Create separate section and table of contents for contours articles #17338
  • Implement QR Code decoder and encoder #17290

• Team are discussion license Apache 2.0 changing
• GSoC update
  • The first evaluation -- June 29th.
  • All mentors connected to students and most of them started work earlier.
• Hackathon:
  • Announcement
  • Tickets & PRs: 1, 2

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• "Google summer of code" in China
  • https://isrc.iscas.ac.cn/summer2020/#/
    • 5 projects submitted drawn from the list we didn't fill from GSoC
    • June 30th decision
• Extend llvm compiler to extend extrinsic to Risk-V
• Ransac: Refactoring Possibly Eigen as a dependency
• Point cloud -- also needs better ransac (plane fitting etc). Possibly Eigen as a dependency
• Text detect and recognition w/o licensing issues
• Tracker project ? keep Stephano in loop
  • Detection and Recognition ... should this be a high level API?
  • Feature extraction -- load network and coordinate on GPU/CPU (when to switch ... impact on inference time)
    • Object tracker should send to Stephano weekly report
  • Sharing of deep backbones -- don't put this on students this summer (creates dependencies -- too hard)
    • Might take a year on optimization on how to share backbones
• Javascript bindings
• Macbeth. Develop as if the model has been downloaded and running in DNN
  • Pre and post-processing in DNN tensors on input and output
• Deepnet glue between models ... vision capsules high level I/O
  • 2 years ago high level API on I/O started
• Pull requests 56 active day by day seems to be equilibrium level
  • 6 pull requests this week
  • Merged several on documentation
• Published on-site about blog and Hackathon
  • Article for blog wasn't posted Valeri Current state of object recognition
• Students who can start should

• Name
  • todo

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• C++11 compatibility is making porting easier according to some in Google
  • TR1 in OpenCV tests could be removed since compatible with C++17, but minimal is C++11
  • More useable for real applications and apps
  • Fuzzing helping with invalid data imreads etc
  • Starting earlier
• Vadim
  • Started communication with Julia bindings project
    • Adding deeplearning support
    • Then rest of time spending on adding automatic bindings
  • Ransac improvements
    • Headers will need refactoring, next meeting on Friday.
    • Wants to start earlier
  • Risc-V optimization -- tomorrow meeting
• Gary
  • Macbeth cart
    • Prepare pull request of existing code
    • Dataset
      • Can maybe get from this paper https://github.com/pedrodiamel/colorchecker-detection
• Anna
  • Pull requests working on several
  • Risc-V project support ongoing
  • Interviews being prepared for OpenCV
  • Working on the blog -- community input
    • Video -- every 2 weeks
    • ROS
• OpenCV hackathon in last week of May
  • Focusing on
• Russian holidays last week, so light progress
  • Next week, focus on GSoC
  • Object tracking
  • SIFT
  • SLAM
    • https://github.com/luigifreda/pyslam
  • Optimization for Javascript
• Hardware partnership program
  • Want to participate in China, but in different format: Arm, Risk-V, 3D sensors, ToF

• Name
  • Set up meeting about hardware program -- Western vs China version more sync'ed

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Two-stage picture reading for processing decompression bombs on the user side.
• Hackaton
• OpenCV blog: new article here
• ObjC / Swift bindings: #17165
• Please add to read me as a development resource: #17179
• Create Non-Transparent OpenCL API for better GPU / Accelerators utilization #17216
• Suggestion for creation of OpenCV technical notes #17217

• Decompression bomb. Filed ticket #1724, targeted for the next Hackathon.
• May 25-31 -- date approved. Lydia works on the announcement. Anna, Garry, Vadim, Satia will help to promote. Alexander & Valdimir are ready to support the activities.
• OpenCV 5.0 release planned for autumn.
• Internal: OpenCV 30 anniversary. Intel sponsors video preparation. Need to ensure on Intel and OpenCV.org collaboration. Anna proposed to prepare a list of people for the interview and write a script. Lydia drives this activity.
• Dmitri’s article is published. Anna will provide feedback on formatting.
• Vadim Pisarevsly will look at Obj-C bindings and Non-Transparent API proposal.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Put together list of video interviews for computer vision and also opencv history
• Vadim will help with the following GSoC OpenCV projects:
  • Text detection, 2 projects (mentored by Shiqi and Vladimir T)
  • Ransac improvements (mentor)
  • Julia bindings (co-mentoring with Sayan)
  • RISC-V support (mentored by Alex from x.ai)
  • Going to write on SIFT improvements (mentored by Vitaly T)
• Vincent
  • New OpenCV being more standard makes OpenCV makes builds easier
• Anna
  • Pull requests being delt with
  • First blog upcoming
  • Risc-V side prep -- make sure can build and compile
• Satya
  • OpenCV/OpenVino -- OpenVino said they'd fund competitions
    • Conference Agricultural AI -- Remote sensing image analysis -- look up page in CVPR
    • Join w/Kaggle in a challenge

• Anna
  • Start spreadsheet for interview contacts
• Gary
  • Look up CVPR
  • Agricultural data from Cognitive Tech
  • Covid-19 data flicker site, mask/no mask
  • NYTimes team
  • CARLA
  • Update mentor email list
• Vadim
  • GSoC project table

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• GSoC
  • 14 Slots -- had 17 good project, 2 went to Tensor Flow,
    • Have 14 solid ones
  • Need mentors to start communicating
• OpenCV 5 postpone to Sept
  • Coronavirus delaying ARM optimization
    • Single threaded is great, but needs better multi-threaded support
    • Need to start extensive testing on ARM using a powerful ARM server (preferably, 16+ cores w. lot's of RAM and fast SSD drive)
  • GSoC results will be integrated
    • Important this year to have the students put out tutorial video
  • Can have Zoom seminar about all this and/or also on Youtube
    • Tutorials, new features, can pyimagesearch
• Create a Video channel for OpenCV on YouTube
  • Shiqui Yu has a bunch of edge content
• Hackathon
  • Mark pull requests for hackathon -- focal on these (procedure to close)
  • Reviewed several
  • Committed part of RISC-V needed for GSoC
• Blog
  • Working with authors for OpenCV blog already. One is ready to go
• Data augmentation guys, fast augmentations based on OpenCV
  • Want functionality for basic things -- simple to scale image
  • Easier optimized interface (scale, lighting ...)
  • Decompression bound -- send a warning to user or crush
    • Could you just resize?
• Managing Large images (Friday meetings) 2am
  • Super quick size check read the header, then data
    • Need to expose 2 steps so user can warn or abort on too huge images
  • Read images into the provided buffer to save memory allocation
  • 1-2 weeks for basic API
  • Needs to be Fuzzed a lot because reading opens security concerns
• oneAPI
  • https://newsroom.intel.com/articles/fact-sheet-oneapi/#gs.5ez60e
  • for CUDA Cudadnn Augmentation kernel
• OpenVino some ideas
  • 20th birthday ... maybe funds
  • Talk to luminaries -- video series -- see where OpenCV is used
    • Scholarship or Challenge
    • $5K for implementing a features
• Come up Challenges list w/Prize money
• Data partners for OpenCV
  • Could do some free data annotation
  • Possibly in conjunction with challenges
  • Windows installer
  • Multi-camera calibration
  • Agricultural stuff

Core

1. GSoC 2020 -- 14 are allocated and ready to go.
2. OpenCV slack channel is community-driven, not handled by OpenCV core team. @Grigory participates in the channel and will provide link. Lydia will update link @opencv.org. Ticket: https://github.com/opencv/opencv/issues/17142
3. Grigory has feedback from Python users community. AI to Grigory to provide list. Recommendation: to authors: submit a ticket to have solution to talk to authors directly.
4. OpenCV team plans to organize Hackaton at the end of May. AI to Alexander to prepare tasks list.
5. April 30 - May 6 CI downtime -- maintenance.
6. PRs:
   1. SIFT: moving to the main repository: #17119. Initial work is done by the core team (Alexander Alekhin). Adjusted GSoC 2020 proposal accordingly. Restored implementation breaks C++ backward compatibility due to Visual Studio compilation issue. AI to Vadim Pisarevsky to review.
   2. core(matrix): Negative values checks #17077 -- the changed code is not well covered with tests. Alexander Alekhin plans to look at code coverage report. The proposed solution is not complete for now.
   3. Fisheye model extended with additional plane to support FOV > 90 in projectPoints #16926 AI to Vadim Pisarevsky to handle the proposal.

• Vadim
  • Bring up managing image read in technical meeting

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• GSoC 16-17
  • All have mentors
• OpenCV 5 (20th anniversary)
  • ARM Optimization (so far, mostly single threaded)
    • Some in multi-core, more work ongoing
  • CUDA optimization -- contribution (inference but it tunes itself once)
  • API, SIFT to main, ASift
  • Color SIFT max gradient over colors
  • Depth/ToF
• Vadim
  • Optimization of Risk-V discussion, team is working on
  • Waiting for Google for SLOTS
• Anna
  • Video writer clean up
  • Initial Risk-V support
  • Blog Post
• Vladamir
  • Gaussian blur polarization ... but code size too large
  • Ipp has this can customize dynamic for each CPU, but OpenCV is static and linker cannot split
  • GSoC Choose mentor for tracker -- Done
  • DNN improved with NGraph. Trying to enable new topologies for optical flow
  • Javascript for "intelligent scissors" -- good for dataset annotation
  • Published an article in Russian, translate for BLOG
  • OpenVino release 2020.3 end of May, long term support release
  • QR detection recognition
  • Object tracking -- separate tracking repository: Datasets for evaluation
    • QR, Obj Tracking
• Edgar
  • O.21 Kornia release
  • Differentiable tensors w/o TorchVision
  • Data augmentation define own pipelines by overloading operators
  • Deep descriptors Haar
  • Online talk https://www.youtube.com/watch?v=zltoYKu4ct0
  • Support for TPU
• Gary
  • Legal docs for joining/leaving OSVF.org coming

• Name
  • todo

• GSoC 2020 status
• Started work on RISC-V support in OpenCV: implemented simple toolchain file for GNU RISC-V64 compiler, enabled minimalistic build
• OpenCV blog status
• Runtime algorithm selection for cudnn convolution #16900
• PIL #17068
• Samples and tutorials for the Dnn High Level API #15240
• Fisheye model extended with additional plane to support FOV > 90 in projectPoints #16926
• Imgproc: templmatch: Add support for mask for all methods #15214
• Added Dashed Rectangle feature in Drawing.cpp #16880

• GSoC: current amount of proposals - 12-16. Action item to all mentors to select projects and press the “mentor” button on the proposal page
• Alexander Smorkalov starts work on RISC-V support in OpenCV as preparation step for RISC-V project in GSOC 2020
• OpenCV blog: Articles verification group has been created (Vadim Pisarevsky, Vladimir Dudnik)
• #16900: Ask author to make benchmarking approach optional. Add item to hackathon to align the solution with OpenCL.
• Samples and tutorials for the Dnn High Level API #15240: -- AI to Vadim to add checklist with remaining items
• Fisheye model extended with additional plane to support FOV > 90 in projectPoints -- AI to Griogry to comment the patch. Should be closed in several days, if author does not respond.
• Added Dashed Rectangle feature in Drawing.cpp PR #16880: -- AI to Vadim to comment.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• GSoC
  • Looks like we might have ~10 "really want"
    • Where to get data from -- generate and collect
  • Line 8 -- April and MacBeth tags
  • Line 10 Deep learning inference on video -- expand
    • Not yet ready to incorporate quantization optimization, but for tutorial
  • Line 12 RISC-V optimization -- company dropped out, but get support from whole University group so we can go ahead -- lines for acceleration of deep learning
  • Line 18 Data augmentation
• Blog
  • Dimitry blog posts in Russian, translating (Deep style augmentation)
  • Several other posts are being studied
  • Over several months
  • Will allow initial, limited blog cross post
• RISC-V
  • Some initial support already for Friday Tech meeting
• Website
  • Will be fixing some of the drop downs
  • Special blog button top right

• Gary
  • See if Reza wants to co-mentor

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• this and the next week main task: select best students for each project (right now about 14-15 potentially interesting projects with good candidates).
• xperience.ai:
  • OpenCV blog - several people already answered. At least 2 people agreed to write posts on OpenCV.org
  • Working on PRs: since OpenCV release is out, ~8 PRs can now be merged.
• Intel:
  • OpenCV 4.3.0 is out
  • continuing working on adding Audio support to OpenCV
  • parallel Gaussian blur (there are faster alternatives)
  • visual trackers. Found several datasets to estimate the quality
• OpenCV hackathon in May is still feasible
• Stefano: made some review
• Vincent: glad that 4.3.0 is out! Will work on updating the internal build

• Finalize GSoC selection

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Google Summer of Code
• OpenJPEG
• OpenCV blog: weekly results

• GSoC proposals evaluation is underway: Alexander will add candidates links to PR, Xperience team will choose their own candidates
• OpenJPEG: looks like encoder using lossless compression parameters - Vadim Levin's task
• OpenCV blog: several people responded, one of the potential authors is starting work on the article

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Note that the GSoC timeline is shifted by a week, now updated on the ideas page
  • Go through OpenCV proposals
  • Mentors only list
  • All proposal list
  • Spreadsheet Vadim take first pass
  • OpenCV.cn group actively working on ARM optimizations for DNN
• Blog -- spreading word about new papers to more
• Everyone is now working from home world-wide
• Vladimir is OpenCV team manager,
• Dmitry is the Graph API team leader, need to connect this with data augmentation

• Vadim
  • First pass on spreadsheet
• Gary
  • Get OSVF membership legal sheet
  • Get in foils to Tong Li

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• List of prospective blog’s authors was preparing
• Feature request #162898
• One more FAQ page
• PR's #14971, #11885

• OpenCV release changelog placeholder
• Lydia is sending invitations to OpenCV blog
• Feature request #162898: Vadim Pisarevsky will look at this task and choose between Eigen and Lapack
• One more FAQ page: need to remove
• PR #14971: need to write evaluation proposal (or feature request possible ways to extend image codecs (video codecs) to accept different parameters types and close after that - Vadim Levin
• PR #11885: refresh future request,extend feature request with to take into account different camera types and mention different solvers #15650

• OpenAI labs proposed patch for T-engine support for Android (integrated).
• Need to restructure OpenCV.org a bit: add platforms (Intel, ARM, etc.) with the relevant information
• Students started submitted proposals. Need to collect all of them into a private spreadsheet visible to all the mentors and start collecting reviews in order to select the best proposals.
• xperiance.ai:
  • merged GSoC 2019 PR (into opencv_contrib)
  • wchar_t support: added to FAQ
• Intel core team:
  • OpenCV 4.3.0 is being prepared, should be in the beginning of April
  • nGraph API support in DNN (added dedicated CMake flag)
  • new DNN-based samples: tracking, scissors, optical flow (in progress: FlowNet 2.0)
• Stefano: need to change "fail-on-the-first-problem" OpenCV DNN importer policy in order to get the complete report about potential problems in each particular ONNX model.
• Edgar: data augmentation project is still very interesting and relevant. Gary: will copy it from GSoC 2019 to GSoC 2020.

• Release Status
• PRs:
  • Reworked and finalized OpenJPEG 2000
  • Multi-channel Mat and scalar handling (MatExpr) #16739
  • MatExpr + InputArray #16653
  • Do not copy standalone IPP libraries to install for static builds #16769
  • Add some docstrings to gen2.py #16767
  • Update resize.cpp #16810

• OpenCV 4.3.0 release status

  • OpenCV release may be moved for 1 week. Target date - the first week of April

• GSoC 2020:

  • Vadim sent requests to all mentors to join the GSoC platform
  • The team will review and vote for submissions to prepare request to Google till April 13.

• PRs:

  • Multi-channel Mat and scalar handling (MatExpr) #16739 - Vadim Pisarevsky will look at the problem
  • MatExpr + InputArray #16653
    • MatExpr + InputArray interop is not properly implemented.
    • Vadim Pisarevsky treats the solution as potentially dangerous.
    • To be merged after 4.3.0 release.
    • MatExpr behavior can be reworked for 5.0.
  • Add some docstrings to gen2.py #16767 would be closed. The patch does not bring sufficient add-value.
  • Update resize.cpp #16810
    • The problem looks like undocumented IPP behavour. Original code is correct. AI Alexander Alekhin to communicate with the IPP team and resolve the issue.
  • Alexander Smorkalov will close the PR with note that documentation issue is communicated to the IPP team.
  • opencv_contrib#2306 patch proposed to be merged as as is. Alexander Smorkalov will fix the second part of the submission contrib, #2247

• xperience.ai: number of open PRs went down to about 50 and stabilized there. Most of "low-hanging fruit" PR have been closed and in average the open PRs are much newer than before. So it just needs consistent maintenance to keep this number that low.
• PlaidML support in G-API is still very basic, at just concept level.
• In TF add-ons (which are similar in status to "opencv_contrib") there is growing amount of image processing functionality. It's probably going to be expanded further during GSoC 2020. On the other hand, most of this functionality is either using OpenCV or not very efficient.
• The text recognition project in OpenCV GSoC 2020 does not necessarily require biLSTM. Stefano will reply the student question about it.
• The patch with DNN acceleration for ARM has been merged, should be a part of OpenCV 4.3.0. It supports only Linux ARM. The patch with Android support has been submitted and is under review.
• Intel core team updates:
  • OpenCV 4.3.0 should come out on time, i.e. 1st week of April. But OpenVINO release might be delayed by about 2 weeks.
  • Magnetic Lasso algorithm implemented. The PR has been submitted.
  • DiamPRN object tracking network, together with the demo, has been converted to ONNX; DNN has been modified to support it. The corresponding PR is being prepared. The algorithms run at 30FPS on CPU!
  • QR detector is being improved to better detect QR codes in the case of radial distortion. PR is in progress.
  • DynamicFusion implementation is being improved.

• OpenCV 4.3.0 release status
• OpenCV development process
• Blog posts & schedule
• PRs for discussion

• OpenCV 4.3.0 release status
  • Release will be launch at the end of month
• Process improvements:
  • Lydia will send invitation to paper authors to contribute to OpenCV blog from OpenCV group
• PRs discussion:
  • #16759 will be closed
  • #16733 need validation and to launch the script
  • #13869 PR will be assigned to Vadim Pisarevsky
  • #16494, #16524 combine them into one PR
  • 15650 do not merged, back to this PR after release
  • #15214 need documentation

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• GSoC

• GSoC
  • https://opencv.org/google-summer-of-code-internship/
  • Mentors until End of May
  • SIFT
    • ASIFT
  • Tensor Flow
• OpenCV 4.3 may be released end of March
  • Better inference speed on ARM for DNN!
• Intel support for embedded to use
  • Regular build for ARM
  • Currently Rasberry Pi can be used with OpenVino+OpenCV+Neural Compute Stick (Movidius)
  • Keem Bay, which is ARM+Movidius SoC, announced in 2019
• Model Zoo
  • OpenZoo open
  • Open (Intel) Model zoo -- more optimized for Intel chipset
  • https://github.com/opencv/open_model_zoo/tree/master/models
    • DNN with OpenVino backend can run all models from both Open Model Zoo
    • DNN compiled w/o OpenVINO can run only the non-Intel models (ONNX, Caffe, TF etc.)
    • Hard to know what OpenVino accelerator doesn't cover (because of the "fail as soon as possible" implementation ideology in OpenCV DNN engine)

• All
  • sign up mentors and get students

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• GSoC considerations
• Blog
  • Note that GSoC isn't even there yet. How to keep this fresh?
• Linux foundation considerations

• GSoC considerations
  • Risk-V, Ransac have good potential already. Optimization CUDA
  • nGraph vs Graph API in OpenCV
  • What process should we use for GSoC this year?
    • What do we do to get students? Don't want too many low quality *
    • How to vet mentors? *
• Blog
  • Note that GSoC isn't even there yet. How to keep this fresh?
• Linux foundation considerations

OpenCV release is coming. This week is soft code freeze stage, the next week -- strong freeze. New features like OpenJPEG support will be merged right after release.

Alexander is working on public Hackathon report. Hopefully, it'll be ready till the next call.

1. FAQ page on Wiki can be useful for beginners and as a solution for often questions and bug reports. The page should contain question or problem and link to ticket, answers.opencv.org, Stackoverflow or any other page with discussion.
2. It's time to update issue template in code OpenCV repository. AI to @Alexander Smorkalov to work on it. Important ideas:
   • Add checklist with 3 items:
     • I updated to latest OpenCV version -- master branch for 4.x and 3.4 branch for 3.x and the issue is still there.
     • I checked the problem with FAQ, open issues, answers.opencv.org, Stackoverflow, etc...
     • There is reproducer and related data (network for dnn, test images, videos, etc).
   • Extra description should be done with xml/html comments to have short description in ticket.
   • It's possible to use a different template for bugs and features.
   • 3.4 branch is for bugs only. New features should go to master (4.x releases).
3. Ideas for blog at OpenCV.org (Alexander plans to work on tech things, Lydia -- relations with external people):
   • Promote Google Summer of Code topics as blog post with problem statements and related work. Publish GSoC results in the same way.
   • Publish technical notes about OpenCV like usage with different cameras, for different embedded platforms, python and java bindings, etc. The articles can be merged to some documentation page or wiki page.
   • Involve speakers of well known conferences to publish texts on OpenCV.org.
   • it's a good idea to promote blog posts over OpenCV mailing list.

• RFC: Jpeg2000 OpenJPEG port (#16494, #16524) -- AI to Alexander Alekhin to list all remaining items. The patch can be merged only after release.
• Support of Unicode file paths under Windows #13368, related task #4292, #5631. - It's decided that OpenCV team will not add functions with wchar_t and std::wstring. UTF8 can be used with std::string and standard C/C++ and imdecode/imencode can be used to work with custom symbols. As alternative OpenCV can support C++ iostream API. AI to @Alexander Smorkalov: close the PR, add item to FAQ page and close related tickets and topics with proper description and links. See https://github.com/opencv/opencv/pull/13368#issuecomment-509225982
• OpenCV addons #6722 -- closed.
• Update test_fitellipse.cpp resolving #10270 #16426 -- closed.

1. OpenCV cannot be compiled with some old versions, even if they are mentioned in the code.
2. @Vadim Levin will collect information about FFMPEG versions in different Linux distributions and user tickets to understand minimal version that should be supported. Current idea for FFmpeg baseline: Ubuntu14.04 or Ubuntu 16.04.
3. It's reasonable to support some old FFmpeg versions for embedded boards that usually do not get OS update.
4. Old FFmpeg versions should be dropped in 3.4 too to resolve further merge conflicts.

• Gary
  • Blog post on GSoC
  • Cuda optimization person email
  • nGraph vs Graph API in OpenCV

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• New vision papers
• GSoC

• OpenCV Blog
  • Under the hood improvements cited in blog
  • DNN update blog
  • Graph api part of OpenCV
  • Blog posts now have commenting added!
  • Vision paper authors
    • Add to blog to keep vision news
    • Once a week newsfeed?
• Best of classical vision
  • That beats deepnets in speed and/or accuracy (SIFT)
  • Color calibration -- Macbeth detectors
  • ASIFT
    • Color sift
  • Susan algorithm for edge detection Finds good features in the presence of noise

Hackathon

1. Core team continue work on merge of hackathon PRs the next week. Remaining PRs contain a lot of changes.
2. Hackathon label will be there and the team will add it to tasks for the next Hackathon.
3. AI to @Alexander Smorkalov write article to OpenCV blog on hackathon results.AI to @Alexander Smorkalov collect information about major hackathon contributors.

PRs discussion

• Support audio module #16578. Audio support is an experimental feature and the team decided to move development to opencv_contrib. The module support motivation -- simple wav files i/o and basic processing to use with DNN module.
• Expose maxIters in findFundamentalMat #16498 -- useful change. Some API adjustment is required. AI to Alexander & Vadim review and comment.
• build: OpenCV includes #12481. -- To be closed.
• Allow access to CUDA pointers for interoperability with other libraries #16513 -- .data field of GpuMat should be exposed to Python bindings as read-only property. GPU memory ownership should be documented.

Algorithm Quality Benchmark

• Vadim Pisarevsky works on concept of algorithms quality benchmarking. Major targets: non DNN algos like RANSAC, Hough Transform, pattern detection (chessboard, checkerboard), QR code detection, etc.

Discussed ideas:

• Several images are included into opencv_extra and used as data for unit testing. Major dataset is not included.
• OpenCV provides some tools & scripting to run benchmark and compute a score. Well known datasets & benchmarks can be used.
• The benchmark or subset can be added to CI as an optional check,
• The initial work can be done in the scope of GSoC, internship or academic project.
• There is a project with relevant scope. https://www.sotabench.com/ runs benchmark for DL models and publish it. https://github.com/catalyst-team/catalyst/ runs testing without GPU in an efficient way.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Considering the Linux foundation ... and/or the Linux AI foundation
  • Meeting later to(my)day. Want to generate list of questions

Questions I have, and we should add to

Background: OpenCV's goal is to drive/accelerate the beneficial uses of computer vision in society.That's the core mission that I want to freedom to achieve. It goes beyond just a codebase, though it is that too. 

• I need a much better understanding of how the funding works for a project under the Linux foundation vs under the Linux AI foundation?

  • In particular, I want funds raised for OpenCV to go to OpenCV and not into a "pool" from which I must then request/justify/argue for allocations. Is this OK?
  • Is that best done under LF or LFAI? Or help me understand better how this works with the understanding that I do not want a gated, or two step allocation process.

• There is an entanglement that needs to be untangled: 

  • I formed the Open Source Vision Foundation OSVF which is a 501c3 umbrella for (filed fictitious names):
    • CARLA -- an autonomous driving simulator
    • Open3D -- 3D point cloud processing (such as comes from LIDAR or other depth sensors)
    • OpenCV -- my computer vision
    • Kornia -- differential computer vision. Possibly this might combine in some way with OpenCV TBD
    • OpenSAM -- wants to join, it does SLAM -- creating 3D maps from sensors and using such maps to find and then track where you are to high precision.
  • These other teams can remain (if they desire) under OSVF, or they can be folded into the LF or LFAI under whatever terms each sub-group can agree on. They each have independent technical teams and it would be up to them ... I didn't choose these randomly, they are all top tier quality.

• Subject to all the above, I really want to understand how OpenCV under LF.org would work. 

  • We raise $N. What part does the LF take or LFAI take? 
  • What resources are at OpenCV's disposal 
    • Legal questions? 
    • Accountants? 
    • PR?
    • ...
  • Many big company names give $s to the LF or LFAI it seems. Does that compete with say, getting Microsoft to donate directly to OpenCV? Or is that some pool which OpenCV could access? How does this work?

• Legal attacks:

  • Let's say someone asserts a trademark, or a patent, or some lawsuit against OpenCV. Does the LF take up the fight? 
  • Let's say under whatever theory, this someone goes after me personally as a founder/director/whatever of OpenCV.
    • My problem?
    • Or LF (a) defends me? (b) indemnifies me?
  • Compliance with or dealing with possible US gov interference in the project, for example in the future imposing AI code restrictions etc.

• I have a lot of questions about my freedom of action under LF as to running OpenCV

  • I'm paying a developer and part of a developer's manager's time to a consulting group. That consulting group offers the developer's and manager's time at cost. My first priority is expanding this group to a minimum of 3 developers, but really targetting having many more: 10 or even more depending on funding.
    • These contractors are provided at cost to OpenCV (substantially below market).
  • Similar for interns. Every year, OpenCV enters Google's Summer of Code ... I want to run OpenCV's Summer of Code.
  • I'd like the option to directly support research. That is, I have many academic contacts and want the freedom of action, should a topic/research and willing professor be found that I could support that student to fill some gap resulting in a contribution of that algorithm/code into OpenCV. 
  • When financially able, I'd like to be able to afford a full-time admin that would take care of the million things that barely get done now or dropped by me to free me up to focus on driving the library's direction.
  • Right now, I run this out of my house, borrowed conference rooms and coffee shops. If finances allow, I want to have an office (the official address is really my accountant's office right now).
  • I and others will attend academic conferences, propose workshops, run tutorials, staff booths etc at existing conferences and tradeshows. This may induce travel, room, conference and other expenses.
  • We may sponsor contests, hackathons, host datasets, collect or create datasets for these functions or to advance the field.
  • I'm fine with attending, participating, giving tutorials etc in LF conferences or events, but I want the freedom, if we choose, to host our own conference, workshop or events.
  • I want to be able to keep mailing lists of people who sign up.
  • I'd like to create a "Open and Free Contributory license". That is, for people using the library for commercial purposes, this would be a voluntary nominal annual payment (for example, $20 or whatever amount desired) per head. This would clearly state this is not required and in no way enforced but is merely a voluntary contribution to give back to the library.
  • Any problem with any of the above?

• Website

  • Without a lot of flack or delay, we want to control OpenCV's website (wherever it clicks through from the LF site) and of course, github repository + wiki. Put up a blog, A/B test donation buttons, host or point to content in accordance with OpenCV's mission. If a member of LF, I don't mind it being within a LF frame/logo etc. Problems? Procedures?

• Revenue w/in mission

  • Books, whether written by OpenCV or not, directly sold or click through. These would be certified -- that is, we think they are of sufficient quality in important areas for people's understanding of computer vision or issues related to it.
  • Camera, Smart Camera, chips/board HW. Similar to the above. We'd certify hardware of high quality and give an honest assessment of important issues (outdoor/indoor/waterproof, frame rate, power, accuracy etc) and in turn, we'd get a membership fee and click through share of sales.
  • We are toying with offering Kickstarters for features. Basically, users vote with their money for features they want to be developed. If the money raised is sufficient to develop that feature, we do it, else the money is returned. 
  • Courseware. We already have professional courseware that contributes 20% to the library. The courseware is of very high quality and so OpenCV links to it in return for 20%

• Foreign operation

  • Especially China? Developers there, how to work with?

• Contacts

  • I want to talk to some existing projects.

• OpenCV Brand management inside LF or LFAI?

• General specific donation? For example, can we put up a donate button that goes to OpenCV specifically?

• If we are funded through the LF or LFAI

• If things don't work out, extracting OpenCV from LF or LFAI: Can we and how do we do that (decision process)?

  • If we want to hedge, can we keep a foundation "home" externally but mainly run internally with the "home" just a "escape path" for the above.

• Trademark? We don't have? Can we or should we attempt to get? What if someone tries to "trump" our trademark?

  • Defense against transfer and then attacks that we don't want.

• OpenCV structure is complex

  • LearnOpenCV
  • Intel team ... OpenVino team
  • China team OpenCV.cn
  • Can other sites live separately?

• Other vision projects ... cannibalization of code or competition by other ops w/in LF or LFAI?

  • Where is the place where we provide technical decisions of what is in or out of the code
  • Only pull requests ... if contribution, what control over that? Intel uses graph API for example, then pulls or changes it w/o any control/consultation?
  • Where do we decide roadmap? OpenCL

• Roadmap

  • OpenCV Evolution Proposals
  • Develop evolution proposals
    • We can devote occasional meetings to this

• I'm forming a contracting consulting company, OpenCV.ai, in AI broadly and Computer Vision in particular whose moral intent it is to contribute some % of profits to OpenCV.org. It could be made a nominal subsidiary or partner of OpenCV.org if any of the above must be instead hosted on a for-profit site.

• Hackathon:

1. Lydia joined Xperience AI team in role of program manager. Welcome Lydia!
2. Hackathon is in progress. The most important contribution is OpenJPEG library integration for JPEG200 support (#16494, #16524). Core team will work on PR review and merge for the next week. Will discuss final status next time.
3. PR & Tickets handling process: PR template implemented and merged. The next steps are:
   • answers.opencv.org resurrection. Stackoverflow is possible alternative. The major goal is to attract community and resolve basic questions there instead of issue tracker. Xperience AI team will try to restart community there.
   • "Good First Issue" label for easy to implement tickets to attract students and newcomers.
4. @Vadim Pisarevsky found good dataset for HoughCircles algorithm testing and rewrote some pieces. It's a good idea to assess some functions and algorithms with public datasets and publish/track score. Possible subjects are: QR codes, chess boards and other calibration patterns, RANSAC. To be discussed on the next meetings in details. AI to @Alexander Smorkalov and @Vadim Pisarevsky prepare an agenda for the next meeting.
5. PRs discussion:
   • core: CV_STRONG_ALIGNMENT macro #16463 - the solution requires extra job on CI. AI to @Alekhin, Alexander add extra job to CI and merge the patch.
   • cmake: split opencv_modules.hpp #12549 - to be merged.
   • Fix bugs in arithm_op() for InputArray (src == dst) case #13570 - to be closed. The patch tries to add fake inplace processing support to functions that are not designed for it. Extra memory allocation and copies are not reasonable for most cases. AI to @Vadim Pisarevsky to close the PR with justification.

• Name
  • todo

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Linux foundation
• Dev Group CN
• GSoC

• Linux foundation
  • Need more information
• Dev Group CN
  • OpenCV on edge
    • Edge
    • Risk-V
  • Offline courses emphasis on edge
  • Online courses emphasis on edge
• Hackathon
  • Developers
  • New people
  • Vadim
  • Organize another hackathon
  • Create schwag for it (send)
• Schwag
  • Revamp/Renew
    • Coats, T-shirts, stickers
  • Store
• Graph API
  • Blog or professional
    • Blog
• Pull requests
  • Down to 56
    • basically, for the team, this seems to be the equilibrium, as many closed as opened
  • New Program Manager,
    • Lidia taking care of priorities
• Need to moderate comments on blog
  • Use discuss as a blog platform
  • Medium or Github.io Seth typepad

• Name
  • todo

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• GSoC
• Linux Foundation
• Update

1. OpenCV team submitted all required papers for Google Summer of Code.
2. Hackathon announcement is published, the team is ready. Alexander Alekhin will work on PRs review. AI to @Vadim Pisarevsky to add QA part to Google Group.
3. PR template from Alexander looks good enough. There are some comments in PR. AI to @Vadim Pisarevsky: propose legal wording about license. AI to @Alexander Smorkalov to move all links, details and recommendations to the "How To Contribute" page and add single link to PR template.
4. Initial work on profiles for cv::VideoWriter support is done. Prepared codec parameters, implemented script for their optimization. Implementation will be submitted right after refactoring.
5. PRs & tickets discussion:
   • Enable 2-channel arrays in resize tests (PR #16189) -- the tests open several bugs in current implementation. AI for @Alekhin, Alexander list all remaining items in PR to do it during the hackathon.
   • Python module: replace config.py files by config.ini (PR #16008). Config.py contains logic to load proper modules for Intel Python, Conda, etc and it cannot be replaced by a plain config file. The expected behavior can be achieved with CMake options. The PR is closed.
   • add REDUCE_SUM2 (PR #13879). Proposed for the Hackathon. Remaining items: disable sanity checks for new test cases and squash commits for merge.
   • VideoIO classes refactoring (PR #16242). Vadim will finish until the next meeting.
   • Fix issue #6450 (ios) (PR #Fix issue #6450 (ios) (PR #15464)15464). The patch is useful AI to @Vadim Pisarevsky to fix warning and merge the patch during the hackathon.
   • Issue #13602 GaussianBlur does not support border type wrap. WRAP mode implementation does not make sense, documentation update only. Added to hackathon scope.
   • Issue #15499 CAP_PROP_FRAME_WIDTH/HEIGHT swapped. The actual issue reason is metadata in videos. Renamed the ticket to describe the issue properly and added the next steps there. Added to the Hackathon scope.
   • Issue #9392 Reference to non-existent parameter name in the documentation of StereoSGBM::create (Still needs the decision). Renaming and documentation update is required. Alexander listed all remaining items on the ticket.

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Browser
• GSoC
• Updates

• Browser -- the below can help make vision integral to browsers that we may engage with

  • Talk with Mozilla about this (GSoC project? Probably too involved or complex)
    • WebGPU (working group inside W3C),
    • WebAssembly (expand WebAssembly outside of the browser, Byte code alliance, WASI standard -- extensions)
    • WebSIMD (in progress, efforts happening, but few people. Extraction layer over SIMD, threading, long vector instruction)
    • Compiler for optimization/vision OpenNP, LLDM (GSoC) ... but looks maybe too big for a student

• GSoC

  • Application is already complete
  • Ideas list
    • Now 15 ideas, the application is complete
    • Adding Tensor flow connection idea
    • OpenNP lldm ... use compiler to optimize vision in browser
      • But probably too complex for a student project
  • We have strong people for overhauling RANSAC. OpenCV foundation may extend student support for this beyond GSoC

• China team

  • Paused by Chinese new year and Coronavirus
  • Collaborating with some startup in China to accelerate OpenCV DNN for ARM. Very promising results. The patch will hopefully be ready and integrated by OpenCV 5 or even earlier.
  • Looking for extra engineers ... again delayed
  • HW for build factory ... also delayed
  • Shiqi ... discussing adapting courseware to Chinese. Not delayed since can be done at home

• Kickstarter for features or toolboxes

  • We should come up with a list we can try this

• Hackathon

  • Feb 2nd
  • Set up a "thankyou" page, and post hackathon thanks
  • Can send now to mailing list and blog list

• Update

  • RISC-V close to RISC pipe committee (they may be the mentors for the idea)
  • Down to 62 pull requests, now more new than old pull requests
    • Probably need to track the average age of pull requests soon
  • Make sure every pull requests is attended to
    • Make a formal process for the response -- what's wrong, what to do if abandoned by the contributor
    • Looking into other projects, how they do it: finding -- no formal processes. Linux is just by fiat by the main guy
      • We want aliveness response
      • Check that it's 3.4 or later
  • What should be done for a pull requests
  • Preparing for hackathon, can provide list

1. Vadim Pisarevsky could not join today. The first and second agenda items were skipped.
2. Hackathon announcement has been published on opencv.org. Thanks, Satya for post promotion.
3. PR handling process was discussed with Alexander Alekhin. Major items:
   • Stale bot produces to many spam e-mails for maintainer, very detailed configuration is required. The idea is postponed.
   • 2 weeks is too short to wait for PR author, e.g. maintainer or author vacations.
   • We can add a checklist and links to test examples directly to the PR template. It's a useful solution and works as a reminder. AI to @Alexander Smorkalov proposes an alternative PR template.
   • Proposed new tag "Discussion Required" for PRs. All PRs with the tag are discussed by the core team before actual review and integration. The tag is designed for PRs with suspicious changes, PRs with small value and changes that breaks existing logic or API. AI to @Alexander Smorkalov introduce the tag on Github and add notes to the Contribution guide.
4. PRs discussion status:
   • optimize cvCeil and cvFloor in fase_math.hpp #16160 -- closed as incomplete; no author replies on questions.
   • core: Workaround flip horiz #16152 -- the PR highlights memory alignment issue on ARM v7 for vectorized code. The PR cannot be merged as-is. AI to @Alekhin, Alexander to file a ticket with a detailed description, propose an alternative solution and then close the original PR.
   • Add package for CMake project #16322 -- The patch rejected as we do not maintain it and most of package management systems store metafiles and extra patches on their side.
   • Fix bugs in arithm_op() for InputArray (src == dst) case. #13570 -- To be discussed with Vadim Pisarevsky.
   • export IMG decoders/encoders #8511 -- To be discussed with Vadim Pisarevsky.

• Gary
  • Set up opencv-gsoc 2020 mailing list
  • Ask lawyers about export licese
• Satya
  • Send site editing access to Gary
  • Contact Microsoft about supporting better OpenCV experience on Microsoft
• Vladamir
  • Post Adam tracker to Ideas list
  • Post audio support to Ideas list

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Hackathon
• GSoC
• Ransac

• Hackathon

  • [email protected]
  • Finalizing the text right now, can publish it now
  • Will be posted to site tomorrow (blog & notification)
  • Banner will also reflect this
  • Hackathon announcement on Google drive: https://docs.google.com/document/d/1lOp1HbEcCQFkO1ydGYI_k7UwHE_0sVRmZTXY2vmVycc/edit?usp=drivesdk

• Website

  • Change order of blog vs buttons
  • Look at font or attention interest site

• GSoC

  • Now officially applied!
  • Feature
    • ASIFT good for perspective distortion
    • Data augmentation module from original GSoC
      • Accelerate taking data set and augment dataset good one to add
  • Calibration
    • Color/Macbeth
    • Lighting with spheres in scene
  • Installing OpenCV Contrib with Windows (exec)
    • Adding NuGet
    • C++ interface
    • Video capture
    • Better tools for Windows environment
  • Integration of Adam tracker in GSoC to enable
  • Audio I/O support
    • Should DNN be extended to text and speech?

• Got down to 60 pull requests

  • Created a metric for response time from pull or bug
    • Finite time for author to reply, or else we can fix or cancel
      • Discussing
  • Have already manually changed some older pull requests to catch up

• China team

  • User from China: What is OpenCV regulation terms (export license) EAR99 label(?)
  • Can we officially state this? Might want to do as Apache does ECCN 5D002 exported

• Updates

1. GSoC 2020 status.
2. Hackathon scope.
3. PR & Tickets process proposal (WIP). Technical solutions and process ideas:
   • Use Stale bot for notifications: https://github.com/apps/stale
   • Add item to contribution guide that OpenCV team can close PR, if it's not complete and abandoned more than 2 weeks.
   • Add item to contribution guide that PRs with RFC label are discussed by core team and can be rejected.
4. PRs for discussion:
   • optimize cvCeil and cvFloor in fase_math.hpp #16160
   • core: Workaround flip horiz #16152
   • Add package for cmake project #16322
   • Fix bugs in arithm_op() for InputArray (src == dst) case. #13570
   • export img decoders/encoders #8511

• Gary
  • Set up opencv-gsoc 2020 mailing list
  • Ask lawyers about export licese
• Satya
  • Send site editing access to Gary
  • Contact Microsoft about supporting better OpenCV experience on Microsoft
• Vladamir
  • Post Adam tracker to Ideas list
  • Post audio support to Ideas list

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

[email protected]

• GSoC

• DNN has Cuda support now
  • Davis King mentored
  • Mentor or student
  • Put in whatever is necessary
  • Cuda backend (has more stuff/layers has more than C++)
    • Add more layers to C++ (Expose C++ from the DNN project)
• Add to model zoo (import new model)
  • How much work is very difficult to estimate
  • Important topology
  • Help to propose what and how
  • DNN importer doesn't give good failure messages for import problems
• ONNX support (importer) parameters and layer types
• Calibration
• New classic computer vision algorithms
• Experience
  • For now, keeping pull requests current
  • Want to hire more staff for addressing tools
• Virtual Hackathon February 3rd or 10th, 2020
  • ONNX versioning
  • Import failure -- know best list (analysis tool)
  • Spotty ONNX version coverage
  • Welcome to join -- one week do good work for OpenCV
    • Ideas
• China development team
  • 4 people assistant, Vadim, 2 people and 2 people coming ... like 5 engineers
    • Conversion to Chinese
    • OpenCV on edge
    • Github extension to another platform China to China
    • Leverage optimization for Arm (maybe 10 engineers)
• Linux foundation: Better camera support (they don't use video for linux)
  • Linux AI project
  • Intel effort for ngraph runtime ... will be presenting today about intent
  • Linux foundation AI for ONNX
    • ML layer mailing list (independent from Google)
    • ARM layer
• Can OpenCV sponsor/be reproducibility track?
  • Reproducible
• Getting pull requests down
  • Clearing out the older one
  • Formal process for pull requests for (2 week reply time) to approve this Friday
    • Other big library policies on this

• Gary
  • Apply for GSoC OpenCV
  • Finish ideas page
  • Connect with Linux foundation
• Anna
  • Communicate Hackathon to Satya/Gary for publication on site

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

My notes from the meeting today:

1. The team agreed WW06 (Feb 2-9) as a preliminary timeframe for OpenCV Hackathon. @A lexander S morkalov and @V adim L evin will create short list of tickets for the hackathon for the next meeting.
2. V adim L evin has done serious cv::VideoCapture refactoring that breaks compatibility. AI to @A lekhin, A lexander and @V adim P isarevsky to review PR to discuss the solution on the next meeting.
3. Most of fixes from V adim L evin for Python bindings have been merged. @V adim L evin will split remaining PR on 2: existing bug fixes and new functionality. The second one will be discussed later.
4. Abandoned PRs:

• BFGS and L-BFGS optimization methods: AI to @A lexander S morkalov to close. The implementation is buggy.
• Clang format: AI to @A lekhin, A lexander to close the PR with comments. The formatter introduces a lot of changes in code and some of them are not well readable.
• Objdetect module const correctness: AI to @A lekhin, Alexander to close the PR and related ticket with resolution "will not fix". Some object detectors are not constant by design and not thread safe. We can force detectors interface and overcome mutability problem with thread local variables and other C++ hacks, but it introduces non-obvious behavior.

5. OpenCV 5.0 is planned for release in June-July time frame. The release is aligned to 20th anniversary of the project.
6. A lexander S morkalov started document to collect common replies to PRs: AI to @A lekhin, A lexander to join and extend the document.

• Name
  • todo

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• Meeting times
• GSoC
• Data augmentation

• Might have to change the meeting time
  • Maybe 11pm California time is the only time that really works out around the world.
• GSoC Ideas
  • Fixing bindings everywhere
    • Issues with binaries, C++11 is not supported in bindings always
    • Want this fixed
  • Ransac improvements
    • Paper
    • PyRansac
  • Fiducial Patterns: robust
  • Calibration improvements (utilities)
  • ONNX, some version
    • Whatever ONNX covers should be runnable in DNN
      • Intel not whole standard, but various architectures (OpenVino)
        • One Resnet 50, UNet specific
  • Data augmentation or differential rendering
• Differentiable data augmentation
• Vacations
  • Anna out 1st to 8th
  • Gary out next 2 weeks
  • => Next meeting will be in Jan 2020

• Gary
  • Meet with AWS, see if we can get a long term deal
  • Get Chase foundation set up
  • Linux foundation AI site
  • Toyota

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

• HW
• Verticals
• Money transfer in the future
• Linux

• HW
  • http://jevois.org/
    • http://ilab.usc.edu/
    • Laurent Itti - [email protected]
    • http://ilab.usc.edu/itti/
  • http://www.raysharp.cn/en/
  • https://luxonis.com/
  • https://www.intelrealsense.com/?cid=sem&source=sa360&campid=2019_q2_egi_us_ntgrs_nach_revs_text-link_brand_exact_cd_realsense-realsense_o-1lngr_google&ad_group=realsense%5Eus%5Erealsense%5Eexact&intel_term=realsense&sa360id=43700043662097307&gclid=CjwKCAiAxMLvBRBNEiwAKhr-nJUI-zj98oLLk8xLaWa0KRjz9cOea_4-uukEw_Yq3KRsljIW_sOV3xoCZxwQAvD_BwE&gclsrc=aw.ds
  • ARCore https://developers.googleblog.com/2019/12/blending-realities-with-arcore-depth-api.html?m=1
  • https://developers.googleblog.com/2019/12/blending-realities-with-arcore-depth-api.html
• Verticals -- collect specialized datasets
  • Security
  • Automotive objects
  • Manufacturing defects
  • Agricultural areas
• Foreign payment
  • Going to switch banks
  • https://transferwise.com/us
  • https://www.xoom.com/
• Linux foundation
  • https://www.linkedin.com/in/ibrahimhaddad/
  • https://www.linkedin.com/company/the-linux-foundation/

• Gary
  • Meet with AWS, see if we can get a long term deal
  • Get Chase foundation set up
  • Linux foundation AI site

---

Meeting_notes