Computer Vision: OpenCV - SCHS-Robotics/Software-Wiki GitHub Wiki

One option for OpenCV is to use FTCVision. The only problem is that it has not been updated in a while, and we here at SCHS do not have any experience using it.

You may follow the directions below to use the OpenCV library by itself:

Go to opencv.org, click on Releases, and click on Android Pack. Download.
Open the Android SDK Manager in Android Studio (Preferences > Appearance & Behavior > System Settings > Android SDK). Click on SDK Tools and download the NDK.
Add the OpenCV library to the FTC Robot Controller using the steps below. The following is derived from this answer on Stack Overflow.

a. Import OpenCV to Android Studio, from File -> New -> Import Module, choose OpenCV-android-sdk/sdk/java folder in the unzipped opencv archive.

b. Update build.gradle (Module: openCVLibrary) to update 4 fields to match your project build.gradle (Module: FtcRobotController). The fields that need updating are compileSdkVersion, buildToolsVersion, minSdkVersion, and targetSdkVersion. buildToolsVersion, however, seems to not be included in the most recent updates to the ftc app. Also, making compileSdkVersion, buildToolsVersion, and minSdkVersion too recent triggers android's security precautions and prevents the app from being uploaded, so as a precaution set compileSdkVersion to 23 and the other two settings to 19. Ignore any red underlines you get; they are warnings, not errors.

c. Right click on FtcRobotController > Open Module Settings, and select the Dependencies tab. Click + icon at bottom, choose Module Dependency and select the imported OpenCV module. Do the same for TeamCode. Due to a recent FTC app update, the computer incorrectly adds the dependency for the teamcode module. To fix this, go into TeamCode's build.gradle file and cut (not copy)
```
dependencies {
    implementation project(path: ':openCVLibrary341dev')
}
```
and paste it right below the line apply from: '../build.common.gradle'

d. In your computer's file browser, copy the libs folder under OpenCV-android-sdk/sdk/native. Above Android Studio's file browser, click the arrow next to "Android" and select "Project". Paste the libs folder under FtcRobotController/src/main. Rename the libs folder to "jniLibs". Do the same for TeamCode/src/main.

e. Go back to the "Android" view by clicking on the arrow next to "Project".
Go to FtcRobotController src > res > layout > activity_ftc_controller.xml. This file codes the UI elements for the main part of the app. It might be useful to learn some things about Android XML files first. The following steps will make the camera view cover the entire screen when an OpMode is running and using OpenCV, but you are welcome to find a different way to add the camera view to the layout.

-. Select "Text" at the bottom of the opened window, as opposed to design.

a. After the line that says xmlns:tools, add the following line:
```
xmlns:opencv="http://schemas.android.com/apk/res-auto"
```
b. Change the LinearLayout at the top to a FrameLayout

c. Define a new LinearLayout below the first chunk of code:
```
<LinearLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:focusable="true"
    android:orientation="vertical">
```
d. Cut (actually cut, don't just copy) android:id="@+id/entire_screen" from the outer layout and paste in the new LinearLayout.

e. Scroll to the bottom. Just before the last line (which should be </FrameLayout>), add </LinearLayout>.

f. In a new line after the </LinearLayout> and before the </FrameLayout> make an OpenCV JavaCameraView:
```
<org.opencv.android.JavaCameraView
    android:id="@+id/cameraView"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:visibility="gone"
    opencv:camera_id="any"
    opencv:show_fps="true" />
```
Edit FtcRobotControllerActivity (FtcRobotController > src > main > java > org.firstinspires.ftc.robotcontroller > internal)
OpenCV has a weird quirk that messes everything up: It only works correctly in landscape orientation. Therefore it's best if you force the screen in landscape mode. Find the onCreate() method in FtcRobotControllerActivity and add the following line somewhere in that method:
```
setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
```

Time to set up random OpenCV stuff.

a. Add the following imports at the top of the file (with all the other imports):

import org.opencv.android.BaseLoaderCallback;
import org.opencv.android.JavaCameraView;
import org.opencv.android.LoaderCallbackInterface;
import org.opencv.android.OpenCVLoader;

b. Make a variable for the camera view near the top of the class:

public static JavaCameraView mOpenCvCameraView;

c. Add the following code somewhere in the class:

private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
    @Override
    public void onManagerConnected(int status) {
        switch (status) {
            case LoaderCallbackInterface.SUCCESS: {
                Log.i(TAG, "OpenCV loaded successfully");
                mOpenCvCameraView.enableView();
            }
            break;
            default: {
                super.onManagerConnected(status);
            }
            break;
        }
    }
};

d. In the onCreate() method, add the following:

mOpenCvCameraView = (JavaCameraView) findViewById(R.id.cameraView);

This must come after setContentView(R.layout.activity_ftc_controller);

e. Add the following code to the end of the onResume() method (Change OpenCV version to closest one to your version):

if (!OpenCVLoader.initDebug()) {
    Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
    OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_2_0, this, mLoaderCallback);
} else {
    Log.d(TAG, "OpenCV library found inside package. Using it!");
    mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
}

f. Add the following code to the end of the onPause() method as well as the onDestroy() method:

if (mOpenCvCameraView != null)
    mOpenCvCameraView.disableView();

Add the following code to FtcRobotControllerActivity (FtcRobotController > src > main > java > org.firstinspires.ftc.robotcontroller > internal), which will allow for displaying and hiding the camera view from an OpMode:

public final static Handler turnOnCameraView = new Handler() {
    @Override
    public void handleMessage(Message msg) {
        if (mOpenCvCameraView.isEnabled())
            mOpenCvCameraView.disableView();
        mOpenCvCameraView.setVisibility(View.VISIBLE);
        mOpenCvCameraView.setCvCameraViewListener((CameraBridgeViewBase.CvCameraViewListener2) msg.obj);
        mOpenCvCameraView.enableView();
    }
};

public final static Handler turnOffCameraView = new Handler() {
    @Override
    public void handleMessage(Message msg) {
        mOpenCvCameraView.setVisibility(View.GONE);
        mOpenCvCameraView.disableView();
    }
};

It's finally OpMode time! The following two methods will be used whenever you use OpenCV in an OpMode, so put them somewhere accessible, preferably in BaseOpMode, if you are familiar with EONS TO COME (Organizing Code):

public void startOpenCV(CameraBridgeViewBase.CvCameraViewListener2 cameraViewListener) {
    FtcRobotControllerActivity.turnOnCameraView.obtainMessage(1, cameraViewListener).sendToTarget();
}

public void stopOpenCV() {
    FtcRobotControllerActivity.turnOffCameraView.obtainMessage().sendToTarget();
}

Here is an example OpMode that uses OpenCV, but does nothing with it:

@TeleOp(name="Some Linear OpMode", group="OpMode")
public class SomeOpMode extends LinearOpMode implements CameraBridgeViewBase.CvCameraViewListener2{

    @Override
    public void runOpMode() {
        // Initialize hardware code goes here

        waitForStart();

        startOpenCV(this);

        while (opModeIsActive()) {
            // Do stuff
        }

        stopOpenCV();
    }

    @Override
    public void onCameraViewStarted(int width, int height) {

    }

    @Override
    public void onCameraViewStopped() {

    }

    @Override
    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
        // This is where the magic will happen. inputFrame has all the data for each camera frame.
        return inputFrame.rgba();
    }
}

Go back to the "OpenCV-android-sdk" folder you downloaded and look at the samples. Depending on your needs, it might be a good idea to build off of the "color-blob-detection" program.

Extra Tips

Check out this improved ColorBlobDetector class. The sole improvement is that it is able to look for the color red while allowing for both pinkish and orangish hues. This isn't possible in the one provided by OpenCV because the color spectrum for HSV starts and ends at red, so only one side of the spectrum could be used at a time. The improved version is able to treat the color wheel as an actual wheel, without sacrificing performance.

This documentation for OpenCV details what you can do with contours, which are the type of object you get from ColorBlobDetector. The documentation is in python but just replace cv2 in their code with Imgproc and you are mostly good to go.