PhotonVision - quasics/quasics-frc-sw-2015 GitHub Wiki

PhotonVision setup

Installing PhotonVision on a "normal" RasPi

The basic installation process for the a "stock" image running on a Raspberry Pi can be found on the PhotonVision web site.

Configuring the RasPi for read-only booting

The PhotonVision team provides an image that is configured to boot as a "read/writable" image. This is great for initial set-up/configuration of the pipeline, but it risks corrupting the filesystem if power is unexpectedly cut to the Raspberry Pi. In our case, this means whenever the robot gets turned off (so... a lot of risk over time).

The solution is to update the OS configuration (preferably after the camera/pipeline is fully in place) so that it will default to booting in a way that the filesystem is treated as read-only data.

There are directions for doing this here, along with information on how to temporarily make the image "writeable" if changes are required (e.g., new versions of the PhotonVision software, updating the camera configuration, etc.).

Step By Step Process:

Step 1: Download current version of photon vision from here - select the most recent revision -> assets -> "photonvision-vXXXX.XX.X-linuxarm64_RaspberryPi.img.xz

Step 2: With the SD card in the computer, use something like Balena Etcher to flash the SD card from the raspberry Pi with the current image.

Step 3: Put the MicroSD card back into the raspberry pi, then configure the pipelines using the web interface. (This should be available here when you're on the same network as the RasPi, so make sure you're connected to the robot and the raspberry pi is also connected before opening this link.) This will let you get the camera set up, resolution, apriltag family, etc. This can be changed later, if needed, but it's easier to do before putting the Pi into read-only mode by default.

Step 4: Disconnect the raspberry pi from the robot's networ, and connect it to a monitor with a mouse, keyboard, ethernet, HDMI, and power.

Step 5: Follow the website guide to configure it for read-only mode, as well as the optional "ro/rw" switcher thing.

  • Note, use 'sudo nano [insert file name here]' to open and edit a file. Make the changes, then hit ctrl+o, enter, then ctrl+x to save then exit.
  • To log onto the Pi, use the username "pi" and the password "raspberry".

Step 6: Once everything is working, MAKE A CLONE AND SAVE IT. This is a tedious process, so saving yourself the pain of redoing all that makes reimaging it much easier.(You will need another img file to clone the current file, you can download a new raspberry pi image from the GitHub and then overwrite it in said cloning software)

Step 7: If you needed to adjust the pipeline after setting up read-only mode, connect to the raspberry pi through the robot, then log onto it using PuTTY to set the pi to read-write mode before making your changes then returning it to read-only mode. If the changes are major, consider cloning the sd card again just to be safe.

Notes:

  • The process of configuring the Pi to boot in read-only mode is somewhat complex (lots of steps). You'll need to have the Pi plugged into a network with Internet access, and it's easiest if you can have both a keyboard/monitor connected to it (for console access) and can connect to it via SSH (e.g., using "Putty" to connect to "photonvision.local" as user "pi", password "raspberry"). The easiest way to do this is to go to one of the monitors, unplug the mouse, keyboard, hdmi, and ethernet cable and plug them all into the raspberry pi. Then plug the power in (micro usb or usb c)
  • Because this process is complex, it's a very good idea to make a backup image of the RasPi's SD card before you get started (so, capturing any initial PhotonVision pipelines, etc.), and after you've got everything working (so that you can quickly deploy a replacement image if things happen like your RasPi's SD card getting physically damaged; "ask us how we know!").
  • When asked to edit a file, using "sudo nano [insert file locations]" and make the required edits
  • Corrections to the tutorial (Check here if anything is spitting back errors):
    • When creating the read-only and read-write swap command, remove the " ; sudo mount -o remount,ro /boot" part from the parts that say alias and replace it with " ; sudo systemctl daemon-reload"
    • When making the bash.bash_logout file, make sure to add "sudo" before the commands, additionally, omit the "mount -o remount,ro /boot" and add the command "sudo systemctl daemon-reload"

Configuring the BeeLink for Smart Dashboard configuration

  • Once the Bee Link is configured (TODO: How was this done?), connect it to the roboRio. Connect to http://photonvision.local:5800/#/dashboard. Note that if you put in https://, you will likely get a security warning. This is because the photon vision setup does not provide secure HTTP, so the browser tries to protect you from treating it like a secure website. (By putting http:// we are telling the browser that we do not expect the website to be secure/encrypted).
  • Go to the "Camera Matching" tab to activate the connected cameras.
  • Go to the "Settings" tab to import old settings (These are checked into Git for 2025 here.), or export current settings (Please check them into git!)
  • If it is not connecting to the smart dashboard, check that your "Team Number/Network Tables address" is set to 2656. Screenshot

Additional resources

  • How to backup/restore a RasPi SD card
    • This is helpful to be able to "snapshot" versions of the RasPi installation as you make progress. (For example, after initial setup of PhotonVision with an initial pipeline, or after configuring the Pi to automatically boot as read-only, etc.)

Setting up PhotonVision on a Mini PC

Additional software:

It was helpful to also install the following software packages (using sudo apt-get install <PACKAGE_NAME>):

  • net-tools
  • avahi-utils
  • avahi-daemon
  • overlayroot

It was also helpful to set up configurations for the avahi-daemom, which lets you "discover" the computer via Zero-conf/Bonjour, advertising the following services (and using "%h" for service names):

Getting rid of boot delays from network manager

As discussed in the set-up docs referenced above, installing PhotonVision will also add the "network manager" service to the machine, which will significantly slow down the boot process. The docs talk about how to disable this service, and this is also discussed in a number of places on the net, including this post on "AskUbuntu".

Configuring the Beelink for read-only booting

The docs above talk about how to use "overlayroot" to boot the box in read-only mode in order to keep the SSD from being corrupted in case of an "unexpected power failure", which is very important for us since power can fail at any time. (Specifically, when the robot is turned off.) However, the "how-to" is a little sketchy.

What Mr. Healy has done on our box for 2024 is outlined below. (Note: at each step along the way, the changes were tested by re-running sudo update-grub and then rebooting the box and performing the specified confirmation steps.)

  1. Update /etc/defaults/grub with the following settings:
# timeout styles: hidden (default on install), menu, countdown
GRUB_TIMEOUT_STYLE=menu
# timeout in seconds (default on install was 0)
GRUB_TIMEOUT=3
  • Confirmation steps: on reboot, the computer should display a menu of boot configurations and let you select which one to use; after 3 seconds, it should simply boot using the first configuration.
  1. Set up an additional option in GRUB2, allowing us to configure an alternate boot mode, using the approach discussed here.
    Changes to /etc/default/grub:
export GRUB_CMDLINE_LINUX_EXTRA=""

Changes to /etc/grub.d/10_linux:

# mhealy: Start of second (custom) option
linux_entry "${OS} (test)" "${version}" simple \
"${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT} ${GRUB_CMDLINE_LINUX_EXTRA}"
# mhealy: End of second (custom) option
  • Confirmation steps: on reboot, the computer should include a new boot configuration ("Ubuntu (test)") in the menu. Selecting this option should boot normally.
  1. Change the GRUB_CMDLINE_LINUX_EXTRA variable in /etc/default/grub to be: export GRUB_CMDLINE_LINUX_EXTRA="quiet splash"
  • Confirmation steps: on reboot, boot using the configuration you've added. The boot should run normally, but nearly all of the messages displayed during the boot should be hidden.
  1. Change the GRUB_CMDLINE_LINUX_EXTRA variable in /etc/default/grub to be: export GRUB_CMDLINE_LINUX_EXTRA="overlayroot=\"tmpfs:swap=1,recurse=0\""
  • Confirmation steps: on reboot, boot using the configuration you've added. Log on, and create a test file (can be empty) in your home directory. Reboot (again, using the new configuration), log on, and confirm that the test file isn't there.
  1. Make some final changes to the configuration files:
    Change the GRUB_DEFAULT variable in /etc/default/grub to be:
GRUB_DEFAULT=1   # Default to the read-only configuration (#2 in the list)

Change the line in /etc/default/grub that generates the new configuration to:

# mhealy: Start of second (custom) option
linux_entry "${OS} (read-only)" "${version}" simple \
"${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT} ${GRUB_CMDLINE_LINUX_EXTRA}"
# mhealy: End of second (custom) option
  • Confirmation steps: on reboot, the new configuration should now be named "Ubuntu (read-only)", and should be selected by default.

Notes:

  • We tried following some guidance on the GRUB2 settings, and making additional changes to /etc/default/grub that would allow us to remember the selected boot configuration, making it a little easier to flip back and forth during testing, etc. However (as might be expected), booting in read-only mode meant that the last boot mode wasn't saved.
  • Some other settings can be configured in GRUB2, as outlined here.

Additional information

  • Per Beelink's technical support, the barrel jack size for the power interface is 5.5mm x 2.5mm.
  • Configuring a Beelink EQ* for autoboot on power-on
  • If someone accidentally installs/enables a graphical login, this can significantly impact the machine's performance. Be sure to switch it back to a "multi-user" default (resource 1, resource 2)

Installing PhotonVision on an OrangePi

Basic outline

  1. Install the OrangePi PhotonVision image on an SD card, and boot the OrangePi.
  • Default account is user pi, password raspberry.
  • Note that the directions provided by PhotonVision indicate that additional steps are needed on an OrangePi 4 that aren't required for an OrangePi 5.
  1. Change default shell to Bash via chsh, and type /usr/bin/bash when prompted for the new shell.
  • This will change the default command interface to something that supports arrow keys for history/editing, along with other "niceties".
  • After changing the shell, you may want to log out and back in again (allowing it to take effect)
  1. Update the system:
  • sudo apt-get update
  • sudo apt-get upgrade
  • sudo apt-get dist-upgrade
  • sudo reboot
  1. Install additional packages: sudo apt-get install net-tools avahi-utils avahi-daemon overlayroot
  2. Set up advertising configurations for the avahi-daemom, which lets you "discover" the computer via Zero-conf/Bonjour, advertising the following services (and using "%h" for service names):

Scratch notes

  • Default Debian image password: orangepi
  • Directions to get a Debian image booting on the OrangePi are on the OrangePi wiki.

Installing PhotonVision on a Romi

Base directions are available on the PhotonVision site

Based on Mr. Healy's experiments with the process outlined above, there appear to be some additional "tweaks" that are needed on the Romi.

  • The Romi currently needs to be in "read/write" mode whenever PhotonVision is running (including when auto-started).

    • The PhotonVision docs suggest that this is only needed when trying to save logs and configurations; however, the program appears to fail to load the configuration or connect to the camera if the Romi is in read-only mode when you connect to the PhotonVision server via a web browser.
    • A bug has been filed about this with the PhotonVision team.
    • For now, a work-around appears to be to connect to the Romi web interface first, and switch the Romi to "writeable" before connecting to the PhotoVision web interface.
  • The memory being allocated for the JVM when running PhotonVision appears to be too low.

    • The file at /lib/systemd/system/photonvision.service should be updated to change the -Xmx512m parameter (.5GB) to -Xmx1024m (1GB). (If you're not familiar with editing files under Linux, please talk to Mr. Healy.)
  • The installer provided for "Other Debian-Based Co-Processor Installation" appears to install the generic JAR file (for use on Windows, Linux, etc.).

    • This is documented in an existing bug with the PhotoVision project.
    • There is a separate JAR file in the PhotonVision download set that is apparently specific to the Raspberry Pi; this file needs to be downloaded to the Pi, moved into the correct directory, and reconfigured for appropriate use.
    • For example:
% sudo wget "https://github.com/PhotonVision/photonvision/releases/download/v2023.1.1-beta-6/photonvision-v2023.1.1-beta-6-raspi.jar" -O /opt/photonvision/photonvision.jar
% sudo chown root.root /opt/photonvision/photonvision.jar
% sudo chmod a+x /opt/photonvision/photonvision.jar

Standard connection info

Additional resources

Setting up image streams on the driver's station

This should simply be a matter of:

  • Bringing up Shuffleboard on the driver's station
  • Pulling open the tab on the left-hand side of the window
  • Clicking on the "Camera Server" section
  • Dragging one of the camera streams onto a sufficiently large region of the main window

Worst-case scenario, you can also try opening a web browser and pointing it at either the raw image stream or the processed stream, and then adjusting windows on the laptop to suit the drive team's needs.

Trouble-shooting

  • If the Camera Server list is empty:
    • Make sure that the coprocessor running PhotonVision is actually running.
    • Make sure that PhotonVision is running on the coprocessor (e.g., try connecting to http://photonvision.local:5800, assuming that the coprocessor's hostname is "photonvision").
    • Make sure that the camera(s) are detected, and the image streams are active (e.g., by checking the main page for PhotonVision on the coprocessor, or browsing the streams directly).
    • Check the configuration settings in PhotonVision, and make sure that the team number is configured. (If it isn't, then PhotonVision won't "know" what robot to connect to in order to publish the information.)
⚠️ **GitHub.com Fallback** ⚠️