Before deploying Vitis-AI applications in the Ultra96-V2 board, one needs to configure a suitable Vitis Platform for the board. In this case, this Vitis-AI tutorial was followed in order to obtain the platform, as not only was it specific for our board, but it did also implement the hardware connection to the camera via MIPI. Although older versions of the IP-Cores and DPU kernel are used, this can be an initial version and can be updated later on.

The local machine used for this was Ubuntu 18.04.02.

• Prerequisites
• Building the Vitis Ultra96V2 and MIPI Platform

• Xilinx Design Tools (Vivado design suite and Vitis unified software platform), version 2019.2
• Board files for Ultra96-V2 installed
• PetaLinux SDK, version 2019.2
• XRT Xilinx Runtime
• Xilinx CSI2 RX MIPI IP License
• The Ultra96-V2 board
• Power supply for Ultra96-V2 board
• A blank, FAT32 formatted microSD card

• Download the Xilinx Unified Installer 2019.2: Linux Self Extracting Web Installer
• Sign in and fill your information, and proceed with the download
• From the directory where it was saved, allow execution permissions to the installer chmod +x Xilinx-Unified_2019.2_1106_2127_Lin64.bin
• Run the installer sudo ./Xilinx-Unified_2019.2_1106_2127_Lin64.bin
• Sign in in your account, and accept all terms and conditions
• Choose to install Vitis (this also includes Vivado), as we will use it in order to add the DPU to the hardware design and to build the system as a whole
• Make sure UltraScale+ is selected in the installation of the devices
• Make any changes if you do not wish it to install into the default directory, accept and proceed with the installation (this will take a while)
• After the installation has finished, you can source both Vivado and Vitis as source {}/Xilinx/Vivado/2019.2/settings64.sh and source {}/Xilinx/Vitis/2019.2/settings64.sh
• If you choose to generate a .desktop file in order to run the applications easier and your local machine is not in English, you might want to include env LANG="en_US.utf8" LC_ALL="en_US.utf8" before the path to the executable at the Exec in order to avoid trouble with the decimal separator

The Ultra96-V2 board files should have been installed alongside with the Vitis and Vivado software. If this was not the case, the board files are available in this Avnet repository. The installation instructions can be seen at the README.md, but as a guide, the following steps are needed.

• Download or clone the repository to your local machine.
• Copy the desired Board Definition Files (in this case ultra96v2) to {}/Xilinx/Vivado/2019.2/data/boards/board_files
• Launch Vivado and you will be able to select the new board at the Project Creation Wizard

A full installation guide can be seen in the PetaLinux Tools Documentation, but, as reference, the following steps are necessary in order to install PetaLinux into your local machine.

• Choose the PetaLinux 2019.2 Installer
• Sign in and fill your information, and proceed with the download
• A full list of the required packages needed in order to install PetaLinux can be consulted in the Table 2 of the PetaLinux Tools Documentation, but if any are missing, you will be notified once you execute the installer
• Change the installer permissions to 744 with chmod 744 petalinux-v2019.2-final-installer.run. Although the installer only recommends not to set the permissions of the installer to 775 as it may cause BitBake errors, using 755 (allowing all to execute) was found to skip a part of the installation, so 744 is recommended.
• Create the installation folder, it must be writable so that one does not need to use sudo in the following steps, as it cannot be installed as a root user. You will not be able to move this folder after the installation, but you can change the permissions to 755 afterwards. You can, for example, use your home directory for the installation, as mkdir -p ~/petalinux/2019.2
• Run the installer and specify the previous folder as ./petalinux-v2019.2-final-installer.run {}/petalinux/2019.2
• Now is the time when the installer will notify you if any of the needed packages are missing. Install them and run the installer again
• Accept all terms and conditions and let the installation complete
• After installation has been successful, you can now source PetaLinux with source {}/petalinux/2019.2/settings.sh

NOTE: After sourcing PetaLinux you might get this message "WARNING: /bin/sh is not bash!", this is because your default shell is linked to dash. If you want to set it to bash, as seen here, you can make sure your shell is /bin/bash with chsh -s /bin/bash log out and back in, and then make /bin/sh link to /bin/bash with sudo rm /bin/sh sudo ln -s /bin/bash /bin/sh

The Xilinx Runtime Library is needed so as to implement Xilinx Runtime (XRT) in our embedded platform, which acts as a combination of userspace and kernel driver components. In order to install it in our local machine one can either download a pre-build .deb file, like this one for Ubuntu 18.04 or make your own using the source files from this repository, which includes building instructions. Once you have the .deb ready, run sudo apt install <deb-dir>/<xrt_filename_OS>.deb in order to install it.

• Access the Xilinx Product Licensing Site
• Sign in and fill the form with your information
• Choose Search now in Evaluation and No Charge Cores
• Search for "MIPI", and choose LogiCORE, MIPI CSI-2 Rx Controller, Evaluation License
• Select and Add
• Select the new license and Generate Node-Locked License
• Cover your host information (Name, operating system, ID type and value. If needed, you can look this up in the Vivado License Manager)
• Click next, next, and accept the terms and conditions
• Download your license directly from the Xilinx Product Licensing Site or from the email from Xilinx

• Open the Vivado License Manager by executing the vlm script in {}/Xilinx/Vivado/{version}/bin as cd {}/Xilinx/Vivado/{version}/bin ./vlm
• Click Load License and Copy License, choose the downloaded file (Xilinx.lic), it will automatically be copied to ~/.Xilinx/, so it can be moved or deleted after this step
• In case you need to check your host information for the license generation, click View Host Information

These steps can be seen in the tutorial's repository in Vitis-AI-Tutorials/files/tutorials/1-Building-a-Vitis-Ultra96V2-and-MIPI-platform.md, but have been included here with added notes so as to make it easier fo follow.

First, you will create the original non-accelerated MIPI project in the Vivado and PetaLinux tools. Next, you will have bootable hardware and software images to launch a pipeline to view the input MIPI video from the Ultra96.

1. Copy the sources folder from the reference-files/vivado/sources directory to the build/vivado directory.
2. Open Vivado 2019.2. These first scrips check the Vivado version before executing, as the configuration of the IP Cores is dependant of their version, and will not work on a different version of Vivado.
3. Create a new project in the [build/ vivado] directory.
   1. Name it ultra96v2_mipi.
   2. Choose to create a project subdirectory.
4. Select the RTL project option and do not specify sources.
5. Navigate to the Boards tab and select the Ultra96V2 board file as the part/board for the project.
6. Click Finish.
7. At the bottom of the screen, within the Tcl Console tab, change the directory to build/vivado.
8. Use the Tcl Console to call source ./sources/u96v2_mipi.tcl. If your local machine uses a "," as a decimal separator, you might encounter an error here about the "," being an unexpected character in the clock configuration. You can bypass it by executing Vivado using the en_US.utf8 language configuration, including env LANG="en_US.utf8" LC_ALL="en_US.utf8" before Vivado either in the .desktop file or the terminal. After this is completed successfully, you should be able to see the following in your Block Design:

9. In the Sources tab, right-click u96v2_mipi.bd, and then click Create HDL Wrapper (let Vivado auto-update).
10. Click File > Add Files (add or create constraints).
11. From build/vivado/sources, select cnst.xdc, and then select to copy the constraints file into the project. This is an important step, because if the path to the file is relative to the project folder, it will not find it from the Vitis workspace later on.

Now, you will make the necessary additions and modifications to the hardware design to prepare it for software-defined acceleration. To get started, open the base Vivado project.

In order for the Vitis tool to insert your hardware acceleration blocks into the design, you need to leave open and designate the interfaces that it can use to connect the blocks. In this design, you need a few memory mapped interfaces, so that your kernels can connect into the PS DDR. You will open three HP Slave ports in this platform. This portion of the process also allows you to "name" the port, giving it a shorter nickname to designate connections later on.

1. Open the base Vivado project.
2. In the Window menu, click Platform Interfaces.
3. To add the three PS HPx slaves that are not already being used, right-click the interface and select Enable on the following:
   • S_AXI_HP0_FPD
   • S_AXI_HP1_FPD
   • S_AXI_HP2_FPD
4. Enable the HPM0_FPD master interface. Make sure this interface is disabled on the Zynq PS block (within the Processor Configuration Wizard), because this master will be used by the tools to connect to the accelerator. You can check this by entering the Zynq UltraSCALE+ configuration and clicking in the Block Design the "4x S-AXI HP" port in the PL and checking the AXI HPMO FPD beneath Master Interface is not selected.
5. For each slave interface enabled, add an sptag value in the Options tab below in the Platform Interface Properties that will be used to reference the port later in the flow: HP0, HP1, and HP2, respectively.

Similar to how you designated the interfaces for the platform, you now must indicate to the tools which clocks it should use for the accelerators that it places in the platform. In this case, your platform should use two clocks (a 1x and a 2x clock with the 2x provided for the optimized double-pumping of DSP48s), so you will indicate to the platform both a 150 and 300 MHz clock. The DPU hardware accelerator can be clocked faster or slower than this rate, and this rate was chosen to balance power and framerate performance in your application.

1. Double-click the clk_wiz_0 IP, and make the following changes in the Output Clocks tab: [clk_out3=150MHz], [clk_out4=300MHz], [Matched routing selected on clk_out3/4], [Reset Type = Active Low]
2. Right-click the block design, select Add IP, and add a processor system reset IP for each of the new clocks.
3. Name the new clocks, proc_sys_reset_dynamic_1 and proc_sys_reset_dynamic_2.
4. Connect the clk_out3 and clk_out4 outputs of clk_wiz_0 block to proc_sys_reset_dynamic_1 and proc_sys_reset_dynamic_2 slowest_sync_clk inputs, respectively.
5. Connect the "locked" output of the Clock Wizard to the dcm_locked port of the processor reset blocks.
6. Connect the ext_reset_in port of each proc_sys_reset block to the pl_resetn0 port on the MPSoC block.
7. In the Platform Interfaces tab, enable clk_out3 and clk_out4 of the clk_wiz_0 instance.
8. Set the slower clock (in this case, clk_out3) as the default by selecting "is_default" at the options window. clk_out3 should have its id set to 0, and clk_out4 should have its id set to 1.
9. Make sure the proc_sys_reset block listed in each window is set to the instance that is connected to that clock. Check the properties/options window when each clock is selected in platform interfaces, and verify the proc_sys_reset parameter matches.

The default scheduling mode for the acceleration kernels is polled. To enable interrupt-based processing within your platform, you need to add an interrupt controller. Within the current design, you will connect a constant "gnd" to the interrupt controller without connecting any valid interrupt sources at this time. Paired with the AXI-Interrupt Controller is a dynamic_postlink Tcl script in the Vivado tool sources, which will select the interrupt constant net, disconnect it from the concatenation block, and then automatically connect up your acceleration kernel after it is added by the Vitis tool.

1. Right-click the block design, select Add IP, and add an AXI Interrupt controller.
2. In the block properties for the interrupt controller, set the name to axi_intc_0.
3. Double-click the controller block to customize it.
4. Change the Auto/Manual switch to manual for "Edge or Level" and to "Single" for Interrupt Output Connection.
5. Select OK to exit.
6. Add a "Concat" IP to concatenate inputs into the interrupt controller.
7. In the block properties for the concat block, set the name to xlconcat_interrupt_0.
8. Double click the Concat block and modify the number of ports to 8.
9. Add a "Constant" IP to provide a constant "0" to the interrupt controller. This constant will get disconnected and replaced by a connection to acceleration interrupts by the tool at compile time.
10. Double-click Constant IP, and modify the constant value to 0.
11. In the block properties, set the name of the constant to xlconstant_gnd.
12. Click the Run Connection Automation link in the Designer Assistance bar to auto-connect the AXI Interrupt controller's Slave AXI interface.
    1. Choose the HPM0_LPD because the HPM1_FPD is being used for the video subsystem.
    2. NOTE: Make sure to select a "new" interconnect
    3. Select the clk_out1 (200 MHz) clock port for all clock sources.
13. Connect the input of the interrupt controller to the concat block output.
14. Connect the constant output to the first input of the concat block and then each subsequent concat input to this net.
15. Connect the output of the interrupt controller to pl_ps_irq0 on the PS block. Your Block Design should now look like this:

Now that you customized this design, it can be exported to the Vitis tool through a Xilinx Support Archive (XSA).

NOTE: You are not going to build this project to a bitstream. The Vitis tool will utilize this archive to import the design, compose in your hardware accelerators, and at that point, it will build a bitstream. You will automate a portion of this process using the xsa.tcl script—this automates naming and platform details before exporting the XSA file to the hw_platform directory. This script also links your dynamic_postlink.tcl script, so that the script specific to this platform is included inside of the archive.

1. Generate the block design.
2. Export the hardware platform by running source ./sources/xsa.tcl in the Tcl Command window.

The software platform requires several changes to the default Petalinux template. Begin by configuring the project to include a meta-layer, which builds in all necessary support for the MIPI mezzanine card and pipeline. Then, finish by adding the necessary Xilinx Runtime (XRT) components into the design.

The first step in creating your acceleration platform is to open a stock Petalinux project in the build directory. You will then add a meta-layer to enable the necessary support for the MIPI mezzanine and other elements on the Ultra96V2. Finally, you will add the acceleration library components previously mentioned XRT. These components come in the form of recipes, which you will add to the user layer within the Petalinux build. First, you will copy over the files and build recipes, and then you will enable them through the Petalinux configuration menus.

1. Change directory in a terminal to the build/ folder.
2. Create a new PetaLinux project with the ZynqMP template.
   petalinux-create -n petalinux --template zynqMP -t project
3. Copy the meta-ultra96v2mipi folder from reference-files/ petalinux] into [build/ petalinux/ components.
4. Update the Petalinux project with the new exported XSA from the Vivado tool, and open the initial configuration menu.
   petalinux-config --get-hw-description=../hw_platform
5. From the main menu, select Subsystem AUTO settings and under the Serial settings, change to psu_uart_1 as the primary.
6. From the main menu, select DTG Settings, and set the machine name to avnet-ultra96-rev1.
7. From the main menu, navigate to Yocto Settings > User Layers and add ${PROOT}/components/meta-ultra96v2mipi as user layer 0, and then exit the initial configuration menu.
8. To add the XRT drviers to the platform, add the recipes by copying the recipes-xrt directory from reference-files/ petalinux to build/ petalinux/ project-spec/ meta-user.
9. Add a recipe for an autostart script to run automatically after boot by copying the autostart directory from [reference-files/ petalinux/ autostart] to [build/ petalinux/ project-spec/ meta-user/ recipes-apps].
10. Add the recipes above to the Petalinux image configuration by editing [build/ petalinux/ project-spec/ meta-user/ conf/ user-rootfsconfig] to add all lines from [reference-files/ petalinux/ plnxrfscfg.txt]
11. Open the Petalinux root filesystem configuration GUI to enable the recipes above.
    petalinux-config -c rootfs
    and then enable all the recipes added to the user-rootfsconfig file within the "User Packages" and "Apps" sub menus
12. Within the rootfs configuration, under the PetaLinux Package Groups, enable the following options:
    • gstreamer
    • matchbox
    • opencv
    • v4lutils
    • x11
13. Exit the rootfs config menu.

The Linux Device Tree needs to be modified so that the several devices not automatically generated by the Device Tree Generator are correctly recognized by Linux—for instance, the OmniVision camera sensor. Modify the system-user.dtsi file to add the all these nodes to the tree.

1. Open build/ petalinux/ project-spec/ meta-user/ recipes-bsp/ device-tree/ files/ system-user.dtsi.
2. Replace the contents of the file with all the text provided in reference-files/ petalinux/ dtfrag.txt.

Now that you have made all the necessary configuration changes for the Petalinux build, start the build. Depending on the processing power on your machine, build process times can vary. After the Linux build is complete, you need to move all the built software components into a common directory. By placing all your boot components in one directory, it makes it easier to package up the Hardware and Software sides into the resulting platform. You will also use Petalinux to build the sysroot to provide the complete cross-compilation environment for this software platform. This sysroot will also be included in the software portion of the platform because it provides the correct version of headers/includes when compiling the platform.

1. Build PetaLinux.
   petalinux-build\
2. Copy all .elf files from the build/ petalinux/ images/ linux directory to build/ sw_platform/ boot.
   This should copy over the following files:
   • Arm Trusted Firmware: b131.elf
   • PMU Firmware: pmufw.elf
   • U-Boot: u-boot.elf
   • Zynq FSBL: zynqmp_fsbl.elf
3. Copy the linux.bif file from the reference_files/ vitis-platform directory to build/ sw_platform/ boot.
4. Copy the image.ub file from the build/ petalinux/ images/ linux directory to build/ sw_platform/ image.
5. Copy the autostart.sh file from the reference_files/ vitis-platform directory to build/ sw_platform/ image.
6. Build the Yocto SDK (this provides your sysroot) from the project. petalinux-build --sdk
7. Move the build/ petalinux/ images/ linux/ sdk.sh file to build/ sw_platform and then extract SDK. cd build/sw_platform ./sdk.sh -d ./ -y

The Vitis Platform is a set of components that comprise everything needed to boot and develop for a particular board/design configuration and contains both a hardware and software component. Now that you have built the hardware (XSA) and software (Linux image and boot elf files) components for the platform, you can use these components to generate and export your custom user-defined platform. You will complete these steps in the Xilinx Vitis IDE.

1. Open the Vitis IDE.
2. For the workspace, select build/ workspace.
3. Select File > New > Platform Project.
4. Name the platform ultra96v2_mipi.
5. Select to create from hardware specification and select the XSA in [build/ hw_platform].
6. Select the Linux operating system and the psu_cortexa53 subsystem.
7. Deselect Generate Boot Components and click Finish.
8. Exit the Welcome tab.
9. In the File Navigator, double-click the platform.spr file to open the Platform Configuration page.
10. On the ultra96v2_mipi Platform page, select Browse for both FSBL and PMU Firmware, and navigate to the zynqmp_fsbl and pmufw elf files that you previously copied into the sw_platform/boot folder.
11. Customize the linux on psu_cortexa53 domain to point to the boot components and bif that were copied earlier to sw_platform/boot.
12. Customize the "linux on psu_cortexa53" domain to point to the image directory in sw_platform/ image.
13. Right click on the top-level project in the explorer and select Build Project to package up the custom Vitis Platform.

Now that the platform is generated, note that there is an export directory. This export directory is the complete, generated platform and can be zipped up and shared—providing the components to enable new developers on the custom platform.

For the final application, target your custom platform to bring up a simple demo showing the input from your MIPI input pipeline. You will also add in the DPUv2 IP from Vitis AI as a hardware accelerator and configure it to enable deployment of Vitis AI applications and demos in the next steps of this tutorial.

Start by creating the new application project. In the Vitis IDE, the Application Project exists inside of a System Project container in order to provide a method for cohesive system development across the enabled domains in the platform (for instance, Cortex-A53 and Cortex-R5). Because you are working in the same workspace, you can simply target the platform that you generated earlier, but you can also add additional platform repositories by clicking the + button and pointing to the directory containing your xpfm while inside the Platform Selection portion of the new app dialog.

1. Open the Vitis IDE.
2. For your workspace, select build/ workspace.
3. Select File> New> Application Project
4. Name the project, hello_world, and use the auto-generated system project name.
5. Select the ultra96v2_mipi platform that you just created.
6. Choose build/sw_platform/sysroots/aarch64-xilinx-linux as the sysroot path.
7. Verify that the Linux domain is selected.
8. Select Empty Application as the template and Finish.

The DPUv2 IP is provided in the sources in the Vitis AI 1.0 Github. You need to clone this repository and reconfigure the IP to meet you needs before packaging it in a .xo file to compose into your custom platform. You will modify a single header file to designate that you want the B2304 configuration of the DPUv2 IP, with the minimal DSP48 configuration.

1. Change directory to the build folder.
2. Clone the Vitis AI repository.
   git clone https://github.com/Xilinx/Vitis-AI.git
3. Change the branch to v1.0, as this was the version that was used in the making of this tutorial, and not only the resulting DPU, but also the makefile change depending of the version
4. Change directory to build/ Vitis-AI/ DPU-TRD/ prj/ Vitis.
5. Open the dpu_conf.vh file, and replace the contents with those in reference-files/ vitis-apps/ hello-world/ dpu_conf.vh.
6. Run "make" on the dpu.xo object to create the package .xo file.
   make binary_container_1/dpu.xo DEVICE=ultra96v2

1. Under your project in the file navigator, right-click the src folder, and select Import Sources.
2. Choose from directory build/ Vitis-AI/ DPU-TRD/ prj/ Vitis/ binary_container_1 as the target location and import the dpu.xo file that you just created.
3. Import sources again, and add the cpp, header, and prj_config files from reference-files/ vitis-apps/ hello_world.
4. Open the hello_world.prj file and in the upper right-hand corner, change Emulation-SW to Hardware
5. Under Hardware Functions, click the lightning bolt icon to add a new accelerator.
6. Select the "dpu_xrt_top" included as part of the dpu.xo file that you included earlier.
7. Click binary_container_1 to change the name to dpu.
8. Right-click dpu and select Edit V++ options.
9. Add --config ../src/prj_config to designate which port of the DPU will connect to your Platform Interfaces you created earlier. In this file, change the configuration under Vivado from prop=run.impl_1.strategy=Performance_Explore to prop=run.impl_1.strategy=Performance_NetDelay_low, this will add extra delay cost to long distance and high fanout connections to ensure the timing requirements of the design are met.
10. In the Explorer tab, right-click the project folder, and in GCC Host Linker > Libraries, click the green + to add the following libraries:
    * opencv_core
    * opencv_imgcodecs
    * opencv_highgui
    * opencv_imgproc
    * opencv_videoio
11. In the GCC Host Compiler sub-menu, select Includes, and click the red X icon to remove the XILINX_VIVADO_HLS entry.

NOTE: You can also open these menus with ALT+Enter when the hello_world project is selected to access its properties, and then under C/C++ Build > Settings > GCC Host Linker / GCC Host Compiler

1. Click Apply and Close.
2. In the Assistant tab, right-click Hardware, and click Build to kick off a build of the Hardware and Software Components.

After the hardware build is complete, you have an sd_card folder in your hello_world project. You can copy everything in this folder over to an SD card, and load it on the Ultra96 to boot the platform and application. By default, the USB ethernet gadget will display with the static IP address, which you can use to SSH into the board from your host PC and forward the X11 graphics (to preview the images from this initial application). However, you are also able to bring up the board with the UART and configure USB to ethernet adapters or display your demo frames to a display port monitor.