4K Camera demonstration - ArrowElectronics/Agilex-5 GitHub Wiki
The 4K Camera Agilex 5 FPGA E-Series 065B Arrow Eagle Board Demo Design showcases a practical glass-to-glass camera solution. The exclusive support for industry-standard Mobile Industry Processor Interface (MIPI) D-PHY and MIPI CSI-2 interface on Agilex 5 FPGAs provides a powerful tool for camera product development. Supporting up to 2.5Gbps per lane and up to 8x lanes per MIPI interface enables seamless data reception from a 4K image sensor to the FPGA fabric for further processing. The MIPI CSI-2 IP converts pixel data to an AXI4-Streaming output, enabling connectivity to other IP cores within Intel's Video and Vision Processing (VVP) Suite. This demo design includes an Image Signal Processing (ISP) subsystem incorporating many standard cameras processing IP cores such that raw sensor image data can be processed into RGB video. The ISP enables auto white balance and exposure correction by continuously collecting and analyzing runtime statistical data using the embedded Hard Processor System. This solution also includes adaptive local tone mapping for handling wide dynamic range scenes, 1DLUT and 3DLUT IPs for color transformations and HDR conversion, and a high-performance Warp IP core for geometric distortion correction. 4Kp30 streaming video is output via Altera DisplayPort IP.
Selectable Input
- MIPI D-PHY connectivity to Raspberry Pi High-Quality Camera (12.3 megapixels Sony IMX477 Sensor) with MIPI CSI-2 interface
- Internal TPG supporting SMPTE Colorbars, Solid Colors, and moving Zone Plate patterns
- 4Kp30 12bit Bayer
- AXI4-Streaming interface
ISP Pipeline (12bit Bayer)
- Clipper
- Image Signal Processing, including Black Level and White Balance Correction, Demosaic, Defective Pixel Correction, Adaptive Noise Reduction, Vignetter Correction, and Statistics.
Video Pipeline (10bit RGB)
- Sharpening
- Warp IP
- Tone Mapping Operator IP
- 3D LUT IP
- 1D LUT IP (12bit)
Output
- 4Kp30 10b DisplayPort
Runtime Processing
- Quad-core ARM Cortex CPU
The following components are required for the demo:
- AXE5-Eagle (TEI0185) development kit,
- 8GB SD card
- Ethernet cable
- Bitec DP FMC card
- Video Sink (3840x2160 resolution up to 30 fps with 10b RGB support)
-
- DisplayPort 2.0 or
- Raspberry Pi High Quality Camera
- Ultra Wide-Angle Lens
- 15 pin to 22 pin flat MIPI cable
- Tripod
- CRUVI ST4 to MIPI FFC Adapter
To run the 4K Camera demo, follow the steps below:
The picture below shows the required components
The camera has an FPC connector which mates to the narrow end of the cable. Both the FPC connector and the cable have conductors on only one side. To make the connection, make sure that the FPC connector has its black tab lifted up on both ends. Insert the narrow end of the cable, with the conductors facing the white side of the connector, the push the tabs down into the connector body.
Attach the camera to the tripod. Thread the wide-angle lens into the camera body.
The AXE5-Eagle Development kit contains the following components:
- Heat sink
- Fan
- Clip
Clean the surface of the FPGA. Remove the plastic film from the underside of the heat sink. This exposes an adhesive surface. Carefully place the heat sink assembly over the surface of the FPGA, centering it. Slide the clip over the fins of the heat sink, in the center. Attach both sides of the clip to the underside of the heat sink ??? on the board. This will require some effort. Use the screws provides to attach the fan to the heat sink. Plug the fan connector into the header on the board (J13).
The MSEL2 and MSEL1 DIP switches must be set to the ON position (left) for Boot from QSPI configuration.
- Insert the SD card in the J24 cage, on the right hand of the board.
- Attach the micro-USB cable to UART (J5) connector
- Plug the Arrow-USB-Blaster (TEI0004-02) into J34 with the USB connector facing to the right.
-
Attach the Ethernet cable to J4 (ETH_HPS). Attach the other end of the cable to a Network device with ethernet connection (with DHCP server)
-
Connect a DisplayPort cable from the DP FMC adapter card to a DisplayPort video sink
OR
- Insert the DP to HDMI adaptor into the DP FMC adaptor card. Then connect an HDMI cable from the adapter to an HDMI video sink.
Connect the cable to the Eagle board. This step is accomplished via the CR00300 CRUVI adapter. Insert the CRUVI adapter CR00300 into the C_HSX connector on the Eagle board.
Lift both ends of the black tabs on the FPC connector as you did on the camera. Insert the cable with the visible conductor side facing the edge of the adapter. Push the black tabs in, all the way.
- Connect the power supply to the AXE5-Eagle J29 barrel connector
- Plug the AC-DC adapter into an AC outlet
The completed demo setup is shown below.
- Download the FPGA jic image axe5_eagle_top_hps.jic
Use the Intel Download cable when programming the AXE5-Eagle board from a Linux environment.
The FPGA JTAG chain will expose 1 or 2 devices when auto-detected from the programmer. Follow the appropriate instructions to program the Agilex 5 FPGA for either scenario.
Open a shell (Nios V Command Shell for Windows)
Determine the number of JTAG devices
$ jtagconfig
One JTAG device 1) Arrow-USB-Blaster [ARA31601-TEI0004] 0364F0DD A5E(C065BB32AR0|D065BB32AR0) Two JTAG devices 1) Arrow-USB-Blaster [ARA31601-TEI0004] 4BA06477 ARM_CORESIGHT_SOC_600 0364F0DD A5E(C065BB32AR0|D065BB32AR0)
Option 1: One Device
$ quartus_pgm -c 1 -m jtag -o "pvi;axe5_eagle_top_hps.jic@1"
Option 2: Two Devices
$ quartus_pgm -c 1 -m jtag -o "pvi;axe5_eagle_top_hps.jic@2"
Program the device
This will take a few minutes to complete.
- A wired Micro USB serial port connection between the host PC and the embedded target is required
- Launch a terminal program (like Tera Term VT or Putty) and connect using serial port
- Select 115200 baud
- Select the appropriate target COM port
Press the 'NCONFIG' button to boot the demo. This will initiate the following boot process.
With the board set up, the HPS Arm processor will start up and boot into an embedded Linux OS. Observe the terminal output and wait for the startup sequence to complete to the Linux prompt.
- At the prompt, type "root," then press enter.
- Next, type "ifconfig" and press enter. This will obtain the device IP address (inet address of eth0), which you will use to connect to the web server running in the FPGA.
- To start the application, type "./VvpIspDemo", then press 'enter'.
- Enter the IP address in a web browser on a device on the same local network to run the application GUI.
The 4K Camera Demo Design software application has a web-server GUI that is used to demonstrate the features of the design. From a PC connected to the same LAN as the development board, open a web browser and enter the IP address of the board into the address bar.
Download the camera profile and save it. Click on the settings icon and select Settings --> Presets --> Import. Navigate to the camera profile and click Open.
The Clipper function in the Input Config section can be utilized to focus on select Regions of Interest in the Image.
The Warp function on the Output Config section provides rotation, zoom and keystone features.
The User Guide provides detailed information about the Camera design and the Video and Vision Processing (VVP) suite.