Video Files - commaai/openpilot GitHub Wiki

Consider another device for serious "dashcam" purposes

If you want a more traditional dashcam experience with reliability and accessibility in mind, consider a traditional dashcam (VIOFO A229 linked as an example).

  • Comma devices can lose and have lost up to a minute of footage in the event of a crash and power-cutout due to the way the comma device and openpilot records video. Traditional dashcams should handle power-cutouts properly, which you can and should easily test.
  • Getting the video off the comma device in a suitable and redistributable/supported format is a bit of a pain. You are at the right page to learn how to do that, but it's obviously a pain. You can plug a traditional dashcam's SD card into a SD card reader and some dashcams can also be themselves SD card readers when plugged into a computer via USB and expose the data over USB mass storage.
  • The comma device does not record behind the vehicle. Recording inside the vehicle is not the same as behind the vehicle. This is an option with another dashcam vendor.
  • There currently isn't an option to record during parking mode. There is no currently maintained software for this. There is plenty of working parking mode recording functionality with traditional dashcam options with their hardwire kits.
  • Audio is not recorded by the comma device. You might want to verbally read out license plates or other details that may not be clearly visible in the video in the event of an incident. In some dashcams, parking mode may even record audio. That may be handy for interactions with law enforcement.
  • Getting the speed out of footage or data in the comma ecosystem is non-trivial. Dedicated dashcams offer the ability to burn in speeds into the footage with a simple menu setting.
  • If you want to power your dashcam with the OBD port while the comma setup is using it, use a splitter or a specific hardwire kit.
  • In comparison to the C3X, you can have more than 128GB of storage for the dashcam. There aren't developed instructions for expandable storage on the C3X via external storage except maybe in this Discord thread: https://discord.com/channels/469524606043160576/1158097882776350842
  • Uninstalling comma openpilot or a custom fork will wipe all data including previously recorded video. This may be surprising.
  • To be frank, comma has many focuses and the dashcam aspect of the comma device is a line item. Dashcam-specific vendors focus solely on dashcam features and reliability in a highly competitive market.

For video, the comma device and infrastructure should be considered as a backup dashcam, an interface if you're too lazy to walk out to the car, and for the purposes of recording and sharing data for openpilot development. This section is a summary of recommendations from many in the community and to be clear, this isn't "word of comma".

While Comma Prime and even some custom forks offer some features to make the comma device slightly more suitable for dashcam purposes, they do not address many of the serious issues above. Installing a traditional dashcam alongside your comma device is highly recommended.

Introduction

comma hardware, software, and cloud services handle recording of data from your device. This includes video footage and log data. This page is to help you understand how to access and manage that data.

Let's clarify the terms/tooling/infrastructure around and what they are in relation to each other:

  • Segments are 1 minute long video and data files of the same timeframe.

  • Data is grouped into "routes" which are a series of segments that are uploaded together. In general, it's every "ignition on" to "ignition off" cycle.

  • comma Connect

    • comma's online cloud for accessing your comma device's data including a polished UI for viewing, uploading, and downloading files.
    • Only official comma.ai devices can upload to comma connect.
    • Low bitrate preview quality video is automatically uploaded to comma connect on Wi-Fi. On cellular, it'll only auto-upload segments that are currently viewed in connect.
    • High bitrate video is uploaded to comma connect when you press the "Upload Video" button for each video segment/minute you are looking at or pressing "Upload All Files" when looking at a series of segments/minutes.
    • If the data doesn't exist on the device, such as being deleted from space pressure, it cannot be uploaded to comma connect.
    • You can download high bitrate video from comma connect that have manually uploaded.
    • "Routes" 3 days old are deleted unless you have Comma Prime, otherwise a year.
    • You can preserve 10 "routes" to keep them from being deleted after the time limit. 100 routes can be preserved with Comma Prime.
    • Setting up SSH/SFTP access to your comma device is not required to use comma connect.

  • comma Device

    • Your device, a comma device that records data to its local storage and uploads it to comma connect.
    • Old data is deleted when the device runs out of space, not by time limits like in comma connect.
    • This means you can access data that doesn't show up on connect if it's on your device.
    • You can access your device's data via SSH and SFTP outside of comma connect.
    • You can download your own data over and store it however you want.
    • Tools such as Openpilot Toolkit for Windows/Android can assist with this.
    • Some custom forks may include built-in webservers.

Retention

[!WARNING] ⚠️ It is important that you try to collect, upload, and preserve data as soon as possible after an incident. Time and space will be working against you.

This section is more of an expansion of the above bullet list.

Comma Connect

  • Unlimited space in terms of MB/GB/etc. for all users. Primarily limited by time.
  • 3 days of route retention for non-Prime users. 1 year for Prime users.
  • 10 routes preservation limit for non-Prime users. 100 for Prime users.
  • Uploads of high resolution data to be retained must be manually initiated by the user. The data must also exist on the device before it can be uploaded.

Comma Device

Files remain on the comma device until they are automatically deleted due to space pressure except for bookmarked segments.

  • 1TB (C3 with 1TB SSD) stores approximately 60 hours of data/footage

  • 250GB (C3 with 250GB SSD) stores approximately 15 hours of data/footage

  • 128GB (C3X) stores approximately 7 hours of data/footage

  • 64GB (Late C3) stores approximately 3 hours of data/footage

  • 32GB (Early C3) stores approximately 2 hours of data/footage

Per minute:

  • 76 MB each for forward/telescopic camera, extended/wide angle camera, and driver camera (if enabled). All in raw HEVC/H.265 files.
  • 12 MB each for low bitrate H.264 encoding forward camera for preview purposes.
  • 3 MB for Full Raw Logs
  • 1 MB for Trimmed Lite Logs

Bookmarked Segments

If the screen is tapped while driving, and the bookmark icon in the bottom left is touched (it's like an upside-down 🔰), the current segment will be bookmarked.

The last 5 bookmarked segments and two segments before each of those are stored on the device and are not deleted when space pressure occurs. As more segments are bookmarked, the oldest bookmarked segment and previous two are deleted.

Bookmarked segments show up as yellow markers in comma connect.

Getting Video

Finally, let's get to the meat of the matter. How do you get video off the device?

Access comma connect at https://connect.comma.ai/. Log in with the same account and account login type you paired your device with.

Click on a route to view it. You will see a timeline.

Click on the timeline to view the video. Drag on the timeline to contract the timeline to a specific time period. Press the <- to re-expand the timeline away from a specific time period.

op-replay-clipper

If you're not interested in the raw files for processing and archiving yourself, you can cut and get high quality clips from this tool and its guidance:

https://github.com/nelsonjchen/op-replay-clipper#quick-usage

If you are interested in the raw files for processing yourself, please continue to read.

Identifying the needed segments to retrieve

Every time you click on the timeline, you will also be browsing segments as well. You will see that as you hover over the timeline, there is a number before the timestamp. That is the segment number for the route. As a reminder, segments are 1 minute long video and data files of the same timeframe.

The best way to use connect to get the needed footage is to contract the timeline to the time period you want, and then hover your mouse across the entire timeline to find all the segment numbers of the segments you want.

With this information, you can identify the segments you want to upload/download to/from via comma connect or directly from the comma device.

Retrieving the raw files

Comma Connect

Click and drag contract the timeline to the time period you want.

Use the Files dropdown and select Upload All Files to queue up all the segments in the time period you want to upload.

Use the Files dropdown to queue raw camera video file HEVC segments for upload and to download them after they have been uploaded. You'll have to click on the timeline in each segment before clicking the necessary action in the Files dropdown.

Note that Connect doesn't show all the files that are on your device. Routes older than a certain time disappear from view. That means a route that hasn't been pruned for space but falls past the 3 days old mark could still be on your device. If you have Prime, that time limit is a year.

To retrieve that you must use the below:

Openpilot Toolkit/SSH/SFTP

Use the Files dropdown in connect to get a copy of the segment ID.

A segment ID looks like this:

fe18f736cb0d7813|2023-10-06--21-30-05--1

The first part is the device ID. The second part is the date, time, and segment ID number of the route. You only need the second part.

The Segment ID is copied to your clipboard as text. You can paste it into a text editor for your reference.

Later, when we're looking for files, you'll need to know the segment ID. As an example, the above segment ID folder would be 2023-10-06--21-30-05--1. You'll need to know the segment IDs of all you want to download from the device. If you wanted minutes 1 through 4, you would need to know the segment IDs of 1, 2, 3, and 4 which as an example would be 2023-10-06--21-30-05--1, 2023-10-06--21-30-05--2, 2023-10-06--21-30-05--3, and 2023-10-06--21-30-05--4.

Now that you have this, setup/get the software needed to download the files.

Openpilot Toolkit

image

This is a GUI tool for Windows and Android that can download segments from the device. It can also download the entire route. It can also download the entire route and convert it to a more consumable format. It can also walk you through the process of setting up SSH/SFTP access to your device.

https://spektor56.github.io/OpenpilotToolkit/

SFTP

Requires a Github account, making encryption keys, enabled SSH on the device, and installing a SFTP client on your computer

Software SFTP Clients:

Host Name: IP Address, Port: 22, Username: comma

It is left as an exercise for you on how to load a generated private key into your SFTP client.

Once connected, start in path:

/data/media/0/realdata

The segment timestamp identified above will match a subfolder name. Download the target .hevc file, and surrounding .hevc files as desired. Name them in some sensical manner. They can be drag-and-dropped into a VLC window to play

The output .hevc files can be uploaded to YouTube straight away for processing, or played in VLC.

SSH

Requires a Github account, making encryption keys and enabled SSH on the device.

To connect to your comma device with SSH please see SSH.

To download the video files locally to your machine you can use SCP (secure copy protocol.) If your use case is to download all files you could run the following from a terminal on your local machine.

Copy all:

scp -r -i ~/.ssh/<your_private_key> comma@<your_commas_ip_address>:/data/media/0/realdata/ ~/<desired_destination>

Copy selective:

scp -r -i ~/.ssh/<your_private_key> comma@<your_commas_ip_address>:/data/media/0/realdata/<timestamp folders> ~/<desired_destination>

Playback and Processing

Very few apps such as VLC can play these files.

To combine multiple segments (if needed), use this Linux command:

cat fcamera1.hevc fcamera2.hevc fcamera3.hevc > output.hevc

or this Windows command

copy /b fcamera1.hevc+fcamera2.hevc+fcamera3.hevc output.hevc

Making the .hevc file more consumable in other applications that support H.265/HEVC .MP4 files.

These steps will produce a .MP4 file that can be played in many video players or uploaded to a web service at the correct 20 FPS if they support .H265/HEVC.

Browser (ffmpeg-online)

  1. Visit https://ffmpeg-online.vercel.app/?inputOptions=-r%2020%20-i&outputOptions=-c%20copy%20-map%200%20-vtag%20hvc1

  2. "Upload" the .hevc file.

    • Technically it never gets uploaded to a server, but the file is loaded into the web page running on your computer to be processed .

  3. Ensure the settings are what it is below, with changes to the output file name as you desire.

    Field 1: -r 20 -i

    Field 3: -c copy -map 0 -vtag hvc1

    It should look somewhat similar to this:

  4. Click the run button, let it process, and then click the download file link that appears below.

Discord's H.264 requirement and Free < 25 MB limit

Discord only inlines H.264 MP4 files for playback and free accounts can only upload 25MB. If you just have an .HEVC file and want to post it on Discord, you should use this alternative preset link to correct the FPS, transcode, and recompress. It will take about 10 minutes or more depending on your processor speed.

https://ffmpeg-online.vercel.app/?inputOptions=-r%2020%20-i&output=output.mp4&outputOptions=-b%3Av%203M

Local, non-browser (ffmpeg)

It is possible to run ffmpeg locally. It can be much, much faster if you need to transcode.

  1. download ffmpeg

  2. Run ffmpeg against the video file

    ffmpeg -r 20 -i input.hevc -c copy -map 0 -vtag hvc1 output.mp4

Cutting

After the file is converted to MP4, you can use a video editor to cut the video.

https://github.com/mifi/lossless-cut is a free cool way to join/trim/cut videos without any loss in quality.

Rendering with UI (Comma 3/3X + Comma Connect Required)

https://user-images.githubusercontent.com/5363/219708673-4673f4ff-9b47-4c57-9be3-65f3ea703f3f.mp4

Video files do not include the UI. You'll need to re-render the video with the UI.

It is possible to use comma's tooling to replay video and logs to an openpilot UI process and see a copy of the UI:

https://github.com/commaai/openpilot/tree/master/tools/replay

A community tool has repackaged and automated that process so it is easy to "replay" a clip and get a recording:

https://github.com/nelsonjchen/op-replay-clipper/

The primary platform for this is Replicate which is extremely cheap and pay as you go. Please see the README of the project for more information.

Rendering 360

Guess what?

https://github.com/nelsonjchen/op-replay-clipper/

does it too.

Example:

https://www.youtube.com/watch?v=dbObmLBRzc4

You can also reframe 360 videos too. See https://github.com/nelsonjchen/op-replay-clipper/?tab=readme-ov-file#reframing-360-and-360-forward-upon-wide-to-a-normal-video

https://github.com/user-attachments/assets/bf55df68-843a-4a76-ab91-7d2d8760f151

Comma Two Errata

The comma two isn't really supported but here's some errata for it on how it differs from modern comma three and up devices.

  • Media is located at /sdcard/realdata/ instead of /data/media/0/realdata/
  • FPS is not correct in the video files, so you will need to correct it with ffmpeg or similar to 20 FPS.
  • The comma two has a driver camera but the FPS is 10 FPS instead of 20 FPS.
  • You may need to SSH into port 8022 instead of 22. The username is root instead of comma.