Filesystem Paths - philrz/scratchwiki GitHub Wiki

Filesystem Paths

Occasionally when using Brim or troubleshooting problems with it, you may need to know the location on the filesystem where the application binaries and/or user data is stored. Below you'll find these details for each supported platform.

Application Binaries

When installed, the binaries and other supporting files that make up the Brim application are unpacked to the following path on each platform:

OS Platform Location
Windows %USERPROFILE%\AppData\Local\Programs\Brim
macOS /Applications/Brim.app
Linux /opt/Brim

This filesystem location should be treated as read-only and is typically only modified when a newer version of the app is being installed.

Since Brim is most often launched directly from the GUI environment in your OS, you'll typically not need to access the application binaries directly. One exception is if you wish to make use of the command line binaries that are bundled with the app, such as zq, zed, brimcap, and so forth. These can be found in the zdeps subdirectory as follows:

OS Platform Location
Windows %USERPROFILE%\AppData\Local\Programs\Brim\resources\app.asar.unpacked\zdeps
macOS /Applications/Brim.app/Contents/Resources/app.asar.unpacked/zdeps
Linux /opt/Brim/resources/app.asar.unpacked/zdeps

While these binaries are also available for download as standalone release packages for Zed or Brimcap, if you're using CLI commands to interact with a pool in the Zed lake that runs behind Brim, you should typically use these bundled zdeps binaries since they've been tested with the same version of Brim and can be considered API-compatible with that version of the app.

User Data

Saved data from your work in Brim is stored in a read/write filesystem path that's separate from the application binaries. The top-level user data path on each supported platform is:

OS Platform Location
Windows %APPDATA%\Brim
macOS $HOME/Library/Application Support/Brim
Linux $HOME/.config/Brim

Particular categories of saved data are held under this path. Specific categories of interest include:

  • lake (subdirectory) - Storage for the local Zed lake that holds data you may have imported into Brim.

  • data (subdirectory) - Storage used by tools bundled with Brim, such as Brimcap.

  • logs (subdirectory) - Logs for the running Brim app and the Zed backend that it launches for data storage and query.

  • appState.json (file) - Persistent app settings such as changes you've made to Preferences, contents of the Query Library, entries in the History panel, and so forth.

Generally you should not need to directly access the saved user data, though it may prove necessary during Troubleshooting or if you use zed or brimcap CLI tools to perform scripted operations from outside the app against data that's stored behind Brim.

Temporary Storage

Brim and the bundled Zed/Brimcap tools often make use of temporary storage. Some examples:

  • The Zed backend may use temporary storage to "spill to disk" when performing sort, fuse, or aggregations on data sets that cannot fit into allocated system memory. For this, Zed uses a directory with a name that starts with zed-spill-. The directory is automatically deleted when the operation finishes.

  • Brimcap uses temporary directories to store the raw logs generated by analyzers (e.g., Zeek and Suricata) before they're loaded into a pool in a Zed lake. For this, Brimcap uses a directory with a name that starts with brimcap-. The directory is automatically deleted when brimcap analyze finishes.

  • In Brim when the Packets button is clicked to extract a flow from a larger pcap, the extracted packets are placed in a temporary file that's opened in Wireshark. For this, Brim uses a file with a name of the format packets-[date+timestamp].pcap (e.g., packets-2020-03-09T17_27_53.399273Z.pcap). This file is not deleted automatically, as Brim does not make any assumptions about what you may continue to do with the pcap file outside the app.

Standard conventions are used for creating these directories/files in the appropriate location for each supported platform. As a result, temporary storage may be freed up automatically by the OS during normal operations, such as when performing a system restart or running common cleanup utilities. However, bugs or abrupt shutdowns of Zed/Brimcap processes (e.g., kill -9 or similar) may prevent the tools from deleting their temporary directories as individual operations finish. This may lead to excess consumption of temporary storage over time, particularly if your temporary storage is not regularly cleared in the course of normal operations.

If during the course of operations or debugging you'd like to check for signs of unreaped temporary storage, you can look below a base temporary directory for your platform. Below we describe the expected base location in an "out of the box" default configuration of each supported platform.

  • Windows - The base path specified in the %TMP% environment variable is used. In a default test environment we've found this variable is already set and is typically of a format like C:\Users\%USERNAME%\AppData\Local\Temp or C:\Users\%USERNAME%\AppData\Local\Temp\2.

  • macOS - The base path specified in the $TMPDIR environment variable is used. In a default test environment we've found this variable is already set and is typically of a format like /var/folders/yn/jbkxxkpd4vg142pc3_bd_krc0000gn/T/.

  • Linux - A base path specified in a $TMPDIR environment variable would be used, if it were set. However, in default test environments we've found this variable is typically not set, and therefore the /tmp path is used.

Note: Detailed explanations for the logic used for locating the temporary storage location can be found in the Electron docs (for Brim) and the Go docs (for Zed/Brimcap).

If you'd like to change the base temporary storage path that's used by Brim and the Zed/Brimcap tools, just ensure that Brim is launched after having set the environment variable mentioned above that's appropriate for your platform.