Remote Lakes

• Summary
• About Cookbooks
• Limitations
• Background: Brim & Zed Lakes
• Starting a Remote Zed Lake
• Accessing Our Remote Lake
• Contact us!

Summary

By default, the Brim application connects to a Zed lake on the system on which it is launched. However, Brim is capable of accessing data stored in a Zed lake on a remote system. This cookbook describes the available options and current limitations.

About Cookbooks

Brim cookbooks provide an opportunity to "test drive" new/experimental features in the Brim application and related Zed tools. They also walk through details of how Brim and Zed tools function and therefore may inspire other creative configurations.

All efforts are made to disclose known caveats and limitations that are relevant to the configurations shown. However, due to the potential to encounter bugs in evolving functionality, it is recommended that you initially follow cookbooks in a non-production, lab-style setting. As such features become more complete and stable, cookbooks may be retired and replaced with regular User Documentation.

Please report any bugs or usability issues you find when working with cookbooks by opening an issue or reaching out on the Brim public Slack. We'd also love to hear your success stories and variations, so please don't be shy!

Limitations

Before diving into the specifics of what's possible, here's an overview of some rough edges you may encounter as you work through the configurations described in this article.

1. If a user imports a packet capture while connected to a remote lake, the generated logs are stored remotely but the pcap file and corresponding index for flow extraction remain local to the user's workstation.

2. While the configuration potentially allows multiple remote users to access the same centrally-stored data, there's currently no concept of user authentication, individual logins, or roles/permissions. Care should be taken to avoid the accidental exposure or loss of centrally-stored data.

Background: Brim & Zed Lakes

Since it's presented as an icon that can be double-clicked to launch it on your desktop, it's easy to think of Brim as a simple standalone application. However, the overall app experience is powered by a distributed "backend" architecture that includes multiple components.

One essential component is the Zed lake which is accessed via a zed serve process that manages the storage and querying of imported data. Operations on the Zed lake are invoked via a REST API that's utilized by a "client", such as the Brim app. The zed command is also available as a command line client that can perform many of the same operations as the Brim app, and therefore may be useful in scripting and automation.

The default location where Zed stores imported data is a lake subdirectory under the Brim user data path.

If you examine the process table while Brim is running, you can observe the command line that was used to start the backend Zed process. For example, here is the process on a Mac laptop being operated by username "phil".

macOS# ps auxww | grep zed
phil              5289   0.0  0.0 34122868    868 s002  S+    4:37PM   0:00.00 grep zed
phil              5254   0.0  0.1 34867788  15340   ??  S     4:37PM   0:00.03 /Applications/Brim.app/Contents/Resources/app.asar.unpacked/zdeps/zed serve -l localhost:9867 -lake /Users/phil/Library/Application Support/Brim/lake -log.level=info -log.filemode=rotate -log.path /Users/phil/Library/Application Support/Brim/logs/zlake.log -brimfd=3

Some useful information revealed in this command line:

1. The inclusion of localhost in the option -l localhost:9867 indicates this zed serve is prepared to accept only connections that arrive from a client running on the same local host.

2. The -lake option points to the lake user data subdirectory for macOS.

3. The -brimfd=3 is an option unique to when zed serve is launched by Brim. This helps ensure that if Brim is killed abruptly, the zed process will also be terminated (see zed/1184 for details).

4. We can see the full path to the zed binary that's packaged with Brim. This binary and other dependencies that are typically launched by Brim can be found in the zdeps directory under Brim's application binaries path.

Now that we know Brim is simply connecting to Zed locally, next we'll vary this approach to instead start a remote zed serve and connect to it to access the data stored there.

Starting a Remote Zed Lake

For our example remote host, we'll use a Linux Ubuntu 18.04 VM running in Amazon AWS. Because Brim interacts with zed serve over a REST API that is still evolving, care should be taken to ensure the Brim version being installed on the remote side matches the version being run locally. In this cookbook we'll use Brim v0.29.0, which includes Zed v1.0.0.

Even though our VM on AWS has no graphical interface, we'll install the full Brim package because it includes the compatible Zed binaries as well as a bundled Brimcap that will prove useful if we want to import packet capture data.

ubuntu# wget --quiet https://github.com/brimdata/brim/releases/download/v0.29.0/Brim-0.29.0.deb
ubuntu# sudo apt update
ubuntu# sudo apt install -y ./Brim-0.29.0.deb

The following additional steps are also currently necessary to work around issue brim/1701.

ubuntu# sudo find /opt/Brim/resources/app.asar.unpacked/zdeps/suricata -exec chmod go+w {} \;
ubuntu# /opt/Brim/resources/app.asar.unpacked/zdeps/suricata/suricataupdater

Variation: Rather than the full Brim package, we could instead download a Zed package. The Zed package includes the binaries that could be used to construct command lines similar to those shown below. However, as the Zed tools are part of a general data system, they do not include Brimcap. This means such a configuration would either lack the ability to import packet data or would need to be augmented with a separate Brimcap install. The Custom Brimcap Config article in the Brimcap wiki provides relevant guidance for this.

Since there's no desktop environment on this VM, there's no "app" interface to see. Therefore we'll start zed serve manually from the application binaries path for the Linux platform as follows:

ubuntu# mkdir -p ~/.config/Brim/lake ~/.config/Brim/data/brimcap-root ~/.config/Brim/logs
ubuntu# /opt/Brim/resources/app.asar.unpacked/zdeps/zed serve \
          -l :9867 \
          -lake $HOME/.config/Brim/lake \
          -log.level=info \
          -log.filemode=rotate \
          -log.path $HOME/.config/Brim/logs/zlake.log

Building on what we learned earlier, we've made two adjustments here compared to the command line Brim would have invoked:

1. localhost was dropped from the -l option. By providing only the port :9867 specification, zed serve is now prepared to accept remote connections as well.

2. The -brimfd=3 was dropped, since we're controlling the start/stop of Zed rather than the Brim app.

At this point zed serve is ready to accept remote connections. However, the network between clients and our remote Zed lake needs to permit this connectivity. You'll need to perform whatever firewall/VPN configuration is necessary for your environment to enable this. In our specific AWS example, one way to achieve this is via a Security Group configuration that permits incoming port 9867 connections from our client's IP address.

Accessing Our Remote Lake

Now that our remote Zed lake is running, we'll access it from the Brim app that's running on our Mac laptop.

The option to initiate a remote connection is available by clicking the pull-down above the pools list in the left panel and selecting Add Lake.

A window will pop up into which we can specify a name for our remote connection along with its hostname or IP address. An optional port specification may also be included if it's listening on a port other than the default 9867.

Existing pools in our remote Zed lake will appear in the left panel just as we're accustomed to seeing when working with local data. We can now enter Zed queries and perform normal workflows.

You can import data (such as the logs shown below) directly from your Brim app to the remote Zed lake in the same manner as you've been doing locally.

When a packet capture is imported, only the generated logs are stored remotely, while the pcap file and corresponding index for flow extraction remain local to your workstation. This maintains the common workflow of clicking the Packets button to extract flows for selected network logs.

Because only your local pcap index was populated, another user remotely accessing the same pool would be able to query the logs, but in order to successfully extract flows from it, they would need to obtain and locally index the same pcap file. The following commands could be executed by such a user on their Mac workstation to index the same sample pcap shown above. See brimcap/105 for more details.

Another_User_macOS# wget --quiet https://archive.wrccdc.org/pcaps/2018/wrccdc.2018-03-23.010014000000000.pcap.gz
Another_User_macOS# gunzip wrccdc.2018-03-23.010014000000000.pcap.gz
Another_User_macOS# export PATH="/Applications/Brim.app/Contents/Resources/app.asar.unpacked/zdeps:$PATH"
Another_User_macOS# brimcap index -root "$HOME/Library/Application Support/Brim/data/brimcap-root" -r wrccdc.2018-03-23.010014000000000.pcap

A connection to a remote lake can be removed by selecting the Get Info option in the pull-down and clicking Logout. This only removes the config in your Brim app that references the remote lake. It does not shutdown the remote zed serve nor does it delete any data stored there.

Contact us!

If you have questions or feedback about this cookbook, we'd like to hear from you! Please join our public Slack or open an issue. Thanks!