Getting Started With Cloud Foundry Sidecars Tutorial - cloudfoundry/cloud_controller_ng GitHub Wiki

Google Doc

We're now editing this in this Google Doc: https://docs.google.com/document/d/129-W0oZ3ZrzhiwXYzRxVwAuQcO5kt3UdAeXTUnI8Z60/edit?usp=sharing

Getting Started With Cloud Foundry Sidecars

What are sidecars?

Sidecars in Cloud Foundry are additional dependent processes that are run in the same container as the main app process. If you've ever found yourself wanting to include an APM agent or proxy alongside your app, read on.

Why use sidecars

You might be wondering how using sidecar processes differs from just pushing separate apps as microservices. While it's true that these use cases overlap, the ability of the sidecar process to run in the same container as its main app grants us several affordances. Fod example, you may want to use sidecar processes if:

  • You have two processes that need to communicate over a unix socket or via localhost.
  • You have two processes that need to share the same filesystem.
  • You have two processes that need to be scaled and placed together.
  • You have two processes that need to have fast interprocess communication.

How do sidecars work in Cloud Foundry

Sidecars are currently an alpha feature available in Cloud Foundry. All required code and configuration needed to run the sidecar and application are packaged together in the same droplet. This droplet is deployed in a single container on Diego and both processes within the container are health checked independently. To learn more about how sidecars work in Cloud Foundry, check out our sidecar docs.

Pushing an app with a sidecar

To demonstrate this pattern, let's consider a simple Ruby app that talks to configuration-service via a separate Golang binary called config-server. The config-server binary provides applications with their required configuration over its /config endpoint and only accepts connections over localhost on the CONFIG_SERVER_PORT port. For example, below you can see the config-server sidecar respond with some configuration that the main application requires.

vcap@f00949bd-6601-4731-6f7e-e859:~$ curl localhost:$CONFIG_SERVER_PORT/config/
{"Scope":"some-service.admin","Password":"not-a-real-p4$$w0rd"}

Since the main application needs to talk to the config-server over localhost, it needs to be colocated with them in the same container. This makes the config-server process a prime candidate for the sidecar pattern. For demonstration purposes, we've added a /config endpoint to the main app that simply calls out to the config-server sidecar and echos back its response:

get '/config' do
  response = Typhoeus.get("localhost:#{ENV['CONFIG_SERVER_PORT']}/config/")
  response.body
end

The diagram below demonstrates this architecture:

Sidecar Diagram https://drive.google.com/a/pivotal.io/file/d/1wUtO4iIy_FBN4PEPQ6xFRJ1TZ9Vy9cTY/view?usp=drivesdk

In order to push our main application and its sidecar we first have to copy the config-server binary into the main application's source directory.

cp config-server <app-source-code-path>

First, let's create the app.

cf v3-create-app my-app

Next we will configure the sidecar via our application manifest so that it is included when we push the app. An example manifest is shown below:

applications:
  - name: my-app
    env:
      CONFIG_SERVER_PORT: 8082
    sidecars:
      - name: config-server
        process_types:
          - web
        command: './config-server'

Here we are setting the CONFIG_SERVER_PORT environment variable so that the sidecar knows which port to listen on and the main app knows which port to connect to. We also name the sidecar, assign it to the app's main web process, and configure the start command for the sidecar.

This manifest can then be applied using the following command:

cf v3-apply-manifest -f <app-manifest-path>

Finally, we push the app.

cf v3-push my-app

You can see both processes running by cf sshing on to the app container. You can run ps aux and see both the config-server sidecar process and rackup main web process command running.

vcap@f00949bd-6601-4731-6f7e-e859:~$ ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.0  0.0   1120     0 ?        S<s  22:17   0:00 /tmp/garden-init
vcap           7  0.0  0.0 106716  4508 ?        S<sl 22:17   0:00 ./config-server
vcap          13  0.0  0.1 519688 35412 ?        S<sl 22:17   0:00 /home/vcap/deps/0/vendor_bundle/ruby/2.4.0/bin/rackup config.ru -p 8080
vcap          24  0.0  0.0 116344 10792 ?        S<sl 22:17   0:00 /tmp/lifecycle/diego-sshd --allowedKeyExchanges= --address=0.0.0.0:2222 --allowUnauthenticatedClients=false --inhe
root          82  0.0  0.0 108012  4548 ?        S<sl 22:17   0:00 /etc/cf-assets/healthcheck/healthcheck -port=8080 -timeout=1000ms -liveness-interval=30s
vcap         215  0.3  0.0  70376  3756 pts/0    S<s  23:12   0:00 /bin/bash
vcap         227  0.0  0.0  86268  3116 pts/0    R<+  23:12   0:00 ps aux

You can also see that it's listening on the port specified by CONFIG_SERVER_PORT and that our main ruby process is connected to it:

vcap@f00949bd-6601-4731-6f7e-e859:~$ lsof -i | grep $CONFIG_SERVER_PORT
config-se   7 vcap    3u  IPv4 17265901      0t0  TCP *:8082 (LISTEN)
config-se   7 vcap    5u  IPv4 17265992      0t0  TCP localhost:8082->localhost:42266 (ESTABLISHED)
ruby       13 vcap   11u  IPv4 17274965      0t0  TCP localhost:42266->localhost:8082 (ESTABLISHED)

Now, as a demonstration, we can navigate to our app at its route and view the configuration that it is fetching from the config-server sidecar:

https://drive.google.com/file/d/19EmiJJgA15sj7o4QAN9DKsUgjut9aRe-/view?usp=sharing

If you would like to try these steps out yourself, we used the following applications to create this tutorial. They are open-source and available at the links below for experimentation:

Additional resources