First implementation of listen@UDS - slimhazard/varnish-cache GitHub Wiki

This commit is a first attempt at hacking Varnish so that it listens at a Unix domain socket (-a /path/to/uds). This is not yet an implementation of everything in VIP17, it's just an example of working code for discussion.

This implementation successfully listens to a UDS and responds to requests. It also passes all of make check, although it has no tests that use a UDS. Nothing has been added to varnishtest yet to support UDSen.

UDS backend addresses are not yet implemented.

This uses no space in session workspaces for Unix sockaddrs, and does not need a destructor for VSA (although I imagine we'll need one when UDS backends come along). That was accomplished like this:

  • Neither of VSA_Malloc or VSA_Build allocate space for a sockaddr_un, but VSA_Clone might.
  • VSA_Malloc and VSA_Build now each have an additional argument, which may be NULL or a pointer to an allocated sockaddr_un. If not NULL, then it becomes the VSA's internal pointer to sockaddr_un. The caller is responsible for managing the space allocated for the sockaddr_un.
  • mgt_acceptor.c sets up the suckaddr_un for a UDS from the -a arg (in this implementation, a UDS for -a is required to begin with a /, anything else is assumed to be an IP address).
  • mgt_acceptor.c sets up the pool of listen_socks (which have a suckaddr field), and these get handed off to the child via heritage.socks.
  • In cache_acceptor.c, when a connection is accepted, it checks if the suckaddr from the pool of listen_socks was a UDS. If so, it uses the pointer to the suckaddr_un from the pool in VSA_Build to create its own suckaddrs.

So the only space allocated for sockaddr_un lives in the pool of listen addresses created in mgt and passed down via heritage. The suckaddrs in session workspace, if UDSen, point back to them.

AFAICT there is no destructor for the heritage.socks in which the sockaddr_uns live, the pool is just allowed to die when Varnish exits. This is why a destructor for VSA is not needed yet.

Some other issues and weirdnesses:

  • Having a pointer to sockaddr_un in suckaddr's union means that it is no longer possible for the untyped sockaddr to live in the union as well, to be able to read the protocol family from there. So I've pulled out the sun_family_t field as a distinct field in the suckaddr, and the untyped sockaddr no longer lives in the union. This means unfortunately that there are two sun_family_t fields with the same value in a suckaddr when it's an IP address (once in the suckaddr, and again in the sockaddr).
  • In this implementation, just for testing, the UDS path is unlinked in VTCP_bind before the bind(2) call, which creates the socket at the path and fails with EADDRINUSE if something exists at the path. unlink fails if there is nothing at the path, so Varnish doesn't start unless you've already touched a file there. This is the opposite of the way I think it should work.
    • Suggestion: Let Varnish fail with EADDRINUSE if the path already exists (and document that), and install an atexit handler that unlinks the path when Varnish stops.
  • Nothing was done yet for the PROXY protocols, but this should be extended to support UDS (there are specifications for that in the haproxy document).
  • Socket options: In cache_acceptor.c, options are not set for a UDS socket if the level is IPPROTO_TCP. That "feels" right to me, and in particular, this means that none of TCP_NODELAY or the TCP_KEEP* options are set, which also feels right to me. However, I cannot find any documentation saying that IPPROTO_TCP options are invalid for AF_UNIX as a generally valid and portable rule. The documentation I've seen is very poor at being specific about which socket options are valid for UDS and which are not.
  • VTCP*name, which assume a string for an IP address and for a port, are all over the damn place in the code. For example, as of now ReqStart says (null) (null) when it's a UDS. I have a few ad hoc workarounds where that had to be avoided. There will need to be many more, or a different and more general solution.
  • Sessions now do not necessarily have IP and PORT attributes, although this has been assumed to be universally the case. This will also lead to a number of necessary changes.
  • What do you add to X-Forwarded-For when the listen address is a UDS? Can't find anything about that, and RFC 7239 does not mention the subject. Maybe check what haproxy and nginx do for inspiration.