VDD23Q1 - varnishcache/varnish-cache GitHub Wiki

revisit last year's notes VDD22Q2

• dridi talks about progress on #1 logging - keep shared memory, but add option to block varnishd - different vsm layout for improved concurrency, more page based - -d option would still be kept, idea of running queries in varnishd is not on the table any more at this time

• discussion about the nature of the problem - phk mentions old idea about split log (complete/incomplete tx) - phk idea to suppress transactions from VCL - phk idea to use log files only once, open a new one once full - phk idea keep req grouping in varnishd (into dynamic memory), flush only once per top level request. - dridi presents vsl scrubbing idea

  • replace log entries or fields with Xes
  • discussion about oob info from length - should redaction be a single fixed token or original length?

  tasks - VIP for scrubbing (dridi) - VIP batching on toplevel req in varnishd (PHK)

• reiterated over status quo

• no progress
• discussion again about the future life of vtest: binary dependency vs. source dependency vs. submodule
• VS has green light to contribute TLS support
• slink has talked to CF guys at H3 workshop, no known show stoppers
• h2o also has keyless "neverbleed" https://github.com/h2o/neverbleed MIT licensed
• should be one or several VEXTs (openssl, libressl, ...)

• TODO

  • vtest
  • VIP the backend interface
  • backend definition like vmod function for vext

  backend newstuff {
    .options = {
      .foo = 123;
    }
  }


options get passed as string at init time

• asad: still working on it with msquic
• also researching alternatives
• haproxy also needs patched openssl

• hermunn/dridi: want to add a parameter to vcl.load as the base path for includes with fall back on global vcl_path, ban absolute includes from vcl

• do we want extension arguments?
• env vars working find for now

• loading old storage requires a control system
• control system can assign the "ban id" (current vtim_real)
• CLI command to return min(max(silo(ban_id))) (or per silo max(silo(ban_id))) ?
• feature flag to deny bans from vcl
• multi tenancy support

General idea to ponder: proper VCL "jails" for multi-tenancy

• local, task scoped variables/objects
• vcl functions
• bump vcl version every time? -> NO

• new include version check?

  • "umbrella" vcl x.y requires x.Y with Y <= y

for (x in vmod.bla(...)) {
}

foreach (key[, val[, idx]]) in vmod.something(arguments) {
    key type is defined by vmod.something iterator
    val type is defined by vmod.something iterator
    idx is integer
}

foreach x in vmod.something(arguments) {
    x type is defined by vmod.something iterator
    can have x.key x.index, x.whatever
}

# built-in structured fields knowledge?
foreach bla in req.http {  // bla has type "http_header"
   what members does `bla` have ?
   // "literal, string, iterable".fields possible too
   if (bla.name == "something") {
     for fld in bla.val.fields { // comma, fld has type sf-element (?)
         for x in fld.attr { // semi-colon, x has type sf-attr (?)
             x.key x.val
         }
     }
   }
}

jobj = vmod_json(input)
for x in jobj.somthing_i_really_want(dont_fail_hard=True) {
}
if (jobj.failed) {
}

foreach bla in req.http {
   if (bla.name ~ "(?i)cache-control") {
       foreach directive in std.split(bla.val, ", ") {
           ...
         }
     }
}

=> Everyone should re-write their complex vcl how they would

want it to look

Fastly-Syntax for SF:

resp.http.Cache-Control["max-age"] = 2h;

function foo(myvmod.mytype x) {
}

What about:

for (x in myvmod.bla()) {
  foo(x);
}

Also, what if the above is in foo()? Answer: Maybe we will allow recursive functions, maybe not

# foo takes type returned by vmod?

Dridi gives a very short talk about a minor change. He will share his 49 page presentation later.

https://github.com/varnishcache/varnish-cache/files/10707808/vdd23q1-compliance-stuff.pdf

if (req.http.Cache-Control[private])  => true if private present
if (req.http.Cache-Control[private] is none) => true if private has no value
if (req.http.Cache-Control[private] == "public") => true if value of private folds to

  string "public"


if (req.http.Cache-Control has private)

if (req.http.Cache-Control[private] is flag) => true if private is present
if (req.http.Cache-Control[private] is not bool) => true if private= is present
if ("private=4"[private] is string) => false
if ("private=*true"[private] is not bool) => false
if ("private"[private]) => bool (true)

if (exists(req.http.Cache-Control[private]))
if (nonzero(req.http.Cache-Control[private]))

if (req.http.Cache-Control[private])
if (req.http.Cache-Control[private].exists)
if (req.http.Cache-Control[private].val)
if (req.http.Cache-Control[private].type)

  "private" => True, None, Bool
  "private=" => True, "", String
  "private=X" => True, "X", String
  "private=43" => True, 43, Number
  "" => False, None, None

if (req.http.Cache-Control[private]) { //exist
  if (req.http.Cache-Control[private].value) { //exist && has value
  }
}

==> PHK to write up a strawman

Specifically on issue #3844:

• HEADER == STRING should be equivalent to STRING == HEADER
• implicit HEADER to BOOL conversion conveys existence
• discussion resumed in the github issue

Does it work as is? - basically yes - but we want to set up IRC & web bridge with simple basic auth

• should keep logs

@slink to ping @thijs about VUG

VUG needs to be a varnish-cache (!) thing find a sponsor for a venue VS, UPLEX, fastly, whoever, are welcome as sponsors

10 days after release idea: 26./27. september

VDD: combine with VUG

sub vcl_init {
      mytls = new newtls();
      mytls.load("/etc/tls/.....pem");
      mytls.load("/etc/tls/.....pem");
}
sub vcl_client_hello {
      mytls.dynload(tls.sni, ttl=10m);  # Dynamicly load cert?

      if (tls.sni == "legacysite.asas") {
              set tls.ciphers = "ssl, tlsv3";
              tls.hint = mytls;
      }

      set tls.ciphers = "sslv1, tlsv3";
      set tls.alpn = "h1,h2,h3=....";
      tls.hint = fallback;
      return(OK);
}

also: vcl_h[23]_settings, proxy2_tlv

---

VEXT client_ssl has vmod tls:

sub vcl_init {
  new something = tls.init(...)
  something.callback(myfunc)
}
func myfunc(tls.type: arg) {
  arg.crytomumblemumble = "bla";
  return(tls.OK)
}

---

sub vcl_accept { // Probably not needed.
  if (...) {
      filter.use();
  }
}

sub vcl_backend_response {
  set beresp.filter_list += "gunzip vmodfoo.spot_foo";
}


sub vcl_backend_body_is_here {
  if (vmodfoo.did_you_see_foo) {
      if (! beresp.body.try_filter({"regsuball(...) gzip"})) {
        return (deliver); # keep the "source" object
      }
      beresp.body.filter("...");
  }
  // trailers goes here
}

we can have h2 in-core, but vext is maybe also viable option.