FACADE input guide: directive, model config, and the JSONL format - duttarnab/nl2CedarPolicy GitHub Wiki

FACADE input guide: directive, model config, and the JSONL format

This guide explains directive.textproto and config.textproto, and shows how to produce action.jsonl and context.jsonl from your own logs so that FACADE can train on them.

  • directive.textproto is the data contract. It lists the action types and context types FACADE reads, and how each one is turned into features. Every type in your JSONL must appear here, or FACADE ignores the record.
  • config.textproto is the model definition. It names the features (produced by the directive) that feed each tower of the neural network, and sets the training hyperparameters.

Example: The synthetic world

The world is a 200-person company (plus 3 new hires) across 11 departments and 30 teams, with 12 cross-team projects. The default window is 2025-01-01 to 2025-04-22 UTC.

Type Kind Details
app action SSO login, resource_id = app:<name>. Global apps (mail, chat, vpn and so on), department apps, team apps
resource action fine-grained access, res:<team>:{repo,db,bucket,secrets,dashboard,drive}, project drives and repos, 13 sensitive shared resources (payroll ledger, IdP signing keys, PII export and so on)
org context, daily team, dept, directed reports_to
groups context, daily IdP groups: team, department, role groups (prod-admins, idp-admins, pii-access and so on), project groups, people-managers
collab context, per meeting daily standups, weekly 1:1s, project syncs, ad-hoc cross-team meetings, with a 14-day half-life decay

directive.textproto

# proto-file: protos/directive.proto
# proto-message: facade.Directive
#
# Featurization for the synthetic IAM dataset produced by generate.py.
# Durations are in seconds: 1d = 86400, 3d = 259200, 14d = 1209600, 30d = 2592000.

model_version: 1

# SSO application logins. History feature "prin" = principals that logged in to
# the same app during the previous 14 days (deduplicated per day).
# max_history_keys_per_day drops company-wide apps (mail, chat, vpn, ...) that
# everyone uses (~67 logins/day each); they carry no signal. Must be set
# explicitly: the code only skips this filter when the value is negative, so
# the proto3 default of 0 drops every action.
action_sources {
  type: "app"
  max_history_keys_per_day: 40
  history_duration { seconds: 1209600 }
  action_deduplication_period { seconds: 86400 }
  max_values_per_feature: 100
  min_values_per_feature: 1
}

# Fine-grained protected resources (repos, DBs, buckets, secrets, ...).
action_sources {
  type: "resource"
  max_history_keys_per_day: 40
  history_duration { seconds: 1209600 }
  action_deduplication_period { seconds: 86400 }
  max_values_per_feature: 100
  min_values_per_feature: 1
}

# Daily directory snapshot.
context_sources {
  type: "org"
  context_lookback { seconds: 259200 }
  peer_feature_configs {
    name: "team"
    max_peers: 25
    aggregation_method: AGG_ACCUMULATE
    bipartite_graph {
      traversal_modes: TM_UNDIRECTED
      edge_weighting_method: EWM_LATEST
    }
  }
  peer_feature_configs {
    name: "dept"
    max_peers: 30
    aggregation_method: AGG_ACCUMULATE
    bipartite_graph {
      traversal_modes: TM_UNDIRECTED
      edge_weighting_method: EWM_LATEST
    }
  }
  # Directed: employee -> manager is D_FORWARD, manager anchor is D_BACKWARD.
  # After FACADE's graph fold (neighbors keyed by walk destination):
  # reports_to_f = my direct reports (managers only), reports_to_b = my manager.
  peer_feature_configs {
    name: "reports_to"
    max_peers: 25
    aggregation_method: AGG_ACCUMULATE
    bipartite_graph {
      traversal_modes: [TM_FORWARD, TM_BACKWARD]
      edge_weighting_method: EWM_LATEST
    }
  }
}

# Daily IdP group membership snapshot.
context_sources {
  type: "groups"
  context_lookback { seconds: 259200 }
  peer_feature_configs {
    name: "group"
    max_peers: 50
    aggregation_method: AGG_ACCUMULATE
    bipartite_graph {
      traversal_modes: TM_UNDIRECTED
      edge_weighting_method: EWM_LATEST
    }
  }
}

# Calendar meetings, older meetings decay with a 14 day half-life.
context_sources {
  type: "collab"
  context_lookback { seconds: 2592000 }
  peer_feature_configs {
    name: "meeting"
    max_peers: 50
    aggregation_method: AGG_ACCUMULATE
    bipartite_graph {
      traversal_modes: TM_UNDIRECTED
      half_life { seconds: 1209600 }
      edge_weighting_method: EWM_SUM_DISCOUNTED
    }
  }
}

dataset_parameters {
  snapshot_period { seconds: 86400 }
  snapshot_time_offset { seconds: 0 }
  max_num_actions_per_contextualized_actions: 64
  max_num_contextualized_actions_per_principal_snapshot: 0
}

config.textproto

# proto-file: protos/config.proto
# proto-message: facade.internal.model.ModelHyperparameters
#
# TF feature names follow common/tf_example.py:
#   context: <context_type>/<peer_feature>_<u|f|b>/{t,w}
#   action : <action_type>/prin/{t,w}

embedding_dims: 32
scoring_function: SF_DOT
action_embeddings_transformations: [TR_SOFTPLUS]
context_embeddings_transformations: [TR_SOFTPLUS]
principal_feature_name: "p"

training_hyperparameters {
  batch_size: 64
  training_examples: 400000
  dropout_tokens: 0.2
  dropout_neurons: 0.1
  learning_rate_schedule {
    one_cycle {
      peak_learning_rate: 0.04
      learning_rate_rampup_factor: 25.0
      learning_rate_rampdown_factor: 10000.0
      rampup: 0.3
      interpolation: I_COSINE
    }
  }
  optimizer {
    sgd {
      global_clipnorm: 100.0
    }
  }
  loss_function {
    pairwise_huber {
      soft_margin: 1.0
      hard_margin: 1.0
      norm_push: 1.0
      lse_scale: 0.0
    }
  }
  synthetic_positives_strategy {
    random_sample_within_minibatch {
      contrastive_scores_per_query: 10
      positive_instances_weight_factor: 1.0
    }
  }
  action_name_to_loss_weight: [
    { key: "app" value: 1.0 },
    { key: "resource" value: 1.0 }
  ]
  commensurable_scores_across_actions: false
  evaluation {
    examples_per_training_epoch: 10000
    batch_size: 256
    examples_per_evaluation_epoch: 2048
    metrics_fpr_thresholds: [1.0, 0.1]
    synthetic_positives {
      contrastive_scores_per_query: 1
      positive_instances_weight_factor: 1.0
    }
  }
}

context_architecture {
  concatenate_then_snn {
    segment_reductions {
      token_feature_name: "org/team_u/t"
      intensity_feature_name: "org/team_u/w"
      token_embedding_name: "context/team"
      segment_weight_scaling: WS_IDENTITY
      segment_weight_normalization: WN_L2
    }
    segment_reductions {
      token_feature_name: "org/dept_u/t"
      intensity_feature_name: "org/dept_u/w"
      token_embedding_name: "context/dept"
      segment_weight_scaling: WS_IDENTITY
      segment_weight_normalization: WN_L2
    }
    segment_reductions {
      token_feature_name: "org/reports_to_f/t"
      intensity_feature_name: "org/reports_to_f/w"
      token_embedding_name: "context/manager"
      segment_weight_scaling: WS_IDENTITY
      segment_weight_normalization: WN_L2
    }
    segment_reductions {
      token_feature_name: "org/reports_to_b/t"
      intensity_feature_name: "org/reports_to_b/w"
      token_embedding_name: "context/reports"
      segment_weight_scaling: WS_IDENTITY
      segment_weight_normalization: WN_L2
    }
    segment_reductions {
      token_feature_name: "groups/group_u/t"
      intensity_feature_name: "groups/group_u/w"
      token_embedding_name: "context/group"
      segment_weight_scaling: WS_IDENTITY
      segment_weight_normalization: WN_L2
    }
    segment_reductions {
      token_feature_name: "collab/meeting_u/t"
      intensity_feature_name: "collab/meeting_u/w"
      token_embedding_name: "context/meeting"
      segment_weight_scaling: WS_LOG
      segment_weight_normalization: WN_L2
    }
    snn {
      layer_sizes: [64]
    }
  }
}

action_name_to_architecture: [
  {
    key: "app"
    value {
      concatenate_then_snn {
        segment_reductions {
          token_feature_name: "app/prin/t"
          intensity_feature_name: "app/prin/w"
          token_embedding_name: "action/app_principal"
          segment_weight_scaling: WS_LOG
          segment_weight_normalization: WN_L2
        }
      }
    }
  },
  {
    key: "resource"
    value {
      concatenate_then_snn {
        segment_reductions {
          token_feature_name: "resource/prin/t"
          intensity_feature_name: "resource/prin/w"
          token_embedding_name: "action/resource_principal"
          segment_weight_scaling: WS_LOG
          segment_weight_normalization: WN_L2
        }
      }
    }
  }
]

token_embedding_name_to_config: [
  { key: "context/team" value {} },
  { key: "context/dept" value {} },
  { key: "context/manager" value {} },
  { key: "context/reports" value {} },
  { key: "context/group" value {} },
  { key: "context/meeting" value {} },
  { key: "action/app_principal" value { oov_dropout_rate: 0.02 } },
  { key: "action/resource_principal" value { oov_dropout_rate: 0.02 } }
]
JSONL field Directive entry Feature name produced Config entry
action type: "app" action_sources { type: "app" } app/prin/t, app/prin/w action_name_to_architecture { key: "app" }
context type: "org" + peer attribute name: "team" context_sources { type: "org" peer_feature_configs { name: "team" } } with TM_UNDIRECTED org/team_u/t, org/team_u/w context_architecture … token_feature_name: "org/team_u/t"

The name has three parts:

app      /  prin   /  t
action      feature   token list
type        name      (w = weights)
  • app: the type of the action. It comes from action_sources { type: "app" } in the directive and from "type": "app" in your action JSONL.
  • prin: the name of the one feature the history featurizer builds for each action. It is hardcoded as K_PRINCIPAL_FEATURE = "prin" in pipelines/history_featurizer/pipeline.py. You cannot rename it through config.
  • t and w: t is the list of tokens and w is the list of weights. Both are hardcoded in tf_example.py.

Context features follow the same pattern: <context type>/<attribute name><suffix>/t.

  • org/team_u/t is context type org, attribute team, and _u for an undirected graph.
  • The suffix is _u, _f or _b, set by the traversal mode in the directive.

action.jsonl

To check the allowed format of action.jsonl please check https://github.com/google/facade/blob/main/protos/action.proto and https://github.com/google/facade/blob/main/protos/context.proto.

  • type: the kind of action. It must match action_sources { type: ... } in the directive. You choose the name, using letters, digits, - or _. Here app means an SSO app login.
  • id: a unique ID for this event across all time. FACADE uses it to join scores back to events. Duplicates cause wrong joins.
  • resource_id: what was accessed. Free text. FACADE copies it onto the score but does not use it as a feature.
  • principal: who did it. Use the same ID format in the context file. If this principal has no context records, the action is dropped.
  • occurred_at: event time in UTC, as epoch seconds. The converter also accepts ISO-8601. This example is 2025-01-01 06:00:08 UTC.
  • history_key: which events count as “the same resource” when building history. FACADE looks up who touched this key in the last 14 days. It is usually equal to resource_id, but you can make it coarser, for example one key per app instead of per URL. The converter fills in resource_id if you omit it. In FACADE itself, an empty key would put every action under one key.
{
  "type": "app",
  "id": "app-00001006",
  "resource_id": "app:calendar",
  "principal": "u0156",
  "occurred_at": 1735711208,
  "history_key": "app:calendar"
}
{"type": "resource", "id": "resource-00000740", "resource_id": "res:sales-emea:dashboard", "principal": "u0155", "occurred_at": 1735714011, "history_key": "res:sales-emea:dashboard"}
{"type": "app", "id": "app-00000958", "resource_id": "app:call-recorder", "principal": "u0149", "occurred_at": 1735715154, "history_key": "app:call-recorder"}
{"type": "resource", "id": "resource-00000730", "resource_id": "res:sales-emea:drive", "principal": "u0154", "occurred_at": 1735715607, "history_key": "res:sales-emea:drive"}
{"type": "app", "id": "app-00000970", "resource_id": "app:cms-admin", "principal": "u0151", "occurred_at": 1735716826, "history_key": "app:cms-admin"}
{"type": "app", "id": "app-00000962", "resource_id": "app:sales-engagement", "principal": "u0149", "occurred_at": 1735718501, "history_key": "app:sales-engagement"}
{"type": "resource", "id": "resource-00000723", "resource_id": "res:sales-emea:bucket", "principal": "u0153", "occurred_at": 1735718667, "history_key": "res:sales-emea:bucket"}
{"type": "app", "id": "app-00000998", "resource_id": "app:call-recorder", "principal": "u0154", "occurred_at": 1735719455, "history_key": "app:call-recorder"}
{"type": "resource", "id": "resource-00000718", "resource_id": "res:sales-emea:db", "principal": "u0150", "occurred_at": 1735719561, "history_key": "res:sales-emea:db"}
{"type": "app", "id": "app-00000975", "resource_id": "app:wiki", "principal": "u0151", "occurred_at": 1735720871, "history_key": "app:wiki"}
{"type": "app", "id": "app-00000986", "resource_id": "app:call-recorder", "principal": "u0153", "occurred_at": 1735721774, "history_key": "app:call-recorder"}
{"type": "app", "id": "app-00001008", "resource_id": "app:calendar", "principal": "u0156", "occurred_at": 1735721813, "history_key": "app:calendar"}
{"type": "resource", "id": "resource-00000748", "resource_id": "res:sales-emea:secrets", "principal": "u0156", "occurred_at": 1735722390, "history_key": "res:sales-emea:secrets"}
{"type": "resource", "id": "resource-00000739", "resource_id": "res:sales-emea:db", "principal": "u0155", "occurred_at": 1735722966, "history_key": "res:sales-emea:db"}
{"type": "resource", "id": "resource-00000727", "resource_id": "res:sales-emea:bucket", "principal": "u0154", "occurred_at": 1735723445, "history_key": "res:sales-emea:bucket"}
{"type": "resource", "id": "resource-00000734", "resource_id": "res:sales-emea:dashboard", "principal": "u0154", "occurred_at": 1735723974, "history_key": "res:sales-emea:dashboard"}
{"type": "resource", "id": "resource-00000733", "resource_id": "res:sales-emea:bucket", "principal": "u0154", "occurred_at": 1735724797, "history_key": "res:sales-emea:bucket"}
{"type": "resource", "id": "resource-00000728", "resource_id": "res:sales-emea:repo", "principal": "u0154", "occurred_at": 1735724914, "history_key": "res:sales-emea:repo"}
{"type": "app", "id": "app-00000979", "resource_id": "app:chat", "principal": "u0151", "occurred_at": 1735725656, "history_key": "app:chat"}
{"type": "resource", "id": "resource-00000724", "resource_id": "res:sales-emea:secrets", "principal": "u0153", "occurred_at": 1735725890, "history_key": "res:sales-emea:secrets"}

context.jsonl

To check the allowed format of action.jsonl please check https://github.com/google/facade/blob/main/protos/context.proto

  • type: the context source. It must match context_sources { type: ... } in the directive. Here org means a daily directory snapshot.
  • principal: who this describes. It must match the action file’s principal.
  • valid_from: when these facts became true, in UTC epoch seconds. Here that is 2025-01-01 00:00. FACADE uses a fact only after this time. A snapshot at time T uses records from the lookback window before T.
  • peer_attributes: a list of shared attributes. Two principals are peers when they share the same name and value.
      • name: the kind of link. It must match a peer_feature_configs { name: ... } entry in the directive, otherwise it is ignored.
      • value: the thing shared. Here that is a team name, a department, or a manager ID.
      • direction: used only for directed links, where 1 is forward and 2 is backward. Leave it out for undirected ones like team.
      • weight (not in this example): link strength, default 1.0, must not be negative.
      • time (not in this example): when this attribute happened, for example a meeting end. It defaults to valid_from.
{
  "type": "org",
  "principal": "u0001",
  "valid_from": 1735689600,
  "peer_attributes": [
    {
      "name": "team",
      "value": "accounting"
    },
    {
      "name": "dept",
      "value": "finance"
    },
    {
      "name": "reports_to",
      "value": "u0056",
      "direction": 1
    },
    {
      "name": "reports_to",
      "value": "u0001",
      "direction": 2
    }
  ]
}
{"type": "groups", "principal": "u0001", "valid_from": 1735689600, "peer_attributes": [{"name": "group", "value": "team-accounting"}, {"name": "group", "value": "dept-finance"}, {"name": "group", "value": "finance-approvers"}, {"name": "group", "value": "people-managers"}]}
{"type": "org", "principal": "u0002", "valid_from": 1735689600, "peer_attributes": [{"name": "team", "value": "accounting"}, {"name": "dept", "value": "finance"}, {"name": "reports_to", "value": "u0001", "direction": 1}]}
{"type": "groups", "principal": "u0002", "valid_from": 1735689600, "peer_attributes": [{"name": "group", "value": "team-accounting"}, {"name": "group", "value": "dept-finance"}, {"name": "group", "value": "finance-approvers"}]}
{"type": "org", "principal": "u0003", "valid_from": 1735689600, "peer_attributes": [{"name": "team", "value": "accounting"}, {"name": "dept", "value": "finance"}, {"name": "reports_to", "value": "u0001", "direction": 1}]}
{"type": "groups", "principal": "u0003", "valid_from": 1735689600, "peer_attributes": [{"name": "group", "value": "team-accounting"}, {"name": "group", "value": "dept-finance"}, {"name": "group", "value": "finance-approvers"}]}
{"type": "org", "principal": "u0004", "valid_from": 1735689600, "peer_attributes": [{"name": "team", "value": "accounting"}, {"name": "dept", "value": "finance"}, {"name": "reports_to", "value": "u0001", "direction": 1}]}
{"type": "groups", "principal": "u0004", "valid_from": 1735689600, "peer_attributes": [{"name": "group", "value": "team-accounting"}, {"name": "group", "value": "dept-finance"}, {"name": "group", "value": "finance-approvers"}]}
{"type": "org", "principal": "u0005", "valid_from": 1735689600, "peer_attributes": [{"name": "team", "value": "accounting"}, {"name": "dept", "value": "finance"}, {"name": "reports_to", "value": "u0001", "direction": 1}]}
{"type": "groups", "principal": "u0005", "valid_from": 1735689600, "peer_attributes": [{"name": "group", "value": "team-accounting"}, {"name": "group", "value": "dept-finance"}, {"name": "group", "value": "finance-approvers"}]}

Files for testing

jsonl_to_tfrecord.py

action.jsonl and context.jsonl

Archive.zip

tfrecod &directive.textproto and config.textproto

Archive.zip

Commands

python -m batch.dataset_maker_main \
  --directive="mydata/directive.textproto" \
  --start_time="2025-01-22 00:00:00" --end_time="2025-04-01 00:00:00" \
  --action_path="mydata/action.tfrecord" --context_path="mydata/context.tfrecord" \
  --train_output="mydata/train.tfrecord"



  python -m batch.dataset_maker_main \
  --directive="mydata/directive.textproto" \
  --start_time="2025-04-01 00:00:00" --end_time="2025-04-08 00:00:00" \
  --action_path="mydata/action.tfrecord" --context_path="mydata/context.tfrecord" \
  --validation_output="mydata/validation.tfrecord"



  python -m model.train_main \
  --train_file="mydata/train.tfrecord" \
  --vocabulary_file="mydata/vocabs.tfrecord" \
  --model_config="mydata/config.textproto" \
  --model_dir="mydata/model"




  python -m model.train_main \
  --eval_file="mydata/validation.tfrecord" \
  --vocabulary_file="mydata/vocabs.tfrecord" \
  --model_config="mydata/config.textproto" \
  --model_dir="mydata/model" \
  --is_evaluation_task



  python -m batch.inference_main \
  --directive="mydata/directive.textproto" \
  --start_time="2025-04-08 00:00:00" --end_time="2025-04-22 00:00:00" \
  --action_path="mydata/action.tfrecord" --context_path="mydata/context.tfrecord" \
  --output_file="mydata/scores.tfrecord" \
  --model_config="mydata/config.textproto" \
  --model_dir="mydata/model"
⚠️ **GitHub.com Fallback** ⚠️