The problem

Many routers exports these flow fields:

• src_mac, dst_mac, post_src_mac, post_dst_mac
• src_ip, dst_ip
• src_port, dst_port
• flow direction

This is OK for a lot of cases, but what if want to know how many traffic is switching a concrete ip? If we have the next table:

src	dst	direction	bytes	pkts
8.8.8.8	10.0.0.2	in	100	2
8.8.6.6	10.0.0.3	in	100	3
10.0.0.2	8.8.8.8	out	200	4

We need to execute this complex query to know that:

SELECT ip, SUM(bytes) FROM (SELECT src AS ip, bytes FROM flows UNION SELECT dst AS ip, bytes FROM flows) GROUP BY ip;

We could save the previous information this way:

lan_ip	wan_ip	direction	bytes	pkts
8.8.8.8	10.0.0.2	in	100	2
8.8.8.8	10.0.0.2	out	200	4
8.8.6.6	10.0.0.3	in	100	3

So query is pretty simple for LAN, for example:

SELECT SUM(bytes), lan FROM flows GROUP BY lan;

Situation is even worse if another WLC is exporting also STA_MAC/IP (IPFIX entities 365, 366), because now we have to add even more columns to table and queries.

Direction normalization

Enabling it

You have to use the parameter --normalize-directions when you start f2k.

Direction of the flow

The direction normalization assumes that flow direction is ingress if it goes from lan to wan, and egress if it goes from wan to lan. I.e., we are in the ISP point of view.

By default, we assume that exporter is in the LAN side of the router. On the contrary, we should indicate it in the observation id with "exporter_in_wan_side":true observation id parameter.

How does normalization works

LAN exporter

If exporter is in the LAN side of the connection (default), all ingress flows will contain LAN fields (interface, mac, ip, port) in src_ variants. on the other side, WAN fields will be the destination ones.

If the flow is egress, we are on the opposite situation: DST fields will be called LAN in the output, and SRC fields will be called WAN.

The direction field will be mantained: INGRESS ones will be INGRESS, and EGRESS flows will be EGRESS.

Note the special case of macs: we don't export WAN mac, and it is selected between src_mac and post_dst_mac. Also, it is called client_mac, not lan_mac.

WAN exporter

If we are using a WAN exporter, ingress flows will be marked as egress one, and they will be processed as if we are using the LAN exporter. Note that flow direction will be reversed in the JSON output.

No direction in the flow

If you are not exporting direction entity in the flow, you still can use normalization: You only need to define your home nets in the observation id:

"observations_id":{
  "default":{
    "home_nets": [
      { "network":"10.13.30.0/16", "network_name":"users" },
      { "network":"2001:0428:ce00:0000:0000:0000:0000:0000/48", "network_name":"users6" }
    ]
  }
}

And flow directions will be guessed. Note that this guessing take precedence over flow direction, and network names will be used in the json output.

Span probe

If you are using a span probe (i.e., the real traffic is being mirrored and you are reading it), client mac will not be post_dst_mac in case of WAN->LAN flows, but dst_mac. You can tell that an observation ID is being using a mirrored traffic with span_port property:

"observations_id":{
  "default":{
    "span_port": true
  }
}