04 Overlay Customization - fortinet-solutions-cse/sdwan-advpn-reference GitHub Wiki

Design Flavor

Dynamic BGP on Loopback (unified)

Jinja Orchestrator versions: 7.4

Basic Settings

The following two optional variables can be used to tweak this design flavor:

Variable Scope Description Values Default stateful
advpn2 project or profiles.options Config ready for ADVPN 2.0 true / false true yes
dynamic_bgp project or profiles.options Dynamic BGP (RR-less) true / false false yes

They can be configured as global variables, for example:

{% set advpn2 = true %}
{% set dynamic_bgp = true %}

The global values can be overriden on per-profile basis, for example:

{% set profiles = {
    'LegacySpoke': {
      'options': {
        'advpn2': false,
        'dynamic_bgp': false
      },
      'interfaces': [
        # ...
      ]
    }
  }
%}

When used for offline rendering, advpn2 = true generates the complete ADVPN 2.0 configuration (most of which is under SD-WAN section).
When used for FortiManager-based deployment, the SD-WAN configuration is handled by the FortiManager. But even in this case it is advised to configure advpn2 = true, in order to configure the best-practice foundation for ADVPN 2.0.

Note that when both these variables are set to false, the result is the "classic" BGP on Loopback configuration, similar to the one generated by the previous version of the Jinja Orchestrator (7.2).

Mixed Deployment

By default, BGP peering configuration on Hubs will be generated either for RR-based or for RR-less design, depending on the global dynamic_bgp variable described above.

However, mixed RR-based/RR-less deployment is also supported. In addition to configuring per-profile settings for the Spokes (with dynamic_bgp set to either false or true for different profiles), it is required to configure custom BGP peering on the Hubs. This is done in the Hubs definition, for example:

{% set hubs = {

    'site1-H1': {
      'peering': {
        'EDGE_RR': { 
          'dynamic_bgp': false,
          'lo_summary': '10.200.1.128/25'
        },
        'EDGE': {
          'dynamic_bgp': true,
          'lo_summary': lo_summary
        }
      },
      'overlays': {
         # ...
      },
      # ...
    }
  }
%}

Each peering must contain the dynamic_bgp value ("true" for RR-less, "false" for RR-based), as well as the respective loopback range for the Spokes.

Multi-VRF Support (Segmentation over Single Overlay)

Variable Scope Description Values Default stateful
pe_vrf project PE VRF for multi-VRF deployments <int> 1 yes
vrf_leak_summary project Internal subnet to use for inter-VRF links e.g. '10.200.255.0/24' 10.200.255.0/24 yes
vrf_rt_as project Base ASN to use for RD/RT values e.g. '65000' 65000 yes

This unified design flavor can be used both for single-VRF is multi-VRF deployments. Most of the necessary distinctions have been already mentioned on the previous pages. Here we summarize them for convenience. The table above also lists all the relevant optional variables that were not mentioned previously.

In order to prepare multi-VRF deployment, the following elements must be configured in the Project Template:

  • One or more VRFs must be defined in the region(s), as explained on this page. The VRFs listed under the region will be used to generate MP-BGP configuration for all devices.

    • The values of MP-BGP RD/RT are using ASN 65000 by default. This can be controlled by an optional global variable listed above.
  • PE VRF is set to 1 by default. This can be controlled by an optional global variable listed above.

  • CE VRFs must be assigned to the LAN interfaces in the device profiles, using the vrf parameter, as explained on this page. Any interface without an explicit VRF value will be assigned to the PE VRF.

    • CE VRFs for which Internet access is required must be configured with allow_dia parameter (in reality, the configuration is per interface, although the effect is for the entire CE VRF).

    • There is a default internal subnet which is used by default for all inter-VRF links. It can be overriden by an optional global variable listed above.

If a region does not contain any VRFs, it is considered to be a single-VRF deployment. As a result, multi-VRF encapsulation is not enabled on the IPSEC tunnels, MP-BGP configuration is not generated, inter-VRF links are not created and all the interfaces without an explicit VRF value are assigned to VRF=0.

Note that reconfiguration between single-VRF and multi-VRF deployment is supported. The Jinja Orchestrator also takes care of deleting the unused inter-VRF links, when switching from a multi-VRF to a single-VRF deployment (see also the description of the stateful allow_dia parameter on this page).

LAN and Loopback Summaries

Variable Scope Description Values Default stateful
lo_summary project or regions Loopback summary e.g. '10.200.1.0/24' -(!) yes
lan_summary project or regions or regions.vrfs LAN summary e.g. '10.0.0.0/16' - yes

LAN Summaries

If the LAN summary is configured, it will be advertised by the Hubs to all their Spokes, as well as to the other regions. The following rules apply:

  • Configuring the LAN summary is optional, but recommended (whenever the network addressing situation permits to do so).

  • In a multi-regional deployment, the LAN summaries must be configured per-region, in order to advertise them properly between the Hubs. For example:

    {% set regions = {
        'Region1': {
          'as': '65001',
          'lan_summary': '10.0.0.0/16',
          # ...
          'hubs': [ 'site1-H1', 'site1-H2' ]
        }
      }
    %}
  • In a multi-VRF deployment, the LAN summaries must be configured per-region per-VRF. For example:

    {% set regions = {
        'Region1': {
          'as': '65001',
          'vrfs': [
            {
              'id': 11,
              'lan_summary': '10.0.0.0/16'
            },
          ],
          'hubs': [ 'site1-H1', 'site1-H2' ]
        }
      }
    %}
  • In a single-region single-VRF deployment, the LAN summary can also be configured globally. For example:

    {% set lan_summary = '10.0.0.0/8' %}

As mentioned above, configuring the LAN summaries is always optional. The following rules summarize the routing behavior:

  • Within a region:

    • If the LAN summary is configured:

      • It is automatically advertised to all the Spokes.
      • In a multi-VRF deployment, this advertisement is done for each CE VRF.
    • If the LAN summary is not configured:

      • The user must make sure that the Spokes have a valid route to all the LAN destinations via the overlay tunnels.
      • In a single-VRF deployment, this can be achieved simply by adding a static default route via the entire SD-WAN zone.
        In the offline mode, this is automatically done by the Jinja Orchestrator.
        In FortiManager-based deployment, this must be done externally.
      • In a multi-VRF deployment, such a default route must be added to each CE VRF, which becomes a burden.
        The Jinja Orchestrator does not handle this.
  • Between regions:

    • If the LAN summary is configured:

      • It is automatically advertised over the Hub-to-Hub tunnels, aggregating the individual Spoke prefixes.
      • In a multi-VRF deployment, this advertisement is done for each CE VRF.
    • If the LAN summary is not configured:

      • All individual Spoke prefixes are advertised over the Hub-to-Hub tunnels, to guarantee inter-regional reachability.
      • These advertisements are not sent down to the Spokes of the remote region.
        The expectation is, again, that Spokes have a valid route (e.g. default route) to all the LAN destinations via the overlay tunnels.

The bottom line is: we recommend configuring LAN summaries whenever the network addressing permits that, to guarantee the most scalable routing design. At the same time, we support network environments where this summarization is not possible.

Loopback Summaries

The loopback summaries are always mandatory. They MUST be configured on the global level, for example:

{% set lo_summary = '10.200.0.0/14' %}

This global loopback summary is advertised by the Hubs to all their Spokes.

In a multi-regional deployment, there must be also a regional loopback summary, in order to advertise it properly between the Hubs. For example:

{% set regions = {
    'Region1': {
      'as': '65001',
      'lan_summary': '10.0.0.0/16',
      'lo_summary': '10.200.1.0/24',
      'hubs': [ 'site1-H1', 'site1-H2' ]
    }
  }
%}

Topology Options

Overlay Mapping and Naming Convention

Jinja Orchestrator versions: 7.2+

Variable Scope Description Values Default stateful
ol_type profiles.interfaces Overlay to connect to over this WAN interface <str> -(!) no
ul_name profiles.interfaces Local underlay name to add to the tunnel <str> - no
hub_name profiles.hubs Hub label to use in the tunnel name (7.4+) <str> H<hub_index> no

An overlay tunnel will be generated from each WAN-facing interface in the Edge (Spoke) device profile to each Hub serving the device region. The target overlay (to which the tunnel will connect) is defined by the ol_type parameter, which is mandatory for all WAN interfaces.

Therefore, it is expected that each of the Hubs is defined in the hubs structure, and each of them describes the overlay referred by the ol_type value.

The default naming convention for the generated tunnels on the Spoke is:

H<hub_index>_<ol_type>

Optionally, a local underlay transport name can be added. This happens automatically, if the ul_name parameter is defined. The naming convention then becomes:

<ul_name>-H<hub_index>_<ol_type>

Another option is to specify a meaningful Hub label, to be able to easily identify to which Hub a certain Spoke tunnel goes. The label will be used instead of the H<hub_index> on all tunnels towards that Hub. To achieve that, define the label using the hub_name parameter under the Hub definition in the hubs structure. The naming convention then becomes:

<hub_name>_<ol_type>
<ul_name>-<hub_name>_<ol_type>

Pay attention to the FOS limitations with regards to the maximum tunnel name length! Keep in mind that child tunnels (ADVPN shortcuts) add indexes to the names, making them even longer. It is recommended using at most 3-4 characters for ul_name, hub_name and ol_type.

On the Hub side, the Dial-Up IPSEC endpoints are generated for each WAN interface with the following naming convention:

EDGE_<ol_type>

For more advanced Spoke-to-Hub connectivity options, please refer to this page.

The naming convention for Hub-to-Hub tunnels between different regions (see [Multi-regional Support]) is the following (note that the optional hub_name parameter is respected):

<region_name>_H<hub_index>_<ol_type>
<region_name>_<hub_name>_<ol_type>

The naming convention for Hub-to-Hub tunnels within the same region (see [Hub-to-Hub Tunnels within the Region]) is the following (note that the hub_name is not used):

H<local_hub_index>H<remote_hub_index>_<ol_type>

Multi-regional Support

Jinja Orchestrator versions: 7.2+

Hub-to-Hub tunnels between regions are automatically generated when more than one region is defined. Hub-to-Hub EBGP peering is generated as well. By default, the Hub-to-Hub tunnels are built over all available overlays, if they are matching on both Hubs.

Jinja Orchestrator versions: 7.4

Variable Scope Description Values Default stateful
multireg project Enable multi-regional configuration true / false true no
hub2hub hubs.overlays Generate Hub-to-Hub tunnel true / false true no

Even when multiple regions are defined in the Project Template, multi-regional configuration can be skipped, by setting multireg = false:

{% set multireg = false %}

Hub-to-Hub tunnels over certain overlays can be skipped, by adding hub2hub = false parameter, for example:

{% set hubs = {
    'site1-H1': {
      'overlays': {
        'INET': {
          'wan_ip': west_h1_isp1,
          'network_id': '11',
          'hub2hub': false
        },
        'MPLS': {
          'wan_ip': west_h1_mpls,
          'network_id': '13'
        }
      }
      # ...
    }
    # ...
  }
%}

ADVPN

Jinja Orchestrator versions: 7.2+

Variable Scope Description Values Default stateful
intrareg_advpn project ADVPN within regions true / false true yes
multireg_advpn project ADVPN between regions true / false true (7.4+) yes
spoke2hub_advpn project or profiles.options ADVPN Spoke-to-Hub (7.4+) true / false false yes

NOTE: Starting from Jinja Orchestrator version 7.4, the multireg_advpn option is supported only when dynamic_bgp is enabled.

Dynamic BGP Options

Jinja Orchestrator versions: 7.4

Variable Scope Description Values Default stateful
regex_as project Regex covering ASNs for ADVPN between regions <str> - yes

This optional parameter may include a regular expression covering all the ASNs in the SD-WAN network, to allow Dynamic BGP peering between the regions (over the established ADVPN shortcuts).

When this parameter is not defined, the Jinja Orchestrator allows BGP peering between all ASNs of the currently defined regions. This is a good approach for most deployments. However, do note that if new regions are added to the project in the future, the configuration will have to be re-installed on all the existing Spokes, to include the new ASNs. On the contrary, using the above optional parameter can help avoiding this, because the regular expression can cover a range beyond the currently used ASNs, thus catering for future extension.

Example (includes ASNs from 65000 to 65009):

{% set regex_as = "6500." %}

Hub-to-Hub Tunnels within the Region

Jinja Orchestrator versions: 7.2+

Variable Scope Description Values Default stateful
intrareg_hub2hub project Hub-to-Hub tunnels within regions true / false false no

Hub-to-Hub tunnels within the region are optional and used only for Hub-to-Hub traffic (that is, only the LAN prefixes on the Hubs are advertised). When building these tunnels, the hub2hub parameter is also respected, allowing to skip certain overlays (see above).

Other Settings

IPSEC Authentication

Jinja Orchestrator versions: 7.2+

Variable Scope Description Values Default stateful
cert_auth project Enable cert-based authentication (otherwise: PSK) true / false true yes
edge_cert_template project Certificate Template name for Spokes Edge yes
hub_cert_template project Certificate Template name for Hubs Hub yes
psk project pre-shared key (PSK), when cert_auth = false S3cr3t! yes

When certificate-based authentication is enabled, the Certificate Template name is used as the certificate name in the IPSEC configuration.

Jinja Orchestrator versions: 7.4+

Variable Scope Description Values Default stateful
cert_auth_filter project Enable certificate filtering for IPSEC authentication true / false false yes
edge_cert_filter project User Peer (PKI User) name for Spokes "TheCA" yes
hub_cert_filter project User Peer (PKI User) name for Hubs (Spoke-facing Dial-Up) "TheCA" yes
hub_cert_filter_hub2hub project User Peer (PKI User) name for Hubs (Hub-to-Hub tunnels) "TheCA" yes

By default, any valid and trusted certificate will be accepted. The above parameters can be used to restrict this behavior.

The most common example is to restrict the list of trusted CAs, because the full list of CAs trusted by the FortiGate normally contains the default Fortinet CA, it might contain public Internet CAs and so on. For the SD-WAN overlay network, the user might want to restrict the list only to the CA actually used to issue the SD-WAN node certificates (it could be a built-in FortiManager CA or an external corporate CA). Optionally, the user may want to add other constraints too.

FOS supports certificate filtering using the config user peer object (known as "PKI User" object in the FortiManager) that is then referred under the IPSEC configuration. Jinja Orchestrator does not generate that object, leaving this task to the user. Thus, the user can include any list of trusted CA certificates, as well as any other constraints supported by the FOS. The Jinja Orchestrator will only add this object to the generated IPSEC configuration.

Optionally, you can specify a custom name of the user peer object, to match the one you create. Otherwise, the Jinja Orchestrator will assume the name "TheCA", as shown in the table above.

Redundant IPSEC Tunnels ("monitor" feature)

Jinja Orchestrator versions: 7.2+

Variable Scope Description Values Default stateful
backup_group profiles.interfaces Backup Group ID - yes
backup profiles.interfaces Interface is backup true / false false yes

We introduce a new abstract concept: a backup group.

The most general definition is as follows. A backup group contains several WAN interfaces, some of which are designated as "backup". Those designated as "backup" are backing up the rest of the group members. In practice, it means that the overlay tunnels over them will be up only when the rest of the group members (non-backup ones) are down. However, as we explain below, there are certain limitations to this general definition, imposed by the FOS capabilities.

The concept is implemented using the IPSEC "monitor" feature available in FOS, also known as "redundant VPN" (see here).

In FOS 7.2, a backup tunnel can monitor only a single "main" tunnel. Therefore, the backup group must currently include only one non-backup interface and one or more backup interfaces. The tunnels over each backup interface will monitor the tunnel over the non-backup interface within the same backup group. Note that in the future FOS releases we will be able to lift this limitation.

Let's consider the most common example:

{% set profiles = {
    'INET_LTE': {
      'interfaces': [
        {
          'name': 'wan1',
          'role': 'wan',
          'ol_type': 'INET',
          'ip': 'dhcp',
          'backup_group': 1,
          'dia': true
        },
        {
          'name': 'wan2',
          'role': 'wan',
          'ol_type': 'LTE',
          'ip': 'dhcp',
          'backup_group': 1,
          'backup': true,
          'dia': true
        },
        {
          'name': 'internal5',
          'role': 'lan',
          'ip': lan_ip
        }
      ]
    } 
  }
%}

There are two optional parameters added to the device profiles on per-interface level:

  • backup_group defines the ID of the backup group
  • backup (true/false) defines whether an interface is designated as backup or not

In the above example, "wan1" and "wan2" belong to the same backup group, in which "wan2" is designated as backup.

Let's assume that we have a Dual-Hub region, so that this example profile is expected to generate four overlay tunnels: H1_INET, H1_LTE, H2_INET and H2_LTE. The configuration above will result in the following redundant VPN configuration:

  • H1_LTE tunnel will monitor H1_INET
  • H2_LTE tunnel will monitor H2_INET

As can be seen, the IPSEC "monitor" feature is applied on per-Hub basis, within the configured backup group.

Overlay Stickiness

Jinja Orchestrator versions: 7.2+

Variable Scope Description Values Default stateful
overlay_stickiness project Enable overlay stickiness (legacy in 7.4+) true / false false (7.4+) yes
⚠️ **GitHub.com Fallback** ⚠️