04 Overlay Customization - fortinet-solutions-cse/sdwan-advpn-reference GitHub Wiki
Jinja Orchestrator versions: 7.4
The following two optional variables can be used to tweak this design flavor:
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
advpn2 |
project or profiles.options | Config ready for ADVPN 2.0 | true / false | true | yes |
dynamic_bgp |
project or profiles.options | Dynamic BGP (RR-less) | true / false | false | yes |
They can be configured as global variables, for example:
{% set advpn2 = true %}
{% set dynamic_bgp = true %}
The global values can be overriden on per-profile basis, for example:
{% set profiles = {
'LegacySpoke': {
'options': {
'advpn2': false,
'dynamic_bgp': false
},
'interfaces': [
# ...
]
}
}
%}
When used for offline rendering, advpn2 = true
generates the complete ADVPN 2.0 configuration (most of which is under SD-WAN section).
When used for FortiManager-based deployment, the SD-WAN configuration is handled by the FortiManager. But even in this case it is advised to configure advpn2 = true
, in order to configure the best-practice foundation for ADVPN 2.0.
Note that when both these variables are set to false
, the result is the "classic" BGP on Loopback configuration, similar to the one generated
by the previous version of the Jinja Orchestrator (7.2).
By default, BGP peering configuration on Hubs will be generated either for RR-based or for RR-less design, depending on the
global dynamic_bgp
variable described above.
However, mixed RR-based/RR-less deployment is also supported. In addition to configuring per-profile settings for the Spokes (with dynamic_bgp
set to either false or true for different profiles), it is required to configure custom BGP peering on the Hubs. This is done in the Hubs definition, for example:
{% set hubs = {
'site1-H1': {
'peering': {
'EDGE_RR': {
'dynamic_bgp': false,
'lo_summary': '10.200.1.128/25'
},
'EDGE': {
'dynamic_bgp': true,
'lo_summary': lo_summary
}
},
'overlays': {
# ...
},
# ...
}
}
%}
Each peering must contain the dynamic_bgp
value ("true" for RR-less, "false" for RR-based), as well as the respective loopback range for the Spokes.
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
pe_vrf |
project | PE VRF for multi-VRF deployments | <int> | 1 | yes |
vrf_leak_summary |
project | Internal subnet to use for inter-VRF links | e.g. '10.200.255.0/24' | 10.200.255.0/24 | yes |
vrf_rt_as |
project | Base ASN to use for RD/RT values | e.g. '65000' | 65000 | yes |
This unified design flavor can be used both for single-VRF is multi-VRF deployments. Most of the necessary distinctions have been already mentioned on the previous pages. Here we summarize them for convenience. The table above also lists all the relevant optional variables that were not mentioned previously.
In order to prepare multi-VRF deployment, the following elements must be configured in the Project Template:
-
One or more VRFs must be defined in the region(s), as explained on this page. The VRFs listed under the region will be used to generate MP-BGP configuration for all devices.
- The values of MP-BGP RD/RT are using ASN 65000 by default. This can be controlled by an optional global variable listed above.
-
PE VRF is set to 1 by default. This can be controlled by an optional global variable listed above.
-
CE VRFs must be assigned to the LAN interfaces in the device profiles, using the
vrf
parameter, as explained on this page. Any interface without an explicit VRF value will be assigned to the PE VRF.-
CE VRFs for which Internet access is required must be configured with
allow_dia
parameter (in reality, the configuration is per interface, although the effect is for the entire CE VRF). -
There is a default internal subnet which is used by default for all inter-VRF links. It can be overriden by an optional global variable listed above.
-
If a region does not contain any VRFs, it is considered to be a single-VRF deployment. As a result, multi-VRF encapsulation is not enabled on the IPSEC tunnels, MP-BGP configuration is not generated, inter-VRF links are not created and all the interfaces without an explicit VRF value are assigned to VRF=0.
Note that reconfiguration between single-VRF and multi-VRF deployment is supported. The Jinja Orchestrator also takes care of deleting the
unused inter-VRF links, when switching from a multi-VRF to a single-VRF deployment (see also the description of the stateful allow_dia
parameter
on this page).
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
lo_summary |
project or regions | Loopback summary | e.g. '10.200.1.0/24' | -(!) | yes |
lan_summary |
project or regions or regions.vrfs | LAN summary | e.g. '10.0.0.0/16' | - | yes |
If the LAN summary is configured, it will be advertised by the Hubs to all their Spokes, as well as to the other regions. The following rules apply:
-
Configuring the LAN summary is optional, but recommended (whenever the network addressing situation permits to do so).
-
In a multi-regional deployment, the LAN summaries must be configured per-region, in order to advertise them properly between the Hubs. For example:
{% set regions = { 'Region1': { 'as': '65001', 'lan_summary': '10.0.0.0/16', # ... 'hubs': [ 'site1-H1', 'site1-H2' ] } } %}
-
In a multi-VRF deployment, the LAN summaries must be configured per-region per-VRF. For example:
{% set regions = { 'Region1': { 'as': '65001', 'vrfs': [ { 'id': 11, 'lan_summary': '10.0.0.0/16' }, ], 'hubs': [ 'site1-H1', 'site1-H2' ] } } %}
-
In a single-region single-VRF deployment, the LAN summary can also be configured globally. For example:
{% set lan_summary = '10.0.0.0/8' %}
As mentioned above, configuring the LAN summaries is always optional. The following rules summarize the routing behavior:
-
Within a region:
-
If the LAN summary is configured:
- It is automatically advertised to all the Spokes.
- In a multi-VRF deployment, this advertisement is done for each CE VRF.
-
If the LAN summary is not configured:
- The user must make sure that the Spokes have a valid route to all the LAN destinations via the overlay tunnels.
- In a single-VRF deployment, this can be achieved simply by adding a static default route via the entire SD-WAN zone.
In the offline mode, this is automatically done by the Jinja Orchestrator.
In FortiManager-based deployment, this must be done externally. - In a multi-VRF deployment, such a default route must be added to each CE VRF, which becomes a burden.
The Jinja Orchestrator does not handle this.
-
-
Between regions:
-
If the LAN summary is configured:
- It is automatically advertised over the Hub-to-Hub tunnels, aggregating the individual Spoke prefixes.
- In a multi-VRF deployment, this advertisement is done for each CE VRF.
-
If the LAN summary is not configured:
- All individual Spoke prefixes are advertised over the Hub-to-Hub tunnels, to guarantee inter-regional reachability.
- These advertisements are not sent down to the Spokes of the remote region.
The expectation is, again, that Spokes have a valid route (e.g. default route) to all the LAN destinations via the overlay tunnels.
-
The bottom line is: we recommend configuring LAN summaries whenever the network addressing permits that, to guarantee the most scalable routing design. At the same time, we support network environments where this summarization is not possible.
The loopback summaries are always mandatory. They MUST be configured on the global level, for example:
{% set lo_summary = '10.200.0.0/14' %}
This global loopback summary is advertised by the Hubs to all their Spokes.
In a multi-regional deployment, there must be also a regional loopback summary, in order to advertise it properly between the Hubs. For example:
{% set regions = {
'Region1': {
'as': '65001',
'lan_summary': '10.0.0.0/16',
'lo_summary': '10.200.1.0/24',
'hubs': [ 'site1-H1', 'site1-H2' ]
}
}
%}
Jinja Orchestrator versions: 7.2+
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
ol_type |
profiles.interfaces | Overlay to connect to over this WAN interface | <str> | -(!) | no |
ul_name |
profiles.interfaces | Local underlay name to add to the tunnel | <str> | - | no |
hub_name |
profiles.hubs | Hub label to use in the tunnel name (7.4+) | <str> | H<hub_index> | no |
An overlay tunnel will be generated from each WAN-facing interface in the Edge (Spoke) device profile to each Hub
serving the device region. The target overlay (to which the tunnel will connect) is defined by the
ol_type
parameter, which is mandatory for all WAN interfaces.
Therefore, it is expected that each of the Hubs is defined in the hubs
structure, and each of them describes the overlay
referred by the ol_type
value.
The default naming convention for the generated tunnels on the Spoke is:
H<hub_index>_<ol_type>
Optionally, a local underlay transport name can be added. This happens automatically, if the ul_name
parameter is defined. The naming convention then becomes:
<ul_name>-H<hub_index>_<ol_type>
Another option is to specify a meaningful Hub label, to be able to easily identify to which Hub a certain Spoke tunnel goes.
The label will be used instead of the H<hub_index>
on all tunnels towards that Hub. To achieve that, define the label using
the hub_name
parameter under the Hub definition in the hubs
structure. The naming convention then becomes:
<hub_name>_<ol_type>
<ul_name>-<hub_name>_<ol_type>
Pay attention to the FOS limitations with regards to the maximum tunnel name length! Keep in mind that child tunnels (ADVPN shortcuts) add indexes to the names, making them even longer. It is recommended using at most 3-4 characters for
ul_name
,hub_name
andol_type
.
On the Hub side, the Dial-Up IPSEC endpoints are generated for each WAN interface with the following naming convention:
EDGE_<ol_type>
For more advanced Spoke-to-Hub connectivity options, please refer to this page.
The naming convention for Hub-to-Hub tunnels between different regions (see [Multi-regional Support]) is the following
(note that the optional hub_name
parameter is respected):
<region_name>_H<hub_index>_<ol_type>
<region_name>_<hub_name>_<ol_type>
The naming convention for Hub-to-Hub tunnels within the same region (see [Hub-to-Hub Tunnels within the Region]) is the following (note that the hub_name
is not used):
H<local_hub_index>H<remote_hub_index>_<ol_type>
Jinja Orchestrator versions: 7.2+
Hub-to-Hub tunnels between regions are automatically generated when more than one region is defined. Hub-to-Hub EBGP peering is generated as well. By default, the Hub-to-Hub tunnels are built over all available overlays, if they are matching on both Hubs.
Jinja Orchestrator versions: 7.4
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
multireg |
project | Enable multi-regional configuration | true / false | true | no |
hub2hub |
hubs.overlays | Generate Hub-to-Hub tunnel | true / false | true | no |
Even when multiple regions are defined in the Project Template, multi-regional configuration can be skipped, by setting multireg = false
:
{% set multireg = false %}
Hub-to-Hub tunnels over certain overlays can be skipped, by adding hub2hub = false
parameter, for example:
{% set hubs = {
'site1-H1': {
'overlays': {
'INET': {
'wan_ip': west_h1_isp1,
'network_id': '11',
'hub2hub': false
},
'MPLS': {
'wan_ip': west_h1_mpls,
'network_id': '13'
}
}
# ...
}
# ...
}
%}
Jinja Orchestrator versions: 7.2+
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
intrareg_advpn |
project | ADVPN within regions | true / false | true | yes |
multireg_advpn |
project | ADVPN between regions | true / false | true (7.4+) | yes |
spoke2hub_advpn |
project or profiles.options | ADVPN Spoke-to-Hub (7.4+) | true / false | false | yes |
NOTE: Starting from Jinja Orchestrator version 7.4, the
multireg_advpn
option is supported only whendynamic_bgp
is enabled.
Jinja Orchestrator versions: 7.4
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
regex_as |
project | Regex covering ASNs for ADVPN between regions | <str> | - | yes |
This optional parameter may include a regular expression covering all the ASNs in the SD-WAN network, to allow Dynamic BGP peering between the regions (over the established ADVPN shortcuts).
When this parameter is not defined, the Jinja Orchestrator allows BGP peering between all ASNs of the currently defined regions. This is a good approach for most deployments. However, do note that if new regions are added to the project in the future, the configuration will have to be re-installed on all the existing Spokes, to include the new ASNs. On the contrary, using the above optional parameter can help avoiding this, because the regular expression can cover a range beyond the currently used ASNs, thus catering for future extension.
Example (includes ASNs from 65000 to 65009):
{% set regex_as = "6500." %}
Jinja Orchestrator versions: 7.2+
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
intrareg_hub2hub |
project | Hub-to-Hub tunnels within regions | true / false | false | no |
Hub-to-Hub tunnels within the region are optional and used only for Hub-to-Hub traffic (that is, only the LAN prefixes on the Hubs are advertised).
When building these tunnels, the hub2hub
parameter is also respected, allowing to skip certain overlays (see above).
Jinja Orchestrator versions: 7.2+
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
cert_auth |
project | Enable cert-based authentication (otherwise: PSK) | true / false | true | yes |
edge_cert_template |
project | Certificate Template name for Spokes | Edge | yes | |
hub_cert_template |
project | Certificate Template name for Hubs | Hub | yes | |
psk |
project | pre-shared key (PSK), when cert_auth = false | S3cr3t! | yes |
When certificate-based authentication is enabled, the Certificate Template name is used as the certificate name in the IPSEC configuration.
Jinja Orchestrator versions: 7.4+
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
cert_auth_filter |
project | Enable certificate filtering for IPSEC authentication | true / false | false | yes |
edge_cert_filter |
project | User Peer (PKI User) name for Spokes | "TheCA" | yes | |
hub_cert_filter |
project | User Peer (PKI User) name for Hubs (Spoke-facing Dial-Up) | "TheCA" | yes | |
hub_cert_filter_hub2hub |
project | User Peer (PKI User) name for Hubs (Hub-to-Hub tunnels) | "TheCA" | yes |
By default, any valid and trusted certificate will be accepted. The above parameters can be used to restrict this behavior.
The most common example is to restrict the list of trusted CAs, because the full list of CAs trusted by the FortiGate normally contains the default Fortinet CA, it might contain public Internet CAs and so on. For the SD-WAN overlay network, the user might want to restrict the list only to the CA actually used to issue the SD-WAN node certificates (it could be a built-in FortiManager CA or an external corporate CA). Optionally, the user may want to add other constraints too.
FOS supports certificate filtering using the config user peer
object (known as "PKI User" object in the FortiManager) that is then referred under the IPSEC configuration. Jinja Orchestrator does not generate that object, leaving this task to the user. Thus, the user can include
any list of trusted CA certificates, as well as any other constraints supported by the FOS. The Jinja Orchestrator will only add this object
to the generated IPSEC configuration.
Optionally, you can specify a custom name of the user peer object, to match the one you create. Otherwise, the Jinja Orchestrator will assume the name "TheCA", as shown in the table above.
Jinja Orchestrator versions: 7.2+
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
backup_group |
profiles.interfaces | Backup Group ID | - | yes | |
backup |
profiles.interfaces | Interface is backup | true / false | false | yes |
We introduce a new abstract concept: a backup group.
The most general definition is as follows. A backup group contains several WAN interfaces, some of which are designated as "backup". Those designated as "backup" are backing up the rest of the group members. In practice, it means that the overlay tunnels over them will be up only when the rest of the group members (non-backup ones) are down. However, as we explain below, there are certain limitations to this general definition, imposed by the FOS capabilities.
The concept is implemented using the IPSEC "monitor" feature available in FOS, also known as "redundant VPN" (see here).
In FOS 7.2, a backup tunnel can monitor only a single "main" tunnel. Therefore, the backup group must currently include only one non-backup interface and one or more backup interfaces. The tunnels over each backup interface will monitor the tunnel over the non-backup interface within the same backup group. Note that in the future FOS releases we will be able to lift this limitation.
Let's consider the most common example:
{% set profiles = {
'INET_LTE': {
'interfaces': [
{
'name': 'wan1',
'role': 'wan',
'ol_type': 'INET',
'ip': 'dhcp',
'backup_group': 1,
'dia': true
},
{
'name': 'wan2',
'role': 'wan',
'ol_type': 'LTE',
'ip': 'dhcp',
'backup_group': 1,
'backup': true,
'dia': true
},
{
'name': 'internal5',
'role': 'lan',
'ip': lan_ip
}
]
}
}
%}
There are two optional parameters added to the device profiles on per-interface level:
-
backup_group
defines the ID of the backup group -
backup
(true/false) defines whether an interface is designated as backup or not
In the above example, "wan1" and "wan2" belong to the same backup group, in which "wan2" is designated as backup.
Let's assume that we have a Dual-Hub region, so that this example profile is expected to generate four overlay tunnels: H1_INET, H1_LTE, H2_INET and H2_LTE. The configuration above will result in the following redundant VPN configuration:
- H1_LTE tunnel will monitor H1_INET
- H2_LTE tunnel will monitor H2_INET
As can be seen, the IPSEC "monitor" feature is applied on per-Hub basis, within the configured backup group.
Jinja Orchestrator versions: 7.2+
Variable | Scope | Description | Values | Default | stateful |
---|---|---|---|---|---|
overlay_stickiness |
project | Enable overlay stickiness (legacy in 7.4+) | true / false | false (7.4+) | yes |