OSVDC Series: Network Services Build with Ubiquiti Networks - rharmonson/richtech GitHub Wiki

OSVDC Series: Network Services Build with Ubiquiti Networks

Revised: April 18, 2016

Article 5 of the Open Source Virtual Data Center Series

Network Service with Ubiquiti Networks

EdgeSwitch

Before beginning, obtain the Quick Start Guide, Administrator Manual, and current firmware from here:

https://www.ubnt.com/download/edgemax

The firmware is under the section "Firmware" and the Guide and Manual are found under "Documentation." Note there is an unobtrusive button "See More Documents" at the bottom of the page.

After downloading the firmware, power-on the EdgeSwitch without any connected devices. Once boot is completed, connect a workstation to eth0 port. Configure the workstation's interface as:

  • IP address: 192.168.1.100
  • Subnet mask: 255.255.255.0

Next, open a browser to 192.168.1.2 and enter the credentials:

  • user: ubnt
  • password: ubnt

Firmware Upgrade

After login, note the version of the installed firmware show in the upper-left corner next to the EdgeMAX logo. Assuming you don't have the most current firmware:

  1. Select the "System" tab or button found in the upper-right corner
  2. Select "Firmware"
  3. Select "Configuration and Upgrade"
  4. Select the icon with the up-arrow next to the inactive image within the "Firmware Upgrade" dialogue window
  5. Select "Browse" and locate the firmware file ending in 'stk'
  6. Select "Begin Transfer"
  7. Wait for transfer to complete for exiting dialogue or browser will stop the process. It takes a while but you will see the status change to "Transfer completed"
  8. Select "Close" within the "Firmware Upgrade" dialogue window
  9. Under "Images" note the "Next Active" and select the radio button for the new firmware version
  10. Select "Submit" button
  11. Select the "Status" tab and verify "Next Active" shows the firmware version you installed
  12. Select "System" tab, "Utilities," then "Restart Switch"
  13. Select "Restart"
  14. Select "OK" to begin restart
  15. login and verify the firmware version

Create Admin User

  1. Select "System" tab
  2. Select "Users"
  3. Select "Add" button
  4. Specify user name, password, and read/write access level.
  5. Select "Submit"
  6. Select "Log Out" upper-right corner and login with the new account.
  7. Select "System" tab
  8. Select "Users"
  9. Select the default user "ubnt" and either "Remove" or "Edit" to update password.
  10. Select "Save Configuration" in the upper-right corner
  11. Select "Save" button
  12. Select "OK"
  13. Select "Close"

System Connectivity

HTTPS

  1. Select "System" tab
  2. Select "Management Access"
  3. Select "HTTPS" tab
  4. Update the "HTTP Session Soft Time Out (Minutes)" to 15 minutes.
  5. Note for "Certificate Status" the value is "Absent"
  6. Selecting the cog button result with prompt "Do you want to generate the certificate?"
  7. Select "OK" to create a self-signed certificate
  8. After a brief wait, the "Certificate Status" will change to "Present"
  9. Update "HTTPS Admin Mode" from Disable to Enable
  10. Select "Submit"
  11. Note that there may be a brief pause, less than ten seconds, where HTTP is not responsive.

SSH

Assumption is you will be using SSH to manage the switch.

  1. Select "System" tab
  2. Select "Management Access"
  3. Select "SSH" tab
  4. Remove check mark from SSH Version 1.
  5. Update the "HTTP Session Soft Time Out (Minutes)" to 15 minutes.
  6. Select the cog button to generate both RSA and DSA keys
  7. Wait briefly for both to display status "Present"
  8. Update "SSH Admin Mode" from Disable to Enable
  9. Select "Submit"

Telnet

  1. Select "System" tab
  2. Select "Management Access"
  3. Select "Telnet" tab
  4. Update "Admin Mode" from Enable to Disable
  5. Select "Submit"

System Connectivity

  1. Select "System" tab
  2. Select "Disable" for "HTTP Admin Mode"
  3. Verify or select "Disable" for "Telnet Server Admin Mode"
  4. Verify or select "Enable" for "HTTPS Admin Mode"
  5. Verify or select "Enable" for "SSH Admin Mode"
  6. Select "Submit" button
  7. Select "Save Configuration" in the upper-right corner
  8. Wait ten seconds, then
  9. Update the browser from HTTP to HTTPS and login using the new credentials you created

System IPv4

  1. Select "System" tab
  2. Select "Connectivity"
  3. Select "IPv4" tab
  4. Network Configuration Protocol: "None"
  5. IP Address: 192.168.11.254
  6. Subnet Mask: 255.255.255.0
  7. Select "Submit" button
  8. Update workstation interface as show below
  9. Change browser IP address from 192.168.1.2 to 192.168.11.254 and login to continue
  10. Select "Save Configuration" in the upper-right corner
  11. Select "Save" button
  12. Select "OK"
  13. Select "Close"

Workstation Interface

  • IP address: 192.168.11.100
  • Subnet mask: 255.255.255.0
  • Gateway: 192.168.11.254

Switch Host Name

  1. Select "System" tab
  2. Select "Summary"
  3. Select "Description" tab
  4. Update "System Name" to whatever
  5. Select "Submit" button
  6. Select "Save Configuration" in the upper-right corner
  7. Select "Save" button
  8. Select "OK"
  9. Select "Close"

VLAN Wizard

Create VLANs

  1. Select "Switching" tab
  2. Select "VLAN"
  3. Select "Wizard"
  4. Note the field next to the "Add" button
  5. Enter 101 and select "Add"
  6. Repeat for VLANs 121, 131, 201, and 211
  7. Update the VLAN "Name" or use the default values
  8. Select "Submit" button

Assign VLANs

  1. For "default" select "E" (exclude) for ports 2 through and including 10 and ports 13 through and including 16
  2. For "VLAN0101" select "U" (untagged) for ports 2 through and including 10 and "T" (tagged) for ports 13 through and including 16
  3. For "VLAN0121" select "T" (tagged) for ports 13 through and including 16
  4. For "VLAN0131" select "T" (tagged) for ports 13 through and including 16
  5. For "VLAN0201" select "T" (tagged) for ports 13 through and including 16
  6. For "VLAN0211" select "T" (tagged) for ports 13 through and including 16
  7. Select "Submit" button

Results with

VLAN Wizard

Note

To review or see the port and VLAN settings made using the "Wizard", select Switching > VLAN > Port Configuration to review.

VLAN Routing

To avoid hair-pinning trusted traffic to a router interface, enable trusted VLANs for routing on the EdgeSwitch. Untrusted VLANS will be forwarded and filtered by the EdgeRouter.

Trusted VLANs are 101, 121, and 131 and untrusted VLANs are 201 and 211.

  1. Select "Switching" tab
  2. Select "VLAN"
  3. Select "Status"
  4. Select "VLAN ID" 101
  5. Select "Edit"
  6. VLAN Routing: Enable
  7. Select "Submit"
  8. Repeat the steps above for VLANs 121 and 131
  9. Done

PVID

  1. Select "Switching" tab
  2. Select "VLAN"
  3. Select "Port Summary" tab
  4. Select ports 0/2 to 0/10
  5. Select "Edit" button
  6. Port VLAN ID: 101
  7. Select "Submit" button

In addition, you may want to limit frames to tagged frames.

  1. Select ports 0/13 to 0/16
  2. Select "Edit" button
  3. Acceptable Frame Type: Only Tagged
  4. Select "Submit" button

Access VLAN ID

  1. Select "Switching" tab
  2. Select "VLAN"
  3. Select "Switchport Summary"
  4. Select interface 0/2 through 0/10
  5. Select "Access"
  6. Access VLAN ID: 101
  7. Select "Submit"

Routing IP Interface

Assign interface for connection to EdgeRouter

  1. Select "Routing"
  2. Select "IP"
  3. Select "Interface Configuration"
  4. Interface: 0/1
  5. Routing Mode: Enable
  6. IP Address Configuration Method: Manual
  7. IP Address: 192.168.69.2
  8. Subnet Mask: 255.255.255.252
  9. Select "Submit"

VLAN Routing IP Interfaces

  1. Select "Routing"
  2. Select "IP"
  3. Select "Interface Configuration"
  4. Interface: VLAN-101
  5. Routing Mode: Enable
  6. IP Address Configuration Method: Manual
  7. IP Address: 192.168.101.254
  8. Subnet Mask: 255.255.255.0
  9. Select "Submit"
  10. Complete the above steps for VLAN-121 using 192.168.121.254/255.255.255.0
  11. Complete the above steps for VLAN-131 using 192.168.131.254/255.255.255.0

Set Routing Mode

  1. Select "Routing"
  2. Select "IP"
  3. Select "Configuration"
  4. Routing Mode: Enable
  5. Select "Submit"

VLAN Routes

Test VLAN Routing

Test #1

Test untagged connection on VLAN 101 for ports 2 - 10

  1. Connect workstation to any port between 2 and 10
  2. Update network interface to use IP address/subnet mask, 192.168.101.100/255.255.255.0 and gateway 192.168.101.254
  3. From the cmd prompt or bash shell, ping 192.168.101.254
  4. Success!
  5. From the cmd prompt or bash shell, ping 192.168.69.2
  6. Expected outcome is success

Warning

Before proceeding, connect a device to one of the ports 13 - 16. Otherwise, the VLAN interface (gateway) will be "inactive" and attempts to ping x.x.x.254 will fail. You can review the current status of interfaces using Routing > IP > Interface Summary.

Test #2

Test tagged connection on VLAN 101 for ports 2 - 10

  1. Update the network interface to tag using vlan "101"
  2. From the cmd prompt or bash shell, ping 192.168.101.254
  3. Success!
  4. From the cmd prompt or bash shell, ping 192.168.69.2
  5. Expected outcome is success

Note

VLAN tagging on a workstation interface is generally supported by current network interface cards. Configuration will differ based on the operating system. For Linux use the interface configuration file that may be found /etc/sysconfig/network-scripts. For Windows select "Network and Internet" within the "Control Panel," properties for the connection, select "Configure" for the device, "Advanced" tab, then update the value for "VLAN" from 0 to 101.

Test #3

Test tagged connection on VLAN 101 for ports 13 - 16

  1. Connect workstation to any port between 13 and 16
  2. Repeat Test #2
  3. Expected outcome is success

Test #4

Test untagged connection on VLAN 101 for ports 13 - 16

  1. Update workstation interface to not use VLAN tagging.
  2. Repeat Test #1
  3. Expected outcome is failure

Test #5

Test tagged connection on VLAN 121 for ports 13 - 16

  1. Update network interface to use IP address/subnet mask, 192.168.121.100/255.255.255.0, gateway 192.168.121.254, and VLAN tag 121
  2. From the cmd prompt or bash shell, ping 192.168.121.254
  3. From the cmd prompt or bash shell, ping 192.168.101.254
  4. From the cmd prompt or bash shell, ping 192.168.131.254
  5. Success!
  6. From the cmd prompt or bash shell, ping 192.168.69.2
  7. Expected outcome is success
  8. Repeat test for VLAN 131 using IP address/subnet mask, 192.168.131.100/255.255.255.0, gateway 192.168.131.254, and VLAN tag 131

Note

VLANs 201 and 211 will fail until interfaces are created on the EdgeRouter.

Results with

ping test

Default Route

In preparation for configuring the EdgeRouter, create a default route for the EdgeSwitch to the EdgeRouter.

  1. Select "Routing" tab
  2. Select "Router"
  3. Select "Configured Routes" tab
  4. Select "Add" button
  5. Route Type: Default
  6. Next Hop IP Address: 192.168.69.1
  7. Select "Submit" button

Port Channeling (LAG)

The process to create a LAG for use with the Storage host and Compute hosts is a three step process.

Step 1

  1. Select "Basic" tab
  2. Select "Port Summary"
  3. Check-mark the ports to participate in LAGs; ports 0/5-0/10 & 0/13-0/16
  4. Select "Edit" button
  5. Verify the port is enabled for "Admin Mode" and "LACP Mode"
  6. Update and select "Submit" button or "Cancel if no changes are required

netsvcs-ports-5_10

netsvcs-ports-13_16

Step 2

  1. Under the "Basic" tab, select "Port Channel (LAG)" tab
  2. Select an unused interface; 3/1
  3. Select "Edit" button
  4. Port Channel Name: mgmt-storage
  5. Admin Mode: Enable
  6. STP Mode: Enable
  7. Static Mode: Disable
  8. Link Trap: Disable
  9. Load Balance: Source/Destination MAC, VLAN, Ethertype, Incoming Port
  10. Members: 0/5 & 0/6
  11. Select "Submit" button

Repeat the steps above to create:

  • 3/2, mgmt-node1, 0/7 & 0/8
  • 3/3, mgmt-node2, 0/9 & 0/10
  • 3/4, vm-node1, 0/13 & 0/14
  • 3/5, vm-node2, 0/15 & 0/16

netsvcs-portchannel_lag

Step 3

  1. Under the "Basic" tab, select "VLAN" tab
  2. For VLAN 1, "E"clude for port channels 3/1 through and including 3/5 on the far right
  3. For VLAN 101, select "U"ntagged for port channels 3/1, 3/2, and 3/3
  4. For VLANs 121, 131, 201, and 211, select "T"agged for port channels 3/4 and 3/5
  5. Select "Submit" button

netsvs-vlans_lags

Testing the LAG interfaces will wait until creation of the first virtual machine.

Save Configuration

  1. Select "Save Configuration"
  2. Select "Save"
  3. Select "OK"
  4. Select "Close"

Backup aka "Transfer"

Advisable to export the configuration as a restore point.

  1. Select "System" tab
  2. Select "Utilities"
  3. Select "Transfer"
  4. Select "Startup Configuration"
  5. Select "Begin Transfer"
  6. Specify save location then select "Close"

EdgeRouter

Before beginning, obtain the Quick Start Guide and current firmware from here:

https://www.ubnt.com/download/edgemax

The firmware is under the section "Firmware" and the Quick Start Guide is found under "Documentation." Note there is an unobtrusive button "See More Documents" at the bottom of the page.

After downloading the firmware, power-on the EdgeRouter without any connected devices. Once boot is completed, connect a workstation to eth0 port. Configure the workstation's interface as:

  • IP address: 192.168.1.100
  • Subnet mask: 255.255.255.0
  • Gateway: 192.168.1.1

Next, open a browser to 192.168.1.1 and enter the credentials:

  • user: ubnt
  • password: ubnt

Firmware Upgrade

After login, note the version of the installed firmware shown in the upper-left corner next to the EdgeMAX logo. Assuming you don't have the most current firmware:

  1. Select the "System" tab or button found in the lower-left corner
  2. Scroll towards the bottom to "Upgrade System Image"
  3. Select "Upload a file"
  4. Browse to the location you saved the firmware file
  5. Wait, briefly, for the file to upload
  6. Select reboot when prompted
  7. login and verify the firmware version

Wizard

To create a base configuration, lets use the Wizard. Select the Wizards tab found in the upper-right corner. Then select either "WAN+2LAN" or "WAN+2LAN2." I will be using the latter.

  1. Open a browser to 192.168.1.1 and enter credentials
  2. Internet port (eth0): accept the defaults
  3. LAN port (eth1): interface will be the next hope from the cluster's EdgeSwitch. My settings are 192.168.69.1/255.255.255.252
  4. Secondary LAN port (eth2): interface will be for an existing access switch. My settings are 192.168.22.254/255.255.255.0 with DHCP enabled
  5. User setup: advise creating a new admin user. It will replace user ubnt.
  6. Select "Apply"
  7. Read the dialogue and select "Apply Changes"
  8. Select "Reboot"
  9. Select "Yes, I'm sure"
  10. Move the workstation Ethernet cable to eth2
  11. Update the workstation interface to use DHCP

Basic Settings

  1. Connected to interface eth2 and using DHCP
  2. Open a browser to 192.168.22.254 and enter credentials
  3. Select the "System" tab or button found in the lower-left corner
  4. Update Host "Name," "Doman Name," "Time Zone," and "Name Server"
  5. Scroll down and select "Save" button found in the lower left corner

DMZ VLAN Interfaces

  1. Select "Dashboard" tab
  2. Select "Add Interface" button
  3. Select "Add VLAN"
  4. VLAN ID: 201
  5. Interface: eth1
  6. Description: dmz1
  7. Address: 192.168.201.254/24
  8. Select "Save"
  9. Note the creation of eth1.201
  10. Select "Add VLAN"
  11. VLAN ID: 211
  12. Interface: eth1
  13. Description: dmz2
  14. Address: 192.168.211.254/24
  15. Select "Save"
  16. Note the creation of eth1.211

Static Routes

Create static routes for the EdgeSwitch VLAN gateways.

  1. Select "Routing" tab
  2. Select "Add Static Route" button
  3. Select Route Type: Gateway
  4. Destination network: 192.168.101.0/24
  5. Next hop address: 192.168.69.2
  6. Select "Save" button
  7. Repeat the above for 192.168.121.0/24 and 192.168.131.0/24 with next hop 192.168.69.2.

Results with

routes

Backup Configuration

  1. Select the "System" tab or button found in the lower-left corner
  2. Back up config: select Download button
  3. Backup done!

Testing

Before testing, connect a cable from EdgeSwitch interface 1 to EdgeRouter interface 1. Next, I would recommend using a combination of the EdgeSwitch ping utility found at System > Utilities > Ping, the EdgeRouter's "Toolbox" found in the upper-right corner, and a workstation to test tagged and untagged traffic. One interesting characteristic of EdgeSwitch's is that pinging a VLAN interface, i.e. 4/0, will result in higher latency than crossing the interface to another VLAN. Here is an excerpt from one of my posts on the Ubiquiti's forums.

All traffic sent to a VLAN interface, e.g. 4/0, suffers a performance penalty for it is using CPU versus an ASIC due to being categorized as "management traffic." This is by design. This can be verified using ping or hping for listening ports on interface 4/0 which will result with 1+ ms times.

However, pinging another host through 4/0 will not be categorized as management traffic and will use the ASIC, thus does not suffer a performance penalty. Again, use ping or hping to verify.

Remember! On the EdgeSwitch, VLAN interfaces 4/0, 4/1, and 4/2 will not be active or in an up state until you connect to a physical port that uses the interface.

Next

Next article in the series is Storage Host Build with FreeNAS 9.3.