Home - ns-3-dpdk-integration/ns-3-dpdk GitHub Wiki
Welcome to the ns-3-dpdk wiki!
Installation
Install DPDK
Is my NIC supported by DPDK?
Check Supported Devices.
Not supported? Use Virtual Machine instead
Install Oracle VM VirtualBox. Create a new VM and install Ubuntu on it. Open settings, create a network adapter with following configuration:
- Attached to: Bridged Adapter
- Name: The host network device you want to use
- In Advanced
- Adapter Type: Intel PRO/1000 MT Server (82545EM)
- Promiscuous Mode: Allow All
- Select Cable Connected
Then rest of the steps are same as follows.
Download the source
Head over to DPDK Downloads page to get the latest stable source. (We have used version dpdk-stable-18.02.2)
Install the source
Refer Installation for detailed instructions.
For a 64 bit linux machine with gcc, run:
make install T=x86_64-native-linuxapp-gcc DESTDIR=install
Build DPDK as a shared library
Goto the build directory. Build directory is the target(T) you used in previous make command.
In build directory, edit .config
file to change following line to compile DPDK as a shared library:
# Compile to share library
CONFIG_RTE_BUILD_SHARED_LIB=y
Then inside target directory, compile dpdk again:
make
Export DPDK Environment variables
Export following environment variables:
- RTE_SDK as the your DPDK source folder.
- RTE_TARGET as the build target directory.
For example:
export RTE_SDK=/home/hrishi/dpdk/dpdk-stable-18.02.2
export RTE_TARGET=x86_64-native-linuxapp-gcc
It is advisable that you export these variables in .bashrc
or similar for reusability.
Load DPDK Drivers to kernel
Execute following:
sudo modprobe uio_pci_generic
sudo modprobe uio
sudo insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko
sudo modprobe vfio-pci
These should be done every time you reboot your system.
Configure hugepages
Refer System Requirements for detailed instructions.
To allocate hugepages at runtime, edit /etc/default/grub
, and add the following to value of variable GRUB_CMDLINE_LINUX_DEFAULT
:
hugepages=256
We suggest minimum of number of 256
to run our applications. Then update the grub configurations using:
sudo update-grub
OR
sudo update-grub2
You will need to reboot your system in order to see these changes.
To check allocation of hugepages, run:
cat /proc/meminfo | grep HugePages
You will see the number of hugepages allocated, they should be equal to the number you used above.
Once the hugepage memory is reserved, to make the memory available for DPDK use, perform the following steps:
mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
The mount point can be made permanent across reboots, by adding the following line to the /etc/fstab
file:
nodev /mnt/huge hugetlbfs defaults 0 0
Install ns-3 with DPDK applications
Clone/Download the project
Clone by:
git clone https://github.com/ns-3-dpdk-integration/ns-3-dpdk
Or download the zip and extract.
Resolve dependencies and Configure ns-3
In folder ns-allinone-3.28.1
, run the command:
./build.py
This will resolve all the dependencies for compiling and building ns-3.
In folder ns-allinone-3.28.1/ns-3.28.1
, run:
./waf configure --build-profile=optimized --enable-examples --enable-tests --enable-sudo
--enable-sudo
is required for ns-3 network emulation
Build ns-3
In folder ns-allinone-3.28.1/ns-3.28.1
, execute:
./waf
Run ping application using DPDK
Edit src/fd-net-device/examples/fd-dpdk-emu-ping.cc
, and change following variables:
deviceName
: The device address of your ethernet. Uselspci
to find this. For example:0000:00:11.0
macClient
: (Optional) The mac address of your ethernet device.remote
: The IP address of the remote you want to ping.localIp
: The local IP address of ethernet device.localMask
: The mask that your subnet uses. Usenetstat -rn
to find this.gateway
: The gateway of your subnet. Usenetstat -rn
to find this.
Since DPDK requires root access, switch to root user using following:
sudo -E su
-E flag to use RTE_SDK and RTE_TARGET environment variables of user
Run the DPDK ping application:
./waf --run src/fd-net-device/examples/fd-dpdk-emu-ping
Run the OnOff Application
The following arguments are required for OnOff Application:
deviceName
: In case of DPDK mode, the device address of your ethernet (Uselspci
to find this), for example:0000:00:11.0
. Otherwise, the interface id which linux uses, for example:eno1
.client
: The IP address of client.server
: The IP address of server.mac-server
: (Optional) The mac address of server.data-rate
: The application rate of application.transportPort
: The transport protocol to use.dpdkMode
:true
to useDpdkNetDevice
.
You can run the OnOff Application using following command as a root user:
./waf --run 'src/fd-net-device/examples/fd-dpdk-emu-onoff \
--deviceName=0000:00:11.0 \
--client=192.168.43.44 \
--server=192.168.43.142 \
--mac-server=20:39:56:6c:47:1b \
--data-rate=10Mb/s \
--transportPort=Udp \
--dpdkMode=true'
On the remote device, iperf
server can be started using following command:
For TCP:
iperf -s -p 8000
For UDP:
iperf -s -p 8000 -u
Statistics
The OnOff Application creates following files which can be used for calculating various statistics:
fd-client-0-0.pcap
: Throughput and RTT using TCP analysis.inflight.plotme
: TCP bytes in flight values at regular interval (1ms).ping.plotme
: Ping RTT at regular interval (1ms).cwnd.plotme
: TCP CWND at values at regular interval (1ms).drops.plotme
: Packet drops at Physical Layer.
Additionally, iperf
can be used with -i
flag to generate throughput statistics.
To generate plots, refer dpdk-plot-scripts.
VTune Hotspot profiling
For installation check Installation Guide.
When OnOff Application is executed, it prints it's pid
and waits for user input. This pid
can be used to profile the program in VTune. To do so:
- Run VTune Amplifier.
- Select Configure Analysis.
- Select Hotspots in analysis type.
- Insert
pid
provided by OnOff Application. - Start analysis.
- Resume the OnOff Application.
VTune provides various profiling statistics. The important ones are:
- CPU Utilization (the measure of parallel efficiency of the program).
- Simultaneously utilized CPU cores.
- Hotspot analysis (total time spent in each function of the program).
netmap in ns-3
Similar to this project, netmap support in ns-3 was added by Pasquale Imputato and Stefano Avallone. Link https://github.com/pasquimp/ns-3-dev-git/tree/socis-next. The installation and usage can be found in their wiki.
OnOff Application (fd-emu-onoff.cc) in netmap can give above statistics. To configure the example to give these statistics, refer fd-dpdk-emu-onoff.cc for adding trace sources.