onos 1.11.1 - levivic/zEcoTest GitHub Wiki

Building ONOS

ONOS (Open Network Operating System) is a software-defined networking (SDN) OS for service providers that has scalability, high availability, high performance and abstractions to make it easy to create apps and services. This page provides the steps to build ONOS version 1.11.1 on Linux on IBM Z for the following distribution:

  • Ubuntu (16.04)
  • RHEL (7.3)
  • SLES (12 SP2)

General Note:

  • Complete the build procedures by using a user role with standard permissions, unless otherwise specified.

  • A directory <source_root> will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it

Step 1: Build and Install the dependencies

1.1 Install Java 8

This article provides instructions to install two different types of JDK: IBM SDK and OpenJDK. Please choose either one of those to follow.

IBM SDK

  1. Install Java and its dependencies

  • Ubuntu:

    sudo apt-get update
sudo apt-get install git wget vim tar

    Download IBM Java 8 sdk binary from IBM Java 8 and follow the instructions as per given in the link.

  • RHEL:

    sudo yum update
sudo yum install git wget vim tar java-1.8.0-ibm java-1.8.0-ibm-devel

  • SLES:

    sudo zypper update
sudo zypper install git wget vim tar java-1_8_0-ibm java-1_8_0-ibm-devel
  1. Set environment variables and working directory

  • Ubuntu:

    export JAVA_HOME=<path-to-java-installation-directory>
export JDK_HOME=<path-to-java-installation-directory>
export PATH=$JAVA_HOME/bin:$PATH

  • RHEL

    export JAVA_HOME=/usr/lib/jvm/java-1.8.0-ibm
export JDK_HOME=/usr/lib/jvm/java-1.8.0-ibm

  • SLES

    export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-ibm-1.8.0
export JDK_HOME=/usr/lib64/jvm/java-1.8.0-ibm-1.8.0

OpenJDK

  1. Install OpenJDK and its dependencies

  • Ubuntu:

    sudo apt-get update
sudo apt-get install openjdk-8-jdk git vim tar wget

  • RHEL:

    sudo yum update
sudo yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel wget git vim tar

  • SLES:

    sudo zypper update
sudo zypper install java-1_8_0-openjdk java-1_8_0-openjdk-devel wget git vim tar
  1. Set environment variables and working directory

  • Ubuntu:

    export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-s390x
export JDK_HOME=/usr/lib/jvm/java-1.8.0-openjdk-s390x

  • RHEL

    export JAVA_HOME=/usr/lib/jvm/java-openjdk
export JDK_HOME=/usr/lib/jvm/java-openjdk

  • SLES

    export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0
export JDK_HOME=/usr/lib64/jvm/java-1.8.0-openjdk-1.8.0

1.2 Install Maven

1.3 Install Karaf

  1. Create two directories called /<source_root>/Downloads and /<source_root>/Applications. Clone the Karaf 3.0.8 into /<source_root>/Downloads and build it to /<source_root>/Applications.

    cd /<source_root>/; mkdir Downloads Applications
cd Downloads
git clone https://github.com/apache/karaf.git
cd karaf
git checkout karaf-3.0.8
export MAVEN_OPTS="-Xms1024m -Xmx4096m -Xss4m"
mvn clean install -DskipTests
tar xzvf assemblies/apache-karaf/target/apache-karaf-3.0.8.tar.gz -C ../../Applications/

  2. Open /<source_root>/Applications/apache-karaf-3.0.8/etc/org.apache.karaf.features.cfg file:

    vim /<source_root>/Applications/apache-karaf-3.0.8/etc/org.apache.karaf.features.cfg

    Locate the featuresRepositories and append the following line (will need a comma before appending the text to separate from the previous value):

    mvn:org.onosproject/onos-features/1.5.0-SNAPSHOT/xml/features

1.4 Build Protobuf and its Pre-compiled Protoc Artifact

1.4.1 Install dependencies

  • Ubuntu:

    sudo apt-get install autoconf automake libtool curl make g++ unzip

  • REHL:

    sudo yum install autoconf automake libtool curl make unzip gcc-c++

  • SLES:

    sudo zypper install autoconf automake libtool curl make unzip gcc-c++

1.4.2 Build Protobuf from source

  1. Run the following commands to prepare Protobuf

    cd /<source_root>/
git clone https://github.com/google/protobuf.git
cd protobuf
git checkout v3.2.0

  2. Prepare s390x patch file, i.e., /<source_root>/protobuf/s390x.patch, as following:

    Description: Add support for s390x architecture.
Based on port of Google V8 JavaScript engine to z Systems
Origin: https://github.com/ibmruntimes/v8z

--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -76,6 +76,7 @@ nobase_include_HEADERS =                                         \
   google/protobuf/stubs/atomicops_internals_generic_gcc.h        \
   google/protobuf/stubs/atomicops_internals_macosx.h             \
   google/protobuf/stubs/atomicops_internals_mips_gcc.h           \
+  google/protobuf/stubs/atomicops_internals_s390_gcc.h           \
   google/protobuf/stubs/atomicops_internals_solaris.h            \
   google/protobuf/stubs/atomicops_internals_tsan.h               \
   google/protobuf/stubs/atomicops_internals_x86_gcc.h            \
--- a/src/google/protobuf/stubs/atomicops.h
+++ b/src/google/protobuf/stubs/atomicops.h
@@ -224,6 +224,8 @@ Atomic64 Release_Load(volatile const Atomic64* ptr);
 #include <google/protobuf/stubs/atomicops_internals_generic_c11_atomic.h>
 #elif defined(GOOGLE_PROTOBUF_ARCH_PPC)
 #include <google/protobuf/stubs/atomicops_internals_ppc_gcc.h>
+#elif defined(GOOGLE_PROTOBUF_ARCH_S390)
+#include <google/protobuf/stubs/atomicops_internals_s390_gcc.h>
 #elif (((__GNUC__ == 4) && (__GNUC_MINOR__ >= 7)) || (__GNUC__ > 4))
 #include <google/protobuf/stubs/atomicops_internals_generic_gcc.h>
 #elif defined(__clang__)
--- a/src/google/protobuf/stubs/platform_macros.h
+++ b/src/google/protobuf/stubs/platform_macros.h
@@ -76,6 +76,13 @@
 #elif defined(__PPC__)
 #define GOOGLE_PROTOBUF_ARCH_PPC 1
 #define GOOGLE_PROTOBUF_ARCH_32_BIT 1
+#elif defined(__s390__) || defined(__s390x__)
+#define GOOGLE_PROTOBUF_ARCH_S390 1
+#if defined(__s390x__)
+#define GOOGLE_PROTOBUF_ARCH_64_BIT 1
+#else
+#define GOOGLE_PROTOBUF_ARCH_32_BIT 1
+#endif
 #elif defined(__GNUC__)
 # if (((__GNUC__ == 4) && (__GNUC_MINOR__ >= 7)) || (__GNUC__ > 4))
 // We fallback to the generic Clang/GCC >= 4.7 implementation in atomicops.h
--- /dev/null
+++ b/src/google/protobuf/stubs/atomicops_internals_s390_gcc.h
@@ -0,0 +1,148 @@
+// Copyright 2016 the V8 project authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+// This file is an internal atomic implementation, use atomicops.h instead.
+
+#ifndef GOOGLE_PROTOBUF_ATOMICOPS_INTERNALS_S390_H_
+#define GOOGLE_PROTOBUF_ATOMICOPS_INTERNALS_S390_H_
+
+namespace google {
+namespace protobuf {
+namespace internal {
+
+inline Atomic32 NoBarrier_CompareAndSwap(volatile Atomic32* ptr,
+                                         Atomic32 old_value,
+                                         Atomic32 new_value) {
+  return (__sync_val_compare_and_swap(ptr, old_value, new_value));
+}
+
+inline Atomic32 NoBarrier_AtomicExchange(volatile Atomic32* ptr,
+                                         Atomic32 new_value) {
+  Atomic32 old_value;
+  do {
+    old_value = *ptr;
+  } while (__sync_bool_compare_and_swap(ptr, old_value, new_value) == false);
+  return old_value;
+}
+
+inline Atomic32 NoBarrier_AtomicIncrement(volatile Atomic32* ptr,
+                                          Atomic32 increment) {
+  return Barrier_AtomicIncrement(ptr, increment);
+}
+
+inline Atomic32 Barrier_AtomicIncrement(volatile Atomic32* ptr,
+                                        Atomic32 increment) {
+  return __sync_add_and_fetch(ptr, increment);
+}
+
+inline Atomic32 Acquire_CompareAndSwap(volatile Atomic32* ptr,
+                                       Atomic32 old_value, Atomic32 new_value) {
+  return NoBarrier_CompareAndSwap(ptr, old_value, new_value);
+}
+
+inline Atomic32 Release_CompareAndSwap(volatile Atomic32* ptr,
+                                       Atomic32 old_value, Atomic32 new_value) {
+  return NoBarrier_CompareAndSwap(ptr, old_value, new_value);
+}
+
+inline void NoBarrier_Store(volatile Atomic32* ptr, Atomic32 value) {
+  *ptr = value;
+}
+
+inline void MemoryBarrier() { __sync_synchronize(); }
+
+inline void Acquire_Store(volatile Atomic32* ptr, Atomic32 value) {
+  *ptr = value;
+  MemoryBarrier();
+}
+
+inline void Release_Store(volatile Atomic32* ptr, Atomic32 value) {
+  MemoryBarrier();
+  *ptr = value;
+}
+
+inline Atomic32 NoBarrier_Load(volatile const Atomic32* ptr) { return *ptr; }
+
+inline Atomic32 Acquire_Load(volatile const Atomic32* ptr) {
+  Atomic32 value = *ptr;
+  MemoryBarrier();
+  return value;
+}
+
+inline Atomic32 Release_Load(volatile const Atomic32* ptr) {
+  MemoryBarrier();
+  return *ptr;
+}
+
+#ifdef GOOGLE_PROTOBUF_ARCH_64_BIT
+inline Atomic64 NoBarrier_CompareAndSwap(volatile Atomic64* ptr,
+                                         Atomic64 old_value,
+                                         Atomic64 new_value) {
+  return (__sync_val_compare_and_swap(ptr, old_value, new_value));
+}
+
+inline Atomic64 NoBarrier_AtomicExchange(volatile Atomic64* ptr,
+                                         Atomic64 new_value) {
+  Atomic64 old_value;
+  do {
+    old_value = *ptr;
+  } while (__sync_bool_compare_and_swap(ptr, old_value, new_value) == false);
+  return old_value;
+}
+
+inline Atomic64 NoBarrier_AtomicIncrement(volatile Atomic64* ptr,
+                                          Atomic64 increment) {
+  return Barrier_AtomicIncrement(ptr, increment);
+}
+
+inline Atomic64 Barrier_AtomicIncrement(volatile Atomic64* ptr,
+                                        Atomic64 increment) {
+  return __sync_add_and_fetch(ptr, increment);
+}
+
+
+inline Atomic64 Acquire_CompareAndSwap(volatile Atomic64* ptr,
+                                       Atomic64 old_value, Atomic64 new_value) {
+  return NoBarrier_CompareAndSwap(ptr, old_value, new_value);
+}
+
+inline Atomic64 Release_CompareAndSwap(volatile Atomic64* ptr,
+                                       Atomic64 old_value, Atomic64 new_value) {
+  return NoBarrier_CompareAndSwap(ptr, old_value, new_value);
+}
+
+inline void NoBarrier_Store(volatile Atomic64* ptr, Atomic64 value) {
+  *ptr = value;
+}
+
+inline void Acquire_Store(volatile Atomic64* ptr, Atomic64 value) {
+  *ptr = value;
+  MemoryBarrier();
+}
+
+inline void Release_Store(volatile Atomic64* ptr, Atomic64 value) {
+  MemoryBarrier();
+  *ptr = value;
+}
+
+inline Atomic64 NoBarrier_Load(volatile const Atomic64* ptr) { return *ptr; }
+
+inline Atomic64 Acquire_Load(volatile const Atomic64* ptr) {
+  Atomic64 value = *ptr;
+  MemoryBarrier();
+  return value;
+}
+
+inline Atomic64 Release_Load(volatile const Atomic64* ptr) {
+  MemoryBarrier();
+  return *ptr;
+}
+
+#endif
+
+}  // namespace internal
+}  // namespace protobuf
+}  // namespace google
+
+#endif  // GOOGLE_PROTOBUF_ATOMICOPS_INTERNALS_S390_H_

(RHEL only) Run the following command to install the missing dependency:

sudo yum install libstdc++-static

and note that this package is from the rhel-s390x-server-7-optional repo on RHEL, so you need to get the repo configured first.

  1. Then run the following commands to install Protobuf: 
    git apply s390x.patch
./autogen.sh
./configure
make
make check
sudo make install
sudo ldconfig

1.4.3 Build Protoc Artifact

  1. Make the following changes to ./protoc-artifacts/build-protoc.sh:

    @@ -71,8 +71,8 @@ checkArch ()
     if [ "$OS" == linux ](/levivic/zEcoTest/wiki/-"$OS"-==-linux-); then
       if [ "$ARCH" == x86_32 ](/levivic/zEcoTest/wiki/-"$ARCH"-==-x86_32-); then
         assertEq $format "elf32-i386" $LINENO
-      elif [ "$ARCH" == x86_64 ](/levivic/zEcoTest/wiki/-"$ARCH"-==-x86_64-); then
-        assertEq $format "elf64-x86-64" $LINENO
+      elif [ "$ARCH" == s390x ](/levivic/zEcoTest/wiki/-"$ARCH"-==-s390x-); then
+        assertEq $format "elf64-s390" $LINENO
       else
         fail "Unsupported arch: $ARCH"
       fi
@@ -114,8 +114,9 @@ checkDependencies ()
     dump_cmd='ldd '"$1"
     if [ "$ARCH" == x86_32 ](/levivic/zEcoTest/wiki/-"$ARCH"-==-x86_32-); then
       white_list="linux-gate\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld-linux\.so\.2"
-    elif [ "$ARCH" == x86_64 ](/levivic/zEcoTest/wiki/-"$ARCH"-==-x86_64-); then
-      white_list="linux-vdso\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld-linux-x86-64\.so\.2"
+    elif [ "$ARCH" == s390x ](/levivic/zEcoTest/wiki/-"$ARCH"-==-s390x-); then
+      white_list="linux-vdso64\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld64\.so\.1"
+
     fi
   elif [ "$OS" == osx ](/levivic/zEcoTest/wiki/-"$OS"-==-osx-); then
     dump_cmd='otool -L '"$1"' | fgrep dylib'
@@ -180,6 +181,8 @@ elif [ "$(uname)" == Linux* ](/levivic/zEcoTest/wiki/-"$(uname)"-==-Linux*-); then
       CXXFLAGS="$CXXFLAGS -m64"
     elif [ "$ARCH" == x86_32 ](/levivic/zEcoTest/wiki/-"$ARCH"-==-x86_32-); then
       CXXFLAGS="$CXXFLAGS -m32"
+    elif [ "$ARCH" == s390x ](/levivic/zEcoTest/wiki/-"$ARCH"-==-s390x-); then
+      CXXFLAGS="$CXXFLAGS -m64"
     else
       fail "Unsupported arch: $ARCH"
     fi

  2. Then run the following commands to build the artifact

    cd protoc-artifacts
./build-protoc.sh linux s390x protoc
cd target
mvn install:install-file -DgroupId=com.google.protobuf -DartifactId=protoc -Dversion=3.2.0 -Dclassifier=linux-s390_64 -Dpackaging=exe -Dfile=./protoc.exe

1.5 Build grpc-java

  1. Run the following commands to prepare grpc-java:

    cd /<source_root>/
git clone https://github.com/grpc/grpc-java.git
cd grpc-java
git checkout v1.2.0

  2. Make changes to ./compiler/build.gradle:

    @@ -48,6 +48,7 @@ model {
     }
     gcc(Gcc) {
       target("ppcle_64")
+      target("unknown")
     }
     clang(Clang) {
     }
@@ -63,6 +64,7 @@ model {
     ppcle_64 {
       architecture "ppcle_64"
     }
+    unknown {}
   }

   components {
@@ -71,6 +73,8 @@ model {
         // If arch is not within the defined platforms, we do not specify the
         // targetPlatform so that Gradle will choose what is appropriate.
         targetPlatform arch
+      } else {
+        targetPlatform "unknown"
       }
       baseName "$protocPluginBaseName"
     }

  3. Make changes to ./compiler/check-artifact.sh:

    @@ -59,6 +59,8 @@ checkArch ()
         assertEq $format "elf32-i386" $LINENO
       elif [ "$ARCH" == x86_64 ](/levivic/zEcoTest/wiki/-"$ARCH"-==-x86_64-); then
         assertEq $format "elf64-x86-64" $LINENO
+      elif [ "$ARCH" == s390x ](/levivic/zEcoTest/wiki/-"$ARCH"-==-s390x-); then
+        assertEq $format "elf64-s390" $LINENO
       else
         fail "Unsupported arch: $ARCH"
       fi
@@ -103,6 +105,8 @@ checkDependencies ()
       white_list="linux-gate\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld-linux\.so\.2"
     elif [ "$ARCH" == x86_64 ](/levivic/zEcoTest/wiki/-"$ARCH"-==-x86_64-); then
       white_list="linux-vdso\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld-linux-x86-64\.so\.2"
+    elif [ "$ARCH" == s390x ](/levivic/zEcoTest/wiki/-"$ARCH"-==-s390x-); then
+      white_list="linux-vdso64\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld64\.so\.1"
     fi
   elif [ "$OS" == osx ](/levivic/zEcoTest/wiki/-"$OS"-==-osx-); then
     dump_cmd='otool -L '"$1"' | fgrep dylib'

  4. Then run the following commands to install grpc-java

    cd compiler
../gradlew java_pluginExecutable
mvn install:install-file -DgroupId=io.grpc -DartifactId=protoc-gen-grpc-java -Dversion=1.2.0 -Dclassifier=linux-s390_64 -Dpackaging=exe -Dfile=build/exe/java_plugin/protoc-gen-grpc-java

Step 2: Build ONOS

  1. Run the following commands to prepare ONOS:

    cd /<source_root>/
git clone https://gerrit.onosproject.org/onos
cd onos
git checkout 1.11.1

  2. Make changes to ./tools/dev/bash_profile

    @@ -19,7 +19,7 @@ if [ -z "${JAVA_HOME}" ]; then
     fi
 fi

-export MAVEN=${MAVEN:-~/Applications/apache-maven-3.3.9}
+export MAVEN=${MAVEN:-/usr/share/maven}

 export KARAF_VERSION=${KARAF_VERSION:-3.0.8}
 export KARAF_ROOT=${KARAF_ROOT:-~/Applications/apache-karaf-$KARAF_VERSION}

  3. Make changes to ./protocols/netconf/ctl/src/main/java/org/onosproject/netconf/ctl/NetconfSessionImpl.java:

    @@ -129,7 +129,7 @@ public class NetconfSessionImpl implements NetconfSession {
             netconfConnection = new Connection(deviceInfo.ip().toString(), deviceInfo.port());

             try {
-                netconfConnection.connect(null, 1000 * connectTimeout, 1000 * connectTimeout);
+                netconfConnection.connect(null, 100000 * connectTimeout, 100000 * connectTimeout);
             } catch (IOException e) {
                 throw new NetconfException("Cannot open a connection with device " + deviceInfo, e);
             }

  4. Then build ONOS by running the following commands:

    export MAVEN_OPTS="-Xms1024m -Xmx4096m -Xss4m"
ulimit -c unlimited
mvn clean install
export PATH=$PATH:/<source_root>/onos/tools/test/bin:/<source_root>/onos/tools/build

Unit tests for ONOS are built using the JUnit framework. Unit tests are used to verify the correctness of implementations and are run as part of every full build of ONOS.

Reference

https://wiki.onosproject.org/display/ONOS/ONOS+from+Scratch