Test design - Paxterra/TestON GitHub Wiki

Test design :

A test in the TestON framework consists of three components.

Test script - Specifies flow of the test OpenSpeak formatted test :

CASE 1
    NAME "Verifying 'SSH protocol' based slicing" 

    STEP "Deleting flowspace using 'removeFlowSpace'"
    STORE ON FlowVisor1 DO removeFlowSpace USING "all" IN removeflowspace_result
    ASSERT removeflowspace_result EQUALS main.TRUE ONPASS "Removed FlowSpace Successfully" ONFAIL "Failed to remove FlowSpace" 

    STEP "Showing connected devices using 'listDevices'"
    STORE ON FlowVisor1 DO listDevices IN listdevices_result
    ASSERT listdevices_result EQUALS main.TRUE ONPASS "Listed devices Successfully" ONFAIL "Failed to list the devices"

    STEP "Verifying hosts reachability through ICMP traffic"
    STORE ON Mininet1 DO pingHost USING src AS 'h1',target AS 'h4' IN ping_result
    ASSERT ping_result EQUALS main.TRUE ONPASS "NO PACKET LOSS, HOST IS REACHABLE" ONFAIL "PACKET LOST, HOST IS NOT REACHABLE"  

    STEP "Showing the flowSpace USING 'listFlowSpace'"
    STORE ON FlowVisor1 DO listFlowSpace IN listflowspace_result
    STORE ON Mininet1 DO pingHost USING src AS 'h1',target AS 'h4' IN ping_result
    ASSERT main.TRUE EQUALS main.TRUE ONPASS "Listed FlowSpace" ONFAIL "Failed to list the FlowSpace"

    STEP "Adding FlowSpace to create the slice of the Network"
    INFO " Geeting the IP-Addresses of Hosts"
    STORE ON Mininet1 DO getIPAddress USING 'h1' IN h1_ipaddress
    STORE ON Mininet1 DO getIPAddress USING 'h4' IN h4_ipaddress 

    INFO " Geeting the MAC-Addresses of Hosts"  
    STORE ON Mininet1 DO getMacAddress USING 'h1' IN h1_macaddress
    STORE ON Mininet1 DO getMacAddress USING 'h4' IN h4_macaddress

    STORE ON FlowVisor1 DO addFlowSpace USING dl_src AS h1_macaddress,nw_dst AS h4_ipaddress IN addflowspace_result_1
    ASSERT addflowspace_result_1 EQUALS main.TRUE ONPASS "Added FlowSpace Successfully" ONFAIL "Failed to add FlowSpace" 
    
    INFO "Showing the flowSpace USING 'listFlowSpace'"
    STORE ON FlowVisor1 DO listFlowSpace IN listflowspace_result    

    STEP "Verifying hosts reachability through ICMP traffic and Connectivity through SSH service"
    STORE ON Mininet1 DO pingHost USING src AS 'h1',target AS 'h4' IN ping_result
    ASSERT ping_result EQUALS main.TRUE ONPASS "NO PACKET LOSS, HOST IS REACHABLE" ONFAIL "PACKET LOST, HOST IS NOT REACHABLE"  

    STORE ON Mininet1 DO verifySSH USING user_name AS CASE['destination_username'],ip_address AS 
    CASE['destination_host'], pwd AS CASE['destination_password'], port AS CASE['destination_port'] IN ssh_result
    ASSERT ssh_result EQUALS main.TRUE ONPASS "Failed to connect remote host through SSH" ONFAIL 
    "Remote host connected through SSH"

Equivalent Python Script :

class MininetSlicing :

     def __init__(self) :
        self.default = ''

     def CASE1(self,main) :

        main.case("Verifying 'SSH protocol' based slicing")

        main.step("Deleting flowspace using 'removeFlowSpace'")
        removeflowspace_result = main.FlowVisor1.removeFlowSpace("all")
        utilities.assert_equals(expect=main.TRUE,actual=removeflowspace_result,onpass="Removed FlowSpace Successfully",
        onfail="Failed to remove FlowSpace")

        main.step("Showing connected devices using 'listDevices'")
        listdevices_result = main.FlowVisor1.listDevices()
        utilities.assert_equals(expect=main.TRUE,actual=listdevices_result,onpass="Listed devices Successfully",
        onfail="Failed to list the devices")

        main.step("Verifying hosts reachability through ICMP traffic")
        ping_result = main.Mininet1.pingHost(src='h1',target='h4')
        utilities.assert_equals(expect=main.TRUE,actual=ping_result,onpass="NO PACKET LOSS, HOST IS REACHABLE",onfail="PACKET LOST, HOST IS NOT REACHABLE")

        main.step("Showing the flowSpace USING 'listFlowSpace'")
        listflowspace_result = main.FlowVisor1.listFlowSpace()
        ping_result = main.Mininet1.pingHost(src='h1',target='h4')


        main.step("Adding FlowSpace to create the slice of the Network")
        main.log.info(" Geeting the IP-Addresses of Hosts")
        h1_ipaddress = main.Mininet1.getIPAddress('h1')
        h4_ipaddress  = main.Mininet1.getIPAddress('h4')

        main.log.info(" Geeting the MAC-Addresses of Hosts"  )
        h1_macaddress = main.Mininet1.getMacAddress('h1')
        h4_macaddress = main.Mininet1.getMacAddress('h4')

        addflowspace_result_1 = main.FlowVisor1.addFlowSpace(dl_src=h1_macaddress,nw_dst=h4_ipaddress)
        utilities.assert_equals(expect=main.TRUE,actual=addflowspace_result_1,onpass="Added FlowSpace Successfully",
        onfail="Failed to add FlowSpace")

        main.log.info("Showing the flowSpace USING 'listFlowSpace'")
        listflowspace_result     = main.FlowVisor1.listFlowSpace()

        main.step("Verifying hosts reachability through ICMP traffic and Connectivity through SSH service")
        ping_result = main.Mininet1.pingHost(src='h1',target='h4')
        utilities.assert_equals(expect=main.TRUE,actual=ping_result,onpass="NO PACKET LOSS, HOST IS REACHABLE",
        onfail="PACKET LOST, HOST IS NOT REACHABLE")

        ssh_result = main.Mininet1.verifySSH(user_name=main.params['CASE1']['destination_username'],ip_address=main.params['CASE1']['destination_host'], 
        pwd=main.params['CASE1']['destination_password'], port=main.params['CASE1']['destination_port'])
        utilities.assert_equals(expect=main.TRUE,actual=ssh_result,onpass="Failed to connect remote host throgh SSH",
        onfail="Remote host connected throgh SSH ")

Params file - Specifies parameters to be used in the test

<PARAMS>
<testcases> 1</testcases>
<mail> [email protected]</mail>
<CASE1>

    <dl_type> 0x800 </dl_type> 
    <nw_proto> 6 </nw_proto>
    <nw_src> 10.0.0.2 </nw_src>
    <tp_dst> 22 </tp_dst>
    <slice> SSH </slice> 
    <permissions> 4 </permissions>
    
    <destination_host> 10.0.0.4 </destination_host>
    <destination_username> openflow </destination_username>
    <destination_password> openflow </destination_password>
    <destination_port> 22 </destination_port>
</CASE1>
</PARAMS>

Topology file - Specifies components to be used in the test and parameters for the components: User can create the topology by dragging the components into the Topology canvas and update the details for each component by clicking on the component.

Corresponding topology file :

<TOPOLOGY>
<COMPONENT>          
    <FlowVisor1>
        <host>192.168.56.101</host>
        <user> openflow </user>
        <fvadmin_pwd></fvadmin_pwd>
        <password>openflow</password>
        <type>FlowVisorDriver</type>
        <COMPONENTS>
        </COMPONENTS>
           
    </FlowVisor1>   
    
    <Mininet1>
        <host>192.168.56.101</host>
        <user> openflow</user>
        <password> openflow</password>
        <type>MininetCliDriver</type>
        <COMPONENTS>
            # Specify the Option for mininet
            <topo> single</topo>
            <topocount>3</topocount>
            <switch> ovsk </switch>
            <controller> remote </controller>
         </COMPONENTS>   
    </Mininet1>        
    
    <POX2>
        <host> 192.168.56.102 </host>
        <user>  openflow </user>
        <password> openflow </password>
        <type> PoxCliDriver </type>
        <test_target> 1 </test_target>
        <COMPONENTS>
            <pox_lib_location> /home/openflow/pox/  </pox_lib_location>
            <samples.of_tutorial></samples.of_tutorial>
        </COMPONENTS>
    </POX2>

           
</COMPONENT>

Tests are divided into test cases which are further divided into test steps. A test case is a sequence of steps meant to test a particular scenario as per the requirement. A test step is the smallest possible sub division of a test case capable of having its own individual assertion. The result of a test step and test case is determined by means of assertions. An assertion compares a value against an expected value using the specified operator. Currently, following assertions are supported.

equals
matches
greater_than
lesser_than

For every assertion, an equivalent not assertion is also supported. A test case is marked as no result if it does not have even a single step that performs an assertion.

The user is advised to follow below guidelines to ensure their tests are easy to debug and reusable.

Parameterise as much as possible. Always separate data from the test flow.
Parameterise intelligently. E.g., The admin IP of a switch will most likely stay the same for all tests. So make it a component parameter in the topology file. But the OSPF route number of an OSPF route can change from test to test. So make it a case/step parameter in the params file.
Make test steps as small as possible (but not any smaller) so that debugging is easy.
Assert at each test step.

Test design - Paxterra/TestON GitHub Wiki

Test design :

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️