Enable the uefi firmware on s2e for uefi academic testing research - shijunjing/s2e-env GitHub Wiki

When you "s2e sync" or "git pull s2e-env", you have to re-run below all build steps again (1 hour at least) to ensure everything is consistent, don't trust the s2e incremental build! Especial for the pip install --user . step

Useful info to understand S2E design: the https://github.com/S2E/docs/blob/master/src/DesignAndImplementation/KvmInterface.rst

S2E install on ubuntu 18.04:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ lsb_release -a
  No LSB modules are available.
  Distributor ID: Ubuntu
  Description:    Ubuntu 18.04.2 LTS
  Release:        18.04
  Codename:       bionic
 jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ cc --version
 cc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
 Copyright (C) 2017 Free Software Foundation, Inc.
 jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ gcc --version
 cc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
 Copyright (C) 2017 Free Software Foundation, Inc.

Install necessary OS dependency tools and libs:

  jshi19@jshi19-desktop:~/wksp_efi/s2e-env-fork$ sudo apt-get install \
   build-essential \
   cmake \
   wget \
   git \
   texinfo \
   flex \
   bison \
   python-dev \
   libdwarf-dev \
   libelf-dev \
   libboost-dev \
   zlib1g-dev \
   libjemalloc-dev \
   nasm \
   pkg-config \
   libmemcached-dev \
   libvdeplug-dev \
   libpq-dev \
   libc6-dev-i386 \
   libboost-system-dev \
   libboost-serialization-dev \
   libboost-regex-dev \
   libprotobuf-dev \
   protobuf-compiler \
   libbsd-dev \
   libglib2.0-dev \
   python-docutils \
   libguestfs-tools \
   genisoimage \
   python-pip \
   xz-utils \
   docker.io \
   p7zip-full \
   pxz \
   fuse \
   libhivex-bin \
   jigdo-file \
   libelf1 \
   libelf-dev

The following commands ask apt-get to install build dependencies for qemu:

   sudo apt-get build-dep qemu

If you are going to be analyzing Windows binaries, you will also need to install mingw to compile the guest tools:

   sudo apt-get install mingw-w64

Now, begin to clone and build it:

  git clone https://github.com/s2e/s2e-env.git
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ set http_proxy=http://******@child-prc.intel.com:***
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ set https_proxy=http://******@child-prc.intel.com:***
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ python -m pip --version
  pip 19.0.3 from /home/jshi19/.local/lib/python2.7/site-packages/pip (python 2.7)
  jshi19@jshi19-desktop:~/wksp_efi/s2e-env$ python -m pip --version
  pip 19.0.3 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)

Uninstall the previous installed s2e_env firstly, then install the latest one:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ pip uninstall s2e_env
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ pip install .

If you meet below pip install error, add the "--user" option:

  The directory '/home/jshi19/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
  jshi19@jshi19-desktop:~/wksp_efi/s2e-env$ pip install --user .

Then, run the setup test:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ python setup.py test
  running test
  running egg_info
  creating s2e_env.egg-info
  writing requirements to s2e_env.egg-info/requires.txt
  ... ...
  ----------------------------------------------------------------------
  Ran 17 tests in 0.618s
  OK

Ensure the /home/jshi19/.local/bin is in the PATH env variable, if not, please add it.

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ env
  ... ...
  PATH=/home/jshi19/.local/bin:/home/jshi19/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ whereis s2e
  s2e: /home/jshi19/.local/bin/s2e

Then create a new S2E environment

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ s2e init edk2_env
  ... ...
  SUCCESS: [s2e_env.utils.repos] Fetched testsuite
  SUCCESS: [init] Environment created in /home/jshi19/wksp_efi/s2e-env/edk2_env. You may wish to modify your environment's s2e.yaml config file. Source ``/home/jshi19/wksp_efi/s2e-env/edk2_env/s2e_activate`` to activate your environment. Then run ``s2e build`` to build S2E

Then set environment variables:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ source s2e_activate
  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$

s2e build need build old qemu, and the "capstone" submoudle in the old version qemu still use the old repository url: git://qemu.org/capstone.git, which has been deprecated and changed to url: https://github.com/qemu/capstone.git. So Need configure git to use https://github.com/qemu/ instead of git://git.qemu.org/

  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ git config --global url.https://github.com/qemu/.insteadOf git://git.qemu.org/
  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e build
  ... ... (build need more than 1 hour, LLVM/Clang, Qemu, KLEE, Z3, S2E, etc.)
  Install the project...
  -- Install configuration: ""
  -- Up-to-date: /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/guest-tools64/include/s2e/s2e.h
  -- Up-to-date: /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/guest-tools64/include/s2e/opcodes.h
  -- Installing: /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/guest-tools64/./s2ecmd.exe
  -- Installing: /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/guest-tools64/./s2eget.exe
  -- Installing: /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/guest-tools64/./s2eput.exe
  make[1]: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools64-win'
  cp /home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/llvm-release/lib/LLVMgold.so /home/jshi19/wksp_efi/s2e-env/edk2_env/install/lib
  make: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e'
  SUCCESS: [build] S2E built

  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e image_build
  /home/jshi19/.local/lib/python2.7/site-packages/s2e_env/__init__.py:39: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
    return yaml.load(f)
  /home/jshi19/.local/lib/python2.7/site-packages/s2e_env/command.py:230: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
    self._config = yaml.load(f)
  Available images:
   * all - Build all images
   * linux - Build all Linux images
   * windows - Build all Windows images
   * cgc_debian-9.2.1-i386 - Debian i386 image with CGC kernel and user-space packages
   * debian-9.2.1-i386 - Debian i386 image
   * debian-9.2.1-x86_64 - Debian x86_64 image
   * windows-10ent1703-x86_64 - Windows 10 Enterprise 1703 x86_64
   * windows-7sp1ent-x86_64 - Windows 7 Enterprise SP1 x86_64
   * windows-7sp1pro-i386 - Windows 7 Professional SP1 i386
   * windows-8.1ent-x86_64 - Windows 8.1 Enterprise x86_64
   * windows-xpsp3pro-i386 - Windows XP Professional SP3 i386
  Run ``s2e image_build <name>`` to build an image. Note that you must run ``s2e build`` **before** building an image

  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e image_build linux
  /home/jshi19/.local/lib/python2.7/site-packages/s2e_env/__init__.py:39: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
    return yaml.load(f)
  /home/jshi19/.local/lib/python2.7/site-packages/s2e_env/command.py:230: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
    self._config = yaml.load(f)
  INFO: [image_build] The following images will be built:
  INFO: [image_build]  * debian-9.2.1-i386
  INFO: [image_build]  * debian-9.2.1-x86_64
  INFO: [image_build]  * cgc_debian-9.2.1-i386
  ERROR: [image_build] KVM interface not found - check that /dev/kvm exists. Alternatively, you can disable KVM (-n option) or download pre-built images (-d option)

Tried "s2e image_build linux -n" later on and found the S2E linux build need long time and still can fails in the middle πŸ‘Ž. Tried to build in the KVM available NUC machine, it still fails because the docker is not correctly configured:

  [S2E:edk2] jshi19@jshi19-desktop:~/wksp_efi/s2e-env/edk2$ s2e image_build debian-9.2.1-i386 --gui
  INFO: [image_build] The following images will be built:
  INFO: [image_build]  * debian-9.2.1-i386
  INFO: [image_build] Kernel repository already exists in /home/jshi19/wksp_efi/s2e-env/edk2/source/s2e-linux-kernel
  make: Entering directory '/home/jshi19/wksp_efi/s2e-env/edk2/images'
  mkdir -p /home/jshi19/wksp_efi/s2e-env/edk2/images/.tmp-output
  [2019εΉ΄ 04月 05ζ—₯ ζ˜ŸζœŸδΊ” 22:13:01 CST] [/home/jshi19/wksp_efi/s2e-env/edk2/images/.stamps/linux-build-i386] Building docker image...
  mkdir -p /home/jshi19/wksp_efi/s2e-env/edk2/images/.stamps
  cd /home/jshi19/wksp_efi/s2e-env/edk2/source/guest-images//Linux/docker && docker build -t linux-build-i386 -f Dockerfile.i386 .
  ERRO[0000] failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: no such file or directory
  Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
  /home/jshi19/wksp_efi/s2e-env/edk2/source/guest-images//Makefile.linux:156: recipe for target '/home/jshi19/wksp_efi/s2e-env/edk2/images/.stamps/linux-build-i386' failed
  make: *** [/home/jshi19/wksp_efi/s2e-env/edk2/images/.stamps/linux-build-i386] Error 1
  make: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2/images'
  ERROR: [image_build]
    RAN: /usr/bin/make --directory=/home/jshi19/wksp_efi/s2e-env/edk2/images --file=/home/jshi19/wksp_efi/s2e-env/edk2/source/guest-images/Makefile -j 2 debian-9.2.1-i386
    STDOUT:
    STDERR:

So just download the pre-built image:

 [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e image_build linux -d --gui

Then follow the PoV Tutorials: https://github.com/S2E/docs/blob/master/src/Tutorials/PoV/index.rst :

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ cd build/s2e/guest-tools32
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32$ make
  [  0%] Built target s2ebios
  [  8%] Built target quicksort
  [ 17%] Built target maze
  [ 26%] Built target vulnerabilities
  [ 39%] Built target s2ecmd
  [ 47%] Built target s2eget
  [ 56%] Built target s2eput
  [ 69%] Built target models_test
  [ 78%] Built target cgccmd
  [100%] Built target s2e

 [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e new_project -i debian-9.2.1-i386 -n vuln-lin32-32 --enable-pov-generation build/s2e/guest-tools32/common/demos/vulnerabilities @@
 [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ cd /home/jshi19/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32
 [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ ./launch-s2e.sh
  • 20190405

Use the s2e built clang 3.9.1 and qemu 3.0.0 to build edk2 and run OVMF platform. Check the s2e clang and qemu version:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/install/bin$ ./clang --version
  clang version 3.9.1 (tags/RELEASE_391/final)
  Target: x86_64-unknown-linux-gnu
  Thread model: posix
  InstalledDir: /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/.
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/install/bin$ ./qemu-system-x86_64 --version
  QEMU emulator version 3.0.0 (-dirty)
  Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

Set CLANG38 toolchain path to s2e clang and build edk2 ovmf firmware:

  git clone https://github.com/tianocore/edk2.git
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ export CLANG38_BIN=~/wksp_efi/s2e-env/edk2_env/install/bin/
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ export WORKSPACE=$(pwd)
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ export EDK_TOOLS_PATH=$(pwd)/BaseTools
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ export CONF_PATH=$(pwd)/Conf
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ rm Conf/tools_def.txt
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ rm Conf/build_rule.txt
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ rm Conf/target.txt
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ source edksetup.sh
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ make -C BaseTools/ clean
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ make -C BaseTools/
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ build -p OvmfPkg/OvmfPkgIa32X64.dsc -t CLANG38 -a IA32 -a X64 -b NOOPT -DDEBUG_ON_SERIAL_PORT
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ build -p OvmfPkg/OvmfPkgIa32.dsc -t CLANG38 -a IA32 -b NOOPT -DDEBUG_ON_SERIAL_PORT

Boot the edk2 ovmf firmware on s2e qemu:

  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ sudo ~/wksp_efi/s2e-env/edk2_env/install/bin/qemu-system-x86_64 -m 5120 -smp 1 -bios ~/wksp_efi/edk2/Build/Ovmf3264/NOOPT_CLANG38/FV/OVMF.fd -global e1000.romfile="" -machine q35 -serial mon:stdio -display none -net none
  jshi19@ub2-uefi-b01:~/wksp_efi/edk2$ sudo ~/wksp_efi/s2e-env/edk2_env/install/bin/qemu-system-i386 -m 5120 -smp 1 -bios ~/wksp_efi/edk2/Build/OvmfIa32/NOOPT_CLANG38/FV/OVMF.fd -global e1000.romfile="" -machine q35 -serial mon:stdio -display none -net none
  • 20190407

Change the vuln-lin32-32 project LD_PRELOAD as below in jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32/launch-s2e.sh

  LD_PRELOAD=$LIBS2E $QEMU $DRIVE \
      -k en-us $GRAPHICS -monitor null -m 256M -enable-kvm \
      -serial file:serial.txt -net none \
      -bios ~/wksp_efi/edk2/Build/OvmfIa32/NOOPT_CLANG38/FV/OVMF.fd

Change the phys_dirty_size as below from 17MB to 128MB in ~/wksp_efi/s2e-env/edk2_env/source/s2e/libcpu/src/exec-ram.c function init_ram_size()

  length += 17 * 1024 * 1024; --> length += 128 * 1024 * 1024;

Rebuild the S2E

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ rm build/s2e/stamps/libs2e-release-make
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ rm build/s2e/stamps/libs2e-release-install
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ source s2e_activate
  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e update
  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e build
  ... ...
  SUCCESS: [build] S2E built

Run the vuln-lin32-32 project again:

  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ ./launch-s2e.sh
  Starting libs2e...
  Opening /dev/kvm
  S2E: output directory = "./s2e-out-8"
  Revision: 70bce7b6b7d60b8af9fec3e223ea32096e6e2687
  Config date: Thu Apr  4 15:33:45 CST 2019

  lua: attempting to load models.lua
  [Z3] Initializing
  Using module /home/jshi19/wksp_efi/s2e-env/edk2_env/install/share/libs2e/op_helper.bc.i386
  Current data layout: e-m:e-i64:64-f80:128-n8:16:32:64-S128
  Current target triple:
  BEGIN searcher description
  ... ...

  BEGIN searcher description
  DFSSearcher
  END searcher description
  0 [State 0] Created initial state
  Adding CPU (addr = 0x558ac37cead0, size = 0x36c60)
  Initializing periodic timer
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-nopiodelay [bit 1]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvmclock [bit 3]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-asyncpf [bit 4]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-steal-time [bit 5]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-pv-eoi [bit 6]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvmclock-stable-bit [bit 24]
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x4000000, hostAddr = 0x7f468fe00000, isSharedConcrete=0, name=pc.ram)
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x400000, hostAddr = 0x7f4699400000, isSharedConcrete=1, name=pc.bios)
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x20000, hostAddr = 0x7f4698c00000, isSharedConcrete=1, name=pc.rom)
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x1000000, hostAddr = 0x7f468ec00000, isSharedConcrete=1, name=vga.vram)
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x10000, hostAddr = 0x7f468ea00000, isSharedConcrete=1, name=vga.rom)
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x20000, hostAddr = 0x7f468e600000, isSharedConcrete=1, name=/rom@etc/acpi/tables)
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x1000, hostAddr = 0x7f468e400000, isSharedConcrete=1, name=/rom@etc/table-loader)
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x1000, hostAddr = 0x7f468e200000, isSharedConcrete=1, name=/rom@etc/acpi/rsdp)

Open another console and cat the serial.txt file and check the ovmf.fd bios has booted into uefi shell:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ cat serial.txt
  ... ...
  UEFI Interactive Shell v2.287477C2-69C7-11D2-8E39-00A0C969723B 30F1594
  EDK IIlProtocolInterface: 752F3136-4E16-4FDC-A22A-E5F46812F4CA 30E4E90
  UEFI v2.70 (EDK II, 0x00010000)008-7F9B-4F30-87AC-60C9FEF5DA4E 280604C
  Mapping table
       BLK0: Alias(s):
            PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)
       BLK3: Alias(s):
            PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)
       BLK1: Alias(s):
            PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(1,MBR,0x33F7EF7F,0x800,0xF3800)
       BLK2: Alias(s):
            PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)/HD(2,MBR,0x33F7EF7F,0xF4000,0x70B800)

  Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
  Shell>

Kill the qemu process to close the project:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ sudo ps -a
  ... ...
   2796 tty1     00:00:00 ibus-engine-sim
   2880 pts/1    00:00:00 launch-s2e.sh
   2881 pts/1    00:00:23 qemu-system-i38
   3033 pts/0    00:00:00 sudo
   3034 pts/0    00:00:00 ps
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ kill 2880
  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ kill 2881

Next task will be to enable the Qemu graphic GUI console for input and output:

Configure the Windows PuTTY SSH tool to let the remote Linux forward X11, then you can enable the Qemu graphic GUI console for input and output.

  1. Install the Xming in the windows:

https://github.com/shijunjing/WikiDoc/blob/master/Xming/About.jpg

  1. Configure the PuTTY Connection to let the remote Linux forward X11:

https://github.com/shijunjing/WikiDoc/blob/master/Xming/X11_forwarding_PuttyConfig.jpg

Comment out the GRAPHICS macro in ~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32/launch-s2e.sh

   # Comment this out to enable QEMU GUI
   #GRAPHICS=-nographic

Run the vuln-lin32-32 project again:

  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ ./launch-s2e.sh

Then, you can see Qemu graphic console and the Ovmf firmware boot into shell later on. Hurrah! All input and output in the graphic console look work well: https://github.com/shijunjing/WikiDoc/blob/master/S2E/OvmfBootIntoShell.jpg And you can enter setup as well: https://github.com/shijunjing/WikiDoc/blob/master/S2E/OvmfBootIntoSetup.jpg

Also tried the -machine q35 option with 1GB memory, it works as well, but the speed is slower than default i440fx machine.

  LD_PRELOAD=$LIBS2E $QEMU $DRIVE \
      -k en-us $GRAPHICS -monitor null -m 1024M -enable-kvm \
      -serial file:serial.txt -net none \
      -machine q35 \
      -bios ~/wksp_efi/edk2/Build/OvmfIa32/NOOPT_CLANG38/FV/OVMF.fd

If need to kill the S2E process (stuck in deadloop):

  jshi19@ub2-uefi-b01:~$ sudo ps -a
  jshi19@ub2-uefi-b01:~$ kill -9 6056

Current problem is the s2e engine cannot do the state switching, and report "error while loading state for instance 0x0 of device 'globalstate'". Need continue to debug.

  ... ...
  27 [State 0] BaseInstructions: Inserted symbolic data @0xfe4af0c of size 0x8: Lba pc=0xea95a22
  27 [State 0] Forking state 0 at pc = 0xea958fa at pagedir = 0x0
      state 0
      state 1
  BEGIN searcher description
  DFSSearcher
  END searcher description
  27 [State 0] Forking state 0 at pc = 0xea958fa at pagedir = 0x0
      state 0
      state 2
  KLEE: WARNING: silently concretizing (instruction: helper_outb:   %17 = call i64 @tcg_llvm_trace_port_access(i64 %15, i64 %16, i32 8, i32 1)) (reason: Symbolic I/O port value) expression (Extract w8 0 (Or w32 (And w32 (ZExt w32 (Extract w8 0 (Or w32 (ZExt w32 (Extract w8 0 (Or w32 (And w32 (ZExt w32 (Extract w8 0 (And w32 (Or w32 (ReadLSB w32 0x3 v0_Lba_0)
                                                                                                                                                    0xe0)
                                                                                                                                            0xff)))
                                                                                                           0xff)
                                                                                                  0xf100000)))
                                                                  0xe0)))
                                  0xff)
                         0xf1a5000)) to value 0xe0
  27 [State 0] BaseInstructions: Killing state 0
  27 [State 0] Terminating state early: State was terminated by opcode
              message: "program terminated"
              status: 0x0
  27 [State 0] Switching from state 0 to state 1
  qemu-system-i386: invalid parameter value:
  qemu-system-i386: error while loading state for instance 0x0 of device 'globalstate'
  qemu-system-i386: Could not restore device state
  ./launch-s2e.sh: line 75: 19472 Aborted                 (core dumped) LD_PRELOAD=$LIBS2E $QEMU $DRIVE -k en-us $GRAPHICS -monitor null -m 256M -enable-kvm -serial file:serial.txt -net none -bios ~/wksp_efi/edk2/Build/OvmfIa32/NOOPT_CLANG38/FV/OVMF.fd
  • 20190408

Continue to debug the problem that s2e engine cannot do the state switching with edk2 firmware and report "loading state error".

Tried the use the -pflash to replace the -bios option. Usage of a flash memory device with kvm requires KVM_CAP_READONLY_MEM, and Qemu will abort if a flash device is used with an older kvm which does not support this capability. The s2e kvm driver implementation happen to not support KVM_CAP_READONLY_MEM capability and abort as below.

  [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ LD_PRELOAD=/home/jshi19/wksp_efi/s2e-env/edk2_env/install/share/libs2e/libs2e-i386-s2e.so /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/qemu-system-i386 -drive file=/home/jshi19/wksp_efi/s2e-env/edk2_env/images/debian-9.2.1-i386/image.raw.s2e,format=s2e,cache=writeback -k en-us -monitor null -m 64M -enable-kvm -serial file:serial.txt -net none -pflash ~/wksp_efi/edk2/Build/OvmfIa32/NOOPT_CLANG38/FV/OVMF.fd
  ... ...
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-nopiodelay [bit 1]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvmclock [bit 3]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-asyncpf [bit 4]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-steal-time [bit 5]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvm-pv-eoi [bit 6]
  qemu-system-i386: warning: host doesn't support requested feature: CPUID.40000001H:EAX.kvmclock-stable-bit [bit 24]
  Adding memory block (startAddr = 0xffffffffffffffff, size = 0x10000000, hostAddr = 0x7f7d1de00000, isSharedConcrete=0, name=pc.ram)
  qemu: pflash with kvm requires KVM readonly memory support
  Terminating node id 0 (instance slot 0)
  Engine terminated.

Need to check whether the Qemu -savevm and -loadvm functions work with -bios edk2 OVMF firmware. If it works, the "loading state error" might not related to the -pflash and -bios option.

  • 20190409

s2e.so seems wrap the stdio and the qemu -monitor stdio cannot work, need to try access the monitor console via Telnet(https://stackoverflow.com/questions/14165158/how-to-switch-to-qemu-monitor-console-when-running-with-curses)

Start QEMU:

  qemu-system-i386 -curses -monitor telnet:127.0.0.1:1234,server,nowait

Then, to access the monitor console, type this in another terminal:

  telnet 127.0.0.1 1234
  • 20190410

Tried the "-monitor telnet:127.0.0.1:1234,server,nowait" on s2e and it works.

Change the -monitor as below:

  LD_PRELOAD=$LIBS2E $QEMU $DRIVE \
      -k en-us $GRAPHICS -monitor telnet:127.0.0.1:1234,server,nowait -m 64M -enable-kvm \
      -serial file:serial.txt -net none \
      -bios ~/wksp_efi/edk2/Build/OvmfIa32/NOOPT_CLANG38/FV/OVMF.fd

Then lauch the s2e:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ ./launch-s2e.sh

In another console, use telnet connect the qemu monitor at 127.0.0.1 1234:

  jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/projects/vuln-lin32-32$ telnet 127.0.0.1 1234
  Trying 127.0.0.1...
  Connected to 127.0.0.1.
  Escape character is '^]'.
  QEMU 3.0.0 monitor - type 'help' for more information
  (qemu) stop
  (qemu)
  • 20190411

  1. For the problem that s2e engine cannot do the state switching with edk2 firmware, has asked it in s2e google forum: https://groups.google.com/forum/#!topic/s2e-dev/39s1tQwAd6c

  2. Update and rebuild s2e will meet the clang-3.9 "-lelf" linker library failure:

     jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ git pull origin master
 jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env$ cd edk2_env/
 jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e update
 jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e build
 ... ...
 [ 96%] Linking C shared library s2e.so
 /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libelf.so when searching for -lelf
 /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libelf.a when searching for -lelf
 /usr/bin/ld: cannot find -lelf
 clang-3.9: error: linker command failed with exit code 1 (use -v to see invocation)
 linux/s2e.so/CMakeFiles/s2e.dir/build.make:276: recipe for target 'linux/s2e.so/s2e.so' failed
 make[3]: *** [linux/s2e.so/s2e.so] Error 1
 make[3]: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32'
 CMakeFiles/Makefile2:581: recipe for target 'linux/s2e.so/CMakeFiles/s2e.dir/all' failed
 make[2]: *** [linux/s2e.so/CMakeFiles/s2e.dir/all] Error 2
 make[2]: *** Waiting for unfinished jobs....
 [100%] Linking CXX executable s2ecmd
 make[3]: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32'
 [100%] Built target s2ecmd
 make[2]: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32'
 Makefile:129: recipe for target 'all' failed
 make[1]: *** [all] Error 2
 make[1]: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32'
 /home/jshi19/wksp_efi/s2e-env/edk2_env/source/s2e/Makefile:766: recipe for target 'stamps/guest-tools32-install' fa
 make: *** [stamps/guest-tools32-install] Error 2
 make: Leaving directory '/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e'
 ERROR: [build]
   RAN: /usr/bin/make --directory=/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e --file=/home/jshi19/wksp_efi/s2e-dk2_env/source/s2e/Makefile install
   STDOUT:
   STDERR:

Below is how I debug above issue and figure out a workaroud:

  1. Add SHELL='sh -x' in the make to show the complete build command in make:

     [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ /usr/bin/make SHELL='sh -x' --directory=/home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e --file=/home/jshi19/wksp_efi/s2e-env/edk2_env/source/s2e/Makefile install
 ... ...
 [ 70%] Linking C shared library s2e.so
 + cd /home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32/linux/s2e.so
 + /usr/bin/cmake -E cmake_link_script CMakeFiles/s2e.dir/link.txt --verbose=
 /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libelf.so when searching for -lelf
 /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libelf.a when searching for -lelf
 /usr/bin/ld: cannot find -lelf
 clang-3.9: error: linker command failed with exit code 1 (use -v to see invocation)

  2. Follow the build commands to run cmake link.txt with --verbose==1

     [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ cd /home/jshi19/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32/linux/s2e.so
 [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32/linux/s2e.so$ /usr/bin/cmake -E cmake_link_script CMakeFiles/s2e.dir/link.txt --verbose=1
 /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/clang -fPIC  -Wall -std=c99 -Werror -g -m32 -march=i386 -mtune=generic -O0 -O3  -shared -Wl,-soname,s2e.so -o s2e.so CMakeFiles/s2e.dir/main.c.o CMakeFiles/s2e.dir/s2e.c.o CMakeFiles/s2e.dir/elf.c.o CMakeFiles/s2e.dir/procmap.c.o CMakeFiles/s2e.dir/modules.c.o CMakeFiles/s2e.dir/__/function_models/libc_wrapper.c.o CMakeFiles/s2e.dir/__/function_models/libz_wrapper.c.o CMakeFiles/s2e.dir/__/function_models/models.c.o -ldl -lelf
 /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libelf.so when searching for -lelf
 /usr/bin/ld: skipping incompatible //usr/lib/x86_64-linux-gnu/libelf.a when searching for -lelf
 /usr/bin/ld: cannot find -lelf
 clang-3.9: error: linker command failed with exit code 1 (use -v to see invocation)

  3. It looks the current ubuntu18.04 "-lelf" lib (libelf) is not compatible with clang-3.9. Anyway, just skip the "-lelf" option to link again:

     [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ /home/jshi19/wksp_efi/s2e-env/edk2_env/install/bin/clang -fPIC  -Wall -std=c99 -Werror -g -m32 -march=i386 -mtune=generic -O0 -O3  -shared -Wl,-soname,s2e.so -o s2e.so CMakeFiles/s2e.dir/main.c.o CMakeFiles/s2e.dir/s2e.c.o CMakeFiles/s2e.dir/elf.c.o CMakeFiles/s2e.dir/procmap.c.o CMakeFiles/s2e.dir/modules.c.o CMakeFiles/s2e.dir/__/function_models/libc_wrapper.c.o CMakeFiles/s2e.dir/__/function_models/libz_wrapper.c.o CMakeFiles/s2e.dir/__/function_models/models.c.o -ldl
 link succeed!

  4. OK, s2e build again:

     [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env/build/s2e/guest-tools32/linux/s2e.so$ cd -
 [S2E:edk2_env] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2_env$ s2e build
 SUCCESS: [build] S2E built

The "-lelf" linker library (which actually because libelf-dev and libelf1 are not installed) failure seems not happen if total regenerate the new environment and clean build in latest s2e.

Tried create the quicksort project, and it has the same problem that s2e engine cannot do the state switching and report "loading state error" if completely booting OS and not use presaved OS snapshot:

  1. create quicksort project:

     [S2E:edk2] jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2$ s2e new_project -i debian-9.2.1-i386 -n quicksort --enable-pov-generation  build/s2e/guest-tools32/common/demos/quicksort @@

  2. Then remove the "-loadvm ready $*" in home\jshi19\wksp_efi\s2e-env\edk2\projects\quicksort\launch-s2e.sh:

    LD_PRELOAD=$LIBS2E $QEMU $DRIVE
    -k en-us $GRAPHICS -monitor null -m 256M -enable-kvm
    -serial file:serial.txt -net none -net nic,model=e1000

  3. Run the launch-s2e.sh and after a while the qemu report state switching error:

     58 [State 0] Switching from state 0 to state 1
 qemu-system-i386: invalid parameter value:
 qemu-system-i386: error while loading state for instance 0x0 of device 'globalstate'
 qemu-system-i386: Could not restore device state
 ./launch-s2e.sh: line 77: 23877 Aborted                 (core dumped) LD_PRELOAD=$LIBS2E $QEMU $DRIVE -k en-us $GRAPHICS -monitor null -m 256M -enable-kvm -serial file:serial.txt -net none -net nic,model=e1000
  • 20190412

I figure out a work around which is to add the "globalstate" device in the s2e kvm_dev_save_snapshot blacklist as below. More discuss about this issue can be found in https://groups.google.com/forum/#!topic/s2e-dev/39s1tQwAd6c

  diff --git a/migration/savevm.c b/migration/savevm.c
  index dd8d6a7c8d..962c307ad3 100644
  --- a/migration/savevm.c
  +++ b/migration/savevm.c
  @@ -798,7 +798,7 @@ void vmstate_unregister(DeviceState *dev, const VMStateDescription *vmsd,
       }
   }

  -static const char *s_blacklist[] = {"cpu", "block", "ram", NULL};
  +static const char *s_blacklist[] = {"cpu", "block", "ram", "globalstate", NULL};

Now, the s2e symbolic execution can works with complete Uefi boot!

OK, below are the simple summary on how to run edk2 OVMF platform on s2e and do the symbolic execution:

  1. Clone and build the s2e. See more detail steps above.

  2. Change the phys_dirty_size as below from 17MB to 128MB in ~/wksp_efi/s2e-env/edk2_env/source/s2e/libcpu/src/exec-ram.c function init_ram_size()

     length += 17 * 1024 * 1024; --> length += 128 * 1024 * 1024;

  3. Add the "globalstate" device in the s2e kvm_dev_save_snapshot blacklist:

     diff --git a/migration/savevm.c b/migration/savevm.c
 @@ -798,7 +798,7 @@ void vmstate_unregister(DeviceState *dev, const VMStateDescription *vmsd,
      }
  }
 -static const char *s_blacklist[] = {"cpu", "block", "ram", NULL};
 +static const char *s_blacklist[] = {"cpu", "block", "ram", "globalstate", NULL};

  4. Rebuild s2e:

    jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2$ s2e build -r qemu libs2e libcoroutine

  5. Create the quicksort project:

    jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2$ s2e new_project -i debian-9.2.1-i386 -n quicksort --enable-pov-generation build/s2e/guest-tools32/common/demos/quicksort @@

  6. Add the s2e-env\edk2\source\s2e\guest\common\include\s2e\s2e.h and s2e-env\edk2\source\s2e\guest\common\include\s2e\opcodes.h to edk2 MdePkg/Include

     MdePkg/Include/opcodes.h
 MdePkg/Include/s2e.h

  7. Add the s2e_make_symbolic() and s2e_kill_state() in your interested edk2 module. Below is a example:

     diff --git a/MdeModulePkg/Bus/Ata/AtaBusDxe/AtaBus.c b/MdeModulePkg/Bus/Ata/AtaBusDxe/AtaBus.c
 @@ -11,7 +11,7 @@
  **/

  #include "AtaBus.h"
 -
 +#include "s2e.h"
  //
  // ATA Bus Driver Binding Protocol Instance
  //
 @@ -1033,6 +1033,8 @@ BlockIoReadWrite (
    UINTN                             NumberOfBlocks;
    UINTN                             IoAlign;

 +  s2e_make_symbolic(&Lba, sizeof(EFI_LBA), "Lba");
 +
    if (IsBlockIo2) {
     Media     = ((EFI_BLOCK_IO2_PROTOCOL *) This)->Media;
     AtaDevice = ATA_DEVICE_FROM_BLOCK_IO2 (This);
 @@ -1083,7 +1085,7 @@ BlockIoReadWrite (
    Status = AccessAtaDevice (AtaDevice, Buffer, Lba, NumberOfBlocks, IsWrite, Token);

    gBS->RestoreTPL (OldTpl);
 -
 +  s2e_kill_state(0, "program terminated");
    return Status;
  }

  8. Build the edk2 OVMF:

     build -p OvmfPkg/OvmfPkgIa32.dsc -t CLANG38 -a IA32 -b NOOPT -DDEBUG_ON_SERIAL_PORT

  9. Change the ~/wksp_efi/s2e-env/edk2/projects/quicksort/launch-s2e.sh, comment out the GRAPHICS and update LD_PRELOAD as below:

       #GRAPHICS=-nographic

   #LD_PRELOAD=$LIBS2E $QEMU $DRIVE \
       # -k en-us $GRAPHICS -monitor null -m 256M -enable-kvm \
       # -serial file:serial.txt -net none -net nic,model=e1000  \
       # -loadvm ready $*
   LD_PRELOAD=$LIBS2E $QEMU $DRIVE \
       -k en-us $GRAPHICS -monitor telnet:127.0.0.1:1234,server,nowait -m 256M -enable-kvm \
       -serial file:serial.txt -net none -net none \
       -bios ~/wksp_efi/edk2/Build/OvmfIa32/NOOPT_CLANG38/FV/OVMF.fd

  10. Run the launch-s2e.sh and use telnet connect the qemu monitor:

     jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2/projects/quicksort$ ./launch-s2e.sh
 jshi19@ub2-uefi-b01:~$ telnet 127.0.0.1 1234

After a while, you should see the OVMF AtaBusDxe module begin to do the symbolic execution on s2e: https://github.com/shijunjing/WikiDoc/blob/master/S2E/OvmfSymbolicExecution.jpg

Check the s2e last run debug.txt to see state switch:

    jshi19@ub2-uefi-b01:~/wksp_efi/s2e-env/edk2/projects/quicksort/s2e-last$ cat debug.txt
    ... ...
    29 [State 48] Switching from state 48 to state 50
    29 [State 50] BaseInstructions: Inserted symbolic data @0xfe4b03c of size 0x8: Lba pc=0xea95ee2
    30 [State 50] Forking state 50 at pc = 0xea95cd3 at pagedir = 0x0
        state 50
        state 51
    30 [State 50] Forking state 50 at pc = 0xea95cd3 at pagedir = 0x0
        state 50
        state 52
    30 [State 50] BaseInstructions: Killing state 50
    30 [State 50] Terminating state early: State was terminated by opcode
                message: "program terminated"
                status: 0x0
    30 [State 50] Switching from state 50 to state 52
    30 [State 52] BaseInstructions: Inserted symbolic data @0xfe4b5cc of size 0x8: Lba pc=0xea95ee2
    31 [State 52] Forking state 52 at pc = 0xea95cd3 at pagedir = 0x0
        state 52
        state 53
    31 [State 52] Forking state 52 at pc = 0xea95cd3 at pagedir = 0x0
        state 52
        state 54
    31 [State 52] BaseInstructions: Killing state 52
    31 [State 52] Terminating state early: State was terminated by opcode
                message: "program terminated"
                status: 0x0
    31 [State 52] Switching from state 52 to state 54
    Terminating node id 0 (instance slot 0)
    Engine terminated.
⚠️ **GitHub.com Fallback** ⚠️