Auto Testing Tools - KU-SKE17/Software-Process GitHub Wiki

  1. Describe each of these. What kind of problems do their apply to?
    • How likely is it to find defects?
  2. What is the difference?
  3. What software can we use? (Java or Python)

Random Testing - QuickCheck

  • black box (nearly), tester only needs to know interface
  • accepted inputs: domain
  • almost anything
  • low probability, which you compensate for by generating many tests

Adaptive Random Testing - QuickCheck with "biased input"

  • random testing but change inputs to increase change of detecting a failure
  • some prior domain knowledge

Symbolic Execution - Z3 from Microsoft, pySym

  • create boolean expressions to represent conditions for the different paths through the code & states of the code
  • not too complex, source available
  • situations where difficult to apply
    • file, database
    • external service
  • change of detection faults: if applicable, high prob. of detecting faults

Dynamic Symbolic Execution

  • combination of Random Testing and Symbolic Execution

Metaheuristic Test Generation - EvoSuite

  • generate tests via some sort of optimization procedure
  • "create test suites that lead to some testing goal"
  • fitness function -> objective function, something to {maximize, minimize}
  • fitness function from static analysis