Automated Testing - openbmc/openbmc GitHub Wiki

Do's and Don'ts

  • Do use code coverage to measure the effectiveness of your test suite - don't fly blind
  • Do isolate your tests to avoid depending on external (unreliable) resources - don't write flaky accidental integration tests
    • Use mocks to avoid external resources
  • Do write separate tests for separate behaviours - don't write one monolithic test case exercising all desired behaviours
    • Writing a monolithic test makes it hard to separate out the dependencies between the exercised behaviours
    • If it's hard to separate the dependencies it's hard to identify the conditions under which the failure has occurred
  • Do prioritise your testing - test public interfaces before testing private interfaces
    • You should be able to exercise all code-paths through your public interfaces
      • If you can't, you've written dead code
    • Test cases for private interfaces have a higher chance of simply being discarded when your code is refactored
  • Do consider in the design phase how you will test your code - don't jump from concept to implementation
  • Do write a test first if you are fixing a bug - don't leave regression testing to people, people are unreliable!
  • Do consider whether your private interfaces could be a public interface of a lower layer of abstraction - don't fall back on fragile techniques (e.g. friend classes) unless absolutely necessary