Chef Testing - chef-boneyard/chef-summit-2014 GitHub Wiki

Chef Testing

Friday, Medina, 10

Convener

Participants

Summary of Discussions

signal in

  • foodcritic
  • rubocop
  • chefspec

signal process

  • chef sw docs this

signal out

  • serverspec
  • minitest
  • bats

"do not use minitest/chef-minitest handler"

  • aspect
  • end with functional
  • BAN

build suite, let business decide what is critical to test

test assertion vs load testing - important for upstream/downstream testing to get RTT

  • chef itself is test with rspec
  • chef server is tested with rspec
  • within provider, etc chefspec

also recommend linters: food critic, rubocop as part of commit process

testing: lint -> unit -> integration -> functional

complex template example: tests for 0 nodes, 1 node, n nodes, confirm valid nginx config

pattern: use chef spec to cap number of searches

eg: chef vault and chef-splunk https://github.com/opscode-cookbooks/chef-vault https://github.com/opscode-cookbooks/chef-splunk use chef zero / chef spec searches

other tools...

minitest Charles: no easy way to help me understand how to write my own

test kitchen codebase - rspec/minitest flava example arai "infratester" project - multimachine testing on top of rspec

other questions

test kitchen/chef metal integration/ relationship?

  • jtmiberman: yes/no/needs work

  • fletcher nichols: tk spawn arbitrary chef metal, just a chef run possibly outside the cluster you're testing

  • sweets to describe platform matrix

  • using server spec - you have access to the node object eg: modify attributes so it no longer hits some resources, you need something outside of chef to catch that.

eg: do chef run -> prove to me that the web site works. what are those things you would do to smoke test/ observe side effect/ prove to yourself that the site is up? These are good tests.

There is a debatable scope regarding this, when is it redundant versus meaningful. It depends on the question you want to ask.

  • To validate that input will not regress - chef spec.
  • server spec could be thought of as acceptance criteria. should work even if you switched cm tool from chef to puppet. serverspec should not normally need to be aware of chef.

**sensu server modules anyone? ** (nope, standalone)

pro's and cons between signal out choices?

  • serverspec goes above and beyond BATS: picks platform, picks right command

manual testing

saner ways to do manual testing? look at bash history -> feed it into (BATS, serverspec) make busser execute it

test kitchen start from making sure the cookbook works... ssh in, cat this, sed that, exit 0? all good, yo. if this helps then it's legitimate

whyrun is useful for:

  • really early when building a new thing - you see the template diffs

  • long running system that was not built with chef

  • refactoring - change service names, post migration config inspection

  • note: does the thing you're trying to test actually support whyrun?

What will we do now? What needs to happen next?

⚠️ **GitHub.com Fallback** ⚠️