Chef Cookbooks
Chef is a powerful configuration management tool that automates system builds, configures software and ensures configurations are kept up-to-date. A recipe is the most basic type of configuration for Chef. A recipe installs, configures or starts a service, updates a configuration file or anything else that is required to configure a node. Cookbooks group multiple recipes in a collection. Chef executes these cookbooks on multiple nodes which could range from a few systems to thousands. Given the scale at which Chef operates, changes to the cookbooks should be rigorously tested prior to deploying them on live systems. This is where kitchen plays an important role. With kitchen, we can spin up a Docker container, test the cookbook changes in a container and then remove the container once the test is complete.
Components of a Cookbook
The main components of a cookbook are recipes, tests and spec directories. The format of a recipe takes is a resource declaration, name of the resource, and an action. Develop the Cookbook Test Cases section of this article covers test cases.
resource name do
action :some_action
end
Note: Anyone with shell scripting experience will naturally end the resource definition with done and not end. This will generate an unexpected end-of-input error.
FATAL: Stacktrace dumped to /root/.chef/local-mode-cache/cache/chef-stacktrace.out FATAL: Please provide the contents of the stacktrace.out file if you file a bug report ERROR: /root/cookbooks/kitchen_test/recipes/default.rb:14: syntax error, unexpected end-of-input, expecting keyword_end FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Creating a Cookbook
For our mongodb example cookbook, we will use two resources: package and service. The package resource will install or remove the software. The service resource will enable, start or disable a service from the system. You will notice on line 6 that we are using a Ruby list and combining two actions, enable and start.
Please note: this cookbook is an example only to showcase using kitchen. The Chef Supermarket has a cookbook available which provides much more functionality. You can download the cookbook from here.
The chef command below will create the cookbook structure. If chef isn’t installed on the system, you can download it as part of the ChefDK found here. Chef is deprecating the knife cookbook create command in favor of using the chef command.
root@ip-172-36-61-107:~/cookbooks# chef generate cookbook mongodb_server Generating cookbook mongodb_server - Ensuring correct cookbook file content - Committing cookbook files to git - Ensuring delivery configuration - Ensuring correct delivery build cookbook content - Adding delivery configuration to feature branch - Adding build cookbook to feature branch - Merging delivery content feature branch to master Your cookbook is ready. Type `cd mongodb_server` to enter it. There are several commands you can run to get started locally developing and testing your cookbook. Type `delivery local --help` to see a full list. Why not start by writing a test? Tests for the default recipe are stored at: test/recipes/default_test.rb If you'd prefer to dive right in, the default recipe can be found at: recipes/default.rb
The chef generate command creates the directory structure below. Note: the default_test.rb has some sample content, so be sure to replace this content as needed.
root@ubuntu-chef:~/cookbooks# tree mongodb_server
mongodb_server
├── Berksfile
├── chefignore
├── LICENSE
├── metadata.rb
├── README.md
├── recipes
│ └── default.rb
├── spec
│ ├── spec_helper.rb
│ └── unit
│ └── recipes
│ └── default_spec.rb
└── test
└── integration
└── default
└── default_test.rb
This is the cookbook we will use for the kitchen tests. Create the recipe file under mongodb_server/recipes and name it mongodb.rb. The “rb” is the extention for Ruby and is used for all recipes and kitchen tests.
package 'mongodb-server' do
action :install
end
package 'mongodb-dev' do
action :install
end
service 'mongodb' do
action [ :enable, :start ]
end
Update the default.rb file in the mongodb_server/recipe to include the mongodb.rb file.
include_recipe 'mongodb_server::mongodb'
Execute the chef-client command to run the recipe locally. Alternatively, if you have setup Hosted Chef, you may omit “–local-mode” from the command below.
root@ip-172-36-61-107:~/cookbooks# chef-client --local-mode -r "recipe[mongodb_server]" [0000-00-00T00:00:00+00:00] WARN: No config file found or specified on command line, using command line options. Starting Chef Client, version 12.13.37 resolving cookbooks for run list: ["mongodb_server"] Synchronizing Cookbooks: - mongodb_server (0.1.0) Installing Cookbook Gems: Compiling Cookbooks... Converging 3 resources Recipe: mongodb_server::mongodb * apt_package[mongodb-server] action install (up to date) * apt_package[mongodb-dev] action install (up to date) * service[mongodb] action enable (up to date) * service[mongodb] action start (up to date) Running handlers: Running handlers complete Chef Client finished, 0/4 resources updated in 01 seconds
Testing Chef Cookbooks with Kitchen
The Chef Development Kit (ChefDK) includes the kitchen command to validate changes to a cookbook . With the kitchen-docker driver, kitchen automatically creates Docker containers, applies the cookbook and then destroys the container. In order to install Docker, please follow the instructions here.
We have to initialize the environment using the command kitchen init. Since we are using the Docker driver, we pass the “-D docker” argument so Chef will use docker instead of the default vagrant. kitchen will automatically download and install the Docker gem package.
root@ubuntu-chef:~/cookbooks/mongodb_server# kitchen init -D docker
create .kitchen.yml
create chefignore
Successfully installed docker-0.4.0
Parsing documentation for docker-0.4.0
Done installing documentation for docker after 0 seconds
1 gem installed
Configuring .kitchen.yml File
The init phase creates the configuration file named .kitchen.yml. You can leave the default .kitchen.yaml, or add or remove any other items required for testing with the cookbook. We removed the CentOS platform since we are only using Ubuntu in this example.
We are using the default verifier named Inspec. The suites heading, describes what we are wanting to test. We are only testing the default recipe, but in a real-world cookbook, we may define different recipes to support the cookbook. Underneath the suites -> verifier subheading lists the directory where the test cases reside. The test cases will not execute since Chef will check a different location.
---
driver:
name: docker
provisioner:
name: chef_zero
# You may wish to disable always updating cookbooks in CI or other testing environments.
# For example:
# always_update_cookbooks: <%= !ENV['CI'] %>
always_update_cookbooks: true
verifier:
name: inspec
platforms:
- name: ubuntu-14.04
suites:
- name: default
run_list:
- recipe[mongodb_server::default]
verifier:
inspec_tests:
- test/recipes/default
attributes:
After running kitchen init, you can see the list of instances available by running list. Below is only one instance since we removed Centos from the .kitchen.yml file.
root@ubuntu-chef:~/cookbooks/mongodb_server# kitchen list Instance Driver Provisioner Verifier Transport Last Action Last Error default-ubuntu-1404 Docker ChefSolo Busser Ssh <Not Created> <None>
Now create the virtual instance used for testing with the kitchen create command. Note: this command usually takes 3-5 minutes to complete.
root@ubuntu-chef:~/cookbooks/mongodb_server# kitchen create
-----> Starting Kitchen (v1.20.0)
-----> Creating <default-ubuntu-1404>...
Sending build context to Docker daemon 201.2kB
Step 1/17 : FROM ubuntu:14.04
14.04: Pulling from library/ubuntu
324d088ce065: Pulling fs layer
2ab951b6c615: Pulling fs layer
9b01635313e2: Pulling fs layer
04510b914a6c: Pulling fs layer
83ab617df7b4: Pulling fs layer
04510b914a6c: Waiting
83ab617df7b4: Waiting
9b01635313e2: Verifying Checksum
9b01635313e2: Download complete
We can use docker commands to see the images and containers kitchen created. If you need to check the container, use the command docker exec -it exec -it 83043dc787e8 /bin/bash
root@ubuntu-chef:~/cookbooks/mongodb_server# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b6948b229272 71cdf1224f9a "/usr/sbin/sshd -D -…" 2 minutes ago Up 2 minutes 0.0.0.0:32768->22/tcp defaultubuntu1604-chefuser-ubuntuchef-9fphugrq root@ubuntu-chef:~/cookbooks/mongodb_server# docker images REPOSITORY TAG IMAGE ID CREATED SIZE <none> <none> 71cdf1224f9a 2 minutes ago 216MB ubuntu 14.04 0b1edfbffd27 3 days ago 113MB
The next step is to install and run the cookbook in the virtual instances created from the step above using the converge sub-command. This command may take several minutes to complete.
root@ubuntu-chef:~/cookbooks/mongodb_server# kitchen converge
-----> Starting Kitchen (v1.21.1)
-----> Converging <default-ubuntu-1404>...
Preparing files for transfer
Preparing dna.json
Preparing current project directory as a cookbook
Removing non-cookbook files before transfer
Preparing solo.rb
-----> Installing Chef Omnibus (install only if missing)
Downloading https://omnitruck.chef.io/install.sh to file /tmp/install.sh
Trying wget...
Download complete.
ubuntu 16.04 x86_64
Getting information for chef stable for ubuntu...
Develop the Cookbook Test Cases
After running kitchen converge, we can run test cases using the verify sub-command. Below is an example test script which verifies the state of Mongo. The script checks that the default mongodb port is in a LISTEN state, the mongodb-server package is installed, a user is defined as mongodb, /etc/mongodb.conf exists and finally checks if the service is running.
describe port(27017) do
it { should be_listening }
end
describe package('mongodb-server') do
it { should be_installed}
end
describe user('mongodb') do
it { should exist }
end
describe file('/etc/mongodb.conf') do
it { should be_file }
end
describe service('mongodb') do
it { should be_running }
end
Use the verify argument to execute the test cases written in the previous step.
root@ubuntu-chef:~/cookbooks/mongodb_server# kitchen verify
-----> Starting Kitchen (v1.20.0)
-----> Verifying <default-ubuntu-1404>...
Loaded tests from {:path=>".root.cookbooks.mongodb_server.test.recipes.default"}
Profile: tests from {:path=>"/root/cookbooks/recipes/test/recipes/default"} (tests from {:path=>".root.cookbooks.mongodb_server.test.recipes.default"})
Version: (not specified)
Target: ssh://kitchen@localhost:32769
Port 27017
✔ should be listening
System Package mongodb-server
✔ should be installed
User mongodb
✔ should exist
File /etc/mongodb.conf
✔ should be file
Service mongodb
✔ should be running
Test Summary: 5 successful, 0 failures, 0 skipped
Finished verifying <default-ubuntu-1404> (0m0.24s).
-----> Kitchen is finished. (0m1.50s)
The last step uses the destroy sub-command which removes the Docker container.
root@ubuntu-chef:~/cookbooks/mongodb_server# kitchen destroy
-----> Starting Kitchen (v1.20.0)
-----> Destroying <default-ubuntu-1404>...
UID PID PPID C STIME TTY TIME CMD
root 27070 19304 0 May03 ? 00:00:00 /usr/sbin/sshd -D -o UseDNS=no -o UsePAM=no -o PasswordAuthentication=yes -o UsePrivilegeSeparation=no -o PidFile=/tmp/sshd.pid
usbmux 34586 27070 0 09:19 ? 00:00:18 /usr/bin/mongod --unixSocketPrefix=/var/run/mongodb --config /etc/mongodb.conf run
41f85bcf65e1707d152d2e8c351ed4289d4e65d355328acc1e4914457d5193d4
41f85bcf65e1707d152d2e8c351ed4289d4e65d355328acc1e4914457d5193d4
Finished destroying <default-ubuntu-1404> (0m1.88s).
-----> Kitchen is finished. (0m2.93s)