machine-check: Unit test your server configuration with Racket

1 Introduction

With machine-check you can write unit tests in Racket to verify if the state of your machine is what you expect.

See the code at on github.

2 Description

If you use configuration management tools you might want to check if the state of your machine is exactly as you expected it after applying the state. For example after a Puppet, SaltStack or Ansible run it might be wise to check if you actually have the packages installed that you wanted to enforce, and if the files you templated exist in the places and with the content you expected.

While very minimal, this project aspires to provide similar functionality to what pytest-testinfra does in Python and serverspec does in Ruby.

3 Usage

See the examples dir for an example of how to use this project. It comes down to placing your Racket unit tests in machine-check/checks-to-perform and running machine-check.

If you want a quick copy-paste-able example:

3.1 Writing tests

This section contains some examples of what type of checks you can perform with machine-check. Any .rkt file in checks-to-perform will be loaded, and each of those files should provide a perform-<name>-checks function.

3.1.1 Testing if packages are installed

Here are some examples for how to test if certain packages are installed on the machine.

To test if xorg is installed on a Debian system you could create a check in checks-to-perform/packages.rkt. After cloning the repository and cd-ing into the machine-check directory you could do so like:

rm -rf checks-to-perform

mkdir checks-to-perform

cat << 'EOF' > checks-to-perform/packages.rkt

#lang racket/base

(require "../machine-check/check-helpers.rkt")

(provide perform-packages-checks)

(define perform-packages-checks

(λ ()

(check-package-installed "xorg")))

EOF

Note that it is significant that packages.rkt provides perform-packages-checks. Similarly a file named somethingelse.rkt would have to provide a function named perform-somethingelse-checks.

After installing xorg this would look like:

root@devenvdebian:~/machine-check# apt-get install xorg

root@devenvdebian:~/machine-check# make run

rm -rf out

rm -rf compiled

rm -rf machine-check/compiled

rm -rf machine-check/test/compiled

racket main.rkt

All tests pass!

root@devenvdebian:~/machine-check# echo $?

0

But imagine that we’d like to also run our test-suite on an Archlinux machine for example when our configuration management is set up to work with multiple distributions. In that case just testing for xorg would not be good enough anymore because on Arch Linux that package has a different name (there it is named xorg-server, not xorg).

For that case we can use detect-os-release. With that we can conditionally test some package names if the Arch Linux os is detected, and some other on our Debian system.

rm -rf checks-to-perform

mkdir checks-to-perform

cat << 'EOF' > checks-to-perform/packages.rkt

#lang racket

(require "../machine-check/check-helpers.rkt")

(require detect-os-release)

(provide perform-packages-checks)

(define perform-packages-checks

(λ ()

; Install desktop packages

(let ((detected-os (detect-os))

; Package names to check on all distros

(packages-to-check (list

"chromium"

"terminator"

)))

(check-packages-installed

(if (equal? detected-os "arch")

; Archlinux packages

(append packages-to-check

(list

"xorg-xinit"

"xorg-server"

))

; Debian packages

(append packages-to-check

(list

"xinit"

"xorg"

)))))))

EOF

3.1.2 Testing file mode and content

If you do a lot of templating in your configuration management it might be smart to test files for their content. Also testing their access mode (permissions) can be convenient.

rm -rf checks-to-perform

mkdir checks-to-perform

cat << 'EOF' > checks-to-perform/files.rkt

#lang racket

(require "../machine-check/check-helpers.rkt")

(provide perform-files-checks)

(define perform-files-checks

(λ ()

(check-file-contains "/etc/ssh/sshd_config"

"PasswordAuthentication no")

; Test that ssh_config is 644

(check-file-mode "/etc/ssh/sshd_config" 420)))

EOF

Take note that the permissions are checked using (file-or-directory-permissions path 'bits) from racket/file which does not use the Unix filesystem three-digit octal value you might expect. Instead it is some other kind of platform-specific integer so it might be best to test it out before using it with check-file-mode.

$ stat -c "%a %n" /etc/ssh/sshd_config

644 /etc/ssh/sshd_config

$ racket

> (file-or-directory-permissions "/etc/ssh/sshd_config" 'bits)

420

3.1.3 Conditional tests

If you install the machine-check test files in checks-to-perform as part of you configuration management as well it might be nice to have only certain checks run on systems with a specific role. For example, if your configuration management installs different packages on different servers you do not want to test for those packages on the servers where they should not be installed. This could be solved by templating the checks-to-perform files in the configuration management, but an alternative approach could be to do this based on a file on disk.

In SaltStack it is possible to see which states (roles) have been applied to this server. On the Salt minion you can see this by running:

We can output this to a file as part of our configuration management run so that later machine-check can access this information:

Then in a check we can decide to skip testing of a certain package is installed if this server has not had a specific state applied to it.

rm -rf checks-to-perform

mkdir checks-to-perform

cat << 'EOF' > checks-to-perform/aptrepo.rkt

#lang racket

(require "../machine-check/check-helpers.rkt")

(provide perform-aptrepo-checks)

(define applied-states (file->lines "/srv/applied_states"))

(define perform-aptrepo-checks

(if (member "aptrepo" applied-states)

(λ ()

(check-packages-installed

(list "reprepro")))

void))

EOF

4 Development

To run the unit tests for machine-check (for the software, not the system unit tests which the software is for) run:

5 Documentation

To generate the documentation run: