How to Develop Puppet Modules – Configuring System

Modules

A module is a collection of manifests and data (such as facts, files, and templates), and they have a specific directory structure. Modules are useful for organizing your Puppet code, because they allow you to split your code into multiple manifests. It is considered best practice to use modules to organize almost all of your Puppet manifests.

To add a module to Puppet, place it in the /etc/puppet/modules directory.

Configuring Our Development System

When developing puppet modules, we usually use a workstation which is a Linux physical system or virtual machine. We do all development under a local user account, not the Root user. Because puppet was written using Ruby it makes use of Gems and installing Gems as Root can cause system-wide issues. Let us begin by creating a development directory under the user account. For the following examples, let the user account is stack admin. Begin by creating your development directory:

$ mkdir -p ~/Development/puppet-modules

This directory will begin to look like your modules directory on your puppet master because it will contain all your modules. The next step to configuring your development environment is to ensure that you have a suitable text editor that makes use of puppet syntax highlighting. We generally use VIM but you can also use EMACS or one of several options. Not all distributions have VIM installed by default. RHEL and its derivatives do not. Use the following command to install it:

$ sudo yum install vim

Now that we have our development environment ready, you can begin coding our sample puppet module.

Installing Ruby

Since puppet was developed in Ruby and makes use of Ruby rspec testing, we need to install ruby on our system for testing our modules. Due to that fact that there are many different versions of Ruby and also Puppet has different requirements for versions, this could be a daunting task. However, we will make use of RBENV which will make managing Ruby easier.

Run the following commands to install Ruby. For the Ubuntu Desktop replace ~/.bash_profile with ~/.bashrc:

$ git clone https://github.com/rbenv/rbenv.git ~/.rbenv
$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile
$ ~/.rbenv/bin/rbenv init
$ source ~/.bash_profile
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build

Now that, we have RBENV installed, it is time to install a version of Ruby:

$ rbenv install 2.1.5

This will more than likely require you to install some additional packages. This is normal as RBENV compiles Ruby from source.

Now, configure your environment to use this newly installed Ruby version:

$ rbenv global 2.1.5

Finally, ensure that it all worked by checking the Ruby version

$ ruby -v
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]

There is only one gem that we need to install globally and that is a bundler. We use the bundler gem to help us install other gems for puppet testing later.

$ gem install bundler

Ruby is now successfully installed on your development system.

Module directory names

In your development environment (and later when you have your modules in Git) you might want a way to separate the modules by functionality or by business unit. This is different than what the module might actually be called on your puppet masters.

For example, when you upload a module to PuppetForge, you put your name in front of the module name, separated by a hyphen (stackadmin-sample). This gives us a nice way of separating our own modules locally. For our example, we will name the directory for our module in our development environment and in git using this convention:

CDBook-Sample

This shows that our module is from this continuous delivery tutorial and that the name of the module is sample. When the module is deployed to our puppet master, the directory will just be a sample.

Docker

It is a container technology. It is NOT virtualization, all containers share the same kernel.

Containers are based on images. Images are pre-configured states of a container. Consider them as a tar.gz of a Linux installation, including all libraries and packages. When a process is executed inside a container, it will be using libraries and configuration inside the container. Containers are single use: once the process terminates it can’t be re-executed in the same container.

Images are layered. Basically, after executing something in the container, the end-state of the container can be saved to a new image. In order to save disk space, not the whole container will be saved, but only the difference between the original image and the end-state of the container. Docker uses AUFS (Another Union File System) to simulate a full filesystem based on these layers.

Ensure puppet is installed

We need to ensure that the puppet is installed on our development system. The easiest way to do this is to install the puppet gem.

$ gem install puppet

Write a simple module

Our sample module is ridiculously simple and is just used to demonstrate how you can integrate continuous delivery in your puppet environment. All our modules are required to do is create an empty text file in /tmp. We can then use continuous delivery to ensure that the module does what we want and deploy it automatically to our environments.

Begin by generating the module using puppet’s module generate command:

$ puppet module generate cdbook-sample

This will ask you a series of questions which are not really important at this stage. After the command is completed, you will get a module directory created with everything you need.

Notice that there is a Gemfile listed. This is what we will use to install the ruby gems needed for rspec testing. We make use of the bundler gem we installed previously to install these gems. We will be installing the gems into a local hidden folder in this module directory. This allows us some added benefit later in the testing phase when we automate these tests on a continuous integration server like Jenkins or Travis-CI.

$ cd sample
 $ bundle install --path=.bundle/gems/

Write unit tests using rspec

One thing that the puppet module generate command misses is creating the fixtures file that will be used by rspec to reference your local code and any dependencies. By using your text editor, create this file in the root directory of your module.

$ vim .fixtures.yml

With the following content:

We already have a sample rspec test created in the spec/classes/init_spec.rb file that checks to make sure that “sample” class was created.

Let’s run our unit tests and see if it passes:

$ bundle exec rake spec

This command uses the gems we downloaded locally to our module folder rather than any global gems. The output of this command should look like this:

This shows that our one test has passed. Now, we will add a test to make sure that our file is created. Edit the spec/classes/init_spec.rb file and add the following line under the should contain_class line:

It { should contain_file(‘test.txt’) }

Save and run the tests again. You will see that our new test fails.

Astute readers will notice that we never actually added the code to our puppet manifests for the file resource. This is an example of Test Driven Development with puppet. We code the tests for stuff that we know we are going to need in our manifests. Then run the tests and observe the failure. We then code our manifests to make a test pass. Rinse and repeat till all tests pass.

This a great way to introduce Pair Programming into your puppet development team. If you have a senior puppet developer and a junior puppet developer, then you would have the senior puppet developer write the initial module framework and a series of tests that they know the module will require.

The junior developer then runs the tests which will fail and codes the puppet manifests to make one test pass at a time until the module passes all of its tests. The developers can sit side-by-side or have a screen sharing session if working remote. This is an excellent way to pass knowledge among your team.

Now, let’s code our file resource into our module. Update your manifests/init.pp file to the following:

Run the rspec tests again.

$ bundle exec rake spec

Write acceptance tests using beaker

Now that we have some basic unit testing capabilities, we need to write some acceptance tests using Beaker. Beaker is found on GitHub at https://github.com/puppetlabs/beaker. Install the requirements for beaker first:

$ sudo apt-get install ruby-dev libxml2-dev libxslt1-dev g++ zlib1g-dev

or for RHEL based systems

$ sudo yum install make gcc gcc-c++ libxml2-devel libxslt-devel ruby-devel

Next, install Beaker by installing the gem:

$ gem install activesupport
 $ gem install beaker

This will take some time, but once complete you will be able to use beaker for your acceptance testing.

$ beaker --help

First, ensure that you have Docker running correctly by running the following command:

$ sudo docker run hello-world

You should see something similar to the following:

If you do not, then go to Docker’s website and install it.

Create a Beaker testing environment

Beaker uses a hosts file that sets up our virtual Docker image for acceptance testing. Create a directory for the hosts files:

$ mkdir spec/hosts

And create the hosts file:

$ vim spec/hosts/sample.cfg

With the following content:

Create the test file spec/acceptance/sample_spec.rb

Add beaker to Gemfile:

Run the Beaker acceptance test:

$ bundle exec rspec spec/acceptance

You should notice that the test fails.

Our puppet rspec tests passed so it appeared that our module was golden. However, this is not the case when we try to do an actual puppet run on the module. We need to give the file resource an explicit directory. Change your init.pp file to the following:

We will need to also update our rspec testing. Edit the spec/classes/init_spec.rb file:

Run the rspec unit tests again:

$ bundle exec rake spec

Next run the Beaker acceptance tests again:

$ bundle exec rspec spec/acceptance

Both our rspec unit testing and Beaker acceptance testing have now passed. Now we have a solid module to start with. We can now use this module for continuous integration and automatic deployment to our puppet environment(s).