Testing cloud-init in a public can be a bit problematic and time consuming, having to relaunch servers to test the config from a pristine state. It's possible to reset the cloud-init config on the server - but if you want to test that your config file works as expected, it's best to test from a freshly booted server.

Thanks to the Ubuntu Vagrant boxes which are built with cloud-init support, we can test our cloud-init configuration with Vagrant easily. We'll be using the Ubuntu 16.04 Xenial Xerus box to test with - as it's the latest Ubuntu LTS release.

Creating a cloud-init ISO

cloud-init can support multiple data sources for configuration, one of them being an CD image. We'll be using this as an easy way to provide our config files. To generate this we can use genisoimage/mkisofs.

To install on Debian/Ubuntu:

$ sudo apt-get install genisoimage

To install on OS X Macports:

$ sudo port install cdrtools

Next up we'll need our user-data file, the main cloud-init config file:

#cloud-config
chpasswd:
  list: |
    ubuntu:RANDOM
  expire: False

manage_etc_hosts: localhost

ssh_authorized_keys:
  - ssh-rsa
    AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ==
    vagrant insecure public key

This can be customised with the configuration you need to test, however in this example we're adding the default Vagrant insecure public SSH key. Don't worry - Vagrant will replace it shortly after rebooting.

Now we need a meta-data file:

instance-id: iid-0123456789abcdef
local-hostname: ubuntu-xenial

This is just the bare minimum - cloud providers usually provide more data. If you depend on other data provided as metadata then just add it as needed.

To create the ISO image we'll create a Makefile:

nocloud.iso: meta-data user-data
    mkisofs \
        -joliet -rock \
        -volid "cidata" \
        -output nocloud.iso meta-data user-data

Don't forget - a Makefile must be indented with tabs and not spaces!

Now to create the ISO file, all we have to do is run:

$ make

And you'll have a brand new nocloud.iso ISO image.

cloud-init Vagrantfile

Now we can create a Vagrantfile to test our newly generated ISO image:

$ vagrant init ubuntu/xenial64

This will create a sample Vagrantfile, which we'll need to edit:

# -*- mode: ruby -*-
# vi: set ft=ruby :

CLOUD_CONFIG_PATH = File.join(File.dirname(__FILE__), "nocloud.iso")

Vagrant.require_version ">= 1.8.0"

Vagrant.configure(2) do |config|
    config.vm.box = "ubuntu/xenial64"

    # Disable SSH password for 16.04 - we'll add the insecure Vagrant key
    # (don't worry, it's just an example and gets replaced anyway)
    config.ssh.password = nil

    # Disable shared folders
    config.vm.synced_folder ".", "/vagrant", disabled: true

    # Tweak virtualbox
    config.vm.provider :virtualbox do |vb|
        # Attach nocloud.iso to the virtual machine
        vb.customize [
            "storageattach", :id,
            "--storagectl", "SCSI",
            "--port", "1",
            "--type", "dvddrive",
            "--medium", CLOUD_CONFIG_PATH
        ]

        # Speed up machine startup by using linked clones
        vb.linked_clone = true
    end
end

This creates a new virtual machine based on the xenial64 Vagrant box, but replaces the CD/DVD port with our nocloud.iso file.

All you have to do at this point is:

$ vagrant up

Vagrant will start up a virtual machine, customised with your cloud-init config file. If anything goes wrong you can easily vagrant destroy the machine and try again - without the cost of starting a new server in a public cloud.

Using your own SSH keys

If you're developing a user-data file and want to use your own SSH keys instead of Vagrant replacing it on every boot, edit the Vagrantfile and add these lines:

config.ssh.private_key_path = File.expand_path("~/.ssh/id_rsa")
config.ssh.insert_key = false

Don't forget to edit the user-data file and add your own public key to ssh_authorized_keys, and run make to generate a new ISO image.

Conclusion

Testing cloud-init with Vagrant is quick and easy once you've got everything setup.

Source code for this is available in the cloud-init-vagrant repo on GitHub.

The problem

Ansible uses an inventory file to define the SSH details for each server, what groups a server belongs to - which then allows you to run certain roles/playbooks on specific servers. However the two options with Vagrant in dealing with the inventory file are rather limited.

The first option is the auto generated inventory, Vagrant will create a fresh inventory file based on the servers listed in the Vagrantfile which are currently up and running. This inventory file gets populated with the correct SSH details, however it means it starts with an empty inventory file which is only populated by setting ansible.groups.

The second option is the static inventory, which Vagrant will use instead of creating a dynamic inventory file. The downside with this would be duplicating the existing hosts file just to edit the SSH details, as well as hardcoding the IP addresses of all your Vagrant virtual machines.

The solution

We can use the ansibler Ruby gem to read the hosts file to automatically populate ansible.groups.

Installation is easy, as Vagrant maintains plugins separately from other Ruby libraries:

$ vagrant plugin install ansibler
Installing the 'ansibler' plugin. This can take a few minutes...
Installed the plugin 'ansibler (0.2.2)'!

Now we need to update the Vagrantfile. First of all we're going to read the existing hosts file, and populate the ANSIBLE_GROUPS variable with a hash of all the groups:

inventory = Ansible::Inventory.read_file('hosts')

ANSIBLE_GROUPS = {}

inventory.groups.each do |group|
    ANSIBLE_GROUPS["#{group.name}"] = group.hosts.map{ |host| host.name }
end

Next up is updating the Ansible provisioner config:

config.vm.provision "ansible" do |ansible|
    # ...
    ansible.groups = ANSIBLE_GROUPS
end

And that's it!

Now that ansible.groups contains an exact copy of all the group config from your hosts file, every time Vagrant creates a new dynamic inventory file it'll be populated with all the group configuration from your main hosts file.