Dynamic inventories for Ansible using Python

Jun 28, 2018
DebianAnsibleDigitalOcean

not only accepts static machine inventories represented in an inventory file, but it is capable of leveraging also dynamic inventories. To use that mechanism the only thing which is needed is a program resp. a script which creates the particular machines which are needed for a certain project and returns their addresses as a JSON object, which represents an inventory like an inventory file does it. This makes it possible to created specially crafted tools to set up the number of cloud machines which are needed for an Ansible project, and the mechanism theoretically is open to any programming language. Instead of selecting an inventory project with the option -i like with ansible-playbook, just give the name of the program you’ve set up, and Ansible executes it and evaluates the inventory which is given back by that.

Here’s a little example of an dynamic inventory for Ansible written in Python. The script uses the library in Debian () to launch a couple of DigitalOcean droplets for a particular Ansible project:

#!/usr/bin/env python
import os
import sys
import json
import digitalocean
import ConfigParser

config = ConfigParser.ConfigParser()
config.read(os.path.dirname(os.path.realpath(__file__)) + '/inventory.cfg')
nom = config.get('digitalocean', 'number_of_machines')
keyid = config.get('digitalocean', 'key-id')
try:
    token = os.environ['DO_ACCESS_TOKEN']
except KeyError:
    token = config.get('digitalocean', 'access-token')

manager = digitalocean.Manager(token=token)

def get_droplets():
    droplets = manager.get_all_droplets(tag_name='ansible-demo')
    if not droplets:
        return False
    elif len(droplets) != 0 and len(droplets) != int(nom):
        print "The number of already set up droplets differs!"
        sys.exit(1)
    elif len(droplets) == int(nom):
        return droplets

key = manager.get_ssh_key(keyid)
tag = digitalocean.Tag(token=token, name='ansible-demo')
tag.create()

def create_droplet(name):
    droplet = digitalocean.Droplet(token=token,
                                   name=name,
                                   region='fra1',
                                   image='debian-8-x64',
                                   size_slug='512mb',
                                   ssh_keys=[key])
    droplet.create()
    tag.add_droplets(droplet.id)
    return True

if get_droplets() is False:
    for node in range(int(nom))[1:]:
        create_droplet(name='wordpress-node'+str(node))
    create_droplet('load-balancer')

droplets = get_droplets()
inventory = {}
hosts = {}
machines = []
for droplet in droplets:
    if 'load-balancer' in droplet.name:
        machines.append(droplet.ip_address)
        hosts['hosts']=machines
        inventory['load-balancer']=hosts
hosts = {}
machines = []
for droplet in droplets:
    if 'wordpress' in droplet.name:
        machines.append(droplet.ip_address)
        hosts['hosts']=machines
        inventory['wordpress-nodes']=hosts

print json.dumps(inventory)

It’s a simple basic script to demonstrate how you can craft something for your own needs to leverage dynamic inventories for Ansible. The parameter of droplets like the size (512mb) the image (debian-8-x64) and the region (fra1) are hard coded, and can be changed easily if wanted. Other things needed like the total number of wanted machines, the access token for the DigitalOcean API and the ID of the public SSH key which is going to be applied to the virtual machines is evaluated using a simple configuration file (inventory.cfg):

[digitalocean]
access-token = 09c43afcbdf4788c611d5a02b5397e5b37bc54c04371851
number_of_machines = 4
key-id = 

The script of course can be executed independently of Ansible. The first time you execute it, it creates the number of machines which is wanted (consisting of always of one load-balancer node and – given the total number of machines, which is four – three wordpress-nodes), and gives back the IP adresses of the newly created machines being put into groups:

$ ./inventory.py 
{"wordpress-nodes": {"hosts": ["", "", ""]}, "load-balancer": {"hosts": [""]}}

Any consecutive execution of this script recognizes that the wanted machines already have been created, and just returns this inventory the same way one more time:

$ ./inventory.py 
{"wordpress-nodes": {"hosts": ["", "", ""]}, "load-balancer": {"hosts": [""]}}

If you delete the droplets then, and run the script again, a new set of machines gets created:

$ for i in $(doctl compute droplet list | awk '/ansible-demo/{print $(1)}'); do doctl compute droplet delete $i; done
$ ./inventory.py 
{"wordpress-nodes": {"hosts": ["", "", ""]}, "load-balancer": {"hosts": [""]}}

As you can see, the JSON object¹ which is given back represents an Ansible inventory, the same inventory represented in a file it would have this form:

[load-balancer]


[wordpress-nodes]

Like said, you can use this “one-trick pony” Python script instead of an inventory file, just given the name of that, and the Ansible CLI tool runs it and works on the inventory which is given back:

$ ansible wordpress-nodes -i ./inventory.py -m ping -u root --private-key=~/.ssh/id_digitalocean
 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}
 | SUCCESS => {
    "changed": false, 
    "ping": "pong"
}

Note: the script doesn’t yet supports a waiter mechanism but completes as soon as there are IP adresses available. It always could take a little while until the newly created machines are completely created, booted, and accessible via SSH, thus there could be errors on the hosts not being accessible. In that case, just wait a few seconds and run the Ansible command again.