Contents
Let’s start playing with Ansible. We’re going to finalize the Ansible configuration and examine first playbooks for basic operations.
File Structure on the ManagementPi
For the start, we need to create a very simple file structure to host our future Ansible playbooks, the inventory, variables, secrets and additionally required scripts.
mkdir -p ~/manager/files
mkdir -p ~/manager/inventory
mkdir -p ~/manager/vars
We will place the inventory file within manager/inventory, and our variables and secrets within an encrypted vault file in manager/vars. For now, our future playbooks will go to the manager directory directly.
Inventory
Ok, before we can start playing with Ansible, we need to tell Ansible about our home lab, of course. In public cloud environments or real-life datacenters, we’d do that with dynamic inventories, since the environment is typically changing continuously. At least within my home lab that is not the case. For that reason, I can stick to a static inventory file.
Let’s go ahead and create one, which is an easy task since we can define that with a simple yaml file. We create that file within the directory manager/inventory and name it hosts.yaml.
Here’s an example how it could look like:
vi ~/manager/inventory/hosts.yaml
all:
children:
smarthome:
hosts:
smarthomesrv:
ansible_host: smarthome.localdomain
ansible_ssh_user: markus
mqttbridge:
ansible_host: mqttbridge.localdomain
ansible_ssh_user: ubuntu
iobroker:
ansible_host: iobroker.localdomain
ansible_ssh_user: pi
media:
hosts:
hyperbian:
ansible_host: hyperbian.localdomain
ansible_ssh_user: pi
dev:
hosts:
devsrv:
ansible_host: dev.localdomain
ansible_ssh_user: markus
k8s:
hosts:
k8sm1.localdomain:
ansible_ssh_user: k8s
k8sm2.localdomain:
ansible_ssh_user: k8s
k8sn1.localdomain:
ansible_ssh_user: k8s
k8sn2.localdomain:
ansible_ssh_user: k8s
k8sn3.localdomain:
ansible_ssh_user: k8s
Within the file above, I’m defining basically four server groups - smarthome, media, dev and k8s. Within the smarthome group there are the smarthome server it self, running multiple containers for my home automation, an mqtt bridge which relays authenticated mqtt messages to my internal mqtt server and the iobroker which reads energy and water consumption data. Similar are the groups media and dev.
The group k8s consolidates my Kubernetes three node cluster.
A Playbook to start with - Update the Development Host
So, this is going to be a very easy playbook. Since in my example the development host I’m using is based on Ubuntu as well, I can simply use the Ansible module for apt to update all the packages to the latest versions.
vi ~/manager/update-dev.yaml
---
- hosts: dev
become: true
become_user: root
tasks:
- name: Update apt repo and cache on all Debian/Ubuntu boxes
apt:
update_cache: true
force_apt_get: true
cache_valid_time: 3600
- name: Upgrade all packages on servers
apt:
upgrade: dist
force_apt_get: true
- name: Check if a reboot is needed on all servers
register: reboot_required_file
stat: path=/var/run/reboot-required get_md5=no
- name: Reboot the box if kernel updated
reboot:
msg: "Reboot initiated by Ansible for kernel updates"
connect_timeout: 5
reboot_timeout: 300
pre_reboot_delay: 0
post_reboot_delay: 30
test_command: uptime
when: reboot_required_file.stat.exists
Being in the manager subdirectory we can easily run the playbook with
ansible-playbook --vault-password-file ../.vault-pass.txt update-dev.yaml
Variables, Secrets and the Ansible Vault
Our upcoming playbooks to require some secrets for authentication or other to be protected confidential information. We will store all that within the Ansible Vault, which eases the handling, encryption and decryption of this kind of information.
To create a vault which we can then include in our playbooks we simply run
ansible-vault create --vault-password-file ../.vault-pass.txt ./vars/secrets.yaml
If we ever need to change, add or remove a variable we run
ansible-vault edit --vault-password-file ../.vault-pass.txt ./vars/secrets.yaml
This will typically launch vi with an editable secrets file. As soon as you close the editor, the file will be automatically encrypted again.
An example for variables stored within the secrets file is shown below:
docker_hub_username: <DOCKER HUB USERNAME>
docker_hub_password: <DOCKER HUB PASSWORD>
A simple Menu to run the Playbooks and Edit the Secrets
Since I don’t like to always type long ansible-playbook commands I created a handy shell based menu to discover available playbooks and run it directly.
vi ~/manager/menu.sh
#!/bin/bash
env_options=("secrets" "playbooks" "last" )
echo 'Please choose: '
select opt in "${env_options[@]}"
do
case $opt in
"secrets")
ansible-vault edit --vault-password-file ../.vault-pass.txt ./vars/secrets.yaml
exit 0
;;
"playbooks")
env_playbooks=( $(find . -maxdepth 1 -name \*.yaml -exec basename {} .yml \; | sort ) )
echo 'Please choose: '
select opt in "${env_playbooks[@]}"
do
echo "running playbook ansible-playbook --vault-password-file ../.vault-pass.txt $opt"
echo "ansible-playbook --vault-password-file ../.vault-pass.txt $opt" > .last.sh
chmod +x .last.sh
ansible-playbook --vault-password-file ../.vault-pass.txt $opt
exit 0
done
;;
"last")
./.last.sh
exit 0
;;
*) echo "invalid option $REPLY";;
esac
done
Create a Container with Configuration Files
Here, we’re going to create a container on the target machine mqttbridge alongside to the necessary configuration files. The container will act as a mqtt bridge, relaying authenticated mqtt messages to an internal mqtt server.
---
- hosts: mqttbridge
# become: true
# become_user: root
tasks:
- name: Include Secrets
include_vars: "{{ item }}"
with_items:
- vars/secrets.yaml
- name: Create Subdirectories
become: true
become_user: root
file:
path: "{{ item }}"
state: directory
mode: '0750'
group: '1883'
owner: '1883'
loop:
- "/home/ubuntu/mosquitto/config/conf.d"
- "/home/ubuntu/mosquitto/config/etc"
- "/home/ubuntu/mosquitto/data"
- "/home/ubuntu/mosquitto/log"
- name: Create Mosquitto Configuration
become: true
become_user: root
copy:
dest: /home/ubuntu/mosquitto/config/mosquitto.conf
mode: '0640'
group: '1883'
owner: '1883'
content: |
persistence true
persistence_location /mosquitto/data/
user mosquitto
password_file /mosquitto/config/etc/passwd
allow_anonymous false
listener {{ mqtt_external_port }}
log_dest file /mosquitto/log/mosquitto.log
include_dir /mosquitto/config/conf.d
- name: Create Mosquitto Bridge Configuration
become: true
become_user: root
copy:
dest: /home/ubuntu/mosquitto/config/conf.d/bridge.conf
mode: '0640'
group: '1883'
owner: '1883'
content: |
connection dmz_int
address {{ mqtt_internal_ip }}:{{ mqtt_internal_port }}
topic # out
- name: Create Mosquitto Passwords
become: true
become_user: root
copy:
dest: /home/ubuntu/mosquitto/config/etc/passwd
mode: '0640'
group: '1883'
owner: '1883'
content: "{{ mqttbridge_passwd }}"
- name: Create Mosquitto Container
docker_container:
name: mosquitto
image: eclipse-mosquitto:2
volumes:
- /home/ubuntu/mosquitto:/mosquitto
- /etc/localtime:/etc/localtime:ro
- /etc/timezone:/etc/timezone:ro
published_ports:
- "{{ mqtt_external_port }}":"{{ mqtt_external_port }}"
# - 9001:9001
detach: true
recreate: true
restart_policy: always
state: started
container_default_behavior: no_defaults