Automating ESP32 deployment via Ansible

Since a few weeks, I’ve done a lot of home automation, and especially of consumption tracking in the house (gas, electricity, water). During this process, I’ve discovered ESPHome, which simplifies a lot the programming of ESP32s and their integration with HomeAssistant.

But as, since a few years, I don’t deploy anything manually, using Ansible to automate everything (and simplify a lot the redeployment of anything, both at home or on my personal server), this was not enough and I want to be able to deploy my ESPs with Ansible.

In this post, I’ll share the relevant parts of my Ansible repository and explain what each part does for deploying the ESP32 that tracks the water meter. I hope it can help someone on the internet, even if this is really basic and probably anybody Ansibling their homes can do the same themselves. I’m sorry I’m not sharing a GitHub link or similar, this is git’d locally and I don’t want to duplicate.

Here is the Ansible tree:

srv-installer
├── playbook.yml
├── secrets.yml
├── inventory
│   ├── group_vars
│   │   └── lan.yml
│   ├── host_vars
│   │   └── esp-water-meter.lan.yml
│   └── master
└── roles
    ├── esphome
    │   └── tasks
    │       └── main.yml
    └── esp-upload
        ├── tasks
        │   └── main.yml
        └── templates
            └── esp-water-meter.yml.j2

The inventory/master file contains all the hosts I manage using this Ansible repository, using groups to mutualize variables (redacted to only show the relevant parts, for shortness):

[lan:children]
esp

[esp]
esp-water-meter.lan

The inventory/group_vars/lan.yml file contains a few variables used across the house, like the Wifi SSID:

wifi_ssid: my-wifi-access-point

The inventory/host_vars/esp-water-meter.lan.yml file contains the ESP-dependant variables, in this case only one, the model:

esp_model: esp-wrover-kit

The secrets.yml file contains the secrets, encrypted using ansible-vault (I’m not yet ready to deploy Hashicorp’s Vault or anything like that, it seems over-engineered, yes, even for me):

wifi_psk: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          6230...464313235613A32423082934579203475920380056639
          1342...464313239859234854657904247890234850348502639

This is it for variables. Now the main playbook calls various roles for various hosts or groups of hosts. The relevant part for esp deployment:

- hosts: esp
  vars_files:
    - secrets.yml
  gather_facts: no
  serial: 1
  roles:
    - { role: 'esphome', tags: ['esphome'] }
    - { role: 'esp-upload', tags: ['esp-upload'] }

I’m setting gather_facts to no, for obvious reason as ESPs don’t have ssh available, much less Python. You will see in the roles that every task is delegated to localhost, because my laptop will do all the work in reality.

I’m also setting serial to 1 because I want the playbook to run sequentially for each ESP32, so that I can plug it in via the USB/FTDI adapter if needed – for the first deployment, it is; afterwards, it is done over the air.

The first role is esphome and is very simple : it just installs the esphome software locally. Here is the roles/esphome/tasks/main.yml file:

---
- name: install esphome
  pip:
    executable: pip3
    name: esphome
    state: present
  become: yes
  delegate_to: localhost

The next role is the one that deploys the relevant yaml code file to each ESP. I am doing it by saving the source yaml as a template, which allows me to use variables for the wifi connections, or other things if needed, in the source file.

Here is the main (and only) tasks file for this role, roles/esp-upload/tasks/main.yml :

- name: build short name
  set_fact:
    short_name: "{{ inventory_hostname | regex_replace('\\.[a-z]+$', '') }}"
  delegate_to: localhost

- name: build source file
  template:
    src: "{{ short_name }}.yml.j2"
    dest: "/tmp/{{ short_name }}.yml"
  delegate_to: localhost

- name: compile source
  shell:
    chdir: /tmp
    cmd: "esphome compile {{ short_name }}.yml"
  delegate_to: localhost

- name: connect ESP
  pause:
    prompt: Please connect ESP to USB FTDI if OTA is not possible
  delegate_to: localhost

- name: upload to esp
  shell:
    chdir: /tmp
    cmd: "esphome upload {{ short_name }}.yml"
  delegate_to: localhost

First of all, we build a short_name variable. It will be used in the ESP source code, and to use the correct source template.

Then we prepare the definitive source file, replacing the variables in it with their values (see below for the source file).

Then we build the firmware using esphome compile.

At that point, the playbook pauses to let us physically connect the ESP32 if this is needed.

Finally, after I hit enter, it continues and uploads the firmware to the ESP32, using esphome upload.

In this playbook and inventory, there is a single ESP32, my water meter tracker ; but if – no, when! – I will add some, the playbook will then continue to the next one.

For completeness, here’s part of my water meter’s code (simplified for shortness), residing in roles/esp-upload/templates/esp-water-meter.yml.j2 :

esphome:
  name: {{ short_name }}

esp32:
  board: {{ esp_model }}
  framework:
    type: arduino

# Enable Home Assistant API
api:
  password: ""

#Yes, I should add passwords on both api and ota.
ota:
  password: ""

wifi:
  ssid: "{{ wifi_ssid }}"
  password: "{{ wifi_psk }}"

sensor:
  - platform: pulse_counter
    id: "pin14_pulse"
    pin:
      number: 14
      inverted: true
      mode:
        input: true
        pullup: true
    name: "pin14_pulse"
    count_mode:
      rising_edge: DISABLE
      falling_edge: INCREMENT
    use_pcnt: false
    internal_filter: 25ms
    total:
      unit_of_measurement: 'L'
      accuracy_decimals: 0
      name: "pin14_liters_index"

And here we are: I can now deploy code changes with a single command:

$ ansible-playbook --ask-become-pass --ask-vault-pass --inventory inventory/ --limit esp playbook.yml 
BECOME password: 
Vault password: 

PLAY [esp] *********************************************************

TASK [install esphome] *********************************************
ok: [esp-water-meter.lan]

TASK [esp-upload : build short name] *******************************
ok: [esp-water-meter.lan]

TASK [esp-upload : build source file] ******************************
changed: [esp-water-meter.lan]

TASK [esp-upload : compile source] *********************************
changed: [esp-water-meter.lan]

TASK [esp-upload : connect ESP] ************************************
[esp-upload : connect ESP]
Please connect ESP to USB FTDI if OTA is not possible:
ok: [esp-water-meter.lan]

TASK [esp-upload : upload to esp] **********************************
changed: [esp-water-meter.lan]

PLAY RECAP *********************************************************
esp-water-meter.lan       : ok=6    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Maybe I should give the hardware part of that water meter the same love the software part received?