On 2025-07-31 01:27 +03, Pavel Korovin <p@tristero.se> wrote:
> Hello Florian,
>
> Can you provide some more details about the playbook which hangs on this
> particular host?

Not really, I have a unified playbook for all my infrastructure and
playbook + roles is about 3000 lines of yaml. I'm not comfortable
sharing that, it just has too much information about my infrastructure.

On a high-level, ansible does this:

- ssh in as unpriv user
- gain privileges with doas
- install packages / copy files / use templates / start services

The role where it hangs on this host looks like this, but note that it
hangs on the very first task:

------------------------------------------------------------------------

$ cat roles/nginx/tasks/main.yml
- name: nginx package
  ansible.builtin.package:
    state: present
    name:
      - nginx

- name: copy tarpit config
  ansible.builtin.copy:
    dest: "/etc/nginx/tarpit.inc"
    src: "tarpit.inc"
    owner: root
    group: wheel
    mode: 00644
  notify: restart nginx

- name: copy ai-bots config
  ansible.builtin.copy:
    # https://raw.githubusercontent.com/ai-robots-txt/ai.robots.txt/refs/heads/main/nginx-block-ai-bots.conf
    dest: "/etc/nginx/nginx-block-ai-bots.conf"
    src: "nginx-block-ai-bots.conf"
    owner: root
    group: wheel
    mode: 00644
  notify: restart nginx

- name: manage /etc/nginx/nginx.conf
  ansible.builtin.template:
    src: nginx.conf.jinja
    dest: /etc/nginx/nginx.conf
    owner: root
    group: wheel
    mode: 0644
    validate: '/usr/local/sbin/nginx -tc %s'
  notify: restart nginx

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    search_string: /var/www/logs/access.log
    line: /var/www/logs/access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644

- name: reload nginx on error.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    search_string: /var/www/logs/error.log
    line: /var/www/logs/error.log			644  7     250  *     ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    line: /var/www/logs/{{ item.domain }}.access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644
  loop: "{{ http.static }}"

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    line: /var/www/logs/{{ nextcloud.servername }}.access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644
  when: "'nextcloud' in group_names"

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    line: /var/www/logs/{{ icinga2.servername }}.access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644
  when: "'icinga2' in group_names"

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    line: /var/www/logs/{{ http.captive_portal_domain }}.access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644
  when: "http.captive_portal is defined and http.captive_portal"

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    line: /var/www/logs/{{ gitea.servername }}.access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644
  when: "'gitea' in group_names"

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    line: /var/www/logs/{{ vaultwarden.servername }}.access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644
  when: "'vaultwarden' in group_names"

- name: reload nginx on access.log rotation
  ansible.builtin.lineinfile:
    path: /etc/newsyslog.conf
    line: /var/www/logs/{{ item.src }}.access.log		644  4     *    $W0   ZB /var/run/nginx.pid SIGUSR1
    owner: root
    group: wheel
    mode: 0644
  loop: "{{ http.domain_redirects | default([]) }}"

- name: enable nginx
  ansible.builtin.service:
    name: nginx
    enabled: yes
    state: started

------------------------------------------------------------------------

All my other roles are basically the same, just for different services.

Since ansible gets stuck at different places for different hosts, to me
it looks like at some point it looses track what it's doing.


> Latest ansible is quite a major change, I've had to rewrite some of my
> playbooks which worked for years, and I had to drop some of
> jmespath-related

I don't know what that is. I'm happy to modernize my playbooks / roles
if there is stuff that's deprecated, but I don't think that's the issue
here.

> stuff, since it didn't work at all. Thing are moving forward, and we also have
> to follow the things.

Good luck, maybe someone else has a more minimal playbook they are
willing to share that can reproduce the issue. Or maybe upstream figures
out what's wrong.

I'll stick with the downgraded version for now and see if things improve
with newer releases.

Thanks,
Florian

>
> On 07/30, Florian Obser wrote:
>> after upgrading to -current, ansible breaks thusly with a fairly large
>> playbook:
>> 
>> Ansible is pretty far into the playbook at this point, running for a few
>> minutes. I'm managing all my infra using ansible. Different hosts hang
>> at different points in the playbook (because they do different things),
>> But the playbook always stops at the same point for a particular host.
>> 
>> Downgrading to ansible-11.7.0 / ansible-core-2.18.6 fixes this.
>
> -- 
> With best regards,
> Pavel Korovin
>

-- 
In my defence, I have been left unsupervised.