How do strategy plugins, pipelining, Mitogen, async tasks, and fork tuning speed up Ansible?

Think of a restaurant kitchen filling a big catering order. The linear strategy is one dish at a time across all plates -- every plate gets soup before any plate gets the main course. The free strategy lets each station work independently -- one plate might be on dessert while another is still on appetizers. The host_pinned strategy assigns each plate to one cook who handles it start to finish. Ansible strategy plugins offer these same execution models, and the right choice depends on whether tasks have dependencies between hosts or can run on their own.

The default linear strategy runs each task across all hosts before moving to the next one. This keeps things in a predictable order -- useful when tasks depend on each other across hosts, like setting up a primary database before its replicas. The free strategy drops this synchronization, letting each host run through tasks as fast as it can without waiting for others. This dramatically speeds up environments where some hosts are fast and others are slow. The host_pinned strategy is similar to free but keeps each host assigned to the same worker process for the entire play, which helps with connection reuse.

Under the hood, the performance bottleneck for large inventories is SSH overhead. By default, Ansible copies a Python module to the remote host as a temporary file, runs it, grabs the output, and deletes the file -- for every single task. Pipelining (pipelining = True in ansible.cfg) skips the file transfer by sending the module code directly through SSH's standard input, which can cut task time by 30-50 percent on high-latency connections. Mitogen takes this further by setting up a persistent Python process on the remote host through an SSH tunnel, completely eliminating per-task module transfers and reusing the Python process across tasks. Mitogen can deliver 4-7x speedups for playbooks with many tasks.

At production scale, fork tuning is the broadest lever. The forks setting (default 5) controls how many hosts Ansible manages at the same time. For 500 servers, forks=50 means 50 hosts process in parallel per task round. Higher forks use more memory and file descriptors on the control node since each fork is a separate Python process. Async tasks (async and poll parameters) let individual tasks run in the background on the remote host while Ansible moves on to other work, which is essential for long-running jobs like package upgrades or database backups that would otherwise block the entire play. Combining the free strategy, pipelining, high forks, and async tasks can cut a 500-host playbook from 45 minutes to under 10.

The non-obvious gotcha is that performance optimizations can break debugging and compatibility. Pipelining requires the requiretty setting to be disabled in sudoers on target hosts. Mitogen has compatibility issues with some connection plugins, callback plugins, and newer Ansible versions -- it tends to lag behind ansible-core releases. The free strategy makes play output non-deterministic, which complicates log analysis and makes it harder to connect failures to specific task ordering. Async tasks need explicit status checking with async_status and can leave orphaned processes if the playbook is interrupted. Architects should benchmark optimizations in staging before production and keep a fallback configuration for troubleshooting where linear strategy and verbose output are needed.