How does Ansible dynamic inventory work with cloud providers like AWS or Azure?

Imagine you are a teacher taking attendance at a school where students transfer in and out every day. Instead of keeping a printed list that is outdated by lunchtime, you check the school's enrollment system each morning to get the current roster. Dynamic inventory works the same way -- instead of a static file listing hosts, Ansible queries your cloud provider's API in real time to discover what instances exist, what IPs they have, and how they should be grouped.

Dynamic inventory comes in two forms: inventory scripts and inventory plugins. The older approach uses executable scripts (usually Python) that output JSON in a specific format when called with --list or --host arguments. The newer and recommended approach uses inventory plugins, which are configured through YAML files. For AWS, you create a file like aws_ec2.yml with the plugin name, AWS region filters, and grouping rules. For Azure, you use the azure_rm plugin with subscription and resource group filters. Plugins are more powerful because they support composable host variables, tag-based grouping, and complex filtering without writing custom code.

Under the hood, when Ansible encounters a dynamic inventory source, it runs the plugin or script before any play starts. The plugin authenticates with the cloud API using credentials from environment variables (like AWS_ACCESS_KEY_ID or AZURE_SUBSCRIPTION_ID), credential files, or IAM instance profiles. It then makes API calls to list instances and their metadata -- instance IDs, private and public IPs, tags, regions, availability zones, and instance types. The plugin converts this metadata into Ansible inventory format: hosts get variables like ansible_host set to their IP, and groups are built automatically based on tags, regions, or any other attribute. For example, all EC2 instances tagged with Environment=production and Role=webserver automatically show up in groups named tag_Environment_production and tag_Role_webserver.

In production, dynamic inventory is essential for cloud-native setups where instances come and go constantly. Auto-scaling groups spin up and tear down servers, and Kubernetes nodes appear and disappear. Teams typically use tag-based grouping conventions -- for example, tagging every instance with Environment, Service, and Team tags, then building Ansible groups from those tags. The plugin config often includes filters to skip terminated instances, limit results to specific VPCs, and build custom host variables from instance metadata. You can also combine multiple inventory sources by pointing Ansible at a directory containing both static files and dynamic plugin configs.

A big gotcha is API rate limiting. If you have thousands of instances and run Ansible frequently, you can hit AWS throttling limits, causing inventory generation to fail. Use the cache plugin (cache: true with cache_plugin: jsonfile) to store inventory results locally and avoid repeated API calls. Another pitfall is credential management -- hardcoding AWS keys in inventory files is a security risk; always use IAM roles, environment variables, or HashiCorp Vault. Also watch for timing issues: if an auto-scaling event adds a new instance between inventory generation and task execution, that new host will not be targeted until the next Ansible run.