Your organization has 500+ Terraform modules across 50 teams and terraform plan takes 45 minutes for the root module. How do you restructure the architecture and what strategies reduce plan/apply time at this scale?
Quick Answer
Decompose into smaller state files using component-based workspaces, implement dependency-based planning with targeted applies, use Terragrunt or custom orchestration for cross-state dependencies, and leverage Terraform Cloud/Spacelift for parallel execution with run triggers.
Detailed Answer
Root Cause Analysis
A 45-minute plan indicates a monolithic state file with thousands of resources. Terraform refreshes every resource in state during plan (API calls to the provider). With 500+ modules in a single state, you're making thousands of API calls sequentially. Additionally, the dependency graph computation becomes expensive for large graphs.
Decomposition Strategy
Split by blast radius and team ownership, not by environment. Instead of one root module, create component stacks: - networking/ (VPC, subnets, TGW) - changes rarely, affects everything - compute/ (EKS, ASG) - changes weekly - data/ (RDS, ElastiCache, S3) - changes monthly - platform/ (IAM, Route53, ACM) - changes rarely Each component has its own state file and can be planned/applied independently.
Cross-State Dependencies
Use terraform_remote_state data source or (better) use SSM Parameter Store / Terraform Cloud outputs as a decoupling layer. Terragrunt's dependency blocks automate this: dependency "networking" { config_path = "../networking" }. This creates a DAG of state files that can be applied in order.
Performance Optimizations
1. -target flag for focused plans (use sparingly, not as a habit) 2. -refresh=false when you know state is current (e.g., immediately after a previous apply) 3. Provider-level parallelism: parallelism=30 (default 10) for providers that support concurrent API calls 4. State file partitioning: Keep each state under 1000 resources for sub-5-minute plans
Organizational Governance
Implement a module registry (Terraform Cloud private registry or Artifactory) with semantic versioning. Enforce module version pinning. Use Atlantis or Spacelift for PR-based workflows with automatic plan-on-PR and policy checks before apply.