Declarative vs Imperative

Declarative infrastructure means writing down the desired end state, for example a VPC with three private subnets in three zones, and letting a tool work out how to get there. Tools such as Terraform, CloudFormation and Kubernetes compare the declaration with what actually exists, compute the difference, and create, change or delete only what is needed. The author models resources and their relationships rather than writing steps, so ordering comes from dependencies, re-running is safe, and removing a block from the code removes the resource. The tool's plan or change set becomes the review artefact. That predictability is why declarative tooling is the default for cloud infrastructure in most organisations.

Imperative infrastructure means writing the ordered steps that create or change resources: call create-vpc, then create-subnet for each zone, then attach route tables, usually in a shell script, SDK program or CLI sequence. It is flexible, because any logic can be expressed, but the author owns everything a declarative tool would handle: checking whether resources already exist, running steps in the right order, recovering from partial failure, avoiding duplicates on re-run and writing explicit delete steps. Without those guards, re-running a script after a timeout can create a second NAT gateway, as happened at ShopNow. Imperative approaches still fit migrations, bootstrap and operational actions, which are naturally sequences.

Because they act on the difference between desired and actual state, not on a list of instructions. On every run, Terraform reads the configuration, refreshes the real state of each resource from the provider APIs, compares the two and plans only the changes needed to converge. If everything already matches, the plan is empty and apply does nothing, so running it twice has the same effect as running it once. Kubernetes controllers do the same continuously in a reconciliation loop. Idempotency also makes partial failure recoverable: if an apply fails after ten of fifteen resources, state records those ten, and the next apply creates only the remaining five, without duplicates.

Terraform builds a dependency graph from references between resources. When a subnet's vpc_id refers to aws_vpc.main.id, Terraform adds an edge meaning the VPC must exist before the subnet. It then walks the graph, creating independent resources in parallel, ten at a time by default, and destroying in reverse order. You never write the order yourself; it falls out of the data flow. When a dependency is real but not visible through attributes, such as an IAM policy that must exist before a service uses the role, you add an explicit depends_on. Overusing depends_on is a smell, because it hides why the order matters and serialises work that could run in parallel.

Imperative is appropriate when the work is genuinely a sequence of actions rather than a state to maintain. Database schema migrations are ordered, one-time steps, best run with versioned tools such as Flyway or Liquibase. Bootstrap tasks, like creating the Terraform state bucket and the first role, are chicken-and-egg problems solved once by a small, documented script, after which the result is imported into declarative code. Operational actions, such as failover, restarts or scaling during an incident, are actions, not state, and belong in runbooks or SSM Automation. Glue between tools fits pipeline steps. Keep imperative work at the edges, versioned and reviewed, and never use it to create core infrastructure.

Partly, and the distinction is worth making. CDK authoring is imperative: you write TypeScript, Java or Python with loops, conditions, classes and functions. But the output is declarative. cdk synth runs your code to produce a CloudFormation template, and CloudFormation then performs a normal declarative deployment with change sets, dependency ordering, rollback and drift detection. Your program describes what the template should contain; it does not call AWS APIs to create resources step by step. That gives you general-purpose language features such as typing, unit tests and abstraction, which suit teams generating many similar resources, while keeping the safety of declarative deployment. Pulumi works in a similar way with its own engine.

Push reconciliation applies changes on demand, typically from a CI pipeline running terraform plan and apply. Its strengths are explicit plans, clear review and change points, and an audit trail for every apply; its weakness is that drift goes unnoticed until the next plan or a scheduled drift job. Pull reconciliation runs controllers that watch desired state and continuously converge reality towards it, as Argo CD, Crossplane and AWS Controllers for Kubernetes do. That gives self-healing and fast drift correction, but previews are harder and you must operate the controllers themselves. Many organisations combine them, as ShopNow does: push-based Terraform for AWS infrastructure and pull-based Argo CD for Kubernetes applications.

Terraform provisioners, local-exec and remote-exec, run arbitrary scripts during resource creation, which smuggles imperative steps into a declarative tool. Their effects do not appear in the plan, so reviewers cannot see what will happen. They are not idempotent and not tracked in state: if a provisioner fails, the resource is marked tainted and replaced on the next apply, and if it succeeds, Terraform never re-runs or reconciles it. They also create hidden dependencies on the machine running Terraform, such as installed tools and network access. Use proper mechanisms instead: baked images, user data or cloud-init, configuration management, or a separate pipeline step. HashiCorp itself describes provisioners as a last resort.

Keep a declarative core and push imperative work to the edges, contained and orchestrated. All long-lived infrastructure is defined in Terraform, CloudFormation or Kubernetes manifests and changed only through reviewed plans. Imperative steps, such as database migrations, bootstrap scripts and operational actions, are versioned, idempotent where possible, and run by pipelines or SSM Automation rather than by people at a terminal. A pipeline sequences them, for example terraform apply, then a Flyway migration job, then an Argo CD sync. Ban out-of-band CLI changes to managed resources, because they create drift, and run drift detection regularly. PayGrid goes further, allowing no imperative changes to cardholder-data infrastructure except approved SSM runbooks.

The plan or change set. terraform plan, or a CloudFormation change set, shows exactly what will be added, changed, replaced or destroyed, computed against real state, so reviewers approve the effect of a change rather than guessing from a code diff. A summary such as 1 to add, 0 to change, 0 to destroy communicates intent, while any destroy or forced replacement deserves close attention, especially on stateful resources such as databases. In a pipeline, save the plan file, show it in the pull request, require approval, and apply exactly that saved plan so nothing changes between review and execution. Add prevent_destroy or policy-as-code checks on critical resources as a further safeguard.