Containers vs Virtual Machines

The isolation boundary is different. A virtual machine virtualises hardware: a hypervisor such as Nitro or KVM gives each VM virtual CPUs, memory and devices, and each VM boots its own operating system kernel. A container virtualises the operating system: containers are ordinary processes on one shared host kernel, isolated by namespaces, which control what they can see, and cgroups, which limit what they can use. Everything else follows. Containers start in milliseconds to seconds, images are megabytes rather than gigabytes and dozens fit on a host, but a kernel flaw affects them all. VMs are heavier and slower, but give stronger isolation and can run different kernels, including Windows.

Containers start much faster, typically in milliseconds to a few seconds, while a new VM takes tens of seconds to minutes. Starting a container only creates namespaces and cgroups and launches a process on a kernel that is already running; booting a VM involves firmware, a guest kernel, init and agents. For scaling, what users feel is time to serve traffic. In the note's figures, a new pod on an existing node served in 5 to 10 seconds for Java with CDS, mostly JVM startup, while a new EC2 instance took 60 to 90 seconds. A pod that needs a new node pays the VM cost as well, which is why spare node capacity matters.

Because all containers on a host share one kernel, and the kernel is a very large attack surface. Namespaces and cgroups are kernel features, so isolation is only as strong as the kernel code enforcing it. A container calls the host kernel directly through hundreds of system calls, and a vulnerability in any of them, or in namespace handling, can let a process escape and reach every other container on the node. Misconfiguration widens the gap: privileged containers, host mounts, broad capabilities or running as root. A VM talks to a much smaller hypervisor interface instead. Mitigations include seccomp profiles, dropped capabilities, non-root users, read-only filesystems and sandboxed runtimes.

A microVM is a minimal virtual machine designed to start almost as fast as a container while keeping hardware-level isolation. Firecracker, the open-source virtual machine monitor that AWS built on KVM, is the main example. It removes legacy devices and emulation and exposes only a few virtio devices, so a microVM boots in about 125 milliseconds with under 5 MiB of memory overhead. Each microVM runs its own small guest kernel, so one tenant's kernel exploit does not reach another tenant. AWS runs every Lambda execution environment and each Fargate task in its own Firecracker microVM, which is how it can safely place many customers' workloads on the same physical servers.

gVisor and Kata Containers are sandboxed container runtimes that add an extra isolation layer while keeping the normal container interface. gVisor, through its runsc runtime, runs a user-space kernel written in Go that intercepts the container's system calls and implements them itself, so the application never talks directly to the host kernel; it costs some performance on syscall-heavy and I/O-heavy workloads. Kata Containers runs each pod inside a lightweight VM with its own guest kernel, giving hardware-backed isolation, but needs bare-metal or nested virtualisation support. In Kubernetes you select either through a RuntimeClass, so only untrusted workloads, such as a renderer for user-supplied files, pay the overhead.

By patching or replacing the host nodes, not by rebuilding container images. An image contains only userland, libraries and the application; the kernel belongs to the node, so a kernel CVE is fixed at the node layer. On EKS this usually means publishing a new node AMI, such as an updated Bottlerocket or EKS-optimised image, and letting Karpenter drift or managed node group updates replace nodes gradually, with pod disruption budgets keeping services available. ShopNow handled a namespace-related kernel CVE exactly this way. Image rebuilds are still needed for library and base-image vulnerabilities, so patching responsibility splits between the platform team for nodes and service teams for their images.

In the public cloud the hypervisor is the isolation boundary between customers, and containers add density and fast deployment inside one customer's boundary. A provider cannot let two customers share a kernel, because a single kernel exploit would expose the other, so customer workloads always sit inside VMs on Nitro, either EC2 instances or Firecracker microVMs. Inside your own nodes, containers pack many trusted services onto each VM, giving high utilisation and quick scaling. ShopNow moved 40 services from 40 small VMs averaging 12% CPU to containers on 14 larger nodes at 55%, cutting compute cost by 48%. You get both layers: VM isolation between tenants and container efficiency within one.

Choose Fargate when you want each task or pod isolated in its own microVM and do not want to manage nodes at all: no AMIs, patching, node scaling or bin-packing decisions. It suits a modest number of services, partner jobs or workloads that need stronger isolation, like ShopNow's partner sync jobs. Choose EC2 nodes when you run many services at scale: bin-packing on EC2, especially with Graviton and Spot, is cheaper, roughly ₹2,900 against ₹6,800 per replica in the note's comparison, and pods start faster on existing nodes. EC2 nodes also support DaemonSets, GPUs, custom kernels and privileged agents, which Fargate does not. Many estates sensibly use both.

Never in plain containers sharing a kernel with other tenants or with your own services. Put each execution inside a hardware-isolated boundary: Lambda or Fargate, which run on Firecracker microVMs, or a Kubernetes RuntimeClass using Kata Containers or gVisor on dedicated, tainted nodes. Then add defence in depth, because the sandbox is only one layer: no network egress by default, no credentials or a minimal IAM role per execution, CPU, memory and time limits, read-only filesystems, and inputs and outputs passed through queues or S3. Make each environment disposable after a single job. ShopNow runs seller-uploaded image transformations in Lambda, so any exploit stays inside a short-lived microVM.

Stay with VMs when the workload needs something containers cannot provide cleanly. Examples are a different kernel or operating system, such as Windows-specific stacks; legacy software that expects a full OS with init systems, local state, cron jobs and agents; vendor software certified only on VMs; licensing counted per VM or per core, where containers would force dedicated nodes anyway; and isolation requirements a shared kernel cannot meet. ShopNow keeps a legacy Windows reporting tool on an EC2 Windows instance and three legacy applications on EC2. Moving such workloads into containers often costs more than it saves, so revisit the decision when the software is replaced or the vendor supports containers.