Of all the sessions I attended at re:Invent 2019, the one I found most interesting was about the Security Benefits of the Nitro architecture.
Although it might sound weird to hear a Cloud Consultant discuss hardware details, what makes it worth going down to this level is, that the Nitro architecture showcases in what ways AWS is innovating in the field of virtualization. I guess that because of this, Werner himself highlighted these features in his keynote.
The Xen hypervisor, which AWS originally used and that still powers some EC2 instance classes, has the concept of a dom0. Each host has one privileged virtual machine, dom0, that is allowed to interact with real hardware; this allows the hypervisor to remain small by offloading work to dom0.
This privileged virtual machine runs a Linux operating system that manages (disk and network) devices and exposes them to the non-privileged guest virtual machines. In dom0 AWS runs, among others, services supporting EBS, VPC networking, and CloudWatch.
The privileged dom0 virtual machine has a large attack surface as it shares hardware with the guest virtual machines and runs a fully-fledged operating system.
What AWS calls the Nitro system is a collection of custom build devices that take most of the work that normally happens in dom0 to support the virtual machines. Not only does offloading this work to the Nitro system leave more capacity for the guests (about 10% of EC2 host resources are regained), it also makes everything much more secure.
It consists of a couple of PCI-card like devices, in addition to this a special security chip is embedded on the motherboard. While implementing the Nitro system, AWS paid attention to a new security paradigm.
(Image taken from the re:Inforce 2019 slides of this talk.)
When building the Nitro system, AWS paid special attention to the following security precautions:
The functionality of the Nitro system is provided by various PCI-card like devices. This design is inspired by microservice software architectures.
Some types of devices include:
The Nitro controller manages the host, hypervisor, and the other Nitro devices in the system.
The Nitro controller:
The Nitro EBS subsystem provides EBS volumes as NVMe devices to the motherboard.
All encryption happens on the Nitro device, encryption keys are managed by KMS, encryption keys can not be accessed by the EC2 host.
The Nitro local storage subsystem provides ephemeral volumes as NVMe devices to the motherboard.
All encryption happens on the Nitro device, encryption keys are generated on, and never leave, the device.
After use, the volumes are cryptographically wiped, i.e., the keys are removed, the actual data in encrypted form remains on the physical disks. Before use, the first and last bytes of the volumes are zeroed to prevent confusing the guest operating system (as the device would still be filled with “random” data from previous uses).
The Nitro VPC subsystem provides network interface controller devices to the motherboard.
The VPC stack runs on the Nitro system; only the Nitro system has access to the private AWS network, the EC2 host and guests can only access the network via the Nitro system.
All traffic between Nitro powered instances is transparently encrypted on the Nitro system, traffic to non-Nitro instances is not encrypted as this would impact the performance.
The Nitro security chip guards the firmware of the motherboard in two ways:
The Security Benefits of the Nitro architecture session taught us: