Docker explained and illustrated
- Marius Ciepluch
- Former user (Deleted)
What is Docker?
Docker solves the problem of operating system environment consistency (packages, libraries, environment variables, configuration state etc.) and build qualification. It allows consistent setups, which can greatly add to reliable service operations.
Before container deployment schemes (like with Docker) convergent end-to-end management of build, test, release and production environments was often lacking, and unsuspecting issues could cause high diagnostics efforts (debugging, system analysis, root cause analysis etc.). Since nobody liked doing this live, in production, container solutions like Docker rose to popularity alongside DevOps. Better testability, reproducibility, convergence, end-to-end control… you name it
On a tech level Docker uses (Linux) kernel features and puts the known chroot concept on steroids. This way we can get a universal system to build and ship applications. Docker is the popular implementation, which has informed the OCI- specification and given rise to other runtimes such as Podman.
The technical foundation
The technical foundation topics concerning Docker are:
Container images
Image registries
Overlay file systems
Linux kernel features: namespaces, cgroups, capabilities
Network bridge interfaces
Routes and IPtables
Build environments
Execution environments
operating system (usually a minimal Linux) and runtime dependencies (of a release build)
What is Cloud Native?
Cloud Native denotes a macro trend related to the standardization of Linux service management and Linux-based cloud computing. It’s when services are made for the cloud (including on-premises private clouds as defined within NIST 800-145). Cloud Native services can be migrated between different kinds of cloud deployment scenarios, which may help with a defined onboarding, migration and exit strategy of the services in scope.
https://nvlpubs.nist.gov/nistpubs/legacy/sp/nistspecialpublication800-145.pdf
Docker under the hood
Unlike a virtual machine, a container does not need to boot the operating system kernel. Therefore, containers can be created in less than a second.
( Aug 25, 2021 ) The process isolation (which uses Linux namespaces, cgroups and other kernel features like capabilities happens within a libcontainer component called “runC”. Once a container gets instantiated, there is another runC instance.
Container images - layered deployable snapshots
Container images are layered deployable snapshots. They are self-contained portable environments which can instantiate a (Micro-)service. Subsequent versions of the same container images are stored as a differential overlay.
Overlay file systems - differential snapshots
Starting from a base image, overlay file systems allow maintaining a lower storage footprint because for the following container image versions it defines the differential. This is allocated onto the disk against the layered base image.
Container image snapshots capture a state, and usually get tagged as build / deploy / release image alike to builds from CI / CD. If you restart a container, changes made during its runtime get discarded.
cgroups - resource quotas
cgroups is a Linux kernel feature for resource compartmentalization.
https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt
Control Groups provide a mechanism for aggregating/partitioning sets of
tasks, and all their future children, into hierarchical groups with
specialized behaviour.
The manpage is man cgroups with an s.
Process segregation with Namespaces and Capabilities
Inside a container, Linux compartmentalizes all processes belonging to that… Namespace. It’s like a label attached to the activities which were instantiated with that running container.
Specific privileges can be assigned to select containers running capabilities. This is similar to running a packet sniffer such as Wireshark (tshark) as an un-elevated Linux user.
Top 3 security concerns
Privileged workstations or servers
Any docker host (such as a developer or DevOps workstation) which can (or does) run Docker containers as root is a privileged workstation or server. It can also mount the host operating system root and persist changes.
Solutions: rootless Docker
Software vulnerabilities
Any security vulnerability that’s reachable from the ENTRYPOINT process can potentially be reached with injection attacks from software exploits.
Solutions: Application Security including Supply-Chain security including SCA (Software Composition Analysis)
Secrets management and omni-capable automation systems
A container gets instantiated with environment secrets (which allow access to confidential, business or proprietary data). The environment secrets can be widely accessible, and often get supplied from omni-capable automation systems, which allow full end-to-end control over production from one single system.
Solutions: CI / CD with secure secrets management, CI and (Micro-)service architecture with segregation
Further concerns should be addressed based on system analysis and hardening. For generic security baselines the CIS benchmark can be helpful
Handy snippets
List dockerized processes
% docker ps marius@sofhelk
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c0ed60585f3a cyb3rward0g/helk-spark-worker:2.4.0 "./spark-worker-entr…" 2 months ago Up Less than a second helk-spark-worker
Prune Docker system
% docker system prune
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
If at the point of the prune action running containers are using a network, it will not get deleted.
Prune Docker containers
% docker container prune
WARNING! This will remove all stopped containers.
This will only remove stopped containers.
Stop all running containers of the host
If you want to delete the containers, you can issue rm instead of the stop. The snapshot-images will still be on the system.
Delete Docker snapshot-images
Given that the containers are stopped:
Alternatively: