Docker and Kubernetes Security – Technical Companion

Docker & Kubernetes Security Mohammad-Ali A'râbi DoĘer and Kubernetes Security The battle for secure realms begins! Docker and Kubernetes Security Supply Chain Security & Runtime Protection by Mohammad-Ali A’râbi Foreword by: Liran Tal Director of Developer Advocacy, Snyk – GitHub Star Hamida Rebai Principal Solutions Architect, Quebec Government – Docker Captain v1.0.1 – Technical Companion Okterakt Verlag ii Docker and Kubernetes Security Edition 1.0.1 – Technical Companion, December 9, 2025 Copyright © 2025 Mohammad-Ali A’râbi All rights reserved. Published by Okterakt Verlag, printed by Amazon KDP. ISBN: 979-8-31-384581-4 (print) No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopying, recording, or otherwise— without prior written permission from the author. The information in this book is distributed on an “as is” basis, without warranty. Every effort has been made to ensure the accuracy of the content. However, the author and contributors shall not be held liable for any damages arising from its use. Docker® and the Docker logo are trademarks or registered trademarks of Docker, Inc. Kubernetes® is a trademark of The Linux Foundation. All other product names and brands are property of their respective owners. Use of these names does not imply endorsement. –Author: Mohammad-Ali A’râbi –Cover Design: Sima Maherani –Technical Editor: Vladimir Mikhalev –Technical Reviewers: Zoe Braiterman, Harpal Dhillon –Design & Layout: Mohammad-Ali A’râbi Docker and Kubernetes Security is an independently published book, crafted by a team of industry experts, reviewed by DevSecOps leaders, and supported by a community of Docker Captains, Kubernetes professionals, and security enthusiasts. This book was built with Pandoc and L A T EX, written in Markdown, and rendered with a lot of love, docker run, and git commit. The index was compiled by the author using imakeidx and a custom Pandoc Lua filter. iii To my wife Farnaz, my friends Melika, Ava, and Atousa, and the Iranian community of Freiburg without whom this book would have been completed months earlier. — Mohammad-Ali iv Contents About This Technical Companion xv Foreword by Liran Tal xvii Foreword by Hamida Rebai xix Praise for the Book xxi About the Contributors xxiii About This Book xxv Preface xxix Acknowledgements xxxi I Introduction to Docker 1 1 Introduction to Containers and Container Security 3 1.1 TL;DR....................................... 4 1.2 Technical Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Introduction to Containers . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.4 Introduction to Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.5 Container Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 1.6 Exploring What Lies Ahead . . . . . . . . . . . . . . . . . . . . . . . . . . 35 1.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 v vi CONTENTS 1.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2 Exploring Container Security Tools 39 2.1 TL;DR....................................... 39 2.2 Technical Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.3 Trivy ....................................... 40 2.4 Snyk ....................................... 49 2.5 Docker Scout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 2.6 Falco ....................................... 65 II Docker Supply Chain Security 69 3 Supply Chain Security for Application Development 71 3.1 TL;DR....................................... 71 3.2 Technical Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 3.3 Selecting Third-Party Libraries and Frameworks . . . . . . . . . . . . . . . 73 3.4 Keeping Your Dependencies Up to Date . . . . . . . . . . . . . . . . . . . . 80 3.5 Signing Your Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.6 Static Code Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 4 Secure Docker Image Building Practices 103 4.1 TL;DR....................................... 104 4.2 Technical Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.3 Dockerfile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.4 Base Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 4.5 Sign Docker Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.6 Installing OS Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 4.7 Installing Application Dependencies . . . . . . . . . . . . . . . . . . . . . . 112 4.8 Generating SBOM Upon Build . . . . . . . . . . . . . . . . . . . . . . . . . 115 4.9 Running as Non-root . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.10 Docker Init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 4.11 Building Docker Images in CI/CD . . . . . . . . . . . . . . . . . . . . . . . 122 4.12 Automating Vulnerability Fixes . . . . . . . . . . . . . . . . . . . . . . . . 124 4.13 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Listings 1 Hello World in Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii 1.1 Pinger Dockerfile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.2 Helm Chart Values File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.1 GitHub Actions workflow for Trivy . . . . . . . . . . . . . . . . . . . . . . 45 2.2 GitHub Actions workflow for Snyk . . . . . . . . . . . . . . . . . . . . . . 54 2.3 GitHub Actions workflow for Docker Scout . . . . . . . . . . . . . . . . . 61 2.4 GitHub Actions workflow for Docker Scout compare . . . . . . . . . . . . 62 3.1 HTTP client interface in TypeScript . . . . . . . . . . . . . . . . . . . . . . 76 3.2 Axios HTTP client implementation in TypeScript . . . . . . . . . . . . . . 76 3.3 User service in TypeScript using HTTP client interface . . . . . . . . . . . 78 3.4 Dockerfile for Python application . . . . . . . . . . . . . . . . . . . . . . . 84 5.1 GitHub Actions workflow to build and push a Docker image . . . . . . . . 139 5.2 Extracting Docker metadata using Docker Metadata Action . . . . . . . . 143 5.3 Complete GitHub Actions workflow to build and push a Docker image . . 144 5.4 GitHub Actions workflow to push the image to multiple registries . . . . . 148 5.5 GitLab CI pipeline to build and push a Docker image . . . . . . . . . . . . 150 5.6 GitLab CI pipeline with default stage for Docker jobs . . . . . . . . . . . . 151 5.7 GitLab CI pipeline to build a Docker image using Kaniko . . . . . . . . . . 152 5.8 GitLab CI pipeline to build a Docker image with SBOM . . . . . . . . . . . 153 6.1 Kubernetes Pod Security Context example . . . . . . . . . . . . . . . . . . 164 6.2 Kubernetes Pod Security Context with Linux capabilities . . . . . . . . . . 165 6.3 Kubernetes Pod Security Context with Seccomp profile . . . . . . . . . . . 166 6.4 Kubernetes Seccomp Profile example . . . . . . . . . . . . . . . . . . . . . 167 6.5 Kubernetes Pod Security Context with SELinux context . . . . . . . . . . . 168 6.6 Kubernetes Pod Security Context with AppArmor profile . . . . . . . . . . 169 6.7 Kubernetes namespace with Pod Security Admission configuration . . . . 175 xiii xiv LISTINGS 6.8 Kubernetes Pod with privileged container . . . . . . . . . . . . . . . . . . 175 6.9 Kubernetes Secret example . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 6.10 Kubernetes Pod with Secret volume mount . . . . . . . . . . . . . . . . . . 178 6.11 Kubernetes Pod with Secret as environment variable . . . . . . . . . . . . 178 6.12 Kubernetes SecretProviderClass for AWS Secrets Manager . . . . . . . . . 180 6.13 Kubernetes Pod with AWS Secrets Manager CSI Driver . . . . . . . . . . . 180 6.14 Kubernetes Pod with AWS Secrets Manager CSI Driver and Secret as environment variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 6.15 Kubernetes NetworkPolicy example . . . . . . . . . . . . . . . . . . . . . . 183 6.16 Kubernetes NetworkPolicy deny all example . . . . . . . . . . . . . . . . . 184 7.1 Kubelet configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 7.2 Kubelet configuration file with TLS authentication and authorization . . . 191 7.3 Kubelet configuration file with authorization . . . . . . . . . . . . . . . . . 191 7.4 Role resource for creating pods . . . . . . . . . . . . . . . . . . . . . . . . 195 7.5 RoleBinding resource for binding the pod-creator role to a user . . . . . . 195 8.1 Prometheus reporter service written in TypeScript . . . . . . . . . . . . . . 207 8.2 Kubernetes Service and ServiceMonitor for Prometheus . . . . . . . . . . . 208 8.3 Kubernetes pod spec with health checks . . . . . . . . . . . . . . . . . . . 211 8.4 Health check endpoint in Node.js . . . . . . . . . . . . . . . . . . . . . . . 212 8.5 Kubernetes audit policy file . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 8.6 Falco rule to detect a container running as root . . . . . . . . . . . . . . . 216 8.7 Falco rule to detect a container executing a binary not present in the image 216 8.8 Tetragon rule to check binary execution in /tmp directory . . . . . . . . . 217 8.9 Tetragon policy to block executing binaries from /tmp . . . . . . . . . . . 222 About This Technical Companion This document is a freely available technical companion to the book Docker and Kubernetes Security (ISBN: 979-83-13845814), published by Okterakt Verlag. It contains selected chapters and foundational material intended for academic reference, evaluation, and citation. The complete commercial edition includes additional chapters, in-depth case studies, extended exercises, and future updates, and is available at: DockerSecurity.io1. Please reach out to the author at DockerSecurity.io/contact2for any questions or feedback regarding this companion, as well as bulk licensing for educational institutions. 1https://DockerSecurity.io 2https://DockerSecurity.io/contact xv xvi ABOUT THIS TECHNICAL COMPANION Foreword by Liran Tal Developers and infrastructure engineers are increasingly expected to be proficient in both building and securing applications. However, knowing where to start can be a significant challenge. In my experience, many professionals find themselves overwhelmed by the sheer volume of resources available online, often leading to confusion and uncertainty about best practices. Even more so, navigating the complexities of Docker, Kubernetes, and the myriad of security challenges can be a daunting task. As a peer of Mohammad in the DevOps and DevSecOps community, I’ve witnessed firsthand his commitment to demystifying this critical domain, and this book is a testament to that passion. This is precisely where this work excels. Mohammad doesn’t just present theory; he provides a practical, hands-on roadmap. Your container security journey will begin with a solid grounding in Docker, making the content accessible to newcomers, before expanding to map out the entire cloud-native landscape. The true power of this book lies in its actionable security guidance. You’ll find DevSecOps-ready recipes for integrating security scanning into your CI pipelines, mastering reporting standards from SARIF to SBOMs, and implementing the best practices that are non-negotiable for any modern security organization, from signing images to securely managing secrets in Kubernetes. As a Docker Captain, Mohammad is more than just a participant in this ecosystem; he is a steward. His deep understanding of the cloud-native attack surface is evident on every page, equipping readers not just with “how-to” knowledge, but with the critical “why” behind securing every link in the supply chain. This book is an essential resource for anyone serious about building and deploying secure applications, and I am confident it will empower you to navigate the complexities of modern software security with clarity and confidence. —Liran Tal, September 2025 xvii xviii FOREWORD BY LIRAN TAL Foreword by Hamida Rebai In today’s fast-changing world of software development, containers have sparked a big change. They allow companies to create, build, and deploy apps faster more and more than ever before. Docker and Kubernetes are leading this shift becoming key tools in modern software creation. But while these technologies offer many advantages, they also bring new security issues that must be addressed to keep our systems and data secure. This book is your essential guide to navigating the complexities of container security, with a focus on mastering the protection of Docker and Kubernetes environments. It gives readers the know-how and practical skills needed to ensure the security of their containerized applications and infrastructure. The book covers everything from detecting vulnerabilities in Docker images to mastering the complexities of Kubernetes security. If you’re a developer on a DevOps team or working in security, this book gives you the information and tools you need to secure Docker containers and Kubernetes clusters. Each chapter provides practical steps, from identifying and fixing vulnerabilities in Docker images to implementing advanced security measures in orchestrated environments, that you can immediately implement in your work. We start with a deep dive into container security tools and how to find and fix vulnerabilities in Docker images. Then we cover supply chain security, secure Docker image building, and integrating security into CI/CD pipelines. As you go through the book you’ll learn to secure Docker containers on Kubernetes, manage Kubernetes cluster security, and implement robust logging, monitoring, and auditing to have full observability of your environments. In the upcoming chapters, you’ll learn how to design an application architecture that’s secure, resilient, and scalable, with Dockerized applications. And the final chapter looks far into the future, exploring emerging trends in Docker and Kubernetes security including xix xx FOREWORD BY HAMIDA REBAI shift-left security, serverless containers and the growing role of DevSecOps. By the end, you’ll be prepared to embrace the future of security and keep ahead of changes so you can remain an expert in your field. What distinguishes this book is that it is extremely hands-on. It provides a step-by-step guide, providing users with real-world recipes and best practices to achieve actual implementation. The chapters on logging, monitoring, and auditing are essential to address security breaches as they track the behavior of a system when issues arise. The rapid rise of containerized applications as the major component of next-generation infrastructure at enterprises is simply leading to a higher engagement and importance of securing those containerized environments. The expansive ecosystem of real-world threats, the moment that you begin working on next-generation infrastructure, requires the knowledge and the core components that this book delivers: a collection of useful strategies, techniques, best practices, and open-source security tools that will help you design, implement, deploy and constantly manage secure containerized applications within Docker containers and Kubernetes clusters. Therefore, please keep in mind that security is not a task performed once; it is more like a state of vigilance and continuous improvement. The knowledge you gain from this book will help you with today’s challenges and future threats. Enjoy and good luck with securing Docker and Kubernetes. —Hamida Rebai Trabelsi, August 2024 Praise for the Book “The Container Security Book You Need!” This book is an absolute masterclass that cuts through the complexity of cloudnative security. It covers everything from building trusted Docker images to locking down Kubernetes with RBAC, Network Policies, and advanced Runtime Defense. It’s clear, practical, and immediately actionable. No more guesswork, just a proven roadmap to secure container deployment. If you want to become an expert in Docker’s deployment and truly secure your cloud-native workloads, this book is for you! —Francesco Corti, Product Manager at Docker Inc. “A practical, must‑read guide that equips you with immediate skills for securing Docker and Kubernetes.” —Pradumna Saraf, Docker Captain and Open Source Contributor “The New Docker and Kubernetes Security Standard Reference!” I know I will carry this book to work for the next few years and I dare to say that, with regard to Docker and Kubernetes security, this work deserves to become the standard reference. —Clemens Scholz, Senior Platform Engineer at Endress+Hauser Group xxi xxii PRAISE FOR THE BOOK Preface What kind of word is “preface” anyway? Sounds like a shape-shifting super-villain. But it’s just a fancy word for “fore-talk”5, so let’s talk. So, what’s this book about? It’s about Docker, Kubernetes, and security. Let’s break it down: •Docker is a thing that runs your code in a container. •Kubernetes is a thing that runs your containers in a cluster. •Attacker is a bad guy who wants to mess with your stuff. •Security is a thing that stops the attacker from messing with your stuff. • This book is a thing that helps you understand security for Docker and Kubernetes. What You’ll Learn It’s probably easier if you just read the book and see for yourself. But here’s a sneak peek: • How to find and fix vulnerabilities in Docker images. • How to secure your Docker image building process, so that Jack can’t add a bitcoin miner to your app. • How to integrate security into your CI/CD pipeline. • How to stop Jack from accessing your Kubernetes cluster and mining bitcoins. • How to monitor and audit your containers, so that you know when Jack is mining bitcoins. So, it’s all about Jack and bitcoins. When I initially wrote these words, I thought I was joking. But Jack really hijacked a few NPM packages to reroute transactions to his bitcoin 5The -face part in preface means “to speak, tell, say”, from Proto-Indo-European root *bha-, from which are also the words phone (e.g. in telephone), photo-, and fatality. xxix xxx PREFACE wallet. So, be careful with Jack. Also, a good thing about Kubernetes is, that it’s changing so often. So, the last book on Kubernetes security is already outdated. This one isn’t—yet. I hope. Well, if you buy it, I send you emails with updates. So, you’re safe. And I publish them also on the book’s GitHub repo, so if I die, you can still get the updates. (TBH, if I die, the info on the repo is probably outdated too.) Why I Wrote This Book One sunny day back in 2023, this publisher came and said I should write a book on Docker security. I said, “No, I’m good”. They said, “Please?”, I said, “OK, sure.” And then I started writing it. Two years later, the person who said “Please?” walked away, the project manager left, and the publisher finally said, “We don’t want your book.” And I said, “OK, I’ll publish it myself.” And here we are. It’s even better now, because they forced me to use Word, and now I can use Markdown. So, it’s a win-win. While working with the publisher, I completed the manuscript, which was then edited and reviewed by Zoe Braiterman and Harpal Dhillon. This phase lasted approximately a year and a half and is known as the “First Era” of the book. The “Second Era” began when the publisher and I parted ways (i.e. the fall of Morgoth). I called upon Vladimir Mikhalev, Docker Captain and Snyk Ambassador, who agreed to be the book’s technical editor. His knowledge breathed new life into the project. Many others from my network joined the fellowship—including Julian König, aka Strider—and together we began forging in the halls of Eregion—known in the common tongue as GitHub. Yet even as we forged our knowledge, a shadow was rising in the East (in the fortress of us-east-16): the growing armies of CVEs… 6The us-east-1 really went down briefly after I published this book, killing half of internet. Acknowledgements This book would not have been possible without the support, feedback, and encouragement of many amazing people. Let’s take a moment to acknowledge them, chronologically: •Surbhi Suman, who first reached out and encouraged me to write this book. •Sepideh Bahramian, my old friend, who read the first chapter draft and encouraged me to keep going. •Kateryna Gridina, my boss at Jodel when I began writing, who supported me and gave me lots of reading material on book writing (including one titled No One Wants to Read Your Shit). •Ashwini Gowda, an unstoppable force behind the early stages of this book — she somehow managed to move the unmovable object (me) and get words on the page. •Apramit Bhattacharya, the book’s editor for more than a year, who asked me to remove many code snippets… which I eventually put back in. •Zoe Braiterman, Snyk Ambassador, who generously agreed to be a technical reviewer in the early days. •Harpal Dhillon, DevSecOps specialist and second technical reviewer, whose feedback made several sections far better. •Hamida Rebai, Docker Captain and author, who kindly wrote the foreword for this book. •Vladimir Mikhalev, Docker Captain and Snyk Ambassador, who jumped in as technical editor after I began self-publishing — a true hero of the second age. •Sima Maherani, the cover designer, for the awesome design and keeping up with my crazy ideas. •Florian Gössele, my colleague at JobRad, who read many chapters and gave thoughtful feedback (like pointing out the typo in hello wowlrd). xxxi xxxii ACKNOWLEDGEMENTS •Farzam Tahmasebmirza, who read the full manuscript and provided invaluable feedback. He will be the first one to add the book to his Goodreads shelf. •Jonas Scholz, fellow Docker Captain, who provided valuable feedback on the second chapter, writing down the comments with his own hands, which I found very impressive but hard to read. •Julian König, a fellow Freiburger Docker Captain who reviewed the first chapter like a machine! •Liran Tal, Director of Developer Advocacy at Snyk, who stepped in like a true hero to write the foreword at the last possible moment. •All the beta readers, who read the book in its early stages and provided invaluable feedback. Your insights helped shape this book into what it is today. If you’ve read the beta version of this book and provided feedback, but your name isn’t here, please reach out — I’d love to thank you properly. If you’ve read the beta version and didn’t provide feedback, the clock is ticking! Part I Introduction to Docker 1 Chapter 1 Introduction to Containers and Container Security Docker was introduced in 2013 in a short talk called The Future of Linux Containers1. Solomon Hykes, co-founder of Docker, Inc., wrote the following line into his terminal: $ docker run busybox /bin/echo hello world Then said: So what just happened here is that it generated a new Linux container and allocated a new file system for it, mounted a read/write layer, allocated a network interface, set an IP for it, and then executed the process in there, captured its output and printed it to me. This command encapsulates what Docker brought to the table. Linux Containers already existed, but were cumbersome to use. Docker introduced a clean CLI, a daemon to manage images and containers, and a registry for sharing them — all with sane defaults. That simplicity opened containers to a wider audience beyond sysadmins. After the hype of Docker, a few Google employees started a new project called Kubernetes. It was a container orchestration system, designed to manage containers at scale. It was open-sourced in 2014 and became the de facto standard for container orchestration in production environments. 1The talk is available on YouTube: https://youtu.be/wW9CAH9nSLs 3 4CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY 1.1 TL;DR This chapter introduces the world of containers, Docker, Kubernetes, and container security: 1. Docker’s Birth: Docker revolutionized Linux containers in 2013. 2. Kubernetes Emergence: Google’s Kubernetes, open-sourced in 2014, became the standard deployment platform for containers. 3. Container Basics: Containers are isolated processes that use Linux namespaces and cgroups for isolation and resource management. 4. Linux Namespaces: Different types of namespaces isolate processes, networks, mounts, hostnames, and IPC resources. 5. Docker vs. VMs: Containers share the host’s kernel, making them lightweight compared to virtual machines. 6. Container Runtimes: Docker isn’t the only runtime; others like containerd and CRI-O exist. 7. Kubernetes Evolution: Note that as of Kubernetes 1.20 (released in late 2020), Docker is no longer supported as a runtime out-of-the-box. Kubernetes now uses runtimes like containerd or CRI-O, though you can still use Docker for building images. 8. Installing Docker: Docker is the most popular development tool, and runs on Linux, macOS, and Windows. 9. Running Containers: Start with simple commands like docker run hello-world and explore image tags and digests. 10. Introduction to Kubernetes: Kubernetes is complex but powerful. 11. Installing Kubernetes: Explore local Kubernetes options like Docker Desktop, Minikube, or Kind. 12. Kubernetes Building Blocks: Pods, services, deployments, namespaces, and ingress. 13. Hello World with Kubernetes: Create and manage pods with kubectl, explore logs and use Helm charts for package management. 14. Container Security: Learn the importance of software bill of materials (SBOMs) and vulnerability checking. This chapter sets the stage for a deeper dive into container technology and security considerations. 1.2. TECHNICAL REQUIREMENTS 5 1.2 Technical Requirements We will need the following technologies and tools for the content and exercises in this chapter: • Docker • Kubernetes • kubectl • Helm Docker Desktop is the easiest way to get started with Docker and Kubernetes on macOS, Windows, and Linux. It offers full support for Apple Silicon chips and integrates seamlessly with modern development environments. You can download it for your operating system from the Docker website2. It comes with Docker and a built-in Kubernetes cluster. If you want to install Docker natively on Linux, you can follow the instructions on the Docker website3. Then you can also use Minikube or Kind for a local Kubernetes cluster. To install kubectl (which is the Kubernetes CLI tool), you can follow the instructions on the Kubernetes website4. And finally, Helm can be installed best by following the instructions on the Helm website5. 1.3 Introduction to Containers In 2015, I joined a new company, hired as a backend engineer, and the main language there was Python. My onboarding to the project took forever as it was almost impossible to set up the dependencies on my machine. A colleague of mine suggested that I use Docker. It was 2 years old at the time and I had never heard of it. It took me 2 weeks to learn the new technology and fix my local setup. A month later, I was containerizing different projects at the company and making CI/CD pipelines for them. Arguably the most important thing about containers is their isolation — they bundle everything an app needs to run, from OS packages to runtimes. If you have two different applications using clashing versions of Python, you can run them in two different con2https://www.docker.com/products/docker-desktop/ 3https://docs.docker.com/engine/install/ 4https://kubernetes.io/docs/tasks/tools/install-kubectl/ 5https://helm.sh/docs/intro/install/ 6CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY tainers and make them both happy. In that sense, containers are similar to Python virtual environments, but more sophisticated: •Support for all packages: Python virtual environments only support Python packages, but containers support all kinds of packages, including OS packages. You can use containers to package Linux applications, C libraries, and even Java applications, all together. •Shareable: You can share containers with others. Once you have packed your application and its dependencies into a container, you can share it with others. This package is called a Docker image6. So, no more “it works on my machine” problems! •Runnable: Another key difference here is that, you can — and probably should — run your application in a container. There are a lot of mechanisms and tools to run your application in this isolated environment. When running a container, Docker orchestrates low-level components to manage networking, storage, and resource limits — and hides all that complexity from the developer. As Solomon Hykes mentioned in his talk, it simplifies container execution. Now that we’ve covered the concepts, let’s try some Docker commands in practice. We’ll first go through Docker basic commands quickly, and then dive deeper into what containers really are behind the curtains. 1.3.1 Docker 101 Let’s go back to the command Solomon Hykes ran in his terminal. $ docker run busybox /bin/echo hello world This command has two parts. The first part is docker run busybox. It tells Docker to run a container based on the busybox image7. The second part is /bin/echo hello world. It tells Docker to run the /bin/echo command inside the container and print hello world to the standard output. 6You could probably argue that these images are not “Docker”-specific, but are rather “container” images. Well, I would say “container image” is rather a confusing term for me, as these things are not images of the containers, but rather images that are used to create containers. Also, we have different types of images in this sense, take OCI (Open Container Initiative) images for example. So, I would rather stick with “Docker image” for now. 7BusyBox is a set of Linux tools packed together. You can compare BusyBox to Alpine, except that Alpine is a proper Linux distribution. BusyBox is roughly 2MB in size and Alpine is now typically 7-10MB. 1.3. INTRODUCTION TO CONTAINERS 13 can run the busybox image and override the default command with /bin/echo hello world. A Docker image name consists of the following parts: <registry>/<organization>/<image-name>:<tag> An example is the following: ghcr.io/dockersecurity-io/book-chapter02:master In this example: •ghcr.io is the registry. It’s the GitHub Container Registry. •dockersecurity-io is the organization. It’s the GitHub organization for the book. •book-chapter02 is the image name. •master is the tag. Here it’s the branch name, but it can be any tag. If the registry is not specified, Docker will use the default registry, which is Docker Hub. If the organization is not specified, Docker will use the default organization, which is library. If the tag is not specified, Docker will use the default tag: latest. This is why when running the hello-world image, we didn’t specify the registry or the organization or the tag, and it started pulling library/hello-world:latest (well, docker.io/library /hello-world:latest to be precise). Now let’s run the hello-world image with a specific command: $ docker run hello-world /bin/echo hello world The output should be something like this: 1docker: Error response from daemon: failed to create shim task: OCI runtime 2create failed: runc create failed: unable to start container process: exec: 3"/bin/echo": stat /bin/echo: no such file or directory: unknown. It says that the /bin/echo command doesn’t exist. This is expected, as the hello-world image doesn’t have normal Linux commands. It’s a minimal image that only has the hello command. Note.Here Docker daemon says it cannot create a shim task. The shim task failed because, in turn, OCI runtime failed to create a container. And yet again that’s because runc failed to 14 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY start the container process. And finally, the container process failed to start because it couldn’t find the /bin/echo command. This is a good example of how the container runtime stack works. Shim is a small piece of code that acts as a proxy between the container runtime and the container process. It’s used to handle signals and other low-level stuff. We’re going to test this command with the ubuntu image, but to do so, we’re going to pull it first: $ docker pull ubuntu The output should be something like this: 1Using default tag: latest 2latest: Pulling from library/ubuntu 35af00eab9784: Already exists 4Digest: sha256:0 bced47fffa3361afa981854fcabcd4577cd43cebbb808cea2b1f33a3dd7f508 5Status: Downloaded newer image for ubuntu:latest 6docker.io/library/ubuntu:latest This command downloaded the ubuntu:latest image from the Docker Hub. We can download a specific version of the image by specifying the version after the image name. For example, we can download the ubuntu:24.04 image: $ docker pull ubuntu:24.04 You can find the available versions of the ubuntu image on the Docker Hub15. Note that ubuntu:24.04 also doesn’t always download the same image. It downloads the latest image for Ubuntu 24.04. It can download a different minor version of Ubuntu 24.04. For example, it can download ubuntu:24.04.3 instead of ubuntu:24.04.4. The only way to download a specific image is to use the image digest. Now that we have downloaded the ubuntu image, let’s learn about its digest: $ docker image inspect ubuntu It returns a huge JSON object. We’re only interested in the RepoDigests field16: 15https://hub.docker.com/_/ubuntu 16Here, the digests are not exact and only for illustration purposes. 1.3. INTRODUCTION TO CONTAINERS 15 1"RepoDigests": [ 2"ubuntu@sha256:33a5cc25d22c45900796a17cb09f09ea00b779e3b2026b4fc2faba" 3] We could download the same image with the following command: $ docker pull ubuntu@sha256:33 a5cc25d22c45900796a17cb09f09ea00b779e3b2026b4fc2faba This command will always download the same image, regardless of the version of Ubuntu 24.04 or their tags. Now let’s run the ubuntu image: $ docker run ubuntu It doesn’t do anything and exits. This is expected, as the default command for the ubuntu image is bash. We can override the default command with /bin/echo hello world: $ docker run ubuntu /bin/echo hello world We can also attach our terminal to the container’s terminal by passing the -it flag: $ docker run -it ubuntu This will attach our terminal to the container’s terminal. We can run commands inside the container now. For example, we can run the ls command: root@c7593865b1ac:/# ls The output should be something like this: 0bin boot dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var You can exit the container by typing exit or pressing Ctrl + D. This will also stop the container. To have the container running in detached mode, you can pass the -d flag: $ docker run -d ubuntu /bin/sh -c "while␣true;␣do␣echo␣hello␣world;␣sleep␣1; ␣done" Docker will return a hash to you, that is the container ID. Now you can attach your terminal to the container’s terminal with the docker attach command: 16 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY $ docker attach <container-id> It will start writing down hello world every second. Doing a Ctrl + C will kill the process now, hence the container. To attach your terminal to the same container, but not the same process on it, you can use the docker exec command: $ docker exec -it <container-id> /bin/bash This will run another Bash instance on the container and attach your terminal to it. You can run the ls command again: root@c7593865b1ac:/# ls And exiting from it won’t kill the container. 1.3.5 Exercises 1. Install Docker on your machine. Create an account on Docker Hub and log in to it with the docker login command. 2. Run the hello-world container. Inspect the container with the docker inspect command. What’s the image name? What’s the image digest? 3. Create your own Docker image. This image will ping 8.8.8.8 every second. Create a new directory and in it create a file called Dockerfile17 with the content below: 1FROM ubuntu:24.04 2 3RUN apt-get update && apt-get install -y iputils-ping 4 5CMD ["/bin/sh", "-c", "while true; do echo Hello World; sleep 1; done"] Listing 1.1: Pinger Dockerfile Build the image with the docker build command in the same directory as the Dockerfile: $ docker build -t pinger . Run the image with the docker run command: 17Yes, Dockerfile has no specific ending. If it’s called Dockerfile, it’s a Dockerfile. 1.3. INTRODUCTION TO CONTAINERS 17 $ docker run pinger You can stop the container with the Ctrl + C command. You can also stop it with the docker stop command: $ docker stop <container-id> 4. Tag the pinger container with your own Docker ID and push it to the Docker Hub: $ docker tag pinger <docker-id>/pinger $ docker push <docker-id>/pinger Now stop all the running containers with the docker stop command: $ docker stop $(docker ps -q) And then do a prune of the images and containers: $ docker system prune -a Now run the pinger container again: $ docker run <docker-id>/pinger It should start pinging. Kill it with the Ctrl + C command. 5. Run the pinger container again, but this time in the background: $ docker run -d <docker-id>/pinger List the running containers with the docker ps command: $ docker ps Execute the following command on the container to get the process IDs of the processes running inside the container: $ docker exec <container-id> ps aux The output should be something like this: 18 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY Table 1.2: Output of the ps aux command on the container User PID %CPU %Mem Start Time Command root 1 0.0 0.0 12:39 0:00 ping 8.8.8.8 root 7 0.0 0.0 12:43 0:00 ps aux You can see that the process ID 1 is the ping command. You can also see that the process ID 7 is the ps aux command. Now let’s try to find the same process IDs on the host machine. Execute the following command on the host machine: $ps aux | grep ping If you’re running Docker Desktop, the command will return only one process which is the grep itself. This is because Docker Desktop runs inside a virtual machine. The easiest way to do that is using a Docker image created by Justin Cormack, the CTO of Docker, Inc.18: $ docker run -it --rm --privileged --pid=host justincormack/nsenter1 Let’s run the ps aux command again: $ps aux | grep ping The output should be something like this: 145074 root 0:00 grep ping 267277 root 0:07 ping 8.8.8.8 You can see that the ping process is visible on the host machine but with a different process ID. Let’s try to kill the process on the host machine: $kill 67277 Of course, you need to adjust the process ID to the one you have on your machine. Now let’s check the list of running containers: $ docker ps 18This tutorial was created by Bret Fisher, a Docker Captain: Getting a Shell in the Docker Desktop Mac VM19. 1.4. INTRODUCTION TO KUBERNETES 19 What would you expect to be the output? 6. Following the instructions on Docker’s website, run Docker in rootless mode. You can find the instructions here: Rootless mode20. Then try to run the pinger container again. What would you expect to be the output? 1.4 Introduction to Kubernetes Following Docker’s impact on the software industry, Google open-sourced Kubernetes in 2014. Kubernetes is a container orchestration platform, designed for running and managing containers at scale. Kubernetes makes deployment of a containerized application to a cluster of machines easy. It also has integrated support for service discovery, load balancing, and scaling. Kubernetes is a Greek word that means helmsman or pilot, helm being a ship’s steering wheel. Speaking of helms, there is another tool usually used with Kubernetes called Helm. Helm makes writing Kubernetes manifests more DRY21 and reusable, by separating the common parts of the manifests into templates. As you can imagine, the Kubernetes ship can get really wet at times. Some of the benefits of using Kubernetes are: •Automation: Kubernetes makes deploying application to a cluster of machines easy. So, instead of SSHing into the machine, getting the last version of the code (e.g. with git pull), building the application, and running it, you can just run a single command and Kubernetes will take care of the rest. •High Availability: Kubernetes can run multiple instances of the same application. If one of the instances fails, Kubernetes will restart it. Servers can fail, containers will die, but you can’t kill the helmsman that easily! •Out of the Box: Kubernetes comes with a lot of features out of the box. It has support for service discovery, load balancing, and scaling. You can also extend it with plugins and custom resources. 20https://docs.docker.com/engine/security/rootless/ 21DRY principle stands for Don’t Repeat Yourself. It’s a software development principle that states that every piece of knowledge must have a single, unambiguous, authoritative representation within a system. If you’re repeating yourself, e.g. writing down the port number in multiple places in your Kubernetes manifests, you’re going to eventually change it in one place and forget to change it in another place. This can lead to bugs and inconsistencies. 20 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY Now that we learned Kubernetes is so awesome as a whole, let’s learn about its pieces. 1.4.1 Kubernetes Terminology In this part, we’re going to discuss the different building blocks of a Kubernetes application. As Kubernetes is already overwhelming for beginners, we’re going to skip details about how it operates. Let’s start with a list of basic Kubernetes concepts: •Pod: A pod is the smallest unit of deployment in Kubernetes. It’s usually used to run a single container. It’s also used to run multiple containers that are tightly coupled and need to share resources. •Service: A service is an abstraction that defines a logical set of pods and a policy to access them. It’s used to expose a pod to the outside world. •Deployment: A deployment is a declarative way to manage pods. It’s used to manage pods that are part of the same application, and to scale them up and down. If you create a deployment with 3 replicas, Kubernetes will make sure that there are always 3 pods running. •Namespace: A namespace is a way to divide cluster resources between multiple users. It’s used to separate different environments like production and staging. •Ingress: An ingress is an API object that manages external access to the services in a cluster. It’s used to expose a service to the outside world. These terms sound overwhelmingly complicated, because they are. But they’ll make more sense when we start using them. So, let’s do it! 1.4.2 Getting Started with Kubernetes A Kubernetes cluster usually consists of multiple machines. There are two types of machines in a Kubernetes cluster: •Master node: It’s the machine that runs the Kubernetes control plane. It’s responsible for managing the cluster. •Worker node: It’s the machine that runs the Kubernetes worker. It’s responsible for running the containers. In the case one of the machines fails, the other machines can take over its responsibilities. So, the cluster is highly available. 1.4. INTRODUCTION TO KUBERNETES 21 To test Kubernetes locally (on a single machine), there are the following options: •Docker Desktop22: This tool is a cross-platform GUI for Docker. It comes with a single-node Kubernetes cluster. •Minikube23: It’s a tool that runs a single-node Kubernetes cluster inside a virtual machine. It’s CLI-based and can be used on Linux, macOS, and Windows. •Kind24: Stands for Kubernetes in Docker. It’s a tool that runs a single-node Kubernetes cluster inside a Docker container. At the time of writing, there are other options to run Kubernetes locally as well, like MicroK8s25, K3s26, and K0s27. And of course there is the option of running Kubernetes on a cloud provider like AWS, GCP, or Azure. Or perhaps smaller cloud providers like DigitalOcean or Linode, for a more cost-effective solution. Now that we have a Kubernetes cluster running, let’s get our hands dirty with some Kubernetes commands. 1.4.3 Kubernetes Hello World Before starting with the example, we need to have kubectl installed. We mentioned how to install it in the technical requirements section. The kubectl can connect to one or more Kubernetes clusters, so we have a concept of context in kubectl. To connect to different clusters, we need to switch between contexts. We can list the available contexts with the following command: $ kubectl config get-contexts The output on my machine is something like this: 1CURRENT NAME CLUSTER AUTHINFO 2docker-desktop docker-desktop docker-desktop 3* minikube minikube minikube 22https://www.docker.com/products/docker-desktop 23https://minikube.sigs.k8s.io/docs/start/ 24https://kind.sigs.k8s.io/ 25https://microk8s.io/ 26https://k3s.io/ 27https://k0sproject.io/ 22 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY Let’s switch to the docker-desktop context: $ kubectl config use-context docker-desktop If you want to use a different cluster, you can of course use the context of your choice. Now let’s create a pod with the kubectl run command: $ kubectl run hello-world --image=hello-world The output should be something like this: 1pod/hello-world created We can list the pods with the kubectl get pods command: $ kubectl get pods The output should be something like this: 1NAME READY STATUS RESTARTS AGE 2hello-world 0/1 CrashLoopBackOff 3 (26s ago) 72s The pod is in the CrashLoopBackOff28 state. This is expected, as the hello-world image doesn’t have a process that keeps running. It just prints a message and exits. We can get more information about the pod with the kubectl describe pod command: $ kubectl describe pod hello-world The output should be something like this: 1Events: 2Type Reason Age Message 3---- ------ ---- ------- 4Normal Scheduled 37s Successfully assigned hello-world to dockerdesktop 5Normal Pulled 34s Successfully pulled image "hello-world" in 1.7959s 6Normal Pulled 33s Successfully pulled image "hello-world" in 1.2793s 28The CrashLoopBackOff state means that the container is crashing and restarting in a loop. Kubernetes usually waits for a few seconds before restarting the container. If the container keeps crashing, Kubernetes will back off and stop restarting it. 1.4. INTRODUCTION TO KUBERNETES 29 1.4.4 Kubernetes Hello World with Helm The standard way of creating Kubernetes applications is using Helm charts. Helm charts are packages of Kubernetes manifests. First, make sure helm is installed on your machine. Then let’s clone the following git repository that contains a link shortener application: $ git clone https://github.com/aerabi/link-shortener-js.git Then let’s go to the link-shortener-js directory: $cd link-shortener-js On the root of the repository, there is a directory called chart. Let’s take a look into it: $ tree chart The output should be something like this: chart Chart.yaml templates _helpers.tpl deployment.yaml ingress.yaml service.yaml values.yaml 2 directories, 6 files Let’s run the following command to install the application in the link-shortener namespace: $ kubectl create namespace link-shortener $ helm install link-shortener-js chart --namespace link-shortener The output should be something like this: 1NAME: link-shortener-js 2LAST DEPLOYED: Sun Sep 24 11:55:14 2023 3NAMESPACE: link-shortener 30 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY 4STATUS: deployed 5REVISION: 1 6TEST SUITE: None Let’s list everything in the link-shortener namespace: $ kubectl get all -n link-shortener The output should be something like this: 1NAME READY STATUS RESTARTS AGE 2pod/link-shortener-js-5458ff4bb-nsrx2 1/1 Running 0 2m17s 3 4NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE 5service/link-shortener-js 10.102.219.230 <none> 3000/TCP 2m17s 6 7NAME READY UP-TO-DATE AVAILABLE AGE 8deployment.apps/link-shortener-js 1/1 1 1 2m17s 9 10 NAME DESIRED CURRENT READY AGE 11 replicaset.apps/link-shortener-js-5458ff4bb 1 1 1 2 m17s Let’s describe the service: $ kubectl describe service link-shortener-js -n link-shortener The output should be something like this: 1Name: link-shortener-js 2Namespace: link-shortener 3Labels: app.kubernetes.io/instance=link-shortener-js 4app.kubernetes.io/managed-by=Helm 5app.kubernetes.io/name=link-shortener-js 6app.kubernetes.io/version=hashmap 7helm.sh/chart=link-shortener-js-0.1.0 8Annotations: meta.helm.sh/release-name: link-shortener-js 9meta.helm.sh/release-namespace: link-shortener 10 Selector: app.kubernetes.io/instance=link-shortener-js, 1.4. INTRODUCTION TO KUBERNETES 31 11 app.kubernetes.io/name=link-shortener-js 12 Type: ClusterIP 13 IP Family Policy: SingleStack 14 IP Families: IPv4 15 IP: 10.102.219.230 16 IPs: 10.102.219.230 17 Port: api 3000/TCP 18 TargetPort: 3000/TCP 19 Endpoints: 10.1.1.1:3000 20 Session Affinity: None 21 Events: <none> As you can see, the service is only accessible from inside the cluster. To make it accessible from outside the cluster, we need to change the service type to NodePort. Go to the chart /values.yaml file and change the service.type value to NodePort. 1imagePullSecrets: [] 2nameOverride: "" 3fullnameOverride: "" 4 5service: 6type: NodePort # <-- Change this line 7 8ports: 9internal: 10 name: "api" 11 number: 3000 Listing 1.2: Helm Chart Values File Then run the following command to upgrade the application: $ helm upgrade link-shortener-js chart --namespace link-shortener The output should be something like this: 1Release "link-shortener-js" has been upgraded. Happy Helming! 2NAME: link-shortener-js 3LAST DEPLOYED: Sun Sep 24 12:24:18 2023 32 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY 4NAMESPACE: link-shortener 5STATUS: deployed 6REVISION: 2 7TEST SUITE: None Let’s list everything in the link-shortener namespace again: $ kubectl get all -n link-shortener You can see that the service is now of type NodePort. The port 3000 is mapped to the port 32045. You can access the service on the port 32045 now. $ curl localhost:32045 The output should be the following: 1Hello World! Voilà! You have a Kubernetes application running. Next stop is talking about container security. 1.4.5 Exercises 1. In the example above, investigate the Chart.yaml file. What’s the name of the chart? What’s the version of the chart? 2. In the example above, investigate the templates/deployment.yaml file. What’s the name of the deployment? What’s the name of the pod template? What’s the name of the container? To find out, you should investigate _helpers.tpl file as well. The _helpers.tpl file is a template that is used by the other templates. It’s usually used to define reusable parts of the templates. 3. The templates/deployment.yaml file has a replicas field. The value is: {{ . Values.replicaCount }} which means it reads the value from the values.yaml file. What’s the value of the replicaCount in the values.yaml file? Change the value to 2and upgrade the application with the helm upgrade command. What would you expect to be the output of the kubectl get all command? 1.5 Container Security Containerized applications are not very different from other types of applications, except that they are packaged in a container. As the means of distribution is different, there 1.5. CONTAINER SECURITY 33 are some security concerns that are specific to containers. Also, the security tools and practices are different. There are two aspects of a container’s security: • The security of the container’s content, i.e. the Docker image. This is not much different from the security of any other software. But containers play an important role here, as they have standardized the way we package and distribute software. There is an ecosystem of tools and services built around containers, that we would focus on in the first part of the book. • The security of the container’s runtime, i.e. the container itself, and the host machine. This is more specific to containers. As containers share the same kernel with the host machine, there are some security concerns that we need to be aware of. We will discuss these concerns in the second part of the book. Containers unified the toolbox for different types of applications. Back in the day, a Java developer perhaps wouldn’t use the same tools as a Python developer. Now, they use Docker, perhaps DevContainers, a tool to scan the Docker images for vulnerabilities and license compliance, and a tool to deploy the whole thing to Kubernetes. 1.5.1 Vulnerability Scanning Java developers remember Log4Shell as the most critical vulnerability incident of 2021. It was a vulnerability in the Log4j library that allowed remote code execution. Log4j is a Java logging library that is used in many Java applications. It’s also used in many Java libraries, like Apache Kafka and Apache Spark. It’s so popular I have never seen a decent Java application that doesn’t use it. When the vulnerability was discovered, it was already exploited in the wild. It was so critical that it was assigned the highest CVSS score of 10.0. But what’s a CVSS score? CVSS stands for Common Vulnerability Scoring System. It’s a standard for assessing the severity of security vulnerabilities. It’s a number between 0 and 10. The higher the number, the more severe the vulnerability is. Security vulnerabilities not only have a CVSS score, but they also have a CVE number. CVE stands for Common Vulnerabilities and Exposures. It’s a list of publicly disclosed cybersecurity vulnerabilities. It’s maintained by the MITRE Corporation, MITRE being a not-for-profit organization that operates research and development centers sponsored by the federal government of the United States. The Log4Shell vulnerability was assigned the CVE-2021-44228 number. You can look it up on the CVE website: https://www.cve.or g/CVERecord?id=CVE-2021-44228. 34 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY There you can find information about the vulnerability, including some description, the CVE number, and the affected software. To get more details one can confer NVD’s website: https://nvd.nist.gov/vuln/detail/CVE-2021-44228. NVD stands for National Vulnerability Database. It’s a database of security vulnerabilities maintained by the United States’ National Institute of Standards and Technology (NIST). With Log4Shell shaking the Java world, the adoption of Software Bill of Materials (SBOMs) has become a hot topic. SBOMs are a list of all the components used in a software project. As an example, the U.S. government has mandated that all software vendors should provide SBOMs for their products. But what’s an SBOM exactly? As discussed earlier, every piece of software has dependencies. And these dependencies have their own dependencies. And so on. An SBOM is a list of all these dependencies. If your final product is a Docker image, then the SBOM is a list of all the packages installed in the image, including the OS packages and the application packages. Having a Docker image at hand, you can generate an SBOM with the syft tool: $ syft docker://<image-name> Docker integrated syft into the Docker CLI, so you can also generate an SBOM with the docker sbom command: $ docker sbom gitweekly/git-weekly Note. The docker sbom command which is included in Docker Desktop, is softly deprecated in favor of Docker Scout: docker scout sbom gitweekly/git-weekly. Still, you can use the docker sbom command in the Docker Desktop, and by installing the SBOM CLI plugin29. SBOMs are important for security, as they help us to identify the vulnerabilities in our software. There are tools that can scan SBOMs and generate reports about the vulnerabilities in the software. We will get to those in the next chapter. 29https://github.com/docker/sbom-cli-plugin 1.6. EXPLORING WHAT LIES AHEAD 35 1.5.2 Exercises 1. Using either Docker CLI or Syft, generate an SBOM for the ubuntu image. 2. Using either Docker CLI or Syft, generate an SBOM for the alpine image. How does the list of dependencies differ from the ubuntu image? 3. Using Docker Scout CLI, list the CVEs in the gitweekly/git-weekly image: $ docker scout cves gitweekly/git-weekly Choose one of the CVEs, say CVE-2023-24540, and look it up on the CVE website: https://www.cve.org/CVERecord?id=CVE-2023-24540. Then look it up on the NVD website: https://nvd.nist.gov/vuln/detail/CVE-202324540. Docker Scout also has its own CVE website where you can look up CVEs: https: //scout.docker.com/v/CVE-2023-24540. What’s the CVSS score of the CVE? 1.6 Exploring What Lies Ahead In this introductory chapter, we’ve dipped our toes into the vast ocean of container security. We’ve covered the basics of containers, container security, and some essential terminology. However, this is just the beginning of your journey. Let’s take a glimpse of what awaits you in the subsequent chapters of this book: Chapter 2: Exploring Container Security Tools We’ll delve deeper into the practical side of container security by introducing you to various tools like Docker Scout, Snyk, Trivy, and Falco. These tools will become your trusted companions as you navigate the container security landscape. Chapter 3: Supply Chain Security for Application Development Learn how to secure your applications from the very beginning with secure coding practices, software dependency management, and code signing techniques. 36 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY Chapter 4: Secure Docker Image Building Practices Discover the art of building secure Docker images. We’ll discuss the importance of base image security and how to generate Software Bill of Materials (SBOMs) during image builds. Chapter 5: Integrating Container Security within CI/CD Pipelines Explore the integration of container security into your CI/CD pipelines. We’ll cover essential topics like scanning Docker images, securing your CI/CD pipelines, and practical setups using GitLab-CI and GitHub Actions. Chapter 6: Secure Docker Containers on Kubernetes Uncover the intricacies of running secure Docker containers on Kubernetes. We’ll explore Kubernetes’ security model, pod security policies, and secrets management. Chapter 7: Kubernetes Cluster Security Dive into the security aspects of your Kubernetes clusters, including network security, role-based access control (RBAC), and safeguarding critical components like etcd and the Kubernetes API server. Chapter 8: Logging, Monitoring, and Auditing in Kubernetes Learn how to implement robust monitoring and auditing practices in Kubernetes using tools like Prometheus and Grafana, ensuring the security of your containerized applications. Chapter 9: Designing a Secure Dockerized Application Architecture Put your knowledge into action by designing a secure Dockerized application architecture. This hands-on chapter will provide practical insights into building resilient containerized applications. Chapter 10: Emerging Trends and Future Trends in Docker Supply Chain and Runtime Security Stay ahead of the curve as we explore emerging trends and future developments in container security, keeping you informed about the ever-evolving landscape. 1.7. CONCLUSION 37 As you embark on this container security journey, each chapter will equip you with valuable insights and practical skills, enabling you to build, deploy, and manage secure containerized applications. So, fasten your seatbelt, and let’s dive deeper into the exciting world of container security! 1.7 Conclusion Containerization has revolutionized the way we build, deploy, and manage applications. However, with this new technology comes new security challenges. As container adoption continues to grow, it’s becoming increasingly important to ensure that your containerized applications and infrastructure are secure. This book is designed to equip you with the knowledge and skills needed to secure your containerized applications and infrastructure. Whether you’re a developer, DevOps engineer, or security professional, this book will provide you with practical insights and hands-on experience to help you build, deploy, and manage secure containerized applications. In this book, you’ll learn about the various security challenges that come with containerization, including image vulnerabilities, network security, and securing critical components like etcd and the Kubernetes API server. You’ll also learn about best practices for securing your containerized applications, including implementing role-based access control (RBAC), monitoring and auditing your Kubernetes clusters, and designing a secure Dockerized application architecture. As you embark on this container security journey, each chapter will equip you with valuable insights and practical skills, enabling you to build, deploy, and manage secure containerized applications. So, if you’re looking to stay ahead of the curve in container security, this book is a must-read. 1.8 Exercises In this chapter, each section had its own exercises. But here are some additional exercises relevant to the whole chapter: 1. Create a project on GitHub that will contain a backend, a frontend, and some Docker files. The backend should be a simple REST API that returns a JSON object. The frontend should be a simple HTML page that displays the JSON object. 38 CHAPTER 1. INTRODUCTION TO CONTAINERS AND CONTAINER SECURITY 2. Create a Dockerfile for the backend and the frontend. The frontend Dockerfile should use the nginx image as a base image. The backend Dockerfile depends on the language you choose, but you can run docker init and it will help you create a Dockerfile. 3. Run both images locally. Then create a Docker network and run both images on the same network. The frontend should be able to access the backend, without exposing a port. 4. Create a Docker Compose file that runs both images. The frontend should be able to access the backend, without exposing a port. 5. Create a Helm chart for both backend and frontend. The Helm chart should be able to deploy both images to a Kubernetes cluster. The frontend should be able to access the backend, using the service name. 2.3. TRIVY 45 12 "licenses": [ 13 { 14 "license": { 15 "name": "GPL-2.0-only" 16 } 17 } 18 ], 19 "purl": "pkg:apk/alpine/[email protected]?arch=x86_64&distro=3.12.12" 20 } The report has the following fields: •bom-ref: A reference to the component in the SBOM. The ID here identifies the package in the Alpine Linux package manager (APK). •type: The type of the component, e.g. library, framework, or application. •licenses: The licenses of the component. In this case, the ssl_client package is licensed under the GPL-2.0-only license. •purl: The Package URL (PURL) of the component. The PURL is a unique identifier for the package, and can be used to reference the package in other systems. The formats SPDX and SPDX-JSON are similar to CycloneDX, and can be used to generate SBOMs for container images. The GitHub format is similar to SARIF, in the sense that it can be used to upload the scan results to the GitHub Security tab. Now let’s see how to use Trivy in a CI/CD pipeline. 2.3.3 GitHub Actions Trivy can be used as a GitHub Action, and the results of the scan can be uploaded to the GitHub Security tab. The following is an example of a GitHub Actions workflow that uses Trivy to scan a Docker image: 1name: "Backend CI/CD" 2 3on: 4push: 5branches: [ master ] 6pull_request: 7branches: [ master ] 8 46 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS 9env: 10 REGISTRY: ghcr.io 11 IMAGE_NAME: ${{ github.repository }}-backend 12 13 defaults: 14 run: 15 working-directory: backend 16 17 jobs: 18 docker_build: 19 runs-on: ubuntu-latest 20 steps: 21 - uses: actions/checkout@v4 22 23 - name: Log in to the Container registry 24 uses: docker/login-action@v3 25 with: 26 registry: ${{ env.REGISTRY }} 27 username: ${{ github.actor }} 28 password: ${{ secrets.GITHUB_TOKEN }} 29 30 - name: Extract metadata (tags, labels) for Docker 31 id: meta 32 uses: docker/metadata-action@v5 33 with: 34 images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} 35 36 - name: Build and push Docker image 37 uses: docker/build-push-action@v5 38 with: 39 context: backend 40 push: true 41 tags: ${{ steps.meta.outputs.tags }} 42 labels: ${{ steps.meta.outputs.labels }} 43 44 docker_scan_trivy: 45 runs-on: ubuntu-latest 2.3. TRIVY 47 46 needs: 47 - docker_build 48 steps: 49 - name: Extract metadata (tags, labels) for Docker 50 id: meta 51 uses: docker/metadata-action@v5 52 with: 53 images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} 54 55 - name: Run Trivy vulnerability scanner 56 uses: aquasecurity/[email protected] 57 with: 58 image-ref: ${{ steps.meta.outputs.tags }} 59 format: 'sarif' 60 output: 'trivy-results.sarif' 61 exit-code: '1' 62 ignore-unfixed: true 63 vuln-type: 'os,library' 64 severity: 'CRITICAL,HIGH' 65 66 - name: Upload Trivy scan results to GitHub Security tab 67 uses: github/codeql-action/upload-sarif@v2 68 with: 69 sarif_file: 'trivy-results.sarif' Listing 2.1: GitHub Actions workflow for Trivy In this example, we assume there is a Dockerfile in the backend directory. The CI pipeline builds the Docker image and pushes it to the GitHub Container Registry. Then, it uses Trivy to scan the image for known vulnerabilities, and uploads the results to the GitHub Security tab. After creating a PR that contains this workflow, the github-code-scanning user will leave the following comment on the PR: This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the ‘Security’ tab will show more code scanning analysis results (for 48 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation. After the merger, the results of the scan will be available in the Security tab of the repository. The following is an example of the results: Figure 2.1: GitHub Security Tab A few notes about the workflow: • The workflow has two jobs. The first job builds the Docker image, and pushes it to the GitHub Container Registry. The second job uses Trivy to scan the image for known vulnerabilities, and uploads the results to the GitHub Security tab. • The workflow uses the docker/metadata-action to extract the tags and labels of the Docker image. The tags and labels are used by Trivy to identify the image. This will ensure that the images stored in the GitHub Container Registry are not overwritten by the workflow and the results of the scan are uploaded to the correct 2.4. SNYK 49 image. • The workflow uses the docker/build-push-action to build the Docker image and push it to the GitHub Container Registry. • The workflow uses the aquasecurity/trivy-action to scan the Docker image for known vulnerabilities. The output format is set to SARIF, and the results are uploaded to the GitHub Security tab using the github/codeql-action. 2.3.4 Exercises 1. Create a GitHub Actions workflow that uses Trivy to scan a Docker image for known vulnerabilities, and uploads the results to the GitHub Security tab. Add it to your project of choice. 2. Generate SBOM for a Docker image using Trivy: docker run --rm -it aquasec/trivy image --format spdx-json alpine:3.12 3. Learn about different Trivy output formats: •table: Human-readable table format, default. •json: JSON format. •template: Go template format. •spdx: SPDX format. •spdx-json: SPDX JSON format. •sarif: SARIF format. •cyclonedx: CycloneDX format. •github: GitHub Security Advisories format. •cosign-vuln: Cosign vulnerability report format. 2.4 Snyk Snyk is a container security tool, developed by Snyk Ltd. It can be used to analyze container images for known vulnerabilities. Snyk was integrated into Docker CLI as a subcommand, docker scan, in 2020 until replaced by Docker Scout in 2023. Snyk is now available as a Docker Desktop extension and a standalone CLI tool. Unlike Trivy, Snyk is not open-source, but is free for public repositories. My Snyk fun fact is that I’ve been a Snyk Ambassador since 2022. So, I have a hoodie from Snyk with my Twitter handle written on it. But now Elon Musk is destroying Twitter, and 50 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS all my other social media accounts have different handles.4 2.4.1 Installation To use Snyk, one needs to create an account on Snyk.io and get an API token. The API token can be obtained from the account settings page. The token can be set as an environment variable, e.g. SNYK_TOKEN. Snyk is available as a Docker Desktop extension, and can be installed from the Docker Desktop UI. I’ll skip the UI stuff as I’m more of a CLI person. To use Snyk CLI, there are two ways: • Use the official Docker image, • Install the CLI binary. Using the Official Docker Image Using the official Snyk Docker image is a bit more verbose than using Trivy: docker run --rm -it \ --env SNYK_TOKEN=${SNYK_TOKEN} \ -v /var/run/docker.sock:/var/run/docker.sock \ snyk/snyk:docker snyk container test alpine:3.12 A few points: • There are multiple Docker images based on the things you want to scan. In this example, we use the snyk/snyk:docker image, which is used to scan Docker images. There is e.g. snyk/snyk:node for Node.js projects. • The SNYK_TOKEN environment variable is set to the API token obtained from Snyk.io. • The Docker socket is mounted to the container, so that the container can access the Docker daemon. This is needed because Snyk will pull the image to be scanned, and analyze it locally. Since the command is quite verbose, you can define an alias to simplify it: alias snyky='docker␣run␣--rm␣-it␣\ ␣␣␣␣--env␣SNYK_TOKEN=${SNYK_TOKEN}␣\ ␣␣␣␣-v␣/var/run/docker.sock:/var/run/docker.sock␣\ ␣␣␣␣snyk/snyk:docker␣snyk' 4It’s @aerabi.com on BlueSky, and @aerabi on GitHub, Medium, DEV.to, and LinkedIn. 2.4. SNYK 51 I called it snyky because it’s a sneaky way to use Snyk and I don’t want it to clash with snyk in case I install the CLI binary later. snyky container test alpine:3.12 Installing the CLI Binary There are multiple ways to install the Snyk CLI binary. The easiest way is to use NPM, if you happen to have Node.js installed on your system: $ npm install -g snyk If not, please confer the official installation instructions5. After the installation, you need to authenticate with Snyk using the API token: $ snyk auth ${SNYK_TOKEN} 2.4.2 Analyzing Container Images From now on, I’ll use the command snyk, but you’ll be able to do all the stuff with snyky as well. To check a Docker image, say alpine:3.12, for known vulnerabilities, run the following command: $ snyk container test alpine:3.12 The output of the command will be similar to the following: 1Testing alpine:3.12... 2 37Critical severity vulnerability found in zlib/zlib 4Description: Out-of-bounds Write 5Info: https://security.snyk.io/vuln/SNYK-ALPINE312-ZLIB-2977082 6Introduced through: zlib/[email protected], apk-tools/[email protected] 7From: zlib/[email protected] 8From: apk-tools/[email protected] > zlib/[email protected] 9Fixed in: 1.2.12-r2 5https://docs.snyk.io/snyk-cli/install-or-update-the-snyk-cli 52 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS 10 11 12 13 Organization: aerabi 14 Package manager: apk 15 Project name: docker-image|alpine 16 Docker image: alpine:3.12 17 Platform: linux/arm64 18 Licenses: enabled 19 20 Tested 14 dependencies for known issues, found 1 issue. 21 22 Alpine 3.12.12 is no longer supported by the Alpine maintainers. The output of the command shows that the image has one critical vulnerability, in the zlib library. There is also a link to Snyk’s vulnerability database: SNYK-ALPINE312ZLIB-29770826. Opening the link will show the details of the vulnerability, including the CVE, the reason for the vulnerability, and the affected versions. 2.4.3 Container Image Scan Output Formats Snyk, like Trivy, supports JSON and SARIF output formats. The following is an example of the JSON output: $ snyk container test alpine:3.12 --json Like Trivy, the output here is huge, and has a list of vulnerabilities among other things. Here is the beginning of the JSON output: 1{ 2"vulnerabilities": [ 3{ 4"id": "SNYK-ALPINE312-ZLIB-2977082", 5"cpes": [], 6"title": "Out-of-bounds Write", 7"CVSSv3": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H", 8"credit": [ 6https://security.snyk.io/vuln/SNYK-ALPINE312-ZLIB-2977082 2.4. SNYK 53 9"" 10 ], 11 "semver": { 12 "vulnerable": [ 13 "<1.2.12-r2" 14 ] 15 }, 16 "exploit": "Not Defined", 17 "patches": [], 18 "insights": { 19 "triageAdvice": null 20 }, 21 "language": "linux", 22 "severity": "critical", 23 "cvssScore": 9.8, 24 "malicious": false, 25 "isDisputed": false, 26 "references": [ 27 { 28 "url": "https://github.com/ivd38/zlib_overflow", 29 "title": "[email protected]" 30 }, The SARIF output is similar to the one of Trivy, and can be used to upload the results to the GitHub Security tab. $ snyk container test alpine:3.12 --sarif To store the SARIF file, you can use the following command: $ snyk container test alpine:3.12 --sarif-file-output=sarif.json This is essentially what we’ll replicate in the CI/CD pipeline. 2.4.4 GitHub Actions Snyk can be used as a GitHub Action, and the results of the scan can be uploaded to the GitHub Security tab. The only difference here is that the token is set as a secret, so that the GitHub Actions workflow can access it. 54 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS The following is an example of a GitHub Actions workflow that uses Snyk to scan a Docker image: 1name: "Scan Docker Application for Vulnerabilities" 2 3on: 4push: 5branches: [ master ] 6pull_request: 7branches: [ master ] 8 9env: 10 REGISTRY: ghcr.io 11 IMAGE_NAME: ${{ github.repository }}-backend 12 13 defaults: 14 run: 15 working-directory: backend 16 17 jobs: 18 docker_build: 19 runs-on: ubuntu-latest 20 steps: 21 - uses: actions/checkout@v4 22 23 - name: Log in to the Container registry 24 uses: docker/login-action@v3 25 with: 26 registry: ${{ env.REGISTRY }} 27 username: ${{ github.actor }} 28 password: ${{ secrets.GITHUB_TOKEN }} 29 30 - name: Extract metadata (tags, labels) for Docker 31 id: meta 32 uses: docker/metadata-action@v5 33 with: 34 images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} 2.5. DOCKER SCOUT 61 2.5.4 Docker Scout Output Formats Docker Scout supports the following output formats: •Packages: A list in plain text •SARIF: The one that we want to use in CI/CD pipelines •Markdown: Which also includes HTML tags •SPDX: SPDX JSON format •GitLab: GitLab Security Report format •SBOM: SBOM in SPDX JSON format The following is an example of the SARIF output: $ docker scout cves --format sarif --output alpine.sarif.json alpine:3.12 As before, CI/CD pipelines can use the SARIF output to upload the scan results to the GitHub Security tab. 2.5.5 GitHub Actions Docker Scout can be used as a GitHub Action for different purposes. One would be comparing the newly built image with the one in the registry in terms of vulnerabilities. The following is an example of a GitHub Actions workflow that uses Docker Scout Action9: 1name: Run Docker Scout to extract vulnerabilities from the image 2uses: docker/scout-action@v1 3with: 4command: cves 5image: '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}' 6sarif-file: scout.sarif 7github-token: '${{ github.token }}' 8 9name: Upload Docker Scout scan results to GitHub Security tab 10 uses: github/codeql-action/upload-sarif@v3 11 with: 12 sarif_file: scout.sarif Listing 2.3: GitHub Actions workflow for Docker Scout 9A full example is available at the book’s GitHub repository: https://github.com/DockerSecurity-io/book 62 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS In this example, we used the docker/scout-action to extract the CVEs from the image. The output is stored in the file scout.sarif, and then uploaded to the GitHub Security tab using the github/codeql-action. We could use another command, e.g. compare to compare the vulnerabilities of the new image with the one in the registry: 1name: Run Docker Scout to compare vulnerabilities with the latest image 2uses: docker/scout-action@v1 3with: 4command: compare 5image: '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}' 6to-latest: true 7github-token: '${{ github.token }}' Listing 2.4: GitHub Actions workflow for Docker Scout compare And we can use multiple commands at the same time: 0command: compare,cves,quickview The github-token is used to write the result as a comment on the pull request. The tolatest option is used to compare the image with the latest one in the registry. 2.5.6 Docker Scout and Vulnerability Exploitability Vulnerability scanning for Docker images was introduced by Docker in 2020, with the docker scan subcommand. Docker Scan was using Snyk under the hood. Docker Scout was initially released as a replacement for the docker scan subcommand, in March 2023. I wrote an article about it on Medium then, comparing their outputs.10 There, I mentioned that the two commands have found different CVEs in the same image. A day later, Docker contacted me and mentioned that they have investigated the differences, and that most of them are false positives. In one case, the installed version was higher than the fixed version, and in another case, package names were different (the package github .com/labstack/echo was being confused with github.com/labstack/echo/v4, which is 10The article is called “Docker Scout vs Scan” and is available at: https://aerabi.medium.com/docker-scoutvs-scan-90ce6d6fd04c. 2.5. DOCKER SCOUT 63 a different package). They also mentioned that they are working on their false positive CVE with the source (that was an advisory on GitLab). The main point here is that it’s always good to check the CVEs manually. The CVEs reported could be false positive, or could be irrelevant to your project. Only because a CVE is in your image, it doesn’t mean that it’s exploitable. The exploitability of a CVE depends on the context of the image, and the application running in it. This is why you can automatically ignore some CVEs when using Docker Scout. Vulnerability Exploitability eXchange (VEX) is a standard format to document the exploitability of a vulnerability in the context of a specific image. You can use this format to create exceptions for vulnerabilities that are not exploitable in your image. To create a VEX document, we need a tool called vexctl. You can install it with Homebrew or using the instruction on their GitHub repo11. After installation, use the following command to create a VEX file: $ vexctl create \ --author="[email protected]" \ --product="pkg:docker/[email protected]" \ --vuln="CVE-2022-37434" \ --status="not_affected" \ --justification="vulnerable_code_not_in_execute_path" \ --file="CVE-2022-37434.vex.json" A JSON file is created with the following content: 1{ 2"@context": "https://openvex.dev/ns/v0.2.0", 3"@id": "https://openvex.dev/docs/public/vex-64dc6e2613b24e0638c10220c4dca ", 4"author": "[email protected]", 5"timestamp": "2025-03-17T14:16:27.900702+01:00", 6"version": 1, 7"statements": [ 8{ 9"vulnerability": { 10 "name": "CVE-2022-37434" 11https://github.com/openvex/vexctl 64 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS 11 }, 12 "timestamp": "2025-03-17T14:16:27.900703+01:00", 13 "products": [ 14 { 15 "@id": "pkg:docker/[email protected]" 16 } 17 ], 18 "status": "not_affected", 19 "justification": "vulnerable_code_not_in_execute_path" 20 } 21 ] 22 } To use this VEX file with Docker Scout, you can use the --vex-location option: $ docker scout cves alpine:3.12 --vex-location . The --vex-location flag is used to specify the location of the VEX files. Here we used the current directory, but you can change it to the directory where the VEX files are stored. The result will be similar to the following: 1## Packages and Vulnerabilities 2 31C 0H 0M 0L zlib 1.2.12-r0 4pkg:apk/alpine/[email protected]?os_name=alpine&os_version=3.12 5 67CRITICAL CVE-2022-37434 7https://scout.docker.com/v/CVE-2022-37434 8Affected range : <1.2.12-r2 9Fixed version : 1.2.12-r2 10 VEX : not affected [vulnerable code not in execute path] 11 : [email protected] To suppress the “not affected” CVEs from the output, you can use the --only-vexaffected option: $ docker scout cves alpine:3.12 --vex-location . --only-vex-affected The output will ignore the CVE altogether and report only the affected ones (which are none at the time of writing, because the suppressed CVE was the only one in the image). 2.6. FALCO 65 2.5.7 Exercises 1. Build an image and generate SBOM for it using BuildKit. Then push the image to Docker Hub. 2. Check the image for known vulnerabilities using Docker Scout: $ docker scout cves aerabi/alpine-curl 3. Generate SBOM for a Docker image using Docker Scout: $ docker scout sbom aerabi/alpine-curl 4. Get recommendations for a Docker image using Docker Scout: $ docker scout recommendations aerabi/alpine-curl 5. Create a VEX file for a CVE in your image, and push it to the registry as an attestation for your image: $ docker scout attestation add \ --file <cve-id>.vex.json \ --predicate-type https://openvex.dev/ns/v0.2.0 \ IMAGE 6. Create a GitHub Actions workflow that uses Docker Scout to scan a Docker image for known vulnerabilities, and uploads the results to the GitHub Security tab. Add it to your project of choice. 7. Create a GitHub Actions workflow that uses Docker Scout to compare a newly built Docker image with the one in the registry in terms of vulnerabilities, and uploads the results to the GitHub Security tab. Add it to your project of choice. 2.6 Falco Falco is a cloud-native runtime security tool, developed by Sysdig Inc. It can be used to detect and alert on anomalous behavior in applications. It can also be used to detect known vulnerabilities in container images. Falco uses eBPF to monitor system calls, and is able to detect malicious behavior in real-time. 66 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS eBPF is a Linux kernel technology that allows running user-defined programs in the kernel. It’s used by Falco to monitor system calls, and is also used by other tools like Cilium and Tracee. 2.6.1 Install Falco Installing Falco is more tricky than the other tools, because it requires a Linux kernel that supports eBPF. If your host is a Linux machine, you can install Falco directly on it. We will use it with Docker here, but it won’t work with Docker Desktop on Mac or Windows. On these machines, you should create a virtual machine with a Linux kernel that supports eBPF. • You can either create the virtual machine using VirtualBox, or Vagrant, which is a wrapper around VirtualBox. Vagrant enables you to create a virtual machine from aVagrantfile, which is a text file that contains the configuration of the virtual machine. In this sense, Vagrant is similar to Docker. • If you are on a Mac with Apple Silicon, you can use Lima to create a virtual machine. Lima is a lightweight virtual machine manager for Apple Silicon and is a wrapper around Hypervisor. After creating the virtual machine with a recent Linux on it, we can use Falco there in the following ways: • Natively on the host, • Inside a Docker container, • Inside a Kubernetes cluster. For the rest of the chapter, I will assume that you are on a Linux machine (either a host or a virtual machine) with a recent Linux kernel that supports eBPF. For more information, please refer to the official documentation12. 2.6.2 Run Falco in a Docker Container To run Falco in a Docker container, you can use the following command: $ sudo docker run --rm -i -t --name falco --privileged \ -v /var/run/docker.sock:/host/var/run/docker.sock \ -v /dev:/host/dev -v /proc:/host/proc:ro -v /boot:/host/boot:ro \ 12https://falco.org/docs/getting-started/falco-linux-quickstart/ 2.6. FALCO 67 -v /lib/modules:/host/lib/modules:ro -v /usr:/host/usr:ro -v /etc:/host/ etc:ro \ falcosecurity/falco:0.40.0 The command runs a Docker container in privileged mode. The container has access to almost all the host’s resources, including: •/var/run/docker.sock: The Docker socket, •/dev: The device on the host, •/proc: The processes, •/boot: The boot files, •/lib/modules: The kernel modules, •/usr: The user files, •/etc: The configuration files. So it’s essentially the same as running Falco natively—just more convenient. After running this command, Falco starts with a pre-configured set of rules, and starts monitoring the host for suspicious activity. 2.6.3 Trigger Falco Alerts To trigger Falco, let’s access a file in the /etc directory: $ sudo cat /etc/shadow Now check the Falco logs to see the alert: 0Warning Sensitive file opened for reading by non-trusted program 2.6.4 Trigger Falco Alerts in Kubernetes To trigger Falco alerts in Kubernetes, you need to install Falco on a Kubernetes cluster. You can use Helm package manager to install Falco on Kubernetes. First, add the Falco Helm repository: $ helm repo add falcosecurity https://falcosecurity.github.io/charts $ helm repo update Then, install Falco on the Kubernetes cluster: 68 CHAPTER 2. EXPLORING CONTAINER SECURITY TOOLS $ helm install --replace falco --namespace falco \ --create-namespace --set tty=true falcosecurity/falco To trigger Falco alerts, let’s create a simple container application that sleeps forever on the Kubernetes cluster. $ kubectl run alpine --image alpine -- sh -c "sleep␣infinity" Return to the terminal prompt, and run the following command to execute a shell command in the container: $ kubectl exec -it alpine -- sh -c "uptime" You can check the Falco logs to see the alert: $ kubectl logs -l app.kubernetes.io/name=falco -n falco -c falco | grep Notice The output of the command will be similar to the following: 0Notice A shell was spawned in a container with an attached terminal You can install Falco Sidekick for example to send the alerts to a Slack channel or a webhook. 2.6.5 Exercises 1. Create a virtual machine with the latest Ubuntu installed on it. 2. Run Falco in a Docker container on the virtual machine, and try to trigger it. 3. Install a K3s Kubernetes cluster on the virtual machine. 4. Try to trigger Falco alerts in the Kubernetes cluster, by accessing a file in the /etc directory: bash kubectl exec -it alpine -- sh -c "cat /etc/shadow" Index AI (Artificial Intelligence), 247 AKS (Azure Kubernetes Service), 190, 202 AppArmor, 158, 161 Argo CD, 202 Bandit, 98 BuildKit, 59, 133 BusyBox, 6 CA (Certificate Authority), 192 Cilium Tetragon, 157 CNCF (Cloud Native Computing Foundation), 11, 248 containerd, 10, 157 control group, 7 Cosign, 109, 144, 157, 158 Cosign-Vuln, 43 CrashLoopBackOff, 22 CRI (Container Runtime Interface), 11 CRI-O, 11 CVE (Common Vulnerabilities and Exposures), 33 CVSS (Common Vulnerability Scoring System), 33 CycloneDX, 42 DAC (Discretionary Access Control), 158 defense in depth, 230 Deployment, 20, 25 Docer Init, 249 Docker, 3, 247, 248 Ask Gordon, 122 Build Cloud, 135, 249 buildx, 59 container, 7 privileged, 130 Desktop, 21 Dockerfile, 16, 104 Hub, 12, 106 image, 6 base image, 106 distroless, 108 signing, 109 Wolfi-based, 106 in Docker, 128 Init, 120 Official Images, 106 SBOM, 34 259 260 INDEX scan, 62 Scout, 56, 249 socket, 131 dockerd, 10 domain name, 8 eBPF, 66, 157, 215, 248, 254 ECR (Amazon Elastic Container Registry), 138 EKS (Amazon Elastic Kubernetes Service), 138, 190, 202 ESLint, 97 etcd, 157, 187, 188 External Secrets Operator, 158 Falco, 65, 157, 205, 215, 231 sidekick, 215 Talon, 205, 215 Fluent Bit, 210 Flux, 202 Git, ii commit signing, 91 GitHub Actions, 45, 53, 61, 139 commit verification, 95 GitLab CI, 129, 150 commit verification, 95 SAST, 100 GKE (Google Kubernetes Engine), 190, 202 GPG (GNU Privacy Guard), 92 Grafana, 205, 209 Grype, 157 Helm, 29, 206 chart, 29 hostname, 8 immutability, 230 Ingress, 20 Java Maven, 89 k3d, 188 k3s, 188 Kaniko, 134, 152 Kind (Kubernetes in Docker), 21, 188 kOps, 202 kubeadm, 202 kubectl, 188 kubelet, 157, 187, 188 Kubernetes, 3, 19, 187, 205, 248 API, 187 API server, 157, 187, 188 controller manager, 187, 188 Deployment, 20, 25 Ingress, 20 liveness probe, 210 Namespace, 20 NetworkPolicy, 157, 158 Pod, 20, 24, 164 Pod Security Admission, 157, 158 Pod Security Policy, 158 PodMonitor, 207 readiness probe, 210 ReplicaSet, 27 scheduler, 187, 188 Secret, 158 security context, 157, 158, 164 Service, 20, 26 Service Account, 192 ServiceMonitor, 207 startup probe, 210