SUBSCRIBE TO OUR NEWSLETTER
Stay informed! Sign up to get expert advice and insight delivered direct to your inbox
What To Do When Containers are Attacked: An Incident Response Plan
Containers introduce a new attack surface that requires different security measures, incident response tactics, and forensics approaches.
Sagar Nangare
May 30, 2022
6 Min Read
(Source: Pixabay)
A container is a software package that contains all dependencies needed to run on any operating system and infrastructure, including code, settings, libraries, and system tools. Each container includes a runtime environment that enables applications to move between various computing environments—for example, moving from a physical machine to the cloud.
Containers offer many benefits but also pose significant security challenges. Containers introduce a new attack surface that requires different security measures, response tactics, and forensics approaches. Container incident response is a standardized approach to responding to container security incidents.
Incident response is a security function that involves detecting security events affecting network resources and information assets and taking appropriate steps to evaluate and remediate. A security event can be a malware infection, compromised credentials, and database exposures. Container incident response helps stop, block, contain, remediate, and prevent security threats against containerized workloads.
Why is container security important?
Containers are widely used as lightweight building blocks in DevOps pipelines, making container security an essential part of a modern IT security strategy. Like a virtual machine (VM), a container holds everything you need to run your application—code, runtime, tools, libraries, settings—and runs as designed in the OS, regardless of environment. Unlike VMs, it is highly portable, starts up faster, and consumes far fewer resources.
Container platforms like Docker and Kubernetes include some basic security controls, but containerized application development often includes third-party software components that can be vulnerable. Also, Docker and Kubernetes are not secure by default, making it critical to ensure secure configuration, which can be complex at large scale.
Containers are vulnerable to malicious processes that can bypass isolation on the host machine. Also, containers are based on images that can be tampered with by attackers or can contain old or vulnerable software components. When containers are not secure, they can enable unauthorized access to other containers and applications running within the container and, in extreme cases, can allow attackers to compromise host machines and entire container clusters.
Why forensics is different in cloud-native environments
Forensics is a critical part of information security, in general and incident response in particular. This is an emerging field, and there are now specialized tools and security certifications centered around DFIR (digital forensics and incident response).
When a suspicious event occurs, security analytics must perform rapid forensic investigation to understand what happened, identify if the event represents a real security incident, and gather the information required to contain and eradicate the threat.
As developers deploy code changes at high speed, it becomes more difficult to perform a forensic investigation. Unlike physical servers or virtual machines, which can remain the same for months, containers are lightweight, ad hoc computing tasks that an orchestrator can start and shut down at any time. A container's lifespan can be as short as a few minutes or seconds, and it is typically measured in hours. Serverless functions have a much shorter lifetime—for example, AWS Lambda functions only run for up to 15 minutes per execution.
These very short life cycles have far-reaching implications for security, visibility, and forensics. Because containers are temporary, data written to the container's file system is usually deleted when the container is shut down (unless it is explicitly saved to another location). An orchestration tool like Kubernetes automatically schedules workloads to run on hosts, and dynamically moves workloads between hosts, meaning that teams typically cannot decide in advance which host the application will run on.
To enable forensics in a cloud-native environment, teams need strategies and technologies that can capture relevant log data in a highly dynamic environment, with the ability to support ephemeral workloads running across multiple clouds and up to thousands of hosts.
Building a container forensics incident response plan
There are three main areas of focus to consider when developing a container incident response plan.
Preventative Measures
Preventive measures can help reduce attack surfaces on containers. For example, you should never provide root access on containers, restrict access to kubectl and the Kubernetes API, and ensure that Kubernetes is running an up-to-date version and clusters have a hardened configuration.
It is important to take advantage of security tools available in Kubernetes, including the Docker Statistics API, which can help collect system metrics. System metrics are useful for analysts who need to understand how a system is affected by container load when the system is running at scale.
Based on these metrics, DevOps and security teams can understand what's happening inside containers and pods. For example, they can determine if sensitive files are missing or unknown files have been added to containers, monitor real-time network traffic, and identify anomalous behavior at the container or application level.
Data Preservation and Investigation
In the event of an accident, it is important to preserve the necessary evidence for further investigation:
Do not shut down a node or container that appears to be compromised. This makes it impossible to determine the root cause.
Take a snapshot of the host that runs the suspected compromised containers.
Identify the critical evidence and get enough visibility to understand the root cause and impact. Try to analyze as many data sources as possible.
Incident Response Planning
Use the following best practices to ensure you are prepared when a container is attacked:
Assign incident response leads to act as key decision-makers in the event of a major incident.
Define an incident response plan with clear steps to follow for different types of security incidents.
Define communication and escalation channels that responders should use when a breach occurs.
Understand legal and compliance obligations, which may require reporting a breach within a specified period of time.
Conduct red team exercises and assessments to continuously improve your security defenses and prepare for a real incident.
Conclusion
In this article, I showed the basics of container security and incident response, explained the fundamentals of container forensics, and provided three building blocks for your container incident response plan:
Preventative measures—things you can do to reduce the attack surface and prevent attacks on containers in the first place.
Data preservation and investigation—ensuring that when an attack takes place, you can preserve data from containers to allow investigation and response.
Incident response planning—create a plan to define who is responsible for container incident response, what are the steps they should take when an attack occurs, and how to test the plan to ensure it is effective.
I hope this will be useful as you improve your organization's ability to respond to threats against cloud-native environments.
Related articles:
About the Author
You May Also Like
More Insights
