A cybersecurity incident can cause severe damage to the reputation of the organization and competitive disadvantage in the market, the imposition of penalties, and unwanted legal issues by end-users. On average, the cost of each data breach is USD 3.92 million as per this IBM report. The biggest challenges providing security in organizations are:

• Lack of skills and training in security tools and practices
• Lack of visibility and vulnerabilities
• Continuous monitoring of the current state of security

In the recent survey by PaloAlto Networks, State of Cloud Security report, it was discovered that 94% of organizations use one or more cloud platforms and around 45% of their compute is on containers or CaaS. The dominance of containers is increasing and thus the security threats. The top issues identified as being a threat in these reports are:

• Data exposure and malware
• Application vulnerabilities
• Weak or broken authentication
• Misconfigurations
• Incorrect or over-permission access
• Insider threats
• Credential leakage
• Insecure Endpoints

In this article, we will go through some of the container security best practices, we can follow and implement to reduce the security risks in the containerized workloads.

Top 10 security best practices to do to secure the application containers

1. Source base image from trusted repositories

When we create a container image, we often rely on the seed image sourced from popular private or public registries. Be aware that in the supply chain of the image production, someone can penetrate and drop malicious code which could open the doors to attackers. Just to give an example of this, in 2018, some hacker targetted British Airways web application with malicious javascript code by attacking their software supply chain. A couple of years back, Docker identified few images on Docker Hub which were having Cryptominers installed in the Image.

Below are some tips:

• When creating the container image, please use a hardened base image sources from a well known trusted publisher.
• Pick those images which are published frequently with the latest security fixes and patches.
• Use signed and labeled images (sign with Notary or similar tools) and verify the authenticity of the image during the pull to stop man-in-the-middle attacks.

2. Install verified packages

As much the sourcing of base image needs to be from trusted sources, the packages installed on top of the base also need to be from verified and trusted sources for the same reason.

3. Minimize attack surface in the image

What I mean by surface area is the number of packages and libraries installed in the image. Common sense is if the number of objects is less, the chances of having vulnerability is also reduced. Keep the image size minimal satisfying the application runtime requirements. Preferably, have only a single Application in one application container.

• Remove unnecessary tools and software like package managers (eg. yum, apt), network tools and clients, shells from the image, netcat (can be used to create reverse shell).
• Use the multi-stage Dockerfiles to remove software build components out of production images.
• Do not expose unnecessary network ports, sockets or run unwanted services (e.g. SSH daemon) in the container to reduce threats.
• Choose alpine images or scratch images or container optimized OS as compared to full-blown OS images for the base image.

4. Do not bake secrets in the image

All the secrets should be kept out of the image and Dockerfile. The secrets include SSL certificates, passwords, tokens, API keys, etc should be kept outside and should be securely mounted through the container orchestration engine or external secret manager. Tools like Hashicorp Vault, Cloud provided secret management services like AWS Secrets Manager, Kubernetes secrets, Docker secrets management, CyberArk, etc. can improve the security posture.

5. Use of secure private or public registries

Often the enterprises have their own base images with proprietary software and libraries which they don’t want to distribute in public. Ensure the image is hosted on a secure and trusted registry to prevent unauthorized access. Use a TLS certificate with trusted Root CA, and implement strong authentication to prevent MITM attack.

6. Do not use privileged or root user to run the application in a container

This is the most common misconfiguration in the containerized workload. With principles of least privileges in mind, create an application user and use it to run the application process inside the container. Why not root? The reason is that a process running in a container is similar to the process running on the host operating system except for the fact that it has additional metadata to identify that it is part of a container. With UID and GID of root user in a container, you can access and modify the files written by root on the host machine.

Note – If you don’t define any USER in the Dockerfile, it generally means that the Container will be running with root user.

7. Implement image vulnerability scanning in CI/CD

When designing CI/CD for the container build and delivery, include image scanning solution to identify vulnerabilities (CVEs) and do not deploy exploitable images without remediation. Tools like Clair, Synk, Anchore, AquaSec, Twistlock can be used. Some of the container registries like AWS ECR, Quay.io are equipped with scanning solutions – do use them.

8. Enable kernel security profiles like AppArmor

AppArmor is a Linux security module to protect OS and its applications from security threats. Docker provides default profile to allow the program to a limited set of resources like network access, kernel capabilities, and file permissions, etc. It reduces the potential attack surface and provides a great in-depth defense.

9. Secure centralized and remote logging

Usually, the containers log everything on STDOUT, and these logs are lost once they are terminated, it is important to securely stream the logs to a centralized system for audit and future forensics. We also need to ensure that this logging system is secured and there is no data leakage from the logs.

10. Deploy runtime security monitoring

Even if you deploy vulnerability scanning solutions based on repository data and take all necessary precautions, there is still a chance of being victimized. It is important to continuously monitor and log the application behavior to prevent and detect malicious activities.

“There is no silver bullet solution with Cyber Security, a layered defence is the only viable defence.” – ICIT Research

By implementing the above best practices, you can make it harder for the attacker to find ways to exploit your system. I am pointing out some tools and references that can be used to audit and secure the containers. Security is a vast topic, we haven’t covered Kubernetes specific controls in this article but stay tuned, we can have a follow-up article focussing on the Kubernetes security best practices.

Tools

To simplify the adoption of Security controls, I am suggesting few opensource and commercial offerings which can be used to discover the current state, to generate advisories for your workload.

• docker-bench-security – Official tool by Docker itself to audit the container workload according to the CIS Benchmark for Docker which is an industry-standard benchmark.

• Hadolint Linter for Dockerfile – Use the linter to do static code analysis of the Dockerfile. The linter helps in implementing the best practices. It can be integrated with popular code editors and integration pipelines.

• Clair – Clair is a popular static vulnerability scanning tool for the application container. It sources metadata from the various vulnerability databases on a regular basis. Alternatives are Anchore, Synk, Trivy.

• OWASP Cheatsheet – OWASP is an open community which is quite popular among security experts. This cheat sheet is a good starting point to start with.

• OpenSCAP for Container – Security Content Automation Protocol (SCAP) is a multi-purpose framework of specifications that supports automated configuration, vulnerability and patch checking, technical control compliance activities, and security measurement. It implements NIST standards.

• Sysdig Falco – Falco can be used to implement runtime security. It uses efficient eBPF to intercept calls and traffic for realtime monitoring and forensics. As hackers continue to evolve, new vulnerabilities are discovered and often not picked up by static scanning tools. A solution with machine learning capability, continuous behavioral monitoring, and advanced AI/ML-based engines can’t be ignored from the list of essentials. Learn more about getting started Kuberentes security using Falco in this handy blog post.

• Commercial offerings from AquaSec, Twistlock, Sysdig, Synk, Qualys for Enterprise-grade security tools, and solutions.

References

1. Container Image Security: Beyond Vulnerability Scanning
2. Top 20 Docker Security Best Practices: Ultimate Guide
3. 10+ top open-source tools for Docker security
4. NIST Application Container Security
5. CIS Benchmark for Docker
6. Best practices for building containers

Hope you enjoyed the post. Do share your thoughts by starting a conversation on Twitter if you have any interesting security incident or a preventable hack involving Containers you want to share with the community.

Looking for help with cloud native security consulting & implementation? do check out our capabilities how we’re helping startups & enterprises as an cloud native consulting & services provider.