Kubernetes is the most popular container orchestration platform in
today’s cloud-native ecosystem. Consequently, Kubernetes security
is also an area of increased interest and attention.
In this blog post, first I will discuss the Pod Security Policy
admission controller. Then we will see how Open Policy Agent can
implement Pod Security Policies. In fact, during Kubernetes SIG Auth at
Kubecon + CloudNaticeCon North
America 2019,
Open Policy Agent / Gatekeeper was touched upon as a potential
alternative to Pod Security Policy.
Firstly, a brief look at containers, security and admission controllers.
Overview of containers and security
What is a container in Kubernetes?
Containers are lightweight, portable and easy to manage. There are no
separate physical / virtual machines for containers running on the same
host. In other words, containers share the resources, hardware and the
OS kernel of the host where they run.
Therefore, it becomes very
important that operators have appropriate security around what processes
can run inside containers, what privileges these processes have, whether
a container will allow escalation of privileges, what images are used
and so on.
What is a Pod in Kuberentes?
A Pod is the
basic execution unit of a Kubernetes application – the smallest and
simplest unit in the Kubernetes object model that you create or deploy.
It is a group of one or more containers with shared storage/network, and
a specification for how to run the containers.
Therefore, when enforcing
security policies on containers, we inspect and apply security policies
on Pod specifications. So, how are these policies enforced? Using
Admission Controllers.
What are admission controllers?
Admission Controllers are part of the kube-apiserver. They intercept
requests to the Kubernetes API server before configuration is stored in
cluster settings (etcd). An admission controller can be validating (one
which validates the incoming request) or mutating (one which modifies
the incoming request) or both. Refer Kubernetes
Documentation
for a quick glance at various admission controllers.
Open Policy Agent as an admission controller
Open Policy Agent (OPA) is an open source, general-purpose policy engine
that makes it possible to write policy as code. OPA provides a high
level declarative language – Rego – to enable policy as code. Using OPA,
we can enforce policies across microservices, CI/CD pipelines, API
gateways and so on. One of the most important use-case for OPA is
Kubernetes policy enforcement as an Admission Controller.
OPA as an admission controller allows you to enforce policies such as
non-root user, requiring specific labels for resources, ensuring all
pods specify resource requests and limits and so on. Basically, OPA
allows you to write any custom policy as code using Rego language.
The policies are written in Rego and loaded into OPA running as an
admission controller on your Kubernetes cluster. OPA will evaluate any
resource create / update / delete request to Kubernetes API server
against the Rego policies. If the request satisfies all the policies,
the request is allowed. But even if a single policy fails, request is
denied.

(image source)
Read more about OPA, Rego and use as admission controller in OPA
documentation.
Now, let’s go into details of Pod Security Policies.
What is Pod Security Policy?
Pod Security Policy (PSP) is a cluster level resource that is
implemented as an Admission Controller. PSP allows users to translate
the security requirements into specific policies governing pod specs. At
first, when a PodSecurityPolicy resource is created, it does nothing.
And in order to use it, the requesting user or target pod’s service
account must be authorized to use the policy by allowing the “use” verb.
You can refer to Enabling Pod Security
Policies
on Kubernetes documentation.
Note that the PSP admission controller acts as both validating and
mutating admission controller. For some of the parameters, the PSP
admission controller uses default values to mutate the incoming request.
Further, the order is always to first mutate and then validate.
What all parameters can we control using PSP?
The table below gives a brief summary of various parameters and fields
used in PSP. A detailed explanation is available in Kubernetes
documentation.
So, can we implement Pod Security Policies with OPA?
I mentioned earlier that Rego language allows us to write any custom
policy as code. That means, we can write the Pod Security Policies
described above using Rego and enforce with OPA as an admission
controller.
Let’s take a quick look at a Rego policy for implementing “privileged”
pod security policy. You can try out this policy on Rego playground here.
package kubernetes.admission
deny[message] {
#applies for Pod resources
input.request.kind.kind == "Pod"
#loops through all containers in the request
container := input.request.object.spec.containers[_]
#for each container, check privileged field
container.securityContext.privileged
#if all above statements are true, return message
message := sprintf("Container %v runs in privileged mode.", [container.name])
}
So, what does this policy do? It will return a message if any container
in the input request is running as privileged container.
Pod Security Policies (PSP) in action
Let’s see this policy in action with a basic minikube based tutorial.
First, setup OPA as admission controller by following the tutorial from
OPA documentation.
This tutorial loads an ingress validation policy. Instead of that, we
will load the privileged policy shown above.
Once OPA is setup as admission controller on minikube, create a file
privileged.rego using the policy above. Then, create the policy as a
configmap in “opa” namespace.
kubectl create configmap privileged-policy --from-file=privileged.rego -n opa
Wait for the policy to be loaded to OPA. You can check that the policy
is loaded when an annotation openpolicyagent.org/policy-status:
'{"status":"ok"}'
appears on the configmap.
Now, let’s create a deployment with privileged container using the
following manifest :
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: default
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
securityContext:
privileged: true
When you try to create this pod, you will notice that the pod is denied
by Open Policy Agent.
Error from server (Container nginx runs in privileged mode.): error when creating "privileged-deploy.yaml": admission webhook "validating-webhook.openpolicyagent.org" denied the request: Container nginx runs in privileged mode.
Likewise, we can write policies for other Pod Security Policy parameters
and enforce using OPA.
While in this tutorial, we loaded the policy using configmap for
simplicity, thats not the best strategy for production deployments. For
production deployments, you can configure OPA to download the policy
bundles from an external bundle server periodically. All your policies
can be maintained in this bundle server. And OPA will keep itself
up-to-date by periodically downloading the policies. Refer Bundle
API
for more details.
In short, using OPA, we can enforce the Pod Security Policies. And not
only that, we can enforce any other custom security / standards based
policies using same setup.
What are the key benefits of using OPA for Pod Security Policies?
Some key benefits that we get from this approach are :
- Centralized management of all your policies (PSP and other custom
policies) in one admission controller instead of managing those
disparately.
- Shift-Left – Enforce the same policies also in the CI/CD pipeline
thus implementing Policy-as-code throughout the stack.
- Ability to maintain OPA policies in a source control repository like
Git. OPA provides http APIs to dynamically manage the policies
loaded.
- Stream the policy decisions to an external logging / monitoring tool
of your choice.
- Customize the denial message as per your setup/implementation.
In addition, We can deploy OPA as a mutating admission controller. That
way, you can also implement the mutating behavior of PSP Admission
Controller.
[Styra Declarative Authorization Service]
If this use case of OPA interests you, it may be worth checking out
Styra Declarative Authorization Service (DAS) for Kubernetes. Styra DAS
makes it easy to enforce and continually monitor security, compliance
and operational policies.
Styra DAS acts as a single control plane over all your clusters.
Moreover, using a pre-built policy library for OPA, you can get started
within minutes. Also, it provides a rich policy editor for writing
custom policies, impact analysis by replaying decisions against changes
to the policies, decision logging, and enterprise readiness with SSO,
access control.
Recently, Styra has also added the Pod Security Policy pack to the
pre-built policy library. So, this takes away all the additional efforts
to write the PSP policies yourself. And, you benefit from the continuous
enhancements to the policies as per industry standards.
Conclusion
We can implement Pod Security Policies with OPA effectively. Moreover,
this allows us to model security policies as code. And, all this in a
single OPA admission controller.
Lastly, I hope you liked this blog post. If you have any feedback or
queries, do reach out to me on
gaurav@infracloud.io / @Gaurav on OPA
slack /
@GGCTwts on twitter.
Looking for help with Kubernetes adoption or Day 2 operations? learn more about our capabilities and why startups & enterprises consider as one of the best Kubernetes consulting services companies.
References :
- Pod Security Policies
- Kubernetes API
- OPA Documentation
- Styra WhitePaper – K8sSecurityAdmissionControl
- How guardrails secure and accelerate K8s deployments
- KubeCon + CloudNativeCon North America 2019 – SIG Auth Update and
Deep
Dive