Can you believe it has already been more than a week since KubeCon NA ended?

We can’t because our experience is still ongoing. KubeCon NA is the largest event in cloud native space. More than 20K people attended and 100+ were sponsoring this year’s event.

Did you miss attending the event or could not attend the interesting talks because of so many parallel tracks? Heck no, you can read this comprehensive blog and know about all the exciting sessions/things that happened in the KubeCon NA 2020.

We have segregated this article based on the following tracks.

• 101 tracks

• CI/CD

• Runtimes

• Service Mesh

• Networking

Each track has the key takeaways from the talks that we found interesting.

Are you ready? Let’s dive in.

101 tracks

Clean Up Your Room! What Does It Mean to Delete Something in K8s - Aaron Alpar, Kasten

Aaron covered how to delete objects and what Kubernetes does to coordinate the delete process. Some specifics of deleting pods, persistent volume claims, deployments, and stateful sets got covered. Some of the key points from the talk:

• Finalizers are keys to resources that control their garbage collection. Finalizers are keys that are designed to be used by controllers to tell them which cleanup actions have to be performed before the resource deletion.
• ‘Dead’ finalizers prevent deletion of the resource. They are the finalizer strings that controllers do not understand and do not know how to deal with.
• Suppose an object has a finalizer on it, and you attempt to delete it. In that case, it will remain in finalization till the controller removes the finalization keys or the finalizer is removed by kubectl.

Five Hundred Twenty-five Thousand Six Hundred K8s CLI’s - Phillip Wittrock & Gabbi Fisher, Apple

With the Kubernetes ecosystem’s success, users now have many choices when it comes to Kubernetes tools. While it’s great for users to have options, many choices can make it difficult for Kubernetes users to make decisions or know where to start. In this talk, Gabbi and Phillip provided an overview of well-known and up-and-coming Kubernetes CLIs. Some of the points they covered:

• Categorized CLIs in understandable and straightforward terms.
• Categorized CLIs as Templating (Helm, ytt), YAML composition (Kustomize), Domain-Specific Languages (Cuelang, Isopod, JSonnet), General purpose Languages (Pulumi, kpt-sdk), and Controller-esque (kpt).
• Gauged these categories using Abstraction, Variance, and cross-cutting abstraction.
• Templating provides better abstraction, while YAML composition provides better Variance. DSL has all three but increases complexity. Controller-esque can be used primarily for abstraction and cross-cutting abstraction, but it is also complicated. General-purpose languages more or less can be used for all three and are also user friendly.

10 More Weird Ways to Blow Up Your Kubernetes - Jian Cheung & Joseph Kim, Airbnb

Airbnb was at KubeCon again to share ten more ways they have messed up in just the past year. They have shared their experience on what can go wrong while running on Kubernetes. This was one of our favorite talks. Some of the key points that were covered:

• We should be aware of Mutating Admission Controllers. It can affect deployments and replicasets.
• There is a correlation between CPU throttling and OOM. It can be due to garbage collection threads or backpressure overloading the services.
• Cross-checking ECR images are always present, prevents breaking of pods due to ImagePullBackOff.
• Be aware of k8s apply ordering.

Inside Kubernetes Ingress - Dominik Tornow, Cisco

Kubernetes Ingress is a core abstraction of Kubernetes: K8s Ingress grants access to K8s HTTP Services outside the K8s Cluster. In this talk, Dominik walked us through a dependable mental model of every aspect of Kubernetes Ingress and comfortably contrasted K8s Ingress with alternative abstractions like the Ambassador API Gateway. Some of the notable points Dominik covered:

• Ingress for Kubernetes consists of two components: Network Ingress and Kubernetes Ingress.
• Network Ingress: admission of traffic. Kubernetes Ingress: routing of traffic
• Network Ingress can be defined as the set of all flows that originate outside the cluster and terminate inside the cluster.
• Kubernetes Ingress can be defined as a set of all flow pairs such that the first flow terminates at the proxy, the second flow ends at a pod, and there exists the rule in the decision table so that the request of the first flow matches the conditions in the rule and the pod matches target service of the rule.

Stop Writing Operators - Joe Thompson, HashiCorp

Since the introduction of the operator pattern by CoreOS in 2016, operators and even operator coding frameworks have proliferated seemingly without limit. But should you write an operator? If not, what should you be doing instead? In this session, Joe Thompson gave a quick overview of the operator ecosystem’s state, points out situations where writing and using them may not be ideal, and offers opinions on alternatives that may be more effective for maintaining your application. Some of the key takeaways:

• The operator pattern may be getting overused. It was initially meant for stateful apps.
• All operators are controllers, but not all controllers are operators. A controller that does nothing stateful is just a controller.
• Operators are generally not needed if there is no statefulness or K8s handles the state and makes the system more complex by over-abstraction.
• Operators also make security auditing more complicated as it needs to reverse engineer the app to check for security flaws.
• Alternatives for operators can be CRDs, Helm chart, etc.
• Things to lookout while writing operators:
  • Maintain loose coupling such that you should not update your operator every time your app is updated.
  • No code is better (maintain less code).
  • Write code you are familiar with.

Managing Developer Workflows with the Kubernetes API - Colin Murphy, Adobe

Kubernetes is incredibly powerful. Its extensibility allows for limitless varieties of architecture. But how do you get hundreds of engineers to follow the same conventions when creating their applications? Some of the critical things he discussed:

• Showed ‘porter2k8s’ open source Adobe Kubernetes client to manage developer workflows.
• Having Kubernetes client manage developer workflows offers greater control over the entire process, better secret management, and easy interfacing with existing deployments.
• KUTTL (Kubernetes Test Tool) declarative te2e testing tool.
• Test manifest with pre-defined assertions of required states or errors.
• How Adobe uses a modular structure of test suite which consists of several tests which in turn consists of several test steps.

CI/CD

Everything You Should Be Doing, But Aren’t: DevSecOps for K8s Workflows - Steven Terrana, Booz Allen Hamilton & Dan (POP) Papandrea, Sysdig.

Steven and Pop described a defense-in-depth approach to secure production workloads running on Kubernetes. They showed a live demonstration of using CNCF projects like Helm, OPA, Falco, and Argo to secure Kubernetes clusters. Some of the key takeaways from the session:

• DecSecOps is integrating security in every step of the software development lifecycle.
• This includes securing dependencies, code, artifacts, configurations, infrastructure, interfaces, maintaining the accessibility of those interfaces, and runtime security.
• Runtime security can be achieved with Falco, which uses pre-defined security rules for runtime assertion and provides alerts in case of violation.
• Open Policy Agent can be used to provide granular RBAC to secure infrastructure access.

Runtimes

Kubelet Deep Dive: Writing a Kubelet in Rust - Kevin Flansburg, Moose Consulting

Kubelet is a critical part of the Kubernetes project. Many intermediate Kubernetes users could benefit from a deeper understanding of Kubelet behavior. This talk discussed the development of a Kubelet in Rust and offered a deep dive into the expected behavior and implementation of Kubelet. Some of the critical points Kevin discussed in this session:

• Rust can produce very high-performance software and frequently matches c++ in performance benchmarks. In part, this is achieved by a policy of zero-cost abstractions, wherein abstract programming features like generic types incur zero runtime cost. This not only helps in performance but also increases the efficiency that is needed in the data center and at the Edge.
• Rust employs a strongly typed system. Rust’s strong concept of memory safety contributes to easier concurrency and parallel programming. It has excellent error handling capabilities.
• Krustlet is developed in Rust following wasm32-wasi architecture and uses wasmtime-based runtime instead of a container runtime to schedule workloads.

Service Mesh

Extending Service Mesh to the Edge - Stephen Wong, OPNFV Clover

The advent of edge computing has led to the trend of splitting applications into edge and cloud components, which makes service mesh a great fit to enable unified application network policies for inter-container communications, regardless of where individual microservice is deployed. Here’s what Stephen talked about in this session:

• Network Functions Virtualization (NFV) is Telco’s way of an attempt to cloudify network services.
• Edge primarily focuses on distributed computing connected via Cloud backends. This requires WAN connectivity between the endpoints/microservices. Software-Defined WAN (SD-WAN) can be implemented for more granular abstraction to create multiple virtual WAN links. A service mesh like Istio can be used to map route related rules and policies into different WAN links depending on their respective characteristics.
• Edge computing is as much, if not more, a networking problem as it is a computing problem.

Networking

Improving K8s Application Experience Over SD-WAN - Alberto Rodriguez-Natal, Cisco & Mark Church, Google

Software-Defined WAN (SD-WAN) technologies have democratized access patterns across the Internet through latency reduction, throughput improvement, and packet loss prevention. However, integration between SD-WAN and K8s is still an emerging field, despite the automation opportunities. This talk showed us how to link K8s application attributes with SD-WAN network capabilities. Some of the key takeaways from the session:

• Kubernetes apps/endpoints are located in many different networks. Moreover, users/clients are situated in many other networks. And these networks are heterogeneous. SD-WAN unifies these underlying sets of networks as a single programmable and controllable WAN fabric.
• For this, DevOps/K8s attributes like App identity, Cluster, Zone, Network, annotations, etc., are mapped to NetOps/SD-WAN attributes like Network security policy, Network Latency policy, Network Bandwidth policy, identities, etc. using a defined policy. SD-WAN controllers and Kubernetes are coupled together using Cloud-Native WAN (CNWAN) integration.
• The core components that enable this are:
  • CNWAN operator: Deployed in Kubernetes cluster. Continuously checks for Metadata as annotations. Sends the metadata to a Service Registry.
  • CNWAN reader: Reads the metadata from Service Registry and passes it to CNWAN adaptor.
  • CNWAN adaptor: Extracts IP: Port from policy mapping and provides the details to SD-WAN controller, which routes the traffic accordingly.

We enjoyed attending the KubeCon NA and interacting with the community. We are hoping that you will find this article useful. Do let us know through comments if we have missed talking about any good session or any point in general.