Client Details

The customer is a global leader in Financial Technology for Banking, Insurance, and other Financial Services.

Context

The customer builds and runs Banking, Treasury, and Risk Management software for financial institutions worldwide. This suite of business applications relies on Platform applications and Big Data applications for logistic and analytics services.

The customer had decided to containerize the applications and adopt relevant tools to manage those. They decided to run the applications on Kubernetes. They were looking for the right partner who could consult, implement, and train them on Kubernetes and other cloud-native technologies and practices.

Challenges

• Customer operated in multiple regions and wanted to setup applications resilient to disaster recovery and ensure high availability for end customer.
• Customer wanted to migrate all the applications to Kubernetes but only some were containerized. Some that were containerized were not in the right way and there were anti-patterns which were discovered during implementation.
• Customer wanted to design continuous integration and delivery platform and pipelines Kubernetes applications.
• Customer ran big data applications for data analytics hence using a scalable storage backend for Stateful containerized services like MongoDB, Spark etc. was important.

Problem Statement

• Various anti patterns around infrastructure and application development existed
• Disaster recovery and Backup was manual, untested and not reliable.
• Storage was tightly couples to cloud provider hence multi cloud was stalled.
• Observability was ad hoc and not optimal

Solution

The customer was looking to modernize their applications and infrastructure by adopting cloud native technologies.

Phase 1:

Containerization and fixing anti patterns

• Containerized and migrated applications like Kong (API Management), Keycloak (IAM) to Kubernetes along with business applications.
• Some of microservices were using anti patterns in Dockerfiles, those were changed to use right way of building Docker images.

Phase 2:

Setting up Kubernetes cluster and repetitive process

• Customer was using AWS as primary cloud and wanted to setup an active passive style cluster for disaster recovery and achieving high availability.
• Infracloud team developed Terraform scripts and used Kops tool to provision entire infrastructure including the Kubernetes cluster.

Phase 3:

Transforming CI/CD pipeline to be cloud native

• A CI/CD pipeline was developed using Jenkins 2.0 pipeline DSL and Helm charts for Kubernetes.
• The entire pipeline was based on changes to one of following repositories. This ensured every change going to a runtime cluster was through source control
  • Source code
  • Helm Charts
  • The environment configs (Values.yaml) of Helm chart
• For stateful applications as well the pipeline was defined using Jenkins pipeline DSL and Helm charts.

Phase 4:

Operations monitoring and auto scaling

• Customer was using monitoring tools which won’t play well with Kubernetes and dynamic nature of infrastructure. We setup and migrated applications to use Prometheus as the tool for monitoring. Customer decided to keep Datadog as monitoring tool for rest of infrastructure and managed services.
• Autoscaling was setup for certain stateless microservices using core and custom metrics.
• The nodes were setup to auto scale up or down using Kubernetes autoscaler and based on need.
• A FluentD, Elastic and Kibana based stack was used to collect and analyze logs from all applications.

Phase 5:

Security

• Initially sealed secret and Vault were evaluated for securing the secrets of application. Eventually Hashicorp Vault was chosen due to it’s maturity and features such as rotating secrets and certificates.
  • As part of CI/CD workflow the images that were built were scanned for CVEs.
  • The runtime defense agent of Twistlock secured runtime components.
• RBAC system with strict policies was built to enable complete isolation of services and communication.

Solution – HA multi region cluster

Solution – Kubernetes Based Continuous Delivery

Multi-Cloud Kubernetes was setup using KOPs on AWS and Azure and all the applications were managed with Helm.

Benefits

• Fixed anti patterns, got a few applications on Kubernetes with “CI/CD” best practices.
• Built a HA and automated Kubernetes platform with built in DR.
• Implemented observability with EFK stack
• Introduced “Infrastructure As Code” practices

Why InfraCloud?

• Our long history in programmable infrastructure space from VMs to containers give us an edge.
• We are one of cloud native technology thought leaders (speakers at various global CNCF conferences, authors, etc.).
• DevOps engineers who have pioneered DevOps at Fortune 500 companies.
• Our teams have worked from data center to deploying apps and across all phases of SDLC, bringing a holistic view of systems.