Monitoring and securing Kubernetes-based healthcare services on Google Cloud: Cota Healthcare

As Cota Healthcare moved to Kubernetes on Google Cloud, it chose Sysdig for Kubernetes monitoring and container security. With Sysdig, Cota accelerates healthcare service development, improves capacity planning, fixes issues rapidly, and strengthens its security posture. Cota builds solutions for healthcare that analyze, visualize, report, and manage real-world evidence and deliver precision medicine. It has an important mission, merging technology and medicine to improve the lives of cancer patients. The company takes a big data approach to increase treatment effectiveness and lower healthcare costs. And, because the company is entrusted with millions of patient records, visibility and security are paramount to its business.

Investing in Kubernetes infrastructure.

Cota began using Docker containers for application delivery in 2017. To keep pace with its expanding business, the company adopted Kubernetes to orchestrate its application stack. “As we were moving to Kubernetes, we decided to look where it would make sense to land our infrastructure to suit how we were building it. Ultimately we made the decision to use Google Cloud because of its pedigree with Kubernetes,” stated Ashley Penney, VP of Infrastructure at Cota Healthcare. “We also knew we would need visibility into everything. You can’t operate a system where you don’t have any idea what’s happening. That doesn’t work.” The team at Cota had experience with a number of monitoring and security products. As a result, they knew they needed a solution that aligned with their new Kubernetes infrastructure. Cota found Sysdig to be the easiest way to get in-depth cloud infrastructure and container metrics with full Kubernetes context. Sysdig also offered security capabilities designed specifically for the unique challenges of containers. “I used other solutions that weren’t a great fit for containers at a previous company,” Penney explained. “With Sysdig, Kubernetes makes sure an agent is running on each of our nodes and immediately we get insight into what’s happening. It’s incredibly powerful. Without any code changes we can look at things like who is making SQL queries, how slow are these queries, and who talks to who.”

Securing sensitive data and systems.

Cota is a HIPAA compliant organization. Because of this, the company must work diligently to ensure secure management and access to systems and data. “We try to move very fast but also have a very careful security posture. Sysdig is at the very heart of the security conversation. It is effectively the first line of defense,” described Penney. “It gives us both the container and host visibility that helps us establish a baseline for how the system should behave and then to spot things that are outside that baseline. We’re able to guard against data exfiltration – that’s the kind of thing that we’re concerned about.”

Increasing developer velocity.

With its success, Cota is doubling the size of its software engineering team. Keeping engineers productive and efficient is a critical requirement. “We’re always trying to get better at enabling developer velocity.” related Penney. “The role of my team is to provide self-service tools that developers can use to build and run the customized services they develop. Because things happen in the environment that we might not even be aware it takes a solution like the Sysdig platform to help us keep up with what’s happening today versus yesterday – and that will change again tomorrow.” Sysdig reduced the time it took to start getting useful metrics. This makes it easier for developers to get on board. Penney pointed out, “Our developers don’t have to spend weeks instrumenting every HTTP call to be able to look for things like regressions that impact performance. It’s this foundational layer that lets us move faster and worry less about breaking stuff with very little effort from our side.”

Improving visibility with custom metrics.

Cota takes advantage of Prometheus and StatsD custom metrics to get even more meaningful data about application performance. These metrics are auto-collected for monitoring and visualization with Sysdig. According to Penney, “From the infrastructure team’s perspective it’s nice that we can tell our developers to emit metrics with StatsD or Prometheus. You won’t have to think about it. They’ll just show up in Sysdig. There are a ton of Prometheus exporters we can use. We can see things like how many patients records were processed, how many succeeded, and how many failed. We had that information in log files before but unless somebody was actively checking for it, it was easy to miss. Now developers have a single dashboard they can go look at.”

Reducing costs with better capacity planning.

One of the biggest uses of Sysdig at Cota is to aid with capacity planning. For example, Penney and team use performance metrics from Sysdig Monitor to help right size their infrastructure. “We recently moved from self-hosted Postgres into RDS. As part of that process we used metrics from Sysdig to decide how big the database instance should be. It was a significant cost savings because the metrics made it very clear that the box we had been using wasn’t busy enough to justify the resources we had given it. In this case it’s not so much finding performance bottlenecks but helping us find things like ‘average CPU usage is 2%,’ which happens all the time.”

Accelerating Troubleshooting.

Seeing system behavior is just one advantage of using Sysdig. Additionally, Penney’s team is able to troubleshoot and discover the cause of issues that would otherwise remain hidden. “We had some reports from the guys who work overnight that the application was incredibly slow. It was taking minutes to do things. We were able to use Sysdig Monitor to drill in and show that there were no slow queries. But then we saw that the number of containers running kept dipping below one. We dug a little deeper and realized that it was actually running out of memory continuously. We were able to spot that and fix it.”

Conclusion.

Sysdig helps Cota Healthcare more efficiently operate and protect the software and systems that serve its business. As a result of selecting Sysdig, Cota has a single solution that helps reduce costs and ensures the data hosted with Docker and Kubernetes is secure, reliable, and available. Interested in reading more? Download the entire case study here.