Sysdig Provides Robust Security for Kubernetes Clusters Supporting Mercari's Flea Market App
Business Impact
- Strong intrusion detection capability and low operational management workload
- Accelerated incident response and troubleshooting
Company Overview
Founded in February 2013, the company’s mainstay product, “Mercari,” is the most widely used flea market app in Japan, with a monthly user count of 17.55 million and an annual gross merchandise volume of over 625.9 billion yen.
Since its founding, Mercari, Inc. has continued to diversify its business. In September 2014, for example, the company expanded to the U.S., and in February 2019, it launched the contactless payment service “Merpay.”
Challenges
Monitoring and Logging Kubernetes Clusters To Protect Sensitive Information
Mercari was established with the mission of “creating a global marketplace that creates new value”. Over time, the Mercari flea market app has grown to become one of the leading products of its kind. Mercari, Inc.’s goal is to enable the circular use of finite resources and thereby pave a path toward a more wholesome society.
Originally, Mercari had a monolithic architecture for its service platform. However, as the number of services increased and a large number of people started to develop in parallel, further improvement of productivity became an issue. Therefore, at the end of 2017, the company decided to make Mercari a micro-service. It adopted Google Cloud Platform (GCP) as its cloud infrastructure and Google Kubernetes Engine (GKE) as its container management tool, and gradually shifted to a microservices architecture.
In the meantime, microservices have changed the way development organizations are organized, and back-end engineers are now responsible for everything from development to operations. As a result, new security strategies became necessary based on the assumption that developers may touch the infrastructure. On a Kubernetes cluster, multiple microservices are running. Therefore, the scope of influence of each microservice needs to be minimized.
Generally speaking, Kubernetes in its current state is not secure by default, and due to the nature of sharing kernels between containers, Mercari believes that it is a technology that needs to be used with careful consideration of security risks. Although Mercari has taken action, for example, if Kubernetes is used in the default environment, there are issues, such as easy privilege escalation.
“We wanted to protect against unauthorized intrusions into containers due to vulnerability exploitation from outside or theft of critical terminals,” said Mr. Hiroki Suezawa of Mercari, Inc.’s Security Engineering Team. “If a node is elevated to privilege, it can acquire credentials on other containers.”
Protecting confidential information is a top priority for Mercari, which handles a huge amount of transaction information. The company has accordingly adopted various measures to bolster Kubenetes security, such as penetration testing and cluster hardening. In the meantime, monitoring and recording of operation logs emerged as a means to further strengthen security.
“We decided to consider new measures to mitigate the risk by immediately detecting and responding to unauthorized intrusions into our infrastructure and applications through monitoring, and by being able to obtain and record operation logs on Kubernetes,” explained Mr. Suezawa.
Solutions
Mercari Adopts the Sysdig Platform
Mercari adopted the Sysdig Platform for its strong intrusion detection capability and low operational management workload. To implement these additional security measures dedicated to container management, Mercari, Inc. evaluated three candidate products against three criteria: intrusion detection capability, recording capability, and operational management. The team ran a PoC for each product, scored it based on the three criteria, and chose the product with the highest total score. The Sysdig Platform, a security monitoring platform for container and Kubernetes environments, received the highest overall score.
“In the PoC, we tested every kind of Kubernetes attack known to us,” said Mr. Suezawa. “For example, we created containers that were deliberately vulnerable. We also enacted scenarios where credentials for operations on Kubernetes clusters are stolen. The result was that Sysdig scored highest in ‘unauthorized access detection.’ Another compellingly factor was that it has a stable SaaS version. Sysdig is offered in forms that are perfect for cloud-native environments and can be implemented simply by deploying its agents on Kubernetes. It can easily be managed from the console without the need for in-house operations.”
The fact that Falco, an open-source Kubernetes runtime security tool created by Sysdig and contributed to the CNCF, is embraced by the community, and is constantly being improved also played a role in the team’s decision, according to Mr. Suezawa. Furthermore, although Sysdig itself is a paid tool, many of its peripheral tools are open source, complete with a process through which they can be swiftly modified to meet Mercari, Inc.’s requirements or incorporate its code.
SCSK, the exclusive distributor in Japan, provided support for the selection and implementation of the PoC. The three products compared were all from overseas, but only Sysdig was able to discuss container security in depth, and SCSK was able to communicate with Sysdig’s headquarters to request new features.
“For some security products, user support is limited to that from its distributors in Japan,” noted Mr. Suezawa. “In the case of Sysdig, SCSK and the Sysdig Japanese subsidiary provided us with joint support, which deepened our understanding of the product. We were also able to use Slack, which our development team uses regularly for speedy communication.”
The Sysdig implementation was completed in approximately one month. The actual work was carried out by Mercari, Inc.’s Kubernetes management team. To minimize impact on performance, Sysdig agents were deployed to Kubernetes clusters in an incremental fashion. This strategy worked, no significant impact was observed upon implementation.
Implementing Sysdig allowed Mercari, Inc. to monitor unauthorized access and greatly improve Kubernetes cluster security. “Because we’re immediately notified whenever an unauthorized access incident occurs, we can take swift action to address the problem. For example, a while back, when a new Linux vulnerability (CVE-2020-14386) was reported, the Security Team was able to analyze how it could be exploited and using Sysdig, immediately protect against any related attacks,” said Mr. Suezawa. “Also, because developer operation logs are recorded and presented in a user-friendly way, we can easily check what happened should an incident occur, which is very reassuring.”
Sysdig Impact and Future Plans
“Google has created a container runtime called gVisor to achieve container orchestration with Secure by default and high isolation between containers. However, I believe that the need to monitor the intrusion itself will remain,” explains Mr. Suezawa. Mercari, Inc., SCSK, and Sysdig will continue to work in close coordination to solve security issues.
To learn more about Mercari, visit www.mercari.com.
Take the Next Step!
See how you can secure every second in the cloud.