Business Impact
- Faster onboarding of applications
- Improved user experience
Company Overview
Société Générale is France’s third largest bank by total assets and the sixth largest in Europe. Headquartered in Paris, the multinational financial services firm has divisions supporting global transaction banking, international retail banking, corporate and investment banking, private banking, asset management, and securities services.
Société Générale uses digital strategies to transform banking relationships with its customers, whether they be individuals, institutions, large companies or private banking clients. To keep up with changing digital usage by consumers, Société Générale is increasing its innovation in web
and mobile services to ensure its customers enjoy greater autonomy, simplicity, and security.
By taking advantage of technologies like Docker, containers with orchestration, platform-as-a-service (PaaS) and public cloud solutions like AWS, Société Générale is able to quickly develop value-added services to stay in step with new client behaviors. The firm’s journey to a modern cloud and a new architecture didn’t happen overnight. A phased approach – including a strategic focus on visibility and security – has helped Société Générale successfully adapt to containers and microservices while maintaining a laser focus on its primary goals of reducing risk and delivering high reliability.
Challenges
Everyone Wants To Do Docker
“Everyone wants to do Docker,” declared Thomas Boussardon, Middleware Specialist at Société Générale as he spoke to an audience at DockerCon 2017. To get there, Boussardon and his team, which include DevOps architect, Stéphan Dechoux, laid out a plan for container adoption and delivery of containers-as-a-service (CaaS) and platform as a service (PaaS) at the financial services firm. In the years since the start of the project, they have successfully built the platform onto which they have onboarded 20 applications with more than 50 applications in the pipeline for containerization.
“You have to understand that we have a lot of applications,” states Boussardon. This includes legacy applications, service oriented architecture (SOA), API REST, monolithic applications, and distributed applications. “In the investment bank we have over 1500 applications – we want people to run exactly in the same infrastructure.” The Société Générale container project seeks to transform and unify the company’s infrastructure with the goal of reaching a new level of agility, scalability, and automation for application rollouts while ensuring security, stability, and performance.
“We want to improve the user experience, to easily deploy apps, to upgrade easily, and decrease time to market,” explained Boussardon. “The use cases in banking are changing. We want now to be able to expose APIs on the internet. We must be able to expose everything in a DMZ to be ready to do Open Banking and to be able to do blockchain – and for this we are building this platform.” The team knew that Docker adoption would not happen overnight. To ensure success, they mapped a four-phase plan to guide their efforts.
Level 0: What Can We Reuse in a Docker Container Environment?
The first phase for the bank was simply to assess what they already had in place. Ideally, the software and hardware solution investments already made by the firm would be integrated and used in the new platform. Illustrating the scale of Société Générale’s IT equipment as it existed then, Dechoux posed this question to the session audience, “If we stack all of our datacenter equipment, what will be the height of this tower?” The answer?
More than 8x the height of the Eiffel Tower! “We can store more than 200 years of HD video, our global fiber network can cover the Tour de France race, and our grid computing can forecast weather faster than Meteo France (the French national meteorological service).”
“We didn’t want to rebuild and recreate everything. We have applications and systems and have people who can run them. What we want to do is build a platform that can host our applications but also use what we already have,” explained Boussardon. Existing services that Société Générale wanted to carry over to the new container environment included Jenkins for CI/CD, GitHub for source control, Nexus for its artifact repository, NetApp for persistent storage, Hortonworks for its data lake, Hashicorp Vault for secrets management, and Consul for its service registry. As much as possible they also wanted to maintain the tools used for its development stack. For Java apps this includes Netflix Open Source Software (OSS), Spring Cloud, RabbitMQ and Zipkin, and for .NET apps consists of .NET core, ASP.net, and Open Web Interface for .NET (OWIN).
Level 1: Introducing Docker Enterprise Edition
The next phase for Société Générale was to introduce Docker Enterprise Edition (EE) featuring Docker Engine to run containers, Docker Universal Control Plane (UCP) with Docker Swarm for orchestration, and Docker Trusted Registry (DTR) to storage images. The team also evolved its continuous integration and continuous delivery (CI/CD) pipeline practice to support Docker and the container lifecycle from test and dev to production. The work completed in this step took place within the first six months of the project.
Prior to Docker, the company utilized virtual machines (VMs) and bare metal servers to host applications. With the shift to containers, the team was tasked to define how the build and deploy process would work in the new platform. As much as possible, Société Générale wanted its new workflow to utilize existing technology to reduce disruption to developers. For its build process, they began to run the Jenkins master and Jenkin slaves in Docker containers.
Now, when the company creates an application, they pull from GitHub and Nexus to build Docker images. Once the application is tested, they push the images to its Docker Trusted Registry (DTR), which makes the application readily available to everyone who has a right to use it. Société Générale’s deploy process follows a similar workflow and provides the flexibility to schedule a deployment, to trigger a deployment after a change is done or a new image is available, or to manually deploy should the team decide to re-deploy an application. For production rollouts, Société Générale leverages the Docker UCP to send orders to Docker workers to deploy containers.
Solutions
Level 2: Stateful Containers and Docker Monitoring
For the next phase, 10 months into the project, Société Générale began onboarding applications into production. During this period they defined what was required to mature the capabilities of the platform to ensure successful operation in production and to enable a wider range of applications to be supported. Three critical enhancements were identified by the team for this phase. First, they needed to support stateful containers to ensure retention of critical data created by applications. Second, they also defined a requirement for a monitoring solution specifically designed to provide visibility into containerized infrastructure and applications. Third, they upgraded how they performed logging for the environment in conjunction with the monitoring solution.
Satisfying its goal of reusing existing infrastructure in the container environment, Société Générale adapted Docker to take advantage of its NetApp storage to support stateful applications that generate data the company wants to keep safe. Two Docker Volume plugins are utilized within the environment, one for NFS from NetApp, and one for CIFS from Netshare. With this functionality in place, the bank can now run stateful applications. Examples of these stateful services include its Jenkins Master, the company’s ELK stack with ElasticSearch, and data generated by batch jobs. “We need to be able to restart without losing information,” said Boussardon. With this rollout, Société Générale is able to onboard stateful applications and ensure that they don’t lose information even if the container crashes.
Choosing Sysdig for Container Monitoring
“Monitoring containers is not the same as monitoring old applications where you know the server, you know the IP, and you know the port. In containers it’s not like this,” explained Boussardon. “With containers, everything changes every time. Your application never runs on the same node, never runs with the same IP, and never runs with the same port. We had to find a solution to monitor this. That’s why we decided to use Sysdig. It gives us a way to introspect what is happening in our containers. It provides us dashboards and also sends metrics and all our logs to our data lake.”
Sysdig Monitor enables the team to see what is occurring not only within the physical environment, but also inside its containers and across a hybrid cloud estate that includes private data centers and public clouds including AWS. Société Générale’s development and operations teams are now able to monitor, alert, and troubleshoot resource usage across all layers of its containerized infrastructure.
Sysdig Monitor Featuring ContainerVision and ServiceVision Enables Société Générale to:
- Analyze process execution, file system activity, and network activity inside containers in a single view
- Visualize the dependencies in containerized environments to quickly isolate the root cause of performance issues
- Inspect application activity inside containers like HTTP error codes, URL response times, and database queries
With this insight, Société Générale can quickly identify and address any issues that occur.
For its initial rollout, the company deployed the Sysdig Monitor solution on-premises to enable the collection of metrics on internal infrastructure within its PaaS. This deployment model lets Société Générale leverage its existing capital investments and ensures it meets its defined security and compliance requirements.
Level 3: Microservices and Security
As Société Générale entered the next phase of its project, the platform was actively supporting a number of applications – both modern apps and traditional legacy apps. At this stage, 15 months into the project, the company began to onboard applications as microservices. Its approach was to enable a parallel run of applications, continuing to support apps on non-container infrastructure while concurrently running the same apps in production on containers. As Dechoux described it, “We already have microservices in the bank running on VMs or bare metal, and we want to be able to migrate to Docker. We want to have a parallel run with the same services running in containers in a canary or bluegreen scenario.”
With apps running in this cross-platform services configuration, Société Générale chose to maintain some services outside of containers. By taking this approach, the team maintains the immutability of its container images – a main principle of containers – but to inject at runtime the needed configuration for the application, the secret (e.g. API key, password), and certificate. “We want to build the image one time in development and the same image will follow all the next environments – UIT integration, pre-prod, and production, etc.,” explained Dechoux.
During this phase Société Générale also introduced Fabio, a containerized dynamic L7 load balancer that delivers “L7-as-a-Service” to route traffic with microservices deployments managed by Consul. Fabio checks with the Consul service registry and adapts its configuration based on state changes it discovers. Société Générale runs a dedicated Fabio container for each containerized application.
The final focus of this phase of Société Générale’s container project was to improve on security. “It must be robust and rock solid,” explained Dechoux. A key part of this process was to utilize Docker security scanning (DSS), an embedded feature of Docker EE that scans images for vulnerabilities. The team also scans Docker files and compose files using an in-house linter tool developed to check that everything respects best practices.
Level 4: Hybrid Cloud and Software-Defined Everything
Société Générale has set clear goals to incorporate public cloud, deploying cross-platform applications, and continuing to improve performance and security along the way to protect its customer data and deliver a great experience with its applications.
“The dream is to have some kind of cross-cluster between Amazure – Amazon Web Services and Azure – and our own site. To have something like a big giant cluster,” said Dechoux. Boussardon added, “We’ve got our own cloud, our private cloud, but we are incorporating public clouds like Amazon or Azure. We want to deploy our applications using immutability in other data centers and other environments.”
To help achieve this goal, the company has outlined a vision for “software-defined everything.” This includes moving toward software-defined networking to standardize the network between everything – VMs, bare-metal servers, and containers. The bank also sees software-defined storage as a technology that can improve the way they deliver storage, offering its customers different classes of service, such as gold, silver, and bronze, to provide a choice of capabilities for performance and persistence to satisfy diverse application requirements.
Because of the nature of its business, at each phase of its evolution, Société Générale diligently works to enhance security. For level 4, the team intends to focus on security policy enforcement. “We are a bank, so security is everywhere,” said Dechoux. “We want to be able to create some rules, like you cannot run somethings as root, you cannot mount a host volume in your container, you cannot run this kind of command, and you cannot modify a bin directory. We want to have some set of policies that can be applied dynamically and for all containers to ensure security. Especially if I want to expose it in a DMZ.”
Société Générale continues to imagine what else it might do to enhance its platform and deliver value-added services to its customers. One possibility is adopting Kubernetes for container orchestration. “It will be a discussion. Does the developer want to have some kube file to deploy or do they prefer the Docker tools?” said Dechoux.
In the final moments of the Société Générale presentation, Dechoux describes four technology areas that are of interest to the bank:
- Serverless: Dynamic allocation of machine resources to shift focus to applications, not servers
- Machine Learning: Predictive monitoring to pro-actively predict and detect failures
- Big Data: A large-scale, distributed operating system for big data applications to support varied processing approaches and a broad array of applications
- GPU: Deploy tasks with containers and use GPUs to accelerate calculations
The Move to Cloud and Containers Creates New Enthusiasm
What previously took one year now is able to be accomplished in three months. The bank now has more than 400 developers working every hour on the platform with a follow-the-sun model.
Boussardon summarizes the excitement of the teams about the new approach, “Everyone wants to onboard to the new platform. Everyone wants to help the platform to run. The UNIX team, the storage team, and the dev team all want to help. Everyone wants to work with Docker. It’s a change of mindset in the company. Everyone runs ONE project.”
Visit https://www.societegenerale.fr to learn more about Société Générale.
Take the Next Step!
See how you can secure every second in the cloud.