About the European Space Agency (ESA)

• Founded in 1975, ESA is a 22-member intergovernmental space exploration organization.
• The European Space Operations Centre (ESOC), one of ESA’s premises in Germany, is home to the engineering teams that control spacecraft in orbit, manage the global tracking station network, and design and build the systems on the ground that support missions in space.
• ESA has operated more than 87 missions from ESOC, ranging from understanding our own planet to exploring our solar system and beyond to help advance scientific knowledge.
• ESOC is responsible for a wide range of IT infrastructure and systems to help fly these unique missions, having been responsible for flying more than 80 custom-built satellites since it was established.

You can’t explore space without infrastructure.

It’s not just expert personnel, satellites, and spacecraft: it takes mountains of software, computer systems, data and networks to get these missions off the ground. These missions are often unique – testing the limits of our technological know-how, having never been done before with each mission presenting its own unique challenges. Building and deploying these systems on a mission-bymission basis often involves significant complexity and costs.

ESA decided to embark on an ambitious challenge: to fly double the number of satellites it does currently by 2030, looking to technology to help it achieve this goal through faster, automated software delivery, shared services, more efficient use of resources, and lower costs of deployment.

To help double its space missions, ESA engaged Canonical to deliver the architecture, design and deployment of Kubernetes and Ceph for persistent storage, along with PostgreSQL and Kafka, needed by the mission control system. ESA also chose Canonical’s Ceph distribution to consolidate the various storage technologies in use. Canonical’s Kubernetes distribution allowed ESA’s Missions Operations Infrastructure – IT (MOI-IT) team to easily deploy its workloads, integrated with the existing IT infrastructure services, and support ESA missions and those of its partners, thanks to automatic updates and deployment, 24/7 monitoring and reduced system complexity.

Kubernetes was perfect for ESA’s next generation multi-mission platform at ESOC. This open source system helps organisations orchestrate their workloads more efficiently with automation. However, Kubernetes’ fast-moving and steep learning curve makes it more demanding to manage. To reduce the engineering workload and management requirements of this new system, ESA chose Canonical to manage its Kubernetes deployments, and provide updates, monitoring and maintenance.

The systems Canonical deployed and now manage for ESA mean that ESOC engineering teams can quickly and centrally manage and deploy mission-critical infrastructure and software, with more security, reliability, and control, and with less downtime and reduced risk of system failures.

In addition, ESA has developed a number of AI- supported software tools that are being used operationally by missions. These tools need a platform to run on, and by utilizing Kubeflow and Spark on top of Kubernetes, ESA was able to focus on solving the operational requirements instead of managing the supporting infrastructure. Shared supporting infrastructure for all these services have been deployed on a MicroCloud, in order to ensure high availability in the event of server failure with the live migration feature.

Copyright: ESA/DLR/FU Berlin/G. Michael CC BY-SA 3.0 IGO

Challenge

From the famous Ulysses space mission in 1990 and the historical Philae comet landing in November 2014, to the Billion Star Survey and Euclid’s exploration of dark matter and energy, the European Space Agency leads some of the most exciting and impactful space missions, pushing into the furthest reaches of our Solar system – and beyond.

Running these intensive and often years-long space missions requires significant infrastructure and computing power. Historically, every ESA mission had its own infrastructure. However, this led to longer deployments and repetitive efforts in rolling out similar software and hardware. It quickly became clear that the organization needed a more effective way to manage operations, especially as ESA is moving to take on more space missions.

Michael Hawkshaw, ESA Mission Operations Infrastructure IT Service Manager at ESOC, explains: “Every mission had its own servers, software support team, and its own databases and infrastructure. Each mission would deploy the specific software on those machines that it needed to operate the mission. Continuing in this way was not sustainable. We realized this was an inefficient way of doing it, and had a goal that we’d support twice the number of missions by 2030”.

At the same time, ESA developed a platform from which to automatically deploy and run the software to operate their missions. However, managing and deploying updates to their software through this Kubernetes-based platform would have required far more resources and cost than they had available.

“Kubernetes is both fast-moving and hard, so supporting it ourselves was not a viable option. The thought of having to support it is daunting. To keep it supported and up to date, it’s better to have experts who do this every day involved and manage it for us,”explains Michael. “Originally before I joined ESA, other products were being explored, but these were not really suitable for our needs. It was also supporting an old version of Kubernetes, which was out of date by the time we wanted to use it. We needed something that was kept on a supported version, and always kept up to date without us having the overhead of having to manage this ourselves”.

The mission control software in use also made great use of Kafka and PostgreSQL for persistent storage, and it is critical that this is deployed in a highly available way, particularly during Launch and Early Orbit Phases (LEOPs), which often take place outside of normal business hours.

To meet its 2030 objective, ESA started work on a solution that would allow it to deploy infrastructure and software automatically for its missions, and simplify the workflow and processes for ESA engineering teams. The solution would need to be kept updated in a controlled and managed way, offer a secure and scalable way to take advantage of public cloud convenience, and support and secure on-premise mission-critical infrastructure. ESA chose to work with Canonical to provide the required private cloud infrastructure as a managed service.

Copyright : ESA–M.Cowan CC BY-SA 3.0 IGO

Solution

To meet these challenges, Canonical deployed its Kubernetes, Ceph, PostgreSQL, and Kafka distributions across data centres located at ESA’s European Space Operations Centre in Darmstadt, Germany. Canonical assisted ESA in the architecture, design and deployment of Kubernetes across hundreds of VMs and dozens of clusters, all connected to physical clusters of Ceph storage, as part of its Managed Services offering.

K8s (Kubernetes) is an open source system to deploy, scale, and manage containerised applications anywhere. Ceph is an open source storage solution that gives users scalable block, file, and object storage from a single distributed cluster. When ‘charmed’, using Canonical’s open source orchestration engine (Juju) , these systems become easy to deploy, integrate and operate.

Canonical Kubernetes was the ideal solution selected for ESA’s needs, as it provides an easy and effective way to automatically deploy software across systems of all sizes. It readily plugs into Ceph and PostgreSQL systems, which are part of ESA’s stack. The ESA chose Canonical’s Ceph distribution for its cloud infrastructure goals, also due to its ease of use. ESA wanted to take advantage of the power and scalability of public cloud, but had mission-critical on-premise cloud infrastructure that needed to be managed at the same time. Ceph provides storage that scales with changing mission requirements, while keeping costs low without the need for specialised storage hardware.

“We wanted one partner for the whole on-premise cloud because we’re not just supporting Kubernetes but also our Ceph clusters, managed Postgres, Kafka, and AI tools such as Kubeflow and Spark. These were all the services that were needed and with this we could have one nice, easy joined-up approach,” added Michael.

Ceph provides storage that scales with changing mission requirements, while keeping costs low without the need for specialised storage hardware.

It also opens up data storage to multiple types, all within a single cluster, cutting the need for building and managing multiple storage solutions. Aside from its high reliability, speed and scalability, Ceph offers a hardware agnostic design that allows it to excel without any reliance on other programs or proprietary systems.

The open source nature of Canonical’s offerings, and its managed services, were both driving factors in ESA’s adoption of these tools. As a managed service that uses open source tools, Canonical provided ESA with better cost effectiveness in the medium and long term, and allowed ESA to have the flexibility and freedom of its own infrastructure. Managed services also help ESA to focus on its core business of operating missions.

“We wanted one service provider for the whole on-premise cloud consisting not only of Kubernetes but also the Ceph storage solution, PostgreSQL and Kafka. These were all the services that were needed and with this we could have one joined-up approach”, added Michael.

The open source nature of Canonical’s solution allowed ESA to retain ownership of their systems.

“It was important, knowing that in the worst case scenario we could always ‘get the keys back and manage it ourselves’, as it were, even if it's very unlikely that we’d need to do this”, explained Michael. “We were stung in the past by proprietary licences and vendor lock-in."

Copyright: ESA/DLR/FU Berlin/G. Michael CC BY-SA 3.0 IGO

Results

This large deployment has given ESA cost savings, easier operations, and more confidence in providing infrastructure across multiple space missions. The open source nature of Ceph, Kubernetes, PostgreSQL, Kafka, Kubeflow, and Spark has reduced the number of software licences and costs associated with proprietary software deployments.

The system also enables 24/7 monitoring, with automatic ongoing updates reducing the risk of outdated services at mission launch. The risk of system failure has been similarly reduced: Kubernetes automatic failovers and backups give ESA missions more certainty of continued operations in cases of technical hiccups. If one node fails, workloads are automatically distributed onto other nodes.

The deployment also brings reduced complexity: all ESOC services and subservices can be managed in one location, with the ability to support Kubernetes, Ceph clusters, and managed data and AI solutions. Finally, this setup provides the ability to scale operations and take advantage of public cloud technologies, while keeping critical infrastructure on site.

"We can develop applications to be more resilient, knowing things won’t fail. It makes us more confident in supporting ESA missions,” said Michael. “We can sleep soundly knowing Canonical is supporting us and we won’t get a call at night telling us that something’s gone wrong."

The impact of Canonical’s open source infrastructure solutions is simple and significant: less time and effort needs to be spent on keeping systems up-to-date resulting in more cost-effective space missions and giving ESA more resources to focus on the space mission.

ESA has an ambitious roadmap for space exploration, continuing its strong presence in low Earth orbit and leading Europe’s human journey into the Solar system, including creating a sustainable presence of the Moon as a stepping stone for further deep-space exploration. Finding new and innovative ways to help support these missions on the ground is key to ESA’s future success.