The dark side of the universe from Oracle to PostgreSQL
Pilar de Teodoro and Sara Nieto are software engineers and database administrators as part of ESA (European Space Agency) working for the Euclid mission. Euclid is the ESA mission to map the geometry of the dark Universe and discover the origin of the accelerating expansion with unprecendented accuracy. We would like to describe the process that we design and implemented to migrate between two extremely different data model paradigms.
In 2020, the European Space Agency will launch a spacecraft to map the geometry of the dark Universe and discover the origin of the accelerating expansion, this mission is called Euclid. Euclid faces two main challenges from the point of view of the data management: 1. The unprecedent volume of data that the mission will produce, in the order of PB per year. 2. The transfer of these data from an Oracle based processing system, to a science archive based on PostgreSQL database. In this talk, we will review the migration system implemented from Oracle to PostgreSQL addressing the challenge of how to map two extremely different data models paradigms, from an Object Oriented data model in Oracle to a pure relational data model in PostgreSQL. The implementation of this transfer layer will rely on an ad-hoc architecture based on Foreign Data Wrappers and the triggering mechanism to synchronize both systems. We will show the tests performed so far to get the best architecture and performance for this large scale migration. Presented by Pilar de Teodoro and Sara Nieto.
- 50 min
- PGConf US 2017 [PgConf.US]
- Use Cases