A technical overview to unlock Seamless Logical Replication from Heterogeneous Databases (MySQL, Oracle, SQL Server, and More) to PostgreSQL
Presented by:
Cary Huang
Cary is a Senior Software Developer in HighGo Software Canada with 8 years of industrial experience developing innovative software solutions in C/C++ in the field of smart grid & metering prior to joining HighGo. He holds a bachelor degree in Electrical Engineering from University of British Columnbia (UBC) in Vancouver in 2012 and has extensive hands-on experience in technologies such as: Advanced Networking, Network & Data security, Smart Metering Innovations, deployment management with Docker, Software Engineering Lifecycle, scalability, authentication, cryptography, PostgreSQL & non-relational database, web services, firewalls, embedded systems, RTOS, ARM, PKI, Cisco equipment, functional and Architecture Design.
PostgreSQL has long excelled at native logical replication and is rapidly becoming a top choice for data analytics, thanks to its powerful features and flexibility. As organizations increasingly use diverse databases like MySQL, Oracle, and SQL Server, the need for seamless, high-performance replication into PostgreSQL (aka Change Data Capture (CDC)) has become significant.
In this talk, I would like to discuss the technical aspects of a PostgreSQL C extension project that I am working on that aims to make Logical Replication from Heterogeneous Databases (MySQL, Oracle, SQL Server, and More) to PostgreSQL a seamless process with high performance. Built on the open-source Java-based Debezium Embedded Engine, this extension acts as a unified command center that controls and monitors data replication from various sources with different database types, eliminating the need for complex replication setups.
I will share the technical challenges and design strategies involved in making heterogeneous logical replication in PostgreSQL possible. This includes:
- Architecture decisions
- Bridging the gap between Java and C scopes
- Data type and DDL translations
- Error handling
- Current status and the next steps
- Date:
- 2024 November 6 10:40 PST
- Duration:
- 50 min
- Room:
- Dev: 422
- Conference:
- Seattle 2024
- Language:
- Track:
- Dev
- Difficulty:
- Medium