Building Reliable & Scalable Data Replication Pipelines from Postgres on Kubernetes
Presented by:

Sanketh Balakrishna
Sanketh Balakrishna is an Engineering Manager at Datadog for the Data Replication & Search teams at Datadog. He manages teams that provide a platform for multiple Product teams at Datadog to easily and reliably replicate data to secondary systems and quickly search their Data. Prior to working at Datadog, he was an Engineer at AWS where he focussed on building a Database Migration/Replication platform for customers and had also received his Masters degree in Computer Science from Umass Amherst. When he's not working, he loves playing racket sports and traveling.
No video of the event yet, sorry!
At Datadog, we built a platform for Data Replication from Postgres to various different targets over the last two years. In doing so, we enabled a lot of product teams to quickly, safely and reliably replicate data to secondary systems to solve business problems. This talk will focus on
- Why we needed a platform solution for Data Replication from Postgres
- How we started, and technologies we explored
- Why not just 'native logical replication', and why we chose to use Debezium and ultimately our own fork of it
- Building a perfect solution vs a quick solution
- A few business use cases at a high level
- Some challenges & complexities we faced along the way including PG WAL related/Schema challenges
- Impact and learnings
- Date:
- 2025 March 20 09:30 EDT
- Duration:
- 50 min
- Room:
- Gold Coast 3&4
- Conference:
- Postgres Conference 2025
- Language:
- Track:
- Essentials
- Difficulty:
- Medium