Real-Time Data Streams from PostgreSQL: A Game Changer in Application Architecture
Jonathan S. Katz is the Director of Customer Success & Communications at Crunchy Data, a leading provider of trusted open source PostgreSQL technology, support, and training. Jonathan is also responsible for the advocacy efforts of the PostgreSQL Global Development Group, is a board member of the nonprofit United States PostgreSQL Association, and is a co-organizer of the NYC PostgreSQL User Group.
Jonathan enjoys building applications with PostgreSQL and revels in showing off all of the wonderful features of PostgreSQL. While he was CTO at VenueBook, Jonathan developed a robust platform using PostgreSQL, taking advantage of its many features, from complex data types to its ability to stream logical changes. Jonathan graduated from Tufts University with a B.S. in Computer Science and a B.A. in Mathematics.
Often in applications, it is necessary to be aware of all changes to your database in order to take additional actions. Prior to PostgreSQL 9.4, this could be accomplished with database-level triggers or hooks in your application code, but these methods have both performance and data consistency tradeoffs.
In version 9.4, PostgreSQL introduced logical decoding, which provides an interface to stream all changes within a database to a listener. With the release of drivers such as psycopg2 and JDBC, logical decoding has become easily accessible to the application developer and provides new architectural opportunities to efficiently and securely manipulate your data.
In this talk, we will cover the following:
- An overview of logical decoding in PostgreSQL
- How to setup and use logical decoding using psycopg2
- An example application requiring changes based on data modifications using:
- In-line with Python code
- Asynchronous messaging framework
- Logical decoding with a serial processor
- Logical decoding + asynchronous messaging framework
- Performance considerations and trade-offs from each of these methods
- 2018 April 20 10:50
- 50 min
- PostgresConf US 2018