Advanced compression in TimescaleDB with hybrid row/columnar storage
Michael J. Freedman is a Professor in the Computer Science Department at Princeton University, as well as the co-founder and CTO of Timescale, which provides an open-souce database that scales out SQL for time-series data. His research broadly focuses on distributed systems, networking, and security, and has led to commercial products and deployed systems reaching millions of users daily. Honors include a Presidential Early Career Award (PECASE), Sloan Fellowship, NSF CAREER Award, ONR Young Investigator Award, DARPA CSSG membership, and multiple award publications.
No video of the event yet, sorry!
Storage systems like databases and file systems have long used compression to reduce their storage footprint. Yet the most effective compression were traditionally limited to column stores, where increased data-type locality provides greater options for advanced techniques. It has often been assumed that fundamental differences between column-store and row-store architectures lead to these opportunities.
In an upcoming release of TimescaleDB, we introduce a compression scheme which challenges this assumption. Our compression technique uses regular Postgres values to store data from many rows in columnar form. This allows us to use state-of-the-art compression techniques to achieve storage usage on par with dedicated column stores. Further, this strategy allows us to compete in performance with dedicated column stores, particularly as we can still store non-columnar metadata to benefit from the full range of index types.
- 2019 September 19 16:20
- 50 min
- Winchester (2)
- Silicon Valley 2019
- Ops and Administration