Advanced compression in TimescaleDB with hybrid row/columnar storage
Presented by:
David Kohn
Storage systems like databases and file systems have long used compression to reduce their storage footprint. Yet the most effective compression were traditionally limited to column stores, where increased data-type locality provides greater options for advanced techniques. It has often been assumed that fundamental differences between column-store and row-store architectures lead to these opportunities.
In an upcoming release of TimescaleDB, we introduce a compression scheme which challenges this assumption. Our compression technique uses regular Postgres values to store data from many rows in columnar form. This allows us to use state-of-the-art compression techniques to achieve storage usage on par with dedicated column stores. Further, this strategy allows us to compete in performance with dedicated column stores, particularly as we can still store non-columnar metadata to benefit from the full range of index types.
- Date:
- 2019 September 19 16:10 PDT
- Duration:
- 50 min
- Room:
- Winchester (2)
- Conference:
- Silicon Valley 2019
- Language:
- Track:
- Ops and Administration
- Difficulty:
- Medium