Profiling PostgreSQL itself to find the slowdown within PostgreSQL
I have been a PostgreSQL administrator since 2002 and have worked for Fred Hutchinson Cancer Research Center since 2008. Within Fred Hutch, I work for the largest group called SCHARP, Statistical Center for HIV/AIDS Research and Prevention.
In 2010 I started SeaPUG, Seattle Postgres Users Group, at the request of Josh Drake. I present at least half the presentations there every year. I started the PostgreSQL track at LinuxFest Northwest in 2014 after my GIS presentation in 2013 was standing room only. In 2017 I got a booth at the SeaGL, Seattle GNU Linux, conference with the idea of having a booth there in 2018 along with also doing a PostgreSQL presentation at the conference. In 2017 I started presenting at various PostgresConf Conferences.
I have also discovered several PostgreSQL bugs which have been fixed, some of them affected every version of PostgreSQL.
- 7553 - Variant of the what alias to use after a rename bug in views - Fixed in 9.3
- 8173 - Inserting heap tuples in bulk in COPY patch return wrong line on failure 999 out of 1000 times. - Fixed in 9.2.5 & 9.3
- 8257 - Multi-Core Restore fails when containing index comments - Fixed in 8.4, 9.0, 9.1, 9.2, 9.3
- 8291 - postgres_fdw does not re-read USER MAPING after change - Updated Docs in 9.3
- 8545 - pg_dump does not backup database level globals - Fixed in 11
- 14147 - Restoration of Materialized View of Foreign Data Tables fails - Fixed in 10
- 15182 - Major Bug - Affecting all versions of PostgreSQL. Partially fixed in 11 and fully fixed in 12
- 15316 - Creation of check constraint functions that rely on data loaded alphabetically after your primary data will fail to restore properly.
I host the Seattle Postgres Users Group where I give many of the presentations. I have many of the presentations posted at Lloyd's Presentations
No video of the event yet, sorry!
At work we bought a brand new server. When testing our in house app against the new server, we found it took 7.5 hours. This same app running against our 7 year old hardware using the same version of PostgreSQL takes only 45 minutes. I was able to track this down to the COPY command the new server being 100 times slower than our 7 year old server. At first I blamed Ubuntu since we had been running openSUSE on the old server. After running a profiler on the PostgreSQL connection, I tracked down the problem to a PostgreSQL "optimization'. Once I change the PostgreSQL config to bypass this "optimization", the COPY command became 300 times faster.
In this presentation I will go over how we figured out what was wrong. This includes how we figured out that PostgreSQL was not being slowed down by the OS and then how we used the profiler to find the problem including finding and reading the source code, etc.
- 50 min
- Postgres Conference 2020
- Case Studies