Joshua D. Drake Blog Posts


How do you use Postgres?

I work Fred Hutchinson Cancer Research Center and within Fred Hutch, I work for the largest group called SCHARP, Statistical Center for HIV/AIDS Research and Prevention. We use Postgres to monitor the AIDS drug trials real time to see if the trials are working or not. This means we collect the data from doctors and labs around the world, and analyzing the data. We also have servers where we receive data from other research institutes and share our randomized data, de-personalized, with other research institutes.


What community contributions have you provided?

In 2010 I started SeaPUG, Seattle Postgres Users Group, at the request of Josh Drake. I have at least have the presentations there every year. I have also discovered several PostgreSQL: bugs which have been fixed. Some of them affected every version of PostgreSQL. Bug numbers: 7553, 8173, 8257, 8291. I also found 8545, which has not been fixed but Core has acknowledge needs to be fixed but they are not sure where it should be fixed, pg_dump or pg_dumpall. I started the PostgreSQL track at LinuxFest Northwest in 2014 after my GIS presentation in 2013 was standing room only. This year I got a booth at the SeaGL, Seattle GNU Linux, conference with the idea of having a booth there next year along with also doing a PostgreSQL presentation next year at the conference.


You recently took a renewed interest in speaking at Postgres Conferences, why?

I have been giving presentations locally now for the last 7 years and so I am now ready to move on to the next step, doing presentations at the Local and National conferences around the United States.


What is the #1 barrier you see to Postgres adoption?

People not knowing about PostgreSQL, most people know about MYSQL, MSSQL and Oracle, but do not know about PostgreSQL. This is changing, some, with the Cloud providers now offering PostgreSQL, but I go to these conferences, LinuxFest Northwest and SeaGL, and people all the time are asking me, "What is Postgres and why should I use it over MYSQL, MSSQL or Oracle", because they have never heard of PostgreSQL.


What is something you feel the wider Postgres community could be better at?

We need to promote PostgreSQL so that new people starting personal projects and starting at companies, will think about PostgreSQL before other databases. This starts with getting the younger generation interested in PostgreSQL and that also means that we need to get the college professors willing to talk about PostgreSQL in their curriculums instead of ignoring PostgreSQL for all the other competing databases. Some of this means that at all the other conferences, we need to have a PostgreSQL presence, aka booth and presentations. We should also come up with a certification method for PostgreSQL DBA's, User's, Engineer's, etc so that prospective employers will have an idea of the prospective employees skill set.

Joshua D. Drake     December 12, 2017

When you are considering a conference about Postgres, one should pick the one that is focused on building the community. PostgresConf is all about building the community and we even captured it on video!
 
 
PostgresConf embraces a holistic view of what community is. We want everyone to feel welcome and encouraged to give back to PostgreSQL.org. However, that is not the only opportunity for you to give back to the Postgres community. We all have different talents and some of those don't extend to writing patches or Docbook XML. 

Giving back

When considering who is part of the community and who is contributing to the community, we want to introduce you to a couple of fantastic organizers of our conference: Debra Cerda and Viral Shah. Some in the community will know Debra. She has been in the community for years and is one of the primary organizers of Austin Postgres.
 
Debra Cerda

Debra is our Attendee and Speaker Liaison as well as our Volunteer Coordinator. She is also a key asset in the development and performance of our Career Fair.

 
Viral Shah

Viral is our on-site logistics lead and is part of the volunteer acquisition team. It is Viral that works with the hotel using a fine tooth comb to make sure everything is on target, on budget, and executed with extreme efficiency.

 
Without her amazing attention to detail and dedication to service we wouldn't be able to deliver the level of conference our community has come to expect from PostgresConf.
 

Building relationships

There a lot of reasons to go to a conference. You may be looking for education on a topic, a sales lead, or possibly just to experience a central location of top talent, products, and services. All of these reasons are awesome but we find that the most important reason is to build relationships. The following are two exceptional examples of community projects.
 
Our first example is ZomboDB. No, they are not a sponsor (yet!) but they have a fantastic Open Source extension to Postgres that integrates Elasticsearch into Postgres. 
 
Our second ecosystem community member is an entity that most have heard of at this point; TimescaleDB. It too is a fantastic showing of what is possible when you combine brilliance with the extensibility of Postgres.
 
What is notable about these two mentions is that they represent what we would call, "Professional Community." Recently ZomboDB wanted to bounce some ideas off of a Postgresql hacker regarding the Index Access Method API. We at PostgresConf were able to facilitate an introduction to Timescale and a couple of amazing minds ended up chewing the fat on their respective projects. It's relationships such as these that enable the community to grow and offer the best opportunities possible.
 
 

Part of the community

Join the Professional user and ecosystem community for Postgres today! You can start by submitting a presentation to the upcoming PostgresConf 2019 being held March 18th - 22nd, 2019 at the Sheraton Times Square.
 
 
 

 
Joshua D. Drake     November 26, 2018

Where is your path leading you?

 

At Postgres Conference Silicon Valley I promised during the launch that after the conference was completed PostgresWarrior and I would be taking a freedom tour to various National Parks. 

 

For us, our path is serving the community through education and professional and personal development. This happens in many forms including these newsletters.

 

Recent projects have been coordinating a successful webinar series from Yugabyte, creating  online live Postgres instructor lead training, and launching a new educational series on PostGIS. This is all happening while the Call for Papers for Postgres Conference 2020 now open! The ongoing goal is to allow any person to receive the education they need to be successful with People, Postgres, Data year around.

Training

The current training options from Postgres Conference can be found here:

https://postgresconf.org/conferences/Postgres-Digital-Training-Series/schedule/events

We have two performance trainings coming up in October:

  • PostgreSQL Performance & Maintenance on October 29th
  • Finding and Fixing Slow Queries on October 30th

 

Both of these training opportunities sell out at the physical conferences. They are solid content and at a reasonable price (149.00 USD ) it is hard to say no to a few hours for education in your day!

Software

Are you frustrated with the limitations and fragility of Logical Replication in PostgreSQL Core? There is a new software on the block called pgcat and it has an impressive list of features to allow your Logical Replication experience to be exceptional.

 

Looking for a simple script to help find tuning opportunities for PostgreSQL? The perl script postgresqltuner may just be what you are looking for. Yes, there really is an active developer community for the Perl language still.

 

A HyperLogLog data type for PostgreSQL from our friends at Citus. This Postgres module introduces a new data type hll which is a HyperLogLog data structure. HyperLogLog is a fixed-size, set-like structure used for distinct value counting with tunable precision. For example, in 1280 bytes hll can estimate the count of tens of billions of distinct values with only a few percent error.

 

Our partner Heimdall Data has been creating a new type of connection pool that removes a significant limitation within other software such as PgBouncer and PgPool. If you are looking for Enterprise Authentication (Active Directory/LDAP) as well as intelligent pooling for many users (and connections), it may be worth a look. There is a webinar next week on how it all works!

Does your path allow people to “Come as you are?”

In consideration of all of the great news from our community we can’t help but reflect on the blessings we have in the world of Open Source. Remember that Open Source is about exceptionalism, creativity, and most importantly freedom. When communities start restricting these three tenets of Open Source, they are no longer Open Source communities, even if their software is.

 

The theme for Postgres Conference 2020 in NYC is “come as you are” and we are asserting this mantra throughout our entire community. Over the past few years there has been an influx of toxicity throughout all circles and it is time for civility and grace to return. It is time to remember that we are all human. We all have angels and demons to our personalities. We are all flawed and we are all exceptional in our own way.

 

"But just because I don't agree with someone on everything doesn't mean that I'm not going to be friends with them. When I say, 'be kind to one another,' I don't only mean the people that think the same way that you do. I mean be kind to everyone."

 

-- Ellen DeGeneres

 

(Yes, this happened. No, it wasn’t planned.) 

Just outside of Moab Utah.

 

Find YOUR path.



Joshua D. Drake     October 17, 2019

PostgresConf US 2018 is in 9 days. Here is the obligatory "Buy your tickets" reminder! If you look around (a Google search of Gold sponsor Google Cloud is a good place to start) you will find a lot of discount codes.

In 2017 we launched a community wide effort to better recognize contributors for not only the conference but the wider Postgres Community. We continued this effort in 2018 and are pleased to have many speaker profiles available, with more being published every day:

As one of the Chairs of PostgresConf, I am honored by the resounding support from sponsors, speakers, and volunteers to help create a fantastic event for all attendees. It has been a pleasure working toward the common goal of creating a global, non-profit, Postgres Conference series.
 
 
 
 

The first ever PGConf US Local: Seattle event is happening in partnership with SEAPUG on August 11th and 12th at the Sheraton Downtown Seattle! On August 11th we have four training options available:
  1. Mastering PostgreSQL Administration by Bruce Momjian
  2. Migrating from Oracle to PostgreSQL by Jim Mlodgenski
  3. Postgres Performance and Maintenance by Joshua (JD) Drake
  4. Database Automation by Robert Bernier 

Tickets are now available!


On August 12th we have 14 break-out sessions between the Development and Operations tracks. A sample of our break-out sessions is available below but please check the full schedule for all of the fantastic content!
We would not be able to produce PGConf Local: Seattle 2017 without the generous support from our sponsors:
    Sponsorship opportunities for the conference are still available. If you use Postgres, support Postgres, or if you are looking to recruit a local Postgres expert, this is a great opportunity. Become a part of our dynamic and growing ecosystem! The prospectus is available at:
    Joshua D. Drake     July 11, 2017




    The presentation includes an introduction and setup for consul as the means of providing highly available PostgreSQL in local and geographically disparate data centers or cloud providers. The presentations includes:

    *) Introduction to consul and its architecture
    *) Setup of a single consul cluster
    *) Setup for a few sample database instances (OLAP and OLTP)
    *) Firewall requirements
    *) Integration with bind, djbdns, and dnsmasq
    *) Setup geographic failover to two different data centers and cloud providers
    *) Various Best Practices tips and suggestions
    *) Q&A

    Joshua D. Drake     April 25, 2017

    Adaptation Lizard

    As Postgres Conference pushes forward in the brave new world, we evolve and increase the ability for the People, Postgres, Data community to succeed. As a part of our positive adaptation we have a new website that features upcoming events, professional content (written and video), and the best in our written community via “Community Content”.

    Discord

    In an effort to provide a modern, friendly, and inclusive community platform, we have launched a Discord server for all things Postgres. We are providing a helpful experience with rule #1 being: Be Nice. Our discussions will branch out beyond the core of PostgreSQL and provide a forum for success with Postgres and related technologies. Join us for what is guaranteed to be a refreshing experience for the community: https://discord.gg/tjxNBCz

    2021

    We are seeking feedback from our community on 2021 in-person events. We are currently considering the East Coast event for October 2021 and the West Coast event for December 2021. Please help us in determining the type of event you would like to participate in!

    Upcoming live events

    We currently have the following webinars scheduled through October:

    Joshua D. Drake     August 25, 2020

    You will want to mark your calendars folks on August 15th the Call for papers for PostgresConf Silicon Valley will close. That is just two weeks away!

    So let's point your Firefox, Google Chrome, or Safari app to this link right here and get your presentation submitted.

    This is the inaugural Silicon Valley conference and from the current submitted papers it is shaping up to be a fantastic event. We can't wait to see everyone and continue our mission of:




    Joshua D. Drake     July 31, 2018

    The Chairs (myself, Jim Mlodgenski, and Amanda Nystrom) have recently decided to bring some visibility to charities that are close to our hearts. They are listed below:

    • Joshua Drake: Navajo Water Project. The Navajo nation is approximately the size of West Virginia and has a population of over 150,000 people (300k in the tribe). Anywhere from 15% - 40% of the residents do not have access to running water. The Navajo Water Project aims to bring clean water to each person and family through support from those that donate. 
    • Jim Mlodgenski: St. Jude Children’s Research Hospital. The hospital is one of the premier research hospitals for cancer and other life threatening illnesses for some of our most vulnerable people. Approximately one in 285 children in the U.S. will be diagnosed with cancer before their 20th birthday. Through donations, St. Jude’s provides treatment to those with cancer, and is actively dedicating resources to the research and cure for cancer. 
    • Amanda Nystrom: ASPCA. The American Society for the Prevention of Cruelty to Animals (ASPCS) was the first humane society to be established in North America, with the goal of providing kind and respectful treatment to animals under the law. Unlike humans, cases of animal abuse aren’t compiled but studies have shown a correlation between domestic violence and animal abuse. The ASPCA prevents animal homelessness and actively rescues animals from dangerous and/or cruel situations.

    Upcoming Webinars

    With the Coronavirus causing the conference market to dry up for 2020, we at Postgres Conference have pivoted to ensure that we continue to provide quality Postgres content to the world of People, Postgres, Data. We have been performing multiple webinars per month. Here is the current schedule and you can register (free) here:

     

    • May 21, 11am PT: A Deep Dive into PostgreSQL Indexing
    • June 2, 10AM PT: How to Move Data from Oracle to Postgres in Near-Real Time
    • June 9, 11am PT: Community vs. Enterprise Open Source – Which is Right for Your Business?
    • June 10, 11am PT: Bring Compression to Postgres at Zero Cost of Performance
    • June 16, 11AM PT: Mostly mistaken and ignored PostgreSQL parameters while optimizing a PostgreSQL database
    • June 30, 11am PT: Deeper Understanding of PostgreSQL Execution Plan: At plan time and run time
    • July 15, 10AM PT: Working with JSON Data in PostgreSQL vs. MongoDB
    • June 17, 11am PT: Postgres vs. MongoDB for real-time machine learning on wind turbine data

    Articles from the community

    Coronavirus Resources:

    Joshua D. Drake     May 19, 2020

     
     

    As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our Diamond and Platinum sponsors for this year in our Sponsor Spotlight Series.

    Jacque Istok, is the Head of Data for Pivotal, one of our Diamond Sponsors for PostgresConf US 2018. Pivotal is hosting the first annual Greenplum Summit at PostgresConf US 2018, with lots of great Greenplum and Postgres-related content. Read what Jacque has to say bout Greenplum and Postgres, as well as why to attend the Greenplum Summit: 

    Greenplum is an Open Source variant of Postgres; what benefits do you bring to the table over vanilla Postgres?

    Postgres is a powerful ORDBMS, but as your data scales, the only way to keep up is to buy bigger and bigger machines to run on. It suffers from the same problems that all SMP databases do: you can only get as big as the machine you’re running on.

    With Greenplum you can put a subset of your data on a Postgres database on one reasonably-sized machine, and another subset on a second machine, and so on. All of your users and applications can then query one of these Postgres databases as if all the data was in a single location - making your data scale limitless. Greenplum manages the distribution, data shuffling, and querying of all of your data across a magically sharded implementation of Postgres databases.

    Greenplum has its own community; what do you hope to achieve by joining the Postgres community and PostgresConf?

    The Postgres community represents some of the most passionate and knowledgeable creators, developers, and users of database technology of our time. We believe that the combination of Postgres and Greenplum becomes the software equivalent of what Oracle Exadata purported to be: an all-purpose database that can do both transactional and analytical workloads across multi-structured data. Simply put, the Greenplum community is looking to join with the Postgres community to further the understanding and adoption of these technologies.

    Do you have plans for cross pollination of technologies with the two open source projects?

    Greenplum forked from Postgres over 10 years, circa Postgres 8.2. Greenplum 5.0 is based off of Postgres 8.3, with our next major release slated for Postgres 9.4 (current open source Greenplum is compatible with 9.0 as of this writing).

    Likewise, we have Postgres committers working at Pivotal looking for opportunities to improve the Postgres code specifically for analytics. We are also ensuring that other projects related to Greenplum, like Apache MADLib, continue to be compatible with Postgres.

    What challenges do you see working with the Postgres community as an open source fork?

    The Postgres community is a long-running and very passionate group, and we want to be both collaborative and respectful in how we continue to grow our participation. We see the products as having synergies which complement each other very well, with some use cases that best fit Postgres, and others that best fit Greenplum. The use of either benefits the other as they both further adoption.

    What would you tell a user who has a choice between Postgres and Greenplum about when they should use which system?

    Postgres is a great ORDBMS that will scale to the performance of a single server. For analytical needs, being restricted to a small number of terabytes does not allow for the type of exploration that most organizations need. Because Greenplum is a Postgres compatible database, you can start out using Postgres and either convert to Greenplum underneath or leverage Greenplum alongside your Postgres systems (making data ETL a ton easier). This then makes the choice of which product to use for your particular use case clearer and clearer.

    What is the number one barrier you see to contributing to the Postgres community?

    The number one barrier we will have to contributing is not seeing the corresponding adoption of our technologies. We feel very strongly that both the transparency and removal of vendor lock-in make our open source commitment the only choice for users. I’m here to implore the community to embrace our technology with zeal and help us continue to drive more and more Postgres adoption in the world.

    What is the best thing about working with the Postgres community?

    Because Greenplum is based on Postgres, we get to interact with this vast community of talent. We are also able to more seamlessly interact with ecosystem products that already work with Postgres, making the adoption of Greenplum that much easier.

    Tell us why you believe people should attend PostgresConf 2018 in April.

    PostgresConf is going to be awesome - with both Pivotal and Amazon headlining as Diamond sponsors - as well as the quality of speakers and their content. I wouldn’t miss it for anything.

    We’re thrilled to organize the first annual Greenplum Summit at PostgresConf. Greenplum co-founder, Scott Yara, will give a keynote on April 18th relating to how data tells the story at the organizations that we help enable (#DataTellsTheStory), and his journey from SMP to MPP. Greenplum Summit on April 19th will be a full day packed with with great use case sessions and tech talks for novices and experts alike.

    Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!



    Joshua D. Drake     March 26, 2018     pivotal Greenplum postgres postgresql