Joshua D. Drake Blog Posts

• Mastering PostgreSQL Administration

• Securing Data in a Multi-Platform Data Lake

• Personal Time Management

• Contributing to PostgreSQL

Why did you attend PgConf US?

Our company is exploring the use of PostgreSQL as an additional DBMS choice to support Vertex Enterprise, our tax technology platform. Attending the conference offered me an opportunity to immerse myself into a variety of topics around PostgreSQL, as well as the chance to interact with other users to tap into their experiences.


Tell us a little bit about your project or how you use PostgreSQL:

Vertex Enterprise is a global tax management end-to-end solution that integrates all tax processes with Enterprise Resource Planning (ERP) and business intelligence (BI) systems on a single platform using a tax performance engine.   The solution is a blend of technologies that surround a Java based processing and calculation engine. This includes rich UI's, import, export, business intelligence / reporting, and big data integration. The foundation of our database leverages a multi-dimensional database structure, as well as other structures whose design varies based upon the needs of a particular function so that we can optimize performance. (ie. import/export/reporting).

The official brochure provides a high level overview of the capabilities and interfaces that our DBMS must support.  

Why did you chose PostgreSQL for your project?

PostgreSQL checks many of the boxes that we are looking for in our consideration of an additional DBMS option.


Some of these include:

* Performance
* Lower cost
* Multi-tenancy support 
* Suitable for deployment on premise, in a hosted environment, and in the cloud
* Industry acceptance
* Ease of administration on locally administered instances
* Features characteristic of an enterprise strength database (auditing, partitioning, replication, procedural language support)


This was your first PGConf US, was it a last minute decision? Do you think it was worth it? If so, why?

The conference was on our radar since the beginning of the year, and it exceeded expectations. Listening to and interacting with actual contributors to the product is not something you have the opportunity to do at many conferences. The sessions covered a variety of topics around PostgreSQL that were of interest (performance, tuning, feature discussion, lessons learned, cloud deployment options, optimization, etc).


Would you attend PgConf US again?

Yes, definitely. 


Are you interested in contributing to the community and if so, in what fashion?

My "day job" keeps me pretty busy, but as far as sharing our experience with PostgreSQL, I am willing to contribute in that way :) .

On occasion, professional developers will drop into the Postgresql.org mailing lists, meetups, and conferences to ask the question, “Why isn’t PostgreSQL development on Github?” In an effort to see if the demand was really there and not just anecdotal we ran a poll/survey over several social media platforms that asked a simple question:

 

Should PostgreSQL development move to Github?

• Yes
• No
• No, but to something like Gitlab would be good

 

We received well over 300 responses and the majority (75%+) chose a move to Github or to something like Github. This was an unscientific poll but it does point out a few interesting topics for consideration:

 

1. We need to recognize that the current contribution model does work for existing contributors. We need to have an honest discussion about what that means for the project as contributors age, change employment, and mature in their skill set, etc..
2. Of the people that argued in comments against the move to a service, only one is a current contributor to PostgreSQL.org core code. The rest were former code contributors or those who contribute in other ways (Advocacy, System administration, etc.).
3. Would a move to Github or similar option produce a higher rate of contribution?

 

This poll does not answer point #3; it only provides a data point that people may desire a modern collaboration platform. The key takeaway from the conversation about migrating to Github or similar service is the future generation of developers use technology such as Slack and Microsoft Teams. They expect a bug/issue tracker. They demand simplicity in collaboration and most importantly they will run a cost->benefit analysis to determine if the effort to contribute is a net positive.

 

It should also be considered that this is not just individual potential contributors. There are many corporations big and small that rely on the success of PostgreSQL. Those corporations will not contribute as much directly to PostgreSQL if the cost to benefit analysis is a net negative. They will instead contribute through other more productive means that produce a net positive when the cost->benefit analysis is run. A good example of this analysis is the proliferation of external projects such as pg_auto_failover, patroni and lack of direct contribution from innovative extension based companies.

Do we need a culture shift within PostgreSQL?

There are those within the Postgresql.org community that would suggest that we do not need a culture shift within PostgreSQL but that does not take into account the very clear market dynamics that are driving the growth of PostgreSQL, Postgres, and the global ecosystem. It is true that 20 years of hard work by Postgresql.org started the growth and it is also true that the majority of growth in the ecosystem and community is from products such as Greenplum, Aurora, Azure, and Timescale. The growth in the ecosystem is from the professional community and that ecosystem will always perform a cost to benefit analysis before contributing.

 

It is not that we should create radical rifts or disrupt our culture. It is to say that we must evolve and shift our community thinking. We need to be able to consider the big picture. A discussion should never start as an opposition to change. The idea of change should be an open discussion about possibility and vision. It should always include whether the change is a good idea and it should always avoid visceral reactions of, "works for me,” “no,” or “we tried that 15 years ago." Those reactions are immature and lacking in the very thing the community needs to continue to grow: positivity, inclusion, vision, and inspiration.

Why did you attend PgConf US?

We have been looking for high quality, experienced, professional support for our application’s database for some time. We have found it difficult to find help online… seemingly every phone number we called was just an answering service. When we heard that the companies offering the level of service we were looking for were all available in the same place, we couldn’t resist.

Tell us a little bit (one or two paragraphs) about your project:

Our project (VX and VO collectively named Victor) is a SaaS solution for 911 emergency response systems. Victor provides analytics and quality assurance tools and services, enabling agencies to assess performance, measure resource & deployment activity, model demand, optimize workload, and even generate financial estimates. For more information see our website.

Why did you chose PostgreSQL for your project?

Victor requires both time and spacial awareness, along with all the other things that are expected from an RDBMS. PostgreSQL is stable, secure, mature, well documented, open source, actively developed, community supported, and generally bad ass. The choice was easy.

As I understand it, attending PgConf US was a last minute decision. Do you think it was worth it? If so, why?

Absolutely! Our objective was to meet with vendors who were able to provide high quality support and we were successful. Additionally, we met a ton of super geniuses (Paragon, TimescaleDB, I’m talking about you) and learned more then expected. For example, we had no idea we could put raster images in PostgreSQL and process them with PostGIS… amazing!

Would you attend PgConf US again?

Yes… see above. Unrelated, but unbelievably cool, we met a guy named Solar… a PhD (from MIT!) passionate about carbon nanotubes.

Are you interested in contributing to the community and if so, in what fashion?

Yes! …and here is our biggest piece of feedback.

It would be great if there was a clear point of entry for people like us… we have been using PostgreSQL for several years, have been through several upgrade cycles (we started on 9.0) and have no idea how our skills could be of use. We make SaaS applications, but make nothing in C. What else does the community need? How can we help?

Attendees of PgConf.US , if you haven't gotten your hotel room booked, now is the time! Our block ends on March 6th. Point your browser over here and book that room. While you wait for your email confirmation from the Westin, you can take some time to review the following talk from PgConf US 2015, ToroDB: a new, open-source, document-oriented, JSON database, built on Postgres.

Pivotal Sponsor Highlight Blog for PostgresConf 2019

 

Written by:

Jacque Istok, Head of Data, Pivotal

 

1. Greenplum has its own community; what do you hope to achieve by joining the Postgres community and PostgresConf?

Both interest and adoption of Postgres have skyrocketed over the last two years, and we feel fortunate to be a part of the extended community. We have worked very hard to uplevel the base version of Postgres within Greenplum to more current levels and to be active in the Postgres community. We see Greenplum as a parallel (and analytic focused) implementation of Postgres, and we encourage the community to continue to embrace both the technology and the goal of the Greenplum project, which is Postgres at scale.

 

2. Are you planning to provide any new tech (PG features, etc.)?

This year we plan to announce several new things for both Greenplum and Postgres. We’re introducing new innovations in our cloud offerings in the marketplaces of AWS, Azure, and GCP. We also have major news about both our natural language at-scale analytics solution based on Apache Solr, and our multi-purpose machine learning and graph analytics library Apache MADlib. The next major release of Greenplum is a major focus as well, differentiating Greenplum from each of its competitors and bringing us ever closer to the latest versions of Postgres.

 

3. Are there any rising stars in the community you’d like to give props to?

While it seems a little self-serving, I would like to take the opportunity to give props to the Pivotal Data Team. This team is a 300+ worldwide organization that helps our customers, our prospects, and the community to solve real world and really hard data problems—solved in part through Postgres technology. They all attack these use cases with passion and truly make a difference in the lives of the people that their solutions touch. I couldn’t wish to work with a finer group.

 

4. What is the number one benefit you see within Postgres that everyone should be aware of?

The number one benefit of Postgres is really its flexibility. This database chameleon can be used for SQL, NoSQL, Big Data, Microservices, time series data, and much more. In fact, our latest analytic solution, MADLib Flow, leverages Postgres as an operational engine. For example, Machine Learning models created in Greenplum can be pushed into a restful API as part of an agile continuous integration/continuous delivery pipeline easily and efficiently—making Postgres the power behind what I still like to think of as #DataOps.

 

5. What is the best thing about working with the Postgres community?

 

I deeply admire the passion and consistency of the community behind Postgres, constantly and incrementally improving this product over decades. And because Greenplum is based on Postgres, we get to interact with this vast community of talent. We are also able to more seamlessly interact with ecosystem products that already work with Postgres, making the adoption of Greenplum that much easier.

 

6. Tell us why you believe people should attend PostgresConf 2019 in March.

 

PostgresConf is going to be awesome, and I can’t wait for it to start! With Pivotal, Amazon, and EnterpriseDB headlining as Diamond sponsors, Greenplum Summit (along with multiple other summits), and high-quality speakers and content across the board, this year’s PostgresConf promises to be bigger and better than ever and surely won’t disappoint.

 

We’re thrilled to be back to present the second annual Greenplum Summit on March 19th at PostgresConf. Our theme this year is “Scale Matters”, and what we’ve seen with our customers is that every year it matters more and more. Our users are part of organizations that are generating tons of data and their need to easily and quickly ingest and interrogate all of it is paramount. This is true even more now than ever before as the insights that can be found not only help differentiate them from their competitors, but are also used to build better products and increase customer loyalty.

 

The day will be filled with real-world case studies from Greenplum users including Morgan Stanley, the European Space Astronomy Centre, Insurance Australia Group, Purdue University, Baker Hughes (a GE company), Conversant, and others, plus presentations and deep-dive tech sessions for novices and experts alike.

The inaugural PGConf US Local: Ohio Conference (PGConf Ohio) will be held September 29th - 30th at the Hyatt Regency Columbus Ohio (350 North High Street Columbus, Ohio, USA, 43215).

• Friday, September 29, 2017: Training. Registration for the PGConf Ohio trainings is open now.

Mastering Postgres Administration: Bruce Momjian

With the pending GA release of Star Link, more cities adopting municipal WIFI, and the growth of remote work due to the pandemic, one would assume that cell phone providers would be falling hand over fist to provide quality services at a quality price. Unfortunately this is not the case and it is hurting the future of our workforce.

Carriers currently think that 30GB of Hotspot data is enough for the average digital nomad. While that may have been the case previously, it isn’t any longer. There is a hundreds-of-thousands-strong workforce specializing in Information Technology, Digital Design, Web Development, and other creative industries. They are living, working and adventuring in RVs, Sprinters, Skooolies (pictured), and other vehicle dwelling options. This community is nomadic by design and should not be artificially limited by technology. 

 

Why is it that we are being charged so much money for so little? Outside of two providers announcing rural initiatives, we are still acting like data needs are small and that they come at a quality price. Data is ubiquitous; it is the new water (for getting work done). In the new economy workers need faster, higher quality, and unlimited amounts of data to get their job done.

 

Where could you go and what could you experience if you weren’t tethered by the faux limitations of bandwidth?

Resources

• Community Chat 
• Upcoming Free Webinars
• Community Blogs

Featured Content

• Using PostgreSQL, PostGIS, and pgRouting for street sweeping
• Zheap dropped by EnterpriseDB: Cybertec takes lead
• Professionalism Matters

After National 2017, the PGConf US team decided to take a long hard road to bring the best in educational content and advocacy to the entire Postgres ecosystem. Today, we welcome 2ndQuadrant as a Platinum sponsor and thank them for helping us with our year over year projects.

Other PGConf US news:

Join the fantastic and growing Postgres community in Cape Town, South Africa for a single day event on October 3rd, 2017! The event is being hosted by fellow Postgres advocates who travel from South Africa each year to attend our National Event in order to increase their knowledge of Postgres and be a part of the community. This year they are joining us and making a commitment to build out our International community and conferences!

• The CFP is open is open from now until August 25th. We will notify speakers by September 1st! Talks are subject to our presentation guidelines.

Local events are designed to bring comprehensive educational content and networking opportunities to the "local" Postgres community where the event is being held. They are perfect opportunities to show support for Postgres, find leads, and build relationships with other professionals and companies using and supporting Postgres.