Posts tagged “Postgresql”

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our Diamond and Platinum sponsors for this year in our Sponsor Spotlight Series.

Brad Nicholson is a database engineer and the PostgreSQL team lead at Compose, an IBM Company, which is one of our Platinum Sponsors for PostgresConf US 2018. Compose runs a PostgreSQL as a service platform, and has long been a supporter of the Postgres community through contributions and support. Read what Brad has to say about Compose and Postgres:

Tell us about your commitment and contribution to the Postgres Community.

Postgres is a big part of our business, and one that is rapidly growing.  As such, our  commitment is pretty self-explanatory – we are committed to postgres and the PG Community. Basically, the better PG is/becomes, the better product we can build on top of it. Our biggest contribution to the community is probably Governor.  While we have since deprecated that project, Patroni is a fork of it, and uses the HA template we created with Governor. 


What particular challenges did you face when building multi platform deployments with Postgres?

Lack of management API was one of the biggest challenges.  This leads to less than desirable patterns like having to run Patroni and Postgres in the same container, effectively tying the lifecycle of the two together. There are also a number of places where log parsing is still required to ensure the validity of an operation (like ensuring PITR \ restores actually restored to the point you specified).  These sorts of patterns are challenging to handle and often lead to less than desirable architectural patterns at the platform level.

What growth pattern do you expect for yourself as well as Postgres as a whole?

I've been using Postgres since 2001. To watch its growth over the years has been impressive, especially over the past few years.

Postgres has long established itself as the number one Open Source RDBMS.  With the huge shift we have seen towards open source adoption in the past several years, I only see it's growth continuing to accelerate.

As an organizer of the Toronto Postgres User Group I'd personally like to get more involved in the community again.  I'm not a C developer, so advocacy and helping people out via the lists/slack/etc where I can.  Now that we've deprecated Governor, I'm also looking forward to contributing to the Patroni project more.

What features would you like to see in v11 and v12?

My number one ask is Failover Slots.  Without them, it makes it difficult to for us to give our customers access to Logical Replication and Logical Decoding.  We use streaming replication for HA, and abstract those details away from our users. A HA failover will break whatever systems are built on these constructs - we lose the replication slot that maintains the place in the decoding stream, most likely requiring a resync of the dependent systems.  That is not a great story for people building downstream systems.

The other thing I'd love to see is connection pooling in core.  This has been a huge Achilles heel in Postgres for ages. Pgbouncer and pgPool are nice products to help work around the limitations, but they are severely limited when it comes to multi-tenant systems.These systems frequently need to spread their connections out across multiple users and/or multiple databases within a given cluster. Because we can't share these connections via external pools, we end up with connection explosions. 

What is the number one barrier you see to contributing to the Postgres community?

 Not a very exciting answer, but time. There just aren't enough hours in the day to fit everything in.

What is the best thing about working with the Postgres community?

How helpful, open and responsive the people in the Community are.  When you have a question or problem, getting direct access to people via the mailing lists, Slack, etc is great.  Often you'll be talking with the folks that wrote the code in the first place.  People are always really helpful.

Tell us why you believe people should attend PostgresConf 2018 in April.

This conference is an amazing opportunity to learn about all sort of different areas from the experts.  Meeting folks face to face is always another huge benefit.

Visit the Compose team in the Exhibit Hall in the Newport Grand Ballroom on Wednesday, April 18, and Thursday, April 19.  IBM Senior Developer Advocate Raj Singh will present  "Do data science and machine learning with Postgres on the IBM Cloud" in his keynote on April 19 at 3 pm, which also takes place in the Newport Grand Ballroom.

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our speakers for this year in our Speaker Spotlight Series.

Malcolm McLean of Apace Systems is a PostgreSQL DBA, Linux admin, Java/PHP developer and according to his bio "a bit of a pedantic perfectionist", with over 12 years of experience with PostgreSQL and somewhat more than that with Linux and development.

Malcolm is presenting "PostgreSQL in a Geographically Distributed Realtime Transactional System" on Thursday, April 19, at 11:20 am. Read what he has to say about Postgres and why to attend his session: 

Why PostgreSQL?

My very first job over 13 years ago introduced me to Postgres after having only worked with MySQL at university. Since then, I've been happier with its performance compared to other DB's I've benchmarked as well as with the features that are continuously introduced to keep the database on the forefront. Across 3 companies I have never looked back and never regretted my decision to always use Postgres.

Tell us about your involvement with the greater Postgres community.

I spoke at the first PgConf South Africa last year and I'm now one of the organisers of PostgresConf South Africa 2018.

What new features of PostgreSQL 10 are you most excited about?

Definitely the integrated logical replication without the need of an extension.

What features should be developed/improved and released in the next major upgrade?

It perhaps won't make Postgres 11, but at least by Postgres 12 we should have BDR integrated with the current logical replication capabilities rounded off.

Why should attendees come to your talk at PostgresConf US 2018? What would you like for them to take away from your session?

We had an interesting replication problem to solve. Multi-master replication where data needed to be replicated betweens servers in multiple countries, some with data restrictions, but without replicating data where it didn't need to go. So a sort of conditional sharding with redundancy in each data center.

What sessions are you most excited about attending at PostgresConf US 2018?

 The talks on replication (anything to make our lives easier) and security. 

What is your favorite aspect of PostgresConf US?

Meeting like-minded people from different backgrounds. And of course, the trip itself. I've done quite a bit of travelling over the years, but I haven't yet been to the US.

What advice would you have for a Computer Science graduate or entry level developer who are interested in learning and engaging with Postgres?

Push yourself to learn. Sitting back doesn't get you very far. 

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our speakers for this year in our Speaker Spotlight Series.

Elizabeth Zalman is the Co-founder & CEO of strongDM, the definitive data security product. Previously she was Co-Founder and CEO of the cross-device profile company Media Armor.

Liz will be presenting a breakout session "Do You Know Your Vendor Dependencies" on Wednesday, April 18, at 4 pm. Read what she has to say about Postgres and why to attend her session: 

Why PostgreSQL? What got you into it, and made you stick with it?

Featureful performance & proven reliability. Over the years I've found fewer and fewer reasons to consider alternatives beyond Postgres (two notable exceptions being redis-style KV and queues).

Tell us about your involvement with the greater Postgres community.

strongDM hears a lot about how its customers use PG and its derivatives (Greenplum, the latest on Aurora, etc.) and we love sharing tips when appropriate.

What new features of PostgreSQL 10 are you most excited about?

As a security product, we're always happy to see security-related upgrades: SASL/SCRAM authentication is a popular option in several of database systems, and is a significant upgrade over the previous salted hashes. We also rely heavily on partitioning, so all related enhancements are welcome.Finally: although we generally wouldn't choose to run parallel queries in a primarily OLTP system, the promise of running analytic queries on a replica (rather than ETL'ing into an analytic database) is very appealing.

What features should be developed/improved and released in the next major upgrade?

Dynamic optimization for queries on partitioned tables. Right now if you run select * from date_partitioned_table where event_timestamp between NOW() and NOW() - interval '7 days'; It scans every partition because the result of NOW() is not knowable ahead of time. But it could optimize that query when it was asked to execute it. Instead of creating a static query plan it could say, I know I'm going to have to compute a list of partitions. So, let's just remember that. Then right before the query runs, it could calculate the list and run the query as though it had been planned with a static date range, and thus only query the affected tables. 

 Why should attendees come to your talk at PostgresConf US 2018?

In this talk, I'll share practical examples of breaches I've personally experienced along with advice on how to hold your vendors accountable, and hopefully make the topic of security a little less intimidating in the process.

What sessions are you most excited about attending at PostgresConf US 2018?

I am looking forward to attending “Reducing the Surface Area of Risk in Data Security” by Tim Gorman and “General Data Protection Regulation (GDPR) with Azure Database for PostgreSQL” by Mark Bolz.

What is your favorite aspect of PostgresConf US?

Being able to network with and learn from a unique community! I’ve found practice to really be one of the best ways to learn about databases.

 What advice would you have for a Computer Science graduate or entry level developer who are interested in learning and engaging with Postgres? 

 I’ve found practice to really be one of the best ways to learn about databases. Get your hands dirty: try breaking things and then fixing them! 

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our speakers for this year in our Speaker Spotlight Series.

Hubert Zhang will co-present with Jack Wu at PostgresConf US 2018 on "Customize and Secure the Runtime and Dependencies of Your Procedural Languages using PL/Container." Hubert is a staff software engineer at Pivotal. He received his Master Degree at Peking University, with a major in artificial intelligence. He is most interested in database systems and distributed computing platform.

Tell us about your involvement with the greater Postgres community. (How long have you been involved? How have you contributed? How else would you want to contribute?)

I've been working on Postgres based MPP database Greenplum and HAWQ since 2014. I contributed on PLContainer in Greenplum, data locality, and Ranger module of HAWQ (a SQL-On-Hadoop system).

What features should be developed/improved and released in the next major upgrade?

Vectorize execution for OLAP queries.

Who should attend your talk at PostgresConf US 2018? What would you like for them to take away from your session?

Data scientists and anyone who want to use Python and R in database to do data analysis and machine learning. You'll learn how to use PLContainer as well as how to build a customized docker image to setup your specialized Python or R environment.

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

Pavan Deolasse is a PostgreSQL consultant for 2ndQuadrant, one of our Platinum Sponsors for PostgresConf US 2018. Learn more about Pavan in this Community Profile:

Pavan Deolasee is a contributor to the development of PostgreSQL and related technologies. He holds a Masters of Technology in Computer Science from the Indian Institute of Technology, Bombay, and has several publications to his credit.

Pavan was introduced to PostgreSQL in the year 2006 when he started working for a PostgreSQL company. Hacking PostgreSQL was a completely new experience, but by accepting help from the community -- especially Simon Riggs, his mentor -- he began working through the code and was soon working on a very important project (Heap-Only-Tuple aka HOT). It was a very complicated piece of work, yet one of the landmark features in PostgreSQL and to this day remains one of his most important contributions to the project. Pavan has since contributed several patches to PostgreSQL. 

HOT was committed to PostgreSQL in 2008 and Pavan has continued to stay fully engaged with the project. In 2009/2010, alongside other colleagues, Pavan started work on Postgres-XC, a distributed, shared-nothing, clustering solution based on PostgreSQL. He was involved in designing the high-level architecture of the solution, and later wrote many important pieces of the code such as the GTM, the GTM Proxy, the 2PC transaction controller at the coordinator, changes required to guarantee global consistency of the data, and many distributed planner and executor pieces.

When Postgres-XL, a fork of Postgres-XC, was released to the public in 2014, Pavan once again became involved with the project and has been leading the development of Postgres-XL ever since. 

Pavan joined 2ndQuadrant not long after in April 2015, and once again started working with long time colleagues and friends; he and 2ndQuadrant have continued to actively work on the development of Postgres-XL. The focus area of development has been stability and performance of the product while keeping the fork as close to open source PostgreSQL as possible. The most recent release (Postgres-XL 10) supports intra-datanode query parallelism, thus taking distributed query processing to the next level.

In 2013, Pavan and two of his colleagues started the India PostgreSQL User Group. They started conducting small meet-ups, in order to promote the technology in India and bring the local community together. These small gatherings have now expanded with a membership of over 600 members, and into a full scale conference: PGConf India, Asia's largest PostgreSQL conference with more than 250 people in attendance each day in 2018. 

As part of the 2ndQuadrant team of PostgreSQL developers, Pavan continues to work on many interesting new features. He is an active part of the production support team and also has the opportunity to investigate interesting customer problems, often leading to important and critical bug fixes to the open source project.

Pavan had this to say about his involvement with the project:

“I love PostgreSQL and the PostgreSQL community, so you will continue to see me around.”

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our Diamond and Platinum sponsors for this year in our Sponsor Spotlight Series.

Jacque Istok, is the Head of Data for Pivotal, one of our Diamond Sponsors for PostgresConf US 2018. Pivotal is hosting the first annual Greenplum Summit at PostgresConf US 2018, with lots of great Greenplum and Postgres-related content. Read what Jacque has to say bout Greenplum and Postgres, as well as why to attend the Greenplum Summit: 

Greenplum is an Open Source variant of Postgres; what benefits do you bring to the table over vanilla Postgres?

Postgres is a powerful ORDBMS, but as your data scales, the only way to keep up is to buy bigger and bigger machines to run on. It suffers from the same problems that all SMP databases do: you can only get as big as the machine you’re running on.

With Greenplum you can put a subset of your data on a Postgres database on one reasonably-sized machine, and another subset on a second machine, and so on. All of your users and applications can then query one of these Postgres databases as if all the data was in a single location - making your data scale limitless. Greenplum manages the distribution, data shuffling, and querying of all of your data across a magically sharded implementation of Postgres databases.

Greenplum has its own community; what do you hope to achieve by joining the Postgres community and PostgresConf?

The Postgres community represents some of the most passionate and knowledgeable creators, developers, and users of database technology of our time. We believe that the combination of Postgres and Greenplum becomes the software equivalent of what Oracle Exadata purported to be: an all-purpose database that can do both transactional and analytical workloads across multi-structured data. Simply put, the Greenplum community is looking to join with the Postgres community to further the understanding and adoption of these technologies.

Do you have plans for cross pollination of technologies with the two open source projects?

Greenplum forked from Postgres over 10 years, circa Postgres 8.2. Greenplum 5.0 is based off of Postgres 8.3, with our next major release slated for Postgres 9.4 (current open source Greenplum is compatible with 9.0 as of this writing).

Likewise, we have Postgres committers working at Pivotal looking for opportunities to improve the Postgres code specifically for analytics. We are also ensuring that other projects related to Greenplum, like Apache MADLib, continue to be compatible with Postgres.

What challenges do you see working with the Postgres community as an open source fork?

The Postgres community is a long-running and very passionate group, and we want to be both collaborative and respectful in how we continue to grow our participation. We see the products as having synergies which complement each other very well, with some use cases that best fit Postgres, and others that best fit Greenplum. The use of either benefits the other as they both further adoption.

What would you tell a user who has a choice between Postgres and Greenplum about when they should use which system?

Postgres is a great ORDBMS that will scale to the performance of a single server. For analytical needs, being restricted to a small number of terabytes does not allow for the type of exploration that most organizations need. Because Greenplum is a Postgres compatible database, you can start out using Postgres and either convert to Greenplum underneath or leverage Greenplum alongside your Postgres systems (making data ETL a ton easier). This then makes the choice of which product to use for your particular use case clearer and clearer.

What is the number one barrier you see to contributing to the Postgres community?

The number one barrier we will have to contributing is not seeing the corresponding adoption of our technologies. We feel very strongly that both the transparency and removal of vendor lock-in make our open source commitment the only choice for users. I’m here to implore the community to embrace our technology with zeal and help us continue to drive more and more Postgres adoption in the world.

What is the best thing about working with the Postgres community?

Because Greenplum is based on Postgres, we get to interact with this vast community of talent. We are also able to more seamlessly interact with ecosystem products that already work with Postgres, making the adoption of Greenplum that much easier.

Tell us why you believe people should attend PostgresConf 2018 in April.

PostgresConf is going to be awesome - with both Pivotal and Amazon headlining as Diamond sponsors - as well as the quality of speakers and their content. I wouldn’t miss it for anything.

We’re thrilled to organize the first annual Greenplum Summit at PostgresConf. Greenplum co-founder, Scott Yara, will give a keynote on April 18th relating to how data tells the story at the organizations that we help enable (#DataTellsTheStory), and his journey from SMP to MPP. Greenplum Summit on April 19th will be a full day packed with with great use case sessions and tech talks for novices and experts alike.

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our speakers for this year in our Speaker Spotlight Series.

Baron Schwartz is the founder and CEO of VividCortex, the best way to see what your production database servers are doing. Baron has written a lot of open source software, and several books. He’s focused his career on learning and teaching about performance and observability of systems generally (including the view that teams are systems and culture influences their performance), and databases specifically.

Baron will be presenting a breakout session "How To Index Your Database" on Friday, April 20th at 12:50 pm. Read what he has to say about Postgres and why to attend his session: 

Why PostgreSQL? What got you into it, and made you stick with it?

The community has always been what’s drawn me to Postgres. From the very first I could immediately sense the deep commitment people had to the database and other people involved with the database. Each database community is special in its own way and has its own culture, and after being involved in many of them you sense it in a way you can’t explain. Postgres’s is loyal to freedom, independence, quality, and principles. All are dear to me as well.

Tell us about your involvement with the greater Postgres community.

I’ve participated since 2008 in various ways. I’ve blogged, written open source, written books, reviewed and tech-edited other people’s books, spoken, and attended. I hope to continue doing so!

What new features of PostgreSQL 10 are you most excited about?

Partitioning! Improved streaming replication! Parallel query execution! There are so many improvements. The pace of development is really impressive.

What features should be developed/improved and released in the next major upgrade?

The database of the microservices-oriented future is only quasi-stateful. By this I mean that it acts almost like a read-through cache in front of “cold storage” like S3. Support for creating an empty database cluster, and having it fetch its data from cold storage on first query, would be game-changing for cloud-native apps. I realize that’s a lot of buzzwords, but it’s going to happen soon!

Why should attendees come to your talk at PostgresConf US 2018? What would you like for them to take away from your session?

Indexing is at once the most important, simplest, and most complicated topic in designing a database; I hope to make it simpler and leave people with three big things they will never struggle to remember.

What sessions are you most excited about attending at PostgresConf US 2018?

I'm really interested in hearing how others are using Postgres, both for creative purposes that showcase its flexibility and extensibility, as well as "use as designed." In my opinion, the best conference talks aren't "you should do this"; types of talks: they are "I did this and here's what happened"; I love stories!

What is your favorite aspect of PostgresConf US?

That’s an easy one to answer: seeing all my old friends again!

What advice would you have for a Computer Science graduate or entry level developer who are interested in learning and engaging with Postgres?

Computer Science seems to feel that algorithms, data structures, and so on are fundamental. Data persistence, especially relational, is equally important. Don’t neglect.

You can invite any three living people from anywhere in the world to dinner. Who do you invite and why?

I invite the three hungriest people in the world and eat mindfully with them, so I learn to appreciate what I have.

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our speakers for this year in our Speaker Spotlight Series.

Jignesh Shah is Product Manager for Amazon RDS with Amazon Web Services. At AWS he works on PostgreSQL engines at AWS including Amazon RDS PostgreSQL and Amazon Aurora with PostgreSQL compatibility. He has worked with PostgreSQL since 2005 and has released application and database lifecycle products using PostgreSQL at appOrbit and VMware.

Jignesh will be co-presenting the full day "Amazon Relational Database Service for PostgreSQL and AWS Database Migration Service Training" on April 17 at 9 am, as well as the "Deep Dive into the RDS PostgreSQL Universe" session on Friday, April 20 at 9:50 am. 

Read what he has to say about Postgres and why to attend his session:

Why PostgreSQL?

Features, Open Source, License!

Tell us about your involvement with the greater Postgres community.

Involved since 2004ish. Early part focussed more on performance on high end servers. (Performance bottlenecks, working to resolve them in standard PostgreSQL) Lately more as an advocate.

What new features of PostgreSQL 10 are you most excited about?

Pub/Sub Logical Replication

What features should be developed/improved and released in the next major upgrade?

CALL Procedure (multi-transaction), SQL Packages

Why should attendees come to your talk at PostgresConf US 2018? What would you like for them to take away from your session?

Learn about the features in Amazon RDS for PostgreSQL and how best to leverage them!

What sessions are you most excited about attending at PostgresConf US 2018?

Keynotes!, Tutorials (Aurora, DMS), Customer sessions!

What is your favorite aspect of PostgresConf US?

Meeting the user community of PostgreSQL

What advice would you have for a Computer Science graduate or entry level developer who are interested in learning and engaging with Postgres?

Start with the use cases. Learn how PostgreSQL supports these use cases. Dive deeper on why PostgreSQL solves them in that particular way!

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our speakers for this year in our Speaker Spotlight Series.

Tim Gorman is a technical consultant for Delphix who enable data virtualization and data masking to increase the agility of IT development and testing securely. Tim is co-author of six books about Oracle data warehousing and performance optimization, and has performed technical review for eight other published books. 

Tim will be presenting a breakout session on Wednesday, April 18 on "Reducing The Surface Area Of Risk In Data Security." Read what he has to say about Postgres and why to attend his session:

Why PostgreSQL? What got you into it, and made you stick with it?

30+ years in IT and databases, and PostgreSQL is the obvious heir apparent as every organization's default choice of data store.

Tell us about your involvement with the greater Postgres community. 

Just becoming involved.  I have been heavily involved in the Oracle user community for almost 25 years, and I have been getting involved in the Microsoft user community for the past 3 years.

What features should be developed/improved and released in the next major upgrade?

pgPL/SQL

Why should attendees come to your talk at PostgresConf US 2018? What would you like for them to take away from your session?

Creating environments for software development on PostgreSQL must include data masking to prevent confidential data from "bleeding" over from production to non-production environments.

What sessions are you most excited about attending at PostgresConf US 2018?

Mastering PostgreSQL Administration

What is your favorite aspect of PostgresConf US?

Networking with my new technical community

What advice would you have for a Computer Science graduate or entry level developer who are interested in learning and engaging with Postgres?

What is learned when optimizing PostgreSQL is also applicable to other databases

Bonus Question: You can invite any three living people from anywhere in the world to dinner. Who do you invite and why?

Vladimir Putin, Michelle Obama, and Justin Trudeau.  Differing people with different accomplishments on the current world stage, and I would be fascinated to listen to how they react to one another (and to me) once the barriers and inhibitions are dropped.

 Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!

As part of the countdown to PostgresConf US 2018, learn more about the engaging content and our speakers for this year in our Speaker Spotlight Series.

Tom Limoncelli is the SRE Manager at StackOverflow.com and author of Time Management for System Administrators (O'Reilly). He is co-author of The Practice of System and Network Administration (3rd edition just released) and The Practice of Cloud System Administration.  

Tom will be presenting a half-day training on Monday, April 16 on "Personal Time Management: The Basics for Overloaded People." Read what he has to say about Postgres and why to attend his session:

Why PostgreSQL? What got you into it, and made you stick with it?

The startup I was at in 2003 needed a database. Since we did a lot of work with IP and network addresses, we picked PostgreSQL because it supports those as native types.

Tell us about your involvement with the greater Postgres community. 

I'm new to the greater Postgres community. I've been a system administrator since 1988 and often speak on related topics.

What features should be developed/improved and released in the next major upgrade?

Anything that makes replication and failovers easier and more reliable. 

Why should attendees come to your talk at PostgresConf US 2018? What would you like for them to take away from your session?

Time management: you will gain an additional day out of every week. Radical ideas: you'll learn how modern software ("distributed computing") is radically changing how we build systems and do operations. I'd like to give them hope for the future.

What is your favorite aspect of PostgresConf US?

That it's in my back yard!  I live 10 miles from here!

What advice would you have for a Computer Science graduate or entry level developer who are interested in learning and engaging with Postgres?

Give yourself projects. Try to break things and then see if you can repair them.  You'll learn more from recovering a failed system than anything else.

Bonus Question: You can invite any three living people from anywhere in the world to dinner. Who do you invite and why?

Obama (because I'd like to thank him), Steve Martin (because he has an amazing life story and is super smart), and Jennifer Lawrence (she also has an interesting life story).  I also think that these three people would get along well.

Check out the full schedule for PostgresConf US 2018, and buy your tickets soon!