Archive for 2020

With the pending GA release of Star Link, more cities adopting municipal WIFI, and the growth of remote work due to the pandemic, one would assume that cell phone providers would be falling hand over fist to provide quality services at a quality price. Unfortunately this is not the case and it is hurting the future of our workforce.

Carriers currently think that 30GB of Hotspot data is enough for the average digital nomad. While that may have been the case previously, it isn’t any longer. There is a hundreds-of-thousands-strong workforce specializing in Information Technology, Digital Design, Web Development, and other creative industries. They are living, working and adventuring in RVs, Sprinters, Skooolies (pictured), and other vehicle dwelling options. This community is nomadic by design and should not be artificially limited by technology. 

Why is it that we are being charged so much money for so little? Outside of two providers announcing rural initiatives, we are still acting like data needs are small and that they come at a quality price. Data is ubiquitous; it is the new water (for getting work done). In the new economy workers need faster, higher quality, and unlimited amounts of data to get their job done.

Where could you go and what could you experience if you weren’t tethered by the faux limitations of bandwidth?

Resources

• Community Chat 
• Upcoming Free Webinars
• Community Blogs

Featured Content

• Using PostgreSQL, PostGIS, and pgRouting for street sweeping
• Zheap dropped by EnterpriseDB: Cybertec takes lead
• Professionalism Matters

By Magnus Brun Falch CoFounder of Bamboo Solutions AS

From zero to PostgreSQL extension in three hours.

I recently had a personal side-project which included setting up a working REST backend for a mobile and web application. There were numerous challenges to overcome but my available hours to work on them were limited. I had a few evenings to spare on this project and didn’t feel like writing a lot of boilerplate in C# to achieve it.

A trusted friend had praised PostgREST many times. My testing showed it outclassing ASP.NET Core for all my test cases by an order of magnitude, half a year earlier.
I finally had a chance to use it as the base for my API layer in one of my projects.

One thing that this project called for which would not normally be easy to achieve using PostgreSQL alone, is to pre sign requests for an S3 compatible storage platform. I had 6 hours left in the budget for this feature, after all the firm requirements were estimated.

I didn’t want to write a full implementation of the relevant parts of the S3 spec from scratch or using pgcrypto. My first instinct was to have a look at the different procedural languages available in PostgreSQL, some of which are Tcl, Java, JS, Python and Julia. None of them were too tempting at the moment.

Luckily I was an early member of the new People, Postgres, Data: Discord server and there are some very interesting and clever people there, Eric Ridge, the main author of ZomboDB (ZDB) among them.

I’ve kept an eye on ZDB for years after doing some research into ElasticSearch. My project included text search so I relished the opportunity to chat with him. Among our many topics of conversation were the upcoming release of a new Rust based version of ZDB and its underlying framework: PGX. I asked him about whether he thought PGX would be a good fit for my remaining challenge. He said “Absolutely”. I spent an hour setting up an Ubuntu VM with all the needed tooling & dependencies including building and testing the “hello world” example. A thorough reading of the man page would likely have saved me half an hour.

The next 30 minutes were spent identifying the most promising Crate to use. Within three hours of writing the first line of code I had a working plugin. About 80% of the time spent was lost to unfamiliarity with the language and writing it using nano over ssh instead of setting up a proper IDE like Jetbrains CLion. I also struggled a bit with finding the correct way of extracting the value from a result of an async function in a foreign language.

In total I spent 4 hours and 25 minutes from the time I started the fresh Ubuntu 20.04 server I was going to use as a Dev environment, to having a working build of an extension. It can be called inline in PostgreSQL to sign download requests in less than 1ms on my Intel I5 2500 testbed.

 I’ve had zero experience with Rust and have never read any teaching materials or looked at any tutorials for it before that evening. What I’d heard about Rust previously was that it’s really fast, and that the compiler would yell at you until your code is of acceptable quality.

Example of the PGX simplicity:

The extension was merely for internal use in a specific project and has some rough edges. It is published on GitHub and needs some more polish before I would consider it a production ready extension. Pull requests are welcome if you have suggestions before I get the time to work on it some more.

Notes:

• PGX is a framework for writing PostgreSQL extensions using Rust.
• ZomboDB integrates Elasticsearch and PostgreSQL.
• PostgREST consumes a PostgreSQL schema and generates API endpoints.

This work is unrelated to my job at Bamboo Solutions AS

What’s your view?

Over 25 years ago I got into an argument with my boss. He was frustrated with all the tasks that needed to be completed and feeling overwhelmed. I said, “Hey, why not go to a park and take in the scenery? You could work there too.” Those were the days before the Internet was required to complete your day to day. He was incredulous, “I don’t pay 1200.00 a month for an office to go work at a park.” It is funny how a single conversation can stick with you throughout life. Now that person is in the Cannabis industry and takes a completely different view on life.

Today I write this newsletter from Henry’s Lake, Idaho just about 30 minutes from Yellowstone National Park and the newsletter image is a photo I took this morning as I was about to make coffee. During these trying times with the pandemic and economic uncertainty it is vital that all of our community take a moment to reflect on what their view is. How is it that you are living your life? Are you putting people first? Are you helping each other as you move through the day? Are you in need of help? These are all questions we should ask ourselves and others every day.

With People, Postgres, Data we make an earnest effort to put people first, professionally and personally. This is why when we have physical events, we have tracks that are not traditional PostgreSQL faire including our Career Fair and Professional Development and Leadership tracks.It is also why we try to keep our digital events free. The better people we are, the better professionals we can be and the better the professional we are, the more we are able to help people. It is also why we started a new community chat server with Discord. Though the server is there to help you with Postgres, it also exists to embrace community with channels such as #watercooler, #food, #games and #professional-advice. It is a holistic approach that allows people to be people, not bytes.

• Community Help and Chat

RSVP for upcoming free digital Events

• September 29, 10AM PST: Introduction to PostgreSQL ColumnStore Indexes

• September 30, 10AM PT: Live Demo: Creating A Single Point Of Access To Multiple Postgres Servers Using Starburst Presto

• October 6, 9am PT: Data processing more than billion rows per second

• October 7, 10am PT: Database Isolation Levels, Data Issues and Global Transaction Consistency

• October 14, 10AM PT: Live Demo: Unlock Data In Postgres Servers To Query It With Other Data Sources LIke Hive, Kafka, Other DBMSs, And More.

• October 20, 10AM PT: PGX: Build Postgres Extensions with Rust

• October 21, 10am PT: Using PostgreSQL, PostGIS, and pgRouting for street sweeping

• October 27, 10AM PT: Logical Replication lessons learned for the Data Warehouse

• October 28, 10AM PT: How to build local communities: a meetup perspective

• November 10, 10AM PT: CYPEX: Revolutionizing PostgreSQL Application Development

• November 12th, 10am PT: Blockchain as a Database

As Postgres Conference pushes forward in the brave new world, we evolve and increase the ability for the People, Postgres, Data community to succeed. As a part of our positive adaptation we have a new website that features upcoming events, professional content (written and video), and the best in our written community via “Community Content”.

Discord

In an effort to provide a modern, friendly, and inclusive community platform, we have launched a Discord server for all things Postgres. We are providing a helpful experience with rule #1 being: Be Nice. Our discussions will branch out beyond the core of PostgreSQL and provide a forum for success with Postgres and related technologies. Join us for what is guaranteed to be a refreshing experience for the community: https://discord.gg/tjxNBCz

2021

We are seeking feedback from our community on 2021 in-person events. We are currently considering the East Coast event for October 2021 and the West Coast event for December 2021. Please help us in determining the type of event you would like to participate in!

Upcoming live events

We currently have the following webinars scheduled through October:

• September 9th, 10AM PST: PostgreSQL Parallelism Do’s and Don’ts
• September 15th, 10AM PT: Monitoring Strategy Considerations
• September 29th, 10AM PST: Introduction to PostgreSQL ColumnStore Indexes

• September 16, 10AM PT: How To Query Against Multiple Postgres Instances As If They're One Server Using Starburst Presto
• September 29, 10AM PST:Introduction to PostgreSQL ColumnStore Indexes
• September 30, 10AM PT: Live Demo: Creating A Single Point Of Access To Multiple Postgres Servers Using Starburst Presto
• October 14, 10AM PT: Live Demo: Live Demo: Unlock Data In Postgres Servers To Query It With Other Data Sources LIke Hive, Kafka, Other DBMSs, And More.

Like most conference organizers we are learning to adapt to the new world; a world where physical events are no longer viable (at least in 2020). A world where people are genuinely and realistically concerned that an in-person event would increase their chances of receiving or spreading a life threatening virus.

The question is: Are in-person events a thing of the past?

The answer to that question is a difficult one. Our friends at O'reilly and Associates have permanently canceled their in person events. Our friends in Europe recently canceled the well respected PgConf.EU and Ibiza. We had to cancel our 2020 marque event in NYC in March and our upcoming Silicon Valley conference. The local community organizer website Meetup.com has even modified their capabilities to allow for online meetups. 

Humans in general seek out fellow human contact. That contact is usually of reasonably like minded individuals or at least mutual interests. This is why events like Postgres Conference are successful, because irrespective of any personal beliefs we are all there to learn and enjoy fellow Postgres professionals. But are virtual meetups and conferences going to be enough to satisfy that connection or are people going to demand a return to a historical norm?

Challenges

Even before COVID-19, in-person events came with challenges that put significant pressure on volunteers. Between cultural communication differences, having an independent Code of Conduct committee, pricing, economies of scale, partner demands, and now social distancing, conferences are now going to be more complicated than ever. A room that once could comfortably seat 100 can now only properly sit 30. An exhibit hall is likely out of the question and one-on-one mentoring and networking are likely not going to be viable.  How do we work around these limitations? Is it worth it? Are the people in our community even interested anymore or is it time to accept a new norm?

Opportunity

Without question this is a time of reflection, continued development of relationships, and looking into the magic 8-ball; a continual asking of questions to find the right path forward. The pandemic is a tough foe but true leaders are looking forward and trying to find ways to continue to serve. For that to be successful we need your help. We have put together a poll (that can be found here) to gain insight into what opportunities we may be able to pursue in the future. Please take a couple minutes and help shape the future of Open Source events. 

As a closing, we are actively moving forward with Digital Events across the globe and have an unending Call for Presentations open for Webinars. If you have any feedback or brilliant ideas, please send them to us via organizers@postgresconf.org.

Blatant Poll Link 

At Postgres Conference we are always optimizing our People, Postgres, Data experience, and as more events move to Digital,  we have worked hard to perfect that collaborative experience for our community. The hard work of our volunteers has paid off, our content is top notch, and our participant account is consistently high.

With a now-proven track record, we are pleased to announce that we are accepting Community and Commercial presentations via the Postgres Webinar Series Call for Presentations through the end of 2020. If your presentation is accepted we will contact you directly to arrange an appropriate time to deliver your content.

Recent awesome news

KDE, arguably the most complete desktop experience for Linux, is now hosted on a private instance of Gitlab. As is proper, Gitlab only supports Postgres as the backend database.

For those who who love #FarmFresh or #farmtotable, there is a new website that showcases Farms that are delivering. It was developed using React, Rails, and Postgres. 

Upcoming Digital Events

• July 7, 10am PT: Deep Dive into PostgreSQL Indexing

A Deep Dive into PostgreSQL Indexing

• July 15, 10AM PT: Working with JSON Data in PostgreSQL vs. MongoDB

Working with JSON Data in PostgreSQL vs. MongoDB

Previous Digital Events:

• PostGIS Training series:
• Getting Started with PostGIS and OpenStreetMap by Ryan Lambert
• Advanced Operations and Features by Ryan Lambert
• Exploring OpenStreetMap from PostGIS:  Tools and Queries by Ryan Lambert
• Loading PostGIS with osm2pgsql by Ryan Lambert
• PostGIS Spatial Queries and Performance Tuning by Ryan Lambert
• PostgreSQL 12 and PostGIS 3.0 by Ryan Lambert
• Ops:
• FPGA: an Easier Path to Parallelism for Postgres? by Andrew Ellicott
• How to Deploy a SQL Database on Kubernetes by Chris Casano
• How to Move Data from Oracle to Postgres in Near-Real Time by Joshua Drake
• A Deep Dive into PostgreSQL Indexing by Ibrar Ahmed
• Mostly mistaken and ignored PostgreSQL parameters while optimizing a PostgreSQL database by Avinash Vallarapu
• Deeper Understanding of PostgreSQL Execution Plan: At plan time and run time by Jobin Augustine
• A Deep Dive into PostgreSQL Indexing by Ibrar Ahmed
• Enterprise Authentication and Connection Pooling by Erik Brandsberg
• Case Studies
• YugabyteDB: a distributed PostgreSQL database by Bryn Llewellyn
• Community vs. Enterprise Open Source – Which is Right for Your Business? by Justin Reock
• Postgres vs. MongoDB for real-time machine learning on wind turbine data by Luis Manuel Carril
• Turning PostgreSQL into a Cloud Native Data Service by Julian Fischer
• Scaling PostgreSQL: Never Shard Again and Gain Effortless Scale by Jim Walker
• Bring Compression to Postgres at Zero Cost of Performance by Tong Zhang

Postgres Conference organizers come to us through a variety of channels, but they all have their own unique story and pathways. Unlike many of our organizers, Lindsay Hooper’s background is non-technical and focused on the logistics, partnerships, and outcomes of Postgres Conference. Read on to learn more about her and her non-profit organization of choice

What is your background?

I majored in art and film at Bucknell University, and I started my career working in art galleries in Manhattan. I quickly moved to events and marketing, and within a few years I started doing events and marketing for tech companies and startups.

What do you do in your full-time job?

I have my own events company called LRH Events, and most of my clients are either in the non-profit or tech worlds. Events are a great tool for gaining brand awareness and for creating community, but getting an event off the ground takes some serious marketing chops, and so I do marketing as well.

What’s your favorite part of Postgres Conference?

There’s so much to love about Postgres Conference. When it comes to the conference itself, I love the logistics and planning that goes into what the event actually looks like. I love everything from brainstorming what the conference should look and feel like, to coordinating with speakers, streamlining talks, and coming up with schedules.

Beyond the actual events, my favorite part is the People, Postgres, Data community that surrounds the organization. I’ve never experienced such a truly people-first group, and I think that that shows in the fact that this isn’t a twice a year conference - it’s a constant connection that’s focused on people, not just Postgres or open source technology.

How do you spend your free time?

I am based in NYC, which affords me the opportunity to spend a lot of my free time exploring the city’s restaurants and museums. I highly recommend an afternoon at The Met, followed by dinner down in the East Village at either Raclette or Dirt Candy. 

Beyond that, I’ve been involved with a non-profit called Mouse for the last five years, so I also spend a few evenings each week working on fundraising and awareness initiatives. Mouse’s mission is to provide vulnerable youth with the computer science skills needed to enter and succeed in higher education and the high tech workforce. We are committed to fostering greater diversity and humanity in STEM and empowering youth - and all those that educate them - to access and amplify technology as a force for good.

How did you begin working with Mouse and what’s your role there?

I started working with Mouse five years ago when they launched the Diversity in Tech Awards, which is their annual awards event that honors folks in the tech ecosystem who foster greater diversity and humanity in STEM and empower youth. They needed event support, and I was able to step in to help.

Since then, I joined Mouse’s associate board, became the VP, and am currently the Associate Board President. 

What are the biggest challenges facing STEM education right now?

The greatest challenges facing education as a whole have shifted in the last few months due to the COVID-19 pandemic. In speaking with Mouse students, I’ve learned that a lot of what they’re up against is the variety of platforms that they’re receiving school work on and the lack of collaboration. On the flip side, most teachers are experiencing a learning curve themselves when it comes to remote teaching.

During the COVID-19 crisis, Mouse has played a frontline role in meeting the emergency educational needs. Responding to a call from the NYC Department of Education after schools closed, Mouse has:

• Trained more than 4,000 teachers in online teaching methods so they can reach their students.
• Helped more than 100,000 NYC students get access to online learning
• Digitized youth programs like the Emoti-Con Virtual Showcase and Mouse Design League for students to continue their computing projects
• Made our STEM and computer science learning platform, Mouse Create, free for all users during the crisis

How can I get involved in Mouse?

If you’re interested in getting involved with Mouse, please feel free to reach out to me directly at Lindsay@postgresconf.org

Alternatively, Mouse is proud to be able to continue our work with students and educators during the COVID-19 crisis. Please give generously today at www.mouse.org/donate so we can continue to make great strides in finding new ways to help our community in these challenging times. 

The Chairs (myself, Jim Mlodgenski, and Amanda Nystrom) have recently decided to bring some visibility to charities that are close to our hearts. They are listed below:

• Joshua Drake: Navajo Water Project. The Navajo nation is approximately the size of West Virginia and has a population of over 150,000 people (300k in the tribe). Anywhere from 15% - 40% of the residents do not have access to running water. The Navajo Water Project aims to bring clean water to each person and family through support from those that donate. 
• Jim Mlodgenski: St. Jude Children’s Research Hospital. The hospital is one of the premier research hospitals for cancer and other life threatening illnesses for some of our most vulnerable people. Approximately one in 285 children in the U.S. will be diagnosed with cancer before their 20th birthday. Through donations, St. Jude’s provides treatment to those with cancer, and is actively dedicating resources to the research and cure for cancer. 
• Amanda Nystrom: ASPCA. The American Society for the Prevention of Cruelty to Animals (ASPCS) was the first humane society to be established in North America, with the goal of providing kind and respectful treatment to animals under the law. Unlike humans, cases of animal abuse aren’t compiled but studies have shown a correlation between domestic violence and animal abuse. The ASPCA prevents animal homelessness and actively rescues animals from dangerous and/or cruel situations.

Upcoming Webinars

With the Coronavirus causing the conference market to dry up for 2020, we at Postgres Conference have pivoted to ensure that we continue to provide quality Postgres content to the world of People, Postgres, Data. We have been performing multiple webinars per month. Here is the current schedule and you can register (free) here:

• May 21, 11am PT: A Deep Dive into PostgreSQL Indexing
• June 2, 10AM PT: How to Move Data from Oracle to Postgres in Near-Real Time
• June 9, 11am PT: Community vs. Enterprise Open Source – Which is Right for Your Business?
• June 10, 11am PT: Bring Compression to Postgres at Zero Cost of Performance
• June 16, 11AM PT: Mostly mistaken and ignored PostgreSQL parameters while optimizing a PostgreSQL database
• June 30, 11am PT: Deeper Understanding of PostgreSQL Execution Plan: At plan time and run time
• July 15, 10AM PT: Working with JSON Data in PostgreSQL vs. MongoDB
• June 17, 11am PT: Postgres vs. MongoDB for real-time machine learning on wind turbine data

Articles from the community

• Why work for hire could be limiting your company and yourself
• Build PostgreSQL extensions on Windows
• Operationalizing your PostgreSQL database health checks using SQL Notebooks
• Building fast refresh capability in Amazon RDS for PostgreSQL
• Multi-tenant data isolation with PostgreSQL Row Level Security

Coronavirus Resources:

• Open-Access Data and Computational Resources to Address COVID-19

People, Postgres, Data,

Due to the health risks and travel restrictions created by the Coronavirus we unfortunately have to cancel Postgres Conference 2020 which was to be held at the Marriott Marquis on March 23rd, 2020 thru March 27th, 2020.

We want to thank all of our attendees, partners and volunteers for all the hard work we all put in to try and pull this event off. Sadly, the stars were not aligned this year. We now focus our efforts on Postgres Conference Silicon Valley 2020 and our Digital Events. 

Thank you all for your patience and support,

Postgres Conference Chairs