Interview Query recently reported that the number of data engineering roles at companies continues to grow annually, where as surprisingly, the number of data scientists has shrank back in comparison.

Data engineers spend much of their time building data pipelines and managing data warehouses, and there are lots of tools for data engineers to choose from when it comes to developing data pipelines.

In this talk, we will discuss how in many companies currently, the data engineer is the bottleneck. As data scientists and analysts are trying to quickly answer questions and create new models, data engineers are often overwhelmed by needing to quickly develop pipelines to meet their counterparts' needs. I have discussed this problem with companies across industries, and it seems to be a consistent complaint. With such a consistent problem comes a whole host of solutions. I will be discussing these solutions whether they be tools, new design methods, or changes in the process altogether. There are pros and cons to most of these alternatives but many companies will likely be attempting to implement these new ideas over the coming years.

By the end of the webinar, the viewers will leave with a deeper understanding of both how the data engineer ends up being the bottleneck, as well as the various solutions many companies are trying to use to fix this problem.

Date:

2021 April 8 13:00 EDT

Duration:

1 h

Room:

Online

Conference:

2021 Postgres Conference Webinars

Language:

Track:

Ops

Difficulty:

Requires Registration:

Yes (Registered: 129)