Sep 07, 2023

Starburst Announces Support for Python DataFrames with PyStarburst and Ibis

Keith Britt

Starburst announces support for Python Dataframes with PyStarburst and Ibis. This helps customers reduce costs and complexity because users no longer need to offload data to frameworks like PySpark and implement complex data transformation workloads.

Today, Starburst, the data lake analytics platform, announced that they will roll out support for Python DataFrames using PyStarburst and Ibis. Now users can take advantage of Starburst’s accelerated query engine, Trino, directly from their Python application code. This can help reduce costs and complexity as users no longer need to offload data to frameworks like PySpark and implement complex data transformation workloads.

Voltron Data partnered with Starburst to ensure their customers can effectively use Ibis as a uniform Python API to build scalable applications that process data in the Starburst Galaxy platform. In turn, Ibis users can now take advantage of Starburst’s capabilities to build analytics expressions across multiple data sources and execute them using Trino at accelerated speeds. This is a win/win for Python users working within this ecosystem.

“Python users struggle to bridge the gap between prototypes on their laptops and production apps running on platforms like Starburst Galaxy. Ibis makes it much easier to bridge this gap,” said Josh Patterson, CEO of Voltron Data.

With Ibis, users can write Python code once and run it anywhere with any supported backend without having to worry about varying SQL dialects. Today, there are 18 Ibis backends available, for query engines and databases including Trino, Oracle, Snowflake, BigQuery, DuckDB, MySQL, and PostgreSQL. For Starburst Galaxy users, this means they can go from crunching gigabyte-scale test data on laptops to crushing petabyte-scale data in production with Ibis.

“At Starburst everything is built with openness in mind, and we are interoperable with nearly any data environment, so we’re extending that commitment to our programming languages. The partnership with Voltron Data and Ibis was a natural fit,” said Harrison Johnson, Head of Technology Partnerships at Starburst.

Starburst joins a growing list of forward-thinking companies (like Google Cloud, Heavy.AI, DuckDB, b.telligent, and more) in using Ibis to help give customers more flexible and performant access to their data. To learn more and get started, check out these resources:

Learn More About Ibis

Ibis isn’t just for connecting sources. It’s also a powerful portable dataframe library that gives Python developers simplicity, flexibility, and increased performance. Learn more by visiting the project site.