Mar 29, 2023

Ibis: Upgraded Interface, Same Stack

Kae Suarez

Overhaul after Overhaul

In this world, great advancements are frequently paired with engineer-weeks of overhauling a system. Switching to the latest database management system requires data movement, stack updates and overhauls for compatibility, and re-training of users. Optimizing for hardware you already have is a cost that is often paid all the way up to the point that you get new hardware — then you start again.

It’s an unfortunate truth: the cost of innovation can slow you down, requiring you to rip, replace, or redo everything.

Wouldn’t it be nice to get a strict upgrade, which doesn’t demand total stack overhauls, and doesn’t require that every single user immediately learn something new or be left confused and slow?

Stay Home and Upgrade with Ibis

Ibis is one upgrade that won’t disturb your stack. Come with what you have. You can stop reading here and go try it if that’s compelling enough, but we’re happy to explain.

API and compute traditionally go hand-in-hand — if you use BigQuery, you’re using BigQuery’s API, the same for Snowflake, the same for DuckDB. While skills are often transferrable, each API comes with quirks that can have consequences that resonate throughout the stack.

By decoupling from compute, Ibis adds to your stack instead of demanding change. It is low-cost to adopt, but brings high impact: you get the rich ecosystem of Python, instead of spending time learning something new. Now your people are already halfway to being experts at any of the 15+ backends Ibis offers, because Ibis has an intuitive dataframe interface not only for Python users, but also folks with a background in R and SQL.

How to get started? Use pip/conda/mamba to install Ibis on a system that can already connect to your backend of choice, or if you’re feeling bold, install it from the source.

Then, take a breath. We’re done touching your stack.

Now, back to the keyboard. You know your stack, and your URI to get to your backend with your existing query methods. That means you’re ready to fill in the following code:

import ibis
ibis.<backend>.connect(<URI>)

Done. Now you can just use Ibis. It’s that easy to add to your stack, and it immediately unlocks a plethora of functionality in the Python ecosystem and taps Python developers in your organization.

To get started with Ibis and find out what else Ibis can do, visit the Ibis Project website. Ibis is just one high-impact, low-cost open source initiative that we at Voltron Data support, so we’d love to show other ways you can revolutionize your stack, such as with Apache Arrow and Substrait. If you want help getting started or even pushing new horizons, check out our enterprise support options.

Photo by Dan Freeman