Apr 13, 2023

Breaking Down the First Principles of Ibis

Kae Suarez

person running infront of white building

TL;DR A first principle is a basic proposition or assumption that cannot be deduced from any other proposition or assumption. (Source: Wikipedia)

The Ibis project is steadily gaining adoption across the enterprise landscape. As new releases come out with more features and capabilities added, we wanted to strip the project down to its basics and offer a clear definition of what Ibis does.

Defining Ibis

We’re going to work backward, so stick with us. First, here’s a sentence to define Ibis:

Ibis is the Python-native interface for X*

*Where X is your query engine

Now, that’s a loaded sentence — we’re going to define every term here, so it makes sense, then come back to it.

Python-native

Python-native means Ibis is built for and in the Python programming language. Plenty of very good libraries — even PyArrow, which we love — are powered by bindings, which are functions that directly funnel calls to code in other languages. However, this can cause confusion in documentation, or necessitate code that isn’t Pythonic in nature. In some cases, it can even make debugging harder, as you have to hunt down problems in other languages. Because Ibis is Python-native, it behaves like Python and handles as many errors as it can in the interpreter.

Ibis aims to be purely Pythonic - you can play and stay with techniques of the Python ecosystem. Without Ibis, a Python interface provided by a database, such as DuckDB, SQLite, or DataFusion, will often require users to write SQL, put it in a string, then pass it to the interface.

Interface

Ibis is not a compute library. Ibis is an interface for data interaction. It is decoupled from compute, using a custom-made set of connectors under-the-hood to get to tools that are good for handling compute. Because of this philosophical difference — emphasizing interface over compute — Ibis shines as an interface for data interactivity. If you take away nothing else, know that Ibis is an interface and aims to be nothing more, and most definitely nothing less.

X

As we hinted at above, Ibis isn’t just the Python-native interface for, say, DataFusion alone. The decoupling from compute allows Ibis to leverage the strengths of the various backends available on the market, from Polars to Snowflake — and also doesn’t lock users into any of them. It takes inspiration from the backends to provide for the needs of users but is not defined by the underlying implementations.

Today, Ibis supports a total of 16 backends — and that number is expected to grow.

The Python-Native Interface for X

With terms defined, allow us to reiterate. Ibis is the Python-native interface for X and aims to be the best interface it can for data interactivity, all achieved in Python, and connecting to as many backends as possible.

To get started with Ibis and experience its highly portable Python-native nature, visit the Ibis Project website. If you want help getting started with Ibis or even pushing new horizons, check out our enterprise support options.

To learn more about Ibis, check out our Ibis Page for more information and links to additional blogs.

Photo by Jakob Braun