Multilanguage Tooling is Table Stakes

We support and maintain open standards built to improve language interoperability. Engineers can scale from local to distributed – all while using their language and tools of choice.

Developers, engineers, analysts, and data scientists can now code in SQL, Python, R, Java, Go, and more – seamlessly operating on the same data inside whatever engine they choose.

Use Arrow and ADBC to start at the finish line and leave data conversions behind

Apache Arrow Logo

Blog | ADBC for Database Developers

url icon

Arrow Database Connectivity (ADBC) gives users a single way to get Arrow data in and out of databases. It bypasses the need for time-intensive, row-to-columnar based data conversions or vendor-specific SDKs. Think of it like JDBC and ODBC, but for Arrow Data, with a strong focus on offering cross-language APIs to maximize productivity. Learn more about this standard and get access to resources to get started in this featured post.

ADBC GitHub Repo link icon
ADBC Documentation link icon
ADBC Primer Blog link icon
Technical Level 2/5
    >>> from adbc_driver_sqlite.dbapi import connect   
  >>> conn = connect()   
  >>> cur = conn.cursor()   
  >>> cur.execute("SELECT 1, 2.0, 'hi'")   
  >>> cur.fetch_arrow_table()   
  1: int64   
  2.0: double   
  'hi': string   
  1: [[1]]   
  2.0: [[2]]   
  'hi': [["hi"]]

↳ Example using Python and SQLite. The Python API is based on Python’s DBAPI standard to make it easy to use.

Bridge Ibis and data engines with Substrait

Substrait Logo

Blog | Supercharge Code Portability

url icon

Substrait is a cross-language, interoperable specification for data compute operations. It connects data analysis tools, like Ibis and dpylr, with compute engines and hardware. It is a flexible way for APIs and compute engines to share the intermediate representations (IR) for analytical computations. In this featured post, we demonstrate how Substrait enables users to write Ibis expressions without targeting a specific backend – effectively increasing productivity through code portability.

  from ibis_substrait.compiler.core 
import SubstraitCompiler   
  compiler = SubstraitCompiler()   
  plan = compiler.compile(topfilms)   
  # type (plan) --> ‹class 'substrait.ibis.plan_pb2.Plan'>

↳ Import Ibis and compile into a Substrait plan.

Technical Level 3/5

Simplify database connectivity for both clients and vendors

Arrow Flight Logo

Blog | Extend Flight SQL with Flight

url icon

Avoid converting between columnar and row-based data. Instead of building the networking layer from scratch, Flight SQL extends Flight — a fast, low-level protocol designed for transporting Arrow data — to avoid unnecessary data copies by hooking into frameworks like gRPC. Get a lay of the land and understand how these two standards work together to simplify database connectivity for both clients and vendors.

A local, dockerized Superset client connected to a Flight SQL-based DuckDB database server

↳ Current State: Every client must support every vendor

Vision: How Flight SQL connects the client API to a database

↳ Vision: How Flight SQL connects the client API to a database

Dig into additional technical resources discussing how to optimize costs and increase flexibility for data system design. 

Discover more on our blog and follow us on social.