datafusion-proto
datafusion-proto
cargo install datafusion-proto
datafusion-proto

datafusion-proto

Apache Arrow DataFusion SQL Query Engine

by apache

11.0.0 (see all)License:Apache-2.0
cargo install datafusion-proto
Readme

DataFusion Proto

DataFusion is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format.

This crate is a submodule of DataFusion that provides a protocol buffer format for representing query plans and expressions.

Serializing Expressions

Based on examples/expr_serde.rs

use datafusion_common::Result;
use datafusion_expr::{col, lit, Expr};
use datafusion_proto::bytes::Serializeable;


fn main() -> Result<()> {
// Create a new Expr a < 32
let expr = col("a").lt(lit(5i32));



// Convert it to an opaque form
let bytes = expr.to_bytes()?;

// Decode bytes from somewhere (over network, etc.)
let decoded_expr = Expr::from_bytes(&amp;bytes)?;
assert_eq!(expr, decoded_expr);
Ok(())




}

}

Serializing Plans

Based on examples/plan_serde.rs

use datafusion::prelude::*;
use datafusion_common::Result;
use datafusion_proto::bytes::{logical_plan_from_bytes, logical_plan_to_bytes};


#[tokio::main]
async fn main() -> Result<()> {
let ctx = SessionContext::new();
ctx.register_csv("t1", "testdata/test.csv", CsvReadOptions::default())
.await
?;
let plan = ctx.table("t1")?.to_logical_plan()?;
let bytes = logical_plan_to_bytes(&plan)?;
let logical_round_trip = logical_plan_from_bytes(&bytes, &ctx)?;
assert_eq!(format!("{:?}", plan), format!("{:?}", logical_round_trip));
Ok(())
}

#[tokio::main] async fn main() -> Result<()> { let ctx = SessionContext::new(); ctx.register_csv("t1", "testdata/test.csv", CsvReadOptions::default()) .await ?; let plan = ctx.table("t1")?.to_logical_plan()?; let bytes = logical_plan_to_bytes(&plan)?; let logical_round_trip = logical_plan_from_bytes(&bytes, &ctx)?; assert_eq!(format!("{:?}", plan), format!("{:?}", logical_round_trip)); Ok(()) }

GitHub Stars

2.4K

LAST COMMIT

7mos ago

MAINTAINERS

2

CONTRIBUTORS

321

OPEN ISSUES

467

OPEN PRs

31
VersionTagPublished
11.0.0
1mo ago
10.0.0
2mos ago
9.0.0
3mos ago
8.0.0
4mos ago
No alternatives found
No tutorials found
Add a tutorial

Rate & Review

100
No reviews found
Be the first to rate