avro

High-performance Avro encoding and decoding utilities.

This module provides optimized Avro serialization and deserialization using serde_avro_fast for best performance, with rayon parallelism for large batches.

​

Performance

• Encoding: Uses serde_avro_fast for 2-3x faster encoding than apache_avro
• Decoding: Uses serde_avro_fast + rayon for up to 8x faster decoding
• Parallelism: Automatically enabled for batches above threshold (500 records)

These helpers split large batches over rayon’s threads, with the results of a sequential pass: records come back in order, and a batch with several bad records reports the first of them. Failures read “Could not encode object: …” and “Could not decode object: …”.

​

Usage

​

Encoding Records

use gpudb_api::avro::{encode_records, parse_fast_schema};

let schema = parse_fast_schema(schema_str)?;
let encoded: Vec<Vec<u8>> = encode_records(&schema, &records)?;

​

Decoding Records

use gpudb_api::avro::{decode_records, parse_fast_schema};

#[derive(serde::Deserialize)]
struct Employee { id: i32, name: String }

let schema = parse_fast_schema(&response.type_schema)?;
let employees: Vec<Employee> = decode_records(&schema, &response.records_binary)?;