GPUdb API Rust Client
A high-performance, async Rust client for Kinetica (GPUdb) - a distributed GPU-accelerated database.
Features
- Async/await - Built on Tokio for non-blocking I/O
- High Performance - Achieves 600K+ records/sec with optimized bulk insert
- Type Safe - Auto-generated API from Avro schemas
- Fast Avro Encoding - Uses
serde_avro_fastfor 6-7x faster serialization - Parallel Processing - Multi-worker architecture with rayon for encoding
- Worker Distribution - Direct routing to worker nodes for maximum throughput
- Logging - Built-in logging via the
logcrate (default level: WARN)
Logging
This library uses the log crate for logging. To enable logging output, add a logging implementation like env_logger to your application:
Then initialize it in your main function:
Log Levels
Messages are logged at these levels:
- ERROR: A request that could not be sent (“Giving up with an exit error after <n>ms …”), and failures the call goes on past, such as a KiFS file that could not be transferred. Errors are also returned via
Result. - WARN: Failing over to another cluster, and settings that are ignored
- INFO: Failing back to the primary cluster, and KiFS files transferred
- DEBUG: Connecting, configuration details, worker setup, routing info
- TRACE: Per-batch details, HTTP request/response info (very verbose)
The level is the application’s logger’s to set, for the gpudb_api target (as with RUST_LOG=gpudb_api=debug above), or for all with log::set_max_level.
Quick Start
Re-exports
pub use crate::gpudb::Kinetica;pub use crate::gpudb::KineticaOptions;pub use crate::gpudb::KineticaVersion;pub use crate::gpudb::HaFailoverOrder;pub use crate::gpudb::HaSynchronicityMode;pub use crate::gpudb::ClusterAddressInfo;pub use crate::gpudb::DEFAULT_HOST_MANAGER_PORT;pub use crate::gpudb::USER_AGENT_PROGRAM_NAME;pub use crate::errors::KineticaError;pub use crate::bulk_inserter::BulkInserter;pub use crate::bulk_inserter::BulkInserterOptions;pub use crate::bulk_inserter::FlushOptions;pub use crate::bulk_inserter::ShardKeyExtractor;pub use crate::bulk_inserter::ShardKeyValue;pub use crate::bulk_inserter::ShardKeyValues;pub use crate::bulk_inserter::insert_records_options;pub use crate::bulk_inserter::BatchInsertionResult;pub use crate::bulk_inserter::BatchInsertionListener;pub use crate::bulk_inserter::FnBatchListener;pub use crate::bulk_inserter::InsertError;pub use crate::bulk_inserter::QueueError;pub use crate::bulk_inserter::BackpressureMetrics;pub use crate::worker_list::WorkerList;pub use crate::avro_schema_builder::AvroSchemaBuilder;pub use crate::avro_schema_builder::AvroType;pub use crate::avro_schema_builder::TypePropertiesBuilder;pub use crate::avro_schema_builder::KineticaColumnProperties;pub use crate::record_retriever::RecordRetriever;pub use crate::record_retriever::RecordRetrieverOptions;pub use crate::record_retriever::RetrievedRecord;pub use crate::record_retriever::END_OF_SET;pub use crate::avro::encode_records;pub use crate::avro::encode_records_parallel;pub use crate::avro::encode_records_sequential;pub use crate::avro::encode_arc_records_parallel;pub use crate::avro::encode_arc_records_partitioned;pub use crate::avro::encode_records_partitioned;pub use crate::avro::encode_single;pub use crate::avro::decode_records;pub use crate::avro::decode_records_parallel;pub use crate::avro::decode_records_sequential;pub use crate::avro::decode_with_fast_schema;pub use crate::avro::decode_single;pub use crate::avro::parse_schema;pub use crate::avro::parse_fast_schema;pub use crate::avro::PARALLEL_THRESHOLD;pub use crate::sql_iterator::SqlIterator;pub use crate::sql_iterator::SqlParameters;pub use crate::json_records::GetRecordsJsonResponse;pub use crate::json_records::InsertRecordsJsonRequest;pub use crate::json_records::JsonOptions;pub use crate::sql::SqlClient;pub use crate::sql::SqlResult;pub use crate::sql::Row;pub use crate::sql::SqlError;pub use crate::sql::SqlOpResult;pub use crate::sql::PaginationStats;pub use crate::sql::BatchingSqlClient;pub use crate::sql::BatchingStats;pub use crate::sql::SqlHints;pub use crate::sql_insert_parser::ParsedInsert;pub use crate::sql_insert_parser::parse_insert_statement;pub use crate::sql_insert_parser::parse_sql_value_to_record_value;pub use crate::generic_record::Type;pub use crate::generic_record::TypeBuilder;pub use crate::generic_record::Column;pub use crate::generic_record::ColumnType;pub use crate::generic_record::ColumnBaseType;pub use crate::generic_record::GenericRecord;pub use crate::generic_record::RecordValue;pub use crate::generic_record::ArrayValue;pub use crate::generic_record::BulkInsertStats;pub use crate::sqlx_support::KineticaPool;pub use crate::sqlx_support::KineticaConnection;pub use crate::sqlx_support::KineticaTransaction;pub use crate::sqlx_support::KineticaPoolOptions;pub use crate::sqlx_support::KineticaPoolExt;pub use crate::sqlx_support::SslMode;pub use crate::sqlx_support::SqlxPaginationStats;pub use crate::sqlx_support::SqlxBulkInserter;pub use crate::sqlx_support::BatchingSqlxClient;pub use crate::sqlx_support::BatchingSqlxStats;pub use crate::sqlx_support::BatchingSqlxError;pub use crate::sqlx_support::connect;pub use crate::sqlx_support::connect_with_url;pub use crate::sqlx_support::connect_with_options;pub use crate::sqlx_support::connect_from_env;pub use crate::sqlx_support::build_connection_url;pub use crate::sqlx_support::connection_url_from_env;pub use crate::sqlx_support::DEFAULT_PORT;pub use crate::sqlx_support::DEFAULT_DATABASE;pub use crate::sqlx_support::KineticaText;pub use crate::sqlx_support::ToKineticaText;pub use crate::sqlx_support::FromKineticaText;pub use crate::sqlx_support::KineticaTimestamp;pub use crate::sqlx_support::KineticaBytes;pub use crate::sqlx_support::KineticaVector;pub use crate::sqlx_support::KineticaArray;pub use crate::sqlx_support::KineticaArrayElement;pub use crate::sqlx_support::KineticaJson;pub use crate::generated::*;Modules
High-performance Avro encoding and decoding utilities.
Avro Schema Builder for Kinetica Records
Shard-aware high-performance bulk insert using serde_avro_fast for encoding.
Common types and utilities shared between bulk inserter implementations.
The column properties a Kinetica type can give its columns, as text. A
Column takes them in its properties, and KineticaColumnProperties builds them.Error types for the Kinetica client.
KiFS (Kinetica File System) API module.
Endpoint layer generated by
gpudb_gen_api_rust.py in gpudb-generators from the gpudb-schemas endpoint schemas. Regenerate with ../gpudb-generators/process_schema_changes.py gen-files —rust; do not edit by hand. Generated endpoint layer. Every endpoint is an inherent method on Kinetica; the transport it calls lives in the hand-written runtime, not here.Generic Record Support for Dynamic Type Handling
Core Kinetica client implementation.
High-performance HTTP client using hyper directly.
Inserting and reading records as JSON.
Whether a text is a JSON object or array, and how a list of JSON records is sent as one array.
Record retriever for efficient record retrieval from Kinetica with multi-head support.
SQL client and result parser for Kinetica’s HTTP API
Parsing of
INSERT INTO … VALUES … statements.Iterating over the records an SQL query returns, a batch at a time.
SQLx Support for Kinetica via PostgreSQL Wire Protocol
Worker list for multi-head ingest and retrieval.
Macros
Helper macro to define a static Avro schema using the builder pattern.
Structs
Re-exported because generated request/response types expose Avro
bytes fields as ByteBuf; callers would otherwise need their own serde_bytes dependency pinned to a matching version. Wrapper around Vec<u8> to serialize and deserialize efficiently.The certificates of a trust store.
Traits
Trait that must be implemented by all types that can be inserted into Kinetica.