Skip to main content

​
Class GPUdbIngestor

class gpudb_multihead_io.GPUdbIngestor(gpudb, table_name, record_type: GPUdbRecordType, batch_size, options=None, workers=None, is_table_replicated=False, json_ingestion=False)

Initializes the GPUdbIngestor instance.

Parameters

get_gpudb()

Return the instance of GPUdb client used by this ingestor.

property retry_count

Return the number of times ingestion will be attempted upon failure.

get_table_name()

Return the GPUdb table associated with this ingestor.

get_batch_size()

Return the batch_size used for this ingestor.

get_options()

Return the options used for this ingestor.

get_count_inserted()

Return the number of records inserted thus far.

get_count_updated()

Return the number of records updated thus far.

set_logger_level(log_level)

Set the log level for the GPUdb multi-head I/O module.

Parameters

insert_record(record, record_encoding=‘binary’, is_data_encoded=True)

Queues a record for insertion into GPUdb. If the queue reaches the batch size, all records in the queue will be inserted into GPUdb before the method returns. If an error occurs while inserting the records, the records will no longer be in the queue nor in GPUdb; catch InsertionException to get the list of records that were being inserted if needed (for example, to retry).

Parameters

Raises

insert_records(records, record_encoding=‘binary’, is_data_encoded=True) → AttrDict | None

Queues a list of records for insertion into GPUdb. If any queue reaches the batch size, all records in that queue will be inserted into GPUdb before the method returns. If an error occurs while inserting the queued records, the records will no longer be in that queue nor in GPUdb; catch InsertionException to get the list of records that were being inserted (including any from the queue in question and any remaining in the list not yet queued) if needed (for example, to retry). Note that depending on the number of records, multiple calls to GPUdb may occur.

Parameters

Raises

flush(forced_flush=True, is_data_encoded=True) → AttrDict | None

Ensures that any queued records are inserted into GPUdb. If an error occurs while inserting the records from any queue, the records will no longer be in that queue nor in GPUdb; catch InsertionException to get the list of records that were being inserted if needed (for example, to retry). Other queues may also still contain unflushed records if this occurs.

Parameters

Raises