Manages the insertion into GPUdb of large numbers of records in bulk, with automatic batch management and support for multi-head ingest. More…
Classes | |
| class | InsertException |
| The exception thrown when a batch of records fails to insert. More… | |
Public Member Functions | |
| InsertException (string msg) | |
| Initializes a new insert exception with the given message. | |
| override string | ToString () |
| KineticaIngestor (Kinetica kdb, string tableName, int batchSize, KineticaType ktype, Dictionary< string, string >? options=null, Utils.WorkerList? workers=null) | |
| Creates a new ingestor for bulk insertion of records into the specified table. | |
| Int64 | getCountInserted () |
| Returns the count of records inserted so far. | |
| Int64 | getCountUpdated () |
| Returns the count of records updated so far. | |
| void | flush () |
| Ensures that all queued records are inserted into Kinetica. | |
| void | insert (T record) |
| Queues a record for insertion into Kinetica. | |
| void | insert (IList< T > records) |
| Queues a list of records for insertion into Kinetica. | |
| InsertException (string msg) | |
| Initializes a new insert exception with the given message. | |
| override string | ToString () |
| KineticaIngestor (Kinetica kdb, string tableName, int batchSize, KineticaType ktype, Dictionary< string, string >? options=null, Utils.WorkerList? workers=null) | |
| Creates a new ingestor for bulk insertion of records into the specified table. | |
| Int64 | getCountInserted () |
| Returns the count of records inserted so far. | |
| Int64 | getCountUpdated () |
| Returns the count of records updated so far. | |
| void | flush () |
| Ensures that all queued records are inserted into Kinetica. | |
| void | insert (T record) |
| Queues a record for insertion into Kinetica. | |
| void | insert (IList< T > records) |
| Queues a list of records for insertion into Kinetica. | |
Public Attributes | |
| Int64 | count_inserted |
| The running count of records inserted so far. | |
| Int64 | count_updated |
| The running count of records that updated existing records (on primary-key collision). | |
Properties | |
| Uri | url [get] |
| The worker URL the failed batch was being sent to, if known. | |
| IList< T > | records [get] |
| The records that were not successfully inserted. | |
| Kinetica | kineticaDB [get] |
| The Kinetica client used to insert records. | |
| string | table_name [get] |
| The name of the table records are inserted into. | |
| int | batch_size [get] |
| The number of records buffered before a batch is flushed to the server. | |
| IDictionary< string, string > | options [get] |
| The insert options applied to each batch. | |
Detailed Description
Manages the insertion into GPUdb of large numbers of records in bulk, with automatic batch management and support for multi-head ingest.
Use the insert(T) and insert(IList<T>) methods to queue records for insertion, and the flush method to ensure that all queued records have been inserted.
| T | The type of object being inserted. |
This class is obsolete. Use BulkInserter<T> instead, which provides better performance and async support among other features.
Definition at line 16 of file KineticaIngestor.cs.
Member Function Documentation
◆ flush() [1/2]
| inline |
Ensures that all queued records are inserted into Kinetica.
If an error occurs while inserting the records from any queue, the records will no longer be in that queue nor in Kinetica; catch InsertException to get the list of records that were being inserted if needed (for example, to retry). Other queues may also still contain unflushed records if this occurs.
| InsertException | If the flushed records fail to insert |
Definition at line 218 of file KineticaIngestor.cs.
◆ flush() [2/2]
| inline |
Ensures that all queued records are inserted into Kinetica.
If an error occurs while inserting the records from any queue, the records will no longer be in that queue nor in Kinetica; catch InsertException to get the list of records that were being inserted if needed (for example, to retry). Other queues may also still contain unflushed records if this occurs.
| InsertException | If the flushed records fail to insert |
Definition at line 218 of file KineticaIngestor.cs.
◆ getCountInserted() [1/2]
| inline |
Returns the count of records inserted so far.
An atomic operation.
Definition at line 191 of file KineticaIngestor.cs.
◆ getCountInserted() [2/2]
| inline |
Returns the count of records inserted so far.
An atomic operation.
Definition at line 191 of file KineticaIngestor.cs.
◆ getCountUpdated() [1/2]
| inline |
Returns the count of records updated so far.
An atomic operation.
Definition at line 202 of file KineticaIngestor.cs.
◆ getCountUpdated() [2/2]
| inline |
Returns the count of records updated so far.
An atomic operation.
Definition at line 202 of file KineticaIngestor.cs.
◆ insert() [1/4]
| inline |
Queues a list of records for insertion into Kinetica.
If any queue reaches the configured batch size, all records in that queue will be inserted into Kinetica before the method returns. If an error occurs while inserting the queued records, the records will no longer be in that queue nor in Kinetica; catch InsertException to get the list of records that were being inserted (including any from the queue in question and any remaining in the list not yet queued) if needed (for example, to retry). Note that depending on the number of records, multiple calls to Kinetica may occur.
| records | The records to insert. |
| InsertException | If the records cannot be inserted |
Definition at line 343 of file KineticaIngestor.cs.
◆ insert() [2/4]
| inline |
Queues a list of records for insertion into Kinetica.
If any queue reaches the configured batch size, all records in that queue will be inserted into Kinetica before the method returns. If an error occurs while inserting the queued records, the records will no longer be in that queue nor in Kinetica; catch InsertException to get the list of records that were being inserted (including any from the queue in question and any remaining in the list not yet queued) if needed (for example, to retry). Note that depending on the number of records, multiple calls to Kinetica may occur.
| records | The records to insert. |
| InsertException | If the records cannot be inserted |
Definition at line 343 of file KineticaIngestor.cs.
◆ insert() [3/4]
| inline |
Queues a record for insertion into Kinetica.
If the queue reaches the configured batch size, all records in the queue will be inserted into Kinetica before the method returns. If an error occurs while inserting the records, the records will no longer be in the queue nor in Kinetica; catch InsertException to get the list of records that were being inserted if needed (for example, to retry).
| record | The record to insert. |
| InsertException | If the record cannot be inserted |
Definition at line 285 of file KineticaIngestor.cs.
◆ insert() [4/4]
| inline |
Queues a record for insertion into Kinetica.
If the queue reaches the configured batch size, all records in the queue will be inserted into Kinetica before the method returns. If an error occurs while inserting the records, the records will no longer be in the queue nor in Kinetica; catch InsertException to get the list of records that were being inserted if needed (for example, to retry).
| record | The record to insert. |
| InsertException | If the record cannot be inserted |
Definition at line 285 of file KineticaIngestor.cs.
◆ InsertException() [1/2]
| inline |
Initializes a new insert exception with the given message.
| msg | The error message. |
Definition at line 29 of file KineticaIngestor.cs.
◆ InsertException() [2/2]
| inline |
Initializes a new insert exception with the given message.
| msg | The error message. |
Definition at line 29 of file KineticaIngestor.cs.
◆ KineticaIngestor() [1/2]
| inline |
Creates a new ingestor for bulk insertion of records into the specified table.
This class is obsolete; use BulkInserter<T> instead for better performance, async support, and feature parity with the Rust API.
| kdb | Kinetica client connection. |
| tableName | Name of the target table to insert into. |
| batchSize | Number of records to queue per worker before flushing to the server. |
| ktype | Type definition of the records being inserted. |
| options | Optional ingest options; recognized keys are defined as constants on InsertRecordsRequest<T>.Options. |
| workers | Optional explicit list of worker URLs. If null, the worker list is fetched from the server. |
Definition at line 81 of file KineticaIngestor.cs.
◆ KineticaIngestor() [2/2]
| inline |
Creates a new ingestor for bulk insertion of records into the specified table.
This class is obsolete; use BulkInserter<T> instead for better performance, async support, and feature parity with the Rust API.
| kdb | Kinetica client connection. |
| tableName | Name of the target table to insert into. |
| batchSize | Number of records to queue per worker before flushing to the server. |
| ktype | Type definition of the records being inserted. |
| options | Optional ingest options; recognized keys are defined as constants on InsertRecordsRequest<T>.Options. |
| workers | Optional explicit list of worker URLs. If null, the worker list is fetched from the server. |
Definition at line 81 of file KineticaIngestor.cs.
◆ ToString() [1/2]
| inline |
Definition at line 38 of file KineticaIngestor.cs.
◆ ToString() [2/2]
| inline |
Definition at line 38 of file KineticaIngestor.cs.
Member Data Documentation
◆ count_inserted
| Int64 kinetica.KineticaIngestor< T >.count_inserted |
The running count of records inserted so far.
Definition at line 56 of file KineticaIngestor.cs.
◆ count_updated
| Int64 kinetica.KineticaIngestor< T >.count_updated |
The running count of records that updated existing records (on primary-key collision).
Definition at line 58 of file KineticaIngestor.cs.
Property Documentation
◆ batch_size
| get |
The number of records buffered before a batch is flushed to the server.
Definition at line 51 of file KineticaIngestor.cs.
◆ kineticaDB
| get |
The Kinetica client used to insert records.
Definition at line 47 of file KineticaIngestor.cs.
◆ options
| get |
The insert options applied to each batch.
Definition at line 53 of file KineticaIngestor.cs.
◆ records
| get |
The records that were not successfully inserted.
Definition at line 25 of file KineticaIngestor.cs.
◆ table_name
| get |
The name of the table records are inserted into.
Definition at line 49 of file KineticaIngestor.cs.
◆ url
| get |
The worker URL the failed batch was being sent to, if known.
Definition at line 23 of file KineticaIngestor.cs.
The documentation for this class was generated from the following files:
- _build/public-os_ubuntu24.04-arch_amd64-cc_gcc_13.3.0/install/Kinetica/KineticaIngestor.cs
- Kinetica/KineticaIngestor.cs