Class BulkInserter\

java.lang.Object

com.gpudb.BulkInserter\

Type Parameters:

T - the type of object being inserted

All Implemented Interfaces:

AutoCloseable

public class BulkInserter\ extends Object implements AutoCloseable

Object that manages the insertion into GPUdb of large numbers of records in bulk, with automatic batch management and support for multi-head ingest. BulkInserter instances are thread safe and may be used from any number of threads simultaneously. Use the insert(Object) and insert(List) methods to queue records for insertion, and the flush() method to ensure that all queued records have been inserted.

Nested Class Summary

Nested Classes

Modifier and Type

Class

Description

static final class

BulkInserter.FlushOptions

This class facilitates customizing the behavior of automatic flush in BulkInserter The default value of the 'flushInterval' is set to a negative value to indicate that the automatic flush feature is not needed.

static final class

BulkInserter.InsertException

An exception that occurred during the insertion of records into GPUdb.

static final class

BulkInserter.WorkerList

Deprecated, for removal: This API element is subject to removal in a future version.
This class has been superseded by com.gpudb.WorkerList.
Constructor Summary

Constructors

Constructor

Description

BulkInserter(GPUdb gpudb, String tableName, int batchSize)

Creates a JSON BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options)

Creates a JSON BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers)

Creates a JSON BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions)

Creates a JSON BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions, GPUdbBase.JsonOptions jsonOptions)

Creates a JSON BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers, GPUdbBase.JsonOptions jsonOptions)

Creates a JSON BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions, GPUdbBase.JsonOptions jsonOptions)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers, GPUdbBase.JsonOptions jsonOptions)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions, GPUdbBase.JsonOptions jsonOptions)

Creates a BulkInserter with the specified parameters.

BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers, GPUdbBase.JsonOptions jsonOptions)

Creates a BulkInserter with the specified parameters.
Method Summary

Modifier and Type

Method

Description

void

close()

Closes BulkInserter resources: Performs a final flush of any data yet to be ingested Terminates the timed flush mechanism, if needed Terminates the ingest executor service
This method will be called automatically if the BulkInserter class is used in a try-with-resources block.

void

flush()

Ensures that any queued records are inserted into GPUdb.

int

getBatchSize()

Gets the batch size (the number of records to insert into GPUdb at a time).

long

getCountInserted()

Gets the number of records inserted into GPUdb.

long

getCountUpdated()

Gets the number of records updated (instead of inserted) in GPUdb due to primary key conflicts.

List\<BulkInserter.InsertException>

getErrors()

Gets the list of errors received since the last call to getErrors().

GPUdb

getGPUdb()

Gets the GPUdb instance into which records will be inserted.

GPUdbBase.JsonOptions

getJsonOptions()

Retrieves options governing JSON ingest.

int

getMaxRetries()

Gets the number of times inserts into GPUdb will be retried in the event of an error.

Map\<String,String>

getOptions()

Gets the optional parameters that will be passed to GPUdb while inserting.

int

getRetryCount()

Deprecated, for removal: This API element is subject to removal in a future version.

String

getTableName()

Gets the name of the table into which records will be inserted.

BulkInserter.FlushOptions

getTimedFlushOptions()

Retrieves options governing the timed flush ingest scheme.

List\<BulkInserter.InsertException>

getWarnings()

Gets the list of warnings received since the last call to getWarnings().

void

insert(List\<T> records)

Queues a list of records for insertion into GPUdb.

void

insert(T record)

Queues a record for insertion into GPUdb.

boolean

isMultiHeadEnabled()

void

setJsonOptions(GPUdbBase.JsonOptions jsonOptions)

This method could be used to set GPUdbBase.JsonOptions in case the user wants to modify the defaults.

void

setMaxRetries(int value)

Sets the number of times inserts into GPUdb will be retried in the event of an error.

void

setRetryCount(int value)

Deprecated, for removal: This API element is subject to removal in a future version.

void

setTimedFlushOptions(BulkInserter.FlushOptions flushOptions)

This method could potentially result in two different scenarios It could start a timed flush thread if it was not already active when the BulkInserter was created.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, int batchSize) throws GPUdbException
  
  Creates a JSON BulkInserter with the specified parameters.
  This constructor will attempt to automatically configure multi-head ingest, if available.
  It will also use default settings for the GPUdb.insertRecords(String, List, Map) call. Details can be found at InsertRecordsRequest.Options.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options) throws GPUdbException
  
  Creates a JSON BulkInserter with the specified parameters.
  This constructor will attempt to automatically configure multi-head ingest, if available.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers) throws GPUdbException
  
  Creates a JSON BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions) throws GPUdbException
  
  Creates a JSON BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  flushOptions - BulkInserter.FlushOptions to use for timed flush operation
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers, GPUdbBase.JsonOptions jsonOptions) throws GPUdbException
  
  Creates a JSON BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  jsonOptions - GPUdbBase.JsonOptions to use for JSON ingest
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
  
  GPUdbBase.JsonOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions, GPUdbBase.JsonOptions jsonOptions) throws GPUdbException
  
  Creates a JSON BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  flushOptions - BulkInserter.FlushOptions to use for timed flush operation
  
  jsonOptions - GPUdbBase.JsonOptions to use for JSON ingest
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
  
  GPUdbBase.JsonOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  This constructor will attempt to automatically configure multi-head ingest, if available.
  It will also use default settings for the GPUdb.insertRecords(String, List, Map) call. Details can be found at InsertRecordsRequest.Options.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  type - the Type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  This constructor will attempt to automatically configure multi-head ingest, if available.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  type - the Type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  type - the Type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  type - the Type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  flushOptions - BulkInserter.FlushOptions to use for timed flush operation
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers, GPUdbBase.JsonOptions jsonOptions) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  type - the Type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  jsonOptions - GPUdbBase.JsonOptions to use for JSON ingest
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
  
  GPUdbBase.JsonOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, Type type, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions, GPUdbBase.JsonOptions jsonOptions) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  type - the Type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  flushOptions - BulkInserter.FlushOptions to use for timed flush operation
  
  jsonOptions - GPUdbBase.JsonOptions to use for JSON ingest
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
  
  GPUdbBase.JsonOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  This constructor will attempt to automatically configure multi-head ingest, if available.
  It will also use default settings for the GPUdb.insertRecords(String, List, Map) call. Details can be found at InsertRecordsRequest.Options.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  typeObjectMap - the TypeObjectMap for the type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  This constructor will attempt to automatically configure multi-head ingest, if available.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  typeObjectMap - the TypeObjectMap for the type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  typeObjectMap - the TypeObjectMap for the type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  typeObjectMap - the TypeObjectMap for the type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  flushOptions - BulkInserter.FlushOptions to use for timed flush operation
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
  
  GPUdbBase.JsonOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers, GPUdbBase.JsonOptions jsonOptions) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  typeObjectMap - the TypeObjectMap for the type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  jsonOptions - GPUdbBase.JsonOptions to use for JSON ingest
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
  
  GPUdbBase.JsonOptions
- BulkInserter
  
  public BulkInserter(GPUdb gpudb, String tableName, TypeObjectMap\<T> typeObjectMap, int batchSize, Map\<String,String> options, WorkerList workers, BulkInserter.FlushOptions flushOptions, GPUdbBase.JsonOptions jsonOptions) throws GPUdbException
  
  Creates a BulkInserter with the specified parameters.
  Parameters:
  
  gpudb - the GPUdb instance to insert records into
  
  tableName - name of the table to insert records into
  
  typeObjectMap - the TypeObjectMap for the type of records being inserted
  
  batchSize - the number of records to insert into GPUdb at a time (records will queue until this number is reached); for multi-head ingest, this value is per worker
  
  options - optional parameters to pass to GPUdb while inserting (null for no parameters)
  This is the same set of options as accepted by the GPUdb.insertRecords(String, List, Map) call.
  The details can be found at InsertRecordsRequest.Options.
  
  workers - worker list for multi-head ingest; use an empty worker list (new WorkerList()) to disable multi-head
  
  flushOptions - BulkInserter.FlushOptions to use for timed flush operation
  
  jsonOptions - GPUdbBase.JsonOptions to use for JSON ingest
  
  Throws:
  
  GPUdbException - if a configuration error occurs
  
  IllegalArgumentException - if an invalid parameter is specified
  
  See Also:
  
  InsertRecordsRequest.Options
  
  BulkInserter.FlushOptions
  
  GPUdbBase.JsonOptions
Method Details
- getTimedFlushOptions
  
  public BulkInserter.FlushOptions getTimedFlushOptions()
  
  Retrieves options governing the timed flush ingest scheme.
  
  Returns:
  
  the timed flush options as a BulkInserter.FlushOptions instance
- getJsonOptions
  
  public GPUdbBase.JsonOptions getJsonOptions()
  
  Retrieves options governing JSON ingest.
  
  Returns:
  
  the JSON ingest options as a GPUdbBase.JsonOptions instance
- setTimedFlushOptions
  
  public void setTimedFlushOptions(BulkInserter.FlushOptions flushOptions) throws GPUdbException
  This method could potentially result in two different scenarios
  
  It could start a timed flush thread if it was not already active when the BulkInserter was created.
  
  If the timed flush thread was already active then setting this to a new value will first terminate the existing thread; set the options to the new value and finally restart a new thread with the options passed in. This case could result in a delay since the thread needs to be cleaned up and restarted.
  Parameters:
  
  flushOptions - BulkInserter.FlushOptions to use for timed flush operation
  
  Throws:
  
  GPUdbException - in case an invalid timeout values is set in the flushOptions parameter.
- setJsonOptions
  
  public void setJsonOptions(GPUdbBase.JsonOptions jsonOptions)
  
  This method could be used to set GPUdbBase.JsonOptions in case the user wants to modify the defaults.
  Parameters:
  
  jsonOptions - GPUdbBase.JsonOptions to use for JSON ingest
  
  See Also:
  
  GPUdbBase.JsonOptions
- close
  
  public void close() throws BulkInserter.InsertException
  Closes BulkInserter resources:
  
  Performs a final flush of any data yet to be ingested
  
  Terminates the timed flush mechanism, if needed
  
  Terminates the ingest executor service
  
  This method will be called automatically if the BulkInserter class is used in a try-with-resources block. If not used that way it is mandatory to call this method to initiate a smooth cleanup of the underlying resources.
       try( BulkInserter inserter = new BulkInserter(...) ) {"{"}
           // Do something with the BulkInserter instance
           // inserter.{"{"}some\_method{"}"}
       {"}"}
       // Here the close method of the BulkInserter class will be called
       // automatically
  
  or
       BulkInserter inserter = new BulkInserter\<>(...)
       // Invoke some methods on the inserter
       //Explicitly call close() method
       inserter.close();
  Specified by:
  
  close in interface AutoCloseable
  
  Throws:
  
  BulkInserter.InsertException - - While doing the final flush
- getGPUdb
  
  public GPUdb getGPUdb()
  
  Gets the GPUdb instance into which records will be inserted.
  
  Returns:
  
  the GPUdb instance into which records will be inserted
- getTableName
  
  public String getTableName()
  
  Gets the name of the table into which records will be inserted.
  
  Returns:
  
  the name of the table into which records will be inserted
- getBatchSize
  
  public int getBatchSize()
  
  Gets the batch size (the number of records to insert into GPUdb at a time). For multi-head ingest this value is per worker.
  
  Returns:
  
  the batch size
- getOptions
  
  public Map\<String,String> getOptions()
  
  Gets the optional parameters that will be passed to GPUdb while inserting.
  Returns:
  
  the optional parameters that will be passed to GPUdb while inserting
  
  See Also:
  
  InsertRecordsRequest.Options
- isMultiHeadEnabled
  
  public boolean isMultiHeadEnabled()
- getMaxRetries
  
  public int getMaxRetries()
  
  Gets the number of times inserts into GPUdb will be retried in the event of an error. After this many retries, BulkInserter.InsertException will be thrown.
  Returns:
  
  the number of retries
  
  See Also:
  
  setMaxRetries(int)
- setMaxRetries
  
  public void setMaxRetries(int value)
  
  Sets the number of times inserts into GPUdb will be retried in the event of an error. After this many retries, BulkInserter.InsertException will be thrown.
  Parameters:
  
  value - the number of retries
  
  Throws:
  
  IllegalArgumentException - if value is less than zero
  
  See Also:
  
  getMaxRetries()
- getRetryCount
  
  @Deprecated(since="7.1.10", forRemoval=true) public int getRetryCount()
  
  Deprecated, for removal: This API element is subject to removal in a future version.
  
  Gets the number of times inserts into GPUdb will be retried in the event of an error. After this many retries, BulkInserter.InsertException will be thrown.
  Returns:
  
  the number of retries
  
  See Also:
  
  setRetryCount(int)
- setRetryCount
  
  @Deprecated(since="7.1.10", forRemoval=true) public void setRetryCount(int value)
  
  Deprecated, for removal: This API element is subject to removal in a future version.
  
  Sets the number of times inserts into GPUdb will be retried in the event of an error. After this many retries, BulkInserter.InsertException will be thrown.
  Parameters:
  
  value - the number of retries
  
  Throws:
  
  IllegalArgumentException - if value is less than zero
  
  See Also:
  
  getRetryCount()
- getCountInserted
  
  public long getCountInserted()
  
  Gets the number of records inserted into GPUdb. Excludes records that are currently queued but not yet inserted and records not inserted due to primary key conflicts.
  
  Returns:
  
  the number of records inserted
- getCountUpdated
  
  public long getCountUpdated()
  
  Gets the number of records updated (instead of inserted) in GPUdb due to primary key conflicts.
  
  Returns:
  
  the number of records updated
- getErrors
  
  public List\<BulkInserter.InsertException> getErrors()
  
  Gets the list of errors received since the last call to getErrors().
  
  Returns:
  
  list of InsertException objects
- getWarnings
  
  public List\<BulkInserter.InsertException> getWarnings()
  
  Gets the list of warnings received since the last call to getWarnings().
  
  Returns:
  
  list of InsertException objects
- flush
  
  public void flush() throws BulkInserter.InsertException
  
  Ensures that any queued records are inserted into GPUdb. If an error occurs while inserting the records from any queue, the records will no longer be in that queue nor in GPUdb; catch BulkInserter.InsertException to get the list of records that were being inserted if needed (for example, to retry). Other queues may also still contain unflushed records if this occurs.
  
  Throws:
  
  BulkInserter.InsertException - if an error occurs while inserting
- insert
  
  public void insert(T record) throws BulkInserter.InsertException
  
  Queues a record for insertion into GPUdb. If the queue reaches the batch size, all records in the queue will be inserted into GPUdb before the method returns. If an error occurs while inserting the records, the records will no longer be in the queue nor in GPUdb; catch BulkInserter.InsertException to get the list of records that were being inserted if needed (for example, to retry). Note: This version of insert(Object) will result in sequential batch inserts in a background thread. Use insert(List) to allow multiple queues to reach their batch size and parallelize all of those batch inserts at once.
  
  Parameters:
  
  record - the record to insert
  
  Throws:
  
  GPUdbException - if an error occurs while calculating shard/primary keys
  
  BulkInserter.InsertException - if an error occurs while inserting
- insert
  
  public void insert(List\<T> records) throws BulkInserter.InsertException
  
  Queues a list of records for insertion into GPUdb. If any queue reaches the batch size, all queues that have reached the batch size will have their records inserted into the database before the method returns. If an error occurs while inserting the records, they will no longer be in that queue or the database; catch BulkInserter.InsertException to get the list of records that were being inserted (including any from the queue in question and any remaining in the list not yet queued) if needed (for example, to retry). Note that depending on the number of records, multiple calls to the database may occur. Note: This version of insert(List) will result in parallelizing the batch inserts in background threads.
  
  Parameters:
  
  records - the records to insert
  
  Throws:
  
  BulkInserter.InsertException - if an error occurs while inserting

Class BulkInserter\

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Details

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

BulkInserter

Method Details

getTimedFlushOptions

getJsonOptions

setTimedFlushOptions

setJsonOptions

close

getGPUdb

getTableName

getBatchSize

getOptions

isMultiHeadEnabled

getMaxRetries

setMaxRetries

getRetryCount

setRetryCount

getCountInserted

getCountUpdated

getErrors

getWarnings

flush

insert

insert