Create Type

Creates a new type describing the layout of a table. The type definition is a JSON string describing the fields (i.e. columns) of the type. Each field consists of a name and a data type. Supported data types are: double, float, int, long, string, and bytes. In addition, one or more properties can be specified for each column which customize the memory usage and query availability of that column. Note that some properties are mutually exclusive--i.e. they cannot be specified for any given column simultaneously. One example of mutually exclusive properties are data and store_only.

A single primary key and/or single shard key can be set across one or more columns. If a primary key is specified, then a uniqueness constraint is enforced, in that only a single object can exist with a given primary key column value (or set of values for the key columns, if using a composite primary key). When inserting data into a table with a primary key, depending on the parameters in the request, incoming objects with primary key values that match existing objects will either overwrite (i.e. update) the existing object or will be skipped and not added into the set.

Example of a type definition with some of the parameters:

{"type":"record",
"name":"point",
"fields":[{"name":"msg_id","type":"string"},
                {"name":"x","type":"double"},
                {"name":"y","type":"double"},
                {"name":"TIMESTAMP","type":"double"},
                {"name":"source","type":"string"},
                {"name":"group_id","type":"string"},
                {"name":"OBJECT_ID","type":"string"}]
}

Properties:

{"group_id":["store_only"],
"msg_id":["store_only","text_search"]
}

Input Parameter Description

NameTypeDescription
type_definitionstringa JSON string describing the columns of the type to be registered.
labelstringA user-defined description string which can be used to differentiate between tables and types with otherwise identical schemas.
propertiesmap of string to arrays of strings

Each key-value pair specifies the properties to use for a given column where the key is the column name. All keys used must be relevant column names for the given table. Specifying any property overrides the default properties for that column (which is based on the column's data type).

Allowed ValuesDescription
dataDefault property for all numeric and string type columns; makes the column available for GPU queries.
text_searchValid only for select 'string' columns. Enables full text search--see Full Text Search for details and applicable string column types. Can be set independently of data and store_only.
store_onlyPersist the column value but do not make it available to queries (e.g. Filter )-i.e. it is mutually exclusive to the data property. Any 'bytes' type column must have a store_only property. This property reduces system memory usage.
disk_optimizedWorks in conjunction with the data property for string columns. This property reduces system disk usage by disabling reverse string lookups. Queries like Filter , Filter by List , and Filter by Value work as usual but Aggregate Unique and Aggregate Group By are not allowed on columns with this property.
timestampValid only for 'long' columns. Indicates that this field represents a timestamp and will be provided in milliseconds since the Unix epoch: 00:00:00 Jan 1 1970. Dates represented by a timestamp must fall between the year 1000 and the year 2900.
ulongValid only for 'string' columns. It represents an unsigned long integer data type. The string can only be interpreted as an unsigned long data type with minimum value of zero, and maximum value of 18446744073709551615.
uuidValid only for 'string' columns. It represents an uuid data type. Internally, it is stored as a 128-bit integer.
decimalValid only for 'string' columns. It represents a SQL type NUMERIC(19, 4) data type. There can be up to 15 digits before the decimal point and up to four digits in the fractional part. The value can be positive or negative (indicated by a minus sign at the beginning). This property is mutually exclusive with the text_search property.
dateValid only for 'string' columns. Indicates that this field represents a date and will be provided in the format 'YYYY-MM-DD'. The allowable range is 1000-01-01 through 2900-01-01. This property is mutually exclusive with the text_search property.
timeValid only for 'string' columns. Indicates that this field represents a time-of-day and will be provided in the format 'HH:MM:SS.mmm'. The allowable range is 00:00:00.000 through 23:59:59.999. This property is mutually exclusive with the text_search property.
datetimeValid only for 'string' columns. Indicates that this field represents a datetime and will be provided in the format 'YYYY-MM-DD HH:MM:SS.mmm'. The allowable range is 1000-01-01 00:00:00.000 through 2900-01-01 23:59:59.999. This property is mutually exclusive with the text_search property.
char1This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 1 character.
char2This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 2 characters.
char4This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 4 characters.
char8This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 8 characters.
char16This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 16 characters.
char32This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 32 characters.
char64This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 64 characters.
char128This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 128 characters.
char256This property provides optimized memory, disk and query performance for string columns. Strings with this property must be no longer than 256 characters.
booleanThis property provides optimized memory and query performance for int columns. Ints with this property must be between 0 and 1(inclusive)
int8This property provides optimized memory and query performance for int columns. Ints with this property must be between -128 and +127 (inclusive)
int16This property provides optimized memory and query performance for int columns. Ints with this property must be between -32768 and +32767 (inclusive)
ipv4This property provides optimized memory, disk and query performance for string columns representing IPv4 addresses (i.e. 192.168.1.1). Strings with this property must be of the form: A.B.C.D where A, B, C and D are in the range of 0-255.
arrayValid only for 'string' columns. Indicates that this field contains an array. The value type and (optionally) the item count should be specified in parenthesis; e.g., 'array(int, 10)' for a 10-integer array. Both 'array(int)' and 'array(int, -1)' will designate an unlimited-length integer array, though no bounds checking is performed on arrays of any length.
jsonValid only for 'string' columns. Indicates that this field contains values in JSON format.
vectorValid only for 'bytes' columns. Indicates that this field contains a vector of floats. The length should be specified in parenthesis, e.g., 'vector(1000)'.
wktValid only for 'string' and 'bytes' columns. Indicates that this field contains geospatial geometry objects in Well-Known Text (WKT) or Well-Known Binary (WKB) format.
primary_keyThis property indicates that this column will be part of (or the entire) primary key.
shard_keyThis property indicates that this column will be part of (or the entire) shard key.
nullableThis property indicates that this column is nullable. However, setting this property is insufficient for making the column nullable. The user must declare the type of the column as a union between its regular type and 'null' in the avro schema for the record type in input parameter type_definition. For example, if a column is of type integer and is nullable, then the entry for the column in the avro schema must be: ['int', 'null']. The C++, C#, Java, and Python APIs have built-in convenience for bypassing setting the avro schema by hand. For those languages, one can use this property as usual and not have to worry about the avro schema for the record.
dictThis property indicates that this column should be dictionary encoded. It can only be used in conjunction with restricted string (charN), int, long or date columns. Dictionary encoding is best for columns where the cardinality (the number of unique values) is expected to be low. This property can save a large amount of memory.
init_with_nowFor 'date', 'time', 'datetime', or 'timestamp' column types, replace empty strings and invalid timestamps with 'NOW()' upon insert.
init_with_uuidFor 'uuid' type, replace empty strings and invalid UUID values with randomly-generated UUIDs upon insert.
optionsmap of string to stringsOptional parameters. The default value is an empty map ( {} ).

Output Parameter Description

NameTypeDescription
type_idstringAn identifier representing the created type. This type_id can be used in subsequent calls to create a table
type_definitionstringValue of input parameter type_definition.
labelstringValue of input parameter label.
propertiesmap of string to arrays of stringsValue of input parameter properties.
infomap of string to stringsAdditional information.