Files & Directories (KiFS)

Kinetica, via KiFS, provides support for staging files within the database for subsequent ingestion. The file structure consists of a single layer of top-level directories, with each file contained within one of those directories. File names can contain / characters to give the appearance of existing under a hierarchy of one or more sub-directories. Directory features accessible via SQL include:

File features accessible via SQL include:

For directory permission management, see:

CREATE DIRECTORY

Creates a new KiFS directory under which files can be uploaded and used for data loading.

CREATE DIRECTORY Syntax

CREATE DIRECTORY '<directory name>'
[WITH OPTIONS (data_limit = <data limit value>)]

Parameters

Name of the directory to create.

WITH OPTIONS

Optional indicator that a comma-delimited list of option/value assignments will follow. The following options are available:

Option	Description
`data_limit`	Total capacity the given directory should not exceed, in bytes; use `-1` for no limit

Examples

To create a directory, kdata, with no size limit:

CREATE DIRECTORY Example

CREATE DIRECTORY 'kdata'

To create a directory, fs_limited, with a 1,000,000 byte maximum capacity:

CREATE DIRECTORY with Limit Example

CREATE DIRECTORY 'fs_limited'
WITH OPTIONS (data_limit = 1000000)

ALTER DIRECTORY

Alters the configuration of an existing KiFS directory. The following facet of a directory can be altered:

Data Limit

Set Data Limit

A directory can have its data limit modified.

Set Directory Data Limit Syntax

ALTER DIRECTORY '<directory name>'
SET data_limit = <data limit value>

Parameters

Name of the directory to alter

Total capacity the given directory should not exceed, in bytes; use -1 for no limit

Examples

To set the maximum capacity of a directory, fs_limited, to 10,000,000 bytes:

ALTER DIRECTORY SET Data Limit Example

ALTER DIRECTORY 'fs_limited'
SET data_limit = 10000000

LIST DIRECTORY

Outputs detail about one or all KiFS directories.

LIST DIRECTORY Syntax

<LIST|SHOW|DESC[RIBE]> DIRECTOR[Y|IES] < '<directory name>' | * >

Parameters

Name of the existing directory for which detail will be output. Use * instead to output detail of all directories.

Response

The response to LIST DIRECTORY is a six-column result set:

Output Column	Description
`DIRECTORY_NAME`	Name of the directory
`CREATED_BY`	User ID of the user who created the directory
`CREATION_TIME`	Date/time at which the directory was created
`DATA_USAGE`	Amount of used file storage within the directory, in bytes
`DATA_LIMIT`	Total amount of file storage within the directory, in bytes
`PERMISSION`	Highest level of access to the directory the command issuer has

Examples

To output the detail about a directory, kdata:

LIST DIRECTORY Example

LIST DIRECTORY 'kdata'

To output the detail for all directories:

LIST DIRECTORIES (All Directories) Example

LIST DIRECTORIES *

DOWNLOAD DIRECTORY

Downloads all files under a directory in KiFS into a local directory.

This command is only available through KiSQL or database clients configured with the Kinetica JDBC driver.

DOWNLOAD DIRECTORY Syntax

DOWNLOAD [DIRECTORY] <kifs directory>
INTO '<local directory>'

Parameters

Examples

To download all files in a KiFS directory to a local directory, /tmp:

DOWNLOAD DIRECTORY Example

DOWNLOAD DIRECTORY 'kdata'
INTO '/tmp'

DROP DIRECTORY

Removes an existing KiFS directory.

DROP DIRECTORY Syntax

DROP DIRECTORY [IF EXISTS] '<directory name>'
[WITH OPTIONS ('<option name>' = '<option value>'[,...])]

Parameters

Name of the directory to remove.

IF EXISTS

Optional error-suppression clause; if specified, no error will be returned if the given directory does not exist.

WITH OPTIONS

Optional indicator that a comma-delimited list of option/value assignments will follow.

Option	Description
`recursive`	Whether to delete all files contained within the directory: `false` (default) - only drop the directory if empty; return an error if the directory contains files `true` - drop the directory and all files in it

Examples

To delete a directory, kbackup:

DROP DIRECTORY Example

DROP DIRECTORY 'kbackup'

To delete a directory, kdata, and all files in it, suppressing the error if it doesn’t exist:

DROP DIRECTORY with Files Example

DROP DIRECTORY IF EXISTS 'kdata'
WITH OPTIONS ('recursive' = 'true')

UPLOAD FILE

Uploads file(s) local to the database client into KiFS, within the given directory.

This command is only available through KiSQL or database clients configured with the Kinetica JDBC driver.

UPLOAD FILE Syntax

UPLOAD FILE[S] <file path(s)>
INTO '<kifs path>'

Parameters

Local path(s) of the file(s) to upload to KiFS, as a comma-separated list of single-quoted file paths; wildcards (*) can be used to specify a group of files.

File Path	Outcome
`’archive/*.csv’`	Upload all files under the archive directory with a csv extension
`’data/q1.csv’,‘data/q2.csv’`	Upload q1.csv & q2.csv that reside under the data directory

Path in KiFS; this can be either the name of a directory or a directory followed by virtual subdirectories separated by / characters.For example, uploading a file named 2021.q1.csv to:

'data' - will upload it to the data directory and make it available as data/2021.q1.csv
'data/sales/Q1' - will upload it to the data directory and make it available as data/sales/Q1/2021.q1.csv

Only the directory specified within this path must exist for the upload to succeed. Any virtual subdirectories added to this path do not need to exist before uploading.

Examples

To upload a file to a directory, kdata:

UPLOAD FILE Example

UPLOAD FILE '../data/products.csv'
INTO 'kdata'

To upload several files to a directory, kdata:

UPLOAD FILES Example

UPLOAD FILES '../data/p*.ssv', '../data/employee.*'
INTO 'kdata'

UPLOAD URL

Uploads file(s) from one or more URLs into KiFS, in one or more directories. There should be a corresponding KiFS full file path specified for each URL specified—the KiFS file name will not be derived from any part of the given URL.

UPLOAD URL Syntax

UPLOAD URL <url(s)>
INTO <kifs path(s)>

Parameters

<url(s)>

URL(s) of the file(s) to upload to KiFS, as a comma-separated list of single-quoted URL(s).

File path(s) in KiFS to upload the file(s) at the URL(s) to, as a comma-separated list of single-quoted KiFS full file paths.

Only the directory specified within this path must exist for the upload to succeed. Any virtual subdirectories within this path do not need to exist before uploading.

Examples

To upload the Kinetica JDBC Driver in GitHub to file kinetica-jdbc-fullshaded.jar under directory drivers:

UPLOAD URL Example

UPLOAD URL 'https://github.com/kineticadb/kinetica-client-jdbc/raw/master/kinetica-jdbc-7.2.3.14-fullshaded.jar'
INTO 'drivers/kinetica-jdbc-fullshaded.jar';

To upload the Kinetica JDBC Driver and the corresponding change log in GitHub to files kjdbc-7.2.jar & kjdbc-CHANGELOG.md, respectively, under directory drivers:

UPLOAD URLs Example

UPLOAD URL
	'https://github.com/kineticadb/kinetica-client-jdbc/raw/master/kinetica-jdbc-7.2.3.14-fullshaded.jar',
	'https://github.com/kineticadb/kinetica-client-jdbc/raw/master/CHANGELOG.md'
INTO
	'drivers/kjdbc-7.2.jar',
	'drivers/kjdbc-CHANGELOG.md';

LIST FILE

Outputs detail about one or all KiFS files.

SHOW FILE Syntax

<LIST|SHOW|DESC[RIBE]> FILE[S] <kifs path(s)>

Parameters

Name(s) of existing directories and/or files for which detail will be output, as a comma-separated list of single-quoted paths.

Path	Outcome
`’archive’`	Output detail of all files under the KiFS directory archive.
`‘data/q1.csv’,‘data/q2.csv’`	Output detail of q1.csv & q2.csv that reside under the data directory.

Response

The response to LIST FILE is a four-column result set:

Output Column	Description
`FILE_NAME`	Name of the file
`SIZE`	Size of the file in bytes
`CREATED_BY`	User ID of the user who created the file
`CREATION_TIME`	Date/time at which the file was created

Examples

To output a listing of all files under the KiFS directory, kdata:

LIST FILES (Directory) Example

LIST FILES 'kdata'

To output a listing of specific files under the KiFS directory, kdata:

LIST FILES (Specific Files) Example

LIST FILES 'kdata/products.csv', 'kdata/employee.parquet'

DOWNLOAD FILE

Downloads file(s) from KiFS into a local directory.

This command is only available through KiSQL or database clients configured with the Kinetica JDBC driver.

DOWNLOAD FILE Syntax

DOWNLOAD [FILE[S]] <kifs file path(s)>
INTO '<local directory>'

Parameters

FILE[S]

Optional keyword for clarity.

KiFS file(s) to download to the local file system, as a comma-separated list of single-quoted file paths.

File Path	Outcome
`’archive/products.csv’`	Download archive/products.csv
`’data/q1.csv’,‘data/q2.csv’`	Download q1.csv & q2.csv that reside under the data directory

Local directory path into which files will be downloaded.

Examples

To download a KiFS file to a local directory, /tmp:

DOWNLOAD FILE Example

DOWNLOAD FILE 'kdata/products.csv'
INTO '/tmp'

To download several KiFS files to a local directory, /tmp:

DOWNLOAD FILES Example

DOWNLOAD FILES 'kdata/products.ssv', 'kdata/employee.parquet'
INTO '/tmp'

DROP FILE

Deletes an existing KiFS file.

DROP FILE Syntax

DROP FILE[S] [IF EXIST[S]] <kifs file path(s)>

Parameters

KiFS file(s) to delete, as a comma-separated list of single-quoted file paths.

IF EXIST[S]

Optional error-suppression clause; if specified, no error will be returned if the given file(s) do not exist.

Examples

To delete a KiFS file:

DROP FILE Example

DROP FILE 'kdata/products.csv'

To delete several KiFS files, suppressing the error if any don’t exist:

DROP FILES Example

DROP FILES IF EXIST 'kdata/products.csv', 'kdata/employee.parquet'

​CREATE DIRECTORY

​Parameters

​Examples

​ALTER DIRECTORY

​Set Data Limit

​Parameters

​Examples

​LIST DIRECTORY

​Parameters

​Response

​Examples

​DOWNLOAD DIRECTORY

​Parameters

​Examples

​DROP DIRECTORY

​Parameters

​Examples

​UPLOAD FILE

​Parameters

​Examples

​UPLOAD URL

​Parameters

​Examples

​LIST FILE

​Parameters

​Response

​Examples

​DOWNLOAD FILE

​Parameters

​Examples

​DROP FILE

​Parameters

​Examples

CREATE DIRECTORY

Parameters

Examples

ALTER DIRECTORY

Set Data Limit

Parameters

Examples

LIST DIRECTORY

Parameters

Response

Examples

DOWNLOAD DIRECTORY

Parameters

Examples

DROP DIRECTORY

Parameters

Examples

UPLOAD FILE

Parameters

Examples

UPLOAD URL

Parameters

Examples

LIST FILE

Parameters

Response

Examples

DOWNLOAD FILE

Parameters

Examples

DROP FILE

Parameters

Examples