The Kinetica File System (KiFS) is a file system interface that's packaged with Kinetica. It provides a repository for users without direct access to the database cluster's file system to store and make use of files within the database.
KiFS can be leveraged by several Kinetica features:
KiFS files can be referenced with the following URI:
kifs://<kifs directory>/<kifs file>
For example, the following URI can be broken down into three components:
The unique KiFS file name, when referenced in the API, is the composite of the directory and file:
KiFS can be configured to use any of the following for file storage:
- Local shared storage, mounted and accessible to every node in the Kinetica cluster
- Azure (Microsoft blob storage)
- HDFS (Apache Hadoop Distributed File System)
- S3 (Amazon S3 Bucket)
To enable KiFS, update the
configuration file in the KiFS section with
one of the following setups, and then restart the database.
Remote storage configuration parameters mirror those used for defining cold storage tiers. See Cold Storage Tier in the Configuration Reference for the full set of parameters.
KiFS can be managed using the following API endpoint calls.
|/create/directory||Creates a directory, a container for files|
|/delete/directory||Removes the directory; can optionally remove all contained files|
|/show/directories||Outputs the properties of one or more specified directories, or optionally, all directories|
|/grant/permission/directory||Grants the permission for a user to access files within a directory|
|/revoke/permission/directory||Revokes the permission for a user to access files within a directory|
|/delete/files||Removes one or more files|
|/download/files||Downloads one or more files|
|/show/files||Outputs the properties of one or more files|
|/upload/files||Uploads one or more files to a directory|
The Kinetica Java API provides a streamlined interface for managing files & directories in KiFS. When using KiFS via the Java API, the handler class must be initialized. The examples in the following sections make use of this interface and assume these steps have been taken.
In KiFS, a directory is a top-level container for files. A directory cannot be nested within another directory, though a file contained within a directory may have a path that gives the appearance of being contained within one or more nested virtual directories. A directory must exist before files can be uploaded into it.
To create a KiFS directory, data:
To show the stats of a KiFS directory, data:
To show the stats of all KiFS directories:
To delete an empty, existing KiFS directory, use the following parameters:
- data - name of the directory
- false - don't delete files contained within the directory, and return an error if any are found
- false - return an error if the directory is not found
To delete a KiFS directory, data, and all files contained within, regardless of whether the directory exists (similar to a Unix rm -rf):
- data - name of the directory
- true - delete any files contained within the directory
- true - suppress the error in the case that the directory is not found
In KiFS, a file is a user-uploaded text or binary object. Each file is located within a directory and must have a unique name within that directory. The name can contain forward slashes to create the appearance of a hierarchical virtual directory structure and help namespace the files. Overall, each file is referenced by the composite of the directory and file name.
To upload a file to a KiFS directory, data, under a virtual directory, geo:
To upload a set of files to a KiFS directory, data, under a virtual directory, geo:
To list the KiFS files in a directory, data:
To download a KiFS file, placing it directly under a local directory, download:
To download a set of KiFS files, placing them all directly under a local directory, download:
To delete a set of KiFS files: