/upload/files

URL: https://<aws.fqdn>/<aws.cluster.name>/gpudb-0/upload/files

Uploads one or more files to KiFS. There are two methods for uploading files: load files in their entirety, or load files in parts. The latter is recommeded for files of approximately 60 MB or larger.

To upload files in their entirety, populate input parameter file_names with the file names to upload into on KiFS, and their respective byte content in input parameter file_data.

Multiple steps are involved when uploading in multiple parts. Only one file at a time can be uploaded in this manner. A user-provided UUID is utilized to tie all the upload steps together for a given file. To upload a file in multiple parts:

  1. Provide the file name in input parameter file_names, the UUID in the multipart_upload_uuid key in input parameter options, and a multipart_operation value of init.
  2. Upload one or more parts by providing the file name, the part data in input parameter file_data, the UUID, a multipart_operation value of upload_part, and the part number in the multipart_upload_part_number. The part numbers must start at 1 and increase incrementally. Parts may not be uploaded out of order.
  3. Complete the upload by providing the file name, the UUID, and a multipart_operation value of complete.

Multipart uploads in progress may be canceled by providing the file name, the UUID, and a multipart_operation value of cancel. If an new upload is initialized with a different UUID for an existing upload in progress, the pre-existing upload is automatically canceled in favor of the new upload.

The multipart upload must be completed for the file to be usable in KiFS. Information about multipart uploads in progress is available in /show/files.

File data may be pre-encoded using base64 encoding. This should be indicated using the file_encoding option, and is recommended when using JSON serialization.

Each file path must reside in a top-level KiFS directory, i.e. one of the directories listed in /show/directories. The user must have write permission on the directory. Nested directories are permitted in file name paths. Directories are deliniated with the directory separator of '/'. For example, given the file path '/a/b/c/d.txt', 'a' must be a KiFS directory.

These characters are allowed in file name paths: letters, numbers, spaces, the path delimiter of '/', and the characters: '.' '-' ':' '[' ']' '(' ')' '#' '='.

Input Parameter Description

Name Type Description
file_names array of strings An array of full file name paths to be used for the files uploaded to KiFS. File names may have any number of nested directories in their paths, but the top-level directory must be an existing KiFS directory. Each file must reside in or under a top-level directory. A full file name path cannot be larger than 1024 characters.
file_data array of bytes File data for the files being uploaded, for the respective files in input parameter file_names.
options map of string to strings

Optional parameters. The default value is an empty map ( {} ).

Supported Parameters (keys) Parameter Description
file_encoding

Encoding that has been applied to the uploaded file data. When using JSON serialization it is recommended to utilize base64. The caller is responsible for encoding the data provided in this payload The default value is none.

Supported Values Description
base64 Specifies that the file data being uploaded has been base64 encoded.
none The uploaded file data has not been encoded.
multipart_operation

Multipart upload operation to perform The default value is none.

Supported Values Description
none Default, indicates this is not a multipart upload
init Initialize a multipart file upload
upload_part Uploads a part of the specified multipart file upload
complete Complete the specified multipart file upload
cancel Cancel the specified multipart file upload
multipart_upload_uuid UUID to uniquely identify a multipart upload
multipart_upload_part_number Incremental part number for each part in a multipart upload. Part numbers start at 1, increment by 1, and must be uploaded sequentially
delete_if_exists

If true, any existing files specified in input parameter file_names will be deleted prior to start of upload. Otherwise the file is replaced once the upload completes. Rollback of the original file is no longer possible if the upload is cancelled, aborted or fails if the file was deleted beforehand. The default value is false. The supported values are:

  • true
  • false

Output Parameter Description

The GPUdb server embeds the endpoint response inside a standard response structure which contains status information and the actual response to the query. Here is a description of the various fields of the wrapper:

Name Type Description
status String 'OK' or 'ERROR'
message String Empty if success or an error message
data_type String 'upload_files_response' or 'none' in case of an error
data String Empty string
data_str JSON or String

This embedded JSON represents the result of the /upload/files endpoint:

Name Type Description
info map of string to strings Additional information.

Empty string in case of an error.