Skip to main content
Package com.gpudb.protocol

​
Class UploadFilesRequest

java.lang.Object
com.gpudb.protocol.UploadFilesRequest
All Implemented Interfaces:
org.apache.avro.generic.GenericContainer, org.apache.avro.generic.IndexedRecord
public class UploadFilesRequest extends Object implements org.apache.avro.generic.IndexedRecord
A set of parameters for GPUdb.uploadFiles.

Uploads one or more files to KiFS. There are two methods for uploading files: load files in their entirety, or load files in parts. The latter is recommended for files of approximately 60 MB or larger.

To upload files in their entirety, populate fileNames with the file names to upload into on KiFS, and their respective byte content in fileData.

Multiple steps are involved when uploading in multiple parts. Only one file at a time can be uploaded in this manner. A user-provided UUID is utilized to tie all the upload steps together for a given file. To upload a file in multiple parts:

1. Provide the file name in fileNames, the UUID in the MULTIPART_UPLOAD_UUID key in options, and a MULTIPART_OPERATION value of INIT.

2. Upload one or more parts by providing the file name, the part data in fileData, the UUID, a MULTIPART_OPERATION value of UPLOAD_PART, and the part number in the MULTIPART_UPLOAD_PART_NUMBER. The part numbers must start at 1 and increase incrementally. Parts may not be uploaded out of order.

3. Complete the upload by providing the file name, the UUID, and a MULTIPART_OPERATION value of COMPLETE.

Multipart uploads in progress may be canceled by providing the file name, the UUID, and a MULTIPART_OPERATION value of CANCEL. If an new upload is initialized with a different UUID for an existing upload in progress, the pre-existing upload is automatically canceled in favor of the new upload.

The multipart upload must be completed for the file to be usable in KiFS. Information about multipart uploads in progress is available in GPUdb.showFiles.

File data may be pre-encoded using base64 encoding. This should be indicated using the FILE_ENCODING option, and is recommended when using JSON serialization.

Each file path must reside in a top-level KiFS directory, i.e. one of the directories listed in GPUdb.showDirectories. The user must have write permission on the directory. Nested directories are permitted in file name paths. Directories are delineated with the directory separator of ’/’. For example, given the file path ‘/a/b/c/d.txt’, ‘a’ must be a KiFS directory.

These characters are allowed in file name paths: letters, numbers, spaces, the path delimiter of ’/’, and the characters: ’.’ ’-’ ’:’ ’[’ ’]’ ’(’ ’)’ ’#’ ’=’.

  • ​
    Constructor Details

    • ​
      UploadFilesRequest

      public UploadFilesRequest()
      Constructs an UploadFilesRequest object with default parameters.

    • ​
      UploadFilesRequest

      public UploadFilesRequest(List<String> fileNames, List<ByteBuffer> fileData, Map<String,String> options)
      Constructs an UploadFilesRequest object with the specified parameters.
      Parameters:
      fileNames - An array of full file name paths to be used for the files uploaded to KiFS. File names may have any number of nested directories in their paths, but the top-level directory must be an existing KiFS directory. Each file must reside in or under a top-level directory. A full file name path cannot be larger than 1024 characters.
      fileData - File data for the files being uploaded, for the respective files in fileNames.
      options - Optional parameters.
      • FILE_ENCODING: Encoding that has been applied to the uploaded file data. When using JSON serialization it is recommended to utilize BASE64. The caller is responsible for encoding the data provided in this payload. Supported values:
        • BASE64: Specifies that the file data being uploaded has been base64 encoded.
        • NONE: The uploaded file data has not been encoded.
        The default value is NONE.
      • MULTIPART_OPERATION: Multipart upload operation to perform. Supported values:
        • NONE: Default, indicates this is not a multipart upload.
        • INIT: Initialize a multipart file upload.
        • UPLOAD_PART: Uploads a part of the specified multipart file upload.
        • COMPLETE: Complete the specified multipart file upload.
        • CANCEL: Cancel the specified multipart file upload.
        The default value is NONE.
      • MULTIPART_UPLOAD_UUID: UUID to uniquely identify a multipart upload.
      • MULTIPART_UPLOAD_PART_NUMBER: Incremental part number for each part in a multipart upload. Part numbers start at 1, increment by 1, and must be uploaded sequentially
      • DELETE_IF_EXISTS: If TRUE, any existing files specified in fileNames will be deleted prior to start of upload. Otherwise the file is replaced once the upload completes. Rollback of the original file is no longer possible if the upload is cancelled, aborted or fails if the file was deleted beforehand. Supported values:The default value is FALSE.
      The default value is an empty Map.

  • ​
    Method Details

    • ​
      getClassSchema

      public static org.apache.avro.Schema getClassSchema()
      This method supports the Avro framework and is not intended to be called directly by the user.
      Returns:
      The schema for the class.

    • ​
      getFileNames

      public List<String> getFileNames()
      An array of full file name paths to be used for the files uploaded to KiFS. File names may have any number of nested directories in their paths, but the top-level directory must be an existing KiFS directory. Each file must reside in or under a top-level directory. A full file name path cannot be larger than 1024 characters.
      Returns:
      The current value of fileNames.

    • ​
      setFileNames

      public UploadFilesRequest setFileNames(List<String> fileNames)
      An array of full file name paths to be used for the files uploaded to KiFS. File names may have any number of nested directories in their paths, but the top-level directory must be an existing KiFS directory. Each file must reside in or under a top-level directory. A full file name path cannot be larger than 1024 characters.
      Parameters:
      fileNames - The new value for fileNames.
      Returns:
      this to mimic the builder pattern.

    • ​
      getFileData

      public List<ByteBuffer> getFileData()
      File data for the files being uploaded, for the respective files in fileNames.
      Returns:
      The current value of fileData.

    • ​
      setFileData

      public UploadFilesRequest setFileData(List<ByteBuffer> fileData)
      File data for the files being uploaded, for the respective files in fileNames.
      Parameters:
      fileData - The new value for fileData.
      Returns:
      this to mimic the builder pattern.

    • ​
      getOptions

      public Map<String,String> getOptions()
      Optional parameters.
      • FILE_ENCODING: Encoding that has been applied to the uploaded file data. When using JSON serialization it is recommended to utilize BASE64. The caller is responsible for encoding the data provided in this payload. Supported values:
        • BASE64: Specifies that the file data being uploaded has been base64 encoded.
        • NONE: The uploaded file data has not been encoded.
        The default value is NONE.
      • MULTIPART_OPERATION: Multipart upload operation to perform. Supported values:
        • NONE: Default, indicates this is not a multipart upload.
        • INIT: Initialize a multipart file upload.
        • UPLOAD_PART: Uploads a part of the specified multipart file upload.
        • COMPLETE: Complete the specified multipart file upload.
        • CANCEL: Cancel the specified multipart file upload.
        The default value is NONE.
      • MULTIPART_UPLOAD_UUID: UUID to uniquely identify a multipart upload.
      • MULTIPART_UPLOAD_PART_NUMBER: Incremental part number for each part in a multipart upload. Part numbers start at 1, increment by 1, and must be uploaded sequentially
      • DELETE_IF_EXISTS: If TRUE, any existing files specified in fileNames will be deleted prior to start of upload. Otherwise the file is replaced once the upload completes. Rollback of the original file is no longer possible if the upload is cancelled, aborted or fails if the file was deleted beforehand. Supported values:The default value is FALSE.
      The default value is an empty Map.
      Returns:
      The current value of options.

    • ​
      setOptions

      public UploadFilesRequest setOptions(Map<String,String> options)
      Optional parameters.
      • FILE_ENCODING: Encoding that has been applied to the uploaded file data. When using JSON serialization it is recommended to utilize BASE64. The caller is responsible for encoding the data provided in this payload. Supported values:
        • BASE64: Specifies that the file data being uploaded has been base64 encoded.
        • NONE: The uploaded file data has not been encoded.
        The default value is NONE.
      • MULTIPART_OPERATION: Multipart upload operation to perform. Supported values:
        • NONE: Default, indicates this is not a multipart upload.
        • INIT: Initialize a multipart file upload.
        • UPLOAD_PART: Uploads a part of the specified multipart file upload.
        • COMPLETE: Complete the specified multipart file upload.
        • CANCEL: Cancel the specified multipart file upload.
        The default value is NONE.
      • MULTIPART_UPLOAD_UUID: UUID to uniquely identify a multipart upload.
      • MULTIPART_UPLOAD_PART_NUMBER: Incremental part number for each part in a multipart upload. Part numbers start at 1, increment by 1, and must be uploaded sequentially
      • DELETE_IF_EXISTS: If TRUE, any existing files specified in fileNames will be deleted prior to start of upload. Otherwise the file is replaced once the upload completes. Rollback of the original file is no longer possible if the upload is cancelled, aborted or fails if the file was deleted beforehand. Supported values:The default value is FALSE.
      The default value is an empty Map.
      Parameters:
      options - The new value for options.
      Returns:
      this to mimic the builder pattern.

    • ​
      getSchema

      public org.apache.avro.Schema getSchema()
      This method supports the Avro framework and is not intended to be called directly by the user.
      Specified by:
      getSchema in interface org.apache.avro.generic.GenericContainer
      Returns:
      The schema object describing this class.

    • ​
      get

      public Object get(int index)
      This method supports the Avro framework and is not intended to be called directly by the user.
      Specified by:
      get in interface org.apache.avro.generic.IndexedRecord
      Parameters:
      index - the position of the field to get
      Returns:
      value of the field with the given index.
      Throws:
      IndexOutOfBoundsException

    • ​
      put

      public void put(int index, Object value)
      This method supports the Avro framework and is not intended to be called directly by the user.
      Specified by:
      put in interface org.apache.avro.generic.IndexedRecord
      Parameters:
      index - the position of the field to set
      value - the value to set
      Throws:
      IndexOutOfBoundsException

    • ​
      equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object

    • ​
      toString

      public String toString()
      Overrides:
      toString in class Object

    • ​
      hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object