com.gpudb.filesystem

Class GPUdbFileHandler

java.lang.Object

com.gpudb.filesystem.GPUdbFileHandler

public class GPUdbFileHandler
extends Object

This is the main class which exposes the API to be used by the end users. This class exposes methods to upload e.g., (upload(String, String, UploadOptions, FileUploadListener), upload(List, String, UploadOptions, FileUploadListener)) and similarly for download. The entire filesystem API is exposed through this class and an instance of this class is enough to consume all functionalities provided by the KIFS. Apart from this class the other classes which could be used by the users of this API are: UploadOptions DownloadOptions FileUploadListener FileDownloadListener

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GPUdbFileHandler.Options This class models the options available for modifying some behaviors of the GPUdbFileHandler class.

Field Summary

Fields

Modifier and Type	Field and Description
static String	KIFS_PATH_PREFIX Prefix to use when referencing KiFS files; e.g., for purposes for file ingest.
static String	KIFS_PATH_SEPARATOR Separator character between a KiFS directory and KiFS file name.
static String	REMOTE_USER_HOME_DIR_PREFIX Alias for a user's home directory.

Constructor Summary

Constructors

Constructor and Description
GPUdbFileHandler(GPUdb db) Constructs a GPUdbFileHandler object that allows the user to upload and download files using default GPUdbFileHandler.Options.
GPUdbFileHandler(GPUdb db, GPUdbFileHandler.Options options) Constructs a GPUdbFileHandler object that allows the user to upload and download files.

Method Summary

Modifier and Type	Method and Description
void	createDirectory(String remoteDirName) This method will create a KIFS directory with default options.
void	createDirectory(String remoteDirName, boolean noErrorIfExists) This method will create a KIFS directory with options as explained below.
void	deleteDirectory(String remoteDirName) This method deletes a KIFS directory.
void	deleteDirectory(String remoteDirName, boolean recursive, boolean noErrorIfNotExists) This method deletes a KIFS directory.
void	deleteFiles(List<String> fileNames) This method takes a list of fully qualified KIFS file paths and deletes the files in one go.
void	deleteFiles(List<String> fileNames, boolean noErrorIfNotExists) This method takes a list of fully qualified KIFS file paths and deletes the files in one go.
void	deleteFilesInDir(String remoteDirName) This method deletes files in the KIFS directory whose name is passed in using remoteDirName.
void	deleteFilesInDir(String remoteDirName, boolean noErrorIfNotExists) This method deletes files in the KIFS directory whose name is passes in using remoteDirName.
void	download(List<String> fileNames, String localDirName) This method downloads files from a KIFS directory to a local directory.
void	download(List<String> fileNames, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) This method downloads files from a KIFS directory to a local directory.
void	download(String fileName, String localDirName) This method downloads a single file from a KIFS directory to a local directory.
void	download(String fileName, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) This method downloads a single file from a KIFS directory to a local directory.
void	downloadDir(String remoteDirName, String localDirName, DownloadOptions downloadOptions, FileDownloadListener callback) This method will download all files in a KIFS directory to a directory on the local file system.
static int	getDefaultFullFileDispatcherThreadpoolSize()
static int	getDefaultThreadPoolTerminationTimeout()
GPUdbFileHandler.Options	getOptions() Returns the GPUdbFileHandler.Options instance
void	ingest(List<String> fileNames, String tableName, IngestOptions ingestOptions, TableCreationOptions createTableOptions) This method will ingest several files in one go.
boolean	kifsDirectoriesExist(Set<String> dirNames) This method checks whether the given KIFS directories exist or not.
boolean	kifsDirectoryExists(String dirName) This method checks whether the given KIFS directory exists or not.
boolean	kifsFileExists(String fileName) This method checks whether the given KIFS file exists or not.
List<KifsDirectoryInfo>	showAllDirectories() This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showAllDirectories(Map<String,String> showDirectoryOptions) This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectories(List<String> remoteDirNames) This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectories(List<String> remoteDirNames, Map<String,String> showDirectoryOptions) This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectories(Set<String> remoteDirNames) Deprecated.
List<KifsDirectoryInfo>	showDirectories(Set<String> remoteDirNames, Map<String,String> showDirectoryOptions) Deprecated.
List<KifsDirectoryInfo>	showDirectory(String remoteDirName) This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects.
List<KifsDirectoryInfo>	showDirectory(String remoteDirName, Map<String,String> showDirectoryOptions) This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects.
List<KifsFileInfo>	showFiles(List<String> remotePaths) This method returns statistics about the given files and/or directories.
void	upload(List<String> fileNames, String remoteDirName) This method facilitates uploading of multiple files from the same local directory to a given remote directory.
void	upload(List<String> fileNames, String remoteDirName, UploadOptions uploadOptions, FileUploadListener callback) This method facilitates uploading of multiple files from the same local directory to a given remote directory.
void	upload(String fileName, String remoteDirName) This method facilitates the upload of a single file from a given local directory to a given remote directory.
void	upload(String fileName, String remoteDirName, UploadOptions uploadOptions, FileUploadListener callback) This method facilitates the upload of a single file from a given local directory to a given remote directory.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

KIFS_PATH_SEPARATOR

public static final String KIFS_PATH_SEPARATOR

Separator character between a KiFS directory and KiFS file name. This can also be used within a file name to create a virtual directory within a path.

See Also:

Constant Field Values

KIFS_PATH_PREFIX

public static final String KIFS_PATH_PREFIX

Prefix to use when referencing KiFS files; e.g., for purposes for file ingest.

See Also:

Constant Field Values

REMOTE_USER_HOME_DIR_PREFIX

public static final String REMOTE_USER_HOME_DIR_PREFIX

Alias for a user's home directory.

See Also:

Constant Field Values

Constructor Detail

GPUdbFileHandler

public GPUdbFileHandler(GPUdb db,
                        GPUdbFileHandler.Options options)

Constructs a GPUdbFileHandler object that allows the user to upload and download files.

Parameters:

db - The GPUdb instance used to access KiFS.

options - Options for setting up the files for transfer.

GPUdbFileHandler

public GPUdbFileHandler(GPUdb db)

Constructs a GPUdbFileHandler object that allows the user to upload and download files using default GPUdbFileHandler.Options.

Parameters:

db - The GPUdb instance used to access KiFS.

Method Detail

getDefaultThreadPoolTerminationTimeout

public static int getDefaultThreadPoolTerminationTimeout()

getDefaultFullFileDispatcherThreadpoolSize

public static int getDefaultFullFileDispatcherThreadpoolSize()

getOptions

public GPUdbFileHandler.Options getOptions()

Returns the GPUdbFileHandler.Options instance

upload

public void upload(String fileName,
                   String remoteDirName)
            throws GPUdbException

This method facilitates the upload of a single file from a given local directory to a given remote directory. The filename passed in is preserved between the client and the server. This method uses the default Upload options and does not use a callback.

Parameters:

fileName - Name of the file along with the full path e.g., "/home/user1/dir1/dir2/a.txt".

remoteDirName - Name of KiFS directory to upload to.

Throws:

GPUdbException - If an error occurs uploading any of the files to the server.

See Also:

FileUploadListener, UploadOptions, UploadOptions.defaultOptions(), upload(String, String, UploadOptions, FileUploadListener)

upload

public void upload(String fileName,
                   String remoteDirName,
                   UploadOptions uploadOptions,
                   FileUploadListener callback)
            throws GPUdbException

This method facilitates the upload of a single file from a given local directory to a given remote directory. The filename passed in is preserved between the client and the server.

Parameters:

fileName - Name of the file along with the full path e.g., "/home/user1/dir1/dir2/a.txt".

remoteDirName - Name of KiFS directory to upload to.

uploadOptions - Various options to be used for uploading a file. If the value passed is null then default values would be used. The default values for the options can also be set explicitly using the method UploadOptions.defaultOptions().

callback - An instance of FileUploadListener class which is used to report status of an ongoing/completed upload to the caller of the method.

Throws:

GPUdbException - If an error occurs uploading any of the files to the server.

See Also:

FileUploadListener, UploadOptions

upload

public void upload(List<String> fileNames,
                   String remoteDirName)
            throws GPUdbException

This method facilitates uploading of multiple files from the same local directory to a given remote directory. The filenames are preserved between the client and the server. This method uses default Upload options and does not use a callback.

Parameters:

fileNames - Names of the files along with the full path e.g., "/home/user1/dir1/dir2/a.txt".

remoteDirName - Name of KiFS directory to upload to.

Throws:

GPUdbException - If an error occurs uploading any of the files to the server.

See Also:

FileUploadListener, UploadOptions, UploadOptions.defaultOptions(), upload(List, String, UploadOptions, FileUploadListener)

upload

public void upload(List<String> fileNames,
                   String remoteDirName,
                   UploadOptions uploadOptions,
                   FileUploadListener callback)
            throws GPUdbException

This method facilitates uploading of multiple files from the same local directory to a given remote directory. The filenames are preserved between the client and the server.

Parameters:

fileNames - Names of the files along with the full path e.g., "/home/user1/dir1/dir2/a.txt".

remoteDirName - Name of KiFS directory to upload to.

uploadOptions - Various uploadOptions to be used for uploading a file. If the value passed is null then default values would be used. The default values for the uploadOptions can also be set explicitly using the method UploadOptions.defaultOptions().

callback - An instance of FileUploadListener class which is used to report status of an ongoing/completed upload to the caller of the method.

Throws:

GPUdbException - If an error occurs uploading any of the files to the server.

See Also:

FileUploadListener, UploadOptions

download

public void download(List<String> fileNames,
                     String localDirName)
              throws GPUdbException

This method downloads files from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory. This method uses default download options and doesn't use a callback.

Parameters:

fileNames - Names of the KiFS files to download e.g., 'a/b/c/d.txt'.

localDirName - Name of local directory to download to. This directory must exist on the local file system.

Throws:

GPUdbException

See Also:

DownloadOptions, DownloadOptions.defaultOptions(), FileDownloadListener

download

public void download(List<String> fileNames,
                     String localDirName,
                     DownloadOptions downloadOptions,
                     FileDownloadListener callback)
              throws GPUdbException

This method downloads files from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory.

Parameters:

fileNames - Names of the KiFS files to download e.g., 'a/b/c/d.txt'.

localDirName - Name of local directory to download to. This directory must exist on the local file system.

downloadOptions - Options to be used for downloading files from KIFS to the local directory. if the value passed is null then default values would be used. The default values for the options can also be set explicitly using the method DownloadOptions.defaultOptions().

callback - An instance of FileDownloadListener class which is used to report status of an ongoing/completed download to the caller of the method.

Throws:

GPUdbException

download

public void download(String fileName,
                     String localDirName)
              throws GPUdbException

This method downloads a single file from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory.

Parameters:

fileName - Name of the KiFS file to download e.g., 'a/b/c/d.txt'.

localDirName - Name of local directory to download to. This directory must exist on the local file system.

Throws:

GPUdbException

download

public void download(String fileName,
                     String localDirName,
                     DownloadOptions downloadOptions,
                     FileDownloadListener callback)
              throws GPUdbException

This method downloads a single file from a KIFS directory to a local directory. The local directory must exist or else this method will throw an exception. The 'fileNames' argument contains a list of fully qualified file names on a KIFS directory.

Parameters:

fileName - Name of the KiFS file to download e.g., 'a/b/c/d.txt'.

localDirName - Name of local directory to download to. This directory must exist on the local file system.

downloadOptions - Options to be used for downloading files from KIFS to the local directory. if the value passed is null then default values would be used. The default values for the options can also be set explicitly using the method DownloadOptions.defaultOptions().

Throws:

GPUdbException

downloadDir

public void downloadDir(String remoteDirName,
                        String localDirName,
                        DownloadOptions downloadOptions,
                        FileDownloadListener callback)
                 throws GPUdbException

This method will download all files in a KIFS directory to a directory on the local file system.

Parameters:

remoteDirName - - Name of the KIFS directory to download.

localDirName - - Name of the local directory.

downloadOptions - Options to be used for downloading files from KIFS to the local directory. if the value passed is null then default values would be used. The default values for the options can also be set explicitly using the method DownloadOptions.defaultOptions().

Throws:

GPUdbException - If: 1. the remoteDirName is not existing 2. the localDirName is not existing 3. something goes wrong with the download

See Also:

DownloadOptions

ingest

public void ingest(List<String> fileNames,
                   String tableName,
                   IngestOptions ingestOptions,
                   TableCreationOptions createTableOptions)
            throws GPUdbException

This method will ingest several files in one go. It will use the method upload(List, String, UploadOptions, FileUploadListener) to upload the files to the directory 'sys_temp' on the KIFS and use the method GPUdb.insertRecordsFromFiles(String, List, Map, Map, Map) to update the table data.

Parameters:

fileNames - Names of the local files to be ingested.

tableName - Name of the table to ingest into.

ingestOptions - Options for ingestion.

createTableOptions - Options for table creation.

Throws:

If - the ingestion or the underlying upload fails.

GPUdbException

See Also:

IngestOptions, TableCreationOptions

deleteFiles

public void deleteFiles(List<String> fileNames)
                 throws GPUdbException

This method takes a list of fully qualified KIFS file paths and deletes the files in one go.

Parameters:

fileNames - List of files in KIFS to delete.

Throws:

GPUdbException

deleteFiles

public void deleteFiles(List<String> fileNames,
                        boolean noErrorIfNotExists)
                 throws GPUdbException

This method takes a list of fully qualified KIFS file paths and deletes the files in one go.

Parameters:

fileNames - List of files in KIFS to delete.

noErrorIfNotExists - True indicates not to throw an error if the files are not found on KIFS and false otherwise.

Throws:

GPUdbException

deleteFilesInDir

public void deleteFilesInDir(String remoteDirName)
                      throws GPUdbException

This method deletes files in the KIFS directory whose name is passed in using remoteDirName. This method uses default options.

Parameters:

remoteDirName - Name of the KIFS directory containing files to delete.

Throws:

GPUdbException - If an error occurs deleting the files.

deleteFilesInDir

public void deleteFilesInDir(String remoteDirName,
                             boolean noErrorIfNotExists)
                      throws GPUdbException

This method deletes files in the KIFS directory whose name is passes in using remoteDirName.

Parameters:

remoteDirName - Name of the KIFS directory containing files to delete.

noErrorIfNotExists - Indicates whether to allow deletion of files in non-existent KIFS directory or not.

Throws:

GPUdbException - If an error occurs deleting the files.

showFiles

public List<KifsFileInfo> showFiles(List<String> remotePaths)
                             throws GPUdbException

This method returns statistics about the given files and/or directories. Wildcards "?" & "*" can be used to represent one character and zero or more characters, respectively.

Parameters:

remotePaths - List of KIFS files and/or directory names to report statistics on.

Returns:

List of KifsFileInfo objects, containing statistics about the given files.

Throws:

GPUdbException - If an error occurs looking up the files.

showDirectories

public List<KifsDirectoryInfo> showDirectories(Set<String> remoteDirNames)
                                        throws GPUdbException

Deprecated.

This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirNames - Set of KIFS directory names to report statistics on.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directories.

Throws:

GPUdbException - If an error occurs looking up the directories.

See Also:

showDirectories(Set, Map)

showDirectories

public List<KifsDirectoryInfo> showDirectories(Set<String> remoteDirNames,
                                               Map<String,String> showDirectoryOptions)
                                        throws GPUdbException

Deprecated.

This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirNames - Set of KIFS directory names to report statistics on.

showDirectoryOptions - Unused.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directories.

Throws:

GPUdbException

showDirectories

public List<KifsDirectoryInfo> showDirectories(List<String> remoteDirNames)
                                        throws GPUdbException

This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirNames - List of KIFS directory names to report statistics on.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directories.

Throws:

GPUdbException - If an error occurs looking up the directories.

See Also:

showDirectories(Set, Map)

showDirectories

public List<KifsDirectoryInfo> showDirectories(List<String> remoteDirNames,
                                               Map<String,String> showDirectoryOptions)
                                        throws GPUdbException

This method retrieves the directory information for a given list of KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirNames - List of KIFS directory names to report statistics on.

showDirectoryOptions - Reserved for future use.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directories.

Throws:

GPUdbException

showDirectory

public List<KifsDirectoryInfo> showDirectory(String remoteDirName)
                                      throws GPUdbException

This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects. This method uses default options.

Parameters:

remoteDirName - Name of the KIFS directory to report statistics on; cannot be null or empty.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directory.

Throws:

GPUdbException

See Also:

showDirectory(String, Map)

showDirectory

public List<KifsDirectoryInfo> showDirectory(String remoteDirName,
                                             Map<String,String> showDirectoryOptions)
                                      throws GPUdbException

This method retrieves the directory information for a given KIFS directory and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

remoteDirName - Name of the KIFS directory to report statistics on; cannot be null or empty.

showDirectoryOptions - Reserved for future use.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directory.

Throws:

GPUdbException

showAllDirectories

public List<KifsDirectoryInfo> showAllDirectories()
                                           throws GPUdbException

This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directory.

Throws:

GPUdbException - If an error occurs looking up the directories.

See Also:

showAllDirectories(Map)

showAllDirectories

public List<KifsDirectoryInfo> showAllDirectories(Map<String,String> showDirectoryOptions)
                                           throws GPUdbException

This method retrieves the directory information all KIFS directories and returns the information as a list of KifsDirectoryInfo objects.

Parameters:

showDirectoryOptions - Reserved for future use.

Returns:

List of KifsDirectoryInfo objects, containing statistics about the given directory.

Throws:

GPUdbException - If an error occurs looking up the directories.

createDirectory

public void createDirectory(String remoteDirName)
                     throws GPUdbException

This method will create a KIFS directory with default options.

Parameters:

remoteDirName - Name of the KIFS directory to create.

Throws:

GPUdbException - If the directory creation fails.

createDirectory

public void createDirectory(String remoteDirName,
                            boolean noErrorIfExists)
                     throws GPUdbException

This method will create a KIFS directory with options as explained below.

Parameters:

remoteDirName - Name of the KIFS directory to create.

noErrorIfExists - True means that if the directory exists, there will be no error thrown; false means an error will be thrown.

Throws:

GPUdbException - If the directory creation fails.

deleteDirectory

public void deleteDirectory(String remoteDirName)
                     throws GPUdbException

This method deletes a KIFS directory. This method uses default options.

Parameters:

remoteDirName - Name of the KIFS directory to delete.

Throws:

GPUdbException - If the directory deletion fails.

See Also:

deleteDirectory(String, boolean, boolean)

deleteDirectory

public void deleteDirectory(String remoteDirName,
                            boolean recursive,
                            boolean noErrorIfNotExists)
                     throws GPUdbException

This method deletes a KIFS directory. It will do a recursive delete if such an option is given.

Parameters:

remoteDirName - Name of the KIFS directory to delete.

recursive - Indicates all files and virtual subdirectories under remoteDirName will also be deleted; if false, an error will be returned if the directory is not empty.

noErrorIfNotExists - True means no error will be returned if the given directory does not exist.

Throws:

GPUdbException

kifsDirectoriesExist

public boolean kifsDirectoriesExist(Set<String> dirNames)

This method checks whether the given KIFS directories exist or not.

Parameters:

dirNames - The set of full KIFS directory paths to check for.

Returns:

True if all the given directories exist, and false if one or more of them does not exist.

kifsDirectoryExists

public boolean kifsDirectoryExists(String dirName)

This method checks whether the given KIFS directory exists or not.

Parameters:

dirName - The full KIFS directory path to check for.

Returns:

True if the given directory exists, and false if it doesn't.

kifsFileExists

public boolean kifsFileExists(String fileName)

This method checks whether the given KIFS file exists or not.

Parameters:

fileName - The full KIFS file path to check for.

Returns:

True if the given file exists, and false if it doesn't.