com.gpudb.filesystem.common

Class FullFileBatchManager

java.lang.Object

com.gpudb.filesystem.common.FullFileBatchManager

public class FullFileBatchManager
extends Object

This is an internal class and not meant to be used by the end users of the filesystem API. The consequences of using this class directly in client code is not guaranteed and maybe undesirable. This class manages the task of creating batches of files where the files are candidates for one-shot uploads.

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<String,org.apache.commons.lang3.tuple.Pair<String,Long>>	fullFileNameToSizeMap This variable is used to maintain a map of file names in the local file system to the pair of values of target file name on the KIFS and the file size on the local file system.
protected List<Map<String,String>>	listOfFullFileNameToRemoteFileNameMap This variable is used to store the maps in a list so that each map has files whose sizes add up to a maximum of the value returned by the method getFileSizeToSplit() of the class GPUdbFileHandler.Options.

Method Summary

Modifier and Type	Method and Description
void	addFile(String localFileName, String remoteFileName, Long size) Add the details for a file to be uploaded one shot.
void	clearBatches() Resets the internal data structures maintained for the batches
void	createBatches() Creates batches of files which could be uploaded one shot.
Map<String,String>	getNthBatch(int n) Gets a particular batch of files which could be uploaded one shot using a single call to the endpoint '/upload/files'.
int	getNumberOfBatches() Gets the number of batches of files which could be uploaded one shot.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fullFileNameToSizeMap

protected Map<String,org.apache.commons.lang3.tuple.Pair<String,Long>> fullFileNameToSizeMap

This variable is used to maintain a map of file names in the local file system to the pair of values of target file name on the KIFS and the file size on the local file system. This is used by the method FileOperation.sortFilesIntoFullAndMultipartLists(List, List). This method, after detecting a file to be a candidate for one shot upload will insert an entry into this map. This map is further split up into a list of maps and stored in the variable listOfFullFileNameToRemoteFileNameMap by the method createBatches(). The purpose of splitting the map into the list of maps is to arrive at batches of full file uploads so that each batch has a cumulative size whose limit is given by the method getFileSizeToSplit() of the class GPUdbFileHandler.Options.

listOfFullFileNameToRemoteFileNameMap

protected List<Map<String,String>> listOfFullFileNameToRemoteFileNameMap

This variable is used to store the maps in a list so that each map has files whose sizes add up to a maximum of the value returned by the method getFileSizeToSplit() of the class GPUdbFileHandler.Options. Each entry in this list is a map of local file name to its corresponding KIFS file name. This is used by the method 'FileUploader#uploadFullFiles()'

Method Detail

clearBatches

public void clearBatches()

Resets the internal data structures maintained for the batches

createBatches

public void createBatches()

Creates batches of files which could be uploaded one shot.

addFile

public void addFile(String localFileName,
                    String remoteFileName,
                    Long size)

Add the details for a file to be uploaded one shot.

Parameters:

localFileName - Name of the file on the local file system.

remoteFileName - Name of the file on KIFS.

size - The size of the file, needed to partition the batches on.

getNumberOfBatches

public int getNumberOfBatches()

Gets the number of batches of files which could be uploaded one shot.

Returns:

The number of batches

getNthBatch

public Map<String,String> getNthBatch(int n)

Gets a particular batch of files which could be uploaded one shot using a single call to the endpoint '/upload/files'.

Parameters:

n - The index of the batch to retrieve.

Returns:

The Nth batch; this batch is a set of entries in a map where each entry has the local file name as the key and the name of the file on KIFS as the value.