The bulk_get_metadata_google_store Module¶

class cloudfusion.store.gs.bulk_get_metadata_google_store.BulkGetMetadataGoogleStore(config)¶

Bases: cloudfusion.store.gs.google_store.GoogleStore, cloudfusion.store.bulk_get_metadata.BulkGetMetadata

Subclass of GoogleStore, extending it by implementing the BulkGetMetadata interface for quick directory lising.

get_bulk_metadata(*args, **kwargs)¶

Returns:	A dictionary mapping the path of every file object in directory to a dictionary with the keys ‘modified’, ‘bytes’ and ‘is_dir’ containing the corresponding metadata for the file object.

The value for ‘modified’ is a date in seconds, stating when the file object was last modified. The value for ‘bytes’ is the number of bytes of the file object. It is 0 if the object is a directory. The value for ‘is_dir’ is True if the file object is a directory and False otherwise.

Raises:	NoSuchFilesytemObjectError if the directory does not exist

The google_store Module¶

Created on 08.04.2011

@author: joe

class cloudfusion.store.gs.google_store.GoogleStore(config)¶

Bases: cloudfusion.store.store.Store

Subclass of Store implementing an interface to the Google Storage storage. File objects are stored in a top level container called bucket. There are no directories. They are simulated by appending a slash to a file object. I.e. “somepath/” is a directory name, but “somepath” without the slash is considered a file. Google Storage provides an interface to list file objects with a certain prefix, and a delimiter, which is used to list single directories. I.e. bucket.list(“somepath/”, delimiter=”/”) returns a list of all file paths beginning with “somepath/” that do not contain a slash after the prefix or end in a slash. For instance: [“somepath/file1”, “somepath/dir1/”, “somepath/file2”]. For compatibility with other tools, file objects with content type “application/x-directory” or file objects ending with _$folder$ are recognized as directories, too.

config is a dictionary with the keys id (access_key_id), secret (secret_access_key), and bucket_name. For instance:

    config['id'] = 'FDS54548SDF8D2S311DF' 
    config['secret'] = 'D370JKD=564++873ZHFD9FDKDD'
    config['bucket_name'] = 'cloudfusion'
    
The bucket will be created if it does not exist. A bucket is similar to a subfolder,
to which access with CloudFusion is restricted.         
Id and secret can be obtained from the developer's console:

* Go to console.developers.google.com/project
* Create a new project
* Select Project dashboard on the left, which opens a new tab
* Go to the new tab
* Select Billing on the left to set up billing
* Select Google Cloud Storage on the left 
* Click on the button labeled "Make this my default project for interoperable storage access"
* Click on Interoperable Access on the left 
* Click Generate new key, to generate the new key pair

Parameters:	config – dictionary with key value pairs

_GoogleStore__get_size(fileobject)¶

_get_time_difference()¶

_handle_error(error, stacktrace, method_name, remaining_tries, *args, **kwargs)¶

Used by retry decorator to react to errors.

_is_dir(key)¶

Parameters:	key – instance of boto’s Key class
Returns:	True iff key is a directory

account_info(*args, **kwargs)¶

create_directory(*args, **kwargs)¶

delete(*args, **kwargs)¶

duplicate(*args, **kwargs)¶

get_directory_listing(*args, **kwargs)¶

get_file(path_to_file)¶

get_logging_handler()¶

get_max_filesize()¶

Return maximum number of bytes per file

get_metadata(*args, **kwargs)¶

get_name()¶

get_overall_space()¶

get_used_space()¶

move(*args, **kwargs)¶

reconnect()¶

store_fileobject(*args, **kwargs)¶

write_gsutil_config()¶

Write credentials to .boto configuration file, for gsutil to use.