:

Contents

1. Overview of package
2. Overview of package
   1. General usage
3. Availability of package by cluster
4. Configuring gdrive
5. Using gdrive
6. Errors/issues/etc
   1. Rate limit errors
   2. Expired or revoked token errors

Overview of package

General usage information

*** DEPRECATED *** The gdrive package/command is deprecated. Please use the rclone package/command instead of gdrive for accessing your Google drive. In addition to having consistent support for many Cloud storage providers including Google drive, we find that rclone supports Google drive better than the gdrive command, in particular with regards to keeping within Google data rate limitations.

Available versions of the package , by cluster

This section lists the available versions of the package on the different clusters.

Available versions of on the Zaratan cluster

Sorry, this is still under construction.

Configuring gdrive

*** DEPRECATED *** The gdrive package/command is deprecated. Please use the rclone package/command instead of gdrive for accessing your Google drive. In addition to having consistent support for many Cloud storage providers including Google drive, we find that rclone supports Google drive better than the gdrive command, in particular with regards to keeping within Google data rate limitations.

Before you can use gdrive, you must create a token to allow gdrive access to your Google G Suite drive. To do this:

1. Run module load gdrive if you have not already done so to add it to your path.
2. By default, gdrive will store your token in the ~/.gdrive directory. If you wish to store the token elsewhere (e.g. you use ~/.gdrive for your personal Google drive, and want a different config file for your team drive), you can either add the flags --config PATH_TO_CONFIG_DIR or set the environmental variable GDRIVE_CONFIG_DIR to the new config dir.
3. Run gdrive about. This command will print out an URL and prompt for a verification code. Enter the URL in a browser, authenticate to Google and allow gdrive access to your account, and cut and past the verification code presented in the browser to the gdrive prompt.

BE SURE TO PROTECT YOUR gdrive configruation directory (either ~/.gdrive or whereever you instructed gdrive to use with the --config flag or GDRIVE_CONFIG_DIR environmental variable). Anyone with read access to that directory can access your Google drive as you.

Using gdrive

You will need to module load gdrive to add gdrive to your path if you have not already done so.

gdrive help will provide some usage instructions, but basically you can use gdrive download FILEID to download files from Google drive to the local filesystem and gdrive upload PATH to upload files from the local file system to Google drive.

Errors/issues/etc

Rate limit errors

Gdrive is prone to fail with errors like Failed to upload file: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded , especially if you are moving large amounts of data as is common among HPC users.

Unfortunately, there is no fix for this in gdrive at the time of writing. Instead, we recommend that users switch to the rclone command. In addition to providing consistent command line support to many Cloud storage providers, it has built in exponential backoff features which will keep the data rate within Google's requirements.

Expired or revoked token errors

If you start getting errors about expired or revoked tokens, you might need to refresh your gdrive token. To do this, just delete the file token_v2.json in your gdrive configuration directory (normally ~/.gdrive unless you set GDRIVE_CONFIG_DIR or are using the --config flag).

The just repeat the gdrive about process about to refresh your token.