How do I access the workspace bucket and copy data to and from it?

The All of Us Researcher Workbench is a cloud application. Each workspace has a permanent storage area called the “workspace bucket” within the Google Cloud Platform (GCP).  Notebooks are loaded in an ephemeral virtual machine and are synced to the “workspace bucket.” Users can use the workspace bucket to save files needed for analysis. The workspace bucket is attached to your workspace, if you delete the workspace, you delete the bucket. If you share the workspace with your colleagues, you will share the workspace bucket. Please view this video tutorial to learn more about using workspace buckets to save and retrieve files.

To learn more about storage options, please visit this support article. For more details about accessing files saved in the workspace bucket please see this support article: Accessing Files in the Workspace Bucket or Persistent Disk.

Accessing your workspace bucket

There are various ways to gain access to your workspace bucket: through your notebook using an environment variable in Python or R, File> Open in an active notebook, or from the “About” tab in your workspace. 

To access your workspace bucket through a notebook, you can utilize the available code snippets. Please note, you will need to run setup prior to utilizing storage code snippets. The workspace bucket can be accessed using an environment variable in Python - os.getenv('WORKSPACE_BUCKET') and in R - Sys.getenv('WORKSPACE_BUCKET'). You can access these code snippets within your workspace notebook by navigating to “Snippet” in the menu bar.

You can access your workspace bucket through File > Open in an active notebook, as indicated below:

You can also access your workspace bucket by navigating to the “About” tab in your workspace, and click the link under 'File Management' to browse files in the GCP Cloud Console.

You can easily delete files from your bucket in this menu, by clicking a file and then clicking the delete option at the top.

Copying data to and from your workspace bucket

Code snippets can be used to copy a data frame to a file in the workspace bucket and vice versa. When the code snippet loads into your notebook, be sure to replace 'test.csv' with the name of the file you're going to store in the bucket.

Bucket files can be shared between your workspaces as long as they share the same data access tier level. You can just use the same file path you would for the file between workspaces. For more information, please see this article. 

Files can be uploaded directly from your local machine to the bucket as long as they don't contain any personal identifiable information. Please see this article for instructions about uploading data. 

Once files are saved to your workspace bucket, they will be available for use. To download files from your workspace bucket to your local environment, follow instructions in this support article.