This article is to help explain the current storage options available within the Researcher Workbench. Here you may find answers to questions such as:

• Where are my data / files of data stored automatically?
• Where can I store data / files permanently? 
• Why did my data / files get deleted?
• Why are my data / files missing from my notebook / directory. 

Supplementary Office Hours recording: 

Standard Environments

Workspace Bucket

The All of Us Researcher Workbench is a cloud application. Each workspace has a permanent storage area called the “workspace bucket.” Notebooks are loaded in an ephemeral virtual machine and are synced to the “workspace bucket.” Users can also use the workspace bucket to save files needed for analysis. The workspace bucket is attached to your workspace, if you delete the workspace, you delete the bucket. If you share the workspace with your colleagues, you will share the workspace bucket. For more information about using buckets, please see this article. 

Persistent Disk 

The persistent disk (PD) storage is part of your VM that is automatically attached to the Cloud Environment VM (like a USB drive) when you recreate it (excluding Dataproc environments}. If you need to update your VM with new software or delete it for different use-cases across workspaces, your PD allows you to save files for later use. Because it's part of your VM, the persistent disk is also personal to you, the user; nobody else has access to it. A persistent disk is saved even when your compute environment is deleted. This incurs storage costs even when you aren't using the workbench.

We offer two types of reattachable persistent disks, a standard persistent disk and a solid state-drive (SSD) persistent disk. You can learn more about the disk types here: https://cloud.google.com/compute/docs/disks#disk-types

For more information regarding Persistent Disk, see this article.

Dataproc Environment

Note: Dataproc clusters do not support Persistent disks.

Standard Disk

A standard disk is created and deleted with your cloud environment. All files and outputs will automatically be saved to the Standard Disk. For permanent storage, copy files to your workspace bucket.

NOTE: All files stored in the Standard Disk are deleted when your environment is terminated. 

Workspace Bucket

The All of Us Researcher Workbench is a cloud application. Each workspace has a permanent storage area called the “workspace bucket.” Notebooks are loaded in an ephemeral virtual machine and are synced to the “workspace bucket.” Users can also use the workspace bucket to save files needed for analysis. The workspace bucket is attached to your workspace, if you delete the workspace, you delete the bucket. If you share the workspace with your colleagues, you will share the workspace bucket. For more information about using buckets, please see this article. 

To learn more about accessing saved files in your preferred storage option, please see this support article - Accessing Files in the Workspace Bucket or Persistent Disk.