Managing Your Researcher Workbench Costs

You can take advantage of All of Us Researcher Workbench and Google Cloud Platform (GCP) console features to help manage and monitor your costs. 

Google Cloud Platform (GCP) features

Let’s start by reviewing some of the GCP console features you’ll want to utilize if you’ve linked your billing account for payment. These features can be accessed by signing in to your GCP billing account.

1. Monitoring costs

   • Billing overview: The Billing overview page provides a summary of your current spending including total costs, usage trends, and billing history.

   • Costs by project: View costs broken down by individual projects to understand which projects are consuming the most resources. Each individual Researcher Workbench workspace linked to your billing account will be labeled as a GCP billing “project” in your console.

2. Setting budgets and alerts

   • Budgets: Set budget limits to control your spending. You can create budget alerts for your projects to receive notifications when your costs approach or exceed the set limits. Set up alerts for unexpected spending spikes, payment issues, or other billing-related events. Ensure that email notifications are configured to alert responsible parties promptly. Go to the “Billing” overview page, click on "Budgets & alerts," and then "Create budget" to set up a new budget.

   • Define the budget amount, time period, and optional alert thresholds (e.g., 50%, 90% of budget).

3. Analyzing costs and usage

   • Use tools like Cost Analysis and Cost Explorer to analyze your spending patterns.

     • Cost Analysis breaks down costs by service, SKU (Stock Keeping Unit), and usage type.

     • Cost Explorer provides interactive visualizations and filtering options to analyze costs over time and by different dimensions (e.g., projects, services, regions).

Researcher Workbench features

Now, let’s look at some of the Researcher Workbench features you can use to manage efficient use of compute resources.

1. Application auto-delete features

   • Applications will auto-delete after a set period of time once idle unless you opt-out of that setting. 

   • We recommend using the default setting unless you intentionally plan to run an application “in the background” for a longer period of time. Read about application costs and default settings.

2. Cloud environment and persistent disk management

   • When you start an application environment or open a persistent disk, it will be logged on your “Cloud Environment” page and can be deleted from the “Cloud Environment” page. 

   • To access the “Cloud Environment” page

     1. Click the menu icon ().
        
        

     2. Click the down arrow ().
        
        
        
        

     3. Click "Cloud Environments."
        
        
        
        

        We recommend reviewing the list before logging off to double check whether you’ve left anything unintentionally running. 

3. Storage bucket management

   • Storage buckets are attached to each workspace you create and can be used to store working files, code, figures, etc. that you export from your application. 

   • While storage costs are relatively low, they can add up over time as your file size and number of files grow. We recommend you review your storage bucket routinely and delete files as they become obsolete for your project.

4. Virtual machine customization

   • All applications are configured with a default setting for RAM, CPU, etc. If you are analyzing data in Jupyter Notebook, you can customize those settings. 

   • Consider customizing depending on the size of your data and analysis plans, but remember that increasing configuration settings can result in higher compute costs.

5. Efficient queries and outputs

   • One of the greatest assets of the All of Us dataset is its large sample size. However, the larger the sample, the larger the compute cost and time. 

   • To avoid unnecessary compute costs you can try running the analysis against a smaller sample and/or a smaller region of the genome or try verifying the outcome before utilizing more compute resources.