An egress alert is issued when a registered researcher violates security policies established by the All of Us Research Program’s Data and Research Center (DRC).
As a reminder, per the Data User Code of Conduct, registered users of the All of Us Researcher Workbench are not permitted to download or remove participant-level data from the platform.
The most common cause of an egress alert is an attempted download of data from the Researcher Workbench (accidental or intentional). However, other events can also trigger an alert.
While the egress alert system is designed to prevent large-scale downloads of participant-level data, legitimate research activity can sometimes trigger an alert due to how the system is configured. In other words, actions performed inside a workspace can trigger egress even when nothing is intentionally downloaded.
The most common triggers are activities that move, return, display, or process a large amount of data at once.
Common egress triggers
| Pattern | Example |
| Attempted download | Downloading a file from Jupyter, RStudio, etc. |
| Large summary-data download | Downloading a file that contains permitted summary data, but the file exceeds the egress size threshold |
| Large query/result | A query returns a very large result set or generates a large output |
| Large notebook | A notebook contains very large dataframe/query outputs, or the notebook file itself is very large |
| Error/output behavior | A command produces a large error or unusually large output |
| Print commands | Using print/output commands that generate or pipe to another script |
| Large data transformations | Sending data through command-line tools (e.g., detailed bash or awk scripts) that generate large outputs |
What happens when an egress alert is triggered
If an egress alert is triggered:
- The virtual machine (VM) that triggered the alert will be shut down.
- The researcher will receive an email from the DRC Security and Support teams requesting completion of a form describing the activity. The Security team typically uses the form to answer questions such as:
- Can you please confirm you were active in your account on [date / time], and also share the details of your activity?
- If you were using SQL, can you please share the SQL query you used so we can validate it against our logs?
- Can you please confirm no data were downloaded to your local device during this activity?
-
If data were downloaded:
- What method was used to download the data?
- How much data were downloaded?
- Did the downloaded data include any participant-level data? What were the column headers in the file?
- How quickly were the data downloaded?
- What ports were used?
After reviewing the form, the DRC’s Security and Support teams may follow up with additional questions about the event.
Note: If a high-volume egress activity is detected, the Researcher Workbench Developer team will automatically disable the researcher’s account. To reinstate the account, the Security team will review the researcher’s responses to the form and, if appropriate, approve reinstatement.
How to reduce the likelihood of egress alerts
If normal research activity is regularly triggering egress alerts, consider adjusting your workflow:
- Split large workflows into smaller stages or notebooks.
- Save intermediate outputs to the workspace bucket.
- Clear notebook outputs when they are no longer needed.
- Minimize large printed outputs and large displayed dataframes.
- Work with smaller subsets of data when feasible.
Large download requests
If your work requires a large export or download, you may submit a Large Download Request. These requests may be approved for temporary relaxation of egress limits to support computationally intensive workflow steps that would otherwise trigger egress. For more information, read our article on Large Download Requests.
If you have any additional questions, please contact us at support@researchallofus.org.
Please review our All of Us Research Program Data and Statistics Dissemination Policy, All of Us Research Program User Appeals Policy and Data User Code of Conduct for further information.
Comments
0 comments
Article is closed for comments.