Introduction to All of Us Survey Collection and Data Transformation Methods

Introduction

Within the All of Us Research Program, participant provided information (PPI) is collected via surveys and is meant as a way to augment data collected from other sources such as the electronic health record (EHR). Surveys are developed and deployed following a methodical process which includes prioritization of scientific domains, content creation based on sourcing of items from well-established studies and the literature, pilot evaluation and refinement, and scheduled deployment. Surveys are deployed on the secure online All of Us Participant Portal. Baseline surveys include questions on sociodemographics, health, and lifestyle. Additional surveys on other domains of interest are regularly developed and deployed. You can view survey questions and learn more about their sources by browsing the All of Us Research Hub Survey Explorer.

Methods

Beginning at program establishment in May 2017, all participants - whether enrolled through their participating health care providers (HPOs) or not (direct volunteers) - responded to surveys via the All of Us Participant Portal supported by the program’s Participant Technology Support Center (Vibrent Health). Beginning in December 2019, a new version of the Participant Portal, supported by the program’s Participant Center (Scripps University with CareEvolution), was introduced as a pilot approach to enrolling a limited number of direct volunteer participants. 

Survey questions and answers are transformed into structural survey metadata and stored and transmitted from the online Participant Portals to the All of Us Data and Research Center’s Raw Data Repository via an artifact referred to as a “survey codebook.” Survey codebooks are created using REDCap data dictionary format and contain not only survey content, but also assigned variables (e.g., codes), branching logic, and field validation rules. Besides serving as the codebook from survey data collection and transmission, resulting REDCap data dictionary artifacts are also made available through the REDCap Consortium’s Shared Library for reuse in other studies.5 Codebook variables (e.g., source codes) are deposited into the OMOP Common Data Model’s observation table and stored as an All of Us specific “PPI” vocabulary. This vocabulary is both browsable and downloadable via Athena, the online OMOP repository.  Survey response data are extracted from the All of Us program’s Raw Data Repository (RDR) and transmitted to the Curated Data Repository (CDR) for the purpose of research-grade dataset creation. Extracted response data are mapped according to the codebook source codes, as well OMOP concept codes and IDs. All survey-specific OMOP concepts are assessed for Registered and Controlled Tier privacy methodology application and resulting decisions are logged and implemented within the corresponding CDRs.

Definitions of Survey Relevant Variable Names in OMOP CDM v 5.2.1. 

Beginning with the v7 Registered and Controlled Tier CDRs (R2022Q4R9 and C2022Q4R9, respectively), select metadata about survey response submissions are made available to researchers via a OMOP Survey Conduct Table. For additional details about this table, see the CDR Data Dictionaries.

Survey Relevant Metadata Elements Available in OMOP Survey Conduct Table

Inclusion Criteria

With the exception of defined CDR “cut-off” dates and CDR privacy methodology, there are no exclusion factors for the survey response data. In terms of inclusion criteria, participants who met eligibility criteria for the surveys are included in the dataset. 

Survey Names, Eligibility Criteria, and Release Date Ranges