Introduction to Geographical Data Collection and Transformation

  • Updated

Accessing Geolocation Data

As part of the curation process, some participant identifiable information has been removed from the Registered and Controlled Tier datasets to protect participant privacy and reduce the risk of re-identification. This includes some participant geolocation data. Geolocation data may come from participant provided information (PPI), or from the electronic health records (EHR) data. In most cases, participants’ self-reported addresses will be used as the primary source of geolocation data. Below is information about available geolocation fields specific to the Registered and Controlled Tier datasets, respectively.  For additional information on how All of Us protects participant privacy by means of data removal, suppression, or generalizations, click here.

Geolocation Data Available within the Registered Tier

State-level data are the most granular geolocation available within the Registered Tier dataset and it is subject to Registered Tier state generalization rules. Researchers will not be able to identify the specific All of Us site where a participant enrolled. 

Participants’ state-level geolocation data are generalized, with the value_as_concept_id = 2000000011, in the Registered Tier dataset if any of the following conditions are met:

  • A participant has EHR data available in the dataset and self-reported state of residence differs from the state of the health provider organization sharing the EHR data.  
  • The participant has reported a state of residence from which there are 200 or fewer total participants enrolled.

The state location data are stored in the data tables in all rows with the observation_source_concept_id = 1585249. The state code is stored in value_as_concept_id in the observation table. 

Geolocation Data Available within the Controlled Tier

Three digit zip code level data are the most granular geolocation data available within the Controlled Tier dataset and are subject to Controlled Tier state generalization rules. If a participant resides in a zip code that is reported to have fewer than 20,000 residents per Census, the participant is generalized to a random neighboring zip3 to protect their privacy. The zip codes with fewer than 20k residences are sourced from Methods for De-identification of PHI | HHS.gov. 

Participants who have self-reported as American Indian or Alaskan Native (AI/AN) will have three digit zip code generalized to 000* in CDR v7 through present CDR versions. 

In the event we receive invalid zip codes from our upstream data contributors, the Data and Research Center (DRC) will recodes these invalid zip codes to 000*. 

Note: Researchers will not be able to identify the specific All of Us site where a participant enrolled using geolocation data.

Data Extraction Query Examples

Registered Tier

To obtain state level information in the Registered Tier: 

  1. Join the value_as_concept_id in the observation table with concept_id from the concept table.
  2. Use concept_name from the concept table to extract the state.

Coding examples for how to retrieve state location are below. 

Example of Code in Python:

# Install dependencies
import pandas_gbq

# Load the CDRv9 dataset (wb-affable-acorn-7941.R2025Q4R6)
dataset = %env WORKSPACE_CDR

# Query the count of participants by states
query = f"""
SELECT 
    concept_name, 
    COUNT(DISTINCT person_id) as n_participant
FROM `{dataset}.observation`
JOIN `{dataset}.concept` on value_as_concept_id=concept_id
WHERE observation_source_concept_id=1585249
GROUP BY 1
"""

# Save states in a dataframe
df = pandas_gbq.read_gbq(query, dialect="standard")
df.head()

Example of Code in R: 

# Import dependencies 
library(bigrquery)
library(DBI)
library(tidyverse)


# Load the CDRv9 dataset (wb-affable-acorn-7941.R2025Q4R6)
dataset <- Sys.getenv('WORKSPACE_CDR')

# Query the count of participants by states
geo_sql <- str_glue("SELECT DISTINCT
   concept_name,
   COUNT(DISTINCT person_id) as n_participant
FROM `{dataset}.observation`
JOIN `{dataset}.concept` ON value_as_concept_id = concept_id
WHERE observation_source_concept_id=1585249
GROUP BY 1
")

# Save states in a dataframe
tb <- bq_project_query(Sys.getenv('GOOGLE_PROJECT'), geo_sql)
geo_df <-  bq_table_download(tb)

head(geo_df)

Controlled Tier

Geolocation data by zip code in the Controlled Tier can be accessed via the Data Explorer or by directly querying in the Jupyter notebook. 

  1. Access via the Data Explorer

Create a cohort. When you add new criteria, search “Postal code” in the criteria search bar and select “Add.”

 

  1. Direct query

You can also access geolocation by zip code by directly querying the data, as shown in the following example queries. 

Example of code in Python: 

# Install dependencies
import pandas_gbq 


# Load the CDRv9 dataset (wb-affable-acorn-7941.R2025Q4R6)
dataset = %env WORKSPACE_CDR


# Query the count of participants by states and zip code
query =  f'''
SELECT COUNT(DISTINCT person_id) AS n_participant, 
    state_of_residence_source_value AS State,
    value_as_string AS Zip_code                
FROM `{dataset}.person_ext`
JOIN `{dataset}.observation` USING(person_id)
WHERE observation_source_concept_id = 1585250 
GROUP BY 2,3'''

# Save states in a dataframe
df = pandas_gbq.read_gbq(query)
df.head()

Example of code in R:

# Install dependencies
library(bigrquery)
library(DBI)
library(tidyverse)

# Load the CDRv9 dataset (wb-affable-acorn-7941.R2025Q4R6)
dataset <- Sys.getenv('WORKSPACE_CDR')


# Query the count of participants by states and zip code
query <- str_glue("
SELECT COUNT(DISTINCT person_id) AS n_participant, 
    state_of_residence_source_value AS State,
    value_as_string AS Zip_code                
FROM `{dataset}.person_ext`
JOIN `{dataset}.observation` USING(person_id)
WHERE observation_source_concept_id = 1585250 
GROUP BY 2,3")


# Save states in a dataframe
tb <- bq_project_query(Sys.getenv('GOOGLE_PROJECT'), query)
df <-  bq_table_download(tb)
head(df)

Registered Tier State and Site Generalizations 

OBSERVATION TABLE 

For rows in the OBSERVATION table where observation_source_concept_id = 1585249 (observation_source_value = StreetAddress_PIIState) and either of the two conditions below are met, we generalize the value_source_concept_id (and the derived value_as_concept_id) of the row to 2000000011 (State information suppressed for privacy).

Condition 1: The value_source_concept_id for a given participant-reported state exists for less than 200 participants. States to be generalized under this condition are listed below.
PARTICIPANT SELECTION (Original value_source_concept_id) PARTICIPANT SELECTION (PMI Code) Generalized value_source_value
1585262 PIIState_AK 2000000011
1585263 PIIState_AS 2000000011
1585270 PIIState_DE 2000000011
1585271 PIIState_FM 2000000011
1585274 PIIState_GU 2000000011
1585275 PIIState_HI 2000000011
1585284 PIIState_MH 2000000011
1585299 PIIState_MP 2000000011
1585291 PIIState_MT 2000000011
1585300 PIIState_ND 2000000011
1585306 PIIState_PR 2000000011
1585304 PIIState_PW 2000000011
1585307 PIIState_RI 2000000011
1585309 PIIState_SD 2000000011
1585315 PIIState_VI 2000000011
1585313 PIIState_VT 2000000011
1585409 PIIState_WV 2000000011
1585411 PIIState_WY 2000000011

 

Condition 2: The value_source_concept_id for the participant-reported state does not match the state of the site providing the participant's EHR; this is defined as eligible OBSERVATION rows for which the mapping table contains an src_hpo_id AND the combination of value_source_concept_id for the observation row and src_hpo_id differs from the below:
PARTICIPANT SELECTION (Original value_source_concept_id) PARTICIPANT SELECTION (PMI Code) Original src_hpo_id Generalized value_source_value
1585261 PIIState_AL saou_uab 2000000011
1585264 PIIState_AZ uamc_banner 2000000011
1585266 PIIState_CA cpmc_ceders 2000000011
1585266 PIIState_CA cpmc_ucd 2000000011
1585266 PIIState_CA cpmc_uci 2000000011
1585266 PIIState_CA cpmc_ucsd 2000000011
1585266 PIIState_CA cpmc_usc 2000000011
1585266 PIIState_CA syhc 2000000011
1585268 PIIState_CT chci 2000000011
1585272 PIIState_FL seec_ufl 2000000011
1585273 PIIState_GA seec_emory 2000000011
1585273 PIIState_GA seec_morehouse 2000000011
1585277 PIIState_IL ipmc_northshore 2000000011
1585277 PIIState_IL ipmc_nu 2000000011
1585277 PIIState_IL ipmc_rush 2000000011
1585277 PIIState_IL ipmc_uchicago 2000000011
1585277 PIIState_IL ipmc_uic 2000000011
1585286 PIIState_MA nec_bmc 2000000011
1585286 PIIState_MA nec_phs 2000000011
1585287 PIIState_MI tach_hfhs 2000000011
1585287 PIIState_MI trans_am_spectrum 2000000011
1585289 PIIState_MS jhchc 2000000011
1585297 PIIState_NY hrhc 2000000011
1585297 PIIState_NY nyc_cornell 2000000011
1585297 PIIState_NY nyc_cu 2000000011
1585297 PIIState_NY nyc_hh 2000000011
1585305 PIIState_PA pitt 2000000011
1585308 PIIState_SC ecchc 2000000011
1585310 PIIState_TN chs 2000000011
1585410 PIIState_WI aouw_mcri 2000000011
1585410 PIIState_WI aouw_mcw 2000000011
1585410 PIIState_WI aouw_uwh 2000000011

For additional information on how All of Us protects participant privacy by means of data removal, suppression, or generalization, click here.

 

Was this article helpful?

0 out of 0 found this helpful

Have more questions? Submit a request

Comments

0 comments

Article is closed for comments.