The All of Us Researcher Workbench team released an incremental update for the All by All tables related to the Personal and Family Health History (PFHH) and Phecode phenotypes to align with data processing and quality assurance updates.
The original dataset tables contained inconsistent case counts in the All by All analysis for the Personal and Family Health History (PFHH) and Phecode phenotypes.
Background
Personal and Family Health History (PFHH) phenotypes
The All by All tables as related to PFHH phenotypes included select instances where cases were incorrectly classified as controls. To address this, we’ve deprecated the results corresponding to the 80 total PFHH phenotypes.
Phecode phenotypes
The All by All tables related to Phecode analysis included select instances where the number of cases were not reported in the All by All results for the All by All Phecode analysis. To address this, we’ve deprecated approximately 1000 of the All by All Phecode results.
As a result of the incremental data release, we have deprecated the following:
- All Hail Matrix Tables with results for all phenotypes including PFHH and Phecode have been deprecated.
-
- The phenotype-specific Hail Tables for all other phenotypes (physical measurements, lab measurements, drugs, phecodeX) are still available for researcher use at the original file paths
-
- The phenotype-specific Hail Tables include the same results as the Hail Matrix Tables but in a different data format.
-
- The phenotype-specific Hail Tables for all other phenotypes (physical measurements, lab measurements, drugs, phecodeX) are still available for researcher use at the original file paths
-
- Phenotype-specific Hail Tables for PFHH and Phecode phenotypes have been deprecated.
For a full list of phenotypes, please see Overview of the All by All tables available on the All of Us Researcher Workbench.
Dataset deprecation details and considerations
The deprecated dataset table files remain accessible in the Researcher Workbench Controlled Tier dataset but have been moved to a new isolated folder at new file paths.
We do not recommend using what’s been deprecated for continued or new analysis. Instead, we recommend researchers take the following action(s):
- For researchers who are working with the All by All Phecode tables, we recommend utilizing the phenotype-specific Hail tables for PhecodeX results instead.
- For researchers who are working with the unaffected All by All results (physical measurements, lab measurements, drugs, phecodeX) in Hail Matrix Table format, we recommend working with the corresponding phenotype-specific Hail Tables.
- For researchers who are working with the All by All PFHH tables, we recommend waiting until the All by All results are updated using a future version of the CDR for most updated results.
Hail Matrix Table
The All by All result Hail MatrixTables use the naming scheme ${POP}_${TYPE}_results.mt. In total, there are 21 Hail Matrix Tables (18 single ancestry tables and three meta-analysis tables) impacted.
- POP specifies either single ancestry results or meta-analysis results (AFR, AMR, EAS, EUR, MID, SAS, META).
- TYPE specifies the type of association test (ACAF, exome, gene).
Original path: gs://fc-aou-datasets-controlled/AllxAll/v1/mt/
Deprecated path: gs://fc-aou-datasets-controlled/AllxAll/v1/mt-deprecated/
Hail Table
The All by All result Hail Tables for individual phenotypes use the naming scheme ${TYPE}/${POP}/phenotype_${PHENONAME}_${TYPE}_results.ht. In total, 9,948 Hail Tables containing the All by All PFHH and Phecode results were deprecated.
- POP specifies either single ancestry results or meta-analsysis results (AFR, AMR, EAS, EUR, MID, SAS, META).
- TYPE specifies the type of association test (ACAF, exome, gene).
- PHENONAME specifies the phenotype of interest by concept ID.
Original path: gs://fc-aou-datasets-controlled/AllxAll/v1/ht/
Deprecated path: gs://fc-aou-datasets-controlled/AllxAll/v1/ht-deprecated/
The above changes are also reflected in Overview of the All by All tables available on the All of Us Researcher Workbench.
Comments
0 comments
Article is closed for comments.