Jupyter Notebook Upgrade - 8.09.2021

The All of Us Researcher Workbench’s Jupyter Notebook server was updated on August 9, 2021 to include upgrades to Jupyter, Python packages and R libraries. Note that these package updates are only the defaults; you may still choose to modify your runtime by installing new packages or changing package versions from within the notebooks housed in your workspaces. We do not anticipate that this update will have a major impact on standard Python or R notebook code, but it is possible that some existing code may need to be updated to account for the changes. Please contact support@researchallofus.org for help if you experience a notebook breakage and are unable to resolve it. See detailed changes below. 

Timing Upgrade

Jupyter Notebook servers will be upgraded on a rolling basis from August 9th through August 23rd  (e.g. while using your notebook during this timeframe, the change may not yet be in effect). If you would like to receive the upgrade to your notebook immediately, you can open the Cloud Analysis Environment panel from the sidebar in your workspace and click the “Delete Environment” button. Create a new runtime or open a notebook afterwards to receive the update.

To confirm you received the upgrade, in Jupyter you can check the menu under Kernel > Change kernel. Here you should now see 4 options:

• Python3
• Python [conda env:root] *
• R
• R [conda env:root] *

Note: these two new “conda env” kernels are identical to their Python and R counterparts. Eventually the Workbench may support alternate kernels managed by conda - but for now, only the framework is in place.

Detailed changes

git support

• nbstripout is installed and enabled by default for all git repositories cloned to the machine to ensure that notebook outputs are removed prior to committing notebooks to source control.
• A global gitignore has been configured so that by default only code and documentation files (e.g. *.ipynb, *.py, *.r, *.sh, *.md) can be committed to git repositories.

python

Packages added:

• ansiwrap (0.8.4)
• anyio (3.2.0)
• appdirs (1.4.4)
• backports.functools-lru-cache (1.6.4)
• beatrix-jupyterlab (0.0.3)
• black (21.5b2)
• blinker (1.4)
• Bottleneck (1.3.2)
• brotlipy (0.7.0)
• caip-notebooks-serverextension (1.0.0)
• cloudpickle (1.6.0)
• colorama (0.4.4)
• conda (4.9.2)
• conda-package-handling (1.7.3)
• docker (5.0.0)
• docker-pycreds (0.4.0)
• fsspec (2021.6.0)
• gitdb (4.0.7)
• GitPython (3.1.18)
• google-api-python-client (2.9.0)
• google-auth-httplib2 (0.1.0)
• google-cloud-appengine-logging (0.1.1)
• google-cloud-audit-log (0.1.0)
• google-cloud-bigquery-storage (2.4.0)
• google-cloud-bigtable (2.2.0)
• google-cloud-dataproc (2.4.0)
• google-cloud-firestore (2.1.3)
• google-cloud-kms (2.3.0)
• google-cloud-language (2.1.0)
• google-cloud-logging (2.5.0)
• google-cloud-monitoring (2.3.0)
• google-cloud-pubsub (1.7.0)
• google-cloud-scheduler (2.2.0)
• google-cloud-spanner (3.5.0)
• google-cloud-speech (2.4.0)
• google-cloud-tasks (2.3.0)
• google-cloud-translate (3.2.0)
• google-cloud-videointelligence (2.2.0)
• google-cloud-vision (2.3.2)
• greenlet (1.1.0)
• grpc-google-iam-v1 (0.12.3)
• grpcio-gcp (0.2.2)
• ipython-sql (0.3.9)
• jmespath (0.10.0)
• json5 (0.9.5)
• jupyter-http-over-ws (0.0.8)
• jupyter-server (1.8.0)
• jupyter-server-mathjax (0.2.3)
• jupyterlab-git (0.11.0)
• kubernetes (17.17.0)
• llvmlite (0.36.0)
• multimethod (1.4)
• nb-conda (2.2.1)
• nb-conda-kernels (2.3.1)
• nbdime (3.1.0)
• notebook-executor (0.2)
• numba (0.53.1)
• olefile (0.46)
• papermill (2.3.3)
• pathspec (0.8.1)
• prettytable (2.1.0)
• pycosat (0.6.3)
• pydantic (1.8.2)
• PySocks (1.7.1)
• pyu2f (0.1.5)
• regex (2021.4.4)
• retrying (1.3.3)
• ruamel-yaml-conda (0.15.80)
• simplejson (3.17.2)
• smmap (3.0.5)
• sniffio (1.2.0)
• SQLAlchemy (1.4.18)
• sqlparse (0.4.1)
• tenacity (7.0.0)
• tensorflow (2.4.2)
• textwrap3 (0.9.2)
• toml (0.10.2)
• ujson (4.0.2)
• Unidecode (1.2.0)
• uritemplate (3.0.1)
• websocket-client (0.57.0)
• whichcraft (0.6.1)

Packages removed:

• jupyter
• jupyter-console
• keyring
• keyrings.alt
• pycrypto
• pygobject
• python-apt
• pyxdg
• qtconsole
• QtPy
• readline
• SecretStorage
• tensorflow-cpu
• unattended-upgrades
• Packages changed:
• absl-py: 0.12.0 -> 0.13.0
• asn1crypto: 0.24.0 -> 1.4.0
• astroid: 2.5.6 -> 2.6.1
• attrs: 20.3.0 -> 21.2.0
• biopython: 1.78 -> 1.79
• certifi: 2020.12.5 -> 2021.5.30
• cftime: 1.4.1 -> 1.5.0
• click: 7.1.2 -> 8.0.1
• cookiecutter: 1.7.2 -> 1.7.0
• dill: 0.3.3 -> 0.3.4
• firecloud: 0.16.25 -> 0.16.31
• google-api-core: 1.26.3 -> 1.30.0
• google-auth: 1.30.0 -> 1.30.2
• google-cloud-bigquery: 2.15.0 -> 2.20.0
• google-cloud-bigquery-datatransfer: 3.1.1 -> 3.2.0
• google-cloud-core: 1.6.0 -> 1.7.0
• google-cloud-datastore: 2.1.2 -> 2.1.3
• google-cloud-storage: 1.31.2 -> 1.25.0
• grpcio: 1.32.0 -> 1.38.0
• idna: 2.6 -> 2.8
• importlib-metadata: 4.0.1 -> 4.5.0
• intel-openmp: 2021.2.0 -> 2021.3.0
• ipykernel: 5.5.4 -> 5.5.5
• ipython: 7.23.1 -> 7.24.1
• isort: 5.8.0 -> 5.9.1
• Jinja2: 2.11.3 -> 3.0.1
• jupyterlab: 0.35.4 -> 1.2.16
• jupyterlab-server: 0.2.0 -> 1.2.0
• libcst: 0.3.18 -> 0.3.19
• MarkupSafe: 1.1.1 -> 2.0.1
• matplotlib: 3.4.1 -> 3.4.2
• mkl: 2021.2.0 -> 2021.3.0
• nbconvert: 6.0.7 -> 5.6.1
• nbstripout: 0.4.0 -> 0.5.0
• netCDF4: 1.5.6 -> 1.5.7
• networkx: 2.5.1 -> 2.5
• notebook: 6.1.1 -> 6.4.0
• oauthlib: 3.1.0 -> 3.1.1
• pandas-profiling: 2.12.0 -> 3.0.0
• pandocfilters: 1.4.3 -> 1.4.2
• pip: 21.1.1 -> 21.1.3
• prometheus-client: 0.10.1 -> 0.11.0
• prompt-toolkit: 3.0.18 -> 3.0.19
• protobuf: 3.15.8 -> 3.16.0
• pyarrow: 3.0.0 -> 4.0.1
• pyasn1-modules: 0.2.8 -> 0.2.7
• pypandoc: 1.5 -> 1.6.3
• pyzmq: 22.0.3 -> 22.1.0
• setuptools: 56.1.0 -> 57.0.0
• six: 1.15.0 -> 1.16.0
• tangled-up-in-unicode: 0.0.7 -> 0.1.0
• tbb: 2021.2.0 -> 2021.3.0
• terminado: 0.9.4 -> 0.10.1
• terra-notebook-utils: 0.7.0 -> 0.8.1
• testpath: 0.4.4 -> 0.5.0
• tornado: 5.1.1 -> 6.1
• typing-extensions: 3.7.4.3 -> 3.10.0.0
• typing-inspect: 0.6.0 -> 0.7.1
• visions: 0.6.0 -> 0.7.1
• Werkzeug: 1.0.1 -> 2.0.1
• xarray: 0.17.0 -> 0.18.2

r

Packages changed:

• Biobase: 2.50.0 -> 2.52.0
• BiocGenerics: 0.36.1 -> 0.38.0
• BiocManager: 1.30.12 -> 1.30.16
• BiocVersion: 3.12.0 -> 3.13.1
• Biostrings: 2.58.0 -> 2.60.1
• broom: 0.7.6 -> 0.7.8
• bslib: 0.2.4 -> 0.2.5.1
• cachem: 1.0.4 -> 1.0.5
• cli: 2.5.0 -> 3.0.0
• colorspace: 2.0-1 -> 2.0-2
• cpp11: 0.2.7 -> 0.3.1
• curl: 4.3.1 -> 4.3.2
• devtools: 2.4.1 -> 2.4.2
• DNAcopy: 1.64.0 -> 1.66.0
• dplyr: 1.0.6 -> 1.0.7
• fansi: 0.4.2 -> 0.5.0
• fitdistrplus: 1.1-3 -> 1.1-5
• gargle: 1.1.0 -> 1.2.0
• gdsfmt: 1.26.1 -> 1.28.0
• GENESIS: 2.20.1 -> 2.22.2
• GenomeInfoDb: 1.26.7 -> 1.28.1
• GenomeInfoDbData: 1.2.4 -> 1.2.6
• GenomicRanges: 1.42.0 -> 1.44.0
• gert: 1.3.0 -> 1.3.1
• ggplot2: 3.3.3 -> 3.3.5
• googledrive: 1.0.1 -> 2.0.0
• gtools: 3.8.2 -> 3.9.2
• GWASTools: 1.36.0 -> 1.38.0
• hms: 1.0.0 -> 1.1.0
• httpuv: 1.6.0 -> 1.6.1
• IRanges: 2.24.1 -> 2.26.0
• IRkernel: 1.1.1 -> 1.2
• leiden: 0.3.7 -> 0.3.8
• matrixStats: 0.58.0 -> 0.59.0
• mime: 0.10 -> 0.11
• parallelly: 1.25.0 -> 1.26.1
• pillar: 1.6.0 -> 1.6.1
• plotly: 4.9.3 -> 4.9.4.1
• quantreg: 5.85 -> 5.86
• quantsmooth: 1.56.0 -> 1.58.0
• r: 4.0.5 -> 4.1.0
• Rcpp: 1.0.6 -> 1.0.7
• RcppArmadillo: 0.10.4.0.0 -> 0.10.5.0.0
• remotes: 2.3.0 -> 2.4.0
• rmarkdown: 2.7 -> 2.9
• rversions: 2.0.2 -> 2.1.1
• S4Vectors: 0.28.1 -> 0.30.0
• SAIGEgds: 1.4.2 -> 1.6.0
• sass: 0.3.1 -> 0.4.0
• SeqArray: 1.30.0 -> 1.32.0
• SeqVarTools: 1.28.1 -> 1.30.0
• Seurat: 4.0.1 -> 4.0.3
• SeuratObject: 4.0.0 -> 4.0.2
• SNPRelate: 1.24.0 -> 1.26.0
• spatstat.core: 2.1-2 -> 2.2-0
• spatstat.geom: 2.1-0 -> 2.2-0
• spatstat.utils: 2.1-0 -> 2.2-0
• stringi: 1.5.3 -> 1.6.2
• testthat: 3.0.2 -> 3.0.4
• tibble: 3.1.1 -> 3.1.2
• tinytex: 0.31 -> 0.32
• xfun: 0.22 -> 0.24
• XVector: 0.30.0 -> 0.32.0
• zip: 2.1.1 -> 2.2.0
• zlibbioc: 1.36.0 -> 1.38.0