HP-CONCORD in FY 2021 Budget Request to Congress

In FY 2021 ASCR budget requests to the Congress, HP-CONCORD appeared as one of the program accomplishments. This is a part of the full budget request made by the Office of Science at Department of Energy

The following is from page 27:

HP-CONCORD Enables Machine Learning in HPC at Unprecedented Scale.
Understanding the interactions between millions of variables such as how a disease may be caused by a subset of the human genes is among the most challenging problems in data-driven science. A powerful machine learning algorithm called CONCORD can identify these relationships however until recently it could only be applied to modest-sized data sets. Researchers from LBNL supported by SciDAC along with their collaborators from Google Brain, University of California Santa Barbara and Carnegie Mellon University developed a high-performance computing version of the algorithm called HP-CONCORD. Using NERSC, they then demonstrated this parallel algorithm on an enormous set of data from the Human Connectome Project, which computed estimates for about 4 billion parameters, and an even larger demonstration problem with over 800 billion parameters. The HP-CONCORD team used functional magnetic resonance imaging data to estimate the underlying conditional dependency structure of the brain and then used the resulting estimate to automatically identify functional regions of the brain. The researchers expect that many other science areas will benefit from HP-CONCORD in the future: In biology it can be used to reconstruct gene regulatory networks, or in environmental sciences it can help in estimating temperature-to-environmental-proxy relationships. HP-CONCORD may also be useful for hypothesis generation in exploratory data analysis to guide further experimental study.

There were a lot of budget cuts, but Computing Research Association reports that ASCR faired reasonably well.

DOE SC Total $6.59B $7.00B $5.84B -$1.16B -17%
ASCR $936M $980M $988M +$8M +1.0%
ARPA-E $366M $425M $0 -$425M -100%


Interactive Tutorial with learnr and Binder

1 minute read

Recently, a colleague pointed out that they would like to host R tutorial created with learnr package for their students. In this post, I describe a way to h...

Back to top ↑


Central Coast Data Science Partnership

less than 1 minute read

We received an NSF grant to train a new generation of data scientists through Central Coast Data Science Partnership! I feel privileged to be a part of the U...

Back to top ↑


Back to top ↑