Sept. 29, 2021

Oregon State Physics is leading a Department of Energy Office of Science funded project to design computing and software infrastructure for the DUNE experiment.   DUNE is a future neutrino experiment that will aim a neutrino beam from Fermilab, in Batavia Illinois, at a very large detector in the Homestake mine in Lead, South Dakota.  The experiment is currently under construction with a 5% prototype running at CERN in 2018 and 2022 and the full detector expected in 2029. These experiments generate data at rates of 1-2 GB/sec, or 30 PB/year which must be stored, processed and distributed to over 1,000 scientists worldwide.

The LBNF neutrino beam traveling (right to left) from Fermilab in Illinois to the Sanford Lab in South Dakota

The project “Essential Computing and Software Development for the DUNE experiment” is funded for 3M$ over 3 years, shared among 4 Universities (Oregon State, Colorado State, Minnesota and Wichita State) and three national laboratories (Argonne National Laboratory, Fermi National Laboratory and Brookhaven National Laboratory). The collaborators will work with colleagues worldwide on advanced data storage systems, high performance computing and databases in support of the DUNE physics mission.  See https://www.dunescience.org/ for more information on the experiment.

PI Heidi Schellman (Oregon State Physics) leads the DUNE computing and software consortium which is responsible for the international DUNE computing project. Physics graduate student Noah Vaughan helps oversee the global grid processing systems that DUNE uses for data reconstruction and simulation and recent graduate Amit Bashyal helped design the DUNE/LBNF beamline.  Graduate student Sean Gilligan is performing a statistical analysis of data transfer patterns to help optimize the design of the worldwide data network.  Postdoc Jake Calcutt recently joined us from Michigan State University and is designing improved methods for producing data analysis samples for the ProtoDUNE experiment at CERN.

One of the major thrusts of the Oregon State project is the design of robust data storage and delivery systems optimized for data integrity and reproducibility.  30 PB/year of data will be distributed worldwide and processed through a complex chain of algorithms. End users need to know the exact provenance of their data –  how was it produced, how was it processed, was any data lost – to ensure scientific reproducibility over the decades that the experiments will run.  Preliminary versions of the data systems have already led to results from the protoDUNE prototype experiments at CERN which are described in https://doi.org/10.1088/1748-0221/15/12/P12004 and https://doi.org/10.1051/epjconf/202024511002.

As an example of this work, three Oregon State Computer Science Majors (Lydia Brynmoor, Zach Lee and Luke Penner) worked with Fermilab scientist Steven Timm on a global monitor for the Rucio storage system shown below. This illustrates test data transfers between compute sites in the US, Brazil and Europe. The dots indicate compute sites in the DUNE compute grid while the lines illustrate test transfers. 

Data transfer diagram for DUNE data.

Other projects will be a Data Dispatcher which optimizes the delivery of data to CPU’s across the DUNE compute systems and monitoring of data streaming between sites.

Amit Bashyal has defended his thesis. You can read all 550 pages here: http://old.inspirehep.net/record/1859505

Done!

His thesis combines his work on neutrino fluxes for the DUNE and MINERvA experiments, which has been submitted to JINST and a new high statistics measurement of anti-neutrino quasi-elastic scattering which we’re writing up.

He is moving to Argonne National Laboratory where he will be working on the HCP-CCE project and the DUNE collaboration.

Mateus Carneiro in the neutrino lab

Postdoc Mateus Carneiro has moved on to a position at Brookhaven National Laboratory but he left us with a really nice paper in Physics Review Letters.

M.Carneiro et al. [MINERvA], “High-Statistics Measurement of Neutrino Quasielasticlike Scattering at 6 GeV on a Hydrocarbon Target”, Phys. Rev. Lett. 124, no.12, 121801 (2020), doi:10.1103/PhysRevLett.124.121801

Graduate Student Amit Bashyal did many of the cross-checks for this complicated measurement and wrote a summary for FermiNews which is copied below. Amit’s thesis topic is the parallel measurement for anti-neutrinos.

Playing pool with neutrinos

Para una versión en español, haga clic aquí. Para a versão em português, clique aqui.

Hard to believe you can play pool with neutrinos, but certain neutrino interaction events are closer to the game than you think.

In these charged-current quasielastic interactions — let’s call them CCQE interactions for short — a neutrino strikes a particle in an atom’s nucleus — a proton or a neutron. Two particles emerge from the collision. One is a muon, a heavier cousin of the electron. The other is either a proton (if the stationary particle is a neutron) or a neutron (if the stationary particle is a proton).

The neutrino interactions that result from these quasielastic reactions are like the collisions between balls in a game of pool: You can guess the energy of the incoming neutrino by measuring the direction and energy of only one of the outgoing particles, provided you know the types of all four particles that were in the interaction in the first place and the original direction of the neutrino.

CCQE interactions are an important interaction mode of neutrinos in current and future neutrino oscillation experiments, such as the international Deep Underground Neutrino Experiment, hosted by Fermilab.

They are similar to the elastic interactions every pool player knows except in one important way: The weak nuclear force allows the particles to change from one kind into another, hence the “quasielastic” name. In this subatomic pool game, the cue ball (neutrino) strikes a stationary red ball (proton), which emerges from the collision as an orange ball (neutron).

This display shows a CCQE-like event reconstructed in the MINERvA detector. Image: MINERvA

Since most modern neutrino experiments use targets made of heavy nuclei ranging from carbon to argon, nuclear effects and correlations between the neutrons and protons inside the nucleus can cause significant changes in the observed interaction rates and modifications to the estimated neutrino energy.

At MINERvA, scientists identify the CCQE interactions by a long muon track left in the particle detector and potentially one or more proton tracks. However, this experimental signature can sometimes be produced by non-CCQE interactions due to nuclear effects inside the target nucleus. Similarly, nuclear effects can also modify the final-state particles to make a CCQE event look like a non-CCQE event and vice versa.

Since nuclear effects can make it challenging to identify a true CCQE event, MINERvA reports measurements based on the properties of the final-state particles only and calls them CCQE-like events (since they will have contributions from both true CCQE and non-CCQE events). A CCQE-like event is one that has at least one outgoing muon, any number of protons or neutrons, and no mesons as final-state particles. (Mesons, like protons and neutrons, are made of quarks. Protons and neutrons have three quarks; mesons have two.)

MINERvA has measured the likelihood of CCQE-like neutrino interactions using Fermilab’s medium-energy neutrino beam, with the neutrino flux peaking at 6 GeV. Compared to MINERvA’s earlier measurements, which were conducted with a low-energy beam (3 GeV peak neutrino flux), this measurement has the advantage of a broader energy reach and much larger statistics: 1,318,540 CCQE-like events compared to 109,275 events in earlier low-energy runs.

MINERvA made these CCQE interaction probability measurements as a function of the square of the momentum transferred by the neutrino to the nucleus, which scientists denote as Q2. The plot shows discrepancies between the data and most predictions in low-Q2 and high-Q2 regions. By comparing MINERvA’s measurement with various models, scientists can refine them and better explain the physics inside the nuclear environment.

This plot shows the ratio of cross-section as a function of Q2 of data and various predictions with respectto one commonly used interaction model. Image: MINERvA

MINERvA has also made more detailed measurements of the probability of neutrino interaction based on the outgoing muon’s momentum. They take into account the muon’s momentum both in the direction of the incoming neutrino’s trajectory and in the direction perpendicular to its trajectory. This work helps current and future neutrino experiments understand their own data over a wide range of muon kinematics.

Mateus Carneiro, formerly of the Brazilian Center for Research in Physics and Oregon State University and now at Brookhaven National Laboratory, and Dan Ruterbories of the University of Rochester were the main drivers of this analysis. The results were published in Physical Review Letters.

Amit Bashyal is an Oregon State University scientist on the MINERvA experiment.

This work is supported by the DOE Office of Science, National Science Foundation, Coordination for the Improvement of Higher Education Personnel in Brazil, Brazilian National Council for Scientific and Technological Development, Mexican National Council of Science and Technology, Basal Project in Chile, Chilean National Commission for Scientific and Technological Research, Chilean National Fund for Scientific and Technological Development, Peruvian National Council for Science, Technology and Technological Innovation, Research Management Directorate at the Pontifical Catholic University of Peru, National University of Engineering in Peru, Polish National Science Center and UK Science and Technology Facilities Council.

Fermi National Accelerator Laboratory is supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit science.energy.gov.

Amit Bashyal (far right) at the Neutrino summer school in Illinois.
Amit Bashyal (far right) at the Neutrino summer school in Illinois.

5th year grad student Amit Bashyal’s team won a group presentation prize at the International Neutrino Summer School for their study of magnetization of DUNE experiment far detectors. Students were divided into groups given a short time to do an original study and then present it! Congratulations Amit and team!

3 other Schellman group members and alumni are pictured. Laura Fields (Fermilab, 6th from left) and Cheryl Patrick (UCL, 7th from left) were lecturers at the summer school and 2nd year student Sean Gilligan (3rd from right) also attended.

Differential QE-like cross section dσ(E_QEν)/dQ^2_QE, in bins of E_QEν. Inner error bars show statistical uncertainties; outer error bars show total (statistical and systematic) uncertainty. The red histogram shows the MINERvA-tuned GENIE model used to estimate smearing and acceptance.

The much anticipated paper version of Cheryl Patrick’s thesis has been accepted by Physical Review D.   Check it out at:

http://inspirehep.net/record/1646253?ln=en

The data release is available at

http://physics.oregonstate.edu/~schellmh/data_release/qelike.html

Update:  It is published in Phys. Rev. D which is now open access:

https://journals.aps.org/prd/abstract/10.1103/PhysRevD.97.052002

Cheryl Patrick successfully defended her thesis:

Measurement of the Antineutrino Double-Differential Charged-Current Quasi-Elastic Scattering Cross Section at MINERvA in March and is now a postdoc on SuperNEMO at University College London.

Heidi Schellman, Cheryl Patrick PhD and Laura Fields
Heidi Schellman, Cheryl Patrick PhD and Laura Fields. Cheryl is wearing the official PhD hat.

She came back to the US to give a fantastic Fermilab Wine and Cheese talk in June 2016 which has been written up in Fermi NewsWatch the video!

Amit Bashyal joined the OSU neutrino group in September 2015 after getting his undergraduate degree at the University of Texas at Arlington working with Jaehoon Yu.  Prior to coming to OSU he was an International Fellow at Fermilab for a year working with Laura Fields and Alberto Marchionni on physics studies for the DUNE/LBNF neutrino beamline design.

Graduate student Amit Bashyal
Graduate student Amit Bashyal

Tim Andeen received his doctorate for work on the D0 experiment at Fermilab in 2008.  His thesis, `Measurement of the W Boson Mass with the D0 Run II Detector using the Electron PT Spectrum’ used a precision measurement of the W boson mass to make tight constraints on the mass of the Higgs boson, several years before the Higgs was finally discovered.  He then went to Columbia University as a postdoc and research associate on the ATLAS experiment at CERN.  He will start as an Assistant Professor of Physics at the University of Texas, Austin in Fall 2015.

Tim at Columbia.
Tim at Columbia.

The W boson mass squeezing the Higgs boson.
The W boson mass squeezing the Higgs boson.  Particles from the Particle Zoo.