Athena releases and nightly builds

Release Numbers

When a particular build of an Athena project is deemed ready for production it will be installed as a numbered release onto the ATLAS production CVMFS server.

ATLAS software is numbered according to the following scheme:

A.B.X[.Y]

• A refers to the release series, e.g., release series 23 is used for Run 3 simulation and data taking in 2023

  Note

  The fact that the release number matches the data taking year is purely coincidental, and does not represent a convention.

• B is the release flavour and usually corresponds to a particular branch in the git repository for which the code base needs to be different. e.g., 23.0 is the release series for MC23 simulation, and 2023 Tier-0 reconstruction & Trigger, while 25.2 is the release series for analysis.

  Principal release flavours are:

  Flavour	Purpose
  A.0.X	Tier-0 reconstruction, trigger, and corresponding simulation production
  A.2.X	Derivations and Analysis
  A.6.X	Event Generation

  Note

  The numbering scheme reflects historical releases where a larger number of different flavours were created. These have been consolidated since release 22.0 resulting in fewer projects, but the numbering for those that remain was kept the same for consistency.

  Note that while there is a strong correspondence between release flavour and Athena project, it is not absolute: e.g., AthDerivation and AthAnalysisBase would both be built from flavour 2.

• X is the major release number, monotonically increasing as code is developed with bugs fixed and enhancements added.

• Occasionally, Y will be used is a minor release number is required for developments that branched from an older major release. e.g., 21.0.20.Y contained any fixes required for MC16a, as releases 21.0.21 onwards contained incompatible changes for MC16c. Most releases will not have a minor release number, and so A.B.X is fully sufficient.

Setting up a Release

To setup a production release the release number and project need to be specified to asetup, e.g.,

asetup Athena,23.0.11

See the full asetup users guide for further options.

Git Branches

As explained above numbered releases of Athena projects are built from specific branches in the git repository.

We try to minimise the number of branches and try to avoid divergence between those branches that do exist. In particular, bug-fix merge requests targeting the frozen production branch are always merged ("swept") into the main branch. This does not apply in the opposite direction, so MRs targeting main are not generally swept into the frozen production branch. This allows long-term developments to be applied without affecting ongoing operations.

In the event that two branches do diverge such that they can never be merged in the future, merge requests which should target both branches can be propagated via Cherry Picks. Once a Merge Request is accepted, the code diff applied to the target branch by the Merge Request can be isolated and a new commit created which applies this diff to another branch. This is called a Cherry-Pick and allows for code to be shared between branches whose development has diverted such that they can never be merged together in the future, or when a change is needed quickly in another branch.

At key points in the life of the experiment, a new frozen production branch is cloned from the main branch. In recent years this has happened annually, a few months before a new data taking year begins. At this point, the main release number for releases built from the new frozen branch is incremented by one, and releases built from the main branch are given a number one higher than that. The old frozen branch is then discontinued.

Finding Releases and Release Information

It is very easy to see which production releases are available, just by looking in CVMFS, e.g.,

$ ls /cvmfs/atlas.cern.ch/repo/sw/software/23.0/Athena
23.0.0  23.0.1  23.0.10  23.0.11  23.0.2  23.0.3  23.0.4  23.0.5  23.0.6  23.0.7  23.0.8  23.0.9

Where the release series and flavour are encoded in the highest level directory, then the project, then the release number (/cvmfs/atlas.cern.ch/repo/sw/software/A.B/PROJECT/RELEASE_NUMBER).

Release Notes

When a git tag is made for a production release the release coordinator will add some release notes to the tag, which you can find directly in the GitLab Releases page or through the git command line:

$ git show release/23.0.11
tag release/23.0.11
Tagger: Tadej Novak <tadej.novak@cern.ch>
Date:   Wed Dec 7 13:27:09 2022 +0100

Release for derivations production and physics validation
commit 115629ba8b6a28ad16c5c09dff994a3c43339faf (tag: release/23.0.11, tag: release/22.6.26, tag: nightly/master/2022-12-07T0501, tag: nightly/master/2022-12-07T0313, tag: nightly/master/2022-12-07T0220, tag: nightly/master/2022-12-07T0001, tag: nightly/master/2022-12-06T2300, tag: nightly/master/2022-12-06T2101, tag: nightly/master/2022-12-06T2001, tag: nightly/master/2022-12-06T2000)
Merge: e823350ccd3 7681d5affec
...

Mapping Releases to Nightly Builds

If you need to find out which nightly build (see below) corresponded to a particular release, you can check the release notes (e.g. 23.0.11). Alternatively, the ReleaseData file contains useful information (see the date row):

$ cat /cvmfs/atlas.cern.ch/repo/sw/software/23.0/Athena/23.0.11/InstallArea/x86_64-centos7-gcc11-opt/ReleaseData
[release_metadata]
release:23.0.11
nightly name:master
project name:Athena
nightly release:115629ba8b6
date:2022-12-06T2101
compiler:GNU-11.2.0
cuda:NVIDIA-11.7.99
cxxpath:/cvmfs/sft.cern.ch/lcg/releases/gcc/11.2.0-8a51a/x86_64-centos7/bin/g++
cudapath:/cvmfs/sft.cern.ch/lcg/releases/LCG_102b_ATLAS_6/cuda/11.7.1/x86_64-centos7-gcc11-opt/bin/nvcc
cmake:3.21.3

You can also search for references with the right tag or hash in git using git describe:

$ git describe --tags --match 'release/*' nightly/22.0/2022-11-01T2101
release/22.0.101

$ git describe --tags --match 'nightly/*' release/22.0.101            
nightly/22.0/2022-11-01T2101

$ git describe --tags --match 'nightly/*' 5a344df38c0
nightly/22.0/2022-11-01T2101

Nightly Builds

For all of the major branches in git, significant projects will always be built from the HEAD once a day. Colloquially these are known as nightly builds, but they might well happen during daylight hours.

Each 'nightly' is date stamped with YYYY-MM-DDTHHMM (e.g. 2025-03-24T2101), which corresponds to the date and time when the git branch for the nightly build was tagged (e.g. 24th of March 2025 at 21:01 CERN local time). The git tag is created with the format nightly/BRANCH/YYYY-MM-DDTHHMM (e.g. nightly/main/2025-03-24T2101). All available tags can be found on the GitLab Tags page.

The results of a nightly build (cmake configuration, build status and tests) is accessible via the ATLAS Nightlies and CI Global Page.

After the release has been built it will be installed on the nightlies CVMFS server for use.

These nightly builds are usually used as a base on which to develop code changes, so knowing how to set them up is important.

Setting up a Nightly Release

To setup a nightly release it is necessary to give the git branch, the project and the timestamp to asetup, e.g.,

asetup 21.0,Athena,r2017-04-26T0710

Note that the datestamp should be prefixed by r (nightly release).

To make life easier asetup will accept several shorthand notations for the datestamp:

Form	Example	Comment
rYYYY-MM-DD	r2017-04-26	Default is to take the oldest time
rMM-DD	r04-26	No year means this year
rDD (zero padded)	r26 or r01	No month or year means this month, this year
latest		The most recent nightly release

See the full asetup users guide for further details.

To see which nightly releases are installed it's possible just to browse CVMFS:

$ ls /cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt
2024-05-06T2101  2025-01-04T2101  2025-01-12T2101  2025-01-20T2101  2025-01-28T2101
2024-12-28T2101  2025-01-05T2101 ...

shows all the nightly builds available for the main branch.

The project is installed within the datestamped directory, along with supporting code (AthenaExternals, Gaudi and some installation logs).

For branches that may build more than one project the timestamps for each project are probably different. The abbreviated asetup commands will handle this smoothly, but do watch out for it if you are directory browsing.

Release Candidate Numbers

Each nightly build is also a release candidate for the next release number for its project. i.e. if the last release of Tier-0 Athena was 21.0.26 then the nightly build will have a release candidate number of 21.0.27.

Release candidate numbers are stored in the version.txt file in the git repository itself, e.g. this is the release candidate number for MC16 and Tier-0 2017 data taking.

The release candidate number is also visible in the cvmfs nightly installation, e.g.:

$ ls /cvmfs/atlas-nightlies.cern.ch/repo/sw/main_Athena_x86_64-el9-gcc13-opt/2025-01-02T2101/Athena
25.0.24

But as explained above it isn't actually needed for setting up a nightly build.

Release History

23.0

Release 23.0 is the release for 2023 data taking, trigger, reprocessing, and MC production. It is a largely technical update with respect to 22.0, in order to provide a stable release through to the end of Run3, and provides full backwards-compatibility with outputs produced in that release. Switching between 22.0 and 23.0 should therefore require only minor adaptations if any.

Some key features are:

• Use of HEPMC3 and 64-bit event numbers, allowing for compatibility with future generator versions and larger event generation files to be produced, respectively.
• Updated the LCG version (LCG_102b) used for providing common libraries such as ROOT, allowing for future compatibility with Centos9s/AlmaLinux9/RHEL9 operating systems.
• Extensive availability of ComponentAccumulator-based job configuration.

AOD files produced in 23.0 can be used to produce derivations in 24.0-series releases (see the description in earlier section).

22.0

Release 22.0 was the release used for initial Run3 data taking, trigger, reprocessing, and MC production in 2022. It represents the culmination of a multi-year development program:

• It was the first Athena release to fully support AthenaMT for multi-threaded event processing
• It also features numerous other improvements to reconstruction, including significant speed-ups of Inner Detector tracking allowing inclusion of dedicated tracking for long-lived particles as part of the standard chain
• It includes reconstruction and simulation of the Muon New Small Wheel
• Additionally, it is the first release series where Run4 detectors can be simulated and reconstructed in the same release as the current detectors.

AOD files produced in 22.0 can be used to produce derivations in 23.0-series releases (up to 23.0.11).

21.0

Release 21.0 was the release for Run2 data taking, trigger, reprocessing, and MC production. It was the release series in which the current CMake-based build system, and gitlab-based code versioning, were first used.

Remark

Future reprocessings of Run2 data, and associated MC productions, will be performed in 23.0 (or newer).

Releases prior to 21.0 used an entirely different release building and versioning infrastructure, and so are not covered here.