Trigger Configuration¶

The trigger configurations are stored in the trigger configuration database, described in section Trigger Configuration Database. The conditions database is used to keep track of which trigger configurations are used during data taking, mapping run and lumiblock number to configuration keys.

This section describes the recording, storage and retrieval of the keys during data taking and later during offline reconstruction.

To investigate which trigger items and chains were used to collect the ATLAS data, please check the section about the Trigger API.

Introduction¶

During data taking, the information about trigger menu, L1 and HLT prescale set, and L1 bunchgroups is recorded in the conditions database, by run and lumiblock number. Thereby only the trigger keys (SMK, L1PSK, HLTPSK and BGK) are stored, with those keys the full configuration can be looked up in the TriggerDB. That recording is done by the CTP, as the information which keys are used, including the HLTPSK, is known to the CTP.

During online data taking at P1 the SMK, and the initial prescale and bunchgroup keys are available in OKS and loaded at the time of the partition configuration. Prescales and bunchgroups can be updated later again during the run, and often are, when the luminosity changes.

During offline data reconstruction the trigger keys are retrieved from the conditions database based on the run and LB number of the event. They keys are used to load the full trigger configuration from the TriggerDB. The trigger configuration is then stored as metadata in ESD, AOD and DAOD pool files, so later stages of the reconstruction chain and physics analysis can access the trigger configuration from the data file without a need to access the TriggerDB.

Trigger keys in the conditions¶

The configuration keys and the name of the TriggerDB are stored in these conditions folders. Note that here the folder names are listed, in Crest these change to tag names, for instance /TRIGGER/HLT/HltConfigKeys becomes TRIGGERHLTHltConfigKeys-HEAD.

Conditions folders with trigger key information

Folder	Filled	Attribute	Information
`/TRIGGER/HLT/HltConfigKeys`	Once per run	`MasterConfigurationKey`	SMK
		`ConfigSource`	DB name and release
`/TRIGGER/LVL1/Lvl1ConfigKey`	LB when key changes	`Lvl1PrescaleConfigurationKey`	L1PSK
`/TRIGGER/HLT/PrescaleKey`	LB when key changes	`HltPrescaleKey`	HLTPSK
`/TRIGGER/LVL1/BunchGroupKey`	LB when key changes	`Lvl1BunchGroupConfigurationKey`	BGSK

Trigger configuration access in athena when reading bytestream files¶

There is no trigger configuration stored in the ATLAS bytestream files, therefor it is provided using the combined information from the conditions and trigger databases.

The L1 Menu and HLT menu are provided by the LVL1ConfigSvc.cxx and HLTConfigSvc.cxx. The HLTConfigSvc also provides the access to the HLT Monitoring. Those two services read the SMK at the start of the job, load the corresponding L1 Menu, HLT menu, and HLT monitoring from the TriggerDB and write them into the detector store from where they can be accessed:

Access to L1 menu, HLT menu, and HLT monitoring

MyAlg.hMyAlg.cxx

#include "TrigConfData/L1Menu.h"
#include "TrigConfData/HLTMenu.h"
#include "TrigConfData/HLTMonitoring.h"

class MyAlg : public AthReentrantAlgorithm {
    SG::ReadHandleKey<TrigConf::L1Menu> m_l1MenuKey{this,
        "L1TriggerMenu", "DetectorStore+L1TriggerMenu", "L1Menu in the DetectorStore"}; 
    SG::ReadHandleKey<TrigConf::HLTMenu> m_hltMenuKey{this,
        "HLTTriggerMenu", "DetectorStore+HLTTriggerMenu", "HLTMenu in the DetectorStore"};
    SG::ReadHandleKey<TrigConf::HLTMonitoring> m_HLTMonitoringKey {this, 
        "HLTMonitoringMenu", "DetectorStore+HLTMonitoringMenu", "HLT Monitoring in the DetectorStore"};
}

StatusCode MyAlg::initialize() {
    ATH_CHECK(m_l1MenuKey.initialize());
    ATH_CHECK(m_hltMenuKey.initialize());
    ATH_CHECK(m_HLTMonitoringKey.initialize());
}

StatusCode MyAlg::execute(const EventContext &ctx) const {
    SG::ReadHandle<TrigConf::L1Menu> l1MenuHandle = SG::makeHandle(m_l1MenuKey, ctx);
    SG::ReadHandle<TrigConf::HLTMenu> hltMenuHandle = SG::makeHandle(m_HLTMenuKey, ctx);
    SG::ReadHandle<TrigConf::HLTMonitoring> hltMonHandle = SG::makeHandle(m_HLTMonitoringKey, ctx);

    ATH_CHECK(l1MenuHandle.isValid());
    ATH_CHECK(hltMenuHandle.isValid());
    ATH_CHECK(hltMonHandle.isValid());

    // L1 menu: e.g. get the thresholds names
    const std::vector<std::string> & thresholds = l1MenuHandle->thresholdNames();

    // HLT menu: e.g. loop over all chains
    for (const Chain& loadedChain : *hltMenuHandle) {
        ...
    }

    // HLT monitoring: e.g. get the monitored signatures and targets
    std::vector<std::string> monSignatures = hltMonHandle->signatureNames();
    const std::set<std::string> & targets = hltMonHandle->targets();
}

Trigger prescales and bunchgroups access in athena¶

L1 prescales, HLT prescales, and L1Bunchgroups are provided by the corresponding conditions algorithms L1PrescaleCondAlg.cxx, HLTPrescaleCondAlg.cxx and BunchGroupCondAlg.cxx.

Those algorithms read the L1PSK, HLTPSK and BGK at each event, and if needed load the corresponding information from the TriggerDB and write them into the conditions store from where they can be accessed through a CondHandle.

Access to prescales and bunchgroups

L1 prescalesHLT prescalesL1 bunchgroups

MyAlg.h

#include "TrigConfData/L1PrescalesSet.h"
class MyAlg : public AthReentrantAlgorithm {
    SG::ReadCondHandleKey<TrigConf::L1PrescalesSet> m_l1pssKey{this,
        "L1Prescales", "L1Prescales", "L1 prescales set condition handle"};
}

MyAlg.cxx

StatusCode MyAlg::initialize() {
    ATH_CHECK(m_l1pssKey.initialize());
}

StatusCode MyAlg::execute(const EventContext &ctx) const {
    SG::ReadCondHandle<TrigConf::L1PrescalesSet> l1pssHandle(m_l1pssKey, ctx);
    ATH_CHECK(l1pssHandle.isValid());
    // get the prescale for item L1_eEM15
    const TrigConf::L1Prescale & ps = l1pssHandle->prescale("L1_eEM15");
}

MyAlg.h

#include "TrigConfData/HLTPrescalesSet.h"
class MyAlg : public AthReentrantAlgorithm {
    SG::ReadCondHandleKey<TrigConf::HLTPrescalesSet> m_hltpsKey{this,
        "HLTPrescales", "HLTPrescales", "HLT prescales set condition handle"};
}

MyAlg.cxx

StatusCode MyAlg::initialize() {
    ATH_CHECK(m_hltpsKey.initialize());
}

StatusCode MyAlg::execute(const EventContext &ctx) const {
    SG::ReadCondHandle<TrigConf::HLTPrescalesSet> hltpssHandle(m_hltpsKey, ctx);
    ATH_CHECK(hltpssHandle.isValid());
    // print all HLT prescales
    hltpssHandle->printPrescaleSet(/*full=*/true);
}

MyAlg.h

#include "TrigConfData/L1BunchGroupSet.h"
class MyAlg : public AthReentrantAlgorithm {
    SG::ReadCondHandleKey<TrigConf::L1BunchGroupSet> m_bgsKey{this,
        "L1BunchGroup", "L1BunchGroup", "L1 bunchgroup set condition handle"};
}

MyAlg.cxx

StatusCode MyAlg::initialize() {
    ATH_CHECK(m_bgsKey.initialize());
}

StatusCode MyAlg::execute(const EventContext &ctx) const {
    SG::ReadCondHandle<TrigConf::L1BunchGroupSet> bgHandle(m_bgInputKey, ctx);
    ATH_CHECK(bgHandle.isValid());
    // get bunch group pattern for BCID 13
    uint16_t bcid13 = bgHandle->bgPattern(/*bcid=*/13);
}

Storing trigger configuration as in-file metadata¶

All trigger configuration data, except the HLT monitoring currently, is stored by the xAODMenuWriter inside the pool file as in-file metadata, from where they can be accessed in any job using pool files as input.

The xAODMenuWriter also stores the trigger configuration keys (TrigConfKeys, BunchConfKey) in the EventStore with the help of the KeyWriterTool. They are used by the xAODConfigSvc to provide the correct configuration to the event that is being executed in the current event slot.

A special case are MC pool files. During simulation the trigger configuration is read not from a TriggerDB like during data taking at P1, but from configuration json-files. Therefor an artificial SMK gets generated based on the combined hash of the L1 and HLT menu. That ensures the same uniqueness of the SMK as for collisions data.

Trigger configuration access when reading pool files¶

For access to the L1 information (L1 menu, L1 prescales, L1 bunchgroups) and HLT information (HLT menu, HLT prescales and HLT monitoring) when processing pool files one should use the xAODConfigSvc, as it is properly set up to read in-file metadata whenever a new pool file is opened.

It provides interfaces to the L1 and HLT information (see ITrigConfigSvc and for details IILVL1ConfigSvc, IIHLTConfigSvc).

Usage of xAODConfigSvc

#include "TrigConfInterfaces/ITrigConfigSvc.h"
class MyAlg : public AthReentrantAlgorithm {
    const ServiceHandle<TrigConf::ITrigConfigSvc> m_trigCfgSvc{this, "TrigConfigSvc", "TrigConf::xAODConfigSvc/xAODConfigSvc", "L1 Trigger configuration access"};
}

StatusCode MyAlg::initialize() {
    ATH_CHECK( m_trigCfgSvc.retrieve() );
}

StatusCode MyAlg::execute (const EventContext& ctx) const {
    m_trigCfgSvc->l1BunchGroupSet(ctx); // as an example
}

Trigger configuration access via DetectorStore vs `xAODConfigSvc`¶

Access via DetectorStore vs xAODConfigSvc

If an algorithm is meant to run on bytestream and and on pool files, then it needs to be configured differently for the two cases, see the two corresponding sections above.

The following is an example for an algorithm that can run on bytestream data and on pool files.

example: algorithm that runs on bytestream and pool input files

MyAlg.hMyAlg.cxxMyAlgConf.py

#include "TrigConfInterfaces/ITrigConfigSvc.h"
#include "TrigConfData/L1Menu.h"
#include "TrigConfData/HLTMenu.h"
#include "TrigConfData/HLTMonitoring.h"
#include "TrigConfData/L1PrescalesSet.h"
#include "TrigConfData/HLTPrescalesSet.h"
#include "TrigConfData/L1BunchGroupSet.h"

class MyAlg : public AthReentrantAlgorithm {
    const ServiceHandle<TrigConf::ITrigConfigSvc> m_trigCfgSvc{this,
        "TrigConfigSvc", "TrigConf::xAODConfigSvc/xAODConfigSvc", "L1 Trigger configuration access"};

    SG::ReadHandleKey<TrigConf::L1Menu> m_l1MenuKey{this,
        "L1TriggerMenu", "DetectorStore+L1TriggerMenu", "L1Menu in the DetectorStore"}; 
    SG::ReadHandleKey<TrigConf::HLTMenu> m_hltMenuKey{this,
        "HLTTriggerMenu", "DetectorStore+HLTTriggerMenu", "HLTMenu in the DetectorStore"};
    SG::ReadHandleKey<TrigConf::HLTMonitoring> m_HLTMonitoringKey {this, 
        "HLTMonitoringMenu", "DetectorStore+HLTMonitoringMenu", "HLT Monitoring in the DetectorStore"};

    SG::ReadCondHandleKey<TrigConf::L1PrescalesSet> m_l1pssKey{this,
        "L1Prescales", "L1Prescales", "L1 prescales set condition handle"};
    SG::ReadCondHandleKey<TrigConf::HLTPrescalesSet> m_hltpsKey{this,
        "HLTPrescales", "HLTPrescales", "HLT prescales set condition handle"};
    SG::ReadCondHandleKey<TrigConf::L1BunchGroupSet> m_bgsKey{this,
        "L1BunchGroup", "L1BunchGroup", "L1 bunchgroup set condition handle"};

    Gaudi::Property<bool> m_poolinput {this, 
        "PoolInput", false, "Flag to control the source of the configuration data"};
}

StatusCode MyAlg::initialize() {
    if(m_poolinput) {
        // use the xAODConfigSvc when running on pool files
        ATH_CHECK( m_trigCfgSvc.retrieve() );
    }
    // use the read handles when running from the bytestream
    ATH_CHECK(m_l1MenuKey.initialize(!m_poolinput));
    ATH_CHECK(m_hltMenuKey.initialize(!m_poolinput));
    ATH_CHECK(m_HLTMonitoringKey.initialize(!m_poolinput));
    ATH_CHECK(m_l1pssKey.initialize(!m_poolinput));
    ATH_CHECK(m_hltpsKey.initialize(!m_poolinput));
    ATH_CHECK(m_bgsKey.initialize(!m_poolinput));
}

StatusCode MyAlg::execute(const EventContext &ctx) const {
    if(m_poolinput) {
        m_trigCfgSvc->l1BunchGroupSet(ctx);
    } else {
        SG::ReadHandle<TrigConf::L1Menu> l1MenuHandle = SG::makeHandle(m_l1MenuKey, ctx);
        SG::ReadHandle<TrigConf::HLTMenu> hltMenuHandle = SG::makeHandle(m_HLTMenuKey, ctx);
        SG::ReadHandle<TrigConf::HLTMonitoring> hltMonHandle = SG::makeHandle(m_HLTMonitoringKey, ctx);
        SG::ReadCondHandle<TrigConf::L1PrescalesSet> l1pssHandle(m_l1pssKey, ctx);
        SG::ReadCondHandle<TrigConf::HLTPrescalesSet> hltpssHandle(m_hltpsKey, ctx);
        SG::ReadCondHandle<TrigConf::L1BunchGroupSet> bgHandle(m_bgInputKey, ctx);
        ATH_CHECK(l1MenuHandle.isValid());
        ATH_CHECK(hltMenuHandle.isValid());
        ATH_CHECK(hltMonHandle.isValid());
        ATH_CHECK(l1pssHandle.isValid());
        ATH_CHECK(hltpssHandle.isValid());
        ATH_CHECK(bgHandle.isValid());

        // L1 menu: e.g. get the thresholds names
        const std::vector<std::string> & thresholds = l1MenuHandle->thresholdNames();
        // HLT menu: e.g. loop over all chains
        for (const Chain& loadedChain : *hltMenuHandle) {
            ...
        }
        // HLT monitoring: e.g. get the monitored signatures and targets
        std::vector<std::string> monSignatures = hltMonHandle->signatureNames();
        const std::set<std::string> & targets = hltMonHandle->targets();

        // get the prescale for item L1_eEM15
        const TrigConf::L1Prescale & ps = l1pssHandle->prescale("L1_eEM15");

        // print all HLT prescales
        hltpssHandle->printPrescaleSet(/*full=*/true);

        // get bunch group pattern for BCID 13
        uint16_t bcid13 = bgHandle->bgPattern(/*bcid=*/13);
    }
}

from AthenaConfiguration.ComponentAccumulator import ComponentAccumulator
from AthenaConfiguration.ComponentFactory import CompFactory
from AthenaConfiguration.Enums import Format

def MyAlgCfg(flags):
    acc = ComponentAccumulator()
    if flags.Input.Format is Format.BS: # => use cond handles
        from TrigConfigSvc.TrigConfigSvcCfg import TrigConfigSvcCfg
        acc.merge( TrigConfigSvcCfg(flags) )
        acc.addEventAlgo(CompFactory.MyAlg("MyAlg"))
    else: # Format.POOL => use xAODConfigSvc with in-file metadata
        from AthenaConfiguration.MainServicesConfig import MainServicesCfg
        cfg = MainServicesCfg(flags)
        myAlg = CompFactory.MyAlg("MyAlg")
        myAlg.TrigConfigSvc = cfg.getPrimaryAndMerge(getxAODConfigSvc(flags))
        myAlg.PoolInput = True
        acc.addEventAlgo(myAlg)
    return acc

Trigger configuration access in python¶

Access to the various parts of the trigger configuration is also provided in python. The information holders are

All classes provide access to the complete configuration json structure through the config() method.

L1 trigger configuration access classes (python)

L1MenuAccessL1PrescalesSetAccessBunchGroupSetAccess

class L1MenuAccess(TriggerConfigAccess):
    """
    this class provides access to the L1Menu
    the methods are self-explanatory for people with knowledge of the configuration
    """
    def __init__(self, filename = None, jsonString = None, dbalias = None, smkey = None, useCrest=False, crestServer=""):
        """
        accessor needs to be initialized with either a filename or the dbalias and smkey
        """
        super().__init__(ConfigType.L1MENU, mainkey = "items",
                         jsonString = jsonString, filename = filename, dbalias = dbalias, dbkey = smkey,
                         useCrest=useCrest, crestServer=crestServer)
        self.loader.setQuery({
            2: "SELECT L1MT.L1TM_DATA FROM {schema}.SUPER_MASTER_TABLE SMT, {schema}.L1_MENU L1MT WHERE L1MT.L1TM_ID=SMT.SMT_L1_MENU_ID AND SMT.SMT_ID=:dbkey", # for new db schema
            1: "SELECT L1MT.L1MT_MENU FROM {schema}.SUPER_MASTER_TABLE SMT, {schema}.L1_MASTER_TABLE L1MT WHERE L1MT.L1MT_ID=SMT.SMT_L1_MASTER_TABLE_ID AND SMT.SMT_ID=:dbkey"  # for current db schema
        })
        self.load()
        if smkey is not None:
            log.info(f"Loaded L1 menu {self.name()} with {len(self)} items from {dbalias} with smk {smkey}{' using CREST' if useCrest else ''}")
        elif filename is not None:
            log.info(f"Loaded L1 menu {self.name()} with {len(self)} chains from file {filename}")

    def itemNames(self):
        return self._config["items"].keys()

    def itemsWithCtpid(self ):
        return self.items( includeKeys = ['ctpid'] )

    def items(self, includeKeys = [] ):
        if includeKeys:
            """ reduce returned dictionary """
            items = self._config["items"]
            return { x : {k : items[x][k] for k in includeKeys if k in items[x]} for x in items }
        else:
            return self._config["items"]

    def thresholdTypes(self):
        thrTypes = list(self._config["thresholds"].keys())
        if "legacyCalo" in thrTypes:
            thrTypes.remove("legacyCalo")
            thrTypes += list(self._config["thresholds"]["legacyCalo"].keys())
        return thrTypes

    def thresholds(self, thresholdType = None, fulldict = False):
        """
        When setting fulldict the full dictionary is returned, else just thresholds
        """
        if thresholdType:
            if thresholdType == "internal":
                return {}
            if thresholdType in self["thresholds"]:
                if fulldict:
                    return self["thresholds"][thresholdType]
                else:
                    return self["thresholds"][thresholdType]["thresholds"]
            if thresholdType in self["thresholds"]["legacyCalo"]:
                return self["thresholds"]["legacyCalo"][thresholdType]["thresholds"]
            raise RuntimeError("Threshold type %s not known in thresholds section of the menu" % thresholdType)
        else:
            thrs = {}
            for thrType in self.thresholdTypes():
                thrs.update( self.thresholds(thrType) )
            return thrs

    def thresholdNames(self, thresholdType = None):
        if thresholdType == "internal":
            return self["thresholds"]["internal"]["names"]
        elif thresholdType is None:
            return list(self.thresholds().keys()) + self.thresholdNames("internal")
        else:
            return list(self.thresholds(thresholdType).keys())

    def thresholdExtraInfo(self, thresholdType):
        if thresholdType in self["thresholds"]:
            thrDef = self["thresholds"][thresholdType]
        elif thresholdType in self["thresholds"]["legacyCalo"]:
            thrDef = self["thresholds"]["legacyCalo"][thresholdType]
        else:
            raise KeyError("Threshold type %s not known in thresholds section of the menu" % thresholdType)
        return {k:thrDef[k] for k in thrDef if k not in ("thresholds", "type")}

    def topoAlgorithmTypes(self):
        return self["topoAlgorithms"].keys()

    def topoAlgorithms(self, topoAlgoType = None):
        if topoAlgoType:
            return self["topoAlgorithms"][topoAlgoType]
        else:
            d = {}
            for x in self["topoAlgorithms"].values():
                for gr in x.values():
                    d.update(gr)
            return d

    def topoAlgorithmNames(self, topoAlgoType = None):
        allAlgs = self.topoAlgorithms(topoAlgoType)
        return allAlgs.keys()


    def boardNames(self):
        return iter(self["boards"])

    def boards(self):
        return self["boards"]

    def board(self, boardName):
        return self["boards"][boardName]

    def connectorNames(self):
        return iter(self["connectors"])

    def connectors(self):
        return self["connectors"]

    def connector(self, connectorName):
        return self["connectors"][connectorName]

    def ctp(self):
        return self["ctp"]

    def ctpInputs(self, inputType):
        """ inputType should be 'optical', 'electrical' or 'ctpin'
        """
        return self["ctp"]["inputs"][inputType]

    def printSummary(self):
        print("L1 menu %s" % self.name())
        print("Number of items: %i" % len(self))
        print("Number of threshold types: %i" % len(self.thresholdTypes()) )
        print("Number of thresholds: %i" % len(self.thresholds()) )
        print("Number of topo algorithms: %i" % len(self.topoAlgorithms()))
        print("Number of boards: %i (%i are legacy boards)" % ( len(self.boards()), sum(["legacy" in b for b in self.boards().values()]) ))
        print("Number of connectors: %i (%i are legacy connetors)" % ( len(self.connectors()), sum(["legacy" in c for c in self.connectors().values()]) ))
        print("CTP has %i optical, %i electrical, and %i CTPIN inputs" % ( len(self.ctpInputs("optical")), len(self.ctpInputs("electrical")), 
                                                                           len(reduce(
                                                                               lambda s1,i: s1.union(self.ctpInputs("ctpin")[f"slot{i}"].values()),
                                                                               [7,8,9], set()) - set([""]) )))

class L1PrescalesSetAccess(TriggerConfigAccess):
    """
    this class provides access to the L1 prescales set
    the methods are self-explanatory for people with knowledge of the configuration
    """
    @staticmethod
    def calcPrescaleFromCut(cut):
        """
        turns cut value (which is what the hardware is configured with), into a float prescale value
        """
        return 0xFFFFFF / ( 0x1000000 - cut )

    def __init__(self, filename = None, jsonString = None, dbalias = None, l1pskey = None, useCrest=False, crestServer=""):
        """
        accessor needs to be initialized with either a filename or the dbalias and l1pskey
        """
        super().__init__(ConfigType.L1PS, mainkey = "cutValues",
                         jsonString = jsonString, filename = filename, dbalias = dbalias, dbkey = l1pskey,
                         useCrest=useCrest, crestServer=crestServer)
        self.loader.setQuery({
            1: "SELECT L1PS_DATA FROM {schema}.L1_PRESCALE_SET L1PS WHERE L1PS_ID=:dbkey" # for current and new db schema
        })
        self.load()
        if l1pskey is not None:
            log.info(f"Loaded L1 prescales {self.name()} with {len(self)} items from {dbalias} with psk {l1pskey}{' using CREST' if useCrest else ''}")
        elif filename is not None:
            log.info(f"Loaded L1 prescales {self.name()} with {len(self)} items from file {filename}")


    def itemNames(self):
        return self["cutValues"].keys()
    def cutValues(self):
        return self["cutValues"]
    def cut(self, itemName):
        return self["cutValues"][itemName]["cut"]
    def prescale(self, itemName):
        return L1PrescalesSetAccess.calcPrescaleFromCut( self.cut(itemName) )

    def enabled(self, itemName):
        return self["cutValues"][itemName]["enabled"]

    def printSummary(self):
        print("L1 prescales set %s" % self.name())
        print("Number of prescales: %i" % len(self) )
        print("Number of enabled prescales: %i" % sum(x["enabled"] for x in self["cutValues"].values()) )

class BunchGroupSetAccess(TriggerConfigAccess):
    """
    this class provides access to the L1 bunchgroup set
    the methods are self-explanatory for people with knowledge of the configuration
    """
    def __init__(self, filename = None, jsonString = None, dbalias = None, bgskey = None, useCrest=False, crestServer=""):
        super().__init__(ConfigType.BGS, mainkey = "bunchGroups",
                         jsonString = jsonString, filename = filename, dbalias = dbalias, dbkey = bgskey,
                         useCrest=useCrest, crestServer=crestServer)
        self.loader.setQuery({
            1: "SELECT L1BGS_DATA FROM {schema}.L1_BUNCH_GROUP_SET BGS WHERE L1BGS_ID=:dbkey" # for current and new db schema
        })
        self.load()
        if bgskey is not None:
            log.info(f"Loaded L1 bunchgroup set {self.name()} with {len(self)} bunchgroups from {dbalias} with bgsk {bgskey}{' using CREST' if useCrest else ''}")
        elif filename is not None:
            log.info(f"Loaded L1 bunchgroup set {self.name()} with {len(self)} bunchgroups from file {filename}")

HLT trigger configuration access classes (python)

HLTMenuAccessHLTPrescalesSetAccessHLTJobOptionsAccessHLTMonitoringAccess

class HLTMenuAccess(TriggerConfigAccess):
    """
    this class provides access to the HLT menu
    the methods are self-explanatory for people with knowledge of the configuration
    """
    def __init__(self, filename = None, jsonString = None, dbalias = None, smkey = None,
                 useCrest=False, crestServer=""):
        """
        accessor needs to be initialized with either a filename or the dbalias and smkey
        """
        super().__init__(ConfigType.HLTMENU, mainkey = "chains",
                         filename = filename, jsonString = jsonString, dbalias = dbalias, dbkey = smkey,
                         useCrest=useCrest, crestServer=crestServer)
        self.loader.setQuery({
            2: "SELECT HMT.HTM_DATA FROM {schema}.SUPER_MASTER_TABLE SMT, {schema}.HLT_MENU HMT WHERE HMT.HTM_ID=SMT.SMT_HLT_MENU_ID AND SMT.SMT_ID=:dbkey", # for new db schema
            1: "SELECT HMT.HMT_MENU FROM {schema}.SUPER_MASTER_TABLE SMT, {schema}.HLT_MASTER_TABLE HMT WHERE HMT.HMT_ID=SMT.SMT_HLT_MASTER_TABLE_ID AND SMT.SMT_ID=:dbkey"  # for current db schema
        })
        self.load()
        if smkey is not None:
            log.info(f"Loaded HLT menu {self.name()} with {len(self)} chains from {dbalias} with smk {smkey}{' using CREST' if useCrest else ''}")
        elif filename is not None:
            log.info(f"Loaded HLT menu {self.name()} with {len(self)} chains from file {filename}")

    def chainNames(self):
        return self["chains"].keys()

    def chains(self):
        return self["chains"]

    def streams(self):
        return self["streams"]

    def sequencers(self):
        return self["sequencers"]

    def printSummary(self):
        print("HLT menu %s" % self.name())
        print("Number of chains: %i" % len(self.chains()) )
        print("Number of streams: %i" % len(self.streams()) )
        print("Number of sequencers: %i" % len(self.sequencers()) )

    def printDetails(self):
        import pprint
        print("Chains:")
        pprint.pprint(list(self.chains()))
        print("Streams:")
        pprint.pprint(list(self.streams()))
        print("Sequencers:")
        pprint.pprint(list(self.sequencers()))

class HLTPrescalesSetAccess(TriggerConfigAccess):
    """
    this class provides access to the HLT prescales set
    the methods are self-explanatory for people with knowledge of the configuration
    """
    def __init__(self, filename = None, jsonString = None, dbalias = None, hltpskey = None,
                 useCrest=False, crestServer=""):
        """
        accessor needs to be initialized with either a filename or the dbalias and hlpskey
        """
        super().__init__(ConfigType.HLTPS, mainkey = "prescales",
                         jsonString = jsonString, filename = filename, dbalias = dbalias, dbkey = hltpskey,
                         useCrest=useCrest, crestServer=crestServer)
        self.loader.setQuery({
            1: "SELECT HPS_DATA FROM {schema}.HLT_PRESCALE_SET HPS WHERE HPS_ID=:dbkey" # for current and new db schema
        })
        self.load()
        if hltpskey is not None:
            log.info(f"Loaded HLT prescales {self.name()} (size {len(self)}) from {dbalias} with psk {hltpskey}{' using CREST' if useCrest else ''}")
        elif filename is not None:
            log.info(f"Loaded HLT prescales {self.name()} with {len(self)} chains from file {filename}")

    def prescales(self):
        return self["prescales"]

    def chainNames(self):
        return iter(self)

    def prescale(self, chainName):
        return self["prescales"][chainName]["prescale"]

    def enabled(self, chainName):
        return self["prescales"][chainName]["enabled"]

    def printSummary(self):
        print("HLT prescales set %s" % self.name())
        print("Number of prescales: %i" % len(self) )
        print("Number of enabled prescales: %i" % sum(x["enabled"] for x in self["prescales"].values()) )

class HLTJobOptionsAccess(TriggerConfigAccess):
    """
    this class provides access to the HLT algorithm configuration
    the methods are self-explanatory for people with knowledge of the configuration
    """
    def __init__(self, filename = None, dbalias = None, smkey = None,
                 useCrest = False, crestServer = ""):
        """
        accessor needs to be initialized with either a filename or the dbalias and smkey
        """
        super().__init__(ConfigType.HLTJO, mainkey = "properties",
                         filename = filename, dbalias = dbalias, dbkey = smkey,
                         useCrest=useCrest, crestServer=crestServer)
        self.loader.setQuery({
            2: "SELECT JO.HJO_DATA FROM {schema}.SUPER_MASTER_TABLE SMT, {schema}.HLT_JOBOPTIONS JO WHERE JO.HJO_ID=SMT.SMT_HLT_JOBOPTIONS_ID AND SMT.SMT_ID=:dbkey", # for new db schema
            1: "SELECT JO.JO_CONTENT FROM {schema}.SUPER_MASTER_TABLE SMT, {schema}.JO_MASTER_TABLE JO WHERE JO.JO_ID=SMT.SMT_JO_MASTER_TABLE_ID AND SMT.SMT_ID=:dbkey"  # for current db schema
        })
        self.load()
        if smkey is not None:
            log.info(f"Loaded HLT job options {self.name()} with {len(self)} algorithms from {dbalias} with smk {smkey}{' using CREST' if useCrest else ''}")
        elif filename is not None:
            log.info(f"Loaded HLT job options {self.name()} with {len(self)} chains from file {filename}")

    def algorithms(self):
        return self["properties"]

    def algorithmNames(self):
        return iter(self)

    def properties(self, algName):
        return self["properties"][algName]

    def name(self):
        # job options don't have a name
        return "HLT JobOptions"


    def printSummary(self):
        print("Job options")
        print("Number of algorithms: %i" % len(self) )
        print("Number of properties: %i" % sum(len(alg) for alg in self.algorithms().values()) )

class HLTMonitoringAccess(TriggerConfigAccess):
    """
    this class provides access to the HLT monitoring json
    """
    def __init__(self, filename = None, jsonString = None, dbalias = None, smkey = None, monikey = None,
                 useCrest=False, crestServer=""):
        """
        accessor needs to be initialized with either a filename or the dbalias and hlpskey
        """
        super().__init__(ConfigType.HLTMON, mainkey = "signatures",
                         jsonString = jsonString, filename = filename, dbalias = dbalias, dbkey = smkey if smkey else monikey,
                         useCrest=useCrest, crestServer=crestServer)
        self.loader.setQuery({
            7: (
                "SELECT HMG.HMG_DATA FROM {schema}.HLT_MONITORING_GROUPS HMG, {schema}.SUPER_MASTER_TABLE SMT WHERE HMG.HMG_IN_USE=1 "
                "AND SMT.SMT_HLT_MENU_ID = HMG.HMG_HLT_MENU_ID AND SMT.SMT_ID=:dbkey ORDER BY HMG.HMG_ID DESC"
            )
        } if smkey else {
            7: "SELECT HMG.HMG_DATA FROM {schema}.HLT_MONITORING_GROUPS HMG WHERE HMG.HMG_ID=:dbkey"
        })
        self.load()
        if smkey is not None:
            log.info(f"Loaded HLT monitoring {self.name()} with {len(self)} signatures from {dbalias} with smk {smkey}{' using CREST' if useCrest else ''}")
        elif filename is not None:
            log.info(f"Loaded HLT monitoring {self.name()} with {len(self)} signatures from file {filename}")


    def monitoringDict(self):
        """
        return stored monitoring dictionary
        """
        return self["signatures"]


    def monitoredChains(self, signatures="", monLevels="", wildcard=""):
        """
        return list of all monitored shifter chains for given signature and for a given monitoring level

        signatures - monitored signature or list of signatures for which to return the chains
                     empty string means all signatures
        monLevels - levels of monitoring (shifter, t0 (expert), val (validation))
        wildcard - regexp pattern to match the chains' names

        if monitoring level is not defined return all the chains for given signature
        if signature is not defined return all the chains for given monitoring level
        if both monitoring level and signature are not defined, raturn all chains

        return can be filtered by wildcard
        """
        chains = set()

        if signatures=="": # empty string means all signatures
            signatures = set(self)

        # turn input (str,list) into a set of signature names
        if isinstance(signatures, str):
            signatures = set([signatures])
        signatures = set(signatures)

        # warn about requested signatures that don't have a monitoring entry and remove from the request
        noMonAvailable = signatures.difference(self)
        if noMonAvailable:
            log.warning("These monitoring signatures are requested but not available in HLT monitoring: %s", ', '.join(noMonAvailable))
            signatures.intersection_update(self) # ignore non-existing signatures

        # turn input (str,list) into a set of monLevels
        if isinstance(monLevels, str):
            monLevels = set([monLevels])
        monLevels = set(monLevels)

        for signature in signatures:
            for chainName, targets in self["signatures"][signature].items():
                if monLevels.intersection(targets+[""]): # if there is an overlap between requested and configured
                    chains.add(chainName)

        try:
            import re
            r = re.compile(wildcard)
            chains = filter(r.search, chains)
        except re.error as exc:
            log.warning("Wildcard regex: %r is not correct!", exc)

        # Create set first to ensure uniquness of elements
        return list(chains)


    @lru_cache(maxsize=5)
    def monitoringLevels(self, signatures = None):
        """
        return all monitoring levels
        If one ore more signatures are specified, return only monitoring levels for those
        """
        if signatures is None:
            signatures = set(self)
        if isinstance(signatures, str):
            signatures = set([signatures])
        signatures = set(signatures)
        levels = set()
        for signatureName, chains in self["signatures"].items():
            if signatureName in signatures:
                levels = reduce( lambda x, y: x.union(y), chains.values(), levels )
        return levels


    def printSummary(self):
        print("HLT monitoring groups %s" % self.name())
        print("Signatures (%i): %s" % (len(self), ", ".join(self)) )
        print("Monitoring levels (%i): %s" % (len(self.monitoringLevels()), ", ".join(self.monitoringLevels())))

They can be retrieved through the functions getL1MenuAccess(flags), getL1PrescalesSetAccess(flags), getBunchGroupSetAccess(flags), getHLTMenuAccess(flags), getHLTPrescalesSetAccess(flags), getHLTJobOptionsAccess(flags), and getHLTMonitoringAccess(flags), which are all defined in TrigConfigSvc/TriggerConfigAccess.py.

All these functions are able to provide the configuration information from json-files, the TriggerDB, or the in-file metadata. In case of the TriggerDB being the source, the trigger keys can be either supplied directly, or are extracted from the conditions database based on the run number. The job-configuration ensures that the exact source is transparent to the user.

Access the trigger configuration in python

def MyAlgCfg(flags)
    # read information from the L1 menu and configure my algorithm
    from TrigConfigSvc.TriggerConfigAccess import getL1MenuAccess
    l1menuacc = getL1MenuAccess(flags)
    l1menu:dict = l1menuacc.config() # the l1 menu (json file as dict)

Online access to trigger configuration¶

During data taking the HLT requires two keys that are external inputs to the HLT setup, the SuperMasterKey (SMK) and the HLT prescale key (HLTPSK). The SMK and initial HLTPSK are selected before the configuration of the ATLAS partition by the shifter and the HLT farm is configured with these keys. For the prescaling a whole set of predefined keys is available, after setting an initial one during the HLT configuration, they can be updated through the "extensible folder mechanism".

Getting the SMK and HLTPSK from OKS to configure the HLT athena job¶

Online, the SMK is selected by the RunControl shifter and saved in OKS before the partition is initialized. When the farm is configured, the information is extracted from OKS and passed on to the TrigPSC/PSC in PSC::configure(ptree&). Inside the JobOptionsSvc loads the job options from the TriggerDB based on the given SMK, with which the job is then configured.

Detailed information how the HLT farm is configured

Online, the SMK and the initial HLTPSK, which are required when the athena jobs in the HLT farm are started, are selected by the RunControl shifter and saved in OKS, in the partition description as part of the TriggerConfiguration in the ATLAS partition.

The executable of an HLT job is HLTMPPU_main. That program is started within an environment that is provided by asetup, which is prepared using the asetup_wrapper.

asetup_wrapper HLTMPPU_main

The main task of the HLTMPPU_main is to instantiate an implementation of the hltinterface::HLTInterface, provided as plugin, and set it up as an HLTReceiver which starts listening to run-control commands.

That interface implementation is the actual HLTMPPU. The first command it receives is to configure itself with the OKS data, which it gets in form of a boost::ptree as argument.

bool HLTMPPU::configure(const boost::property_tree::ptree& args )

configuration ptree

The ptree is contains among other information the trigger configuration keys that the HLT farm starts with.

Partition : 
    UID : ATLAS
    TriggerConfiguration : 
        TriggerConfiguration : 
        UID : TriggerConfiguration-1
        L1TriggerConfiguration : 
            L1TriggerConfiguration : 
            UID : L1TrigConf
            Lvl1PrescaleKey : 20317
            Lvl1BunchGroupKey : 2497
            ConfigureLvl1MenuFrom : DB
        TriggerDBConnection : 
            TriggerDBConnection : 
            UID : TRIGGERDB_RUN3_CoralServer
            Alias : TRIGGERDB_RUN3
            User : ATLAS_CONF_TRIGGER_RUN3_R
            SuperMasterKey : 3504

While the HLTMPPU implements the interface hltinterface/HLTInterface, it lives in tdaq and hence can not execute the athena-based HLT jobs. Therefore, in the different methods to go from state INITIAL to state RUNNING, it simply forwards these calls to the TrigPSC/PSC, a different application, which also implement the interface HLTInterface, but which lives in athena and is loaded as a plugin into the HLTMPPU_main run-control application.

The PSC then creates the Gaudi::ApplicationMgr in bool Psc::configure(const ptree& config) and configures the JobOptionsSvc with the TriggerDB connection and the trigger keys. The JobOptionsSvc then reads the HLT configuration from the database, which results in a fully configured HLT, that can start initializing its algorithms, etc.

HLTMPPU log output

Pesa JobOptions file is = server=TRIGGERDB_RUN3;smkey=3504;lvl1key=20317;hltkey=14656
JobOptionsSvc        INFO Initializing TrigConf::JobOptionsSvc
JobOptionsSvc        INFO Reading SMK 3504 from 'TRIGGERDB_RUN3'

Extensible folder mechanism¶

Some conditions which are used by the HLT can change during the run. These conditions must be stored in so-called "extensible folders". As their content can change during an ongoing run and the HLT needs to be notified of such a change and be able to load the new conditions data consistently across the farm.

Currently there are three conditions folders which are "extensible", they are added in athena with IOVDbSvc.addFolder(...,extensible=True)

HLT prescale key (folder /TRIGGER/HLT/PrescaleKey)
Beamspot for HLT (folder /Indet/Onl/Beampos)
Online luminosity for HLT (folder /TRIGGER/LUMI/HLTPrefLumi)

Thereby the mapping between the flag position in the CTPFragment and the folder name is stored in /TRIGGER/HLT/COOLUPDATE.

Step-by-step explanation of the HLT live conditions update

New information available: Whenever the HLT configuration (prescale keys, beamspot, instantaneous luminosity) needs to be updated for the HLT during the run, the configuration supplier sends the new data to the CTP. For the HLT prescales this can be either the run control shifter via the TriggerPanel in the IGUI, or CHIP changing prescales when ATLAS changes the state, e.g. from STANDBY to PHYSICS. For beamspot and lumi updates other applications exist. Let's assume the current luminosity block x.
Conditions folder update: The CTP updates the corresponding conditions folder with the received data and for a validity range (IOV) starting with the next luminosity block. The CTP knows, depending on the type of configuration data, in which conditions folder it needs to store that data. If needed, multiple folders can be updated for the same LB.
Informing the HLT farm: The CTP informs all HLT nodes about the update of the conditions folder for LB x+1 using the CTPFragment in the event data itself as information carrier (see Table 2 of the CTPCore event format document). That ensures that the information reaches all HLT jobs, even if they are being restarted right at that moment. In the CTPFragment the folder is not identified by name, but by a folder_index.
- The mapping of folder_index to folder_name is stored in the conditions database in /TRIGGER/HLT/COOLUPDATE, from where the HLT gets that information.
- The information that the CTP sends in the CTPFragment is (folder_index, LB=x+1). The reason for sending the LB=x+1 value along is to allow the HLT to keep track if it already requested the data from the conditions database and therefor avoid sending conditions DB queries for each new event.
Refresh the conditions data: In the athena HLT job the IOVSvc-internal conditions cache of folder F is invalidated for LB=x+1 onwards. Any request of conditions data for that or future _LB_s then requires the IOVDbSvc to go back to the conditions database and fetch the data again, thus loading the updated conditions.
- The IOVDbSvc always includes the LB number it got in the CTPFragment along with the conditions database query. This disables the DB response caching in the HLT farm when requests for different _LB_s are made, but still supports DB response caching across the farm for queries for identical queries.
Load trigger config data: In case of the HLT prescale keys a query to the TriggerDB might be necessary, if the prescale set for that key has not been loaded yet.

Testing the mechanism¶

There is a unit test for this mechanism CondReadWrite.py

Trigger Configuration¶

Introduction¶

Trigger keys in the conditions¶

Trigger configuration access in athena when reading bytestream files¶

Trigger menu access¶

Trigger prescales and bunchgroups access in athena¶

Storing trigger configuration as in-file metadata¶

Trigger configuration access when reading pool files¶

Trigger configuration access via DetectorStore vs xAODConfigSvc¶

Trigger configuration access in python¶

Online access to trigger configuration¶

Getting the SMK and HLTPSK from OKS to configure the HLT athena job¶

Extensible folder mechanism¶

Testing the mechanism¶

Trigger configuration access via DetectorStore vs `xAODConfigSvc`¶