KMTF fixes on propagation coefficients (CMSSW_11_3_X) #33078

Tyler-Lam · 2021-03-05T01:38:13Z

PR description:

New version of KMTF algorithm as discussed in https://indico.cern.ch/event/1012620/contributions/4250430/attachments/2200192/3721090/Run3_KMTF.pdf

PR validation:

Nominal tests with just KMTF algorithm. No core changes, just incremental changes to KMTF

if this PR is a backport please specify the original PR and why you need to backport that PR:

@rekovic The lookup tables located in L1Trigger/L1TMuon/data/bmtf_luts/kalmanLUTs.root are usually in the https://github.com/cms-l1t-offline/L1Trigger-L1TMuon.git repository. They have to be moved but I do not have access
@dinyar please check that we didn't revert your changes

cmsbuild · 2021-03-05T01:46:23Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33078/21389

This PR adds an extra 252KB to repository
Found files with invalid states:
- L1Trigger/L1TMuon/data/bmtf_luts/kalmanLUTs.root:
  - Added: 4e78c82, 88f3cd1
  - Deleted: 5e66a6a

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-checks:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33078/21389/code-checks.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33078/21389/code-checks.patch | patch -p1
You can also run scram build code-checks to apply code checks directly
code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33078/21389/code-format.patch
e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33078/21389/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2021-03-05T20:50:38Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33078/21402

This PR adds an extra 256KB to repository
Found files with invalid states:
- L1Trigger/L1TMuon/data/bmtf_luts/kalmanLUTs.root:
  - Added: 4e78c82, 88f3cd1
  - Deleted: 5e66a6a

cmsbuild · 2021-03-05T20:51:02Z

A new Pull Request was created by @Tyler-Lam for master.

It involves the following packages:

L1Trigger/L1TMuon
L1Trigger/L1TMuonBarrel

@cmsbuild, @rekovic, @cecilecaillol can you please review it and eventually sign? Thanks.
@dinyar, @Martin-Grunewald, @thomreis this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

silviodonato · 2021-03-09T11:54:35Z

please test
kind reminder @cms-sw/l1-l2

cmsbuild · 2021-03-09T14:39:00Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-77a5cf/13366/summary.html
COMMIT: f192774
CMSSW: CMSSW_11_3_X_2021-03-08-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33078/13366/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2849195
DQMHistoTests: Total failures: 1695
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2847478
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
Checked 160 log files, 37 edm output root files, 38 DQM output files

dinyar · 2021-03-09T17:18:00Z

Hi @Tyler-Lam,

Thanks for the ping, looks good to me.

Cheers,
Dinyar

rekovic · 2021-03-23T16:21:44Z

Ok to sign off for L1 after passing tests.

cmsbuild · 2021-03-23T17:02:37Z

-1

Failed Tests: RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-77a5cf/13700/summary.html
COMMIT: 0936dc5
CMSSW: CMSSW_11_3_X_2021-03-23-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33078/13700/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

1325.81

----- Begin Fatal Exception 23-Mar-2021 17:09:49 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 0
   [1] Running path 'nanoAOD_step'
   [2] Calling method for module SimpleCandidateFlatTableProducer/'fatJetMCTable'
   Additional Info:
      [a] Fatal Root Error: @SUB=TInterpreter::AutoParse
Error parsing payload code for class reco::JetFlavourInfo with content:

#line 1 "SimDataFormatsJetMatching_xr dictionary payload"

#ifndef CMS_DICT_IMPL
  #define CMS_DICT_IMPL 1
#endif
#ifndef _REENTRANT
  #define _REENTRANT 1
#endif
#ifndef GNUSOURCE
  #define GNUSOURCE 1
#endif
#ifndef __STRICT_ANSI__
  #define __STRICT_ANSI__ 1
#endif
#ifndef GNU_GCC
  #define GNU_GCC 1
#endif
#ifndef _GNU_SOURCE
  #define _GNU_SOURCE 1
#endif
#ifndef EIGEN_DONT_PARALLELIZE
  #define EIGEN_DONT_PARALLELIZE 1
#endif
#ifndef TBB_USE_GLIBCXX_VERSION
  #define TBB_USE_GLIBCXX_VERSION 90300
#endif
#ifndef TBB_SUPPRESS_DEPRECATED_MESSAGES
  #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
#endif
#ifndef TBB_PREVIEW_RESUMABLE_TASKS
  #define TBB_PREVIEW_RESUMABLE_TASKS 1
#endif
#ifndef BOOST_SPIRIT_THREADSAFE
  #define BOOST_SPIRIT_THREADSAFE 1
#endif
#ifndef PHOENIX_THREADSAFE
  #define PHOENIX_THREADSAFE 1
#endif
#ifndef BOOST_MATH_DISABLE_STD_FPCLASSIFY
  #define BOOST_MATH_DISABLE_STD_FPCLASSIFY 1
#endif
#ifndef BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX
  #define BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX 1
#endif
#ifndef CMSSW_GIT_HASH
  #define CMSSW_GIT_HASH "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef PROJECT_NAME
  #define PROJECT_NAME "CMSSW"
#endif
#ifndef PROJECT_VERSION
  #define PROJECT_VERSION "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef CMSSW_REFLEX_DICT
  #define CMSSW_REFLEX_DICT 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
// Inline headers
#include <utility>
#include <vector>

#include "DataFormats/Common/interface/Wrapper.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfo.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfoMatching.h"
#include "SimDataFormats/JetMatching/interface/JetFlavour.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourMatching.h"
#include "SimDataFormats/JetMatching/interface/MatchedPartons.h"
#include "SimDataFormats/JetMatching/interface/JetMatchedPartons.h"

#undef  _BACKWARD_BACKWARD_WARNING_H


----- End Fatal Exception -------------------------------------------------

10024.0

----- Begin Fatal Exception 23-Mar-2021 17:26:33 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'nanoAOD_step'
   [2] Calling method for module SimpleCandidateFlatTableProducer/'fatJetMCTable'
   Additional Info:
      [a] Fatal Root Error: @SUB=TInterpreter::AutoParse
Error parsing payload code for class reco::JetFlavourInfo with content:

#line 1 "SimDataFormatsJetMatching_xr dictionary payload"

#ifndef CMS_DICT_IMPL
  #define CMS_DICT_IMPL 1
#endif
#ifndef _REENTRANT
  #define _REENTRANT 1
#endif
#ifndef GNUSOURCE
  #define GNUSOURCE 1
#endif
#ifndef __STRICT_ANSI__
  #define __STRICT_ANSI__ 1
#endif
#ifndef GNU_GCC
  #define GNU_GCC 1
#endif
#ifndef _GNU_SOURCE
  #define _GNU_SOURCE 1
#endif
#ifndef EIGEN_DONT_PARALLELIZE
  #define EIGEN_DONT_PARALLELIZE 1
#endif
#ifndef TBB_USE_GLIBCXX_VERSION
  #define TBB_USE_GLIBCXX_VERSION 90300
#endif
#ifndef TBB_SUPPRESS_DEPRECATED_MESSAGES
  #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
#endif
#ifndef TBB_PREVIEW_RESUMABLE_TASKS
  #define TBB_PREVIEW_RESUMABLE_TASKS 1
#endif
#ifndef BOOST_SPIRIT_THREADSAFE
  #define BOOST_SPIRIT_THREADSAFE 1
#endif
#ifndef PHOENIX_THREADSAFE
  #define PHOENIX_THREADSAFE 1
#endif
#ifndef BOOST_MATH_DISABLE_STD_FPCLASSIFY
  #define BOOST_MATH_DISABLE_STD_FPCLASSIFY 1
#endif
#ifndef BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX
  #define BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX 1
#endif
#ifndef CMSSW_GIT_HASH
  #define CMSSW_GIT_HASH "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef PROJECT_NAME
  #define PROJECT_NAME "CMSSW"
#endif
#ifndef PROJECT_VERSION
  #define PROJECT_VERSION "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef CMSSW_REFLEX_DICT
  #define CMSSW_REFLEX_DICT 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
// Inline headers
#include <utility>
#include <vector>

#include "DataFormats/Common/interface/Wrapper.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfo.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfoMatching.h"
#include "SimDataFormats/JetMatching/interface/JetFlavour.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourMatching.h"
#include "SimDataFormats/JetMatching/interface/MatchedPartons.h"
#include "SimDataFormats/JetMatching/interface/JetMatchedPartons.h"

#undef  _BACKWARD_BACKWARD_WARNING_H


----- End Fatal Exception -------------------------------------------------

25202.0

----- Begin Fatal Exception 23-Mar-2021 17:28:28 CET-----------------------
An exception of category 'FatalRootError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 1
   [1] Running path 'nanoAOD_step'
   [2] Calling method for module SimpleCandidateFlatTableProducer/'fatJetMCTable'
   Additional Info:
      [a] Fatal Root Error: @SUB=TInterpreter::AutoParse
Error parsing payload code for class reco::JetFlavourInfo with content:

#line 1 "SimDataFormatsJetMatching_xr dictionary payload"

#ifndef CMS_DICT_IMPL
  #define CMS_DICT_IMPL 1
#endif
#ifndef _REENTRANT
  #define _REENTRANT 1
#endif
#ifndef GNUSOURCE
  #define GNUSOURCE 1
#endif
#ifndef __STRICT_ANSI__
  #define __STRICT_ANSI__ 1
#endif
#ifndef GNU_GCC
  #define GNU_GCC 1
#endif
#ifndef _GNU_SOURCE
  #define _GNU_SOURCE 1
#endif
#ifndef EIGEN_DONT_PARALLELIZE
  #define EIGEN_DONT_PARALLELIZE 1
#endif
#ifndef TBB_USE_GLIBCXX_VERSION
  #define TBB_USE_GLIBCXX_VERSION 90300
#endif
#ifndef TBB_SUPPRESS_DEPRECATED_MESSAGES
  #define TBB_SUPPRESS_DEPRECATED_MESSAGES 1
#endif
#ifndef TBB_PREVIEW_RESUMABLE_TASKS
  #define TBB_PREVIEW_RESUMABLE_TASKS 1
#endif
#ifndef BOOST_SPIRIT_THREADSAFE
  #define BOOST_SPIRIT_THREADSAFE 1
#endif
#ifndef PHOENIX_THREADSAFE
  #define PHOENIX_THREADSAFE 1
#endif
#ifndef BOOST_MATH_DISABLE_STD_FPCLASSIFY
  #define BOOST_MATH_DISABLE_STD_FPCLASSIFY 1
#endif
#ifndef BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX
  #define BOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX 1
#endif
#ifndef CMSSW_GIT_HASH
  #define CMSSW_GIT_HASH "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef PROJECT_NAME
  #define PROJECT_NAME "CMSSW"
#endif
#ifndef PROJECT_VERSION
  #define PROJECT_VERSION "CMSSW_11_3_X_2021-03-22-2300"
#endif
#ifndef CMSSW_REFLEX_DICT
  #define CMSSW_REFLEX_DICT 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
// Inline headers
#include <utility>
#include <vector>

#include "DataFormats/Common/interface/Wrapper.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfo.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourInfoMatching.h"
#include "SimDataFormats/JetMatching/interface/JetFlavour.h"
#include "SimDataFormats/JetMatching/interface/JetFlavourMatching.h"
#include "SimDataFormats/JetMatching/interface/MatchedPartons.h"
#include "SimDataFormats/JetMatching/interface/JetMatchedPartons.h"

#undef  _BACKWARD_BACKWARD_WARNING_H


----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

1325.611325.61_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOAODMC2017_106XMiniAODv1/step2_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOAODMC2017_106XMiniAODv1.log
1325.811325.81_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOEDMMC2017_106XMiniAODv1+HARVESTNANOAODMC2017_106XMiniAODv1/step2_TTbar_13_106Xv1NanoAODINPUT+TTbar_13_106Xv1NanoAODINPUT+NANOEDMMC2017_106XMiniAODv1+HARVESTNANOAODMC2017_106XMiniAODv1.log

silviodonato · 2021-03-24T15:30:43Z

please test
the error was already in the IB

cmsbuild · 2021-03-24T18:12:45Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-77a5cf/13738/summary.html
COMMIT: 0936dc5
CMSSW: CMSSW_11_3_X_2021-03-23-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/33078/13738/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2639935
DQMHistoTests: Total failures: 478
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2639435
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 155 log files, 37 edm output root files, 37 DQM output files
TriggerResults: found differences in 2 / 36 workflows

bachtis · 2021-03-26T21:55:00Z

@rekovic seems this is ready?

panoskatsoulis · 2021-03-26T23:32:18Z

Had a quick look at some of the comparison-plots as well and I think all look good - no unexpected changes in the WFs
I assume that minor changes in the PT distributions are because of the modified coefficients that lead to slightly different tracks and curvatures. (example here https://tinyurl.com/yzrwk4a5)

I think this is ready for the backport
(we need to catch the deadline of the DQM so if also @rekovic and @silviodonato think that this is ready to go,
please open the 11_2_X PR too)

bachtis · 2021-03-26T23:35:00Z

@panoskatsoulis as you see Tyler has opened the backport

panoskatsoulis · 2021-03-26T23:38:18Z

@panoskatsoulis as you see Tyler has opened the backport

right my bad, mentioning this to the DQM
thx

silviodonato · 2021-03-27T08:50:42Z

@cms-sw/l1-l2 do you have any comments?

cecilecaillol · 2021-03-29T07:44:11Z

+l1

cmsbuild · 2021-03-29T07:44:36Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-03-29T07:49:44Z

+1

… via cms-sw/cmssw#33078

Tyler-Lam added 4 commits March 3, 2021 19:00

Migrated run3 KMTF changes from 11_2_X to 11_3_X

a779b90

Added new kalman gains

4e78c82

Removed gains

5e66a6a

Added gains for run 3 kbmtf

88f3cd1

cmsbuild added this to the CMSSW_11_3_X milestone Mar 5, 2021

cmsbuild added code-checks-pending l1-pending orp-pending pending-signatures tests-pending labels Mar 5, 2021

cmsbuild added code-checks-rejected and removed code-checks-pending labels Mar 5, 2021

Tyler-Lam added 2 commits March 5, 2021 12:53

code checks

ee936f5

code-format

f192774

cmsbuild added code-checks-pending and removed code-checks-rejected labels Mar 5, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 5, 2021

cmsbuild added tests-started and removed tests-pending labels Mar 9, 2021

cmsbuild added tests-approved and removed tests-started labels Mar 9, 2021

Tyler-Lam added 2 commits March 10, 2021 17:12

Scripts to make gains for high/low quality stubs

91be504

gain file

c78ef02

cmsbuild added the tests-started label Mar 23, 2021

cmsbuild added tests-rejected and removed tests-started labels Mar 23, 2021

cmsbuild added tests-started and removed tests-rejected labels Mar 24, 2021

cmsbuild added tests-approved and removed tests-started labels Mar 24, 2021

Tyler-Lam mentioned this pull request Mar 26, 2021

[L1TMuonBarrel] KMTF fixes on propagation coefficients (CMSSW_11_2_X) #33281

Merged

cmsbuild added fully-signed l1-approved and removed l1-pending pending-signatures labels Mar 29, 2021

cmsbuild added orp-approved and removed orp-pending labels Mar 29, 2021

cmsbuild merged commit f3c690b into cms-sw:master Mar 29, 2021

smuzaffar mentioned this pull request Mar 31, 2021

Duplicate L1 data files causing cmsdist PRs to fails #33303

Closed

smuzaffar added a commit to cms-data/L1Trigger-L1TMuon that referenced this pull request Mar 31, 2021

Updated bmtf_luts/kalmanLUTs.root which was mistakenly added in cmssw…

ecb8691

… via cms-sw/cmssw#33078

smuzaffar mentioned this pull request Mar 31, 2021

Updated bmtf_luts/kalmanLUTs.root which was mistakenly added in cmssw cms-data/L1Trigger-L1TMuon#20

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KMTF fixes on propagation coefficients (CMSSW_11_3_X) #33078

KMTF fixes on propagation coefficients (CMSSW_11_3_X) #33078

Tyler-Lam commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

silviodonato commented Mar 9, 2021

cmsbuild commented Mar 9, 2021

dinyar commented Mar 9, 2021

rekovic commented Mar 23, 2021

cmsbuild commented Mar 23, 2021

silviodonato commented Mar 24, 2021

cmsbuild commented Mar 24, 2021

bachtis commented Mar 26, 2021

panoskatsoulis commented Mar 26, 2021

bachtis commented Mar 26, 2021

panoskatsoulis commented Mar 26, 2021

silviodonato commented Mar 27, 2021

cecilecaillol commented Mar 29, 2021

cmsbuild commented Mar 29, 2021

qliphy commented Mar 29, 2021

KMTF fixes on propagation coefficients (CMSSW_11_3_X) #33078

KMTF fixes on propagation coefficients (CMSSW_11_3_X) #33078

Conversation

Tyler-Lam commented Mar 5, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

cmsbuild commented Mar 5, 2021

silviodonato commented Mar 9, 2021

cmsbuild commented Mar 9, 2021

Comparison Summary

dinyar commented Mar 9, 2021

rekovic commented Mar 23, 2021

cmsbuild commented Mar 23, 2021

RelVals

RelVals-INPUT

silviodonato commented Mar 24, 2021

cmsbuild commented Mar 24, 2021

Comparison Summary

bachtis commented Mar 26, 2021

panoskatsoulis commented Mar 26, 2021

bachtis commented Mar 26, 2021

panoskatsoulis commented Mar 26, 2021

silviodonato commented Mar 27, 2021

cecilecaillol commented Mar 29, 2021

cmsbuild commented Mar 29, 2021

qliphy commented Mar 29, 2021