General information

EGI Core Services: new bidding ongoing

  • The bidding for the EGI Services covering 2024-01 - 2026-12 was announced last December
    • applications/expression of interest were received and under discussion with the suppliers
  • EGI Core services delivered in best effort mode between July and December 2023
    • Broadcast circulated on July 6th
    • Gap between EGI-ACE project (ended in June 2023) and the EOSC procurement that is going to fund them (starting from Jan 2024)
    • ensured continuous operation and security system maintenance

Middleware


UMD

  • moving the UMD infrastructure to production.
  • new UMD update to be announced soon.

Operations

Accounting Repository

Downtime from Jan 20th to Feb 16th:

  • Pub/Sync system taken offline for a security issue. APEL Repository operation unaffected, but Repository test is provided via the pub/sync hosts.

ARGO/SAM

  • Monitoring of xrootd endpoints
    • some endpoints are exposed outside the site in read-only mode
    • the new service type "eu.egi.readonly.xrootd" was created for this purpose (see GGUS 160848)
    • new version of the xrootd probe executing only "read" tests: to be added in UMD and deployed in ARGO (GGUS 163071)
  • New version of srm probe to be deployed (GGUS 162411) and to be included in UMD (GGUS 162424)
    • support for py3 only
    • support for SRM+HTTPS
    • updated default Top-BDII endpoint

FedCloud


Feedback from DMSU


New Known Error Database (KEDB)

The KEDB has been moved to Jira+Confluence: https://confluence.egi.eu/display/EGIKEDB/EGI+Federation+KEDB+Home

  • problems are tracked with Jira tickets to better follow-up their evolution
  • problems can be registered by DMSU staff and EGI Operations team


Monthly Availability/Reliability

Under-performed sites in the past A/R reports with issues not yet fixed:

Under-performed sites after 3 consecutive months, under-performed NGIs, QoS violations: (Jan 2024):

sites suspended: 

  • INFN-COSENZA

Documentation

IPv6 readiness plans

Campaign to upgrade HTCondor to version 10 with SSL authentication enabled

  • The campaign to decommission HTCondor <= 9 was started
    • Upgrade to HTCondor 10 (or 23) with SSL authentication enabled

Enabling SSL authentication on HTCondor 9 and 10

The HTCondor team set-up an upgrade procedure to help sites and VOs with the migration from X509 personal certificates to tokens.

Essentially it was created an intermediate step where the plain SSL authentication can be used to authenticate a client' proxy, in addition to the GSI one or to the token one:

In summary, the steps are:

  • update to HTCondor 9.0.20
  • enable the SSL authz (with priority over GSI)
  • map the users' DNs
  • test the SSL authz successfully
  • update to HTCondor 10.7.0 or later
  • install and configure the Check-in plugin

Note the usage in the last step of the HTCondor Feature channel since it is the one supporting the EGI Check-in plugin from 10.4.0.

  • In this way the sites can accept clients’ proxies and tokens at the same time while waiting for the supported VOs moving completely to tokens.

The new HTCondor version not yet included in UMD (GGUS 162689). WLCG kindly set-up a dedicated repository for HTCondor 9.0.20.

Early next year WLCG is going to start a new campaign for updating to HTCondor 23

  • in the ticket the sites will receive further information if updating to HTCondor 10 first (and already) or if waiting for that campaign. 

Important for the sites:

  • Please start collecting information from the VOs you support about the DNs that should be mapped on your endpoints
  • Mapping for the ops VO - at least the following certificates:
    • EGI Monitoring Service:
      • "/DC=EU/DC=EGI/C=GR/O=Robots/O=Greek Research and Technology Network/CN=Robot:argo-egi@grnet.gr"
      • "/DC=EU/DC=EGI/C=HR/O=Robots/O=SRCE/CN=Robot:argo-egi@cro-ngi.hr"
    • EGI Security monitoring:
      • "/DC=EU/DC=EGI/C=GR/O=Robots/O=Greek Research and Technology Network/CN=Robot:argo-secmon@grnet.gr"

Important for the VOs:

  • update the condor-client as well in coordination with the sites

Monitoring:

Issues:

  • some issues with LHCB clients (v. 8.8.10) when SSL is used as a primary authentication. It works fine when on CE it is set SEC_CLIENT_AUTHENTICATION_METHODS=GSI,SSL.
    • fixed with HTCondor 9.0.20 version.

New server for dteam VO

  • The current VOMS server voms2.hellasgrid.gr is going to be decommissioned soon.
  • CERN provided an Indigo IAM server to replace it: https://dteam-auth.cern.ch/
    • Users have been imported from the voms server
    • for the time being, new memberships will still be handled with the voms server
  • The sites need to update their configuration as soon as possible
    • Created the rpm wlcg-iam-lsc-dteam containing the .lsc file of the new server
  • Follow the instruction in https://twiki.cern.ch/twiki/bin/view/LCG/VOMSLSCfileConfiguration

Configuration example for dteam VO:

----------------------------------------------------------------------
# ls -l /etc/grid-security/vomsdir/dteam/
total 8
-rw-r--r--. 1 root root 102 Dec  6 22:04 voms-dteam-auth.cern.ch.lsc
-rw-r--r--. 1 root root 129 Jan 19  2017 voms2.hellasgrid.gr.lsc
----------------------------------------------------------------------
# cat /etc/grid-security/vomsdir/dteam/voms-dteam-auth.cern.ch.lsc
/DC=ch/DC=cern/OU=computers/CN=dteam-auth.cern.ch
/DC=ch/DC=cern/CN=CERN Grid Certification Authority
----------------------------------------------------------------------
# cat /etc/grid-security/vomsdir/dteam/voms2.hellasgrid.gr.lsc
/C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr
/C=GR/O=HellasGrid/OU=Certification Authorities/CN=HellasGrid CA 2016
----------------------------------------------------------------------

  • The information about the "vomses" file for the UI will be added to the wiki mentioned above within a few days, waiting for more sites updating their configuration.
  • Broadcast circulated to the sites on Dec 7th

DPM Decommission and migration

  • Suppor of DPM ended in June 2023
    • CERN IT will provide a minimal support to DPM until the EOL of CentOS 7, with very little effort:
      • only critical issues will be looked into
  • DPM provides a migration script to dCache (migration guide)
  • In September 2022 opened tickets to the sites to plan the migration and decommission:
  • Migrations still pending
    • BEIJING-LCG2
    • BG05-SUGrid (EOS)
    • CYFRONET-LCG2 (EOS)
    • GRIF (EOS)
    • INFN-COSENZA (dCache)
    • INFN-FRASCATI (dCache)
    • INFN-ROMA1 (dCache)
    • NCP-LCG2 (dCache)
    • UKI-LT2-Brunel (XrootD/CEPHFS)
    • UNIBE-LHEP (dCache)
    • PSNC (EOS)

    • By Q1 2024

      • UKI-NORTHGRID-MAN-HEP (XrootD/CEPHFS)
    • not clear/no reply
      • ATLAND
      • GR-07-UOI-HEPLAB
  • Please note that after June 30th no support is provided with the migration to dCache in case of issues.

New benchmark HEPscore23

The benchmark HEPscore23 is replacing the old Hep-SPEC06

Recent activities:

  • Some tests in particular with sites sending normalised reports were performed.
  • APEL
    • APEL client 1.9.2 released that adds basic HEPscore23 publishing using existing message format
      • It needs to be added to UMD
    • APEL server release candidate in testing
      • Test sending of old format normalised records to new APEL server software at Portal
      • Create an update schema to apply to Repository database during rollout
    • Working on python 3 compatibility and EL8/EL9
    • Testing a fix in ARC-CE for the proper configuration of HEPscore23
  • Accounting Portal
    • Implementation of new features
      • General data filtering by benchmark
      • Specific data filtering by benchmark
      • Custom plots according to the benchmark data filtering
    • Staging environment (https://accounting-staging.egi.eu)
  • Expected completion: mid-March
  • Please contact us if you'd like to make tests with the new benchmark
  • Information for testing the publication of accounting records with the new benchmark:

HEPSCORE application:

April 2023 GDB:

Feb 2024 WLCG Operations Coordination meeting:

Monitoring of webdav and xrootd protocols/endpoints

AOB


Next meeting

March

  • No labels