Document control

AreaEGI Federation Operations
Procedure status

FINAL

OwnerAlessandro Paolini 
ApproversOperations Management Board
Approval status

APPROVED

Approved version and date

v15,  

Statement

A procedure describing the steps for registering and certifying new Resource Centres (sites) in the EGI infrastructure. The certification steps can also be used to re-certify suspended Resource Centres (sites).

Dissemination Level

TLP:WHITE - Public

Next procedure reviewupon request

Procedure reviews

The following table is updated after every review of this procedure.

DateReview bySummary of resultsFollow-up actions / Comments

 

Alessandro Paolini copy from PROC09_Resource_Centre_Registration_and_Certification in EGI Wiki; updated some information

 

Alessandro Paolini updated step 6 of RC registration: new link to SEC05 procedure

 

Catalin Condurache updated step 5 of RC certificationno need for BDII configuration at Clouud sites; also only ops VO is needed to be configured at Cloud sites

Table of contents

Overview

Certification is a verification process for a Resource Centre (aka site) to become part of a Resource Infrastructure such as a National Grid Initiative (NGI), an EIRO, or a multi-country Resource Infrastructure.


This procedure applies to Grid and/or Cloud Resource Centres.


This document describes the steps required to

  1. register and certify a new Resource Centre,
  2. re-certify a Resource Centre which has been suspended.

A separate document provides the process for decommissioning a Resource Centre.

Through its parent Resource Infrastructure, a certified Resource Centre becomes a member of the EGI Resource Infrastructure to make resources available to international user communities.

The main difference between a certified Resource Centre and an uncertified or test Resource Centre is that a certified Resource Centre provides and guarantees a minimum quality of service of the resources (currently expressed in terms of monthly availability and reliability). All the requirements can be found in the Resource Centre OLA.

Definitions

Please refer to the EGI Glossary for the definitions of the terms used in this procedure.

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", “MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Entities involved in the procedure

  • Resource Centre Operations Manager: A person who is responsible for initiating the certification process by applying for membership to a Resource Infrastructure (e.g site administrator).

  • Resource Infrastructure Operations Manager: A person who is responsible for approving the integration of a new Resource Centre into the respective Infrastructure (e.g. NGI manager).

    • EGI Resource infrastructure Providers are listed on the EGI website.

  • Operations Centre (Resource Infrastructure): An entity which is technically responsible for carrying out the Resource Centre certification part of the procedure, once the membership is approved.

    • A list of EGI Operations Centres with their respective contact information is available from the GOCDB (access restricted - grid certificate needed)

  • EGI CSIRT (Computer Security Incident Response Team): EGI entity which is technically responsible for carrying out the security certification.

The Resource Infrastructure Operations Manager can determine with the Resource Centre Operations Manager the level of involvement of other actors.

Prerequisites and responsibilities

Resource Centre Operations Manager

Resource Center Operations Manager is:

  1. responsible for all Resource Centres within its respective jurisdiction. For this reason, the Resource Centre Operations Manager is REQUIRED to
    • contact the respective Operations Center if the Resource Centre is located in Europe,
    • contact the respective Resource infrastructure Provider active in a relevant geographical area if the Resource Centre is outside Europe.
      • If needed, EGI Operations can assist the Resource Centre Operations Manager to get in contact with the relevant partner.
  2. REQUIRED to provide the necessary Resource Centre information needed to complete the registration process, and he/she is responsible for its accuracy and maintenance.
    • on a yearly basis, he/she has to review the information registered on GOC-DB regarding his/her Resource Centre, in particular:
      • E-Mail
      • telephone numbers
      • CSIRT E-Mail
      • people and roles
      • service endpoints
  3. responsible for reading, understanding and accepting:

To become Site administrator go through following steps.

Resource Infrastructure Operations Manager

Resource Infrastructure Operations Manager:

  1. is REQUIRED to be responsible for all Resource Centres within its respective jurisdiction.
  2. MUST attend Resource Centre certification applications and MUST provide feedback to the requesting partners in a timely manner to accept or reject the requests received.
  3. is responsible for keeping records of the Resource Centre Operations Manager OLA agreement, as deemed suitable by the Resource infrastructure Provider
    • for example, through a signed e-mail agreement, a collection of signatories on a paper copy of the OLA, or other means.

Operations Centre

The Operations Centre:

  1. is responsible for registering (if applicable) and certifying the Resource Centre.
  2. (In the case of re-certification) MUST ensure that the issue that caused the suspension has been resolved
    • (After suspension for security reason) MUST contact the EGI CSIRT to verify that all requested repair operations have been successfully applied to fix the issue.
  3. is responsible for maintaining accurate the information registered on GOC-DB regarding the NGI itself and the RCs:
    • on a yearly basis, he/she is requested to review in particular:
      • E-Mail
      • ROD E-Mail
      • Security E-Mail
      • people and roles
      • the status of the "not certified" RCs, in according to the RC Status Workflow
      • Service Groups

Resource Center status Workflow

The general status flow that a Resource Centre is allowed to follow is illustrated by the following diagram.

Information on Resource Centre status and on how to manipulate it is available from GOCDB Documentation.


Timelines

A Resource Centre cannot be in

  • Candidate state for more than two months
  • Suspended state for more than four months

After this period the Resource Centre SHOULD be closed.

Resource Centre registration

Requirements

A Resource Centre MUST

  1. Find a respective Resource Infrastructure which will provide operational services to the Resource Center. If a provider is not yet available for your country, then an alternative existing Operations Centre can be contacted. 

  2. Provide required information as documented below.

    Required information

    The following fields are mandatory and must be sent with the application.

    • ROC: the ROC/NGI (e.g. NGI_France or ROC_LA)
    • Country: the country (e.g. Italy)
    • Domain: DNS domain used by the machines at this site (eg: cnaf.infn.it)
    • Short Name: Generic name for the site (Alphanumeric, dot dash and underscore, eg: INFN-CNAF)
    • Official Name: Official name (Alphanumeric and basic punctuation), e.g. Mysite, University of Mytown, Mycountry
    • E-Mail: Generic contact used for broadcasts, notifications and general purpose contact. The mailing list must include all the site managers responsible for that site. A suggested e-mail alias would take the form of grid-prod@site-domain.tld
    • Contact Telephone Number: Generic phone contact (numbers, optional +, dots spaces or dashes)
    • Security Contact E-mail: A mailing list for security information communication. The mailing list must be closed and its archives not published. A suggested e-mail alias would take the form of grid-sec@site-domain.tld
    • Security Contact Telephone Number: Site’s Computer Security Incident Response Team (CSIRT) telephone number (numbers, optional +, dots spaces or dashes)

    Additional information

    This information should be filled in at a later time. Although it is not required, it is recommended to complete this as much as possible.

    • Timezone: e.g. Europe/Amsterdam
    • Home URL: Site web homepage if any (e.g.: http://www.cnaf.infn.it)
    • GIIS URL: SITE-BDII LDAP URL, e.g. ldap://sibilla.cnaf.infn.it:2170/mds-vo-name=infn-cnaf,o=grid
    • IP Range: (a.b.c.d/e.f.g.h) for the case of no firewall: 0.0.0.0/255.255.255.255
    • Location: An increasing resolution ending with Country (Town, City, Country), e.g. Soho, London, United Kingdom (Alphanumeric and basic punctuation)
    • Latitude: Latitude of the site e.g. 52.370216 (+/-a.b)
    • Longitude: Longitude of the site e.g. 4.895168 (+/-a.b)
    • Description: (Alphanumeric and basic punctuation)
    • Emergency Telephone Number: phone contact for emergency procedure (numbers, optional +, dots spaces or dashes)
    • Alarm E-Mail: (for LCG Tier 1 sites) (valid email format)
    • Helpdesk E-Mail: endpoint to the local helpdesk for direct ticketing from GGUS. If not set, the ROC/NGI helpdesk contact will be used instead (valid email format)

Notes: If a Resource Centre wishes to leave the Infrastructure or the Infrastructure decides to remove the Resource Centre, the registration information MUST be kept by GOCDB for at least the same period defined for logging in the Security Traceability and Logging Policy. Personal registration information of the Resource Centre Operations Manager and Security Contact of the Resource Centre leaving the Infrastructure MUST NOT be retained for longer than one year.

Steps

The following steps are only applicable if the Resource Centre is not already registered in GOCDB.

  • Actions tagged RC are the responsibility of the Resource Centre Operations Manager.
  • Actions tagged RP are the responsibility of the Resource Infrastructure Operations Manager.
  • Actions tagged OC are the responsibility of the Operations Centre
#ResponsibleAction
0RC

Contact your Resource Infrastructure Operations Manager (contact information is available on GOCDB).

  • Provide the required information according to the template available in the "Requirements" section above.
1RP

Accept or reject registration request and communicate this result back to applicant.

  • If the Resource Centre is accepted, notify the relevant Operations Centre, handle the Resource Centre information received, and put the Operations Centre in contact with the Resource Centre Operations Manager.
2OC
  1. Forward all documentation:
  2. Clarify any doubts or questions.

Include the Operations Centre ROD, CSIRT, or help-desk teams in the step if necessary.

3OC
  1. Add the Resource Centre to the GOCDB and flag it as "Candidate".
  2. Approve in GOCDB 'Resource Centre Operations Manager' role. 
  3. Approve in GOCDB 'Resource Centre Security Officer' role. 
  4. If the RC is joining the Federated Cloud with some services, open a GGUS ticket to EGI Operations asking to flag the RC with the scope tag "FedCloud".
4RC
  1. Complete any missing information for the Resource Centre's entry in the GOCDB
    • It includes services that are to be integrated into the infrastructure:
      • The production services must have the flags "Production" and "Monitored";
      • If the RC is joining the Federated Cloud, flag the relevant services with the scope tag "FedCloud";
        • This can be done after the FedCloud tag is assigned to the site;

      • the required information for registering a service is documented at docs.egi.eu: Service registration requirements.

  2. Notify the Operations Centre that the Resource Centre information update is concluded.
  3. Note: If the external RC is considering buying host certs, please make sure they source them from an EGI recognised authority. The local national CA (usually for free or at flat rate) is likely the best source of their SSL certificates.
5OC

Check GOC DB that the Resource Centre's information is correct.

  • Resource Centre (site) roles and any other additional information.
  • Check that contacts receive email (if they are mailing lists, check that outside EGI members are allowed to post there). Site administrator MUST reply to the test email.
  • Check that the required services for a Resource Centre are properly registered.
    • Note that for Resource Centres sending accounting data from HTC jobs, a “glite-APEL” node must be registered in GOCDB. For Resource Centres sending accounting data from Cloud systems, a “eu.egi.cloud.accounting” node must be registered in GOCDB. The entry must include the correct DN to allow the message broker and accounting repository’s Access Control Lists to get automatically updated. Resource Centres can start publishing usage records after about four hours.
  • Check domain names and forward and reverse DNS.
6RCONLY for Cloud RCs - Security survey: follow Security Resource Centre Certification Procedure
7OC

Any other Operations Centre-specific requirements (e.g. join a certain VO and/or mailing list, etc.)

8OC

If all previous actions have been completed with success, notify the Resource Centre Operations Manager that the Registration is completed, and contact the Resource Infrastructure Operations Manager to notify that a new candidate Resource Centre exists and is ready to be certified.

Resource Centre certification

Requirements

  1. The Resource Centre Certification procedure is only applicable for both Resource Centres in "Candidate" or "Suspended" status state in GOC DB.
  2. A Resource Centre can successfully pass certification only if the conditions required by the Resource Centre OLA are met.

Steps

The following is a detailed description of the steps required for the transition from the "Candidate"/"Suspended" to the "Certified" state of the Resource Centre.

  • Actions tagged RC are the responsibility of the Resource Centre Operations Manager.
  • Actions tagged RP are the responsibility of the Resource Infrastructure Operations Manager.
  • Actions tagged OC are the responsibility of the Operations Centre
#ResponsibleAction
0RP

The Resource Infrastructure Operations Manager contacts the Resource Centre Operations Manager to request the subscription of the Resource Centre OLA.

1RC

The Resource Centre Operations Manager notifies the Resource Infrastructure Operations Manager that the Resource Centre OLA is accepted (if the Resource Centre has not already endorsed it before for example in case of a suspended Resource Centre), and the Resource Centre is ready to start certification.

2RP

The Resource Infrastructure Operations Manager contacts the Operations Centre asking to start the certification process.

3OC

If the Resource Centre is in the "Candidate" or "Suspended" state, then flag the Resource Centre as "Uncertified".

  • If it was in the "Suspended" state then check that the reason for suspension has been cleared.


4OC

Add Resource Centre contact information to any regional mailing list and provide access to regional tools as required.

5OC

Check:

  1. GOC-DB: All services are registered in GOCDB according to the requirements of the Resource Centre OLA, these are published and ALSO that services published in the GOCDB are valid.
  2. Information system (NOT for Cloud sites): Check that the information of compute and storage services is properly published by the Site-BDII and that the values are coherent
  3. Monitoring and troubleshooting should be possible:
    • ARC, HTCondorCE, DPM-DMLite, dCache, STORM: the OPS VO (monitoring) and the DTEAM VO (troubleshooting) are configured and supported by the Resource Centre.
    • QCG:
    1. All certificates used by monitoring users must be mapped to a local account with the same permissions as assigned to an ordinary infrastructure user.
    2. The certificate of the regional instance of QCG-Broker must be authorised.
  4. Accounting
  5. ops, dteam VOs are configured and supported. Regional VOs are configured and supported as needed.
    • ONLY ops VO should be configured for Cloud sites, along with other (regional) VOs as needed.
  6. Site is integrated in any regional tool as needed (for example, the regional accounting infrastructure if present).
6OC

Check that the registered services are fully functional either by performing manual tests or by checking on the dedicated Nagios server.

Contact the Resource Centre admins if there are problems, and ensure that they fix them. Include the ROD, CSIRT and help-desk teams if necessary. Iterate this step with the Resource Centre admins until tests pass successfully. 

Details for manual tests can be found at Manual tests.

7RCONLY for HTC RCs - Security monitoring: follow Security Resource Centre Certification Procedure.
8OC

If all preliminary tests are passed for 3 consecutive calendar days, declare an initial maintenance downtime and switch the Resource Centre status to 'Certified'.

  • This ensures that Resource Centre will appear in NAGIOS.
  • The target 'Infrastructure' value should be set to 'Production'.
9OC

The downtime should not be closed until the Resource Centre

  • appears in all operational tools
    •  ARGO/SAM NAGIOS: server 1 and server 2
    • GGUS - the Resource Centre appears in the "Notified Site" field (GGUS queries GOC-DB once per day, during the night) - GGUS search
  • All Nagios tests are passed
  • accounting data is properly published


If there are problems with a specific tool, open GGUS tickets to the relevant Support Units.

Wait at least two days after the switch to the 'Certified' status to open the ticket, the propagation of the new status to the operational tools or the publication of accounting data may take one or two days.

10OCNotify the Resource Centre Operations Manager that the Resource Centre is certified

11OC

The Operation Center can broadcast that a new Resource Centre is now part of the EGI infrastructure.

This step is OPTIONAL.

After the successful completion of these steps, the Resource Centre is considered as "Certified".