Ambition
The ICOS - eLTER Competence Centre groups two scientific communities:
- The Integrated Carbon Observation Systems (ICOS) research infrastructure: https://www.icos-ri.eu
- Long-Term Ecosystem Research in Europe (eLTER) community: http://www.lter-europe.net/
The ICOS group integrates and tests generic services from the EOSC-hub portfolio with the ICOS Carbon portal to provide a scalable environment for researchers wishing to monitor and analyse carbon processes. The CC expects to have the first runs of the portal with actual near real-time data performed before the end of 2018 and to start full production or near real-time data in January 2019. The resulting data is to be automatically ingested and be published in the ICOS Carbon Portal and at the Thematic Center. At this stage the integration of the data flow from EAA can also start. The same data workflow adapted for final data quality data will be implemented in February 2019. The final production ready version of the workflow should be finished by M18.
The eLTER group aims to develop and share data analysis workflows with the Long Term Ecosystem Research communities through user-friendly interfaces and on top of scalable compute systems. The group will test suitable common services from EOSC-hub to assess the best fit for the eLTER purposes, then based on the experience arrange the long-term setup at the core eLTER RI sites.
ICOS
Architecture & EOSC-hub technologies considered/assessed
Requirements for EOSC-hub
Technical and capacity requirements
None at the moment. Requirements are handled at the IT provider of the CC: CSC.
- Containerized data processing module, Ansible orchestration, OpenStack
- Community Linked Open Data repository (RDF, SparQL endpoint) publishes to global (meta)data systems (GEO, WMO, DataCite)
- FAIR, data license, identification, usage tracking essential
Validation plan
During the the first year of EOSC-hub the ICOS CC tested the new implementation of B2Safe at CSC and integrated this using the improved http API to the data object storage workflow in parallel with the production level iRODs data storage at PDC. Data transfer to the new B2Safe implementation has been very reliable except for 1 interruption and a few announced maintenance periods.
The CC worked on the automated receipt of the raw ecosystem flux data directly from the field data loggers and system into the Carbon Portal into the data staging area. From the staging area the data is transferred in daily packages to the ICOS CC object data store that is connected to B2Safe as its trusted repository.
The CC transferred the solution for ecosystem data processing developed in ENVRIplus based of the D4Science platform to the Cloud system, using the orchestration tools developed for the ICOS STILT footprint tool. Personnel from the ICOS Ecosystem thematic center have received training in the orchestration and integration with the ICOS semantic link framework that through persistent identifiers gives access to the input and output data through the B2SAFE repository. A preliminary test indicated that the processing of the daily raw data from 100 stations into the near real-time product could be completed within the hour, using only 20 cores.
eLTER
User stories
Instruction
Requirements are based on a user story, which is is an informal, natural language description of one or more features of a software system. User stories are often written from the perspective of an end user or user of a system. Depending on the community, user stories may be written by various stakeholders including clients, users, managers or development team members. They facilitate sensemaking and communication, that is, they help software teams organize their understanding of the system and its context. Please do not confuse user story with system requirements. A user story is an informal description of a feature; a requirement is a formal description of need (See section later).
User stories may follow one of several formats or templates. The most common would be:
"As a <role>, I want <capability> so that <receive benefit>"
"In order to <receive benefit> as a <role>, I want <goal/desire>"
"As <persona>, I want <what?> so that <why?>" where a persona is a fictional stakeholder (e.g. user). A persona may include a name, picture; characteristics, behaviours, attitudes, and a goal which the product should help them achieve.
Example:
“As provider of the Climate gateway I want to empower researchers from academia to interact with datasets stored in the Climate Catalogue, and bring their own applications to analyse this data on remote cloud servers offered via EGI.”
No. | User stories |
|---|---|
US1 | |
US2 | |
... |
Use cases
Instruction
A use case is a list of actions or event steps typically defining the interactions between a role (known in the Unified Modeling Language as an actor) and a system to achieve a goal.
Include in this section any diagrams that could facilitate the understanding of the use cases and their relationships.
Step | Description of action | Dependency on 3rd party services (EOSC-hub or other) |
|---|---|---|
UC1 | ||
UC2 | ... | |
... |
Requirements
Technical Requirements
Instruction
- Requirement number: Use numbers RQ1, RQ2, RQ3, ...
- Requirement title: Use a short but descriptive title. Use the same title in the Jira ticket 'Summary' field
- Link to requirement JIRA ticket: Open a ticket in <this JIRA queue https://jira.eosc-hub.eu/projects/EOSCWP10/issues/EOSCWP10-4?filter=allopenissues> (click on 'CREATE' button in the middle-top of JIRA)
- Source use case: Refer back to the use cases above (UC1, 2, ...)
Requirement number | Requirement title | Link to Requirement JIRA ticket | Source Use Case |
|---|---|---|---|
Example | EOSC-hub to provide an FTS data transfer service | EOSCWP10-21 - Getting issue details... STATUS | UC1 |
RQ1 | |||
RQ2 |
Capacity Requirements
EOSC-hub services | Amount of requested resources | Time period |
|---|---|---|
Validation plan
Instruction
How are you planning to test the system when it's ready? Maybe you need small scale functional tests, then stress tests at large scale? Maybe you need multiple, different test cases? How will you select and invite testers? This is the section to describe all of these.
