Commit 170c5964 authored by Laurent Ganne's avatar Laurent Ganne
Browse files

Starting a page describing a template

parent 5299fa33
# Description of a workflow running several jobs
To describe the structure of a workflow and how HEAppE jobs are run,
the RISICO workflow at will be taken as an example.
This workflow has a preprocessing step run on a cloud instance that will produce
results for 3 dates: a given day D, the previous day D - 1, and the day before D - 2.
One HEAppE job will have to be run for each of these days.
Then a post-processing step will be run taking in input the results of the 3 HEAppE jobs.
The TOSCA application template for RISICO is available at https://github.com/lexis-project/application-templates/blob/master/weather-climate/applications/risico/risico_template.yaml
It is made of the following sections:
```yaml
# Section medata provides the name and version of the template
metadata:
template_name: RisicoTemplate
template_version: 0.1.0-SNAPSHOT
template_author: lexis
description: RISICO template
# Section imports declare which types to import from Alien4Cloud catalog
# so that they can be instantiated in our template
imports:
- yorc-types:1.1.0
- heappe-types:1.0.3
# Section topology_template describe the application template:
# input parameters, components and relationships, outputs, workflows
topology_template:
# Input parameters provided by the user and referenced in node templates below
inputs:
token:
type: string
required: true
description: "Access token"
...
# Description of components and relationships between these components
node_templates:
# Here a HEAppE job referencing in its properties the token input parameter
WRF_DAY_1:
type: org.heappe.nodes.Job
metadata:
task: computation
properties:
token: { get_input: token }
JobSpecification:
Name: WRFJob
...
# Outputs: values of attributes exposed by components described above
outputs:
ddi_post_process_results:
description: DDI path to RISICO post-processing results
value: { get_attribute: [ CloudToDDIJob, destination_path ] }
ddi_wrf_results:
description: DDI path to RISICO WRF results
value: { get_attribute: [ CloudToDDIWRFJob, destination_path ] }
# Workflows: sequences of operations on components described above
workflows:
Run:
steps:
WRF_DAY_1_create:
target: WRF_DAY_1
operation_host: ORCHESTRATOR
activities:
- call_operation: Standard.create
on_success:
- WRF_DAY_1_enable_file_transfer
...
```
Each of these sections will be described below in more details
## Section imports
This section allow to imports archives from the Alien4Cloud catalog, to add
data types and node types definitions that will be used to build the application template
For example, this import:
```yaml
imports:
- heappe-types:1.0.3
```
will import from Alien4Cloud the definition of HEAppE data types and node types
that can be found in this TOSCA file: https://github.com/lexis-project/yorc-heappe-plugin/blob/master/src/a4c/heappe-types-a4c.yaml
This file provides definitions of data types and node types needed for HEAppE.
For example a data type `org.heappe.types.JobSpecification` :
```yaml
data_types:
org.heappe.types.JobSpecification:
derived_from: tosca.datatypes.Root
properties:
Name:
description: Job name
type: string
required: true
Project:
description: Accounting project
type: string
required: true
ClusterId :
description: Cluster ID
type: integer
required: true
Tasks :
description: Tasks (at leat one task needs to be defined)
type: list
entry_schema:
type: org.heappe.types.TaskSpecification
required: true
...
```
and a node type using this data type as a property:
```yaml
node_types:
org.heappe.nodes.pub.Job:
derived_from: tosca.nodes.Root
abstract: true
description: >
HEAppE Job
# Properties having static values
properties:
token:
description: Access token
type: string
required: false
JobSpecification:
description: Specification of the job to create
type: org.heappe.types.JobSpecification
required: true
# Attributes, which will take value at runtime.
attributes:
job_id:
type: string
description: >
ID of the HEAppE job created
file_transfer:
type: org.heappe.types.FileTransfer
description: >
File transfer settings
changed_files:
type: list
description: List of files created or changed by the job execution
tasks_name_id:
type: map
description: Map of task name - task ID
# A capability (which feature this job provides), that other components
# requiring to be associated to a job, will use as a requirement
capabilities:
heappejob:
type: org.heappe.capabilities.HeappeJob
# Operations implemented by this component:
interfaces:
Standard:
create:
...
delete:
...
custom:
enable_file_transfer:
...
disable_file_transfer:
...
list_changed_files:
...
tosca.interfaces.node.lifecycle.Runnable:
submit:
...
run:
...
cancel:
...
```
So here the node type declares :
* properties (token and JobSpecification) whose value are static and must be provided before the deployment
* attributes, whose values are set by the orchestrator at runtime, for example job_id will be set once the operation create has been called
* a capability, ie. a feature provided by this component, here we describe this component is a HEAppE Job, and each component requiring to be associated to a HEAppE job will have to declare in its section `requirements` that it needs to be associated with a component having this capability `org.heappe.capabilities.HeappeJob` described later in the file
* interfaces, ie. operations supported by this component:
* the section `Standard` describe standard operations defined in the TOSCA specification, here the component supports `create` and `delete`, other standard operations that can be implemented are `start` and `stop`.
* the section `custom` describe any specific operations the developper needs to defined, here we define operations `enable_file_transfer` and `disable_file_transfer` to enable/disable the ability to transfer files to the job, and `list_changed_files` to list the files that changed during a job execution. When this `list_changed_files` is called, the attribute `changed_files` described above will be set by the orchestrator.
* the section `tosca.interfaces.node.lifecycle.Runnable` describes an extension to TOSCA that Alien4Cloud/Yorc have defined to support jobs:
* `submit` to submit a job
* `run` to monitor a job execution until the job ends
* `cancel` to cancel a job execution.
The description of these data types and node types will be imported in our template at
https://github.com/lexis-project/application-templates/blob/master/weather-climate/applications/risico/risico_template.yaml
thanks to these lines in the import section:
```yaml
# Section imports declare which types to import from Alien4Cloud catalog
# so that they can be instantiated in our template
imports:
- heappe-types:1.0.3
```
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment