Following are the attributes of the task.
script: This denotes that a bash script would be executed.
path: Remote path of the script
value: The script content that will be executed.
target (Optional): The resource's alias where this script would be executed. If the path is specified, then the script location will be considered to be of the resource mentioned in the target attribute.
The following launch configuration would add an EMR step in order to execute the script defined in the
value attribute. To know more about EMR steps, refer the AWS document.
project: Demoresources: /path/to/resources.ymlstages:- exec_demo:- exec:script:value: |set -xecho "Running the hadoop commands. This will take some time."sleep 3mecho "Successfully ran all the commands" > /home/hadoop/commands.outtarget: playground-emr-cluster
Similarly, we can execute a script, that is located at a path say
project: Demoresources: /path/to/resources.ymlstages:- exec_demo:- exec:script:path: /opt/modules/scripts/exec_me.shtarget: playground-emr-cluster
Corresponding resources configuration would be as below.
resources.yaml- emr:alias: test-clusterkeyFile: /path/to/key-pair.pemclusterId: J-xxxxxxxxxxxx
demo.flow.ymlproject: Demoresources: /path/to/resources.ymlstages:- exec_demo:- exec:script:path: /opt/modules/scripts/exec_me.shtarget: remote
Corresponding resource target
remote would be defined as below.
resources.ymlresources:- server:alias: remotehost: xx.xx.xx.xxxusername: gigakeyFile: /path/to/keyFile.pem
If the launch configuration
demo.launch.yml is stored in the current working directory, then you can run the launch command as below.
$ teraflow demo
Else, you need to specify the path of the launch configuration.
$ teraflow -c /path/to/demo.flow.yml