Command reference

moa run

usage: moa run [-h] [-v] [–bg] [–profile] [-j THREADS] [–ol]
[–olq OPENLAVAQUEUE] [–olx OPENLAVAEXTRA] [–oln OPENLAVAPROCS] [–oldummy] [–olm OPENLAVAHOST]

execute the template ‘run’ command. Execution depends on the template. This command can only be exeucted from within a template.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output
--bg Run moa in the background (implies -s)
--profile Run the profiler
-j THREADS No threads to use when running Ruffus
--ol Use OpenLava as actor
--olq OPENLAVAQUEUE
 The Openlava queue to submit this job to
--olx OPENLAVAEXTRA
 Extra arguments for bsub
--oln OPENLAVAPROCS
 The number of processors the jobs requires
--oldummy Do not execute - just create a script to run
--olm OPENLAVAHOST
 The host to use for openlava

~~~~~~~~~~~~~~~~~~.. _command_moa_archive:

moa archive

Archive a job, or tree with jobs for later reuse.

This command stores only those files that are necessary for execution of this job, that is: templates & configuration. In & output files, and any other file are ignored. An exception to this are all files that start with ‘moa. If the name is omitted, it is derived from the jobid parameter.

It is possible to run this command recursively with the -r parameter - in which case all (moa job containing) subdirectories are included in the archive.

positional arguments:
name archive name (default: None)
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-s, --sync Alternative approach to deal with sync type jobs - only include _ref directories (default: False)
-t, --template Store this archive as a template (default: False)

moa archive_excl

Toggle a directory to be included in an moa archive.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa archive_incl

Toggle a directory to be included in an moa archive.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa cp

Copy a moa job, or a tree with jobs (with -r).

moa cp copies only those files defining a job: the template files and the job configuration. Additionaly, all files in the moa directory that start with moa. (for example moa.description are copied as well. Data and log files are not copied!. If used in conjunction with the -r (recursive) flag the complete tree is copied.

positional arguments:
from copy from to copy to (default: None)
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-r, --recursive
 copy recursively - including all subdirectories (default: False)
-o, --overwrite
 if the target dir exists - overwrite (instead of copying into that dir (default: False)

moa dumpTemplate

moa template_dump - Show raw template information

Usage:

moa template_dump [TEMPLATE_NAME]

Show the raw template sysConf.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa err

Show the stderr of the most recently executed moa job

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa files

Show in and output files for this job

Display a list of all files discovered (for input & prerequisite type filesets) and inferred from these for map type filesets.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-a, --all Show all filesets (default: False)
-n NO_FILES, --no_files NO_FILES
 No filesets to show (default 10) (default: 10)

moa kill

Kill a running job.

This command checks if a job is running. If so - it tries to kill it by sending SIGKILL (-9) to the job.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa list

Lists all known local templates

Print a list of all templates known to this moa installation. This includes locally installed templates as well.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa lock

Lock a job - prevent execution

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa log

Show activity log

Shows a log of moa commands executed. Only commands with an impact on the pipeline are logged, such as moa run & moa set.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa map

create an adhoc moa ‘map’ job

Moa will query the user for process, input & output files. A map job maps a set of input files on a set of output files, executing the process command for each combination. The process parameter is interpreted as a Jinja2 template with the input file available as {{ input }} and the output as {{ output }}.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-t TITLE, --title TITLE
 A title for this job (default: None)

moa map!

create an adhoc moa ‘map’ job

This command is exactly the same as moa map but uses the Moa local (or user) bash history instead.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-t TITLE, --title TITLE
 A title for this job (default: None)

moa mv

Move, rename or renumber a moa job.

positional arguments:
from copy from to copy to (default: None)
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa new

Create a new job.

This command creates a new job with the specified template in the current directory. If the directory already contains a job it needs to be forced using ‘-f’. It is possible to define arguments for the job on the commandline using KEY=VALUE after the template. Note: do not use spaces around the ‘=’ sign. Use quotes if you need spaces in variables (KEY=’two values’)

positional arguments:
template name of the template to use for this moa job parameter arguments for this job, specifyas KEY=VALUE without spaces (default: None)
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-t TITLE, --title TITLE
 mandatory job title (default: )

moa out

Show the stdout of the most recently executed moa job

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa pause

Pause a running job

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa postcommand

Execute ‘postcommand’

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa precommand

Execute ‘precommand’

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa raw_commands

return a list available commands

Print a list of known Moa commands, both global, plugin defined commands as template specified ones. This command meant to be used by software interacting with Moa.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa raw_parameters

Print a list of all known parameters

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa reduce

Create a ‘reduce’ adhoc job.

There are a number of ways this command can be used:

$ moa reduce -t 'a title' -- echo 'define a command'

Anything after will be the executable command. If omitted, Moa will query the user for a command.

Moa will also query the user for input & output files. An example session:

$ moa map -t 'something intelligent'
process:
> echo 'processing {{ input }} {{ output }}'
input:
> ../10.input/*.txt
output:
> ./*.out

Assuming you have a number of text files in the ../10/input/ directory, you will see, upon running:

processing ../10.input/test.01.txt ./test.01.out
processing ../10.input/test.02.txt ./test.02.out
processing ../10.input/test.03.txt ./test.03.out
...
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-t TITLE, --title TITLE
 A title for this job (default: None)

moa reduce!

Create a ‘reduce’ adhoc job using the bash history

This command is exactly the same as moa reduce, but uses the bash history instead of the moa process history.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-t TITLE, --title TITLE
 A title for this job (default: None)

moa refresh

Refresh the template

Reload the template from the original repository.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa rehash

cache a list of variables for command line completion

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa resume

Resume a running job

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa set

Set one or more variables

This command can be used in two ways. In its first form both parameter key and value are defined on the command line: moa set KEY=VALUE. Note that the command line will be processed by bash, which can either create complications or prove very useful. Take care to escape variables that you do not want to be expandend and use single quotes where necessary. For example, to include a space in a variable: moa set KEY=’VALUE WITH SPACES’.

Alternative use of the set command is by just specifying the key: ‘moa set PARAMETER_NAME’, in which case Moa will prompt the user enter a value - circumventing problems with bash interpretation.

Note: without -s, moa needs to be executed from within a Moa job

System configuration

By specifying -s or –system, the variable is stored as a system configuration variable in the YAML formatted ~/.config/moa/config. Please, use this with care!

The dots in the key name are interpreted as nested levels, so, running:

moa set -s plugins.job.completion.enabled=false

will result in the following section added on top of the YAML:

plugins:
    job:
        completion:
            enabled: false

Adding keys like this mixes safely with configuration information that is already present. So, setting:

moa set -s plugins.job.completion.something=else

will not remove the enabled: false heading under completion:, resulting in:

plugins:
    job:
        completion:
            enabled: false
            someting: else
positional arguments:
parameter arguments for this job, specifyas KEY=VALUE without spaces
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-s, --system store this a system configuration variable (default: False)

moa show

Show parameters known to this job.

The command outputs three columns, parameter name, flag and value. The two flags have the following meaning:

  • Origin: (l) locally defined; (d) default value; (r) recursively defined; (s) system defined; (x) extra value, not in the template; and (.) not defined.
  • Private: a p indicates this variable to be private.
  • Mandatory: a lower case o indicates this to be an optional variable and M means mandatory.
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-u show unrendered values (default: False)
-a show all parameters (default: False)
-p show private parameters (default: False)

moa simple

Create a ‘simple’ adhoc job.

Simple meaning that no in or output files are tracked. Moa will query you for a command to execute (the process parameter). Note that Moa tracks a history for all ‘process’ parameters used.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-t TITLE, --title TITLE
 A title for this job (default: None)

moa simple!

Create a ‘simple’ adhoc job.

This command is exactly the same as moa simple except for the fact that Moa uses the bash history specific for the moa job or, if absent, the user bash history. This is convenient if you would like to register or reuse a command that you have alreayd executed.

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-f, --force Force this action (default: False)
-t TITLE, --title TITLE
 A title for this job (default: None)

moa status

Show job status

Print a short status of the job, including configuration

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)
-u show unrendered values (when using inline parameters) (default: False)
-R show recursively defined parameters not specified by the local template (default: False)
-p show private parameters (default: False)
-a show all parameters (default: False)

moa template

moa template - Print the template name of the current job

Usage:

moa template
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa test

Test the job parameters

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa tree

Show a directory tree and job status

positional arguments:
filter show only directories that match this filter (default: None)
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

-a, –all


moa unlock

Unlock a job - allow execution

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa unset

Remove a parameter from the configuration

Remove a configured parameter from this job. In the parameter was defined by the job template, it reverts back to the default value. If it was an ad-hoc parameter, it is lost from the configuration.

positional arguments:
parameter parameter to unset
optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

moa version

print moa version number

optional arguments:
-h, --help show this help message and exit
-v, --verbose Show debugging output (default: False)
--profile Run the profiler (default: False)

msp

moa set process

Usage:

msp

this is an alias for the often used:

moa set process