Cluster configurationΒΆ
Our pipeline is developed using CGAT-core as the workflow engine. For more information on how tRNAnalysis is written and executed. In order for our workflows to be executed over a cluster you will need to configure the cluster options by following the example below:
Currently SGE, SLURM, Torque and PBSPro workload managers are supported. The default cluster options for
cgatcore are set for SunGrid Engine (SGE). Therefore, if you would like to run an alternative workload manager
then you will need to configure your settings for your cluster. In order to do this you will need to
create a .cgat.yml
within the user`s home directory.
This will allow you to overide the default configurations. To view the hardcoded parameters for cgatcore please see the parameters.py file.
For an example of how to configure a PBSpro workload manager see this link to this config example.
The .cgat.yml is placed in your home directory and when a pipeline is executed it will automatically prioritise the
.cgat.yml
parameters over the cgatcore hard coded parameters. For example, adding the following to the
.cgat.yml file will implement cluster settings for PBSpro:
memory_resource: mem
options: -l walltime=00:10:00 -l select=1:ncpus=8:mem=1gb
queue_manager: pbspro
queue: NONE
parallel_environment: "dedicated"