3.1 Infrastructure Configuration

Further details on infrastructure configuration, including example config files, are available at the nf-core/configs github repository. Finer details on individual parameters are available in the Nextflow documentation here.

An example of the Imperial College Dementia Research Institute configuration file for High Performance Computing (HPC) job submission with PBSPRO is below: -

//Profile config names for nf-core/configs

params {
  // Config Params
  config_profile_description = 'Imperial College London - DRI - HPC Profile -- provided by nf-core/configs.'
  config_profile_contact = 'Combiz Khozoie (c.khozoie@imperial.ac.uk)'
  config_profile_url = 'http://www.github.com/combiz/scFlow'

  // Resources
  max_memory = 640.GB
  max_cpus = 32
  max_time = 168.h

  // Analysis Resource Params
  ctd_folder = "/rds/general/user/$USER/projects/ukdrmultiomicsproject/live/Analyses/scFlowResources/refs/ctd"
  ensembl_mappings = "/rds/general/user/$USER/projects/ukdrmultiomicsproject/live/Analyses/scFlowResources/src/ensembl-ids/ensembl_mappings.tsv"

}

executor {
    $pbspro {
        queueSize = 50
    }

    $local {
          cpus = 2
        queueSize = 1
        memory = '32 GB'
    }
}

singularity {
  enabled = true
  autoMounts = true
  cacheDir = "/rds/general/user/$USER/projects/ukdrmultiomicsproject/live/.singularity-cache"
  runOptions = "-B /rds/,/rdsgpfs/,/rds/general/user/$USER/ephemeral/tmp/:/tmp,/rds/general/user/$USER/ephemeral/tmp/:/var/tmp"

}

process {
  beforeScript = 'module load singularity'
  executor = 'pbspro'
  queue = 'pqmedbio-tput'

  //queue = 'med-bio' !! this is an alias and shouldn't be used

  withLabel:process_large {
    queue = 'pqmedbio-large'
  }

}

workDir = "/rds/general/user/$USER/ephemeral/tmp"

These parameters are highly infrastructure specific. Some points to highlight include: (1) the singularity {cacheDir} parameter specifies the cache location of the downloaded singularity image files and can be shared between different users, (2) the singularity {runOptions} parameter specifies locations to mount/bind between the container and the local storage; binding of the /tmp and /var/tmp locations inside the container to a suitable local location is essential, (3) the temporary working directory location specified by workDir should be used for a single analysis.