Skip to content

Argonne Leadership Computing Facility

Continuous Integration via Gitlab-CI For Sunspot/Aurora

Changes from the general documentation needed for Aurora/Sunspot:

Instead of gitlab-ci.alcf.anl.gov use gitlab-sunspot.alcf.anl.gov.

ALCF Specific Variables for Aurora and Sunspot:

Cluster Scheduler Variable Name Support docs
Aurora PBS ANL_AURORA_SCHEDULER_PARAMETERS Aurora Getting Started
Sunspot PBS ANL_SUNSPOT_SCHEDULER_PARAMETERS Sunspot Getting Started

Examples which have been modified for Aurora and Sunspot:

Example: A .gitlab-ci.yml file for an Aurora project 

variables:
  ANL_AURORA_SCHEDULER_PARAMETERS: "-A ProjectName -l walltime=0:30:00  -q AuroraQueueName"
stages:
  - stage1
  - stage2
shell_test1:
  stage: stage1
  tags:
    - shell
    - aurora
  script:
    - echo "Shell Job 1"
batch_test:
  stage: stage2
  tags:
    - batch
    - aurora
  script:
    - echo "Job 2 start"
    - echo "Job end"

Example: Running a batch job on the Aurora HPC 

variables:
 ANL_AURORA_SCHEDULER_PARAMETERS: "-A ProjectName -l walltime=0:30:00  -q AuroraQueueName"

batch_test:
  tags:
    - aurora
    - batch
  script:
    - echo "Job start"
    - echo "Job end"

Example: Aurora pipeline with custom stages 

variables:
 ANL_AURORA_SCHEDULER_PARAMETERS: "-A ProjectName -l walltime=0:30:00  -q AuroraQueueName"

stages:
  - stage1
  - stage2

test1:
  stage: stage1
  tags:
    - aurora
    - shell
  script:
    - export
    - id
    - hostname
    - echo "Running on aurora with shell runner" 
    - echo test > test.txt
test2:
  stage: stage2
  tags:
    - aurora
    - batch
  script:
    - echo "Job 2 start"
    - echo "Job 2 end"

Example: Gitlab job designed to only run on merge requests 

test1:
  rules:
    - if: $CI_COMMIT_TAG                    # Do not execute jobs for tag context
      when: never
    - if: $CI_COMMIT_BRANCH == "master"     # Do not run on master, since will run on the merge request just prior
      when: never
    - if: $CI_MERGE_REQUEST_IID             # CI_MERGE_REQUEST_IID exists, so run job
  stage: stage1
  tags:
    - aurora
    - shell
  script:
    - echo "Run test 1"