OpenCL on Cooley
Overview
OpenCL (Open Computing Language) is an open standard from the Khronos group that enables heterogeneous device programming (e.g. CPUS, GPUs, and FPGAs). Complete descriptions of the API, memory hierarchy, and usage can be found in OpenCL documentation and technical specifications found on the Khronos website.
Using OpenCL at ALCF
OpenCL is provided on Cooley via the NVIDIA GPU drivers for the Kepler K80 GPUs. OpenCL 1.2 is supported on Cooley. There is not an OpenCL CPU driver available on Cooley to launch OpenCL kernels on the CPUs.
Building on Cooley
To build an OpenCL application for the GPUs on Cooley, one can modify their software environment to load CUDA.
soft add +cuda-10.0
soft add +gcc-7.1.0
export CPATH=/soft/visualization/cuda-10.0/include:$CPATH
export LIBRARY_PATH=/soft/visualization/cuda-10.0/include:$LIBRARY_PATH
The following is an example to compile a simple OpenCL application.
Running jobs on Cooley
An example ‘test.sh’ job submission script follows.
#!/bin/sh
NODES=`cat $COBALT_NODEFILE | wc -l`
PROCS=$((NODES * 1))
mpirun -f $COBALT_NODEFILE -n $PROCS ./a.out
To request a single node with 10 minutes of walltime, charging to the MyProject project, one can use the following command.
ALCF Tutorials
There is an OpenCL tutorial available from the ALCF on GitHub that provides a walk through on several concepts of the programming model: introduction to API, querying device info, compiling kernels, using buffers, profiling, etc… The repo can be cloned to your working directory with one of the following commands.
git clone https://github.com/alcf-perfengr/alcl.git --branch cooley
or
git clone git@github.com:alcf-perfengr/alcl.git --branch cooley
The tutorial provides examples for C, C++, Python, and Ruby. Individual C and C++ examples can be built and run, from their respective directories using ‘make’ and the name of the example (e.g. platform).
Note: OpenCL examples will only run correctly on the Cooley compute nodes as there is no CPU driver installed. For the Python examples, users will need to first install pyopencl before running similar ‘make’ commands.