We include below a set of instructions to get
EleutherAI/gpt-neox running on Polaris.
A batch submission script for the following example is available here.
The instructions below should be ran directly from a compute node.
Explicitly, to request an interactive job (from
Refer to job scheduling and execution for additional information.
Load and activate the base
We've installed the requirements for running
gpt-neoxinto a virtual environment. To activate this environment,
EleutherAI/gpt-neoxrepository if it doesn't already exist:
Navigate into the
The remaining instructions assume you're inside the
Create a DeepSpeed compliant
hostfile(each line is formatted as
.deepspeed_envfile to ensure a consistent environment across all workers