PyTorch

Machine learning framework for Python.

Available

Available on Puhti only. Currently supported PyTorch versions and corresponding modules to load:

  • 1.8.0a0 using Singularity: pytorch/nvidia-20.11-py3
  • 1.7.0a0 with experimental Horovod support using Singularity: pytorch/nvidia-20.08-py3
  • 1.6.0: pytorch/1.6
  • 1.6.0a0 using Singularity: pytorch/nvidia-20.07-py3
  • 1.5.0a0 using Singularity: pytorch/nvidia-20.03-py3
  • 1.5.0a0 using Singularity: pytorch/nvidia-20.02-py3
  • 1.4.0: pytorch/1.4
  • 1.4.0a0 using Singularity: pytorch/nvidia-19.11-py3
  • 1.3.1: pytorch/1.3.1
  • 1.3.1 with Horovod support using hpcx MPI: pytorch/1.3.1-hvd
  • 1.3.1 with Horovod support using mpich MPI: pytorch/1.3.1-hvd-mpich
  • 1.3.0: pytorch/1.3.0
  • 1.2.0: pytorch/1.2.0
  • 1.1.0: pytorch/1.1.0
  • 1.0.1: pytorch/1.0.1
  • 0.4.1: pytorch/0.4.1

All modules include PyTorch and related libraries with GPU support via CUDA.

Modules starting with nvidia are based on NVIDIA's optimized container images from NGC with some CSC specific additions. For these, the included PyTorch versions are typically not the release versions, but the newest git commit at the time of creation. See NVIDIA's PyTorch container release notes for more information on provided software versions.

Note

In Singularity-based modules you need to launch Python with singularity_wrapper, see here for a usage example.

If you find that some package is missing, you can often install it yourself with pip install --user, or singularity_wrapper exec pip install --user in Singularity-based modules. If you think that some important PyTorch-related package should be included in the module provided by CSC, you can send an email to servicedesk@csc.fi.

License

PyTorch is BSD-style licensed, as found in the LICENSE file.

Usage

To use this software on Puhti, initialize it with:

module load pytorch

to access the default version, or if you wish to have a specific version (see above for available versions):

module load pytorch/1.4

Please note that the module already includes CUDA and cuDNN libraries, so there is no need to load cuda and cudnn modules separately!

This command will also show all available versions:

module avail pytorch

To check the exact packages and versions included in the loaded module you can run:

list-packages

Note

Note that Puhti login nodes are not intended for heavy computing, please use slurm batch jobs instead. See our instructions on how to use the batch job system.

Example batch script

Example batch script for reserving one GPU and 10 CPUs in a single node:

#!/bin/bash
#SBATCH --account=<project>
#SBATCH --partition=gpu
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=10
#SBATCH --mem=64G
#SBATCH --time=1:00:00
#SBATCH --gres=gpu:v100:1

module load pytorch/1.4
srun python3 myprog.py <options>

For Singularity-based modules the last two lines would instead look like:

module load pytorch/nvidia-20.02-py3
srun singularity_wrapper exec python3 myprog.py <options>

Note

Please do not read a huge number of files from the shared file system, use fast local disk or package your data into larger files instead! See the GPU-accelerated machine learning guide for more details.

Big datasets, multi-GPU and multi-node jobs

Please see our tutorial for GPU-accelerated machine learning, which covers more advanced topics, including how to work with big data sets, multi-GPU and multi-node jobs.

More information

Last edited Fri Nov 27 2020