NAMD
NAMD is a parallel molecular dynamics code designed for high-performance simulation of large biomolecular systems. The software is developed and distributed by the Theoretical and Computational Biophysics Group at the Beckman Institute of the University of Illinois.
Available
The following versions are available:
- Roihu-CPU: 3.0.2, 3.0.3
- Roihu-GPU: 3.0.3
- LUMI: 3.0.2-cpu, 3.0.2-gpu
License
CSC has obtained a computing center license, which allows usage for non-commercial research. For commercial use, contact namd@ks.uiuc.edu. See also acknowledging usage below.
Usage
NAMD can be run either with CPUs or with GPUs + CPUs. GPU versions support single-node jobs only.
See available NAMD versions on your system with:
And load the version you want (e.g. on Roihu-CPU):
Performance considerations
Tests show that leaving one core for communication for each task is beneficial when running on multiple CPU nodes:
This is also recommended by the NAMD manual. Please test with your input.
Make sure --ntasks-per-node multiplied by --cpus-per-task equals 384
(Roihu-CPU) or 128 (LUMI-C), i.e. all cores in a node. Try different ratios and
select the optimal one.
- Optimal settings depend on the amount of resources in addition to system and
run parameters. Try different numbers of
namd_threads(as set in the Roihu script below) to find the best balance for your system. - Remember that using more resources to get results faster is also more expensive in terms of consumed Billing Units. To avoid wasting resources, ensure that your job actually benefits from increasing the number of cores. You should get at least a 1.5-fold speedup when doubling the amount of resources.
- To test your own system, run e.g. 10 000 steps of dynamics and search for the
Benchmark time:line in the output.
To run on multiple GPUs efficiently, you typically need a rather large system
composed of at least several hundred thousand atoms. Check with your system and see the
NAMD website
for available features that allow you to maximize the performance
of multi-GPU runs. Importantly, enabling GPU-resident mode using configuration
file option GPUresident on is extremely beneficial.
Info
We currently recommend running multi-GPU NAMD simulations on LUMI-G, which offers more abundant GPU resources than Roihu.
Batch script examples
The script below requests 24 tasks per node and 16 threads per task on one full Roihu node (384 cores). One thread per task is reserved for communication.
#!/bin/bash
#SBATCH --job-name=apoa1-namd
#SBATCH --account=<project>
#SBATCH --partition=medium
#SBATCH --time=00:20:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=24
#SBATCH --cpus-per-task=16
module purge
module load namd/3.0.2
# leave one core per process for communication
(( namd_threads = SLURM_CPUS_PER_TASK - 1 ))
srun namd3 +ppn ${namd_threads} apoa1.namd > apoa1.out
# while NAMD suggests using 1 thread per task for communication
# (as above), all cores for computing can be tested with:
# srun namd3 +ppn ${SLURM_CPUS_PER_TASK} apoa1.namd > apoa1.out
The script below requests 1 GCD and 7 CPU cores. Note that each GPU node on LUMI contains 4 GPUs, which are in turn composed of 2 GCDs (graphics compute dies) that are recognized by Slurm as individual GPU devices. Moreover, there are 56 CPU cores available per node, so use at most 7 cores per reserved GCD.
#!/bin/bash
#SBATCH --partition=small-g
#SBATCH --account=<project>
#SBATCH --time=00:15:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=7
#SBATCH --gpus-per-node=1
module use /appl/local/csc/modulefiles
module load namd/3.0.2-gpu
srun namd3 +p ${SLURM_CPUS_PER_TASK} +setcpuaffinity +devices 0 stmv.namd > stmv.out
The script below requests 8 GCDs and 50 CPU cores. To mitigate load
imbalance due to PME, less CPU cores are assigned to the single GPU device
performing the PME work using the +pmepes option. For the STMV case
using 8 GCDs, best performance is obtained by assigning 7 cores for each
non-PME device and only 1 core for the PME device. Note that +p is set to
the total number of CPU cores, i.e. 7*7 + 1 = 50. Please test different
options for your system.
#!/bin/bash
#SBATCH --partition=standard-g
#SBATCH --account=<project>
#SBATCH --time=00:15:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=50
#SBATCH --gpus-per-node=8
module use /appl/local/csc/modulefiles
module load namd/3.0.2-gpu
srun namd3 +p ${SLURM_CPUS_PER_TASK} +pmepes 1 +setcpuaffinity +devices 0,1,2,3,4,5,6,7 stmv.namd > stmv.out
Submit batch jobs with:
References
The NAMD License Agreement specifies that any reports or published results obtained with NAMD shall acknowledge its use and credit the developers as:
NAMD was developed by the Theoretical and Computational Biophysics Group in the Beckman Institute for Advanced Science and Technology at the University of Illinois at Urbana-Champaign.
Also, any published work which utilizes NAMD shall include the following reference:
James C. Phillips, David J. Hardy, Julio D. C. Maia, John E. Stone, Joao V. Ribeiro, Rafael C. Bernardi, Ronak Buch, Giacomo Fiorin, Jerome Henin, Wei Jiang, Ryan McGreevy, Marcelo C. R. Melo, Brian K. Radak, Robert D. Skeel, Abhishek Singharoy, Yi Wang, Benoit Roux, Aleksei Aksimentiev, Zaida Luthey-Schulten, Laxmikant V. Kale, Klaus Schulten, Christophe Chipot, and Emad Tajkhorshid. Scalable molecular dynamics on CPU and GPU architectures with NAMD. Journal of Chemical Physics, 153:044130, 2020. https://doi.org/10.1063/5.0014475
In addition, electronic documents should include a direct link to the official NAMD page.