Collection of Python libraries for data analytics and machine learning.
python-data module is available on Puhti only. Versions are numbered as
X.Y.Z is the version of the Python interpreter included, and
N is a running version number starting from 1. Current versions are:
python-data/3.7.6-1installed in June 2020, includes newer versions, for example: Scikit-learn 0.23.1, Pandas 1.0.4 and JupyterLab 2.1.4.
python-data/3.7.3-1installed in July 2019.
Python-data tries to include a comprehensive selection of Python libraries for data analytics and machine learning, for example:
- Dask: Scalable analytics in Python
- Gensim: Topic modelling
- Jupyter and JupyterLab
- NLTK: Natural language toolkit
- SciPy, including NumPy, Matplotlib and Pandas
- Scikit-learn: machine learning in Python
- Seaborn: statistical data visualization
If you find that some package is missing, you can often install it yourself with
pip install --user, or alternatively create a separate virtual environment with venv, for example:
module load python-data python -m venv --system-site-packages my-venv source my-venv/bin/activate pip install my_package_to_install
With venv, you can keep separate environments for each program. The next time you wish to activate the environment you only need to run
If you think that some important data analytics or machine learning package for Python should be included in the module provided by CSC, you can send an email to email@example.com. Note that some machine learning frameworks have their own specific modules, for example: PyTorch, TensorFlow, MXNet and RAPIDS.
All packages are licensed under various free and open source licenses (FOSS).
To use this software on Puhti, initialize it with:
module load python-data
to access the default version.
This will show all available versions:
module avail python-data
To check the exact packages and versions included in the loaded module you can run:
Note that Puhti login nodes are not intended for heavy computing, please use slurm batch jobs instead. See our instructions on how to use the batch job system.
The GPU nodes in Puhti have fast local storage which is useful for IO-intensive applications. See our general instructions on how to take the fast local storage into use.
Last edited Thu Sep 3 2020