Skip to content

Docs CSC now features an automatic Finnish translation. Click here for more information.

Warning!

Puhti and Mahti will be decommissioned after Roihu becomes available. Users should clean up unnecessary files and move any required data by the end of August 2026. See the Roihu data preparation instructions for details.

Puhti scratch is very full: keep only active data there and move or delete everything else. No new Puhti scratch quota will be granted.

cProfile: Python profiler

Available

  • Mahti: Any Python version
  • Puhti: Any Python version

License

Usage is possible for both academic and commercial purposes.

Usage

cProfile is a built-in profiler for Python programs. There are two ways to use the profiler. Within the code (or from the interpreter):

import cProfile
cProfile.run('functba(list_parameters)')

Now the script can be ran as a normal Python job. This will give information about how long and how many times the function gets called.

Alternatively cProfile can also be invoked as a script to profile another script:

python -m cProfile [-o output_file] [-s sort_order] myscript.py

The results can be printed out or saved to a file. By default they are ordered by name, but other options are available, too.

A report saved in a file can e.g. be visualized and interpreted by a graphical tool such as pyprof2calltree. Here is an example of a profiling done for a function:

93 function calls in 0.065 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.065    0.065 <string>:1(<module>)
        1    0.000    0.000    0.028    0.028 _numpy_fft.py:1086(irfftn)
        1    0.000    0.000    0.003    0.003 arraypad.py:102(_do_append)
        2    0.000    0.000    0.000    0.000 arraypad.py:107(_prepend_const)
        2    0.000    0.000    0.004    0.002 arraypad.py:136(_append_const)
        1    0.000    0.000    0.008    0.008 arraypad.py:964(pad)
        1    0.000    0.000    0.065    0.065 compute_correlations.py:4(compute_correlations)
        1    0.032    0.032    0.065    0.065 normxcorr2.py:33(normxcorr2)
        1    0.000    0.000    0.065    0.065 {built-in method builtins.exec}
        1    0.003    0.003    0.003    0.003 {built-in method numpy.concatenate}
        2    0.002    0.001    0.002    0.001 {built-in method numpy.copyto}
        1    0.002    0.002    0.002    0.002 {built-in method numpy.where}
        1    0.000    0.000    0.000    0.000 {built-in method numpy.zeros}
        2    0.000    0.000    0.000    0.000 {method 'astype' of 'numpy.ndarray' objects}
        2    0.003    0.002    0.003    0.002 {method 'copy' of 'numpy.ndarray' objects}
        1    0.020    0.020    0.028    0.028 {mkl_fft._pydfti.irfftn_numpy}

Keep in mind that running the profiler has to be done in the same way as running a (puhti or mahti) batch or interactive job.