Skip to content

Docs CSC now features an automatic Finnish translation. Click here for more information.

Warning!

Puhti and Mahti are being decommissioned in stages, and their storage areas will become fully unavailable from 15 October 2026. Clean up unnecessary files and move any data you need to keep by 31 August 2026. See the Roihu data migration guide for instructions on transferring your data to Roihu.

Puhti scratch is very full: keep only active data there and move or delete everything else. No new Puhti scratch quota will be granted.

HeLI-OTS

HeLI-OTS (Helsinki Language Identification method — Off-The-Shelf) is a language identifier that detects which language a text is written in. It ships with language models for 220 languages, identified by their ISO 639-3 codes, and can process on the order of hundreds to over a thousand sentences per second on a single CPU core.

Available

  • Puhti: 2.0
  • Roihu: 2.0

License

HeLI-OTS is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. If you use it in academic work, please cite:

Tommi Jauhiainen, Heidi Jauhiainen and Krister Lindén (2022). HeLI-OTS, Off-the-shelf Language Identifier for Text. Proceedings of the Language Resources and Evaluation Conference (LREC 2022), pages 3912–3922.

Usage

Initialize HeLI-OTS with:

module load heliots

HeLI-OTS is a Java application; on CSC systems the module provides a heliots wrapper command. It reads text from an input file and writes the identified language for each line to an output file:

heliots -r input.txt -w output.txt

Useful options include -c (confidence scores), -t (several ranked language guesses), -l (restrict to a subset of languages) and -s (language-set detection). Once the module is loaded, run module help heliots for the details of the CSC installation.

More information