HeLI-OTS

HeLI-OTS (Helsinki Language Identification method — Off-The-Shelf) is a language identifier that detects which language a text is written in. It ships with language models for 220 languages, identified by their ISO 639-3 codes, and can process on the order of hundreds to over a thousand sentences per second on a single CPU core.

Available

• Roihu: 2.0

License

HeLI-OTS is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. If you use it in academic work, please cite:

Tommi Jauhiainen, Heidi Jauhiainen and Krister Lindén (2022). HeLI-OTS, Off-the-shelf Language Identifier for Text. Proceedings of the Language Resources and Evaluation Conference (LREC 2022), pages 3912–3922.

Usage

Initialize HeLI-OTS with:

module load heliots

HeLI-OTS is a Java application; on CSC systems the module provides a heliots wrapper command. It reads text from an input file and writes the identified language for each line to an output file:

heliots -r input.txt -w output.txt

Useful options include -c (confidence scores), -t (several ranked language guesses), -l (restrict to a subset of languages) and -s (language-set detection). Once the module is loaded, run module help heliots for the details of the CSC installation.

More information

• HeLI-OTS 2.0 on Zenodo
• HeLI-OTS on GitHub
• HeLI-OTS in the Language Bank of Finland
• Metadata record in the Language Bank of Finland

Do you find this page useful?

Yes

Thank you for your feedback!

No

Thank you! Your feedback is appreciated.

Please let us know how we could improve this page by contacting us or by creating an issue in GitHub. You can also edit this page yourself and contribute your improvements by creating a pull request.