Skip to content

Docs CSC now features an automatic Finnish translation. Click here for more information.

Warning!

Puhti and Mahti are being decommissioned in stages, and their storage areas will become fully unavailable from 15 October 2026. Clean up unnecessary files and move any data you need to keep by 31 August 2026. See the Roihu data migration guide for instructions on transferring your data to Roihu.

Puhti scratch is very full: keep only active data there and move or delete everything else. No new Puhti scratch quota will be granted.

vrt-tools

vrt-tools is a collection of tools for working with VRT (Vertical Text) format corpus files. It currently contains a single tool, vrt-to-json, which converts VRT files into JSON.

vrt-to-json reads VRT input where field names for token lines are given in special comments, and where the text, paragraph and sentence levels of structure are properly nested. It writes the data out as JSON objects, with each token becoming a JSON object of named fields (or an array of values in positional mode). The output can be split into a hierarchy of files with a token limit per file.

Available

  • Puhti: 1.0
  • Roihu: 1.0

License

vrt-tools is open source, licensed under the MIT License.

Usage

Initialize vrt-tools with:

module load vrt-tools

Convert a VRT file into JSON, splitting the output into numbered files of about one million tokens each:

vrt-to-json --out=/path/to/##/part-##.json --limit=1000000 --nat=ref,dephead input.vrt

The # characters in the output path are replaced with counter digits. Multiple VRT files can also be concatenated into a single JSON object via standard input.

More information