GreynirTranslate - mBART25 NMT models for Translations between Icelandic and English (1.0)

  Authors

Snæbjarnarson, Vésteinn ; et al.show everyone

Snæbjarnarson, Vésteinn ; Símonarson, Haukur Barri ; Ragnarsson, Pétur Orri ; Jónsson, Haukur Páll ; Ingólfsdóttir, Svanhvít Lilja ; Þorsteinsson, Vilhjálmur

  Item identifier

http://hdl.handle.net/20.500.12537/125

 Project URL

https://velthyding.is

 Demo URL

https://velthyding.is

 Referenced by

https://arxiv.org/abs/2109.07343

 Date issued

2021-09-23

 Type

toolService

 Language(s)

English , Icelandic

 Description

Provided are a general domain IS-EN and EN-IS translation models developed by Miðeind ehf. They are based on a multilingual BART model (https://arxiv.org/pdf/2001.08210.pdf) and finetuned for translation on parallel and backtranslated data. The model is trained using the Fairseq sequence modeling toolkit by PyTorch. Provided here are a model files, sentencepiece subword-tokenizing model and dictionary files for running the model locally. You can run the scripts infer-enis.sh and infer-isen.sh to test the model by translating sentences command-line. For translating documents and evaluating results you will need to binarize the data using fairseq-preprocess and use fairseq-generate for translating. Please refer to the Fairseq documentation for further information on running a pre-trained model: https://fairseq.readthedocs.io/en/latest/ - Pakkinn inniheldur almenn þýðingarlíkön fyrir áttirnar IS-EN og EN-IS þróuð af Miðeind ehf. Þau eru byggð á margmála BART líkani (https://arxiv.org/pdf/2001.08210.pdf) og fínþjálfuð fyrir þýðingar. Líkönin eru þjálfað með Fairseq og PyTorch. Líkönin sjálf og ásamt sentencepiece tilreiðingarlíkani eru gerð aðgengileg. Skripturnar infer-enis.sh og infer-isen.sh gefa dæmi um hvernig er hægt að keyra líkönin á skipanalínu. Til að þýða stór skjöl og meta niðurstöður þarf að nota fairseq-preprocess skipunina ásamt fairseq-generate. Frekari upplýsingar er að finna í Fairseq leiðbeiningunum: https://fairseq.readthedocs.io/en/latest/

 Publisher

Miðeind ehf

 Acknowledgement

Ministry of Education, Science and Culture

Project code: Back-translation data selection and filtering (V2b)

Project name: Language Technology for Icelandic 2019-2023

 Subject(s)

machine translation neural machine translation model

 Collection(s)

Clarin IS

Show full item record