En-Is Synthetic Parallel Corpus (21.07)

Símonarson, Haukur Barri; Snæbjarnarson, Vésteinn; Þorsteinsson, Vilhjálmur

dc.contributor.author	Símonarson, Haukur Barri
dc.contributor.author	Snæbjarnarson, Vésteinn
dc.contributor.author	Þorsteinsson, Vilhjálmur
dc.date.accessioned	2021-09-28T09:24:19Z
dc.date.available	2021-09-28T09:24:19Z
dc.date.issued	2021-07-01
dc.identifier.uri	http://hdl.handle.net/20.500.12537/127
dc.description	Synthetic back-translated training corpus for neural machine translation. An mBART25 finetuned on English-Icelandic data created the corpus by translating Icelandic and English sentences. The English sentences (44,7m) are retrieved from the Wikipedia, Newscrawl and Europarl corpora. The Icelandic sentences (31,3m) are sourced from the Icelandic Gigaword Corpus. The source texts are the same as last years corpus https://repository.clarin.is/repository/xmlui/handle/20.500.12537/70 . The files include all 4-5 beams from the beam-search generated during the translation process. The format is a tab-separated document (line-index , text). Samhliða gervimálheild með bakþýddum þjálfunargögnum fyrir vélþýðingar. mBART25 líkan var yfirfært á ensk-íslenskar þýðingar og það líkan notað til að mynda málheildina með því að þýða enskar og íslenskar setningar. Ensku setningarnar (44,7m) eru fengnar úr Wikipedia, Newscrawl og Europarl málheildunum. Þær íslensku eru fengnar úr Risamálheildinni (31,3m). Frumtextar eru þeir sömu og málheild síðasta árs https://repository.clarin.is/repository/xmlui/handle/20.500.12537/70 . Skrárnar innihalda alla 4-5 geisla úr geislaleitinni sem var framkvæmd sem hluti af þýðingarferlinu. Skráarsnið er .tsv (línunúmer, texti).
dc.language.iso	isl
dc.language.iso	eng
dc.publisher	Miðeind ehf
dc.relation.isreferencedby	https://arxiv.org/abs/2109.07343
dc.rights	Creative Commons - Attribution 4.0 International (CC BY 4.0)
dc.rights.uri	https://creativecommons.org/licenses/by/4.0/
dc.rights.label	PUB
dc.source.uri	https://velthyding.is/
dc.subject	machine translation
dc.subject	back translation
dc.subject	neural machine translation
dc.subject	parallel corpus
dc.title	En-Is Synthetic Parallel Corpus (21.07)
dc.type	corpus
metashare.ResourceInfo#ContentInfo.mediaType	text
has.files	yes
branding	Clarin IS Repository
demo.uri	https://velthyding.is/
contact.person	Vésteinn Snæbjarnarson vesteinn@mideind.is Miðeind ehf
sponsor	Ministry of Education, Science and Culture Back-translation data selection and filtering (V2b) Language Technology for Icelandic 2019-2023 nationalFunds
size.info	76000000 sentences
files.size	11568547022
files.count	12