OCR Post-Processing Tool for Icelandic 22.10

 
Clarin IS Repository
  Authors
Jasonarson, Atli ; Steingrímsson, Steinþór ; Magnússon, Árni Davíð and Ingimundarson, Finnur Ágúst
  Item identifier
http://hdl.handle.net/20.500.12537/271
 Project URL
https://github.com/atlijas/ocr-post-processing
 Date issued
2022-10-01
 Type
toolService
 Language(s)
Icelandic
 Description
ENGLISH: This entry consists of two trained transformer models to correct OCR errors, along with ca 50,000 line pairs of OCRed/corrected text. The models were trained on ca 900,000 lines (~7,000,000 tokens) of which only 50,000 (~400,000 tokens) were from real OCRed texts. It can be assumed that increasing the amount of such data can significantly improve the tool. More info in README.md. ICELANDIC: Þessi gagnahirsla inniheldur tvö þjálfuð transformer-líkön til leiðréttingar á ljóslestrarvillum, auk u.þ.b. 50.000 línupara úr ljóslesnum/leiðréttum textum. Líkönin voru þjálfuð á u.þ.b. 900.000 línum (~7.000.000 orð) en af þeim voru ekki nema um 50.000 (~400.000 orð) úr raunverulegum ljóslesnum gögnum. Ætla má að aukið magn slíkra gagna geti bætt tólið umtalsvert. Nánari upplýsingar í README.md.
 Publisher
The Árni Magnússon Institue for Icelandic Studies
 Acknowledgement

Ministry of Education, Science and Culture (Mennta- og menningamálaráðuneytið)

Project code: L11 - Error models for OCR

Project name: Language Technology for Icelandic 2019-2023

 Subject(s)
ocr
 Collection(s)
Clarin IS
Show full item record
 

 Files in this item

 Download all files in item (1.24 GB)
This item is
Publicly Available
and licensed under:
Apache License 2.0
Icon
Name
ocr-post-processing.zip
Size
1.24 GB
Format
application/zip
Description
ocr-post-processing
MD5
2d501a7485cafb73b3d4d6f287a92726
 Download file  Preview
 File Preview  
  • ocr-post-processing-clarin
    • setup.py3 kB
    • README.md11 kB
    • infer.py8 kB
    • train.py10 kB
    • .git
      • logs
      • info
        • exclude240 B
      • config387 B
      • COMMIT_EDITMSG18 B
      • ORIG_HEAD41 B
      • FETCH_HEAD106 B
      • packed-refs112 B
      • index51 kB
      • HEAD21 B
      • refs
      • description73 B
      • hooks
        • post-commit278 B
        • push-to-checkout.sample2 kB
        • applypatch-msg.sample478 B
        • commit-msg.sample896 B
        • pre-push.sample1 kB
        • pre-rebase.sample4 kB
        • post-update.sample189 B
        • post-checkout282 B
        • pre-receive.sample544 B
        • pre-applypatch.sample424 B
        • pre-push272 B
        • update.sample3 kB
        • pre-commit.sample1 kB
        • pre-merge-commit.sample416 B
        • fsmonitor-watchman.sample4 kB
        • post-merge276 B
        • prepare-commit-msg.sample1 kB
      • branches
        • objects
          • a0
            • 0cc45077e6e2ab325c4508fc17a1d86e6ac3c1114 B
          • 4f
            • e1ffbb39d90d8c17c99abcd955dac6b345aaca162 B
            • 851128e66c60390dcc18a5c83059ca0fcc15fa344 B
          • 4e
            • bfd271f4582a9756a0551df63ee7f9bde4e5bb331 B
          • 4b
            • 0b22ce995b8be4b4354974da7ed6001942ed6451 kB
          • ef
            • d033c7abbb2f3b657d27eba319512abbc893a3114 B
          • ee
            • 9b30bec9b2e347e60e0b618fe0aefe51c4ff96743 B
          • ed
            • b270b00a61058848785b5ca3a7bd7247895b50273 B
            • 9ef00e015491898b70c90ce86ef400a6d1cca5152 B
          • ec
            • e3293c58783185b83930dcfab06d8c74223ac42 kB
          • ea
            • 04ddde886b1132fd7b248826fb77da42f7a745157 B
          • 58
            • 2dd1063940671ae8faafce4ef0ff3bd7caad68114 B
          • 56
            • 32a4e02d144eafc1a2f1bf58556466b97458c0277 B
            • 2ebd51f8790fcfd0e7c4453a70cb53401e8a1e299 B
          • 55
            • f59aeff4ddab2f7ee0f3b2a72b978dc23a6753114 B
          • 54
            • b59f212448e455b13793cc65172351fadbe0ea743 B
          • 52
            • 46f94310e8cff3cc47c01d48e10a2e6ca2ee761014 B
          • f9
            • 037ef315a9089618630dac1dd3aa9fd1e0140f743 B
            • 1f9e72346889c20f603274aa062a857c45aa78157 B
          • f8
            • 50a43b27ca03792692a305ddbd269d94f67a16167 B
          • info
            • f5
              • 7553ec521fdc50b33e778369ef7e9cd86e4aca1 kB
            • f2
              • 8c63f4b3825166a0b3939b19919e91864a66a4153 B
            • f1
              • a2bf790ee8c467f218a222bb3fac01fede31a5171 B
            • 3e
              • 802cd1b515d0fbfd9ee992605c513e62270f69114 B
            • f0
              • cad4069f7efe1e5440c974ecece52531b0ddf1146 B
              • 29de0dbe0ab24c93d4a7467e5285a15ba3e064842 B
            • 3d
              • 332846513a88288e46b761887e8fc21804f4c44 kB
              • 7e6378540b65b7bd7d75e7fe75fc648fc9283d303 B
              • 72ab7d1036c966f22c96f214d7bdfb0464a0cc156 B
            • 3a
              • e7ad7bf4945e9c6991287b63495e9876a9fac1286 B
              • 23ccd7f4b88bfbd2cacffe21674c6f875f3a75352 B
            • 9f
              • aebc0fb0159f70b4ca823748ea79216b70d3837 kB
            • 9e
              • 74cb3f8a6a54cab4cea6b246552353d0eef5d5114 B
              • 1fb39ae845afc1a3aa461f42186b9dca0dcb9581 kB
            • 9c
              • 9a30ebbd06b039f9e31dfec194da5b7210b610114 B
              • 40f08ef81c46e4473c7d63770acc63ab5a75dc81 kB
            • dd
              • e32a93cb2506cf7eb03cdc654f76b83214dada114 B
            • db
              • d9e4cb370ca4d2c8a84345a7ced29e0d02d3222 kB
            • pack
              • pack-84f47de17fadded1ab13e39c24b0ad90acb91428.pack40 MB
              • pack-84f47de17fadded1ab13e39c24b0ad90acb91428.idx35 kB
            • 49
              • 765efc2e8c7195a5e2c17b000e5a4feda49406813 B
              • e50c77969f430e70d30658dcb3072f47cdefca156 B
            • 48
              • 80aad488d53ddbc70305571e4f81bcce846b3f2 kB
            • 46
              • bef4ecdaecae8ab8350d7eaafe15b884db9ea6369 B
            • 43
              • d510234ef1ecd2ee1ee2818d47b52e0e647196897 B
              • 8a9f0a82ef46e84466d91d3682b5d1f01f532a926 B
              • 81dacaaa501f667e55c1c1f759d3423a5232a5165 B
            • 41
              • c904f69733dbe3da21951597b5cbf54ddbf861854 B
            • e8
              • c2cd29fe131cd62dfb0a7ab256c66e81b8dd3b64 B
            • e7
              • 76239819c1e745ed58d64694b63070e5d16cba55 kB
              • 7d36b77381a1db712bc6289c1816766d0356d5169 B
            • e5
              • 6f93f1788ce4f472a104c4e9cd1a7c6c1841577 kB
            • e3
              • a8b95b51ba3bdd457606c6a503a05adf6aa74582 kB
            • e2
              • 03537558caa3a78239acc136acaeeb8e5fa0d01 kB
            • 2f
              • 67b3cbc610c15d51a59b315562ead979ecf0a0164 B
            • 2d
              • 430bc8b5488a9f91fd88d9bf13b8dbf811ac9d2 kB
            • 2c
              • 408c8fd3096a93dddea00daafb1149638175473 kB
              • f0b939575d96a9f492c38c64757a54ade2d27521 kB
            • 8d
              • dd112e757282f17bd21e50435f039c63cd0d14319 B
            • 8c
              • f337f3ee6b7f46597bb3ffb499c4a56a768fc12 kB
            • cf
              • 3963885737e7548ace51b71c77cdf95b939855114 B
            • ce
              • 7c5ea1b7a1573c3117895d814a25ac0520db2e159 B
              • b1962d8158db3f557c228287d9a8b2c154163e172 B
            • 39
              • 71101422afee964ca2ca2d6dae30a0f986a8d5153 B
            • ca
              • 38e12677ea191d1b0db9491184376bf308f83c369 B
            • 33
              • 42acbf15bbfb1144daff5283269294d1cf0bfd1 kB
            • 98
              • eebe6eacf966f90dda246e1b86a32c7953a513172 B
              • 626a349a85b2299d76d5b7eef9bafb58d3ed7e50 kB
            • 32
              • 0f9ebf9857fd3a2065750f8986eec8d377e998338 B
            • 31
              • 94911cbd3d7cc835a34e1648207b3ac11ac2fa329 B
            • 96
              • a0845946705ef0acb1e180258796181776ac372 kB
            • 30
              • 5380f94dcef50f4ad21624365501be1ef93144148 B
            • 95
              • 6256715f8c4c372d10edf3d9755e0ba3254171855 B
              • 01f3d07a794652f020c16d0168a8a75d4f7975166 B
            • 94
              • 163b27a1bd1e15beae1f852552fc84054a548610 kB
              • 1f45937a9db6dcef0b47d2903c26e68ec3f9e03 kB
            • 93
              • 26724a50a941fe2349621dd22f08a59eb6daa43 kB
            • d7
              • 860bb5feaecbd93673d313b38184eef830495c926 B
            • 92
              • 0a380888f92f85353c8b547b66b460704fdc0618 kB
              • e66ff0827c441224c285a94b4478e16fa207ac81 kB
            • 91
              • e782a5350c17ccc7ce8cbe5c11102fd70927ad113 B
              • 8df4c2b6ae3c1181cbe1eb52c99b27124548eb114 B
            • d5
              • 6686fc7426c8a1ee2f7c7b1e1d3c25fc4205c981 kB
              • 9de644123c68416d9c721f6aa6e3b5007df34a7 kB
            • d4
              • d3e76580db959c6a8ef89968940e535fbe8970163 B
            • d2
              • 830de5f2c08adabd812dbdcb42e7d750a55bf5926 B
              • a2cb6e36534d325a070a5570d2169e2a683e1f11 kB
            • 1e
              • 52cd3546b62decd8b7d4b905fa9889c91cebe14 kB
            • 1b
              • 813b80856ca949504469f0ad4c1bc2a2d9545d359 B
              • 51b328fed5a7e16ef477ab4c6a3c1402b3b7d4334 B
            • 7f
              • 84b6a027e6f9ebcc694092f062b0eedcdab3943 kB
              • 7643b34d0a9fabeebd475d5a1327e447cee6ee675 B
            • 7e
              • f346fe9bd03ff6f96c367a3f7dd24bf9b1cd9d462 B
            • 7d
              • b4612192b520501e160e2b93cfe415d7866df1927 B
            • bb
              • 65d3f5c71f59e0f0322c16076fed6629fe0b1c3 kB
            • ba
              • 4106d35c3c72aad66645525d2391c7b079ed1581 kB
            • 29
              • deeb5b049371dba7127823add8254a8b16bd1d171 B
            • 28
              • fbddf0e1558393bdff34fd20f14bc57d25c65d81 kB
              • 28187188a8993992a49c4af61aee08d1e50aea1 kB
              • 8c9f8631fa79368e7699ba89c94bd5ec8bf59a925 B
            • 26
              • 362fd7ec33713089d3a44c34ff30a8066a9a7c7 kB
              • bdd9b28c764de38dca84bab0fc94f0cdeeb3cd157 B
            • 25
              • b3909ed3ec0e7b0373e2bd5bab26e59f33528a335 B
            • 23
              • b2ef331dde484d65321f858c7cb254356ee06e927 B
            • 86
              • d830807984d6a8827af812e93fd844312ff7f1115 B
            • 21
              • 9915078a82cff1d22a3421340559c672ce6d832 kB
              • 508116fe9c2bc095a80166b08a1a2b290878fa823 B
              • f0257c66ad7dfd2c7ca85e1288c57765d8fbdd708 B
            • 85
              • 9b3e2378a832a44376f41eb03db5602218c0eb334 B
            • c9
              • 0f530291efe50d62cc1c48d0b9035ce20114c1333 B
            • 82
              • 885928337caf4cc828faef7e4ec720984a1601221 B
              • 9f01d28346906748bb74fcd1778cc927bc65d73 kB
            • 80
              • ed6504d179cf8129ad20a654d67beda0f51a5a267 B
            • c2
              • 6aa4ad871aa264b057a37f1a5a1661d6e92b16926 B
              • 3ad1be17e338964ab950d2bb4600b9e9f4d378141 B
              • 52a822c8132397fb3d68e56ce45122ea11cc8a114 B
            • 0d
              • ea69e6e5aec1c1f60eada07bb52d61cd06e8113 kB
            • 0b
              • 1fe5305222f726b584419f7ac87798eae368892 kB
            • 6f
              • 3434335593d2c53cb68e2b293c3a1d31a70ddb157 B
            • 6a
              • 458ee0b4c40db90a79d463c74d168a62381f3d2 kB
              • d04d4abb63adc2d561c4a05e0837868c6437e7171 B
              • 3410bb8450ea8c9ef220d90532cf480b8f281a171 B
            • ae
              • e9223f3f6a356ab71b5c4d174fc534d2ec87c6171 B
              • 7c11b1b61309ca90a07f2936de1459e08ecabb927 B
            • ac
              • e5859c090eb735592a6e6de351b78cfd406cf5927 B
            • ab
              • 4048777f1d65604fd1479e36cd9274674ba3f7380 B
            • aa
              • e58051db029d1709df36da79ad1a7a5a6ee546299 B
              • 39d72248b9b19c640674c40cf7d457c118f9d62 kB
            • 18
              • e1283fc5265ff6942243fc459526cc2f75b77c437 B
            • 14
              • fb6ec68a9259f53516b0455ab65cd4f088814f81 kB
            • 12
              • 63e13de5820a8a158ced6aabc871ea89b3ea95825 B
              • 735cc1b06ce25b688c6e092ffdaeb28b7ec4492 kB
            • 77
              • 490082ee40f3936f0417c1533c1017f660a6c7825 B
              • 8bdd55107a0c34477e1a3c6ceb90f9a09f3dd8926 B
              • 3c3491d1ccf941623b2777861849b4482e3549327 B
              • 6f973539ba4d9c723115d63b3e33b4acd8d17c172 B
              • 58d966c5ad0b8d673a6f2c261aafa4803feb16300 B
            • 11
              • 640006a3c699de7fb70299191cbb886ededca92 kB
            • b9
              • 00b5c0f3032901701722dc66af1db0932869c9172 B
            • 74
              • d250ac0ec4d907e95dd0192dc4076a98530df12 kB
              • ec78514b9301b2c4190bcb979c1d565a03fd56157 B
              • 5667071c96e3c6f15ef646a8eb9d27c7e97cd06 kB
            • b8
              • f287c97cc6932fd58d4f04000a7d773df5a888166 B
            • b7
              • 5290208b83b6eca6025e5144d78f02926a7fa2200 B
              • 1a0b0030dc2ec5596516613c85016f20118a783 kB
            • b5
              • edefc936af2801d8b4ab9eb7511b870eab1dd7372 B
            • 71
              • e41965b90afc3e19ef8baba3a38daf6ed3cdd8926 B
            • b4
              • 2d0f61d9596e6c6a122fa21d74f05676bf8073152 B
            • 5f
              • c29d0f9331c8ebf50d17dce1f8261000a338f02 kB
              • 05eb243d601d61b94e804b85b75c16b3aea761824 B
            • 5e
              • da91028fffd62ab7efbcbc6269c98d6aa08d177 kB
            • 5d
              • b4a3a7c2d2b371febe8275c8585bd34d4033712 kB
              • 01e0294afdeabd620ec058f4c7abef22dc255f824 B
            • 5b
              • 8dacc04f8a495900a9206644e21278b7cd67842 kB
            • 5a
              • 47f5324f625fd53b0ec0fb3175910829cbcc45888 B
            • fe
              • 307de3fe098134072857f7b70d9e59c4cba44f842 B
            • 08
              • 1672aae2dc21871a60ece18713e38259f8e963331 B
            • fa
              • 7034550853b8751c8c491fa1dc0ede84848d847 kB
              • 0e48b40e02f34375b7eccff14f523b2972f3c7115 B
            • 68
              • 4a70c703f48bd58ff90e2787ccfd7208a5c3892 kB
            • 02
              • d7c16895ecc0ddb1512194f4e0c3f959a69897150 B
            • 67
              • cfb214df98ba99149e3de14f681f380c16fa762 kB
            • 01
              • 617005c2d9c5fe0446e0577f4e3385bb00ef312 kB
              • 1e9cfc37cab649bde50e8be600801f4db373a9438 B
            • 00
              • a587d8f38c005d8b82aead45a72a41662d3cf64 kB
            • a7
              • 294a59d85fb35f45dceac050207250146a5f1b239 B
              • 4e952993ddbc72ec3404a828261b490458e38d156 B
            • 61
              • b5938dcd2e02778ab21bb7a139d7527c6b80cd3 kB
            • 60
              • df0838282762ab3f540fc01bb43ac0c0f9ca60824 B
            • a3
              • 3c2c7a598ca47db289fa45ef7a8a0c6a5f7f83157 B
            • a2
              • b86512f88ce027c6043ef682e8680fc614f792949 B
          • lfs
            • incomplete
              • cache
              • tmp
                • objects
                  • 81
                    • 68
                      • 8168e4f6b00714aed0c32f4d721d02b789933cf4f7ac985b7704a35247e401122 MB
                  • e9
                    • 5c
                      • e95c3a2a66136f5c98f4e585909cf37785e83933f96104406ffe2ff456fb877f47 MB
                  • 6d
                    • 38
                      • 6d3852eb1b1b963ea9ee0a7c66c6f939ee7e1458602bc9a1222ffc1d66aeabd035 kB
                  • 96
                    • 14
                      • 96145e9cddac18b5ea54004349151bc9a00a7edb7cf5e67b55972cb8fe44901a76 MB
                  • 0b
                    • b4
                      • 0bb43aa36e4a66727aed36a7e1597b4b61a2edaa46b455717aa4ba86b7ab4d66167 MB
                  • 64
                    • ab
                      • 64ab4483488ae1d63f9cffd42ecff7fc7eab737b1dca67a0dbe2159564a8c4e428 kB
                  • 93
                    • b1
                      • 93b12bca0b417d90838e88493c28b839ff81caad74292c1f6f9ec276412cb08f274 kB
                  • 9b
                    • 4f
                      • 9b4f3b494e720654bbef3a1fba82ab12e9c845895fe9966a47a40041b071e5c93 MB
                  • 00
                    • 99
                      • 00998d23429f5712a9e00b1da472563d3d2ba0f35310ec397f1558f795df698b2 MB
                  • 90
                    • 42
                      • 904283e53a01ba3f2d02d76158bcf0da75bf84a6362dce2d90b1f8f8b451696849 MB
                  • 19
                    • 82
                      • 198297655fddc94a946e0fdc2d14648b0a6dd4c8a15d2e77bea6d85bd4857f8c34 kB
                  • 18
                    • d3
                      • 18d33630e429bf09eb728ed4311d7b997718a312d4ae67161ebfbca02cae0e809 kB
                  • f9
                    • 96
                      • f996d81f13b7f36828e72975e2e89823f04c44f1072720ddcadc612676c327df3 MB
                  • 78
                    • e5
                      • 78e582171bf97e9c47637b67c488aec27fdfaf0ad0b872e3c8d9baeb45f3de764 MB
                  • 15
                    • b2
                      • 15b25c0cf6e27ed4076ea1a8ee770202f9f11ca9d29ba526b2d7f37f3fa4a6a52 kB
                  • 4e
                    • c0
                      • 4ec0b8af331a0d49f7dd423f605115d01ac9f8c2b322cb2898dd35e79dec9d674 MB
                  • ff
                    • d9
                      • ffd9a44d2522fcdac23de7306013bf4467d0c46122257e80facaa338342e21e3450 MB
                  • 7d
                    • 9a
                      • 7d9af5556e74d9ef48f74e40a51e706cede112bd9fbe419cd902c58e203bc9c29 kB
                  • fd
                    • 2a
                      • fd2ad2ad019d6cf6334e0f2ea12417ac2276a2231a601047cda9d20f6244131345 kB
                  • f3
                    • 7b
                      • f37b01e0e1f9b27c4e379396443236240645b8007a727a489f5c5ac9b31efeba18 kB
                  • 73
                    • 6a
                      • 736a9e14e2492712f8edf3ed4c35df06ed8b80cc80d93c6e6d491d3c470c428029 MB
                  • ca
                    • 40
                      • ca40fdd3a185cd8abbe4ff4991089dd983063cd3dfd2b3ef190782b2f5bc62e818 kB
                  • d9
                    • 37
                      • d937b1ffd1ba6497e7d938387d1362ca07ede5949a244b450fcb73c56af1d44528 kB
                  • 2f
                    • 3d
                      • 2f3dac972eb6efbeddf38c1d07ef5817dc09f8f809d62c59238c0cfd304b81f1653 kB
                  • ae
                    • 9f
                      • ae9f6e08bda81270c32fdf82c662bddf604af9ea522045605e44b836220b7d9d10 MB
                  • 8f
                    • 96
                      • 8f96e4d7e7ffb9cc103a5dcc7081af9c1c0a879b0b5c453a2e6448d09f640fb5653 kB
                  • ad
                    • 5e
                      • ad5efc9a8b4ca2548a3fb76f9abaeb08e9d00668aea54e1cf45a86d4be9f574145 kB
                  • d5
                    • 38
                      • d538dcc7b3c6b35f9d16a108b39ae3d9d02e1e61a6a49db516ea516ac0abc71130 MB
                  • 52
                    • 51
                      • 52517a1dbe11ac760c6e8c43ce6e43f752da6311438b01cab74489a21685e3b810 MB
                    • e0
                      • 52e0c4e2ae4984472308d1fada464fdfaebd5fa96be37fe8d23659d5fc6dfe7f78 MB
            • train_wordpiece_tokenizer.py1005 B
            • ocr_dataset.py1 kB
            • test_data
              • corrected
                • jon_raudi-1886.txt12 kB
                • jafnadamadurinn-1926.txt46 kB
                • althydubladid_1949-2-1-bls-4.txt13 kB
                • lesbok_morgunbladsins-1925.txt30 kB
                • ulfur.txt15 kB
                • test.txt118 kB
                • short_test.txt368 B
                • 80s_90s_test.txt173 kB
              • original
                • jon_raudi-1886.txt12 kB
                • althydubladid_1949-2-1-bls-4.txt13 kB
                • lesbok_morgunbladsins-1925.txt31 kB
                • jafnadarmadurinn-1926.txt48 kB
                • ulfur.txt15 kB
                • test.txt118 kB
                • short_test.txt512 B
                • 80s_90s_test.txt172 kB
              • outputs
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_torch_only_80s_90s_test.txt172 kB
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_torch_only_lexicon_lookup_80s_90s_test.txt172 kB
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_80s_90s_test.txt172 kB
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_fairseq_only_80s_90s_test.txt172 kB
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_None_lexicon_lookup_80s_90s_test.txt172 kB
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_fairseq_only_lexicon_lookup_80s_90s_test.txt172 kB
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_test.txt114 kB
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13_short_test.txt503 B
            • .gitignore178 B
            • params.py175 B
            • dbs
              • .gitignore17 B
              • spellfix.so46 kB
            • globals.py1 kB
            • hyperparams
              • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13.py331 B
              • __pycache__
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13.cpython-39.pyc625 B
                • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13.cpython-310.pyc634 B
            • validate_sents_with_greynir.py660 B
            • .gitattributes192 B
            • ocr_tokenizers
            • evaluate.py2 kB
            • dataframes
              • .gitignore14 B
            • dehyphenate.py2 kB
            • __pycache__
              • params.cpython-310.pyc392 B
              • transformer_classes.cpython-39.pyc3 kB
              • dehyphenate.cpython-310.pyc2 kB
              • globals.cpython-310.pyc2 kB
              • transformer_classes.cpython-310.pyc3 kB
              • train.cpython-310.pyc6 kB
              • ocr_dataset.cpython-310.pyc1 kB
              • train.cpython-39.pyc6 kB
              • dehyphenate.cpython-39.pyc2 kB
              • ocr_dataset.cpython-39.pyc1 kB
              • params.cpython-39.pyc383 B
              • globals.cpython-39.pyc2 kB
            • transformer_classes.py3 kB
            • requirements.txt199 B
            • utils
              • format_long_file.py1 kB
              • lexicon_lookup.py9 kB
              • pair_lines.py6 kB
              • bin_tree.pickle71 MB
              • noise_to_corpus.py976 B
              • __pycache__
                • lexicon_lookup.cpython-39.pyc6 kB
                • tokens.cpython-310.pyc7 kB
                • error_files_setup.cpython-39.pyc3 kB
                • pair_lines.cpython-39.pyc3 kB
                • format.cpython-310.pyc2 kB
                • tokens.cpython-39.pyc7 kB
                • lexicon_lookup.cpython-310.pyc6 kB
                • format.cpython-39.pyc2 kB
                • pair_lines.cpython-310.pyc3 kB
              • tr_test.txt0 B
              • format.py1 kB
              • tokens.py8 kB
              • old_words.pickle6 MB
              • error_files_setup.py3 kB
              • sql
                • db_setup.py4 kB
                • lookup.py4 kB
                • __pycache__
                  • db_setup.cpython-39.pyc3 kB
                  • lookup.cpython-310.pyc4 kB
                  • lookup.cpython-39.pyc4 kB
              • noise_to_text.py3 kB
            • LICENSE11 kB
            • models
              • 937152_512_4_2048_24_5_5_0dot1_40_0_3e-05_3000_3_EPOCH_13.model167 MB
            • data
              • val_tgt_3000_3.vocab21 kB
              • parallel
                • 60k_gold
                  • corrected
                    • Fjallkonan_1886-04-13.txt28 kB
                    • Isafold_1875-07-26.txt37 kB
                    • Austri_1884-06-21.txt28 kB
                    • Isafold_1875-01-27.txt36 kB
                    • jon_raudi-1886.txt12 kB
                    • Nordlingur_1875-11-06.txt41 kB
                    • Bjarki_1896-10-09.txt20 kB
                    • Thjodviljinn_1900-04-30.txt66 kB
                    • Isafold_1893-04-22.txt33 kB
                    • Austri_1900-06-09.txt39 kB
                    • Fjallkonan_1890-03-25.txt26 kB
                    • Fjallkonan_1899-07-06.txt19 kB
                    • Nordlingur_1875-07-02.txt41 kB
                    • Thjodviljinn_1900-07-26.txt31 kB
                    • Austri_1900-03-01.txt44 kB
                    • Isfirdingur_1898-09-09.txt13 kB
                    • Fjallkonan_1898-06-28.txt23 kB
                    • Kvennabladid_1900-08-01.txt29 kB
                    • Framsokn_1900-06-01.txt24 kB
                    • Nordlingur_1875-08-31.txt38 kB
                    • althydubladid_1949-2-1-bls-4.txt13 kB
                    • Nordlingur_1875-12-29.txt38 kB
                    • Althydubladid_1906-01-21.txt20 kB
                    • Austurland_1907-08-01.txt34 kB
                    • Islendingur_1875-04-19.txt30 kB
                    • Fjallkonan_1889-02-18.txt25 kB
                    • Isafold_1875-05-25.txt35 kB
                    • Isafold_1882-05-25.txt34 kB
                    • ulfur.txt15 kB
                    • Fjallkonan_1884-02-29.txt25 kB
                    • Thjodviljinn_1900-01-26.txt65 kB
                    • Bjarki_1900-05-12.txt32 kB
                    • Fjallkonan_1887-11-08.txt27 kB
                    • Isafold_1895-06-08.txt35 kB
                    • Akureyrarposturinn_1886-08-19.txt15 kB
                    • Fjallkonan_1896-10-21.txt25 kB
                    • Akureyrarposturinn_1886-02-04.txt14 kB
                    • Isafold_1875-10-20.txt36 kB
                    • Fjallkonan_1891-04-28.txt26 kB
                    • Kvennabladid_1900-02-01.txt24 kB
                    • Bjarki_1900-02-09.txt32 kB
                    • Nordri_1913-10-08.txt16 kB
                    • Fjallkonan_1895-09-17.txt24 kB
                    • Kvennabladid_1900-11-01.txt29 kB
                    • Islendingur_1875-07-26.txt28 kB
                    • Sudri_1883-02-03.txt31 kB
                    • Fjallkonan_1891-10-20.txt25 kB
                    • Austurland_1907-10-15.txt39 kB
                    • Austri_1900-12-10.txt39 kB
                    • Bjarki_1900-11-14.txt33 kB
                    • Isafold_1879-02-19.txt33 kB
                    • Fjallkonan_1888-06-30.txt28 kB
                    • Fjallkonan_1894-07-03.txt21 kB
                    • Fjallkonan_1893-12-13.txt25 kB
                    • Bjarki_1896-11-14.txt22 kB
                    • Fjallkonan_1892-11-08.txt27 kB
                    • Framsokn_1900-09-01.txt23 kB
                    • Fjallkonan_1884-08-26.txt20 kB
                    • Thjodviljinn_1900-10-30.txt29 kB
                    • Isafold_1889-12-28.txt16 kB
                    • Akureyrarposturinn_1885-12-30.txt4 kB
                    • lesbok_morgunbladsins-1925.txt30 kB
                    • Dagskra_1896-07-13.txt22 kB
                    • Fjallkonan_1900-01-13.txt37 kB
                    • Isafold_1887-10-13.txt27 kB
                    • Dagskra_1896-12-19.txt22 kB
                    • Fjallkonan_1887-05-08.txt26 kB
                    • Frettir_fra_Islandi_1875-01-01.txt180 kB
                    • jafnadarmadurinn-1926.txt46 kB
                    • Isafold_1884-07-16.txt30 kB
                    • Framsokn_1900-12-01.txt9 kB
                    • Islendingur_1875-10-14.txt29 kB
                    • Dagskra_1896-12-24.txt25 kB
                    • Isafold_1877-12-05.txt33 kB
                    • Kvennabladid_1900-05-01.txt26 kB
                    • Islendingur_1875-01-25.txt32 kB
                    • Sudri_1883-02-17.txt33 kB
                    • Isafold_1874-09-19.txt30 kB
                    • Framsokn_1900-03-01.txt24 kB
                    • Nordlingur_1881-05-07.txt32 kB
                    • Bjarki_1896-11-07.txt22 kB
                    • Austurland_1907-10-02.txt40 kB
                    • Isafold_1876-11-27.txt34 kB
                    • Austri_1900-09-01.txt42 kB
                    • Fjallkonan_1897-11-17.txt25 kB
                    • Bjarki_1900-08-11.txt31 kB
                  • original
                    • Fjallkonan_1886-04-13.txt28 kB
                    • Isafold_1875-07-26.txt38 kB
                    • Austri_1884-06-21.txt29 kB
                    • Isafold_1875-01-27.txt37 kB
                    • jon_raudi-1886.txt12 kB
                    • Nordlingur_1875-11-06.txt42 kB
                    • Bjarki_1896-10-09.txt20 kB
                    • Thjodviljinn_1900-04-30.txt66 kB
                    • Isafold_1893-04-22.txt36 kB
                    • Austri_1900-06-09.txt40 kB
                    • Fjallkonan_1890-03-25.txt26 kB
                    • Fjallkonan_1899-07-06.txt20 kB
                    • Nordlingur_1875-07-02.txt40 kB
                    • Thjodviljinn_1900-07-26.txt32 kB
                    • Austri_1900-03-01.txt44 kB
                    • Isfirdingur_1898-09-09.txt13 kB
                    • Fjallkonan_1898-06-28.txt23 kB
                    • Kvennabladid_1900-08-01.txt29 kB
                    • Framsokn_1900-06-01.txt24 kB
                    • Nordlingur_1875-08-31.txt39 kB
                    • althydubladid_1949-2-1-bls-4.txt13 kB
                    • Nordlingur_1875-12-29.txt39 kB
                    • Althydubladid_1906-01-21.txt20 kB
                    • Austurland_1907-08-01.txt35 kB
                    • Islendingur_1875-04-19.txt30 kB
                    • Fjallkonan_1889-02-18.txt25 kB
                    • Isafold_1875-05-25.txt36 kB
                    • Isafold_1882-05-25.txt34 kB
                    • ulfur.txt15 kB
                    • Fjallkonan_1884-02-29.txt25 kB
                    • Thjodviljinn_1900-01-26.txt67 kB
                    • Bjarki_1900-05-12.txt33 kB
                    • Fjallkonan_1887-11-08.txt28 kB
                    • Isafold_1895-06-08.txt36 kB
                    • Akureyrarposturinn_1886-08-19.txt15 kB
                    • Fjallkonan_1896-10-21.txt25 kB
                    • Akureyrarposturinn_1886-02-04.txt14 kB
                    • Isafold_1875-10-20.txt36 kB
                    • Fjallkonan_1891-04-28.txt26 kB
                    • Kvennabladid_1900-02-01.txt25 kB
                    • Bjarki_1900-02-09.txt33 kB
                    • Nordri_1913-10-08.txt16 kB
                    • Fjallkonan_1895-09-17.txt25 kB
                    • Kvennabladid_1900-11-01.txt29 kB
                    • Islendingur_1875-07-26.txt29 kB
                    • Sudri_1883-02-03.txt32 kB
                    • Fjallkonan_1891-10-20.txt25 kB
                    • Austurland_1907-10-15.txt40 kB
                    • Austri_1900-12-10.txt39 kB
                    • Bjarki_1900-11-14.txt35 kB
                    • Isafold_1879-02-19.txt33 kB
                    • Fjallkonan_1888-06-30.txt29 kB
                    • Fjallkonan_1894-07-03.txt22 kB
                    • Fjallkonan_1893-12-13.txt25 kB
                    • Bjarki_1896-11-14.txt22 kB
                    • Fjallkonan_1892-11-08.txt27 kB
                    • Framsokn_1900-09-01.txt23 kB
                    • Fjallkonan_1884-08-26.txt20 kB
                    • Thjodviljinn_1900-10-30.txt30 kB
                    • Isafold_1889-12-28.txt17 kB
                    • Akureyrarposturinn_1885-12-30.txt4 kB
                    • lesbok_morgunbladsins-1925.txt31 kB
                    • Dagskra_1896-07-13.txt22 kB
                    • Fjallkonan_1900-01-13.txt38 kB
                    • Isafold_1887-10-13.txt28 kB
                    • Dagskra_1896-12-19.txt22 kB
                    • Fjallkonan_1887-05-08.txt26 kB
                    • Frettir_fra_Islandi_1875-01-01.txt185 kB
                    • jafnadarmadurinn-1926.txt48 kB
                    • Isafold_1884-07-16.txt30 kB
                    • Framsokn_1900-12-01.txt9 kB
                    • Islendingur_1875-10-14.txt29 kB
                    • Dagskra_1896-12-24.txt25 kB
                    • Isafold_1877-12-05.txt34 kB
                    • Kvennabladid_1900-05-01.txt26 kB
                    • Islendingur_1875-01-25.txt32 kB
                    • Sudri_1883-02-17.txt33 kB
                    • Isafold_1874-09-19.txt30 kB
                    • Framsokn_1900-03-01.txt24 kB
                    • Nordlingur_1881-05-07.txt32 kB
                    • Bjarki_1896-11-07.txt23 kB
                    • Austurland_1907-10-02.txt40 kB
                    • Isafold_1876-11-27.txt35 kB
                    • Austri_1900-09-01.txt43 kB
                    • Fjallkonan_1897-11-17.txt25 kB
                    • Bjarki_1900-08-11.txt32 kB
                • 50k_gold
                  • corrected
                    • Fjallkonan_1886-04-13.txt28 kB
                    • Isafold_1875-07-26.txt38 kB
                    • Austri_1884-06-21.txt28 kB
                    • Isafold_1875-01-27.txt36 kB
                    • Nordlingur_1875-11-06.txt41 kB
                    • Bjarki_1896-10-09.txt20 kB
                    • Thjodviljinn_1900-04-30.txt66 kB
                    • Isafold_1893-04-22.txt33 kB
                    • Austri_1900-06-09.txt39 kB
                    • Fjallkonan_1890-03-25.txt26 kB
                    • Fjallkonan_1899-07-06.txt19 kB
                    • Nordlingur_1875-07-02.txt41 kB
                    • Thjodviljinn_1900-07-26.txt31 kB
                    • Austri_1900-03-01.txt44 kB
                    • Isfirdingur_1898-09-09.txt13 kB
                    • Fjallkonan_1898-06-28.txt23 kB
                    • Kvennabladid_1900-08-01.txt29 kB
                    • Framsokn_1900-06-01.txt24 kB
                    • Nordlingur_1875-08-31.txt38 kB
                    • Nordlingur_1875-12-29.txt38 kB
                    • Althydubladid_1906-01-21.txt20 kB
                    • Austurland_1907-08-01.txt34 kB
                    • Islendingur_1875-04-19.txt30 kB
                    • Fjallkonan_1889-02-18.txt25 kB
                    • Isafold_1875-05-25.txt35 kB
                    • Isafold_1882-05-25.txt34 kB
                    • Fjallkonan_1884-02-29.txt25 kB
                    • Thjodviljinn_1900-01-26.txt65 kB
                    • Bjarki_1900-05-12.txt32 kB
                    • Fjallkonan_1887-11-08.txt27 kB
                    • Isafold_1895-06-08.txt35 kB
                    • Akureyrarposturinn_1886-08-19.txt15 kB
                    • Fjallkonan_1896-10-21.txt25 kB
                    • Akureyrarposturinn_1886-02-04.txt14 kB
                    • Isafold_1875-10-20.txt36 kB
                    • Fjallkonan_1891-04-28.txt26 kB
                    • Kvennabladid_1900-02-01.txt24 kB
                    • Bjarki_1900-02-09.txt32 kB
                    • Nordri_1913-10-08.txt16 kB
                    • Fjallkonan_1895-09-17.txt24 kB
                    • Kvennabladid_1900-11-01.txt29 kB
                    • Islendingur_1875-07-26.txt28 kB
                    • Sudri_1883-02-03.txt31 kB
                    • Fjallkonan_1891-10-20.txt25 kB
                    • Austurland_1907-10-15.txt39 kB
                    • Austri_1900-12-10.txt39 kB
                    • Bjarki_1900-11-14.txt33 kB
                    • Isafold_1879-02-19.txt33 kB
                    • Fjallkonan_1888-06-30.txt28 kB
                    • Fjallkonan_1894-07-03.txt21 kB
                    • Fjallkonan_1893-12-13.txt25 kB
                    • Bjarki_1896-11-14.txt22 kB
                    • Fjallkonan_1892-11-08.txt27 kB
                    • Framsokn_1900-09-01.txt23 kB
                    • Fjallkonan_1884-08-26.txt20 kB
                    • Thjodviljinn_1900-10-30.txt29 kB
                    • Isafold_1889-12-28.txt16 kB
                    • Akureyrarposturinn_1885-12-30.txt4 kB
                    • Dagskra_1896-07-13.txt22 kB
                    • Fjallkonan_1900-01-13.txt37 kB
                    • Isafold_1887-10-13.txt27 kB
                    • Dagskra_1896-12-19.txt22 kB
                    • Fjallkonan_1887-05-08.txt26 kB
                    • Isafold_1884-07-16.txt30 kB
                    • Framsokn_1900-12-01.txt9 kB
                    • Islendingur_1875-10-14.txt29 kB
                    • Dagskra_1896-12-24.txt25 kB
                    • Isafold_1877-12-05.txt33 kB
                    • Kvennabladid_1900-05-01.txt26 kB
                    • Islendingur_1875-01-25.txt32 kB
                    • Sudri_1883-02-17.txt33 kB
                    • Isafold_1874-09-19.txt30 kB
                    • Framsokn_1900-03-01.txt24 kB
                    • Nordlingur_1881-05-07.txt32 kB
                    • Bjarki_1896-11-07.txt22 kB
                    • Austurland_1907-10-02.txt40 kB
                    • Isafold_1876-11-27.txt34 kB
                    • Austri_1900-09-01.txt42 kB
                    • Fjallkonan_1897-11-17.txt25 kB
                    • Bjarki_1900-08-11.txt31 kB
                  • original
                    • Fjallkonan_1886-04-13.txt28 kB
                    • Isafold_1875-07-26.txt38 kB
                    • Austri_1884-06-21.txt29 kB
                    • Isafold_1875-01-27.txt37 kB
                    • Nordlingur_1875-11-06.txt42 kB
                    • Bjarki_1896-10-09.txt20 kB
                    • Thjodviljinn_1900-04-30.txt66 kB
                    • Isafold_1893-04-22.txt36 kB
                    • Austri_1900-06-09.txt40 kB
                    • Fjallkonan_1890-03-25.txt26 kB
                    • Fjallkonan_1899-07-06.txt20 kB
                    • Nordlingur_1875-07-02.txt40 kB
                    • Thjodviljinn_1900-07-26.txt32 kB
                    • Austri_1900-03-01.txt44 kB
                    • Isfirdingur_1898-09-09.txt13 kB
                    • Fjallkonan_1898-06-28.txt23 kB
                    • Kvennabladid_1900-08-01.txt29 kB
                    • Framsokn_1900-06-01.txt24 kB
                    • Nordlingur_1875-08-31.txt39 kB
                    • Nordlingur_1875-12-29.txt39 kB
                    • Althydubladid_1906-01-21.txt20 kB
                    • Austurland_1907-08-01.txt35 kB
                    • Islendingur_1875-04-19.txt30 kB
                    • Fjallkonan_1889-02-18.txt25 kB
                    • Isafold_1875-05-25.txt36 kB
                    • Isafold_1882-05-25.txt34 kB
                    • Fjallkonan_1884-02-29.txt25 kB
                    • Thjodviljinn_1900-01-26.txt67 kB
                    • Bjarki_1900-05-12.txt33 kB
                    • Fjallkonan_1887-11-08.txt28 kB
                    • Isafold_1895-06-08.txt36 kB
                    • Akureyrarposturinn_1886-08-19.txt15 kB
                    • Fjallkonan_1896-10-21.txt25 kB
                    • Akureyrarposturinn_1886-02-04.txt14 kB
                    • Isafold_1875-10-20.txt36 kB
                    • Fjallkonan_1891-04-28.txt26 kB
                    • Kvennabladid_1900-02-01.txt25 kB
                    • Bjarki_1900-02-09.txt33 kB
                    • Nordri_1913-10-08.txt16 kB
                    • Fjallkonan_1895-09-17.txt25 kB
                    • Kvennabladid_1900-11-01.txt29 kB
                    • Islendingur_1875-07-26.txt29 kB
                    • Sudri_1883-02-03.txt32 kB
                    • Fjallkonan_1891-10-20.txt25 kB
                    • Austurland_1907-10-15.txt40 kB
                    • Austri_1900-12-10.txt39 kB
                    • Bjarki_1900-11-14.txt35 kB
                    • Isafold_1879-02-19.txt33 kB
                    • Fjallkonan_1888-06-30.txt29 kB
                    • Fjallkonan_1894-07-03.txt22 kB
                    • Fjallkonan_1893-12-13.txt25 kB
                    • Bjarki_1896-11-14.txt22 kB
                    • Fjallkonan_1892-11-08.txt27 kB
                    • Framsokn_1900-09-01.txt23 kB
                    • Fjallkonan_1884-08-26.txt20 kB
                    • Thjodviljinn_1900-10-30.txt30 kB
                    • Isafold_1889-12-28.txt17 kB
                    • Akureyrarposturinn_1885-12-30.txt4 kB
                    • Dagskra_1896-07-13.txt22 kB
                    • Fjallkonan_1900-01-13.txt38 kB
                    • Isafold_1887-10-13.txt28 kB
                    • Dagskra_1896-12-19.txt22 kB
                    • Fjallkonan_1887-05-08.txt26 kB
                    • Isafold_1884-07-16.txt30 kB
                    • Framsokn_1900-12-01.txt9 kB
                    • Islendingur_1875-10-14.txt29 kB
                    • Dagskra_1896-12-24.txt25 kB
                    • Isafold_1877-12-05.txt34 kB
                    • Kvennabladid_1900-05-01.txt26 kB
                    • Islendingur_1875-01-25.txt32 kB
                    • Sudri_1883-02-17.txt33 kB
                    • Isafold_1874-09-19.txt30 kB
                    • Framsokn_1900-03-01.txt24 kB
                    • Nordlingur_1881-05-07.txt32 kB
                    • Bjarki_1896-11-07.txt23 kB
                    • Austurland_1907-10-02.txt40 kB
                    • Isafold_1876-11-27.txt35 kB
                    • Austri_1900-09-01.txt43 kB
                    • Fjallkonan_1897-11-17.txt25 kB
                    • Bjarki_1900-08-11.txt32 kB
              • 50k_list.txt1 kB
              • 50k_in_training.txt1 kB
              • 50k_in_validation.txt0 B
              • errors
                • all_replacements.tsv19 kB
              • dev_target_3000_3.vocab22 kB
              • dev_source_3000_3.vocab22 kB
              • val_src_3000_3.vocab21 kB
            • frsq
              • data
                • data-bin.3000
                  • test.original-corrected.original.bin18 kB
                  • test.original-corrected.corrected.idx9 kB
                  • valid.original-corrected.corrected.bin2 MB
                  • dict.original.txt34 kB
                  • test.original-corrected.corrected.bin18 kB
                  • train.original-corrected.corrected.idx10 MB
                  • valid.original-corrected.original.idx653 kB
                  • dict.corrected.txt34 kB
                  • train.original-corrected.original.idx10 MB
                  • train.original-corrected.corrected.bin29 MB
                  • valid.original-corrected.original.bin2 MB
                  • train.original-corrected.original.bin30 MB
                  • test.original-corrected.original.idx9 kB
                  • valid.original-corrected.corrected.idx653 kB
                  • preprocess.log2 kB
                • sentencepiece
                  • data
                    • sentencepiece_3000.bpe.model274 kB
                    • sentencepiece_3000.bpe.vocab35 kB
              • models
                • checkpoint_best.pt450 MB
            • val
              • .gitignore14 B
          Icon
          Name
          README.md
          Size
          11.34 KB
          Format
          Unknown
          Description
          README
          MD5
          6cf2097655e8ab6cb16c52bafcec4969
           Download file

          Partners, Coordination, Funding

          Repository

          More

          CLARIN-IS is fully supported by the Ministry of Culture and Business Affairs

          Copyright (c) 2023. Arni Magnusson Institute for Icelandic Studies. All rights reserved.