The Icelandic Confusion Set Corpus (ICoSC) 2.0 (2020-05-06)

 
Clarin IS Repository
  Authors
Friðriksdóttir, Steinunn Rut and Ingason, Anton Karl
  Item identifier
http://hdl.handle.net/20.500.12537/19
 Project URL
https://github.com/steinunnfridriks/ICoSC
 Referenced by
https://www.insticc.org/Primoris/Resources/PaperPdf.ashx?idPaper=93715
https://26679e8b-5ba3-4907-88f4-76d04d4fe176.filesusr.com/ugd/c75ba8_db0942eec06f4c09a71654b1be7e4efe.pdf
 Date issued
2020-05-06
 Type
corpus, text
 Language(s)
Icelandic
 Description
The Icelandic Confusion Set Corpus (ICoSC) is available under a CC-BY licence. It was compiled during the course of 8 months by Steinunn Rut Friðriksdóttir and Anton Karl Ingason of the language technology department in the University of Iceland. Included in the ICoSC are CSV spreadsheets containing all collected confusion sets of each phonetic category and their frequencies. The spreadsheets are organized so that for each set, the total frequency of each candidate is calculated along with the frequency of each possible PoS tag for that candidate. The seventh and eight column of the tables contain binary values referring to whether the confusion set is grammatically disjoint (all PoS tags differ for the two candidates) or grammatically identical (all PoS tags are identical for the two candidates). The final column shows the frequency of the less frequent candidate of the set which can be used to determine which sets are viable in an experiment. Also included are text files containing the list of words from each category and text files containing sentence examples from the Icelandic Gigaword Corpus which contain the words for each category. As the n/nn examples are by far the most frequent confusion sets, the corpus also includes a word list and sentence examples for the 55 most frequent sets. There are also spreadsheets containing all of the collected word pairs which are grammatically identical, grammatically disjoint or neither of the aforementioned categories. All files have UTF-8 encoding. The ICoSC consists of the following categories of confusion sets, selected for their linguistic properties as homophones, separated orthographically by a single letter. The categories are: •197 pairs containing y/i (leyti ’extent’ / leiti ’search’) •150 pairs containing ý/í (sýn ’vision’ / sín ’theirs (possessive reflexive)’) •1203 pairs containing nn/n (forvitinn ’curious(masc.)’ / forvitin ’curious (fem.)’) •8 pairs commonly confused by Icelandic speakers (mig 'me' (accusative) / mér 'me' (dative)) •24 pairs containing hv/kv (hvað ’what’ / kvað ’chanted’) •42 pairs containing rð/ðr (veðri ’weather’ / verði ’will become’) •110 pairs containing rr/r (klárri ’smart’ / klári ’smart’) •25 pairs containing f/ff (kafi 'underwater' / kaffi 'coffee') •87 pairs containing g/gg (vegi 'roads' / veggi 'walls') •113 pairs containing s/ss (kysi 'would choose' / kyssi 'kiss') •161 pairs containing k/kk (taka 'to take' / takka 'button') •34 pairs containing p/pp (tapa 'loose' / tappa 'cap/plug') •456 pairs containing l/ll (vel 'well' / vell 'gush') •38 pairs containing m/mm (ama 'bother' / amma 'grandmother') •401 pairs containing t/tt (ættu 'should' / ætu 'would eat') •261 pairs containing a/á (láta 'put' / lata 'lazy') •92 pairs containing e/é (lék 'played' / lek 'leaks') •290 pairs containing i/í (sín 'theirs (possesive)' / sin 'tendon') •189 pairs containing o/ó (góðum 'good' / goðum '(heathen) gods') •100 pairs containing u/ú (skútum 'sailboats' / skutum 'shot') •582 pairs containing g/k (leika 'to play' / leiga 'rent') •62 pairs containing gg/kk (bakka 'go backwards' / bagga 'bale') •162 pairs containing p/b (best 'best' / pest 'illness') •19 pairs containing pp/bb (pabbi 'dad' / pappi 'cardboard') •432 pairs containing t/d (myndir 'pictures' / myntir 'coins/currencies') •115 pairs containing tt/dd (fædd 'born (feminine)' / fætt 'born (neutral)') •314 pairs containing v/f (fara 'leave' / vara 'lips') __ Íslenska ruglingsmengjamálheildin er birt með CC-BY leyfi. Hún var smíðuð á 8 mánaða tímabili af Steinunni Rut Friðriksdóttur og Antoni Karli Ingasyni í máltæknideild Háskóla Íslands. Málheildin inniheldur CVS skjöl sem innihalda öll ruglingsmengi viðkomandi flokks og tíðni þeirra. Skjölin eru þannig skipulögð að fyrir hvert mengi birtist tíðni viðkomandi orða ásamt tíðni allra mögulegra málfræðilegra marka þeirra. Sjöundi og áttundi dálkurinn innihalda sanngildi sem vísa til þess hvort ruglingsmengið er málfræðilega sundurlægt (orðin í parinu eiga ekkert mark sameiginlegt) eða málfræðilega eins (öll mörk eru eins). Síðasti dálkurinn sýnir tíðni orðsins sem er sjaldgæfara innan parsins sem má nýta til þess að ákveða hvaða mengi eru nýtileg við prófanir. Þá inniheldur pakkinn jafnframt textaskjöl sem innihalda orðalista úr hverjum flokki og textaskjöl sem innihalda setningadæmi fengin úr Risamálheildinni sem innihalda orðin sem um ræðir. Þar sem n/nn dæmin eru langalgengust inniheldur pakkinn jafnframt setningadæmi fyrir 55 algengustu n/nn mengin. Einnig má finna skjöl sem innihalda öll þau mengi sem safnað var sem eru málfræðilega eins, málfræðilega sundurlæg og þau sem falla undir hvorugan fyrrnefndan flokk. Skjölin eru í UTF-8 sniðmáti. Málheildin inniheldur eftirfarandi flokka en aðeins einn stafur aðskilur orðin innan paranna í flokkunum: •197 pör með y/i (leyti / leiti) •150 pör með ý/í (sýn / sín) •1203 pör með nn/n (forvitinn / forvitin) •8 pör annars eðlis (mig / mér) •24 pör með hv/kv (hvað / kvað) •42 pör með rð/ðr (veðri / verði) •110 pör með rr/r (klárri / klári) •25 pör með f/ff (kafi / kaffi) •87 pör með g/gg (vegi / veggi) •113 pör með s/ss (kysi / kyssi) •161 pör með k/kk (taka / takka) •34 pör með p/pp (tapa / tappa) •456 pör með l/ll (vel / vell) •38 pör með m/mm (ama / amma) •401 pör með t/tt (ætu / ættu) •261 pör með a/á (láta / lata) •92 pör með e/é (lék / lek) •290 pör með i/í (sín / sin) •189 pör með o/ó (góðum / goðum) •100 pör með u/ú (skútum / skutum) •582 pör með g/k (leika / leiga) •62 pör með gg/kk (bakka / bagga) •162 pör með p/b (best / pest) •19 pör með pp/bb (pabbi / pappi) •432 pör með t/d (myndir / myntir) •115 pör með tt/dd (fædd / fætt) •314 pör með v/f (fara / vara)
 Publisher
Háskóli Íslands
 Subject(s)
homophones confusion sets context dependency rich morphology disambiguation
 Collection(s)
Clarin IS
 Other versions
Show full item record
 

 Files in this item

This item is
Publicly Available
and licensed under:
Creative Commons - Attribution 4.0 International (CC BY 4.0)
Icon
Name
ICoSC_2.0.zip
Size
951.8 MB
Format
application/zip
Description
zip file containing the ICoSC
MD5
a2c61e3e88eb7cbae06dd5d31e2f7832
 Download file  Preview
 File Preview  
  • ICoSC
    • Frequency tables
      • e_spread.csv-1 B
      • rr_freq.csv-1 B
      • ff_spread.csv-1 B
      • ttdd_spread.csv-1 B
      • i_spread.csv-1 B
      • ss_spread.csv-1 B
      • kk_spread.csv-1 B
      • vf_spread.csv-1 B
      • hvkv_freq.csv-1 B
      • gg_spread.csv-1 B
      • ppbb_spread.csv-1 B
      • o_spread.csv-1 B
      • ggkk_spread.csv-1 B
      • tt_spread.csv-1 B
      • gk_spread.csv-1 B
      • pp_spread.csv-1 B
      • nn_mostfreq_freq.csv-1 B
      • various_spread.csv-1 B
      • nn_freq.csv-1 B
      • ll_spread.csv-1 B
      • nn_mostfreq_spread.csv-1 B
      • u_spread.csv-1 B
      • rddr_freq.csv-1 B
      • mm_spread.csv-1 B
      • y_freq.csv-1 B
      • y_elongated_spread.csv-1 B
      • y_elongated_freq.csv-1 B
      • rddr_spread.csv-1 B
      • y_spread.csv-1 B
      • td_spread.csv-1 B
      • various_freq.csv-1 B
      • hvkv_spread.csv-1 B
      • a_spread.csv-1 B
      • pb_spread.csv-1 B
      • rr_spread.csv-1 B
      • nn_spread.csv-1 B
    • Wordlists
      • ll_wordlist.txt-1 B
      • vf_wordlist.txt-1 B
      • y_elongated_wordlist.txt-1 B
      • ff_wordlist.txt-1 B
      • ttdd_wordlist.txt-1 B
      • o_wordlist.txt-1 B
      • various_wordlist.txt-1 B
      • e_wordlist.txt-1 B
      • gk_wordlist.txt-1 B
      • u_wordlist.txt-1 B
      • pp_wordlist.txt-1 B
      • nn_mostfreq_wordlist.txt-1 B
      • ggkk_wordlist.txt-1 B
      • td_wordlist.txt-1 B
      • tt_wordlist.txt-1 B
      • rddr_wordlist.txt-1 B
      • mm_wordlist.txt-1 B
      • nn_wordlist.txt-1 B
      • a_wordlist.txt-1 B
      • gg_wordlist.txt-1 B
      • hvkv_wordlist.txt-1 B
      • i_wordlist.txt-1 B
      • pb_wordlist.txt-1 B
      • y_wordlist.txt-1 B
      • ppbb_wordlist.txt-1 B
      • rr_wordlist.txt-1 B
      • kk_wordlist.txt-1 B
      • ss_wordlist.txt-1 B
    • .git
      • logs
      • config-1 B
      • info
        • exclude-1 B
      • COMMIT_EDITMSG-1 B
      • ORIG_HEAD-1 B
      • FETCH_HEAD-1 B
      • index-1 B
      • packed-refs-1 B
      • HEAD-1 B
      • refs
      • description-1 B
      • hooks
        • post-commit-1 B
        • applypatch-msg.sample-1 B
        • pre-push.sample-1 B
        • commit-msg.sample-1 B
        • post-checkout-1 B
        • post-update.sample-1 B
        • pre-rebase.sample-1 B
        • pre-receive.sample-1 B
        • pre-push-1 B
        • update.sample-1 B
        • pre-applypatch.sample-1 B
        • pre-commit.sample-1 B
        • fsmonitor-watchman.sample-1 B
        • post-merge-1 B
        • prepare-commit-msg.sample-1 B
      • objects
        • ee
          • 3d623cce412288d6c26f3beb534f9549db0b86-1 B
        • ed
          • 125c0db350888935acbded1d371d99050a7218-1 B
          • cb42ab378cbe9c5a9a9238f76c0285a88dd619-1 B
        • 57
          • f0153e7419f38c7164aa968bf8882bf1d39c72-1 B
          • 56a09c44bed42bd5aaa31239ba5e4edd4507c0-1 B
        • 51
          • 6e9d5e9b9c147a7cd3abbcde1c6b5503cba91e-1 B
        • 50
          • ff126d05957b0704b5d6b0437ac6cf75af1dba-1 B
          • 33a5b0c21f57b4f6d08f85cadd1513c3420554-1 B
        • f8
          • ced83fa67de314ea58aa59cf50875adec892b8-1 B
        • info
          • f5
            • 75f76c4f1f3d1bd0ff6a658026552f7988bb4c-1 B
          • f4
            • ab2087a83b8f9f207fbd9b4e19a3428c930272-1 B
            • f147e88550e65b94b608ee599d0aea33d64b1e-1 B
          • f0
            • 36ae1a4f08316427d86df1ce9324acb8783f87-1 B
            • d7c7499ec3fb3a56bee1741fcde2106b7c3d57-1 B
          • 3d
            • 6c7ccb07f9701071ae729a5b2897a00e800b41-1 B
          • de
            • b8a4c734a6299a23d90eb2f6f86c3194d4c5f6-1 B
          • pack
            • pack-9fe52c3f962d07cb7067631d093520769e78a44a.pack-1 B
            • pack-9fe52c3f962d07cb7067631d093520769e78a44a.idx-1 B
          • 49
            • d84bc56e50daa08df8a32a4379789335269fd7-1 B
            • 69cd82532fc9597d405cec1b3df312617be3e3-1 B
          • 46
            • 2772cb6c65f5068cfdbf634d1d67c09de7b511-1 B
          • 44
            • 67b03a5f6f8e3c55e94ac82912016781a34763-1 B
          • e3
            • 03b9e98a5d017338b1bca8de557d2d6c0045e0-1 B
            • d7574dea3f13515d06403788589919559f5abe-1 B
          • 2e
            • 3f7408a8e70bfc5d0bbce111a6f1a590aa2990-1 B
          • 2d
            • f24bc0f9748cd5794a553ea4be7c0624064c12-1 B
          • 2b
            • d2131dc1a3503fc4e96051fc09171742dc66f4-1 B
          • 8c
            • 1db4f9f18f42a8c67e73c8504825b0dddba4b8-1 B
          • 8b
            • 0be6de7085958af069a98d7898830628834d13-1 B
            • ddf2c10d33e92f7d48f4467f827c46bfd54411-1 B
          • cf
            • d5fd297a2e4aed8d835a2f96d958a8d292e44e-1 B
          • ca
            • 262c79973e659f4764f2d137340d43eb964f56-1 B
          • 39
            • ac2bf746321b0922264f5553706ac8a04cd0fa-1 B
          • 37
            • 9dd50321fd794cd746a3f0022ffbdbda51dee7-1 B
          • 99
            • b6215fe81dedaf22eb4dfa69920aa5e94310f9-1 B
          • 31
            • c8dba7756a5a6a616583365a9f0af74806e592-1 B
          • 96
            • be6133c106e395e525d1e0000b547ec14e2ddc-1 B
          • d4
            • b20a0f7b45116e00b626c4c2f5edcac16df1e8-1 B
            • a5e1b78267ae9e44be4bdad4d07a177f37f1e3-1 B
          • d2
            • 6978f9819d8eabb6beafd6807876d9cbf6c088-1 B
          • 1f
            • 4a48a544c0404d5a9bee3b0aa50c60cb726703-1 B
          • d0
            • cdc8b929ada684c1abe68877c281cd08e01f84-1 B
          • 1d
            • afabbf2bf467360d4cf6d26ac2e396c58b1875-1 B
          • 1c
            • 814a01aae0d97e231fa98cc1018e7bbf446445-1 B
          • 7c
            • 16cf79bcfb26a2e934e9a931a6eb9921df10c0-1 B
          • 7b
            • 90b006fa3ccea3652c5f4e52049fad1f56372c-1 B
          • be
            • a5699e3c6f85c9fbba9aaaa9b0ff5948e91017-1 B
          • ba
            • ab43a3042defc138aca19efeb8e620dddd38b3-1 B
            • feaf296ae774f2d4c4c74e054b263073f6a3fa-1 B
          • 25
            • a8ebf59d67681eb7477b3329dbc05a87886b59-1 B
            • c4d2944f521777eb32855f5a9ca8c4292e676e-1 B
          • 24
            • cea6f4cc1c238c69e9746d688bc9b372e40bda-1 B
          • 89
            • 6266c24e5b525e3c532018f8dd48605daf8bff-1 B
          • 20
            • 9a4cc2b6dbe21b2f0bed553589157c32641007-1 B
          • c9
            • 0b6ee45e4761c7bba594b3d74413bf73618cf1-1 B
          • 84
            • 80a6d34981bf398985df933060e4d15c295e55-1 B
          • c6
            • e9bce38bf66a33b35ebedb5f2801dccc7d64f8-1 B
            • 390349c51804a2f408045ec7d9fa5320a3fb1b-1 B
          • c5
            • 2998daab3142cb558e36cae9840f1b403645c6-1 B
          • c4
            • 86993a3b25be64e1b164fbf75f1252b87c1a64-1 B
          • 0f
            • 4a34b5a5d220d044ee5dc5405a192c74ac8b93-1 B
          • 0a
            • a982f56f16a9e62f6016f54bbc1db886aa4357-1 B
            • 57e4823cb8aaa8426a539b183f14d58d1baff8-1 B
            • 204f95ca6458aa6488f08a573539f2f36ad3d8-1 B
          • 6c
            • 9df7908946f4a250476b3aa6842c1b2dc44f6f-1 B
          • ad
            • 2e0d509d9cbc67484a145e98f917fc9e5ddd02-1 B
            • bd1b6d73fd3dcd121567093a9516550d93d2ab-1 B
          • 19
            • 3266f5570b35b42811e130c1ad55e79a46ea78-1 B
          • aa
            • 8d2125541deeebac384fc27899da6eee90bfe3-1 B
          • 15
            • 4c2d80c3ec57f37e31ff43ad5bb2b826b79390-1 B
          • 14
            • e8161f118dba22ab740042f219c5f6b71d5811-1 B
          • 12
            • df2190a015bed339397eeb65fd9994a153d883-1 B
          • 76
            • 4266688dd5aec44b49cbfeff16c7dabd93f76c-1 B
          • 10
            • d161f58fe91a42956364457c0f3b9f4bce4c93-1 B
          • 73
            • 8880dbb2e9625c3debe2b0f7c74d77ba9c30e5-1 B
          • b7
            • 8d7ce341a97dfe4116cfd849d0a08512d4fc4b-1 B
          • 71
            • 6f3abe744c704b182370dd459b0dcd2be8dc48-1 B
          • 70
            • 4d110735c70ebb4c5532d5511a22e88e2dd15f-1 B
          • b4
            • 7d99caf784219d25a6d0e80c4866de8cc89fad-1 B
          • b2
            • 24645da0b7c34c7ac761055ea8ea521f992848-1 B
          • b0
            • 356677cfa72419609f48e8c5f5275ed30cc1fa-1 B
          • 5c
            • da04392cd72baba7e02f8cee2bc783c8f2806c-1 B
            • 2146e0ad90f0881792f089127ea915bf416a5d-1 B
          • 5b
            • aa9cf16e8192fc9851a8719a2bda2a61f6fa1f-1 B
          • 09
            • 8aa5e9c669840b2edb85e99dc9863a7fb14167-1 B
          • 08
            • 8e15407633e0d9a5d6315b6c6a81acf4ce0ee9-1 B
          • 07
            • ebd4b11e9f5ab708efa617567b6c5488e59df4-1 B
          • fc
            • 32edfe45d32851b128ec988506442930dfee3d-1 B
          • 05
            • a2d49c442aa004b2dba3dae49ad7c276a1d84c-1 B
          • fb
            • 4e073f31a091ecb72ac06f223f8429a4a74115-1 B
          • 62
            • ee0f9f1359116e8ff806d1e30dbf0199637aae-1 B
          • a5
            • fbcec2b275c948ac0a3ee3b4a30c8ed997d5a7-1 B
          • 60
            • 82889bfeb2ec9aee7d4d88b93897b0e2011088-1 B
          • a4
            • 7dc424bf5fbf7ff893c73bc1dee145002ed758-1 B
            • 1d8a513305e175bdc2e8128e79ecf3bb01400c-1 B
          • a3
            • 23fa891a6d62a63789182c782eb91fa519c59f-1 B
        • lfs
          • cache
          • tmp
            • objects
              • 4f
                • 08
                  • 4f0858450f974cb8d1e2959a476ae2fceeae6d5e8f67d1abcad3fbbb6d659756-1 B
              • 8c
                • 01
                  • 8c0134a37c2c88f4178f60bd2602beb15f23667241158c5ac33e06bde83a77f8-1 B
              • 62
                • 34
                  • 623494a07a117814d4c0a52d0f4ead9dd13651f2977f0fc8a05be0b3c26fc590-1 B
              • 4d
                • a5
                  • 4da5edaa2aef3602cc947256061044c858e6ea9797d3f7755e2ae0da462ea151-1 B
              • 0b
                • 27
                  • 0b270afd5f297314873b7570ab9e509f59dabc43091da853d376950415956260-1 B
              • 6b
                • 72
                  • 6b7205d458fe4a7a8749d8ed3b0bc7779f6cc554e1220f9251d7db85ac41a847-1 B
              • 41
                • 03
                  • 41030f9cdbc89b6f213009ce1ee224f140175bed4d07ce93acf4bd029db5f12b-1 B
              • 2d
                • e8
                  • 2de8c250767b4196261f0d1137c3b570afbfbe38e2bcc1b9a3e6253f58c3c88c-1 B
        • Most common words - frequency tables
          • Grammatically identical.csv-1 B
          • NeitherGInorGD.csv-1 B
          • Grammaticaly disjoint.csv-1 B
        • Sentence examples
          • rr_sent.txt-1 B
          • mm_sent.txt-1 B
          • various_sent.txt-1 B
          • tt_sent.txt-1 B
          • ylong_sent.txt-1 B
          • nn_mostfreq_sent.txt-1 B
          • ggkk_sent.txt-1 B
          • e_sent.txt-1 B
          • rddr_sent.txt-1 B
          • ff_sent.txt-1 B
          • ll_sent.txt-1 B
          • gg_sent.txt-1 B
          • .gitattributes-1 B
          • hvkv_sent.txt-1 B
          • y_sent.txt-1 B
          • ppbb_sent.txt-1 B
          • pb_sent.txt-1 B
          • ss_sent.txt-1 B
          • td_sent.txt-1 B
          • nn_sent.txt-1 B
          • gk_sent.txt-1 B
          • ttdd_sent.txt-1 B
          • a_sent.txt-1 B
          • vf_sent.txt-1 B
          • pp_sent.txt-1 B
          • kk_sent.txt-1 B
          • u_sent.txt-1 B
          • o_sent.txt-1 B
          • i_sent.txt-1 B

      Partners, Coordination, Funding

      Repository

      More

      CLARIN-IS is fully supported by the Ministry of Culture and Business Affairs

      Copyright (c) 2023. Arni Magnusson Institute for Icelandic Studies. All rights reserved.