Punctuation model (2020-08-04)

 
Clarin IS Repository
  Authors
Sigurðardóttir, Helga Svala ; Helgadóttir, Inga Rún
  Item identifier
http://hdl.handle.net/20.500.12537/48
 Project URL
http://github.com/cadia-lvl/punctuation-prediction
 Date issued
2020
 Type
toolService
 Language(s)
Icelandic
 Description
A python package that punctuates Icelandic text. The input data is unpunctuated text and punctuated text is returned. The user can choose between three different punctuation models, a BERT-based Transformer, a seq2seq Transformer and a bidirectional RNN ([Punctuator 2](www.github.com/ottokart/punctuator2)) in Tensorflow 2. We have written a paper on this topic and sent to Interspeech, the paper is pending publication.
 Publisher
Reykjavík University
 Acknowledgement

Ministry of Education, Science and Culture (Mennta- og menningamálaráðuneytið)

Project code: Punctuation and sentence boundary detection (H12)

Project name: Language Technology for Icelandic 2019-2023

 Subject(s)
punctuation prediction
 Collection(s)
Clarin IS
 
This item is replaced by a newer submission:
Show full item record
 

 Files in this item

 Download all files in item (159.03 MB)
This item is
Publicly Available
and licensed under:
The MIT License (MIT)
Icon
Name
config.json
Size
658 bytes
Format
Unknown
Description
Electra config file
MD5
b4f432f1e85789e98ada42e2d3fcb421
 Download file
Icon
Name
pytorch_model.bin
Size
52.26 MB
Format
Unknown
Description
Electra model file
MD5
7c8a2a8b02689215900690856c8aaf20
 Download file
Icon
Name
special_tokens_map.json
Size
138 bytes
Format
Unknown
Description
Electra special tokens
MD5
cf25419989519ed218ed34a279a5dceb
 Download file
Icon
Name
tokenizer_config.json
Size
210 bytes
Format
Unknown
Description
Electra tokenizer config file
MD5
82a0aa0e14596b79da6dceb9c7bc3d98
 Download file
Icon
Name
vocab.txt
Size
254.19 KB
Format
Text file
Description
Electra vocab file
MD5
50d2b6ba7979f6d0f4b17bc27854af37
 Download file  Preview
 File Preview  
[SEP] [UNK] [CLS] [PAD] [MASK] [unused0] [unused1] [unused2] [unused3] [unused4] [unused5] [unused6] [unused7] [unused8] [unused9] [unused10] [unused11] [unused12] [unused13] [unused14] [unused15] [unused16] [unused17] [unused18] [unused19] [unused20] [unused21] [unused22] [unused23] [unused24] [unused25] [unused26] [unused27] [unused28] [unused29] [unused30] [unused31] [unused32] [unused33] [unused34] [unused35] [unused36] [unused37] [unused38] [unused39] [unused40] [unused41] [unused42] [unused43] [unused44] [unused45] [unused46] [unused47] [unused48] [unused49] [unused50] [unused51] [unused52] [unused53] [unused54] [unused55] [unused56] [unused57] [unused58] [unused59] [unused60] [unused61] [unused62] [unused63] [unused64] [unused65] [unused66] [unused67] [unused68] [unused69] [unused70] [unused71] [unused72] [unused73] [unused74] [unused75] [unused76] [unused77] [unused78] [unused79] [unused80] [unused81] [unused82] [unused83] [unused84] [unused85] [unused86] [unused87] [unused88] . . .
Icon
Name
isl-model.pcl
Size
105.43 MB
Format
Unknown
Description
Punctuator 2 model
MD5
336989fe5178c2575c3e0b3cca145931
 Download file
Icon
Name
punctuations
Size
35 bytes
Format
Unknown
Description
Punctuator 2 punctuations list
MD5
b060415aa079cfad9572b3ca4c6fec73
 Download file
Icon
Name
vocabulary
Size
1.09 MB
Format
Unknown
Description
Punctuator 2 vocabulary list
MD5
1e06c279d7521061bc697d5bffaafdc8
 Download file

Partners, Coordination, Funding

Repository

More

CLARIN-IS IS FULLY SUPPORTED BY THE MINISTRY OF EDUCATION, SCIENCE and CULTURE

Copyright (c) 2019 Stofnun Árna Magnússonar. All rights reserved.