In linguistics, a corpus (plural corpora) or text corpus is a language resource consisting of a large and structured set of texts (nowadays usually electronically stored and processed). In corpus linguistics, they are used to do statistical analysis and hypothesis testing, checking occurrences or validating linguistic rules within a specific language territory.
In search technology, a corpus is the collection of documents which is being searched.
A corpus may contain texts in a single language (monolingual corpus) or text data in multiple languages (multilingual corpus).
In order to make the corpora more useful for doing linguistic research, they are often subjected to a process known as annotation. An example of annotating a corpus is part-of-speech tagging, or POS-tagging, in which information about each word's part of speech (verb, noun, adjective, etc.) is added to the corpus in the form of tags. Another example is indicating the lemma (base) form of each word. When the language of the corpus is not a working language of the researchers who use it, interlinear glossing is used to make the annotation bilingual.
Some corpora have further structured levels of analysis applied. In particular, smaller corpora may be fully parsed. Such corpora are usually called Treebanks or Parsed Corpora. The difficulty of ensuring that the entire corpus is completely and consistently annotated means that these corpora are usually smaller, containing around one to three million words. Other levels of linguistic structured analysis are possible, including annotations for morphology, semantics and pragmatics.
Applications
Corpora are the main knowledge base in corpus linguistics. Other notable areas of application include:
The analysis and processing of various types of corpora are also the subject of much work in computational linguistics, speech recognition and machine translation, where they are often used to create hidden Markov models for part of speech tagging and other purposes. Corpora and frequency lists derived from them are useful for language teaching. Corpora can be considered as a type of foreign language writing aid as the contextualised grammatical knowledge acquired by non-native language users through exposure to authentic texts in corpora allows learners to grasp the manner of sentence formation in the target language, enabling effective writing.[1]
Multilingual corpora that have been specially formatted for side-by-side comparison are called aligned parallel corpora. There are two main types of parallel corpora which contain texts in two languages. In a translation corpus, the texts in one language are translations of texts in the other language. In a comparable corpus, the texts are of the same kind and cover the same content, but they are not translations of each other.[2] To exploit a parallel text, some kind of text alignment identifying equivalent text segments (phrases or sentences) is a prerequisite for analysis. Machine translation algorithms for translating between two languages are often trained using parallel fragments comprising a first-language corpus and a second-language corpus, which is an element-for-element translation of the first-language corpus.[3]
Text corpora are also used in the study of historical documents, for example in attempts to decipher ancient scripts, or in Biblical scholarship. Some archaeological corpora can be of such short duration that they provide a snapshot in time. One of the shortest corpora in time may be the 15–30 year Amarna letters texts (1350 BC). The corpus of an ancient city, (for example the "Kültepe Texts" of Turkey), may go through a series of corpora, determined by their find site dates.
^Yoon, H., & Hirvela, A. (2004). ESL Student Attitudes toward Corpus Use in L2 Writing. Journal of Second Language Writing, 13(4), 257–283. Retrieved 21 March 2012.
^Wołk, K.; Marasek, K. (7 April 2014). "A Sentence Meaning Based Alignment Method for Parallel Text Corpora Preparation". Advances in Intelligent Systems and Computing. Springer. 275: 107–114. arXiv:1509.09090. doi:10.1007/978-3-319-05951-8_11. ISBN978-3-319-05950-1. ISSN 2194-5357. S2CID 15361632.
^Wolk, Krzysztof; Marasek, Krzysztof (2015). "Tuned and GPU-accelerated parallel data mining from comparable corpora". In Král, Pavel; Matousek, Václav (eds.). Text, Speech, and Dialogue – 18th International Conference, TSD 2015, Pilsen, Czech Republic, September 14–17, 2015, Proceedings. Lecture Notes in Computer Science. Vol. 9302. Springer. pp. 32–40. arXiv:1509.08639. doi:10.1007/978-3-319-24033-6_4.
External links
ACL SIGLEX Resource Links: Text Corpora 2013-08-13 at the Wayback Machine
Developing Linguistic Corpora: a Guide to Good Practice
Free samples (not free), web-based corpora (45-425 million words each): American (COCA, COHA, TIME), British (BNC), Spanish, Portuguese
Intercorp Building synchronous parallel corpora of the languages taught at the Faculty of Arts of Charles University.
Sketch Engine: Open corpora with free access
TS Corpus – A Turkish Corpus freely available for academic research.
Turkish National Corpus – A general-purpose corpus for contemporary Turkish
Corpus of Political Speeches, Free access to political speeches by American and Chinese politicians, developed by Hong Kong Baptist University Library
Russian National Corpus
January 13, 2023
text, corpus, linguistics, corpus, plural, corpora, text, corpus, language, resource, consisting, large, structured, texts, nowadays, usually, electronically, stored, processed, corpus, linguistics, they, used, statistical, analysis, hypothesis, testing, check. In linguistics a corpus plural corpora or text corpus is a language resource consisting of a large and structured set of texts nowadays usually electronically stored and processed In corpus linguistics they are used to do statistical analysis and hypothesis testing checking occurrences or validating linguistic rules within a specific language territory In search technology a corpus is the collection of documents which is being searched Contents 1 Overview 2 Applications 3 Some notable text corpora 4 See also 5 References 6 External linksOverview EditA corpus may contain texts in a single language monolingual corpus or text data in multiple languages multilingual corpus In order to make the corpora more useful for doing linguistic research they are often subjected to a process known as annotation An example of annotating a corpus is part of speech tagging or POS tagging in which information about each word s part of speech verb noun adjective etc is added to the corpus in the form of tags Another example is indicating the lemma base form of each word When the language of the corpus is not a working language of the researchers who use it interlinear glossing is used to make the annotation bilingual Some corpora have further structured levels of analysis applied In particular smaller corpora may be fully parsed Such corpora are usually called Treebanks or Parsed Corpora The difficulty of ensuring that the entire corpus is completely and consistently annotated means that these corpora are usually smaller containing around one to three million words Other levels of linguistic structured analysis are possible including annotations for morphology semantics and pragmatics Applications EditCorpora are the main knowledge base in corpus linguistics Other notable areas of application include Language technology natural language processing computational linguistics The analysis and processing of various types of corpora are also the subject of much work in computational linguistics speech recognition and machine translation where they are often used to create hidden Markov models for part of speech tagging and other purposes Corpora and frequency lists derived from them are useful for language teaching Corpora can be considered as a type of foreign language writing aid as the contextualised grammatical knowledge acquired by non native language users through exposure to authentic texts in corpora allows learners to grasp the manner of sentence formation in the target language enabling effective writing 1 Machine translation Multilingual corpora that have been specially formatted for side by side comparison are called aligned parallel corpora There are two main types of parallel corpora which contain texts in two languages In a translation corpus the texts in one language are translations of texts in the other language In a comparable corpus the texts are of the same kind and cover the same content but they are not translations of each other 2 To exploit a parallel text some kind of text alignment identifying equivalent text segments phrases or sentences is a prerequisite for analysis Machine translation algorithms for translating between two languages are often trained using parallel fragments comprising a first language corpus and a second language corpus which is an element for element translation of the first language corpus 3 Philologies Text corpora are also used in the study of historical documents for example in attempts to decipher ancient scripts or in Biblical scholarship Some archaeological corpora can be of such short duration that they provide a snapshot in time One of the shortest corpora in time may be the 15 30 year Amarna letters texts 1350 BC The corpus of an ancient city for example the Kultepe Texts of Turkey may go through a series of corpora determined by their find site dates Some notable text corpora EditMain article List of text corporaSee also EditConcordance Corpus linguistics Distributional relational database Linguistic Data Consortium Natural language processing Natural Language Toolkit Parallel text alignment Search engines they access the web corpus Speech corpus Translation memory Treebank Zipf s LawReferences Edit Yoon H amp Hirvela A 2004 ESL Student Attitudes toward Corpus Use in L2 Writing Journal of Second Language Writing 13 4 257 283 Retrieved 21 March 2012 Wolk K Marasek K 7 April 2014 A Sentence Meaning Based Alignment Method for Parallel Text Corpora Preparation Advances in Intelligent Systems and Computing Springer 275 107 114 arXiv 1509 09090 doi 10 1007 978 3 319 05951 8 11 ISBN 978 3 319 05950 1 ISSN 2194 5357 S2CID 15361632 Wolk Krzysztof Marasek Krzysztof 2015 Tuned and GPU accelerated parallel data mining from comparable corpora In Kral Pavel Matousek Vaclav eds Text Speech and Dialogue 18th International Conference TSD 2015 Pilsen Czech Republic September 14 17 2015 Proceedings Lecture Notes in Computer Science Vol 9302 Springer pp 32 40 arXiv 1509 08639 doi 10 1007 978 3 319 24033 6 4 External links EditACL SIGLEX Resource Links Text Corpora Archived 2013 08 13 at the Wayback Machine Developing Linguistic Corpora a Guide to Good Practice Free samples not free web based corpora 45 425 million words each American COCA COHA TIME British BNC Spanish Portuguese Intercorp Building synchronous parallel corpora of the languages taught at the Faculty of Arts of Charles University Sketch Engine Open corpora with free access TS Corpus A Turkish Corpus freely available for academic research Turkish National Corpus A general purpose corpus for contemporary Turkish Corpus of Political Speeches Free access to political speeches by American and Chinese politicians developed by Hong Kong Baptist University Library Russian National Corpus Retrieved from https en wikipedia org w index php title Text corpus amp oldid 1110037402, wikipedia, wiki, book, books, library,
