ROUGE, or Recall-Oriented Understudy for Gisting Evaluation,[1] is a set of metrics and a software package used for evaluating automatic summarization and machine translation software in natural language processing. The metrics compare an automatically produced summary or translation against a reference or a set of references (human-produced) summary or translation.
The following five evaluation metrics are available.
ROUGE-N: Overlap of n-grams[2] between the system and reference summaries.
ROUGE-1 refers to the overlap of unigram(each word) between the system and reference summaries.
ROUGE-2 refers to the overlap of bigrams between the system and reference summaries.
ROUGE-L: Longest Common Subsequence (LCS)[3] based statistics. Longest common subsequence problem takes into account sentence level structure similarity naturally and identifies longest co-occurring in sequence n-grams automatically.
ROUGE-W: Weighted LCS-based statistics that favors consecutive LCSes .
ROUGE-S: Skip-bigram[3] based co-occurrence statistics. Skip-bigram is any pair of words in their sentence order.
ROUGE-SU: Skip-bigram plus unigram-based co-occurrence statistics.
^Lin, Chin-Yew. 2004. ROUGE: a Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain, July 25 - 26, 2004.
^Lin, Chin-Yew and E.H. Hovy 2003. Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In Proceedings of 2003 Language Technology Conference (HLT-NAACL 2003), Edmonton, Canada, May 27 - June 1, 2003.
^ abLin, Chin-Yew and Franz Josef Och. 2004. Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip-Bigram Statistics. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL 2004), Barcelona, Spain, July 21 - 26, 2004.
External links
ROUGE Usage Tutorial
Java Implementation of ROUGE
February 09, 2023
rouge, metric, rouge, recall, oriented, understudy, gisting, evaluation, metrics, software, package, used, evaluating, automatic, summarization, machine, translation, software, natural, language, processing, metrics, compare, automatically, produced, summary, . ROUGE or Recall Oriented Understudy for Gisting Evaluation 1 is a set of metrics and a software package used for evaluating automatic summarization and machine translation software in natural language processing The metrics compare an automatically produced summary or translation against a reference or a set of references human produced summary or translation Contents 1 Metrics 2 See also 3 References 4 External linksMetrics EditThe following five evaluation metrics are available ROUGE N Overlap of n grams 2 between the system and reference summaries ROUGE 1 refers to the overlap of unigram each word between the system and reference summaries ROUGE 2 refers to the overlap of bigrams between the system and reference summaries ROUGE L Longest Common Subsequence LCS 3 based statistics Longest common subsequence problem takes into account sentence level structure similarity naturally and identifies longest co occurring in sequence n grams automatically ROUGE W Weighted LCS based statistics that favors consecutive LCSes ROUGE S Skip bigram 3 based co occurrence statistics Skip bigram is any pair of words in their sentence order ROUGE SU Skip bigram plus unigram based co occurrence statistics See also EditBLEU F Measure METEOR NIST metric Noun phrase chunking Word error rate WER References Edit Lin Chin Yew 2004 ROUGE a Package for Automatic Evaluation of Summaries In Proceedings of the Workshop on Text Summarization Branches Out WAS 2004 Barcelona Spain July 25 26 2004 Lin Chin Yew and E H Hovy 2003 Automatic Evaluation of Summaries Using N gram Co occurrence Statistics In Proceedings of 2003 Language Technology Conference HLT NAACL 2003 Edmonton Canada May 27 June 1 2003 a b Lin Chin Yew and Franz Josef Och 2004 Automatic Evaluation of Machine Translation Quality Using Longest Common Subsequence and Skip Bigram Statistics In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics ACL 2004 Barcelona Spain July 21 26 2004 External links EditROUGE Usage Tutorial Java Implementation of ROUGE Retrieved from https en wikipedia org w index php title ROUGE metric amp oldid 1123710489, wikipedia, wiki, book, books, library,
