Similarity-Based Models of Word Cooccurrence Probabilities

by: Ido Dagan, Lillian Lee, Fernando C Pereira

Machine Learning, Vol. 34, No. 1. (1 February 1999), pp. 43-69.

View FullText article

Reviews [Write a review of this article]

There are no reviews of this article

Find related articles from these CiteULike users

briordan

Find related articles with these CiteULike tags

computational-lexical-semantics, distributional-similarity

Abstract

In many applications of natural language processing (NLP) it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations “eat a peach” and ”eat a beach” is more likely. Statistical NLP methods determine the likelihood of a word combination from its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in any given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on “most similar” words.

@article{citeulike:2713176, abstract = {In many applications of natural language processing (NLP) it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations “eat a peach” and ”eat a beach” is more likely. Statistical NLP methods determine the likelihood of a word combination from its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in any given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on “most similar” words.}, author = {Dagan, Ido and Lee, Lillian and Pereira, Fernando C. }, citeulike-article-id = {2713176}, doi = {10.1023/A:1007537716579}, journal = {Machine Learning}, keywords = {computational-lexical-semantics, distributional-similarity}, month = {February}, number = {1}, pages = {43--69}, posted-at = {2008-04-24 15:26:12}, priority = {2}, title = {Similarity-Based Models of Word Cooccurrence Probabilities}, url = {http://dx.doi.org/10.1023/A:1007537716579}, volume = {34}, year = {1999} }

RIS record

TY - JOUR ID - citeulike:2713176 L3 - citeulike-article-id:2713176 TI - Similarity-Based Models of Word Cooccurrence Probabilities JF - Machine Learning VL - 34 IS - 1 SP - 43 EP - 69 N2 - In many applications of natural language processing (NLP) it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations “eat a peach” and ”eat a beach” is more likely. Statistical NLP methods determine the likelihood of a word combination from its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in any given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on “most similar” words. KW - computational-lexical-semantics KW - distributional-similarity AU - Dagan, Ido AU - Lee, Lillian AU - Pereira, Fernando PY - 1999/02/01/ UR - http://dx.doi.org/10.1023/A:1007537716579 ER -

RIS BibTeX