Options
Indexing and searching strategies for the Russian language
Auteur(s)
Dolamic, Ljiljana
Date de parution
2009
In
Journal of the American Society for Information Science and Technology, Wiley, 2009/60/12/2540-2547
Résumé
This paper describes and evaluates various stemming and indexing strategies for the Russian language. We design and evaluate two stemming approaches, a light and a more aggressive one, and compare these stemmers to the Snowball stemmer, to no stemming, and also to a language-independent approach (<i>n</i>-gram). To evaluate the suggested stemming strategies we apply various probabilistic information retrieval (IR) models, including the Okapi, the <i>Divergence from Randomness</i> (DFR), a statistical language model (LM), as well as two vector-space approaches, namely, the classical <i>tf idf</i> scheme and the <i>dtu-dtn</i> model. We find that the vector-space dtu-dtn and the DFR models tend to result in better retrieval effectiveness than the Okapi, LM, or <i>tf idf</i> models, while only the latter two IR approaches result in statistically significant performance differences. Ignoring stemming generally reduces the MAP by more than 50%, and these differences are always significant. When applying an <i>n</i>-gram approach, performance differences are usually lower than an approach involving stemming. Finally, our light stemmer tends to perform best, although performance differences between the light, aggressive, and Snowball stemmers are not statistically significant.
Identifiants
Type de publication
journal article
Dossier(s) à télécharger
