weka.core.stemmers

Class KeseljSipkaStemmer

java.lang.Object

weka.core.stemmers.SCStemmer

weka.core.stemmers.SerbianStemmer

weka.core.stemmers.KeseljSipkaStemmer

All Implemented Interfaces:

java.io.Serializable, weka.core.RevisionHandler, weka.core.stemmers.Stemmer

Direct Known Subclasses:

KeseljSipkaStemmerGreedy, KeseljSipkaStemmerOptimal

public abstract class KeseljSipkaStemmer
extends SerbianStemmer

Ova apstraktna klasa implementira zajedničke funkcije za stemere za srpski opisane u radu:
Pristup izgradnji stemera i lematizora za jezike s bogatom fleksijom i oskudnim resursima zasnovan na obuhvatanju sufiksa, Vlado Kešelj, Danko Šipka, Infoteka 9(1-2), 21-31 (2008).
http://infoteka.bg.ac.rs/pdf/Srp/2008/04%20Vlado-Danko_Stemeri.pdf
(originalna implementacija u Perlu i drugi resursi su dostupni na: http://www.cs.dal.ca/~vlado/nlp/2007-sr/)

Ovi stemeri koriste tzv. dual1 kodovanje u kome se sva ćirilična slova prevode u latinična a svako latinično slovo koje sadrži dijakritičke oznake - š, đ, č, ć, ž, dž - se piše kao skup dva latinična slova bez dijakritičkih oznaka. Pored toga, slova lj/Lj i nj/Nj se prevode u oblike ly/Ly i ny/Ny.

This abstract class implements the common functions of the stemmers for Serbian described in the paper:
A Suffix Subsumption-Based Approach to Building Stemmers and Lemmatizers for Highly Inflectional Languages with Sparse Resources, Vlado Kešelj, Danko Šipka, Infotheca 9(1-2), 23a-33a (2008).
http://infoteka.bg.ac.rs/pdf/Eng/2008/INFOTHECA_IX_1-2_May2008_23a-33a.pdf
(the original implementation in Perl and other resources are available at: http://www.cs.dal.ca/~vlado/nlp/2007-sr/)

These stemmers use the so-called dual1 coding system in which all Cyrillic letters are transformed into their Latin equivalents and every Latin letter that contains diacritical marks - š, đ, č, ć, ž, dž - is written as a set of two Latin letters without the diacritical marks. Furthermore, the letters lj/Lj and nj/Nj are transformed into the ly/Ly and ny/Ny forms.

Author:

Vuk Batanović

See Also:

Reliable Baselines for Sentiment Analysis in Resource-Limited Languages: The Serbian Movie Review Dataset, Vuk Batanović, Boško Nikolić, Milan Milosavljević, in Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), pp. 2688-2696, Portorož, Slovenia (2016).
https://github.com/vukbatanovic/SCStemmers
, Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
private static long	serialVersionUID

Fields inherited from class weka.core.stemmers.SerbianStemmer

maxSuffixLen, rules

Constructor Summary

Constructors

Constructor and Description
KeseljSipkaStemmer()

Method Summary

Modifier and Type	Method and Description
protected java.lang.String	convertToDual1Character(int intCharacter, char oldChar) Konvertuje jedan karakter iz standardnog oblika (ćirilice ili latinice) u dual1 kodiranje
java.lang.String	stemDual1Line(java.lang.String line) Stemuje liniju teksta koja je napisana u dual1 kodiranju
java.lang.String	stemDual1Word(java.lang.String word) Stemuje reč koja je napisana u dual1 kodiranju

Methods inherited from class weka.core.stemmers.SerbianStemmer

convertToDual1File, convertToDual1String, convertToNormalFile, convertToNormalString, initMaxSuffixLen, initRules, stemDual1File, stemLine, stemWord

Methods inherited from class weka.core.stemmers.SCStemmer

getRevision, main, replaceSpaceWithNewLine, stem, stemFile, stemText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

serialVersionUID

private static final long serialVersionUID

See Also:

Constant Field Values

Constructor Detail

KeseljSipkaStemmer

public KeseljSipkaStemmer()

Method Detail

stemDual1Word

public java.lang.String stemDual1Word(java.lang.String word)

Description copied from class: SerbianStemmer

Stemuje reč koja je napisana u dual1 kodiranju

Stems a word written in the dual1 coding system

Specified by:

stemDual1Word in class SerbianStemmer

Parameters:

word - Reč koju treba stemovati
The word to be stemmed

Returns:

Stemovana reč
The stemmed word

stemDual1Line

public java.lang.String stemDual1Line(java.lang.String line)

Description copied from class: SerbianStemmer

Stemuje liniju teksta koja je napisana u dual1 kodiranju

Stems a line of text written in the dual1 coding system

Specified by:

stemDual1Line in class SerbianStemmer

Parameters:

line - Linija teksta koju treba obraditi
The line of text to be processed

Returns:

Linija teksta sa stemovanim rečima
The line of text with stemmed words

convertToDual1Character

protected java.lang.String convertToDual1Character(int intCharacter,
                                                   char oldChar)

Description copied from class: SerbianStemmer

Konvertuje jedan karakter iz standardnog oblika (ćirilice ili latinice) u dual1 kodiranje

Converts a given character from the standard form (in the Cyrillic or Latin script) to the dual1 coding system

Specified by:

convertToDual1Character in class SerbianStemmer

Parameters:

intCharacter - Unicode kod karaktera koji treba prevesti u dual1 sistem
Unicode code point of the character that should be translated into the dual1 system

oldChar - Karakter koji je u tekstu prethodio trenutno zadatom karakteru
The character which preceded the currently given one within the text

Returns:

String koji sadrži dual1 reprezentaciju zadatog karaktera
A string which contains the dual1 representation of the given character