weka.core.stemmers

Class LjubesicPandzicStemmer

java.lang.Object

weka.core.stemmers.SCStemmer

weka.core.stemmers.LjubesicPandzicStemmer

All Implemented Interfaces:

java.io.Serializable, weka.core.RevisionHandler, weka.core.stemmers.Stemmer

public class LjubesicPandzicStemmer
extends SCStemmer

Ova klasa implementira stemer za hrvatski "Simple stemmer for Croatian v0.1" Nikole Ljubešića i Ivana Pandžića. Originalna implementacija u Python-u je dostupna na adresi:
http://nlp.ffzg.hr/resources/tools/stemmer-for-croatian/

Stemer predstavlja poboljšanje ranijeg algoritma opisanog u radu:
Retrieving Information in Croatian: Building a Simple and Efficient Rule-Based Stemmer, Nikola Ljubešić, Damir Boras, Ozren Kubelka, Digital Information and Heritage, 313–320 (2007).

This class implements the "Simple stemmer for Croatian v0.1" by Nikola Ljubešić and Ivan Pandžić. The original implementation in Python is available at:
http://nlp.ffzg.hr/resources/tools/stemmer-for-croatian/

The stemmer represents an improvement of an earlier algorithm described in the paper:
Retrieving Information in Croatian: Building a Simple and Efficient Rule-Based Stemmer, Nikola Ljubešić, Damir Boras, Ozren Kubelka, Digital Information and Heritage, 313–320 (2007).

Author:

Vuk Batanović

See Also:

Reliable Baselines for Sentiment Analysis in Resource-Limited Languages: The Serbian Movie Review Dataset, Vuk Batanović, Boško Nikolić, Milan Milosavljević, in Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), pp. 2688-2696, Portorož, Slovenia (2016).
https://github.com/vukbatanovic/SCStemmers
, Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
private static long	serialVersionUID
private java.util.HashSet<java.lang.String>	stopset Lista stop-reči.
private java.util.HashMap<java.lang.String,java.lang.String>	transformations Mapa sufiksnih transformacija.
private static java.util.regex.Pattern	vowelPattern Skup samoglasnika.
private java.util.ArrayList<java.lang.String>	wordEnd Lista završetaka reči.
private java.util.ArrayList<java.util.regex.Pattern>	wordPatterns Lista morfoloških obrazaca reči.
private java.util.ArrayList<java.lang.String>	wordStart Lista početnih delova reči.

Constructor Summary

Constructors

Constructor and Description
LjubesicPandzicStemmer()

Method Summary

Modifier and Type	Method and Description
private java.lang.String	capitalizeSyllabicR(java.lang.String word) Kapitalizuje slogotvorno R u zadatoj reči, ako postoji
private boolean	hasAVowel(java.lang.String word) Proverava da li reč sadrži samoglasnik/slogotvorno R
protected void	initRules() Inicijalizuje pravila za stemovanje
java.lang.String	stemLine(java.lang.String line) Stemuje liniju teksta
java.lang.String	stemWord(java.lang.String word) Ako se naiđe na neku od stop-reči, ona se preskače.
private java.lang.String	transform(java.lang.String word) Zamenjuje sufiks reči transformisanom varijantom tog sufiksa

Methods inherited from class weka.core.stemmers.SCStemmer

getRevision, main, replaceSpaceWithNewLine, stem, stemFile, stemText

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

serialVersionUID

private static final long serialVersionUID

See Also:

Constant Field Values

transformations

private java.util.HashMap<java.lang.String,java.lang.String> transformations

Mapa sufiksnih transformacija.

The map of suffix transformations.

stopset

private java.util.HashSet<java.lang.String> stopset

Lista stop-reči. Korišćena je implementacija u vidu hashseta radi brzine.

The list of stop-words. A hashset implementation was used for the sake of efficiency.

wordStart

private java.util.ArrayList<java.lang.String> wordStart

Lista početnih delova reči.

The list of word beginnings.

wordEnd

private java.util.ArrayList<java.lang.String> wordEnd

Lista završetaka reči.

The list of word endings.

wordPatterns

private java.util.ArrayList<java.util.regex.Pattern> wordPatterns

Lista morfoloških obrazaca reči.

The list of morphological patterns of words.

vowelPattern

private static final java.util.regex.Pattern vowelPattern

Skup samoglasnika.

The set of vowels.

Constructor Detail

LjubesicPandzicStemmer

public LjubesicPandzicStemmer()

Method Detail

stemWord

public java.lang.String stemWord(java.lang.String word)

Ako se naiđe na neku od stop-reči, ona se preskače. U suprotnom, sufiks reči se najpre transformiše a zatim i uklanja.

If a stop-word is encountered, it is skipped. Otherwise, the suffix of the word is first transformed and then removed.

Specified by:

stemWord in class SCStemmer

Parameters:

word - Reč koju treba obraditi
The word that should be processed

Returns:

Stemovana reč
The stemmed word

stemLine

public java.lang.String stemLine(java.lang.String line)

Stemuje liniju teksta

Stems a line of text

Specified by:

stemLine in class SCStemmer

transform

private java.lang.String transform(java.lang.String word)

Zamenjuje sufiks reči transformisanom varijantom tog sufiksa

Replaces the word suffix with a transformed variant of that suffix

Parameters:

word - Reč koju treba obraditi
The word that should be processed

Returns:

Transformisana reč
The transformed word

capitalizeSyllabicR

private java.lang.String capitalizeSyllabicR(java.lang.String word)

Kapitalizuje slogotvorno R u zadatoj reči, ako postoji

Capitalizes the syllabic R in the given word, if it exists

Parameters:

word - Reč koju treba obraditi
The word that should be processed

Returns:

Reč sa kapitalizovanim slogotvornim R
The word with the syllabic R capitalized

hasAVowel

private boolean hasAVowel(java.lang.String word)

Proverava da li reč sadrži samoglasnik/slogotvorno R

Checks whether the word contains a vowel/syllabic R

Parameters:

word - Reč koju treba obraditi
The word that should be processed

Returns:

True ako reč sadrži samoglasnik/slogotvorno R, false u suprotnom
True if the word contains a vowel/syllabic R, false otherwise

initRules

protected void initRules()

Description copied from class: SCStemmer

Inicijalizuje pravila za stemovanje

Initializes the stemming rules

Specified by:

initRules in class SCStemmer