Home » Server Options » Text & interMedia » Problem with stop words
Problem with stop words [message #272371] Thu, 04 October 2007 14:56 Go to next message
amazing
Messages: 46
Registered: September 2007
Location: Venezuela
Member
Hi. I create my own stop list with the preference BASIC_STOPLIST that have my personalized stop words. For some contains query operators, when return a stop word, it do not appear in the resulting query, or highlighted. Doesn't it..?

Here is the problem. I have an 'Y' stop word. And when I use for example

...contains(text, '$Y', 1) > 0 ...

the resulting query or highlighted comes with that word and it also happen with the others stop words. So what am i doing wrong...???

I'm dealing with Oracle 10g patch 10.2.0.3 and i know that it makes some problems. Am I dealing with this kind of problem?
Re: Problem with stop words [message #272397 is a reply to message #272371] Thu, 04 October 2007 20:35 Go to previous messageGo to next message
yumyumyeh
Messages: 8
Registered: October 2007
Junior Member
Did you re-create or rebuild your index with your stoplist preference?
Re: Problem with stop words [message #272403 is a reply to message #272397] Thu, 04 October 2007 22:50 Go to previous messageGo to next message
amazing
Messages: 46
Registered: September 2007
Location: Venezuela
Member
Yes. Why?
Re: Problem with stop words [message #272580 is a reply to message #272403] Fri, 05 October 2007 10:32 Go to previous messageGo to next message
yumyumyeh
Messages: 8
Registered: October 2007
Junior Member
What results are you getting when you run your query?
Re: Problem with stop words [message #272588 is a reply to message #272580] Fri, 05 October 2007 10:57 Go to previous messageGo to next message
amazing
Messages: 46
Registered: September 2007
Location: Venezuela
Member
If I query with a text that includes a stop word, and use an operator that when return a stop word, do not include this stop word, the resulting query also includes a stop word. I' am developing an spanish solution: For example, a stop word is 'Y'. If i query by 'BANCOS Y TIENDAS', the resulting query is like:

'BANCOS Y TIENDAS'
'... Y ...'
'Y...Y...'

Re: Problem with stop words [message #272648 is a reply to message #272588] Fri, 05 October 2007 15:24 Go to previous message
yumyumyeh
Messages: 8
Registered: October 2007
Junior Member
It looks like your index has not really been updated with your stopword.

Example:

create table test_index(search_text CLOB);

INSERT INTO test_index VALUES ('on?demand');
INSERT INTO test_index VALUES ('on demand');
INSERT INTO test_index VALUES ('on demand?');

BEGIN
CTX_DDL.CREATE_PREFERENCE ('test_lexer', 'BASIC_LEXER');
CTX_DDL.SET_ATTRIBUTE ('test_lexer', 'PRINTJOINS', '?');
END;

CREATE INDEX i_tk_test_index ON test_index (search_text) INDEXTYPE IS CTXSYS.CONTEXT PARAMETERS ('LEXER test_lexer STOPLIST CTXSYS.DEFAULT_STOPLIST');

SELECT token_text FROM dr$i_tk_test_index$i

SELECT * FROM test_index WHERE CONTAINS (search_text, '$on', 1) > 0

The above query is trying stemming ($) on a stop word (on) and it is not returning any results. So, if your index has been built correctly then it should not return any rows containing the stop word.

I would try to drop and recreate the index or atleast try rebuilding it with your set of preferences again.
Previous Topic: BUG in contains?
Next Topic: Ending printjoin character issue and CONTAINS function not working (merged 2 threads)
Goto Forum:
  


Current Time: Thu Mar 28 16:11:38 CDT 2024