Searching with synonym in elastic search - php

I am implementing search with synonym.
These are changes for searching with synonym.
This is my setting in index. I used bubble search extension for that and modified that code.
$indexSettings['analysis']['analyzer'] = array(
'std' => array(
'tokenizer' => 'standard',
'char_filter' => 'html_strip', // strip html tags
'filter' => array('standard', 'elision', 'asciifolding', 'lowercase', 'stop', 'length'),
),
'keyword' => array(
'tokenizer' => 'keyword',
'filter' => array('asciifolding', 'lowercase'),
),
'keyword_prefix' => array(
'tokenizer' => 'keyword',
'filter' => array('asciifolding', 'lowercase', 'edge_ngram_front'),
),
'text_prefix' => array(
'tokenizer' => 'standard',
'char_filter' => 'html_strip', // strip html tags
'filter' => array('standard', 'elision', 'asciifolding', 'lowercase', 'stop', 'edge_ngram_front'),
),
'text_suffix' => array(
'tokenizer' => 'standard',
'char_filter' => 'html_strip', // strip html tags
'filter' => array('standard', 'elision', 'asciifolding', 'lowercase', 'stop', 'edge_ngram_back'),
),
/*This is my analyzer for synonym*/
'synonym' => array(
'tokenizer' => 'standard',
'type' => 'custom',
// 'char_filter' => 'html_strip', // strip html tags
'filter' => array('synonym','standard', 'elision', 'asciifolding', 'lowercase', 'stop',"kstem", "length" )
),
);
$indexSettings['analysis']['filter'] = array(
'edge_ngram_front' => array(
'type' => 'edgeNGram',
'min_gram' => 2,
'max_gram' => 10,
'side' => 'front',
),
'edge_ngram_back' => array(
'type' => 'edgeNGram',
'min_gram' => 2,
'max_gram' => 10,
'side' => 'back',
),
'stop' => array(
'type' => 'stop',
'stopwords' => '_none_',
),
'length' => array(
'type' => 'length',
'min' => 2,
),
/*This is filter for synonym */
'synonym' => array(
'type' => 'synonym',
'expand' => true,
'synonyms_path' => 'synonyms.txt',
),
);
$properties[$key]['analyzer'] = 'synonym';
I am getting result but not up to mark.
eg lkjhgf => arylacetamide deacetylase like 3
If i search for lkjhgf then it will give result for
arylacetamide deacetylase like 1
arylacetamide deacetylase like 2
arylacetamide deacetylase like 3
arylacetamide deacetylase like 4
I want exact match and my fuzzy login is also disabled.

Related

Typo3 use realurl to mask tx_news_pi1 parameter in URL

I am pretty new to typo3. The Website is all set up now i want to use realurl to make some human readable URLS.
Out of the boy it works pretty well. But not for my news sites.
http://myDomain/news-events/news/Name-of-the-Article/?tx_news_pi1%5Bday%5D=19&tx_news_pi1%5Bmonth%5D=7&tx_news_pi1%5Byear%5D=2017&cHash=6af067caeb037b4de744f6b9e07b73e8
Please help me to get rid of the last Parameters.
to get something like:
http://myDomain/news-events/news/Name-of-the-Article/
Here is my realurl_conf
[ See Update]
Thank you all for your time =)
Update:
So i worked my way through the manual.
I copied from them and inserted my IDs and Stuff
this is my new Configuration [ Deleted the first one ]
<?php
$GLOBALS['TYPO3_CONF_VARS']['FE']['addRootLineFields'] .= ',tx_realurl_pathsegment';
// Adjust to your needs
$domain = 'http://mydomain.de';
$rootPageUid = 1;
#$rssFeedPageType = 9818; // pageType of your RSS feed page
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['realurl'][$domain] = array(
'pagePath' => array(
'type' => 'user',
'userFunc' => 'EXT:realurl/class.tx_realurl_advanced.php:&tx_realurl_advanced->main',
'spaceCharacter' => '-',
'languageGetVar' => 'L',
'expireDays' => '3',
'rootpage_id' => $rootPageUid,
'firstHitPathCache' => 1
),
'init' => array(
'enableCHashCache' => TRUE,
'respectSimulateStaticURLs' => 0,
'appendMissingSlash' => 'ifNotFile,redirect',
'adminJumpToBackend' => TRUE,
'enableUrlDecodeCache' => TRUE,
'enableUrlEncodeCache' => TRUE,
'emptyUrlReturnValue' => '/',
),
'fileName' => array(
'defaultToHTMLsuffixOnPrev' => 0,
'acceptHTMLsuffix' => 1,
'index' => array(
# 'feed.rss' => array(
# 'keyValues' => array(
# 'type' => $rssFeedPageType,
# )
# )
)
),
'preVars' => array(
array(
'GETvar' => 'L',
'valueMap' => array(
# 'en' => '1',
),
'noMatch' => 'bypass',
),
array(
'GETvar' => 'no_cache',
'valueMap' => array(
'nc' => 1,
),
'noMatch' => 'bypass',
),
),
'fixedPostVars' => array(
'newsDetailConfiguration' => array(
array(
'GETvar' => 'tx_news_pi1[action]',
'valueMap' => array(
'detail' => '',
),
'noMatch' => 'bypass'
),
array(
'GETvar' => 'tx_news_pi1[controller]',
'valueMap' => array(
'News' => '',
),
'noMatch' => 'bypass'
),
array(
'GETvar' => 'tx_news_pi1[news]',
'lookUpTable' => array(
'table' => 'tx_news_domain_model_news',
'id_field' => 'uid',
'alias_field' => "CONCAT(uid, '-', IF(path_segment!='',path_segment,title))",
/** OR ***************/
'alias_field' => 'IF(path_segment!="",path_segment,title)',
/** OR ***************/
'alias_field' => "CONCAT(uid, '-', title)",
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => 1, # 1?
'useUniqueCache_conf' => array(
'strtolower' => 1,
'spaceCharacter' => '-'
),
'languageGetVar' => 'L',
'languageExceptionUids' => '',
'languageField' => 'sys_language_uid',
'transOrigPointerField' => 'l10n_parent',
'expireDays' => 180,
),
#Depends ?
array(
'GETvar' => 'tx_news_pi1[day]',
'noMatch' => 'bypass',
),
array(
'GETvar' => 'tx_news_pi1[month]',
'noMatch' => 'bypass',
),
array(
'GETvar' => 'tx_news_pi1[year]',
'noMatch' => 'bypass',
),
),
'newsCategoryConfiguration' => array(
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][categories]',
'lookUpTable' => array(
'table' => 'sys_category',
'id_field' => 'uid',
'alias_field' => 'title',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => 1,
'useUniqueCache_conf' => array(
'strtolower' => 1,
'spaceCharacter' => '-'
)
)
)
),
'newsTagConfiguration' => array(
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][tags]',
'lookUpTable' => array(
'table' => 'tx_news_domain_model_tag',
'id_field' => 'uid',
'alias_field' => 'title',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => 1,
'useUniqueCache_conf' => array(
'strtolower' => 1,
'spaceCharacter' => '-'
)
)
)
),
#TODO: ID-News Seite Finden
#'145' => 'newsDetailConfiguration',
#'147' => 'newsDetailConfiguration', // For additional detail pages, add their uid as well
#'134' => 'newsDetailConfiguration', // For additional detail pages, add their uid as well
'148' => 'newsDetailConfiguration', // For additional detail pages, add their uid as well
'149' => 'newsDetailConfiguration', // For additional detail pages, add their uid as well
#'71' => 'newsTagConfiguration',
#'72' => 'newsCategoryConfiguration',
),
'postVarSets' => array(
'_DEFAULT' => array(
'controller' => array(
array(
'GETvar' => 'tx_news_pi1[action]',
'noMatch' => 'bypass'
),
array(
'GETvar' => 'tx_news_pi1[controller]',
'noMatch' => 'bypass'
)
),
'dateFilter' => array(
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][year]',
),
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][month]',
),
),
'page' => array(
array(
'GETvar' => 'tx_news_pi1[#widget_0][currentPage]',
),
),
),
),
)
);
But still there are the
tx_news_pi1[day], tx_news_pi1[month], tx_news_pi1[year] and cHash
Parameters in the URL.
Please Help me to get rid of them.
thats my configuration:
<?php
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['realurl']=array (
'_DEFAULT' => array (
'init' => array (
'appendMissingSlash' => 'ifNotFile,redirect',
'emptyUrlReturnValue' => '/',
),
'pagePath' => array (
'rootpage_id' => '2',
),
'fileName' => array (
'defaultToHTMLsuffixOnPrev' => 0,
'acceptHTMLsuffix' => 1,
'index' => array (
'print' => array (
'keyValues' => array (
'type' => 98,
),
),
),
),
'preVars' => array(
array(
'GETvar' => 'L',
'valueMap' => array(
'en' => '1',
),
'noMatch' => 'bypass',
),
array(
'GETvar' => 'no_cache',
'valueMap' => array(
'nc' => 1,
),
'noMatch' => 'bypass',
),
),
'fixedPostVars' => array(
// config for single/detail news:
'newsDetailConfiguration' => array(
array(
'GETvar' => 'tx_news_pi1[action]',
'valueMap' => array(
'detail' => '',
),
'noMatch' => 'bypass'
),
array(
'GETvar' => 'tx_news_pi1[controller]',
'valueMap' => array(
'News' => '',
),
'noMatch' => 'bypass'
),
array(
'GETvar' => 'tx_news_pi1[news]',
'lookUpTable' => array(
'table' => 'tx_news_domain_model_news',
'id_field' => 'uid',
'alias_field' => 'concat(uid,\'_\',title)',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => 1,
'useUniqueCache_conf' => array(
'strtolower' => 1,
'spaceCharacter' => '-'
),
'languageGetVar' => 'L',
'languageExceptionUids' => '',
'languageField' => 'sys_language_uid',
'transOrigPointerField' => 'l10n_parent',
'autoUpdate' => 1,
'expireDays' => 180,
)
)
),
// config for category selection:
'newsCategoryConfiguration' => array(
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][categories]',
'lookUpTable' => array(
'table' => 'sys_category',
'id_field' => 'uid',
'alias_field' => 'title',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => 1,
'useUniqueCache_conf' => array(
'strtolower' => 1,
'spaceCharacter' => '-'
)
)
)
),
// configuration for tag selection:
'newsTagConfiguration' => array(
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][tags]',
'lookUpTable' => array(
'table' => 'tx_news_domain_model_tag',
'id_field' => 'uid',
'alias_field' => 'title',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => 1,
'useUniqueCache_conf' => array(
'strtolower' => 1,
'spaceCharacter' => '-'
)
)
)
),
// add your page uids where you have detail view of news:
'70' => 'newsDetailConfiguration',
//'701' => 'newsDetailConfiguration', // For additional detail pages, add their uid as well
/connect page uid for further views: tag-selection, category-selection"
'71' => 'newsTagConfiguration',
'72' => 'newsCategoryConfiguration',
),
'postVarSets' => array(
'_DEFAULT' => array(
'controller' => array(
array(
'GETvar' => 'tx_news_pi1[action]',
'noMatch' => 'bypass'
),
array(
'GETvar' => 'tx_news_pi1[controller]',
'noMatch' => 'bypass'
)
),
'dateFilter' => array(
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][year]',
),
array(
'GETvar' => 'tx_news_pi1[overwriteDemand][month]',
),
),
'page' => array(
array(
'GETvar' => 'tx_news_pi1[#widget_0][currentPage]',
),
),
),
),
),
);
look for the right insertion in the array. For that have a clean indention!
The already mentioned link is quite useful and contains detailed info:
https://docs.typo3.org/typo3cms/extensions/news/AdministratorManual/BestPractice/Realurl/Index.html
There are 3 ways explained there:
a basic one which still shows action and controller name as an URL path segment
an advanced example (the one Bernd Wilke is using in his sample as well)
a TypoScript solution to avoid controller and action name in URL
Option 2 can only be used if you show your detail news on a dedicated pages (separate page for list and detail view) as you configure a preset for a controller and action name for a special page (uid) - your news detail view.
Option 3 basically does the same as Option 2, but on a TypoScript level and it can be used in a case where list and detail view is on one single page.
Both options can / should be used in combination with the setting skipControllerAndAction = 1.
In case you want to use the basic setup. The postVarSets is on the same level as fixedPortsVars or preVars
Additional hint:
Do you use any other language than DE or EN as your default language? If so your default language has sys language uid of 0 and thus all good.
The default language usually has sys language uid of 0. If you do not need a default language path segment, you leave it out in our real url config thus it gets bypassed.
More info on language setup: https://github.com/dmitryd/typo3-realurl/wiki/Notes-for-Integrators#configuring-languages
RealUrl Doc:
https://github.com/dmitryd/typo3-realurl/wiki

Cannot search string in elastic search after adding synonym

I have implemented synonym in elasticsearch but now i am facing problem. Whole string is splited on whitespace and i can not search with "see all result option".
Here is my settings.
$indexSettings['analysis']['analyzer'] = array(
'std' => array( // Will allow query 'shoes' to match better than 'shoe' which the stemmed version
'tokenizer' => 'standard',
'char_filter' => 'html_strip', // strip html tags
'filter' => array('standard', 'elision', 'asciifolding', 'lowercase', 'stop', 'length'),
),
'keyword' => array(
'tokenizer' => 'keyword',
'filter' => array('asciifolding', 'lowercase'),
),
'keyword_prefix' => array(
'tokenizer' => 'keyword',
'filter' => array('asciifolding', 'lowercase', 'edge_ngram_front'),
),
'text_prefix' => array(
'tokenizer' => 'standard',
'char_filter' => 'html_strip', // strip html tags
'filter' => array('standard', 'elision', 'asciifolding', 'lowercase', 'stop', 'edge_ngram_front'),
),
'text_suffix' => array(
'tokenizer' => 'standard',
'char_filter' => 'html_strip', // strip html tags
'filter' => array('standard', 'elision', 'asciifolding', 'lowercase', 'stop', 'edge_ngram_back'),
),
'synonym' => array(
'tokenizer' => 'standard',
'type' => 'custom',
'char_filter' => 'html_strip', // strip html tags
'filter' => array('standard','synonym', 'elision', 'asciifolding', 'kstem', 'length','snowball' ,'lowercase', 'stop',)
),
);
$indexSettings['analysis']['filter'] = array(
'edge_ngram_front' => array(
'type' => 'edgeNGram',
'min_gram' => 2,
'max_gram' => 10,
'side' => 'front',
),
'edge_ngram_back' => array(
'type' => 'edgeNGram',
'min_gram' => 2,
'max_gram' => 10,
'side' => 'back',
),
'stop' => array(
'type' => 'stop',
'stopwords' => '_none_',
),
'length' => array(
'type' => 'length',
'min' => 2,
),
'synonym' => array(
'type' => 'synonym',
'expand' => true,
'synonyms_path' => 'synonyms.txt',
),
);
$properties[$key]['analyzer'] = 'synonym';
In dropdown i give 5 result to show and one option see all result.
On searching of "Anti Ssl like protein" this word i got q=Anti ssl="" like="" protein="" in anchor tag of see all result.Cant get full word with see all result option.

Aggregations are empty with keyword tokenizer

For an aggregation result i have a field (type_name2) in my mapping with a lowercased filter and a keyword tokenizer. But if I set this filter and tokenizer to the field my bucket is now empty. I have set this token and filter to ignoring spaces in words like: Fun X or Viva X.
Here my mapping (in PHP):
'cars' => array(
'properties' => array(
/**'type_name2' => array(
'type' => 'string',
'fields' => array(
'raw' => array(
'type' => 'string',
'analyzer' => 'keyword'
)
)
),**/
'type_name2' => array(
'type' => 'string',
'analyzer' => 'keyword_lower'
),
'type_kw' => array(
'type' => 'float'
),
'type_hp' => array(
'type' => 'float'
)
)
),
and the custom analyzer:
'analysis' => array(
'analyzer' => array(
'keyword_lower' => array(
'tokenizer' => 'keyword',
'filter' => 'lowercase'
),
'nGram_analyzer' => array(
'type' => 'custom',
'tokenizer' => 'whitespace',
'filter' => array(
'lowercase',
'asciifolding',
'nGram_filter'
)
),
'whitespace_analyzer' => array(
'type' => 'custom',
'tokenizer' => 'whitespace',
'filter' => array(
'lowercase',
'asciifolding'
)
)
),
'filter' => array(
'nGram_filter' => array(
'type' => 'ngram',
'min_gram' => 2,
'max_gram' => 20,
'token_chars' => array(
'letter',
'digit',
'punctation',
'symbol'
)
),
)
)
but the result of type_name2 is empty:
'type_name2' =>
array (size=1)
'buckets' =>
array (size=0)
empty
Any ideas, what is wrong?
Thx and regards
Stefan

how to pre-fill select box in typo3 backend

First of all excuse my bad english.
I got a problem with my select-field in the BE. I would like to prefill (preselect) all of the available items.
Code in ext_tables.php:
'teilnehmer' => array(
'exclude' => 0,
'label' => 'LLL:EXT:kiwanisext/Resources/Private/Language/locallang_db.xlf:tx_kiwanisext_domain_model_veranstaltung.teilnehmer',
'config' => array(
'type' => 'select',
'foreign_table' => 'fe_users',
'MM' => 'tx_kiwanisext_veranstaltung_user_mm',
'size' => 10,
'autoSizeMax' => 30,
'maxitems' => 9999,
'multiple' => 0,
'wizards' => array(
'_PADDING' => 1,
'_VERTICAL' => 1,
'edit' => array(
'type' => 'popup',
'title' => 'Edit',
'script' => 'wizard_edit.php',
'icon' => 'edit2.gif',
'popup_onlyOpenIfSelected' => 1,
'JSopenParams' => 'height=350,width=580,status=0,menubar=0,scrollbars=1',
),
'add' => Array(
'type' => 'script',
'title' => 'Create new',
'icon' => 'add.gif',
'params' => array(
'table' => 'fe_users',
'pid' => '###CURRENT_PID###',
'setValue' => 'prepend'
),
'script' => 'wizard_add.php',
),
),
),
),
I found nothing helpful in the documentation.
Any hint, tip or help will be much appreciated!
Its not possible to do that with plain TCA config, afaik. You can however define a default value wich will be selected (if none is defines, 1st item will be selected).
'default' => 'myValue'
But to preselect multiple values at once, you have to use JavaScript I guess.
This code worked for me in typo3 6.2. I have a selectbox filled with database records.
I want the records with ID's 1 ans 2 to be preselected in the selectbox:
'thematique' => array(
'exclude' => 0,
'label' => 'LLL:EXT:dk_actus/locallang_db.xml:tx_dkactus_thematique',
'config' => array(
'type' => 'select',
'foreign_table' => 'tx_dkactus_thematique',
'foreign_table_where' => 'ORDER BY tx_dkactus_thematique.uid',
'size' => 10,
'minitems' => 0,
'maxitems' => 99,
'default' => '1,2',
),
),

typo3 realurl chash

I am trying to setup realurl to work with my extension.
I think I am doing something wrong because I can't get rid of the cHash
parameter. My realurl config is
REALURL START
$TYPO3_CONF_VARS['EXTCONF']['realurl'] = array(
'fileName' => array(),
'init' => array(
'enableCHashCache' => true,
'appendMissingSlash' => 'ifNotFile,redirect',
'adminJumpToBackend' => true,
'enableUrlDecodeCache' => true,
'enableUrlEncodeCache' => true,
'emptyUrlReturnValue' => '/',
),
'_DEFAULT' => array(
'redirects' => array(),
'preVars' => array(),
'pagePath' => array(
'type' => 'user',
'userFunc' => 'EXT:realurl/class.tx_realurl_advanced.php:&tx_realurl_advanced->main',
'spaceCharacter' => '-',
'expireDays' => 30,
'rootpage_id' => '1',
),
'postVarSets' => array(
'_DEFAULT' => array(
'kategori' => array(
array(
'GETvar' => 'tx_projgallery_pi1[cat]',
'lookUpTable' => array(
'table' => 'tx_projgallery_categories',
'id_field' => 'uid',
'alias_field' => 'category',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => '1',
'useUniqueCache_conf' => array(
'strtolower' => '1',
'spaceCharacter' => '-'
)
),
),
),
'album' => array(
array(
'GETvar' => 'tx_projgallery_pi1[album]',
'lookUpTable' => array(
'table' => 'tx_projgallery_gallery',
'id_field' => 'uid',
'alias_field' => 'navi_title',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => '1',
'useUniqueCache_conf' => array(
'strtolower' => '1',
'spaceCharacter' => '-'
)
),
),
),
'billede' => array(
array(
'GETvar' => 'tx_projgallery_pi1[show]',
'lookUpTable' => array(
'table' => 'tx_projgallery_photos',
'id_field' => 'uid',
'alias_field' => 'uid',
'addWhereClause' => ' AND NOT deleted',
'useUniqueCache' => '1',
'useUniqueCache_conf' => array(
'strtolower' => '1',
'spaceCharacter' => '-'
)
),
),
),
),
),
),
);
REAL URL END
I have googled it like a maniac but did find any helpful info.
Can someone see what I am doing wrong?
Check your extension configuration. Are all tables and fields created? If not try to create the tables and fields via phpMyAdmin on your own.
Check if table tx_realurl_chashcache is there, it should not be empty.
Activate enableChashUrlDebug in realurl extension configuration.
Activate enableDevLog in realurl extension configuration
Install extension devlog
So, check the log and the cache-tables for errors.
Deactivate realurl and check the links. Perhaps you find some issues then.

Categories