Extracting cyrillic terms/keywords from text in php - php

Question

I'm trying to build keywords for my webpage and I want that keywords to be extracted from text.
I have that function
function extractCommonWords($string){
$stopWords = array('и', 'или');
$string = preg_replace('/ss+/i', '', $string);
$string = trim($string);
$string = preg_replace('/[^a-zA-Z0-9 -]/', '', $string);
$string = strtolower($string);
preg_match_all('/\b.*?\b/i', $string, $matchWords);
$matchWords = $matchWords[0];
foreach ( $matchWords as $key=>$item ) {
if ( $item == '' || in_array(strtolower($item), $stopWords) || strlen($item) <= 3 ) {
unset($matchWords[$key]);
}
}
$wordCountArr = array();
if ( is_array($matchWords) ) {
foreach ( $matchWords as $key => $val ) {
$val = strtolower($val);
if ( isset($wordCountArr[$val]) ) {
$wordCountArr[$val]++;
} else {
$wordCountArr[$val] = 1;
}
}
}
arsort($wordCountArr);
$wordCountArr = array_slice($wordCountArr, 0, 10);
return $wordCountArr;
}
Here is what I try:
$text = "Текст кирилица";
$words = extractCommonWords($text);
echo implode(',', array_keys($words));
The problem is dosen`t work with cyrillic letters. How to fix that ?

Answer 1

Cyrillic letters are multi-byte characters. You'll need to use multi-byte character function of PHP.
For regular expressions, you'll need to add the /u modifier to make them unicode compliant.
See also Are the PHP preg_functions multibyte safe?

Answer 2

Your pattern to replace will also remove all cyrillic characters, because a-z will not match them.
Add this to the character-class to keep the cyrillic characters:
\p{Cyrillic}
...and use the modifiier u like suggested by GolezTrol.
$string = preg_replace('/[^\p{Cyrillic} a-zA-Z0-9 -]/u', '', $string);
If you only like to extract cyrillic words, you don't need to replace anything, just use this to match the words:
preg_match_all('/\b(\p{Cyrillic}+)\b/u', $string, $matchWords);