Hello i had to use older version for my SS project (SS 3.1.5). I add SS fluent (3.2.3) to make my project multilanguage, everything works fine with , ENG, LAT, SW, DE languages but when i add Russian (RU) and then run dev/build i get duplicating DB column error. I get them even if i manually remove the columns.
This is one of the error messages what i get. Full build list and error list i posted here - https://github.com/tractorcow/silverstripe-fluent/issues/340
Couldn't run query: ALTER TABLE "SiteTree_Live" ADD "URLSegment_ru_RU" varchar(255) character set utf8 collate utf8_general_ci, ADD "Title_ru_RU" varchar(255) character set utf8 collate utf8_general_ci, ADD "MenuTitle_ru_RU" varchar(100) character set utf8 collate utf8_general_ci, ADD "Content_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "MetaDescription_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "ExtraMeta_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "ReportClass_ru_RU" varchar(50) character set utf8 collate utf8_general_ci, ADD index "URLSegment_ru_RU" ("URLSegment_ru_RU") Duplicate column name 'URLSegment_ru_RU'
MySQLDatabase.php:598
MySQLDatabase->databaseError(Couldn't run query: ALTER TABLE "SiteTree_Live" ADD "URLSegment_ru_RU" varchar(255) character set utf8 collate utf8_general_ci, ADD "Title_ru_RU" varchar(255) character set utf8 collate utf8_general_ci, ADD "MenuTitle_ru_RU" varchar(100) character set utf8 collate utf8_general_ci, ADD "Content_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "MetaDescription_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "ExtraMeta_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "ReportClass_ru_RU" varchar(50) character set utf8 collate utf8_general_ci, ADD index "URLSegment_ru_RU" ("URLSegment_ru_RU") | Duplicate column name 'URLSegment_ru_RU',256)
MySQLDatabase.php:150
MySQLDatabase->query(ALTER TABLE "SiteTree_Live" ADD "URLSegment_ru_RU" varchar(255) character set utf8 collate utf8_general_ci, ADD "Title_ru_RU" varchar(255) character set utf8 collate utf8_general_ci, ADD "MenuTitle_ru_RU" varchar(100) character set utf8 collate utf8_general_ci, ADD "Content_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "MetaDescription_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "ExtraMeta_ru_RU" mediumtext character set utf8 collate utf8_general_ci, ADD "ReportClass_ru_RU" varchar(50) character set utf8 collate utf8_general_ci, ADD index "URLSegment_ru_RU" ("URLSegment_ru_RU"))
MySQLDatabase.php:320
MySQLDatabase->alterTable(SiteTree_Live,Array,Array,Array,Array,,)
Database.php:223
SS_Database->endSchemaUpdate()
DatabaseAdmin.php:215
DatabaseAdmin->doBuild(,1)
DatabaseAdmin.php:100
DatabaseAdmin->build()
DatabaseAdmin.php:80
DatabaseAdmin->index(SS_HTTPRequest)
RequestHandler.php:288
RequestHandler->handleAction(SS_HTTPRequest,index)
Controller.php:194
Controller->handleAction(SS_HTTPRequest,index)
RequestHandler.php:200
RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
Controller.php:153
Controller->handleRequest(SS_HTTPRequest,DataModel)
DevelopmentAdmin.php:146
DevelopmentAdmin->build(SS_HTTPRequest)
RequestHandler.php:288
RequestHandler->handleAction(SS_HTTPRequest,build)
Controller.php:194
Controller->handleAction(SS_HTTPRequest,build)
RequestHandler.php:200
RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
Controller.php:153
Controller->handleRequest(SS_HTTPRequest,DataModel)
Director.php:366
Director::handleRequest(SS_HTTPRequest,Session,DataModel)
Director.php:152
Director::direct(/dev/build,DataModel)
main.php:189
Related
I'm creating a website design in PHP with aquarium ads - www.akwa-market.pl
I wanted the site to be international.
I have created a database and tables, one of them is with classifieds entries. I also have a built-in word search.
Suppose I typed in the polish word turtle - żółw
I run the command:
SELECT FROM * table_name WHERE item REGEXP '[[:<:]]{$keyword}[[:>:]]';
or:
SELECT FROM * table_name WHERE item REGEXP '{$keyword}';
and find a record in the database that contains that word.
The problem occurs when I enter a word in Russian, for example Привет
The search engine does not show a record with the given Russian word.
After connecting to the database, I call.
mysqli_set_charset($db, "utf8mb4");
mysqli_query($db, "SET NAMES utf8mb4");
mysqli_query($db, "SET CHARACTER SET utf8mb4");
I tried to set in db the string comparison method to utf8mb4_bin or to utf8mb4_unicode_ci
but this does not give successful results. It is still possible to search only words containing Latin characters, including diacritics of European languages.
I don't know how to set the query to be able to search for words in other languages, e.g. Russian, Chinese or Arabic.
I mean to use it together with REGEXP, preferably to search for several words at the same time, separated by the 'I' sign, i.e. OR, where, for example, the $keyword variable takes the value:
$keyword = 'Привет | Андрей';
P.S. Additionally, I don't know how to solve the case when someone types - polish word tortoise - zółw with z letter without dot, so that the word tortoise - żółw can be searched. Or even written without pressing ALT, e.g. zolw
Can anyone help? The site is almost functioning normally there is no errors given by the server, but the search engine is not working correctly and I don't really know how to solve it.
Sorry, but English is my second language.
Edit:
Output show create table as user #ysth asked.
CREATE TABLE `items` (
`index` int(11) NOT NULL AUTO_INCREMENT,
`id` varchar(11) COLLATE utf8_unicode_ci NOT NULL,
`user_name` varchar(30) COLLATE utf8_unicode_ci DEFAULT NULL,
`user_id` varchar(30) COLLATE utf8_unicode_ci DEFAULT NULL,
`price` decimal(10,2) DEFAULT NULL,
`main_category` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`category` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`sub_category` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`description` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL,
`country` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`voivodeship` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`town` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`latitude` decimal(11,8) DEFAULT NULL,
`longitude` decimal(11,8) DEFAULT NULL,
`email` varchar(50) COLLATE utf8_unicode_ci NOT NULL,
`entry_start` timestamp NULL DEFAULT NULL,
`entry_expire` timestamp NULL DEFAULT NULL,
`adv_type` varchar(30) COLLATE utf8_unicode_ci NOT NULL,
`adv_status` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL,
`adv_views` decimal(9,0) DEFAULT 0,
`users_report` text COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`index`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
as I see now the table description column where the Russian text was saved has different collation utf8mb4_bin, if that is the case of problem sorry for disturbing, I have to check that right now and let You know.
Recently I noticed few queries are taking very long time in execution, checked further and found that MySQL Optimizer is trying to use COLLATE in Where clause and that's causing performance issue, if I run below query without COLLATE then getting quick response from database:
SELECT notification_id FROM notification
WHERE ref_table = 2
AND ref_id = NAME_CONST('v_wall_detail_id',_utf8mb4'c37e32fc-b3b5-11ec-befc-02447a44a47c' COLLATE 'utf8mb4_unicode_ci')
MySQL version 5.7
Database Character Set: utf8mb4
Column Character set: UTF8
Column Data Type: CHAR(36) UUID
From PHP in Connection object passing: utf8mb4
Index is applied
This query is written in MySQL stored procedure
SHOW CREATE TABLE
CREATE TABLE `notification` (
`notification_id` CHAR(36) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`title` VARCHAR(500) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`notification_id`)
) ENGINE=INNODB DEFAULT CHARSET=utf8mb4
SHOW VARIABLES LIKE 'coll%';
collation_connection utf8_general_ci
collation_database utf8mb4_unicode_ci
collation_server latin1_swedish_ci
SHOW VARIABLES LIKE 'char%';
character_set_client, Connection,Result, System: utf8
character_set_database utf8mb4
character_set_server latin1
character_set_system utf8
Any suggestion, what improvements are needed to make my queries faster?
The table's character set is utf8, so I guess its collation is one of utf8_general_ci or utf8_unicode_ci. You can check this way:
SELECT collation_name from INFORMATION_SCHEMA.COLUMNS
WHERE table_schema = '...your schema...' AND table_name = 'notification'
AND column_name = 'ref_id';
You are forcing it to compare to a string with a utf8mb4 charset and collation. An index is a sorted data structure, and the sort order depends on the collation of the column. Using that index means taking advantage of the sort order to look up values rapidly, without examining every row.
When you compared the column to a string with a different collation, MySQL cannot infer that the sort order or string equivalence of your UUID constant is compatible. So it must do string comparison the hard way, row by row.
This is not a bug, this is the intended way for collations to work. To take advantage of the index, you must compare to a string with a compatible collation.
I tested and found that the following expressions fail to use the index:
Different character set, different collation:
WHERE ref_id = _utf8mb4'c37e32fc-b3b5-11ec-befc-02447a44a47c' COLLATE utf8mb4_general_ci
WHERE ref_id = _utf8mb4'c37e32fc-b3b5-11ec-befc-02447a44a47c' COLLATE utf8mb4_unicode_ci
Same character set, different collation:
WHERE ref_id = _utf8'c37e32fc-b3b5-11ec-befc-02447a44a47c' COLLATE 'utf8_unicode_ci'
The following expressions successfully use the index:
Different character set, default collation:
WHERE ref_id = _utf8mb4'c37e32fc-b3b5-11ec-befc-02447a44a47c'
Same character set, same collation:
WHERE ref_id = _utf8'c37e32fc-b3b5-11ec-befc-02447a44a47c' COLLATE 'utf8_general_ci'
Same character set, default collation:
WHERE ref_id = _utf8'c37e32fc-b3b5-11ec-befc-02447a44a47c'
To simplify your environment, I recommend you should just use one character set and one collation in all tables and in your session. I suggest:
ALTER TABLE notification CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
This will rebuild the indexes on string columns, using the sort order for the specified collation.
Then using COLLATE utf8mb4_unicode_ci will be compatible, and will use the index.
P.S. In all cases I omitted the NAME_CONST() function, because it has no purpose in a WHERE clause as far as I know. I don't know why you are using it.
These say what the client is talking in:
collation_connection utf8_general_ci
character_set_client, Connection,Result, System: utf8
Either change them or change the various columns to match them.
If you have Stored routines, they need to be dropped, do SET NAMES to match what you picked, then re-CREATEd.
Since you are using 5.7, I recommend using utf8mb4 and utf8mb4_unicode_520_ci throughout.
I am trying to insert some utf8 encoded data into a mysql database. The special characters are correctly displayed in the browser, but not in my database. After manually changing the collation to utf8_unicode_ci and confirming that "this operation will attempt to convert your data to the new collation", the data gets displayed correctly. However if I create the table using
CREATE TABLE IF NOT EXISTS table_name (
date date NOT NULL,
searchengine VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
location VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
keyword VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
position INT NOT NULL,
competition VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
url VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL
)
and insert the data after creating the table, the data is still not shown correctly, even though the colletion is utf8_unicode_ci. Any ideas on how to fix this?
A collation is a set of rules that defines how to compare and sort character strings. Each collation in MySQL belongs to a single character set. Every character set has at least one collation, and most have two or more collations. A collation orders characters based on weights.
utf8mb4_unicode_ci is based on the Unicode standard for sorting and comparison, which sorts accurately in a very wide range of languages.
I want to use cascading in my project to simplify certain processes. I created these two queries as well as a couple more but all child tables throw the same error on execution. They worked when I used MyISAM as the engine, but on further testing and research I figured that it does not support cascading, so I switched the engine to InnoDB which triggered these errors. I've looked at a couple of forums and threads with the same problem but I just can't seem to figure out where the actual problem lies. Can someone help?
ParentTable:
CREATE TABLE IF NOT EXISTS `branches` ( `branch_id` int(11) NOT NULL AUTO_INCREMENT, `key` VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `name` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `short_name` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `city` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `timezone` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, UNIQUE(`key`), PRIMARY KEY (`branch_id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8
Child Table:
CREATE TABLE IF NOT EXISTS `files` ( `file_id` int(11) NOT NULL AUTO_INCREMENT, `branch_id` int(11) NOT NULL, `path` text CHARACTER SET utf8 COLLATE utf8_bin NOT NULL, `use_google_analytics` BOOLEAN NOT NULL, FOREIGN KEY(`branch_id`) REFERENCES `branches`(`branch_id`) ON DELETE CASCADE ON UPDATE CASCADE, PRIMARY KEY (`file_id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8
Error:
1005: Can't create table 'files' (errno: 150)
You defined your foreign key
ON DELETE CASCADE
But your branch_id is defined as NOT NULL. That won't work.
You want your branch_id never be null but then the foreign key related data gets removed you want it set to NULL with the CASCADE option.
So either change removd the ON DELETE CASCADE or allow NULL in the foreign key. This works
SQLFiddle demo
I have a MySQL database with all the table fields collation as
latin1_swedish_ci
It has almost 1000 of the records already stored and now I want to convert all these data into
utf8_general_ci
So that I can display any language content. I have already altered the field collations into utf8_general_ci but this does not CONVERT all the old records into utf8_general_ci
one funny thing.
CONVERT TO CHARSET and CONVERT()/CAST() suggested by Anshu will work fine if charset in the table is in right encoding.
If for some reason latin1 column containts utf8 text, CONVERT() and CAST() will not be able to help. I had "messed" my database with that setup so spend bit more time on solving this.
to fix this in addition to character set conversion, there are several exercises required.
"Hard one" is to recreate the database from dump that will be converted via console
"Simple one" is to convert row by row or table by table:
INSERT INTO UTF8_TABLE (UTF8_FIELD)
SELECT convert(cast(convert(LATIN1_FIELD using latin1) as binary) using utf8)
FROM LATIN1_TABLE;
basically, both cases will process string to original symbols and then to right encoding, that won't happen with simple convert(field using encoding) from table; command.
Export your table.
Drop the table.
Open the export file in the editor.
Edit it manually where the table structure is created.
old query:
CREATE TABLE `message` (
`message_id` int(11) NOT NULL,
`message_thread_id` int(11) NOT NULL,
`message_from` int(11) NOT NULL,
`message_to` int(11) NOT NULL,
`message_text` longtext NOT NULL,
`message_time` varchar(50) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
new query: ( suppose you want to change message_text field. )
CREATE TABLE `message` (
`message_id` int(11) NOT NULL,
`message_thread_id` int(11) NOT NULL,
`message_from` int(11) NOT NULL,
`message_to` int(11) NOT NULL,
`message_text` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`message_time` varchar(50) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
save the file and import back to the database.