I have noticed when dealing with some names that are not of normal spelling ie standard alphabet UK/US are getting lost from my inserting of a record to what actually shows up in the database. I have done quiet a bit of reading regarding the Collation type, which is what I thought was causing the issue, but not sure if this is the case or I'm still doing it wrong as my problem is still persisting.
Below is an example of a record I am creating as well as my database structure, and as you can also see the last_name field has "ö", when I lookup the record I actually see the last_name "Körner"
CREATE TABLE `data` (
`id` bigint(20) NOT NULL,
`profile_id` int(11) NOT NULL,
`first_name` varchar(100) NOT NULL,
`last_name` varchar(100) NOT NULL,
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
ALTER TABLE `data`
ADD PRIMARY KEY (`id`),
ADD UNIQUE KEY `profile_id` (`profile_id`);
ALTER TABLE `data`
MODIFY `id` bigint(20) NOT NULL AUTO_INCREMENT;
INSERT IGNORE INTO data (profile_id, first_name, last_name) VALUES (1, 'Brent', 'Körner');
The field collation on the last_name is set to 'utf8_general_ci' which that I understand or should I say thought would sort this issue out.
This seems to be something I am doing wrong / missing with PHP, as when I execute the INSERT query within PhpMyAdmin it saves fine.
it seems the issue was down to PHP in the end, and i wasn't setting the charset.
For mysql
mysql_set_charset('utf8');
For mysqli
mysqli_set_charset('utf8');
ref https://akrabat.com/utf8-php-and-mysql/
Related
I'm going to prepare an installation script for my website which is automatically creates database and it's tables.
How can I send every queries in one?
Here is the PHP code:
$table_query = "CREATE TABLE IF NOT EXISTS ".TP."_usertypes(
utid INT(11) NOT NULL AUTO_INCREMENT,PRIMARY KEY (utid),
utname VARCHAR(255)
)
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci;
CREATE TABLE IF NOT EXISTS ".TP."_users(
uid INT(11) NOT NULL AUTO_INCREMENT,PRIMARY KEY (uid),
utid INT(11) NOT NULL,FOREIGN KEY (utid) REFERENCES ".TP."_usertypes(utid) ON DELETE RESTRICT,
username VARCHAR(255),
password VARCHAR(255),
avatar VARCHAR(255)
)
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci";
mysqli_query($connection,$table_query) or die(mysqli_error($connection));
Thank you.
With mysqli you're able to use multiple statements for real using mysqli_multi_query().
Read more on multiple statements in the PHP Docs.
How can I send every queries in one?
You can't.
You need to send each query in a separate php call.
I've recently started using laravel for a project I'm working on, and I'm currently having problems displaying data from my database in the correct character encoding.
My current system consists of a separate script responsible for populating the database with data, while the laravel project is reponsible for displaying the data. The view that is used, is set to display all text as utf-8, which works as I've successfully printed special characters in the view. Text from the database is not printed as utf8, and will not print special characters the right way. I've tried using both eloquent models and DB::select(), but they both show the same poor result.
charset in database.php is set to utf8 while collation is set to utf8_unicode_ci.
The database table:
CREATE TABLE `RssFeedItem` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`feedId` smallint(5) unsigned NOT NULL,
`title` varchar(250) COLLATE utf8_unicode_ci NOT NULL,
`url` varchar(250) COLLATE utf8_unicode_ci NOT NULL,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
`text` mediumtext COLLATE utf8_unicode_ci,
`textSha1` varchar(250) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `url` (`url`),
KEY `feedId` (`feedId`),
CONSTRAINT `RssFeedItem_ibfk_1` FOREIGN KEY (`feedId`) REFERENCES `RssFeed` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=6370 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
I've also set up a test page in order to see if the problem could be my database setup, but the test page prints everything just fine. The test page uses PDO to select all data, and prints it on a simple html page.
Does anyone know what the problem might be? I've tried searching around with no luck besides this link, but I haven't found anything that might help me.
I did eventually end up solving this myself. The problem was caused by the separate script responsible for populating my database with data. This was solved by running a query with SET NAMES utf8 before inserting data to the database. The original data was pulled out, and then sent back in after running said query.
The reason for it working outside laravel, was simply because the said query wasn't executed on my test page. If i ran the query before retrieving the data, it came out with the wrong encoding because the query stated that the data was encoded as utf8, when it really wasn't.
I have a MySQL database with all the table fields collation as
latin1_swedish_ci
It has almost 1000 of the records already stored and now I want to convert all these data into
utf8_general_ci
So that I can display any language content. I have already altered the field collations into utf8_general_ci but this does not CONVERT all the old records into utf8_general_ci
one funny thing.
CONVERT TO CHARSET and CONVERT()/CAST() suggested by Anshu will work fine if charset in the table is in right encoding.
If for some reason latin1 column containts utf8 text, CONVERT() and CAST() will not be able to help. I had "messed" my database with that setup so spend bit more time on solving this.
to fix this in addition to character set conversion, there are several exercises required.
"Hard one" is to recreate the database from dump that will be converted via console
"Simple one" is to convert row by row or table by table:
INSERT INTO UTF8_TABLE (UTF8_FIELD)
SELECT convert(cast(convert(LATIN1_FIELD using latin1) as binary) using utf8)
FROM LATIN1_TABLE;
basically, both cases will process string to original symbols and then to right encoding, that won't happen with simple convert(field using encoding) from table; command.
Export your table.
Drop the table.
Open the export file in the editor.
Edit it manually where the table structure is created.
old query:
CREATE TABLE `message` (
`message_id` int(11) NOT NULL,
`message_thread_id` int(11) NOT NULL,
`message_from` int(11) NOT NULL,
`message_to` int(11) NOT NULL,
`message_text` longtext NOT NULL,
`message_time` varchar(50) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
new query: ( suppose you want to change message_text field. )
CREATE TABLE `message` (
`message_id` int(11) NOT NULL,
`message_thread_id` int(11) NOT NULL,
`message_from` int(11) NOT NULL,
`message_to` int(11) NOT NULL,
`message_text` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`message_time` varchar(50) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
save the file and import back to the database.
I have ~38 columns for a table.
ID, name, and the other 36 are bit-sized settings for the user.
The 36 other columns are grouped into 6 "settings", e.g. Setting1_on, Setting1_colored, etc.
Is this the best way to do this?
Thanks.
If it must be in one table and they're all toggle type settings like yes/no, true/false, etc... use TINYINT to save space.
I'd recommend creating a separate table 'settings' with 36 records one for each option. Then create a linking table to the user table with a value column to record the user settings. This creates a many-to-many link for the user settings. It also makes it easy to add a new setting--just add a new row to the 'settings' table. Here is an example schema. I use varchar for the value of the setting to allow for later setting which might not be bits, but feel free to use TINYINT if size is an issue. This solution will not use as much space as the one table with the danger of a large sparsely populated set of columns.
CREATE TABLE `user` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(64) DEFAULT NULL,
`address` varchar(64) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `setting` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(64) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `setting_user` (
`user_id` int(11) NOT NULL DEFAULT '0',
`setting_id` int(11) unsigned NOT NULL,
`value` varchar(32) DEFAULT NULL,
PRIMARY KEY (`user_id`,`setting_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
All depends on how you want to access them. If you want to (or must) just select one of them, then go with the #Ray solution. If they can be functionally grouped (really, not some pretend grouping for all those that start with F) ie. you'll always need number of them for a function and reading and writing them doesn't make sense as an individual flag, then perhaps storing them as ints and using logic operaoprs on them might be a goer.
Saying that, unless you are doing a lot of read and writes to the db during a session, bundling them up into ints gives you very little performance wise, it would save some space on the DB, if all the options had to exist. If doesn't exist = false, it could be a toss up.
So all things being unequal, I'd go with Mr Ray.
MySQL has a SET type that could be useful here. Everything would fit into a single SET, but six SETs might make more sense.
http://dev.mysql.com/doc/refman/5.5/en/set.html
Let's say I have a screenshots table and a replies table. So: every screenshot can have multiple replies (one-to-many). Now, I want to create a combined timeline of the two, but in reality, they're pretty unrelated (in structure).
How can I select data from both tables, ordering by their publish time, descending; when for example, I can have a post, a few comments, then another post; because that would be how the timeline happened?
Normally selecting from both combines the tables; but I don't want that to happen. On that subject, I also need the tables to be distinguishable. Here's the structure for the tables...
--
-- Table structure for table `screenshots`
--
CREATE TABLE IF NOT EXISTS `screenshots` (
`id` int(11) NOT NULL auto_increment,
`user` int(11) NOT NULL,
`description` text NOT NULL,
`time` timestamp NOT NULL default CURRENT_TIMESTAMP,
`ext` varchar(4) NOT NULL default 'png',
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=14 ;
-- --------------------------------------------------------
--
-- Table structure for table `screenshot_replies`
--
CREATE TABLE IF NOT EXISTS `screenshot_replies` (
`id` int(11) NOT NULL auto_increment,
`user` int(11) NOT NULL,
`parent` int(11) NOT NULL,
`time` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
`text` text NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=27 ;
Note: I realize this may be a duplicate of this question but I didn't find the solution there working for me.
Thanks in advance! :)
You should use UNION in this case:
(SELECT id, time, 'screenshots' as tableName FROM screenshots)
UNION
(SELECT id, time, 'replies' as tableName FROM screenshot_replies)
ORDER BY time ASC
You can get the tablename of a field by using the mysql_tablename function in php
You can indicate the tablename as a column in the result set
Whenever I come across a problem like this, where you're finding it impossible to do something because you can't built a query which will extract the data, I have to start wondering do you have the right data model?
Start with what data you want to extract, and then build a model which allows that, not the reverse, and you'll find it much easiler in the long term, and also probably faster performance and quite often better flexibility.
Looking at those tables, I'd have to ask why have two of them? What's wrong with:
CREATE TABLE IF NOT EXISTS `screenshots` (
`id` int(11) NOT NULL auto_increment,
`user` int(11) NOT NULL,
`description` text NOT NULL,
`time` timestamp NOT NULL default CURRENT_TIMESTAMP,
`ext` varchar(4) default 'png',
`parent` int(11),
`text` text,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
That would allow you to do what you want, easily telling if it's a screen shot (if ext is set), a reply (if parent is set) or ... since the model now allows it ... a screenshot which is a reply!!!