I have been looking for this problem since a while so I guess I stuck.
Case is: I'm importing a large csv file into mysql database with Bigdump (thanks to A. Ozerov), delimiter is |, all is fine, this works.
In my last varchar field the values are usually: '', 'X', '1', '0'. Problem is if I do a select query to this field and look up values with '', I find none or one only. I checked a few lines one by one and found that the phpmyadmin shows a "textarea" instead of a textbox. I went and selected the field data and nothing was there (not even a carriage return). Still I could save the line and reopen it, when the textarea changed to textbox.
I wonder if anyone ever saw this error, and if there is a way to fix this database runtime from php (part of daily process), not line-by-line.
I even tried to copy paste both the csv and the database field to text editor to see if a char is there that I don't see.
The data I want to use is:
0 if the value is empty, 1 if the value is anything else. Currently I am not able to separate this two for the above reason.
Thank you in advance!
Check for hexcidecimal values in the column by doing this:
SELECT HEX(columnname)
Then use REPLACE to change the Hex values to Null or ''
Related
I have a PHP web application developed with CakePHP 4 an MySQL.
One of the databases the application works with is a table named trg_registros_vip that contains 77 columns. One of the columns is named trg_tipo_proyecto and is defined as a non-nullable varchar with a maximum length of 100 characters.
The table is populated by reading an excel file with SimpleXLSX (a library obtainable as shuchkin/simplexlsx by using composer). For each row in the file, starting from the second row (as the first one has the headers for the sheet), the data on each cell from left to right, starting from the first cell, is stored in an entity class named TRegistrosVip in the same order as defined for each column in the table starting from the second (since the first column is an autoincremental primary key). Then the entity is inserted in the table. The column trg_tipo_proyecto is the fifth one in the table and the data for this column is obtained from fourth column in the excel file.
However, when I try to read the excel from the application, I get the following error message: Error: [PDOException] SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'trg_tipo_proyecto' at row 1 in C:\xampp\htdocs\CPSC2\vendor\cakephp\cakephp\src\Database\Statement\MysqlStatement.php
The stacktrace for this error indicates that the error is generated in the very moment I try to insert one row in the table.
I searched in the excel file if there is a cell in the fourth column whose length is greater than 100 and got none.
I searched the table class associated to the entity class, named TRegistrosVipTable in case of a bad defined column and I got this:
$validator->scalar('trg_tipo_proyecto')
->maxLength('trg_tipo_proyecto', 100)
->requirePresence('trg_tipo_proyecto', 'create')
->notEmptyString('trg_tipo_proyecto');
That means the column is well defined.
I check the config/app.php file in case I would be working with a wrong database but I'm using the correct one.
Also, due to possible presence of foreign accents, I called the utf8_decode function on each value of that column in the excel file before inserting it on the table.
Considering the described above, what would be the cause of this error?
Thanks in advance.
SOLVED
I implemented logging in the application and printed in the debug log file the value in the excel file's fourth column for each row after the call to the utf8_decode function. The error is triggered when trying to insert the word "Regularización" (without the double quotes, without leading or trending spaces and with an accent).
I removed the accent from that word in the Excel and the error dissappeared.
However, I cannot remove the accents from the Excel programmatically because client's restrinctions forbid it. So, I tested removing the call to utf8_decode and inserting the value directly without any codification or decodification. The error is not triggered anymore.
Thanks to all who have read and answered this question.
I have some php scripts that read data from csv files and insert them to a mysql database. When I look at the table rows in phpmyadmin their data seems fine but when i open one of the rows for editing, the varchar fields become full of question marks. Also when I search the tables for any string the result is empty. the collation of the fields is utf8_general_ci....
Even a word like KAMZA displays like ��K�A�M�Z�A��
Whats the problem?
This is my first post, and there is a similar post here: phpMyAdmin doesn't show added columns - Stack Overflow
But since no one has answered, I will ask here and provide more details in hoping to resolve this issue. I have a table in my database that I needed to be able to add additional columns to. I've done this using the following query:
$sql = "ALTER TABLE tablename ADD columnname INT(11)"; //Run the query....
This worked perfect on day one, however on day two when I tried to add an additional column via this php script, it appears to work from the phpMyAdmin Structure view. The column is in there with the correct columnname and datatype. However when I switch to the Browse view, there is no Column Name, just blank columns filled with "null" values. But, if you click on an individual row, it shows the correct column name, and a value (if one exists for that row).
I've tried running Analyze Table, as I read somewhere that that would update the Schema. However, I haven't had any success with that fixing it. I'd prefer to not have to delete the table and restart, especially if I run into this issue again. As this is my first post, I've tried to format everything correctly, but please forgive me if I didn't. Also, I can grab screenshots if anyone is having issues understanding my question.
if you can, restart mysql, it should flush everything, including mysql caches phpmyadmin may be using.
I used this example : http://woork.blogspot.com/2007/10/insert-record-into-database-using-ajax.html
it is working when using with Numbers for example if i enter number in these textbox it is work but when i enter string i get this error: "Unknown column 'text' in 'field list'"
'text' is text that write in the textbox instead of 'text' i write '123' it is working
Could you please suggest me way urgent.
Thanks all,
This error means that in your table there is no column named text. To fix this, either rename the field you wish to update/insert to text, or figure out which column you really wish to update/insert the text into.
Since your field is going to be updating/inserting a string, you need to also make sure you wrap the string in quotes.
INSERT INTO `table` (`val1`,`val2`) VALUES('".$val1."','".$val2."')
Make sure your column has a text or varchar datatype. Theoretically it should throw a different error if that was the case. But the fact that you're able to insert numbers and not text makes me think you've got some sort of INT datatype.
Obviously most importantly make sure your hello table really exists.
the data base data is stored as the image shows. the table is products.
the data in excel is as the following shows.
but when i open it by notepad++. it shows like this:
1, the data's format which i prepared is right? if not? how to correct it?
supposed the csv filed named product.csv. when i using LOAD DATA INFILE .when should i put the csv file into? could i load it from my local computer to my site's server?
i want to update the price, if the price is null, then using 0 to fill. this is my command. but it doesn't work
LOAD DATA LOCAL INFILE 'D:\\product.csv' REPLACE INTO TABLE products
FIELDS TERMINATED BY ' '
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(products_model, #var1)
SET products_price = IF(#var1 = '', 0, #var1); -- Replace empty string with 0
i put the above command in phpmyadmin, it can run, but the data doesn't be updated. what's wrong with it?
ps: is there a way to use PHP to get this? if have, how to do it?
I haven't used the setup you're working with, but it looks like the code is trying to read a tab-delimited file. The file is comma-delimited.
So, maybe all you need to do is change:
FIELDS TERMINATED BY ' '
to
FIELDS TERMINATED BY ','
Probably this issue is solved already, but just to give some extra input (also for others who visit this page):
Did you check your excel sheet? It seems that you have your data (price) in a different column than your sql database.
In excel the price is located in sixth column, in your sql it is located in the THIRD column... you can see that in your notepad csv file. Each comma represent a column.