I have a csv in .txt who need to be imported to MySql
I'm importing it manualy to my table but it keep breaking giving me this error :
Invalid column count in CSV input on line 706.
checking it on excel the only information who could make the problem is that one of the cells contains : 5.5 x 18"
probably the quotation mark is doing it
how can i fix this for the import ?
thank you
I believe all you need is this:
str_replace('"', '\"', $value);
By replacing the " with \" you are escaping it and it should insert properly.
Related
We are writing some php to move data from a Microsoft SQL database to a MySQL database. The Microsoft data contains user input text, with line breaks and all sorts of wacky stuff
The process we've been working with is as follows:
select from a Microsoft SQL database into php array (works fine)
processes this into a .csv by looping the fputcsv() function:
fputcsv($delimiter = ',', $enclosure = '"', $escape_char = "\\");
(the .csv file looks good when I open it with google sheets)
upload the .csv file to the MySQL server with the following code:
LOAD DATA LOCAL INFILE 'c:/filename.csv'
INTO TABLE tablename
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS; ```
The problem is that when step 3 is executed, the resulting MySQL table is all jumbled up because it appears to get confused with the line breaks in the data, thinking it's a new line in the table.
I noticed the MySQL LOAD DATA command doesn't take the escape character as an argument, but I can't seem to find a way to give it to it. Is this the issue? How can we make this work?
i am importing a CSV file into mysql database. One whole column (Project_ID_) in the excel sheet has leading space in it and after importing, the space is converted into garbage value in mysql table.
For example:
Value in Excel is : 000000000015382
Value in Mysql becomes: � 000000000015382
how do i trim the space
code for importing is given below:
$import="INSERT INTO streammapping(Project_ID_Numeric
,Project_ID_
,Project_Description_
,Action
,Transition_Steady_State
,Stream
,Stream_Lead
,Fixed_Variable
,Domain
,Pillar
,Account_Plan_Pilar
,Account_Plan_Project_Type)
VALUES('".mysql_real_escape_string($data[0])."','".mysql_real_escape_string($daa[1])."','".mysql_real_escape_string($data[2])."','".mysql_real_escape_string($data[3])."','".mysql_real_escape_string($data[4])."','".mysql_real_escape_string($data[5])."','".mysql_real_escape_string($data[6])."','".mysql_real_escape_string($data[7])."','".mysql_real_escape_string($data[8])."','".mysql_real_escape_string($data[9])."','".mysql_real_escape_string($data[10])."','".mysql_real_escape_string($data[11])."')";
mysql_query($import) or die(mysql_error());
Try this
From:
mysql_real_escape_string($data[0])
To:
mysql_real_escape_string(trim($data[0]))
It's new for me to use richtext but a user has to be able to insert pictures or to change his text. So I'm using ckeditor but i've some trouble with it.
When a user write into the rich text he can write things using apostrophe and comas.
I'll use an example. Let's imagine i want to write :
It's quite difficult to open a picture, a file blablabla
The problem is that in the inserting query looks like that
Insert into tab (txt1,txt2) values ('value1','value2')
If the user uses comas or apostrophe the richtext cannot be correctly inserted. Moreover, I use longtext in my MySQL database to store the text.
My questions are :
How to store the richtext written by the user
What type should be the column in my MySQL database to correctly store the richtext to be able to give it back
The problem i have is
Erreur SQL !INSERT INTO detail_article(ID,Problem,Num,URL,Solving,Description) VALUES(16,'Chrome : clearing datas',4,'','
\r\nCHROME CACHE CLEARING :
\r\n\r\n
blablabala
\r\n\r\n
In the 'windows explorer', do the fallowing statment....blablabla
\r\n','')
It's a sample of my text but i think the problem is caused by the quotes and the coma
Your problem must come from your PHP script which seems not to escape the pieces of data it send.
If it's not working with the richtext containing commas and apostrophes, then it's probably that the query send to your database isn't escape.
For example, if you have such content :
$value2 = "Catch',em all";
$value1 = "Hello boys";
And if you try to store it in database without escaping it, then the resulting SQL query will looks like :
INSERT INTO tab (value1, value2) VALUES ('Hello boys', 'Catch', em all);
which will be an incorrect SQL statement because of incorrect syntax.
If you use some escape method like PDO::quote(), your input will be inserted in a safe way that won't produce any error (syntax).
Make sure your script does escape the params you send with either PDO::quote() (https://php.net/manual/fr/pdo.quote.php) or either mysqli_real_escape_string (http://php.net/manual/fr/mysqli.real-escape-string.php)
I am writing the following script up import a csv file into a mysql database.
The user has to log in first and their security rights to upload are checked by another file, if this evaluates to true, the file can then be uploaded by the file up-loader which will only allow csv's to be uploaded and then the following part imports the file.
Am I using the correct code for this below ?, also if the csv layout is wrong is it possible to refuse the import ?, im just concerned that this could go badly wrong if the csv is formatted correctly. This feature has been requested as a requirement for this project so I am just trying to make it as idiot proof as possible for them.
<?php
$uploadedcsv = './uploads/'.$filename.'';
$sql = 'LOAD DATA LOCAL INFILE "'.$uploadedcsv.'" INTO TABLE '.$table.' FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY """" IGNORE 1 LINES' or die(mysql_error());
?>
Unless you are 100% rely on the user and it seems that you are not,
it will be good not to blindly import uploaded file, but first to check if it's correct.
To do it, you'll need to open and to read the file, e.g. with fgetcsv and to check the data consistency line-by-line.
You can find a lot of examples on the web.
Here are just some:
http://www.damnsemicolon.com/php/php-upload-csv-mysql
http://www.programmingfacts.com/import-csvexcel-data-mysql-database/
It can be done via MySQL, but by default LOAD DATA INFILE expects a different format than CSV. From the documentation:
If you specify no FIELDS or LINES clause, the defaults are the same as if you had written this:
FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\'
LINES TERMINATED BY '\n' STARTING BY ''
The documentation notes, for when dealing with CSV files:
LOAD DATA INFILE 'data.txt' INTO TABLE tbl_name
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES;
Which is an example of dealing with comma separated values enclosed by "" and line-separated by \r\n
To use this to import a file, make sure your statement matches the CSV format of your upload files and you can import via this manner.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
PHP code to convert a MySQL query to CSV
Hi I am generating a csv file from mysql record seperated by "," comma. I fetch data from database and concate it in a string. I am using following code
for($i=0;$i<count($all_logs);$i++)
{
$rd=$project->fetchRow($project->select()->where('id='.$all_logs[$i]->user2project_id));
$username=$userid->fetchRow($userid->select()->where('id='.$all_logs[$i]->user_id));
$outstr .=$all_logs[$i]->log_date.",";
$outstr .=$username->username.",";
$outstr .=$rd->title.",";
$outstr .=$all_logs[$i]->task.",";
$outstr .=$all_logs[$i]->workdesc.",";
$outstr .=$all_logs[$i]->hours.",";
$outstr .="\n";
}
return $outstr;
Now what is my problem. If any column data is itself is having a " ," comma then it splits this 1 column to 2 columns. For example if my column workdesc is having data like this I worked on this,that,these and those then it will put this 1 column to 3 columns in generted csv. Any body can tell me how can I escape from this situation......
In order to construct a CSV file, you should not use strings concatenations.
Instead, you should use the fputcsv() function -- and if you do not want to scrite to a file, but get a string as a result, take a look at the users notes, in which you'll find several possible solutions ;-)
Using that function, you will not have to deal with escaping the enclosure nor delimiter yourself : all this will already be done for you.
You have to put values in quotes something like this
$outstr .='"'.$username->username.'",'; //output ://"somvalue contain , comma"
Apply this format on all the data columns and that's it.
If the column data itself has comma, then you should use double quotes to escape it.
example if your columndata = "some,data"; then your csv should look like -
col1,col2,"some,data",col4
So better have double quotes enclosed for your column data.