I have csv file which looks:
0,00018332,2016-08-29 00:00:00,2016-08-29 00:00:00,9999,Sale of Parts only,0,DMS Maritime Pty Ltd,xyz,,00000250,1971-01-01 00:00:00,1971-01-01 00:00:00,218335,Sale of parts only,9999
Then I have php script which is importing this csv into table
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "test.csv";
$dbtable = "res";
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$dbtable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator));
But when I check it back in db table, values 2016-08-29 00:00:00 (column is datetime) is 0000-00-00 00:00:00 , 218335 and 9999 is set to 0 event columns are set to int(11).Everything else is written correctly. I tried with csv and txt and I have same problem.
Do you know why?
EDIT:
CREATE TABLE `res` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` text,
`start` datetime DEFAULT NULL,
`end` datetime DEFAULT NULL,
`room_id` int(11) DEFAULT NULL,
`status` varchar(30) DEFAULT NULL,
`paid` int(11) DEFAULT NULL,
`customer` varchar(50) DEFAULT NULL,
`name_ship` varchar(50) DEFAULT NULL,
`equipment` text,
`port` varchar(30) DEFAULT NULL,
`ETA` datetime DEFAULT NULL,
`ETD` datetime DEFAULT NULL,
`service_id` int(11) NOT NULL,
`service_classification` varchar(30) DEFAULT NULL,
`partners_id` int(10) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=5392 DEFAULT CHARSET=latin1
Your file is encoded in utf16, mysql will not check that on its own. Either change that in your datasource (since your table encoding indicates that you don't want to use utf16 strings anyway) or inform mysql about the file's encoding.
For your file, the following should work:
LOAD DATA LOCAL INFILE 'test.csv' INTO TABLE res
CHARACTER SET utf16le
FIELDS TERMINATED BY ';\0'
LINES TERMINATED BY '\r\0\n\0';
It sets the characterset of the file and defines the delimiters in unicode. You can use just '\n\0' too.
Why not execute : show warnings in mysql cmd and see what's the exact error.
https://dev.mysql.com/doc/refman/5.5/en/show-warnings.html
Related
So i have a table that has multiple date time columns and i am trying to select certain records based on a certain date using
SELECT * FROM `posdata` WHERE `CommissionDate` >= '2019-01-01 00:00:00'
the table structure
CREATE TABLE `posdata` (
`ID` int(11) NOT NULL,
`DISTYNAME` varchar(30) DEFAULT NULL,
`ENDCUST` varchar(75) DEFAULT NULL,
`MFGCUST` varchar(50) DEFAULT NULL,
`EXTPRICE` double DEFAULT NULL,
`POSPERIOD` datetime DEFAULT NULL,
`PAYMENTDATE` date DEFAULT NULL,
`QTY` double DEFAULT NULL,
`UNITCOST` double DEFAULT NULL,
`UNITPRICE` double DEFAULT NULL,
`COMMISSION` double DEFAULT NULL,
`SALESORDER` varchar(40) DEFAULT NULL,
`PO` varchar(40) DEFAULT NULL,
`POLineItem` varchar(20) DEFAULT NULL,
`ENTRYDATE` datetime DEFAULT NULL,
`AdjustedCommission` int(11) DEFAULT NULL,
`CustomerPart-NO` varchar(50) DEFAULT NULL,
`CommissionDate` datetime DEFAULT NULL,
`EXTCOST` double DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
ALTER TABLE `posdata`
ADD PRIMARY KEY (`ID`),
ADD KEY `CommissionDate` (`CommissionDate`);
ALTER TABLE `posdata` ADD FULLTEXT KEY `posdata_endcustomer_index` (`ENDCUST`);
a very weird thing happens, it returns all the fields as required, but the CommissionDate column has only '2019-01-01 00:00:00' as the date. The actual CommissionDate column in the database has only '2016-01-01 00:00:00' as the data.
I am using phpmyadmin to to run this query and have used the search filter on that and it always gives me the same result whether i run it thorough a php script or phpmyadmin. What am i doing wrong ?
In your query SELECT * FROM 'posdata' means get all the fields from the table post data and then the next WHERE clause applies.If you only want to get the data from CommissionDate column
Change Your Query to
SELECT `CommissionDate` FROM `posdata`
In the way you get the desired data.
the query was correct can some one delete this question !
So, I have an issue with loading my CSV into SQL database with PHP and Wordpress. Both are running on local xammp atm.
This is my code for loading the CSV file. It first uploads it via html form to some temp folder (data) and then tries with LOAD DATA to copy the content into the table (wp_data)
<?php
if (isset($_POST['submit'])) {
global $wordpress,$wpdb;
$file = $_FILES['fileToUpload']['tmp_name'];
$target_dir = $_SERVER['DOCUMENT_ROOT']."/data";
$target_file = $target_dir . '/' . basename($_FILES["fileToUpload"]["name"]);
move_uploaded_file($file, $target_file);
$sql="
LOAD DATA INFILE '$target_file' INTO TABLE wp_data FIELDS TERMINATED BY ';' IGNORE 1 ROWS";
$query = $wpdb->query($sql);
}
?>
The uploading part works fine, however when it tries to fill the table with the data, it just fills "0s"/NULL values, like this:
The file as text looks like this:
And visually, it looks like this:
Table structure from SHOW CREATE TABLE wp_data:
CREATE TABLE `wp_data` (
`Unix_time` int(11) NOT NULL,
`Message` text NOT NULL,
`Time` float NOT NULL,
`DF` int(11) NOT NULL,
`Type_code/BDS_number` int(11) NOT NULL,
`Latitude` double NOT NULL,
`Longitude` double NOT NULL,
`Altitude` int(11) NOT NULL,
`Ground_speed` double NOT NULL,
`Baro_diff` int(11) NOT NULL,
`Heading` int(11) NOT NULL,
`TAS` int(11) NOT NULL,
`Mach` float NOT NULL,
`FOM_SOURCE` varchar(40) DEFAULT NULL,
`Wind_speed` varchar(40) DEFAULT NULL,
`Wind_direction` varchar(40) DEFAULT NULL,
`Temperture` text,
`Pressure` varchar(40) DEFAULT NULL,
`Turbulence` varchar(40) DEFAULT NULL,
`Humidity` varchar(40) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
What am I doing wrong here? I am quite new at PHP and got stuck on this
:(...
Thx.
Fixed it by setting the encoding in query to CHARACTER SET UTF8 like this:
LOAD DATA INFILE '$target_file'
INTO TABLE wp_data
CHARACTER SET UTF8
FIELDS TERMINATED BY ';'
IGNORE 1 ROWS;
i'm trying to insert datas in a table of 8 columns but it's doesn't work . i'm using mysql version 5.7.11 . however it works when i reduce it to 6 columns 7 columns refuse to work as well .
please find the tables and their php code below
6 columns mysql/php
1.1 php
$res=$pdo->prepare('INSERT INTO '.$table.'(title,description,image,city,price) VALUES(:titl,:decri,:img,:cty,:prce)');
$res->bindParam(':titl',$title);
$res->bindParam(':decri',$decri);
$res->bindParam(':img',$item_img);
$res->bindParam(':cty',$city);
$res->bindParam(':prce',$price);
1.2 mysql
CREATE TABLE `mobiles`
(`id` int(11) NOT NULL,
`title` varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`description` varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`image` longblob NOT NULL,
`city` varchar(20) NOT NULL,
`price` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
8 columns tables
2.1 mysql code
CREATE TABLE `mobiles` (
`id` int(11) NOT NULL,
`title` varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`description` varchar(100) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`image` longblob NOT NULL,
`city` varchar(20) NOT NULL,
`price` int(11) NOT NULL,
`user_id` int(10) NOT NULL,
`date` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
2.2 php code
$res=$pdo->prepare('INSERT INTO'.$table.'(title,description,image,city,price,user_id,date) VALUES(:titl,:decri,:img,:cty,:prce,:user_id,:dte)');
$res->bindParam(':titl',$title);
$res->bindParam(':decri',$decri);
$res->bindParam(':img',$item_img);
$res->bindParam(':cty',$city);
$res->bindParam(':prce',$price);
$res->bindParam(':user_id',$user_id);
$res->bindParam(':dte',$date);
i have executed right after , the six columns works but the 8 columns one
why is that more than 6 columns mysql can't insert while the number limit of columns can reach 255.
is there any issue in my code ? i echoed all the entries and they all work fine.
any help please ?
You have no space between 'INSERT INTO' and $table in second query, so query fails because of syntax error.
I have the following function:
function bulk_insert_file($filename) {
$file_location = 'assets/temp/'.$filename;
$sql = 'LOAD DATA LOCAL INFILE '."'$file_location'".' INTO TABLE p4p.users_csv_import
FIELDS TERMINATED BY \',\'
LINES TERMINATED BY \'\\r\\n\'
IGNORE 1 LINES';
$this->db->query($sql);
}
And the following CSV file:
,2,unique_id,first_name,last_name,email,company,nonprofit,username,password,dimension_data,raw_csv_data,
,2,unique_id,first_name,last_name,email,company,nonprofit,username,password,dimension_data,raw_csv_data,
,2,unique_id,first_name,last_name,email,company,nonprofit,username,password,dimension_data,raw_csv_data,
However, when I run the code no data is imported. If I remove the IGNORE 1 LINES part I will at least get the first row imported.
CREATE TABLE `users_csv_import` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`role_id` int(11) NOT NULL DEFAULT '1',
`unique_id` varchar(255) DEFAULT '',
`first_name` varchar(50) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`last_name` varchar(50) CHARACTER SET latin1 NOT NULL,
`email` varchar(100) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL,
`company` int(11) NOT NULL DEFAULT '0',
`nonprofit` int(11) NOT NULL,
`username` varchar(100) CHARACTER SET latin1 NOT NULL DEFAULT '',
`password` varchar(34) CHARACTER SET latin1 NOT NULL DEFAULT '',
`dimension_data` text,
`raw_csv_data` text,
PRIMARY KEY (`id`)
)ENGINE=InnoDB DEFAULT CHARSET=utf8;
Here is the specs on the table..
I assume the csv file is in unix line terminators format.
You should therefore try importing with LINES TERMINATED BY \'\\n\'.
It might sound a bit confusing, and perhaps what I'm doing isn't right at all, but here goes.
Our users can upload .csv files containing their school's data, exported from an external management system called SIMS. Once they upload that file I want to run a .sql script that updates our database with the information they provide in this file. What I've done, on the upload file page (works like a charm), is create a session variable containing the file path and name of the uploaded file, as shown below (file path is blanked out):
$target_path = "xxx";
$target_path = $target_path . basename( $_FILES['uploadedfile']['name']);
if(move_uploaded_file($_FILES['uploadedfile']['tmp_name'], $target_path))
{
echo "<p>The file ". basename( $_FILES['uploadedfile']['name']). " has been uploaded.</p>" ;
echo "<p><a href='sqltest.php'>Click here</a> to upload your files into the database.</p>" ;
$_SESSION['file'] = $target_path ;
}
Below is a portion of my .sql file operated within PHP containing the session variable. Because most of this populate script uses the same kind of commands throughout it will probably not make sense to show you all 973 lines of code (of course there is a session_start() active at the top of the page).
$filename = $_SESSION['file'] ;
mysqli_query($dbc, 'SET foreign_key_checks = 0') ;
$populate =
"CREATE TEMPORARY TABLE IF NOT EXISTS `mldb`.`TempSchool`
(
`CentreNo` INT UNSIGNED NOT NULL,
`School` VARCHAR(255) NULL,
`Street` VARCHAR(255) NULL,
`Town` VARCHAR(255) NULL,
`County` VARCHAR(255) NULL,
`Postcode` VARCHAR(10) NULL,
`Tel` VARCHAR(45) NULL,
`URL` VARCHAR(512) NULL,
`Email` VARCHAR(255) NULL,
`Headteacher` VARCHAR(255) NULL,
`LEA` VARCHAR(45) NULL,
`LEANo` INT UNSIGNED NULL,
`EstablishmentNo` INT UNSIGNED NULL,
`URN` INT UNSIGNED NULL,
`Governance` VARCHAR(45) NULL,
`Phase` VARCHAR(45) NULL,
PRIMARY KEY (`CentreNo`)
)
ENGINE = InnoDB ;
LOAD DATA INFILE '$filename'
IGNORE INTO TABLE `TempSchool`
FIELDS TERMINATED BY ' , '
OPTIONALLY ENCLOSED BY ' \" '
LINES TERMINATED BY ' \r\n '
IGNORE 1 LINES (etc...)
I'm getting an error on line 25 of the code (LOAD DATA INFILE '$filename') displaying the following:
Invalid query:You have an error in your SQL syntax; check the manual that corresponds to your MySQL
server version for the right syntax to use near 'LOAD DATA INFILE
'C:/Users/Public/Dropbox/mlwebfiles/Trial/uploads/MarksLive Set' at line 25
Presumably it's got something to do with the fact that, even though it's recognizing the file path and file name stored in the $_SESSION['file'] variable, but it's not actually doing anything with it. Is there a way where this .sql file can operate with through PHP set up variables for the file names? Many thanks!
After searching long and hard, and far and wide we've found a (what we think is the only) solution: create a function to loop through every one of these 'mini-queries' individually and it seems to be a lot more happy now! We actually came across this suggestion before but we rejected it initially because it would (and did) take a lot of time to separate 973 lines of code to split all the individual database additions...
$populate =
"
CREATE TEMPORARY TABLE IF NOT EXISTS `mldb`.`TempSchool`
(
`CentreNo` INT UNSIGNED NOT NULL,
`School` VARCHAR(255) NULL,
`Street` VARCHAR(255) NULL,
`Town` VARCHAR(255) NULL,
`County` VARCHAR(255) NULL,
`Postcode` VARCHAR(10) NULL,
`Tel` VARCHAR(45) NULL,
`URL` VARCHAR(512) NULL,
`Email` VARCHAR(255) NULL,
`Headteacher` VARCHAR(255) NULL,
`LEA` VARCHAR(45) NULL,
`LEANo` INT UNSIGNED NULL,
`EstablishmentNo` INT UNSIGNED NULL,
`URN` INT UNSIGNED NULL,
`Governance` VARCHAR(45) NULL,
`Phase` VARCHAR(45) NULL,
PRIMARY KEY (`CentreNo`)
)
ENGINE = InnoDB ;
" ;
populate ($dbc, $populate);
$populate =
"
LOAD DATA INFILE '$path'
IGNORE INTO TABLE `mldb`.`TempSchool`
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\\r\\n'
IGNORE 1 LINES
(#AdNo, #UPN, #ULN, #UCI, #CandidateNo, #LegalSurname, #LegalForename,
#PreferredSurname, #PreferredForename, #Gender, #DOB, #Email,
#InCare, #EverInCare, #FSM, #FSMEver6, #EAL, #SENStatus, #AMA,
#Attendance, #RegGroup, #YearGroup, #EnteredYearDate,
#Class, #Subject, #Staff, #Initials,
CentreNo, School, Street, Town, County, Postcode, Tel, URL,
Email, Headteacher, LEA, LEANo, EstablishmentNo, Governance, Phase)
" ;
populate ($dbc, $populate);