Importing CSV into MYSQL using PHP not working - php
I am trying to import a CSV file into MySQL I keep getting an error on Line 4 I know this has to be a simple problem but I have never done this before...
Here is the code I have.
<?php
$file = fopen("http://*******.com/*****/********.csv", "r");
$servername = "*******";
$username = "*************";
$password = "**************";
$dbname = "*********";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error){
die("Connection failed: " . $conn->connect_error);
}
while (($line = fgetcsv($file, 0, ",")) !== FALSE){
echo $sql = "INSERT INTO inventory_two (
`Add_City_State_Zip`,
`CommentsInstalledOptions`,
`TodaysDate`,
`total_Acquired_Service`,
`total_Acquired_Service_PakFee`,
`total_Service`,
`year_make_model_trim`,
`AcquiredDate`,
`AcquiredPrice`,
`ACV`,
`AdditionalNotes`,
`AltVehicleLocation`,
`ASIS`,
`AskDown`,
`Askprice`,
`AskPrice_Low`,
`AskTerm`,
`AutoBoingURL`,
`BodyStyle`,
`BodyStyle_Ebay`,
`ClearTitle`,
`CommentOptionsFuelStereo`,
`Comments`,
`Condition_ID`,
`ConditionDesc`,
`DaysInInventory`,
`DriveType`,
`Engine`,
`ESN`,
`ExtColor`,
`ExtTrim`,
`Flags`,
`FuelType`,
`Images`,
`Images_Primary`,
`ImageUpdate_Epoc`,
`Inspected`,
`Inst_Add1`,
`Inst_City`,
`Inst_DBA`,
`Inst_Email`,
`Inst_ID`,
`Inst_IDLot_ID`,
`Inst_Name`,
`Inst_Phone1`,
`Inst_State`,
`Inst_Website`,
`Inst_Zip`,
`InstalledOptions`,
`IntColor`,
`Inventory_ID`,
`InvType_ID`,
`InvTypeDesc`,
`Lot_Add1`,
`Lot_Add2`,
`Lot_City`,
`Lot_Email`,
`Lot_ID`,
`Lot_Phone1`,
`Lot_Phone2`,
`Lot_State`,
`Lot_Zip`,
`LotLegalName`,
`LotLocation`,
`Make`,
`Mileage`,
`MileageStatus_ID`,
`Model`,
`NewFlag`,
`PakFee`,
`Provider_ID`,
`ProviderCode`,
`SellerNotes`,
`Status`,
`Stereo`,
`StockNumber`,
`TitleLocation`,
`TitleStatus_ID`,
`TotalExpenses`,
`Transmission`,
`Transmission_Common`,
`UserDefined1`,
`VIN`,
`WarrantyTerms`,
`Weight`,
`WholeSalePrice`,
`Year`
)
VALUES ($line[0]','$line[1]','$line[2]','$line[3]','$line[4]','$line[5]','$line[6]','$line[7]','$line[8]','$line[9]','$line[10]','$line[11]','$line[12]','$line[13]','$line[14]','$line[15]','$line[16]','$line[17]','$line[18]','$line[19]','$line[20]','$line[21]','$line[22]','$line[23]','$line[24]','$line[25]','$line[26]','$line[27]','$line[28]','$line[29]','$line[30]','$line[31]','$line[32]','$line[33]','$line[34]','$line[35]','$line[36]','$line[37]','$line[38]','$line[39]','$line[40]','$line[41]','$line[42]','$line[43]','$line[44]','$line[45]','$line[46]','$line[47]','$line[48]','$line[49]','$line[50]','$line[51]','$line[52]','$line[53]','$line[54]','$line[55]','$line[56]','$line[57]','$line[58]','$line[59]','$line[60]','$line[61]','$line[62]','$line[63]','$line[64]','$line[65]','$line[66]','$line[67]','$line[68]','$line[69]','$line[70]','$line[71]','$line[72]','$line[73]','$line[74]','$line[75]','$line[76]','$line[77]','$line[78]','$line[79]','$line[80]','$line[81]','$line[82]','$line[83]','$line[84]','$line[85]','$line[86])";
if ($conn->query($sql) === TRUE){
echo "New record created successfully";
}
else{
echo "Error: " . $sql . "<br />" . $conn->error;
}
}
fclose($file);
?>
If I Echo the $line[] items in the while loop everything echos right. So I don't understand why this does not work?
Related
MySQL insert inside foreach only being called once
I have an array $alert_note. I iterate through a loop and fill it up with a few strings: $n = 0; $alert_note = array(); $results = array(); while($row = mysqli_fetch_assoc($query)){ //some code in here populates the $results[$n] array with results from $row $thisnote = "<b>Location alert</b><br> Alert ID: {$results[$n]['alert-id']}<br> Start: {$results[$n]['start-formatted']}<br> End: {$results[$n]['end-formatted']}<br> Radius: {$results[$n]['radius-km']} km<br> Distance: {$results[$n]['distance-km']} km<br> <ul>\n"; //$results[$n]['data'] is a nested array, so iterate through it: foreach($results[$n]['data'] as $name => $data){ $thisnote .= "<li>$name: $data</li>\n"; } $thisnote .= "</ul>"; $alert_note[$n] = $thisnote; $n++; } Then I call a foreach function: foreach($alert_note as $alert_note_contents){ error_log("Note: $alert_note_contents"); mysqli_query($dblink, "INSERT INTO `incident_events` (`incident`, `data`, `time`, `operator`) VALUES ('$incident', '$alert_note_contents', '$now', '$operator')"); } Each string in $alert_note shows up as expected in the PHP error log, but only the last one is inserted into the MySQL table. No PHP errors are being thrown. Any ideas why this may be?
plz before foreach($alert_note as $alert_note_contents){ do this so i can get the error var_dump($alert_note ); and do this plz so i can see if there is any error ind db $conn = new mysqli($servername, $username, $password, $dbname); // Check connection if ($conn->connect_error) { die("Connection failed: " . $conn->connect_error); } $sql = "INSERT INTO incident_events (incident, data, time, operator) VALUES ('$incident', '$alert_note_contents', '$now', '$operator')"; if ($conn->query($sql) === TRUE) { echo "New record created successfully"; } else { echo "Error: " . $sql . "<br>" . $conn->error; } $conn->close();
Codeigniter XML Datafeed to Multiple (MYSQL Tables)
goodness.. it's been a quite a headache for me. I'm trying to get variable SQL1, SQL2 and SQL3 to update different mysql tables.. but how on earth will SQL2 and SQL3's "product_id" fields, correlate with SQL1? See 'id' in the insert sections in SQL2 and SQL3.. i'm not sure how to replace and grab this generated data into this sections without making more ugly code. Also, if any of you have any idea how to make this easier, please help a fellow out here.. this looks pretty dirty to me.. surely there are better ways :( // DB Settings $servername = "localhost"; $username = "user"; $password = "pass"; $dbname = "db"; // Create connection $conn = new mysqli($servername, $username, $password, $dbname); // Check connection if ($conn->connect_error) { die("Connection failed: " . $conn->connect_error); } // Load data from XML file $xml = simplexml_load_file("Datafeed.xml"); // Capture XML Data foreach($xml->children() as $product) { // Change stock wording $stock = $product->stock; if ($stock == 'yes') { $stock = 'on'; } else { $stock = 'off'; } // Convert XML data and insert into MYSQL $sql = "INSERT INTO testshop_products (product_id, product_name, product_type, product_price, product_status) VALUES ("echo $product->code;", "echo $product->cat;", "echo $product->price;","echo $stock;")"; $sql2 = "INSERT INTO testshop_product_details (product_id, product_color, product_image, details_status) VALUES ('id', "Generating...", "echo $product->img;","echo $stock;")"; $sql3 = "INSERT INTO testshop_spesifications (product_id, specs_meta, specs_details, specs_slug, specs_status) VALUES ('id', General, "echo $product->img;", general, "echo $stock;")"; if ($conn->query($sql) === TRUE) { echo "New record created successfully"; } else { echo "Error: " . $sql . "<br>" . $conn->error; } $conn->close(); if ($conn->query($sql2) === TRUE) { echo "New record created successfully"; } else { echo "Error: " . $sql . "<br>" . $conn->error; } $conn->close(); if ($conn->query($sql3) === TRUE) { echo "New record created successfully"; } else { echo "Error: " . $sql . "<br>" . $conn->error; } $conn->close(); }
write store procedure and inset all records and all table it will find. store procedure like this CREATE PROCEDURE `addJobApplicant`(IN XMLINPUT text, OUT vresult VARCHAR(100)) BEGIN DECLARE iCounter INT DEFAULT 1; DECLARE maxCount INT; SET vcompID=ExtractValue(XMLINPUT, '/ROOT/HEADER/COMPANYID[$iCounter]'); SET vjpID=ExtractValue(XMLINPUT, '/ROOT/HEADER/JOBID[$iCounter]'); insert command here SET vresult = 'Successfully Inserted'; end
PHP > Invalid Argument supplied for foreach()
In short, I am trying to figure out what is wrong with my foreach statement. I have been trying to work on finding the error for over a day know and I'm running out of time. This program is supposed to parse a json array and post it up to a mysqli database. <?php error_reporting(E_ALL); ini_set('display_errors', 1); $a = print_r(var_dump($GLOBALS),1); echo htmlspecialchars($a); $servername = "#"; $username = "#"; $password = "#"; $dbname = "#"; // Create connection $conn = mysqli_connect($servername, $username, $password, $dbname); echo "Connection Successful : "; // Check connection if (!$conn) { die("Connection failed: " . mysqli_connect_error()); } // Read JSON file $jsondata = file_get_contents('scripts/AUDIT_DIR/report.json'); echo "JSON File Read : "; // Convert and Loop $item = json_decode($jsondata, true); echo "JSON File Decoded : "; foreach($item as $arr) { $id = $arr["id"]; $hostname = $arr["hostname"]; $ip = $arr["ip"]; $package = $arr["package"]; $publisher = $arr["publisher"]; $origin = $arr["origin"]; $version = $arr["version"]; $size = $arr["size"]; $sql = "INSERT INTO testtable(id, hostname, ip, package, publisher, origin, version, size) VALUES ('10', '$hostname', '$ip', '$package', '$publisher', '$origin', '$version', '$size')"; if (mysqli_query($conn, $sql)) { echo "New record created successfully : "; } else { echo "Error: " . $sql . "<br>" . mysqli_error($conn); } } ?>
You likely have an invalid return from your json_decode() you can check this with a var_dump($item); after your json_decode() In php json_decode() will return NULL if the json cannot be decoded or if the encoded data is deeper than the recursion limit. http://php.net/manual/en/function.json-decode.php You need to properly guard for such a case that $item === null and not assume you will always get a valid return for your foreach() params. Example showing your error happens when $item = null https://3v4l.org/oNr8P
Parse Large XML File in PHP Efficiently to Generate SQL
I am trying to parsing a large XML file and load it into MySQL. I have used simplexml to parse it, and it works perfectly, but its way to slow for this large XML file. Now i am trying to use XMLReader. Here is the sample of the XML: <?xml version="1.0" encoding="UTF-8"?> <drug type="biotech" created="2005-06-13" updated="2015-02-23"> <drugbank-id primary="true">DB00001</drugbank-id> <drugbank-id>BIOD00024</drugbank-id> <drugbank-id>BTD00024</drugbank-id> <name>Lepirudin</name> <description>Lepirudin is identical </description> <cas-number>120993-53-5</cas-number> <groups> <group>approved</group> </groups> <pathways> <pathway> <smpdb-id>SMP00278</smpdb-id> <name>Lepirudin Action Pathway</name> <drugs> <drug> <drugbank-id>DB00001</drugbank-id> <name>Lepirudin</name> </drug> <drug> <drugbank-id>DB01373</drugbank-id> <name>Calcium</name> </drug> </drugs> ... </drug> <drug type="biotech" created="2005-06-15" updated="2015-02-25"> ... </drug> Here is my approach using simplexml: <?php $xml = simplexml_load_file('drugbank.xml'); $servername = "localhost"; // Example : localhost $username = "root"; $password = "pass"; $dbname = "dbname"; // Create connection $conn = new mysqli($servername, $username, $password, $dbname); // Check connection if ($conn->connect_error) { die("Connection failed: " . $conn->connect_error); } $xmlObject_count = $xml->drug->count(); for ($i=0; $i < $xmlObject_count; $i++) { $name = $xml->drug[$i]->name; $description = $xml->drug[$i]->description; $casnumber = $xml->drug[$i]->{'cas-number'}; // ... $created = $xml->drug[$i]['created']; $updated = $xml->drug[$i]['updated']; $type = $xml->drug[$i]['type']; $sql = "INSERT INTO `drug` (name, description,cas_number,created,updated,type) VALUES ('$name', '$description','$casnumber','$created','$updated','$type')"; if ($conn->query($sql) === TRUE) { $last_id = $conn->insert_id; } else { echo "outer else Error: " . $sql . "<br>" . $conn->error. "<br>" ; } } $conn->close(); It works okay and it gives me 7,789 rows. But, I want to use XMLReader to parse this. But the problem with XMLReader I am finding it give more than 35,000 rows. If you look at the XML you can see that inside the <drug /> nodes there are also some other <drugs><drug> child nodes. How can I overcome this? Here is my procedure with XMLReader: <?php $servername = "localhost"; // Example : localhost $username = "root"; $password = "pass"; $dbname = "dbname"; // Create connection $conn = new mysqli($servername, $username, $password, $dbname); // Check connection if ($conn->connect_error) { die("Connection failed: " . $conn->connect_error); } $reader = new XMLReader(); $reader->open('drugbank.xml'); while ($reader->read()) { if ($reader->nodeType == XMLReader::ELEMENT && $reader->name == 'drug') { $doc = new DOMDocument('1.0', 'UTF-8'); $xml = simplexml_import_dom($doc->importNode($reader->expand(),true)); $name = $xml->name; $description = $xml->description; $casnumber = $xml->{'cas-number'}; // ... $sql = "INSERT INTO `drug` (name, description,cas_number,created,updated,type) VALUES ('$name', '$description','$casnumber','$created','$updated','$type')"; if ($conn->query($sql) === TRUE) { $last_id = $conn->insert_id; } else { echo "outer else Error: " . $sql . "<br>" . $conn->error. "<br>" ; } } } $conn->close(); With this example, I am finding it give more than 35,000 rows.
Alright, I have a working example for you with much improvement in execution speed, memory usage, and database load: <?php define('INSERT_BATCH_SIZE', 500); define('DRUG_XML_FILE', 'drugbank.xml'); $servername = "localhost"; // Example : localhost $username = "root"; $password = "pass"; $dbname = "dbname"; function parseXml($mysql) { $drugs = array(); $xmlReader = new XMLReader(); $xmlReader->open(DRUG_XML_FILE); // Move our pointer to the first <drug /> element. while ($xmlReader->read() && $xmlReader->name !== 'drug') ; $drugCount = 0; $totalDrugs = 0; // Iterate over the outer <drug /> elements. while ($xmlReader->name == 'drug') { // Convert the node into a SimpleXMLElement for ease of use. $item = new SimpleXMLElement($xmlReader->readOuterXML()); $name = $item->name; $description = $item->description; $casNumber = $item->{'cas-number'}; $created = $item['created']; $updated = $item['updated']; $type = $item['type']; $drugs[] = "('$name', '$description','$casNumber','$created','$updated','$type')"; $drugCount++; $totalDrugs++; // Once we've reached the desired batch size, insert the batch and reset the counter. if ($drugCount >= INSERT_BATCH_SIZE) { batchInsertDrugs($mysql, $drugs); $drugCount = 0; } // Go to next <drug />. $xmlReader->next('drug'); } $xmlReader->close(); // Insert the leftovers from the last batch. batchInsertDrugs($mysql, $drugs); echo "Inserted $totalDrugs total drugs."; } function batchInsertDrugs($mysql, &$drugs) { // Generate a batched INSERT statement. $statement = "INSERT INTO `drug` (name, description, cas_number, created, updated, type) VALUES"; $statement = $statement . ' ' . implode(",\n", $drugs); echo $statement, "\n"; // Run the batch INSERT. if ($mysql->query($statement)) { echo "Inserted " . count($drugs) . " drugs."; } else { echo "INSERT Error: " . $statement . "<br>" . $mysql->error. "<br>" ; } // Clear the buffer. $drugs = array(); } // Create MySQL connection. $mysql = new mysqli($servername, $username, $password, $dbname); if ($mysql->connect_error) { die("Connection failed: " . $mysql->connect_error); } parseXml($mysql); I tested this example using the same dataset. Using SimpleXML in the way that you are leads to parsing the entire document in memory, which is slow and memory-intensive. This approach uses XMLReader, which is a fast pull-parser. You can probably make this faster still using the PHP SAX XML Parser, but it's a bit more complex of a pattern, and the above example will be noticeably better than what you started with. The other significant change in my example is that we're using MySQL Batched Inserts, so we only actually hit the database every 500 (configurable) items we process. You can tweak this number for better performance. After a certain point, the query will become too large for MySQL to process, but you may be able to do a lot more than 500 at one time. If you'd like me to explain any part of this further, or if you have any problems with it, just let me know in the comments! :)
Warning: mysqli_query(): Couldn't fetch mysqli in my code
I know this question has been asked many times , I tried lot of other answers but it didn't work for me Here is my class <?php class saveexceltodb { var $inputFileName; var $tableName; var $conn; var $allDataInSheet; var $arrayCount; /** * Create a new PHPExcel with one Worksheet */ public function __construct($table=0) { $this->initiatedb(); } private function initiatedb(){ //var_dump($allDataInSheet); $servername = "localhost"; $username = "root"; $password = ""; $dbname = "xyx"; // Create connection $this->conn = mysqli_connect($servername, $username, $password, $dbname); // Check connection if (!$this->conn) { die("Connection failed: " . mysqli_connect_error()); } } public function updateIntermidiate(){ $allocationeventwise=array(); mysqli_query($this->conn,"DELETE FROM `allocationeventwise` WHERE 1"); $sql = "INSERT INTO `allocationeventwise` (`empid`, `event1`, `event2`, `event3`, `event4`, `event5` `event6`, `event7`) VALUES "; $result = mysqli_query($this->conn, "SELECT usdeal.empid , usdeal.event1 , usdeal.event2, usdeal.event3, usdeal.event4, usdeal.event5, usdeal.event6, usdeal.event7 , salary.salary from usdeal INNER JOIN salary ON usdeal.empid = salary.empid where usdeal.allocated=0"); $i=0; if (mysqli_num_rows($result) > 0) { // output data of each row while($row = mysqli_fetch_assoc($result)) { $i++; $totaleventDays = $row["event1"]+$row["event2"]+$row["event3"]+$row["event4"]+$row["event5"]+$row["event6"]+$row["event7"]; $allocationeventwise[$row["empid"]]['event1']=($row['salary']/$totaleventDays)*$row["event1"]; $allocationeventwise[$row["empid"]]['event2']=($row['salary']/$totaleventDays)*$row["event2"]; $allocationeventwise[$row["empid"]]['event3']=($row['salary']/$totaleventDays)*$row["event3"]; $allocationeventwise[$row["empid"]]['event4']=($row['salary']/$totaleventDays)*$row["event4"]; $allocationeventwise[$row["empid"]]['event5']=($row['salary']/$totaleventDays)*$row["event5"]; $allocationeventwise[$row["empid"]]['event6']=($row['salary']/$totaleventDays)*$row["event6"]; $allocationeventwise[$row["empid"]]['event7']=($row['salary']/$totaleventDays)*$row["event7"]; $sql .='("'.$row["empid"].'", '.$allocationeventwise[$row["empid"]]["event1"].', '.$allocationeventwise[$row["empid"]]["event2"].', '.$allocationeventwise[$row["empid"]]["event3"].', '.$allocationeventwise[$row["empid"]]["event4"].', '.$allocationeventwise[$row["empid"]]["event5"].', '.$allocationeventwise[$row["empid"]]["event6"].', '.$allocationeventwise[$row["empid"]]["event7"].',)'; if($i<mysqli_num_rows($result)) $sql .=","; else $sql .=";"; } echo $sql; if (mysqli_query($this->conn, $sql)) { echo "New record created successfully<br/>"; } else { echo "Error: " . $sql . "<br>" . mysqli_error($this->conn); } } else { echo "0 results"; } } } When I use it like this include('saveexceltodb.php'); $obj = new saveexceltodb(0); $obj->updateIntermidiate(); I get following error . Warning: mysqli_query(): Couldn't fetch mysqli in C:\xampp\htdocs\import-excel\saveexceltodb.php
You should follow below steps Check database connection Check query syntax You have to pass connection object in mysqli_query function in you case it is ' $this->conn ' Hope, It may help you.