I haven't found anything useful about my problem on google or on here.
Currently, I'm storing a CSV file from an URL to my server using cronjob and file_get_contents(); once per hour.
Right now there are several steps executed whenever a user loads the page, but it's all PHP and Javascript from there so it takes about 9 second to load the page with the end result (html table) as described here.
I want to change that by importing the CSV file stored on my server to a SQL database but I have no idea how to realise this. Tried converting it to an array with PHP and then storing it to a db like here, but it didn't work for me. How would you approach this problem?
Thanks, Innerwolf.
file looks like:
username,score,numWins,numKills,numKillsWallshot,numDeaths,numShots,highestSpree,timePlayed
/\ssa,14104,26,2113,0,867,28083,15,43695
"∀ЈAIIX",10166,18,2641,0,1291,34201,14,59346
i tried this new cron
<?php
function download_remote_file($file_url, $save_to)
{
$content = file_get_contents($file_url);
file_put_contents($save_to, $content);
}
download_remote_file(//link', realpath(".//path") . '/dump.csv');
if(!function_exists('str_getcsv')) { ------|
//define str_getcsv |until now, cron
} |job executes
} |part above.
$url = '//path/dump.csv'; |
$csvData = file_get_contents($url); |this part was in
$lines = explode(PHP_EOL, $csvData); |index.htm before
$array = array(); |and gets exe-
foreach ($lines as $line) { |cuted when
$line = str_replace("\\", "\", $line); |reload
$line = str_replace("#", "#", $line); |
$array[] = str_getcsv($line); |
} ------|
$fields = implode(', ', array_shift($array));
$values = array();
foreach ($array as $rowValues) {
foreach ($rowValues as $key => $rowValue) {
$rowValues[$key] = mysql_real_escape_string($rowValues[$key]);
}
$values[] = "(" . implode(', ', $rowValues) . ")";
}
//mySQL connection values
// Create connection
$conn = new mysqli($mysql_host, $mysql_user, $mysql_password, $mysql_database);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$query = "INSERT INTO ranking ($fields) VALUES " . implode (', ', $values);
if ($conn->query($query) === TRUE) {
echo "New record created successfully";
} else {
echo "Error: " . $query . "<br>" . $conn->error;
}
$conn->close();
?>
Related
I get a text from html-form, didnt mention it here, but it looks like:
John:John
Mike:Mike
Root:Admin
Here is my php code:
$text = explode("\n", $_POST["info"]);
// - get data from html form and //explode it to pieces
print_r($text);
// result is: Array ( [0] => John:John [1] => Mike:Mike [2] => Root:Admin )
foreach ($text as $key => $value) {
$val = explode (":", $value);
// want to explode it to pieces, result must be 0=>John 1=>John, 0=>Mike 1=>Mike, [0]=>Root [1]=>Admin
$sql = "INSERT INTO `redtable`(`NAME`,`NAME2`) VALUES('$val[0]','$val[1]');";
}
When this code runs, it inserts into database only the last line, which are (Root:Admin), why it doesn't inserts John:John, Mike:Mike ...?
Where is the mistake?
Here is the result of echo $sql:
INSERT INTO `redtable`(`IGNAME`,`IGPASS`) VALUES('John','John ');INSERT INTO `redtable`(`IGNAME`,`IGPASS`) VALUES('Mike','Mike ');INSERT INTO `redtable`(`IGNAME`,`IGPASS`) VALUES('Root','Admin');
Here is the full code:
<?php
$servername = "localhost";
$username = "mysql";
$password = "mysql";
$dbname = "red";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$pieces = explode("\n", $_POST["info"]);
foreach ($pieces as $key => $value) {
$val = explode (":", $value);
$sql = "INSERT INTO `redtable`(`IGNAME`,`IGPASS`) VALUES('$val[0]','$val[1]');";
echo $sql;
}
if ($conn->query($sql) === TRUE) {
echo "Days left updated";
} else {
mysqli_error($conn);
echo "Error: " . $sql . "<br>" . $conn->error;
}
$conn->close();
?>
Have trying using if-else statements, like this:
# code...
$val = explode (":", $value);
# print_r($val);
if (1 == 1) {
$sql = "INSERT INTO `redtable`(`IGNAME`,`IGPASS`) VALUES('$val[0]','$val[1]');";
}
else {
echo "esle";
}
}
The same result, only the last line have been inserted to the DB.
GUYS, If SOMEONE NEED THE WORKING SOLUTION WATCH #Matt Rabe answer - working like a charm, you need just replace the brackets!
Mark Baker is right - you are executing your sql outside of your foreach loop. You are defining the $sql var inside your foreach, but the actual execution of it ($conn->query($sql)) occurs outside of the foreach.
Change this:
foreach ($pieces as $key => $value) {
$val = explode (":", $value);
$sql = "INSERT INTO `redtable`(`IGNAME`,`IGPASS`) VALUES('$val[0]','$val[1]');";
echo $sql;
}
if ($conn->query($sql) === TRUE) {
echo "Days left updated";
} else {
mysqli_error($conn);
echo "Error: " . $sql . "<br>" . $conn->error;
}
To this:
foreach ($pieces as $key => $value) {
$val = explode (":", $value);
$sql = "INSERT INTO `redtable`(`IGNAME`,`IGPASS`) VALUES('$val[0]','$val[1]');";
echo $sql;
if ($conn->query($sql) === TRUE) {
echo "Days left updated";
} else {
mysqli_error($conn);
echo "Error: " . $sql . "<br>" . $conn->error;
}
}
Your code has numerous issues beyond this, but this should address your stated question.
Im trying to insert json data using php into mysql,
I get success msg, but no records are inserted.
My json data is :
jsondata.json:
{"users": { "bert":6.44, "earnie":0.25, "bigbird":34.45 }}
My php code:
<?php
//First: read data
$fo=fopen("data.json","r");
$fr=fread($fo,filesize("data.json"));
$array=json_decode($fr,true);
//Second: create $values
$rows = array();
foreach ($array['users'] as $key => $value)
$rows[] = "('" . $key . "', '" . $value . "')";
$values = implode(",", $rows);
//To display all values from JSON file
echo '<pre>';print_r($array);
//Save to DB
$hostname = 'localhost';
$username = 'root';
$password = '';
try
{
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
echo 'Connected to database<br />';
//$count = $dbh->exec("INSERT INTO USERSAMOUNTS(USERNAME, AMOUNT) VALUES " . $values) or die(print_r($dbh->errorInfo(), true));
$count = $dbh->exec("INSERT INTO json(firstName) VALUES " . $values) or die(print_r($dbh->errorInfo(), true));
echo $count;// echo the number of affected rows
$dbh = null;// close the database connection
echo 'Success<br />';
}
catch(PDOException $e)
{
echo $e->getMessage();
}
?>
I believe the problem could be the order in which the actions are performed.
<?php
//First: read data
$fo = fopen("jsondata.json", "r");
$fr = fread($fo, filesize("jsondata.json"));
$array = json_decode($fr, true);
//Second: create $values
$rows = array();
foreach ($array['users'] as $key => $value)
$rows[] = "('" . $key . "', '" . $value . "')";
$values = implode(",", $rows);
//Third: display
echo '<pre>';
print_r($array);
//Fourth: save to db
$hostname = 'localhost';
$username = 'root';
$password = '';
try {
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
echo 'Connected to database<br />'; // echo a message saying we have connected
$count = $dbh->exec("INSERT INTO USERAMOUNTS(USERNAME, AMOUNT) VALUES " . $values);
echo $count; // echo the number of affected rows
$dbh = null; // close the database connection
echo 'Success<br />';
} catch (PDOException $e) {
echo $e->getMessage();
}
?>
Enables or disables emulation of prepared statements. Some drivers do not support native prepared statements or have limited support for them for more info please check - http://php.net/manual/en/pdo.setattribute.php
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
$dbh->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$count = $dbh->exec("INSERT INTO USERAMOUNTS(USERNAME, AMOUNT) VALUES " . $values);
Hope this help.
The issue is with how your are trying to insert data. I'm surprised you're not getting an error.
You should use a prepared statement. See the following... https://stackoverflow.com/a/4629088/2033178
Some things are funky here.
At first it looks like you're expecting the data to come magically from $data (unless that is passed somewhere?)
$array = json_decode($data, true);
$rows = array();
foreach($array['users'] as $key => $value)
$rows[] = "('" . $key . "', '" . $value . "')";
$values = implode(",", $rows);
And then it looks like you're opening a file and parsing the JSON (but not doing the above magic with $rows[])
$fo=fopen("jsondata.json","r");
$fr=fread($fo,filesize("jsondata.json"));
$array=json_decode($fr,true);
Why not insert on the for each loop?
$fo=fopen("jsondata.json","r");
$fr=fread($fo,filesize("jsondata.json"));
$array=json_decode($fr,true);
$count = 0;
$dbh = new PDO("mysql:host=$hostname;dbname=nodejs", $username, $password);
try {
foreach($array['users'] as $key => $value)
$count = $count + $dbh->exec("INSERT INTO USERAMOUNTS(USERNAME, AMOUNT) VALUES " . $key . " " . $value . ")";
} catch ...
I've the following script:
<?php
//Define basepath for codeigniter
define('BASEPATH', '/');
//Include constants.php config file from codeigniter
require_once "../../my_manager/system/application/config/constants.php";
//Check username and password from GET
if( $_GET['username'] != IMPORT_USERNAME || $_GET['password'] != IMPORT_PASSWORD )
{
header('HTTP/1.0 401 Unauthorized');
echo "Denied Access";
return;
}
//running scripts
include 'export_table1.php';
include 'export_table2.php';
include 'export_table3.php';
include 'export_table4.php';
?>
Eache export is like the following:
<?php
//$host="localhost";
$host="";
$user="";
$pass="";
$db_name="";
$table="table1";
$conn = mysqli_connect($host,$user,$pass,$db_name) or die("Connection Error");
$query = "SELECT * FROM $table ORDER BY ID";
$result = mysqli_query($conn,$query) or die("sql error");
if(mysqli_num_rows($result)>0)
{
$csv = "";
$row = mysqli_fetch_assoc($result);
$delim = "";
//retrieving first line fields
foreach($row as $k => $v)
{
$csv .= $delim . '"' . str_replace('"', '""', $k) . '"';
$delim= ";";
}
$csv .= "\n";
//retrieving value into fields
while($row = mysqli_fetch_assoc($result))
{
$delim = "";
foreach($row as $v)
{
$csv .= $delim . '"' . str_replace('"', '""', $v) . '"';
$delim = ";";
}
$csv .= "\n";
}
header("Content-type: text/csv");
header("Content-Disposition: attachment; filename=".$table.".csv");
echo $csv;
exit
}
else
{
echo "No records";
}
exit
?>
The issue is that only the first script is fired, so I can get only the first table downloaded into csv file. If I launch first the table2 export, only table2 export is fired. How can I manage that? Thanks in advance!
As far as i know "exit" kills both child- and parent-script - that's the cause why you only get the first table exported in a csv. Use "return" instead to continue.
I have a lot of csv files in a directory. With these files, I have to write a script that put their content in the right fields and tables of my database. I am almost beginner in php language : I have found some bits of code on the Internet. It seems to work, but I am stuck at a stage. Some topics are related on this website, but I did not found the ecat problem.
I have written a php script that permits to get the path and the name of these files. I managed too to create a table whose name depends of each csv (e.g : file ‘data_1.csv’ gives the table data_1.csv in my-sql). All the tables have the three same fields, id, url, value.
The last thing to do is to populate these tables, by reading each file and put the values separated by ‘|’ character in the right tables. For example, if a line of ‘data_1.csv’ is
8756|htttp://example.com|something written
I would like to get a record in data_1.csv table where 8756 is in id, htttp://example.com in url field, and something written in value field.
I have found the way to read and print these csv with fcsvget function. But I do not know how to make these lines go into the SQL database. Could anyone help me on this point?
Here is my script below
<?php
ini_set('max_execution_time', 300);
$dbhost = 'localhost';
$dbuser = 'root';
$dbpass = '';
$conn = mysql_connect($dbhost, $dbuser, $dbpass, true) or die ('Error connecting to mysql');
$dbname = 'test_database';
mysql_select_db($dbname);
$bdd = mysqli_connect($dbhost, $dbuser, $dbpass, $dbname);
$mysqli = new mysqli($dbhost, $dbuser, $dbpass, $dbname);
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
else {
echo "hello <br>";
}
$dir = 'C:\wamp\www\test';
$imgs = array();
if ($dh = opendir($dir)) {
while (($file = readdir($dh)) !== false) {
if (!is_dir($file) && preg_match("/\.(csv)$/", $file)) {
array_push($imgs, $file);
}
}
closedir($dh);
} else {
die('cannot open ' . $dir);
}
foreach ($imgs as $idx=>$img) {
$class = ($idx == count($imgs) - 1 ? ' class="last"' : '');
$filePath=$dir . '\\'. $img;
$file = fopen($filePath, "r") or exit("Unable to open file!");
$nom = 'FILE: ' . $dir . '\\'. $img;
echo $nom;
settype($nom, "string");
echo '<br>';
$chemin = '<img src="' . $dir . $img . '" alt="' .
$img . '"' . $class . ' />' . "\n";
echo $chemin;
$file_name = basename($filePath);
$sql = 'CREATE TABLE `' . $file_name . '` (id int(20000), url varchar(15), value TEXT)';
mysql_query($sql,$conn);
$handle = fopen($filePath, 'r');
while (($row = fgetcsv($handle)) !== false) {
foreach ($row as $field) {
echo $field . '<br />';
}
}
fclose($handle);
}
echo ("VERIFY");
for ($i = 1; $i <= 1682; $i++) {
echo ("<br>A : " . $i);
$checkRequest= "select * from movies where movieID='". $i."'";
echo ("<br>". $checkRequest);
if ($result = $mysqli->query($checkRequest)) {
printf("<br> Select ". $i ."returned %d rows.\n", $result->num_rows);
if ($result->num_rows == 0) {
echo ("I : " . $i);
}
$result->close();
}
}
$mysqli->close();
?>
MySQL provides a wonderful feature that allows you to import a CSV file directly, in a single query.
The SQL command you're looking for is LOAD DATA INFILE
Manual page here: http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Quick example:
LOAD DATA INFILE 'fileName'
INTO TABLE tableName
FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
(
field1,
field2,
field3,
#variable1,
#variable2,
etc
)
set
(
field4 = concat(#variable1,#variable2)
);
That's a fairly basic example, but it covers most of what you'd want. The manual page gives full details of how to do some very complex stuff with it.
Hope that helps.
after
foreach ($row as $field) {
echo $field . '<br />';
You need to parse the result after ; like:
$pieces = explode(";", $field);
and then insert every piece into the database
$sql = ' INSERT INTO X VALUES ("'.$pieces[0].'","'.$pieces[1].'","'.$pieces[2].'","'.$pieces[3].'","'.$pieces[4].'")';
mysql_query($sql, $conn);
Also mysqlimport can be used
private function runImport($host, $username, $password, $schema, $csvFilePath) {
$output = array();
$cmd = "/usr/bin/mysqlimport --host=$host";
$cmd .= ' --fields-terminated-by=\',\' --fields-optionally-enclosed-by=\'"\' ';
$cmd .= ' --user=' . $username . ' --password='. $password;
$cmd .= " $schema $csvFilePath";
exec($cmd, $output, $retVal);
foreach ($output as $line) {
echo "$line\n";
}
echo "\n\nReturn code : $retVal";
}
I feel like I'm making a simple mistake but I can't seem to figure out what.
I have some code for exporting a mySQL table to an Excel file.
However, when I do the export, the entire HTML source code gets exported along with my data. I open the file in Excel and my table data in there but it's also got all the HTML inside.
What could be causing all the source code to be exported along with the data?
I should mention that I'm using this code as part of a Wordpress plugin I'm writing. When I test the export outside wordpress, it works fine. But when I try to export from a Wordpress admin page, I get all the extra HTML source code.
Try this code.
$host = 'localhost';
$user = 'mysqlUser';
$pass = 'myUserPass';
$db = 'myDatabase';
$table = 'products_info';
$file = 'export';
$link = mysql_connect($host, $user, $pass) or die("Can not connect." . mysql_error());
mysql_select_db($db) or die("Can not connect.");
$res = mysql_query("SELECT * FROM $table");
// fetch a row and write the column names out to the file
$row = mysql_fetch_assoc($res);
$line = "";
$comma = "";
foreach($row as $name => $value) {
$line .= $comma . '"' . str_replace('"', '""', $name) . '"';
$comma = ",";
}
$line .= "\n";
fputs($fp, $line);
// remove the result pointer back to the start
mysql_data_seek($res, 0);
// and loop through the actual data
while($row = mysql_fetch_assoc($res)) {
$line = "";
$comma = "";
foreach($row as $value) {
$line .= $comma . '"' . str_replace('"', '""', $value) . '"';
$comma = ",";
}
$line .= "\n";
fputs($fp, $line);
}
fclose($fp);
header('Content-type: application/csv');
header('Content-Disposition: attachment; filename="export.csv"');
readfile('export.csv');
Thanks,
Kanji