MySQL INSERT INTO from PHP://INPUT multiple rows - php

I have data (exact) from this HTTP POST:
rowno=1.00000000&date_line=2014-10-07&name=Dan%20Volunteer&affiliation=Enterprise&checkno=1701&amount=20025.00000000&total=20250.00000000&notes=&date_deposit=&rowno=2.00000000&date_line=2014-10-07&name=Harper%20Lee&affiliation=Enterprise%20B&checkno=1702&amount=225
then this code to process
<?php
file_get_contents("php://input");
$db = null;
if (isset($_SERVER['SERVER_SOFTWARE']) &&
strpos($_SERVER['SERVER_SOFTWARE'],'Google App Engine') !== false) {
// Connect from App Engine.
try{
$db = new pdo('mysql:unix_socket=/cloudsql/wonder:bread;dbname=loaf', 'root', '');
}catch(PDOException $ex){
die(json_encode(
array('outcome' => false, 'message' => 'Unable to connect.')
)
);
}
};
try {
if (array_key_exists('name', $_POST)) {
$stmt = $db->prepare('INSERT INTO entries (name, affiliation) VALUES (:name, :affiliation)');
$stmt->execute(array(':name' => htmlspecialchars($_POST['name']), ':affiliation' => htmlspecialchars($_POST['affiliation'])));
$affected_rows = $stmt->rowCount();
// Log $affected_rows.
}
} catch (PDOException $ex) {
// Log error.
}
$db = null;
?>
<?php
header("Content-type: application/vnd.fdf");
// read and store the data however you want
// reply with some FDF data
echo <<<RESPONSE
%FDF-1.2
1 0 obj
<< /FDF <<
/Status (Wham bam! File sent.)
>>
>>
endobj
trailer
<< /Root 1 0 R >>
%%EOF
RESPONSE;
?>
This http post has two records (row/recount count always varies), but only data from the last row is being inserted. Need all rows.

I'm going to stab at this one....I think what is happening is that you are just processing the return post as is, so the first rowno is being skipped over (rather the second rowno is overwriting the first). If you receive that post back as a string, you need to split it by preg_match() or explode() so that you can loop over it with your try.
Try this class on your string. This class will split the string into arrays based on rows. Then you need to take the resulting array $insert then process each array in your an sql loop...does that make sense?
class ProcessPost
{
public static function Split($value = '',$splitVal = 'rowno=')
{
if(!empty($value)) {
// Explode by row values
$rows = explode($splitVal,$value);
$rows = array_filter($rows);
if(is_array($rows) && !empty($rows)) {
foreach($rows as $_row => $querystring) {
parse_str($splitVal.$querystring,$_array[]);
}
foreach($_array as $row_key => $row_val) {
if(empty($row_val))
unset($_array[$row_key]);
}
return $_array;
}
}
}
}
$test = 'rowno=1.00000000&date_line=2014-10-07&name=Dan%20Volunteer&affiliation=Enterprise&checkno=1701&amount=20025.00000000&total=20250.00000000&notes=&date_deposit=&rowno=2.00000000&date_line=2014-10-07&name=Harper%20Lee&affiliation=Enterprise%20B&checkno=1702&amount=225';
$insert = ProcessPost::Split($test);

Related

PHP writing to database, discovering what happens

I have this mysql table
****** login_attempts **********
email (varchar 40)
attempts (tinyint 1)
ipaddress (CHAR 45)
I want to write some data to this table with this PHP code, which seems good.
$thecols = "(email,attempts,ipaddress)";
$parndata = array(':one'=>'1111',':two'=>2,':three'=>'3');
$bindings = "(:one,:two,:three)";
insertInto($db,'login_attempts',$thecols,$parndata,$bindings);
function insertInto($db,$table,$tablevals,$pars,$forbinds){
echo "INSERT INTO $table$tablevals VALUES $forbinds";
echo '<br>';
try{
$one = $db->prepare("INSERT INTO $table$tablevals VALUES $forbinds");
foreach ($pars as $key => $value) {
$one->bindParam($key,$value);
//echo $key.','.$value.'<br>';
}
if($one->execute()){ return true; }else{ return false; }
}catch(PDOException $ex){
throw $ex;
}catch(Exception $ex){
throw $ex;
}
}
but the above code writes the data like that
email [3]
attempts [3]
ipaddress [3]
why this happened ?
This is because PDOStatement::bindParam($param, &$variable) accepts $variable by reference. All parameters are binded to the same $value variable which has a value of 3 at the end of foreach loop.
Replace ->bindParam($key,$value); with ->bindValue($key,$value); to fix an issue.

PHP Improve performance to execute multiple queries while reading a file with thousand lines

I'm trying to build a script where I need to read a txt file and execute some process with the lines on the file. For example, I need to check if the ID exists, if the information has updated, if yes, then update the current table, if no, then insert a new row on another temporary table to be manually checked later.
These files may contain more than 20,30 thousand lines.
When I just read the file and print some dummie content from the lines, it takes up to 40-50ms. However, when I need to connect to the database to do all those verifications, it stops before the end due to the timeout.
This is what I'm doing so far:
$handle = fopen($path, "r") or die("Couldn't get handle");
if ($handle) {
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
$segment = explode('|', $buffer);
if ( strlen($segment[0]) > 6 ) {
$param = [':code' => intval($segment[0])];
$codeObj = Sql::exec("SELECT value FROM product WHERE code = :code", $param);
if ( !$codeObj ) {
$param = [
':code' => $segment[0],
':name' => $segment[1],
':value' => $segment[2],
];
Sql::exec("INSERT INTO product_tmp (code, name, value) VALUES (:code, :name, :value)", $param);
} else {
if ( $codeObj->value !== $segment[2] ) {
$param = [
':code' => $segment[0],
':value' => $segment[2],
];
Sql::exec("UPDATE product SET value = :value WHERE code = :code", $param);
}
}
}
}
fclose($handle);
}
And this is my Sql Class to connect with PDO and execute the query:
public static function exec($sql, $param = null) {
try {
$conn = new PDO('mysql:charset=utf8mb4;host= '....'); // I've just deleted the information to connect to the database (password, user, etc.)
$q = $conn->prepare($sql);
if ( isset($param) ) {
foreach ($param as $key => $value) {
$$key = $value;
$q->bindParam($key, $$key);
}
}
$q->execute();
$response = $q->fetchAll();
if ( count($response) ) return $response;
return false;
} catch(PDOException $e) {
return 'ERROR: ' . $e->getMessage();
}
}
As you can see, each query I do through Sql::exec(), is openning a new connection. I don't know if this may be the cause of such a delay on the process, because when I don't do any Sql query, the script run within ms.
Or what other part of the code may be causing this problem?
First of all, make your function like this,
to avoid multiple connects and also o get rid of useless code.
public static function getPDO() {
if (!static::$conn) {
static::$conn = new PDO('mysql:charset=utf8mb4;host= ....');
}
return static::$conn;
}
public static function exec($sql, $param = null) {
$q = static::getPDO()->prepare($sql);
$q->execute($param);
return $q;
}
then create unique index for the code field
then use a single INSERT ... ON DUPLICATE KEY UPDATE query instead of your thrree queries
you may also want to wrap your inserts in a transaction, it may speed up the inserts up to 70 times.

How to insert large amount of data in MySQL DB using JSON

I am using 7 JSON File, and I am trying to insert 95 coloums + 50-100 rows at a time.
This is how my code looks like: I removed some 6 JSON files and other rows&colums. This is a short version of my code
<?php
$requestsDone = 2;
$maxRequests = 100;
$currentnumber = 1;
while ($requestsDone <= $maxRequests) {
$response = file_get_contents("https://api.themoviedb.org/3/movie/".$requestsDone."?api_key=522cexxxxxxxxxxxxx9ax34a");
if ($response != FALSE) {
$response = json_decode($response, true);
}
if ( ($response["title"]) != "" )
{
$requestsDone++;
include 'admin/connectdb.php';
try {
// prepare sql and bind parameters
$stmt = $conn->prepare("INSERT INTO TABLE (title,image,money,tag1,tag2)
VALUES (:title, :image, :money,:tag1,:tag2)");
$stmt->bindParam(':title', $title);
$stmt->bindParam(':image', $image);
$stmt->bindParam(':money', $money);
$stmt->bindParam(':tag1', $tag1);
$stmt->bindParam(':tag1', $tag2);
// insert a row
$movie_title = "".$response["title"]."";
if ( $response["image"] != "" )
$image = "".$response["image"]."";
if ( $response["money"] != "" )
$money = "".$response["money"]."";
if ( ($response["tag"][0]["name"]) != "" )
$tag1 = "".$response["tag"][0]["name"]."";
if ( ($response["tag"][1]["name"]) != "" )
$tag2 = "".$response["tag"][1]["name"]."";
$stmt->execute();
echo "New records created successfully ".$currentnumber++." <br/>";
}
catch(PDOException $e)
{
echo "Error: " . $e->getMessage();
}
$conn = null;
}
}
?>
I am getting 2 type of errors: Sometime 1st error, sometime 2nd error.
1st Error: Cloudflare time-out: Error 504 Screenshot
2nd Error: Error: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
Please let me know, if you need more information.
Edit:
This is how one of the JSON file looks like
{"adult":false,"backdrop_path":"/87hTDiay2N2qWyX4Ds7ybXi9h8I.jpg","belongs_to_collection":null,"budget":63000000,"genres":[{"id":18,"name":"Drama"}],"homepage":"http://www.foxmovies.com/movies/fight-club","id":550,"imdb_id":"tt0137523","original_language":"en","original_title":"Fight Club","overview":"A ticking-time-bomb insomniac and a slippery soap salesman channel primal male aggression into a shocking new form of therapy. Their concept catches on, with underground \"fight clubs\" forming in every town, until an eccentric gets in the way and ignites an out-of-control spiral toward oblivion.","popularity":9.765478999999999,"poster_path":"/adw6Lq9FiC9zjYEpOqfq03ituwp.jpg","production_companies":[{"name":"Regency Enterprises","id":508},{"name":"Fox 2000 Pictures","id":711},{"name":"Taurus Film","id":20555},{"name":"Linson Films","id":54050},{"name":"Atman Entertainment","id":54051},{"name":"Knickerbocker Films","id":54052}],"production_countries":[{"iso_3166_1":"DE","name":"Germany"},{"iso_3166_1":"US","name":"United States of America"}],"release_date":"1999-10-15","revenue":100853753,"runtime":139,"spoken_languages":[{"iso_639_1":"en","name":"English"}],"status":"Released","tagline":"How much can you know about yourself if you've never been in a fight?","title":"Fight Club","video":false,"vote_average":8.199999999999999,"vote_count":8047}

Undefined offset: 0 in first PHP app

I'm currently building my first PHP application. I want to read in bus times from a csv file and give back ti users the next bus from their residence to our university. It is my first try with PHP, so be gentle:
<?php
// configuration
require("../includes/config.php");
if (!empty($_SESSION["id"]))
{
//lookup database entries for house
$users = query("SELECT * FROM users WHERE id = ?", $_SESSION["id"]);
if ($users === false)
{
apologize("Sorry, there was an error");
}
//lookup database entries for house
$residences = query("SELECT * FROM residences WHERE id = ?", $users[0]["residence"]);
if ($residences === false)
{
apologize("Sorry, there was an error");
}
//if user specified a residence in his profile
if($residences[0]["id"] != 0)
{
$times = array();
//if there is no bus today, in this case sat and sun
if(date( "w", $timestamp) == 0 || date( "w", $timestamp) == 6)
{
$times[0] = "There is no bus today";
}
//load the busplan for his residence
else
{
//open file and load in array if time is higher than date("His");
$timesList = file_get_contents($users[0]["residence"] . ".csv");
$nextbuses = explode(',', $timesList);
$hoursMins = date("Gi");
$num = 0;
for($i = 0; $i < count($nextbuses); $i++)
{
if($hoursMins < $nextbuses[$i])
{
$times[$num] = $nextbuses[$i];
$num++;
}
}
}
render("shuttle_show.php", ["title" => "Next Shuttle from your residence.", "times" => $times]);
}
}
This uses the function query:
function query(/* $sql [, ... ] */)
{
// SQL statement
$sql = func_get_arg(0);
// parameters, if any
$parameters = array_slice(func_get_args(), 1);
// try to connect to database
static $handle;
if (!isset($handle))
{
try
{
// connect to database
$handle = new PDO("mysql:dbname=" . DATABASE . ";host=" . SERVER, USERNAME, PASSWORD);
// ensure that PDO::prepare returns false when passed invalid SQL
$handle->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
}
catch (Exception $e)
{
// trigger (big, orange) error
trigger_error($e->getMessage(), E_USER_ERROR);
exit;
}
}
// prepare SQL statement
$statement = $handle->prepare($sql);
if ($statement === false)
{
// trigger (big, orange) error
trigger_error($handle->errorInfo()[2], E_USER_ERROR);
exit;
}
// execute SQL statement
$results = $statement->execute($parameters);
// return result set's rows, if any
if ($results !== false)
{
return $statement->fetchAll(PDO::FETCH_ASSOC);
}
else
{
return false;
}
}
the other functions it uses are not relevant I guess. Now I cant seem to find why this keeps producing:
Notice: Undefined offset: 0 in /Applications/MAMP/htdocs/html/shuttle.php on line 16
Notice: Undefined offset: 0 in /Applications/MAMP/htdocs/html/shuttle.php on line 22
The relevant lines are
$residences = query("SELECT * FROM residences WHERE id = ?", $users[0]["residence"]);
and
if($residences[0]["id"] != 0)
Would appreciate some help! :)
Edit:
I transferred the same file to my linux system and without any changes I get a vardump. If I use the same vardump on MAMP on my Mac the array is empty. Now I get for:
var_dump($users);
array (size=1)
0 =>
array (size=5)
'id' => int 12
'username' => string 'frechdaxx' (length=9)
'mail' => string '*************#gmail.com' (length=23)
'hash' => string '$1$qr5axj4C$BET5zZGJza2DcHI8eD8fV0' (length=34)
'residence' => int 9573
Why is this a problem at all? the function query worked with the exact same syntax before when I accessed the user table and as we can see it gives back all other values correctly.
Why the difference between my environments? The array is empty on my MAMP server, but works on Linux. However, other code examples with the query function work perfectly on both environments
Why the big int of 9573? The value in the table is 2.
This is simply that $users[0]["residence"] doesn't exists (is not set). You should check it before attempting to use it. As you can see, it's only a notice, not a error. This means that in some case scenarios, the variable can be unset and PHP won't complain THAT much, but this is definitely not good as a general rule.
I would change it to:
$users = query("SELECT * FROM users WHERE id = ?", $_SESSION["id"]);
if (empty($users[0]["residence"]))
{
apologize("Sorry, there was an error");
}
Furthermore, that only "fixes" it. If you want to go to the root of the problem, it's in the query() function that is called in $users = query("SELECT * FROM users WHERE id = ?", $_SESSION["id"]);:
// execute SQL statement
$results = $statement->execute($parameters);
// return result set's rows, if any
if ($results !== false)
{
$fetched = $statement->fetchAll(PDO::FETCH_ASSOC);
// Check that ther was actually something fetched
if(!empty($fetched))
{
return $fetched;
}
}
/* You don't need to return false, null is the default returned value.
* If you *want* it, then delete the else. */
}
From the documentation, fetchall returns an empty array if there's nothing, so the function might return an empty array.
$users = query("SELECT * FROM users WHERE id = ?", $_SESSION["id"]);
if ($users === false) { }
You are strictly checking if query returned false, that only occurs when something was wrong with query or parameters. That does not check if any result was returned, and you are referring to the first returned record ($users[0]["residence"]) that is not present ($users is an empty array).
Since user id comes from $_SESSION["id"] it should be present and available in database, but there is possibility that somewhere in your code you are overwriting it with other id.

ajax to a php and get JSON from it

I have a php file that i connect to it with ajax and callback value is JSON
when i get data from php it dosnt show and when alert data i see Object
Where is my problem ?
PHP:
if(isset($_SERVER["HTTP_X_REQUESTED_WITH"])){
$query = mysql_query("select * from tab");
for ($i=0;$i<mysql_num_rows($query);$i++){
while($row = mysql_fetch_assoc($query)){
$title['t'][i] = $row['title'];
$title['d'][i] = $row['description'];
}
}
echo(json_encode($title));
exit();
?>
JS:
$('#button').click(function(){
$.ajax({
url : "test2.php",
data : $("#tab"),
type : "GET",
success : function(b){
b = eval('('+ b +')');
console.log((b['t']));
alert(b);
}
});
});
How can i get all of data from this JSON and show me corect it ?
Here's a full working example with single row fetch and multi row fetch, without using mysql_ syntax and using prepared statements to prevent sql injections.
And yes, DON'T use mysql specific syntax, like I mentioned here: I cant get the form data to go into database. What am I doing wrong?
function example()
{
var select = true;
var url = '../scripts/ajax.php';
$.ajax(
{
// Post select to url.
type : 'post',
url : url,
dataType : 'json', // expected returned data format.
data :
{
'select' : select // the variable you're posting.
},
success : function(data)
{
// This happens AFTER the PHP has returned an JSON array,
// as explained below.
var result1, result2, message;
for(var i = 0; i < data.length; i++)
{
// Parse through the JSON array which was returned.
// A proper error handling should be added here (check if
// everything went successful or not)
result1 = data[i].result1;
result2 = data[i].result2;
message = data[i].message;
// Do something with result and result2, or message.
// For example:
$('#content').html(result1);
// Or just alert / log the data.
alert(result1);
}
},
complete : function(data)
{
// do something, not critical.
}
});
}
Now we need to receive the posted variable in ajax.php:
$select = isset($_POST['select']) ? $_POST['select'] : false;
The ternary operator lets $select's value become false if It's not set.
Make sure you got access to your database here:
$db = $GLOBALS['db']; // An example of a PDO database connection
Now, check if $select is requested (true) and then perform some database requests, and return them with JSON:
if($select)
{
// Fetch data from the database.
// Return the data with a JSON array (see below).
}
else
{
$json[] = array
(
'message' => 'Not Requested'
);
}
echo json_encode($json);
flush();
How you fetch the data from the database is of course optional, you can use JSON to fetch a single row from the database or you can use it return multiple rows.
Let me give you an example of how you can return multiple rows with json (which you will iterate through in the javascript (the data)):
function selectMultipleRows($db, $query)
{
$array = array();
$stmt = $db->prepare($query);
$stmt->execute();
if($result = $stmt->fetchAll(PDO::FETCH_ASSOC))
{
foreach($result as $res)
{
foreach($res as $key=>$val)
{
$temp[$key] = utf8_encode($val);
}
array_push($array, $temp);
}
return $array;
}
return false;
}
Then you can do something like this:
if($select)
{
$array = array();
$i = 0;
$query = 'SELECT e.result1, e.result2 FROM exampleTable e ORDER BY e.id ASC;';
foreach(selectMultipleRows($db, $query) as $row)
{
$array[$i]["result1"] = $row['result1'];
$array[$i]["result2"] = $row['result2'];
$i++;
}
if(!(empty($array))) // If something was fetched
{
while(list($key, $value) = each($array))
{
$json[] = array
(
'result1' => $value["result1"],
'result2' => $value["result2"],
'message' => 'success'
);
}
}
else // Nothing found in database
{
$json[] = array
(
'message' => 'nothing found'
);
}
}
// ...
Or, if you want to KISS (Keep it simple stupid):
Init a basic function which select some values from the database and returns a single row:
function getSingleRow($db, $query)
{
$stmt = $db->prepare($query);
$stmt->execute();
// $stmt->execute(array(":id"=>$someValue)); another approach to execute.
$result = $stmt->fetch(PDO::FETCH_ASSOC);
if($result)
{
$array = (
'result1' => $result['result1'],
'result2' => $result['result2']
);
// An array is not needed for a single value.
return $array;
}
return false;
}
And then fetch the row (or the single value) and return it with JSON:
if($select)
{
// Assume that the previously defined query exists.
$results = getSingleRow($db, $query);
if($results !== false)
{
$json[] = array
(
'result1' => $results['result1'],
'result2' => $results['result2'],
'message' => 'success'
);
}
else // Nothing found in database
{
$json[] = array
(
'message' => 'nothing found'
);
}
}
// ...
And if you want to get the value of $("#tab") then you have to do something like $("#tab").val() or $("#tab").text().
I hope that helps.
I suggest to use either:
b = jQuery.parseJSON(data)
see more here
or
$.getJSON
instead of eval()

Categories