I created a PHP handler to receive a JSON payload from a POST request and then insert it into a database in phpMyAdmin. I'm not sure why this is not working.
JSON:
payload = {
"version":"1.0",
"event":"video_recorded",
"data":{
"videoName":"vs1457013120534_862",
"audioCodec":"NellyMoser ASAO",
"videoCodec":"H.264",
"type":"FLV",
"orientation":"landscape",
"id":"0",
"dateTime":"2016-03-03 15:51:44",
"timeZone":"Europe/Bucharest",
"payload":"111.111.111.11",
"httpReferer":"http://site_from_where_video_was_recorded.com"
}
}
The PHP code I got from a tutorial online. The tutorial was from 2017 so I'm assuming everything is up to date, but yet it still does not work:
<?php
/* db variables */
$dbhost = 'localhost';
$dbname = 'name_db';
$dbuser = 'user_db';
$dbpass = 'pass_db';
/* grab the json */
$data = $_POST['payload'];
/* put json into php associative array */
$data_array = json_decode($data);
/* store in PHP variables */
$ip_address = $data_array['data']['payload'];
$vid_name = $data_array['data']['videoName'];
$date_time = $data_array['data']['dateTime'];
$time_zone = $data_array['data']['timeZone'];
/* connect to mysql db */
$con = mysql_connect($dbuser, $dbpass, $dbhost) or die('Could not connect: ' . mysql_error());
/* select the specific db */
mysql_select_db($dbname, $con);
/* insert the values into the db */
$sql = "INSERT INTO ip_and_videos(IpAddress, VideoName, DateTime, Timezone) VALUES('$ip_address','$vid_name','$date_time','$time_zone')";
if(!mysql_query($sql,$con))
{
die('Error : ' . mysql_error());
}
?>
I have the primary key set to an int and have it on auto increment. If I understand correctly I don't need to insert anything into that column because it will assign a number each time. Or do I still need to pass it when I INSERT the other variables?
This works for the array part, you get correct answer. So, your code is not bad, but you should check all errors (as stated by Bhavin in comment). And I'm retty sure you have a typo -> $vid_name = $data_array['data']['videName']; is NOT like $vid_name = $data_array['data']['videoName']; Thereforer, error_reporting will be very helpful, and after that, check the query if other errors (prepared statements ^^)
<?php
error_reporting(E_ALL);
ini_set('display_errors', 1);
$payload = '{
"version":"1.0",
"event":"video_recorded",
"data": {
"videoName":"vs1457013120534_862",
"audioCodec":"NellyMoser ASAO",
"videoCodec":"H.264",
"type":"FLV",
"orientation":"landscape",
"id":"0",
"dateTime":"2016-03-03 15:51:44",
"timeZone":"Europe/Bucharest",
"payload":"111.111.111.11",
"httpReferer":"http://site_from_where_video_was_recorded.com"
}
}';
$data_array = json_decode($payload, true);
/* store in PHP variables */
$ip_address = $data_array['data']['payload'];
$vid_name = $data_array['data']['videoName'];
$date_time = $data_array['data']['dateTime'];
$time_zone = $data_array['data']['timeZone'];
echo"[ $ip_address / $vid_name / $date_time / $time_zone ]";
// EDIT : added query
include"config.inc.php";
// connect to DB
$mysqli = mysqli_connect("$host", "$user", "$mdp", "$db");
if (mysqli_connect_errno()) { echo "Error connecting : " . mysqli_connect_error($mysqli); }
$query = " INSERT INTO ip_and_videos (`IpAddress`, `VideoName`, `DateTime`, `Timezone`) VALUES (?,?,?,?) ";
$stmt = $mysqli->prepare($query);
print_r($stmt->error_list);
$stmt->bind_param("ssss", $ip_address, $vid_name, $date_time, $time_zone );
if (!$stmt->execute()) { echo $stmt->error; } else { echo"true"; }
?>
Not sure if you had the same issue but this does not work for me...
$vid_name = $data_array['data']['videName'];
I had to use
$vid_name = $data_array->data->videoName;
Can't use stdClass as an array.
Should be better this way
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
/* db variables */
$dbhost = 'localhost';
$dbname = 'name_db';
$dbuser = 'user_db';
$dbpass = 'pass_db';
/* grab the json */
$data = $_POST['payload'];
/* put json into php associative array */
$alldata = json_decode($data,true);
print_r($alldata);
/* connect to mysql db */
$con = mysql_connect($dbuser, $dbpass, $dbhost) or die('Could not connect: ' . mysql_error());
/* select the specific db */
mysql_select_db($dbname, $con);
/* insert the values into the db */
foreach($alldata as $data_array) {
$ip_address = $data_array['data']['payload'];
$vid_name = $data_array['data']['videoName'];
$date_time = $data_array['data']['dateTime'];
$time_zone = $data_array['data']['timeZone'];
$sql = "INSERT INTO ip_and_videos(IpAddress, VideoName, DateTime, Timezone) VALUES('".$ip_address."','".$vid_name."','".$date_time."','".$time_zone."')";
mysql_query($sql);
echo mysql_errno($con) . ": " . mysql_error($con) . "\n";
}
?>
Hope this helps
Related
I have puzzled over this for some time. It is puzzling because a very similar query just a few lines above works fine. I am very new to mysqli, so there may be something very fundamental I am missing.
The connection is set up like this:
Class dbObj{
/* Database connection start */
var $servername = "myserver";
var $username = "myusername";
var $password = "mypassword";
var $dbname = "mydb";
var $conn;
function getConnstring() {
$con = mysqli_connect($this->servername, $this->username, $this->password, $this->dbname) or die("Connection failed: " . mysqli_connect_error());
/* check connection */
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
} else {
$this->conn = $con;
}
return $this->conn;
}
}
Then, in an Ajax processing file:
include_once({the connection file above});
$db = new dbObj();
$connString = $db->getConnstring();
$params = $_REQUEST;
$action = isset($params['action']) != '' ? $params['action'] : '';
$NeedsCls = new Needs($connString);
Then, inside a class called "Needs":
protected $conn;
protected $data = array();
function __construct($connString)
{
$this->conn = $connString;
}
function insertNeeds($params)
{
$ExpDate = $params[Expire];
$sql = "INSERT INTO `pNeeds` (PubCode, Title, Description, Keywords, Expire) VALUES('".$_SESSION["PubCode"]."','".$params["Title"]."','".$params["Description"]."','".$params[NeedsTags]."','".$ExpDate."'); ";
echo $result = mysqli_query($this->conn, $sql) or die("Error - Failed inserting Needs data");
$Record = $db->insert_id;
$KW = explode(';',$params[NeedsTags]);
$KWCount = count($KW);
for($x=0;$x<$KWCount;$x++)
{
$Keyword = $KW[$x];
$sql = "INSERT INTO `Keywords` (Keyword, PubCode, Table, Record, Expire) VALUES ('".$Keyword."','".$_SESSION["PubCode"]."','pNeeds','".$Record."','".$ExpDate."'); ";
echo $result = mysqli_query($this->conn,$sql) or die("Error - Keywords not saved<br />".$mysqli_error());
}
}
The first query works fine. The second fails with the error "Function name must be a string". I've verified the data going into the Ajax code is correct. It doesn't appear that I am missing something stupid. (famous last words) The error message makes no sense to me. Similar posts here on StackOverflow and elsewhere do not seem to pertain in this instance.
It looks like you're just grabbing $params wrong:
$ExpDate = $params[Expire];
$params[NeedsTags];
Should be:
$ExpDate = $params['Expire'];
$params['NeedsTags'];
Edit
You're actual error is from:
$mysqli_error()
Remove the $
Edit
RationalRabbit: To avoid confusing anyone, I should mention that the whole syntax was wrong. Using object oriented mysqli, the error syntax should have been
db->error
OR
mysqli($this->conn)
I am trying to parsing a large XML file and load it into MySQL. I have used simplexml to parse it, and it works perfectly, but its way to slow for this large XML file. Now i am trying to use XMLReader.
Here is the sample of the XML:
<?xml version="1.0" encoding="UTF-8"?>
<drug type="biotech" created="2005-06-13" updated="2015-02-23">
<drugbank-id primary="true">DB00001</drugbank-id>
<drugbank-id>BIOD00024</drugbank-id>
<drugbank-id>BTD00024</drugbank-id>
<name>Lepirudin</name>
<description>Lepirudin is identical </description>
<cas-number>120993-53-5</cas-number>
<groups>
<group>approved</group>
</groups>
<pathways>
<pathway>
<smpdb-id>SMP00278</smpdb-id>
<name>Lepirudin Action Pathway</name>
<drugs>
<drug>
<drugbank-id>DB00001</drugbank-id>
<name>Lepirudin</name>
</drug>
<drug>
<drugbank-id>DB01373</drugbank-id>
<name>Calcium</name>
</drug>
</drugs>
...
</drug>
<drug type="biotech" created="2005-06-15" updated="2015-02-25">
...
</drug>
Here is my approach using simplexml:
<?php
$xml = simplexml_load_file('drugbank.xml');
$servername = "localhost"; // Example : localhost
$username = "root";
$password = "pass";
$dbname = "dbname";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$xmlObject_count = $xml->drug->count();
for ($i=0; $i < $xmlObject_count; $i++) {
$name = $xml->drug[$i]->name;
$description = $xml->drug[$i]->description;
$casnumber = $xml->drug[$i]->{'cas-number'};
// ...
$created = $xml->drug[$i]['created'];
$updated = $xml->drug[$i]['updated'];
$type = $xml->drug[$i]['type'];
$sql = "INSERT INTO `drug` (name, description,cas_number,created,updated,type)
VALUES ('$name', '$description','$casnumber','$created','$updated','$type')";
if ($conn->query($sql) === TRUE) {
$last_id = $conn->insert_id;
} else {
echo "outer else Error: " . $sql . "<br>" . $conn->error. "<br>" ;
}
}
$conn->close();
It works okay and it gives me 7,789 rows. But, I want to use XMLReader to parse this. But the problem with XMLReader I am finding it give more than 35,000 rows.
If you look at the XML you can see that inside the <drug /> nodes there are also some other <drugs><drug> child nodes. How can I overcome this?
Here is my procedure with XMLReader:
<?php
$servername = "localhost"; // Example : localhost
$username = "root";
$password = "pass";
$dbname = "dbname";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$reader = new XMLReader();
$reader->open('drugbank.xml');
while ($reader->read())
{
if ($reader->nodeType == XMLReader::ELEMENT && $reader->name == 'drug')
{
$doc = new DOMDocument('1.0', 'UTF-8');
$xml = simplexml_import_dom($doc->importNode($reader->expand(),true));
$name = $xml->name;
$description = $xml->description;
$casnumber = $xml->{'cas-number'};
// ...
$sql = "INSERT INTO `drug` (name, description,cas_number,created,updated,type)
VALUES ('$name', '$description','$casnumber','$created','$updated','$type')";
if ($conn->query($sql) === TRUE) {
$last_id = $conn->insert_id;
} else {
echo "outer else Error: " . $sql . "<br>" . $conn->error. "<br>" ;
}
}
}
$conn->close();
With this example, I am finding it give more than 35,000 rows.
Alright, I have a working example for you with much improvement in execution speed, memory usage, and database load:
<?php
define('INSERT_BATCH_SIZE', 500);
define('DRUG_XML_FILE', 'drugbank.xml');
$servername = "localhost"; // Example : localhost
$username = "root";
$password = "pass";
$dbname = "dbname";
function parseXml($mysql)
{
$drugs = array();
$xmlReader = new XMLReader();
$xmlReader->open(DRUG_XML_FILE);
// Move our pointer to the first <drug /> element.
while ($xmlReader->read() && $xmlReader->name !== 'drug') ;
$drugCount = 0;
$totalDrugs = 0;
// Iterate over the outer <drug /> elements.
while ($xmlReader->name == 'drug')
{
// Convert the node into a SimpleXMLElement for ease of use.
$item = new SimpleXMLElement($xmlReader->readOuterXML());
$name = $item->name;
$description = $item->description;
$casNumber = $item->{'cas-number'};
$created = $item['created'];
$updated = $item['updated'];
$type = $item['type'];
$drugs[] = "('$name', '$description','$casNumber','$created','$updated','$type')";
$drugCount++;
$totalDrugs++;
// Once we've reached the desired batch size, insert the batch and reset the counter.
if ($drugCount >= INSERT_BATCH_SIZE)
{
batchInsertDrugs($mysql, $drugs);
$drugCount = 0;
}
// Go to next <drug />.
$xmlReader->next('drug');
}
$xmlReader->close();
// Insert the leftovers from the last batch.
batchInsertDrugs($mysql, $drugs);
echo "Inserted $totalDrugs total drugs.";
}
function batchInsertDrugs($mysql, &$drugs)
{
// Generate a batched INSERT statement.
$statement = "INSERT INTO `drug` (name, description, cas_number, created, updated, type) VALUES";
$statement = $statement . ' ' . implode(",\n", $drugs);
echo $statement, "\n";
// Run the batch INSERT.
if ($mysql->query($statement))
{
echo "Inserted " . count($drugs) . " drugs.";
}
else
{
echo "INSERT Error: " . $statement . "<br>" . $mysql->error. "<br>" ;
}
// Clear the buffer.
$drugs = array();
}
// Create MySQL connection.
$mysql = new mysqli($servername, $username, $password, $dbname);
if ($mysql->connect_error)
{
die("Connection failed: " . $mysql->connect_error);
}
parseXml($mysql);
I tested this example using the same dataset.
Using SimpleXML in the way that you are leads to parsing the entire document in memory, which is slow and memory-intensive. This approach uses XMLReader, which is a fast pull-parser. You can probably make this faster still using the PHP SAX XML Parser, but it's a bit more complex of a pattern, and the above example will be noticeably better than what you started with.
The other significant change in my example is that we're using MySQL Batched Inserts, so we only actually hit the database every 500 (configurable) items we process. You can tweak this number for better performance. After a certain point, the query will become too large for MySQL to process, but you may be able to do a lot more than 500 at one time.
If you'd like me to explain any part of this further, or if you have any problems with it, just let me know in the comments! :)
I am using this same code `
php $postId = 41;
<!-- hidden items and variables. Elements that will not be revealed !-->
<span id="gameLength"><?php
// MySQL connect configuration
$dbname="my_db";
$host="localhost";
$user="guessthe";
$dbh=mysql_connect ($host,$user,"correctPassword?") or die ('I cannot connect to the database because: ' . mysql_error(). '');
mysql_select_db ("$dbname") or die('I cannot select the database because: ' . mysql_error());
$sql="SELECT * FROM games WHERE postId = $postId";
$result=mysql_query($sql);
$rows=mysql_fetch_array($result);
$gameId = $rows['id'];
$game100s = $rows['game100s'];
$gamesPlayedAllTime = $rows['gamesPlayed'];
$gamesPointsAllTime = $rows['gameScore'];
$gameLength = $rows['gameLength']; // get number of questions
$gameScore = $rows['gameScore'];
$gameType = $rows['gameType'];
$gametitle = $rows['gameSubTitle'];
echo $gameLength;
There is a value in the gameLength row! I can't get this code to pull any of the rows! Any idea what i'm doing wrong?
You're using MySQL, which is depcirated - and will be phased out. You should use MySQLi or PDO instead. Also, your $postId is defined outside a PHP-tag? Might just be a copy/paste mistake? Anyway, you can try the code below, which is in MySQLi:
<?php
$postId = 41;
?>
<!-- hidden items and variables. Elements that will not be revealed !-->
<span id="gameLength"><?php
// MySQL connect configuration
$dbname = "my_db";
$host = "localhost";
$user = "guessthe";
// Connecting to the database
$mysqli = new mysqli($host, $user, "correctPassword?", $dbname);
if ($mysqli->connect_errno) {
// If we are here, the connection failed
echo "Failed to connect to MySQL: (".$mysqli->connect_errno.") ".$mysqli->connect_error;
}
$sql ="SELECT * FROM games WHERE postId = $postId";
if ($result = $mysqli->query($sql)) {
// If the query was sucsessfull, we can get the rows
while ($row = $result->fetch_assoc()) {
$gameId = $row['id'];
$game100s = $row['game100s'];
$gamesPlayedAllTime = $row['gamesPlayed'];
$gamesPointsAllTime = $row['gameScore'];
$gameLength = $row['gameLength']; // get number of questions
$gameScore = $row['gameScore'];
$gameType = $row['gameType'];
$gametitle = $row['gameSubTitle'];
}
} else {
// If the query failed, do something here
}
echo $gameLength;
?>
I see some people commenting that you need to put the $postId variable inside quotes in the query, but when using double-quotes (") variables will be posted, so it's not really needed. Also note that things are case-sensitive, so if your results doesn't show, check for spelling-mistakes.
There are many errors in your code
Try this...
<?php
$postId = 41;
?>
<!-- hidden items and variables. Elements that will not be revealed !-->
<span id="gameLength">
<?php
// MySQL connect configuration
$host = "localhost";
$dbname = "my_db";
$user = "username";
$password = "password";
$dbh = mysql_connect ($host,$user,$password) or die ('I cannot connect to the database because: ' . mysql_error() . '');
mysql_select_db($dbname, $dbh) or die('I cannot select the database because: ' . mysql_error());
$sql = "SELECT * FROM games WHERE postId='$postId'";
$result = mysql_query($sql);
while($rows = mysql_fetch_array($result)){
$gameId = $rows['id'];
$game100s = $rows['game100s'];
$gamesPlayedAllTime = $rows['gamesPlayed'];
$gamesPointsAllTime = $rows['gameScore'];
$gameLength = $rows['gameLength']; // get number of questions
$gameScore = $rows['gameScore'];
$gameType = $rows['gameType'];
$gametitle = $rows['gameSubTitle'];
echo $gameLength;
}
?>
You need to fix this is your code and that should fix the error.
$sql="SELECT * FROM games WHERE postId ='".$postId."' ";
If you want all the records you can use a while loop. Here is some pseudo code.
while($row = mysql_fect_assoc($query)){
echo $row["THE THING YOU WANT"];
...
}
The script below seems to stall for no reason during the loops. There are 6700 results to loop through, but it stalls between 375-460 results processed. No errors given or logged.
<?php
/* Example usage of the Amazon Product Advertising API */
include("amazon_api_class.php");
ignore_user_abort(true);
set_time_limit(0);
ini_set('memory_limit','256M');
ini_set("display_errors", 1);
ini_set("track_errors", 1);
ini_set("html_errors", 1);
error_reporting(E_ALL);
// define variables
$username = "xxxxxxx";
$password = "xxxxxxx";
$hostname = "localhost";
//connection to the database
$dbhandle = mysql_connect($hostname, $username, $password)
or die("Unable to connect to MySQL");
//echo "Connected to MySQL<br>";
//select a database to work with
mysql_select_db("xxxxxxxx",$dbhandle)
or die("Could not select database");
//execute the SQL query and return records
$result = mysql_query("SELECT * FROM input ORDER BY mpn");
if (!$result) { // add this check.
die('Invalid query: ' . mysql_error());
}
//clear results table before processing
mysql_query("TRUNCATE TABLE results");
mysql_close($dbhandle);
//loop
while($row = mysql_fetch_array($result)) {
$mfg = $row['mfg'];
$mpn = $row['mpn'];
$keyword = $mfg." ".$mpn;
$obj = new AmazonProductAPI();
try{
$data = $obj->getItemByKeyword($keyword);
$asin = $data->Items->Item->ASIN;
$weight = $data->Items->Item->ItemAttributes->PackageDimensions->Weight;
$height = $data->Items->Item->ItemAttributes->PackageDimensions->Height;
$length = $data->Items->Item->ItemAttributes->PackageDimensions->Length;
$width = $data->Items->Item->ItemAttributes->PackageDimensions->Width;
$manufacturer = $data->Items->Item->ItemAttributes->Manufacturer;
$brand = $data->Items->Item->ItemAttributes->Brand;
$partnumber = $data->Items->Item->ItemAttributes->MPN;
$title = $data->Items->Item->ItemAttributes->Title;
$upc = $data->Items->Item->ItemAttributes->UPC;
$ean = $data->Items->Item->ItemAttributes->EAN;
$pricenew = $data->Items->Item->OfferSummary->LowestNewPrice->FormattedPrice;
$priceused = $data->Items->Item->OfferSummary->LowestUsedPrice->FormattedPrice;
$description = $data->Items->Item->ItemAttributes->Feature;
$title = $data->Items->Item->ItemAttributes->Title;
echo ' processing '.$counter++;
//connection to the database
$dbhandle1 = mysql_connect($hostname, $username, $password)
or die("Unable to connect to MySQL");
//select a database to work with
mysql_select_db("xxxxxxxx",$dbhandle1)
or die("Could not select db");
//execute the SQL query and return records
mysql_query('INSERT INTO `results`(`mpn`, `mfg`, `brand`, `asin`, `ean`, `upc`, `description`, `low_price_new`, `low_price_used`, `length`, `width`, `height`, `net_weight`, `gross_weight`, `shipped_weight`, `title`) VALUES ("'.$partnumber.'","'.$manufacturer.'","'.$brand.'","'.$asin.'","'.$ean.'","'.$upc.'","'.$description.'","'.$pricenew.'","'.$priceused.'","'.$length.'","'.$width.'","'.$height.'","'.$weight.'","'.$weight.'","'.$weight.'","'.$title.'")');
mysql_error();
mysql_close($dbhandle1);
//unset variables
unset($asin);
unset($weight);
unset($height);
unset($length);
unset($width);
unset($manufacturer);
unset($brand);
unset($partnumber);
unset($packagequantity);
unset($title);
unset($upc);
unset($data);
unset($keyword);
//flush
flush();
//force garbage collection
gc_enable();
gc_collect_cycles();
usleep(50000);
}
catch(Exception $e)
{
//echo $e->getMessage();
}
//print_r($result);
}
echo "<br><h3>Job Completed</h3><br><br><h1><a href='csv_amz.php'>Download Results As CSV File</a></h1><br><br><h1><a href='index.php'>Upload New File</a></h1>";
?>
My primary concern is to get this script to run through the entire list. When testing with 50-100 results, works without any problem.
As a secondary question, within the loop I have an echo with a counter. I would like to get that to display on screen in real time, instead of all at once when script finishes.
This is a follow up question , earlier i had asked
about inserting json into mysql . I encoded it again and now i want it to be printed back to mysql . I don't know how am i supposed to print the encoded json output as string back into mysql . Folowing is my current code
<?php
$json = array
(
array("pineapple","yellow"),
array("watermelon","red"),
array("orange","orange")
);
var_dump($json);
var_dump(json_decode($json, true));
$newelements = json_encode( $json, JSON_FORCE_OBJECT | JSON_UNESCAPED_UNICODE );
echo $newelements;
$username = "root";
$password = "";
$hostname = "localhost";
$dbhandle = mysql_connect($hostname, $username, $password)
or die("Unable to connect to MySQL");
echo "Connected to MySQL<br>";
$selected = mysql_select_db("json",$dbhandle)
or die("Could not select json");
// foreach ($enc as $fruit => $color) {
$db_insert = mysql_query("INSERT INTO fruits (fruit,color) VALUES('$fruit','$color')");
mysql_query($db_insert);
if (!$db_insert)
{
die('Could not connect - event insert failed: ' . mysql_error());
}
// }
?>
Any help would be much appreciated .Thanks in advance :)
Because you have an array of arrays, the correct foreach would look like this:
$values = array();
foreach ($newelement as $element) {
$values[] = "('".mysql_real_escape_string($element[0])."','".mysql_real_escape_string($element[1])."')";
}
$db_insert = mysql_query("INSERT INTO fruits (fruit,color) VALUES ".implode(",", $values);