PHP Script Stalls During Loops - php

The script below seems to stall for no reason during the loops. There are 6700 results to loop through, but it stalls between 375-460 results processed. No errors given or logged.
<?php
/* Example usage of the Amazon Product Advertising API */
include("amazon_api_class.php");
ignore_user_abort(true);
set_time_limit(0);
ini_set('memory_limit','256M');
ini_set("display_errors", 1);
ini_set("track_errors", 1);
ini_set("html_errors", 1);
error_reporting(E_ALL);
// define variables
$username = "xxxxxxx";
$password = "xxxxxxx";
$hostname = "localhost";
//connection to the database
$dbhandle = mysql_connect($hostname, $username, $password)
or die("Unable to connect to MySQL");
//echo "Connected to MySQL<br>";
//select a database to work with
mysql_select_db("xxxxxxxx",$dbhandle)
or die("Could not select database");
//execute the SQL query and return records
$result = mysql_query("SELECT * FROM input ORDER BY mpn");
if (!$result) { // add this check.
die('Invalid query: ' . mysql_error());
}
//clear results table before processing
mysql_query("TRUNCATE TABLE results");
mysql_close($dbhandle);
//loop
while($row = mysql_fetch_array($result)) {
$mfg = $row['mfg'];
$mpn = $row['mpn'];
$keyword = $mfg." ".$mpn;
$obj = new AmazonProductAPI();
try{
$data = $obj->getItemByKeyword($keyword);
$asin = $data->Items->Item->ASIN;
$weight = $data->Items->Item->ItemAttributes->PackageDimensions->Weight;
$height = $data->Items->Item->ItemAttributes->PackageDimensions->Height;
$length = $data->Items->Item->ItemAttributes->PackageDimensions->Length;
$width = $data->Items->Item->ItemAttributes->PackageDimensions->Width;
$manufacturer = $data->Items->Item->ItemAttributes->Manufacturer;
$brand = $data->Items->Item->ItemAttributes->Brand;
$partnumber = $data->Items->Item->ItemAttributes->MPN;
$title = $data->Items->Item->ItemAttributes->Title;
$upc = $data->Items->Item->ItemAttributes->UPC;
$ean = $data->Items->Item->ItemAttributes->EAN;
$pricenew = $data->Items->Item->OfferSummary->LowestNewPrice->FormattedPrice;
$priceused = $data->Items->Item->OfferSummary->LowestUsedPrice->FormattedPrice;
$description = $data->Items->Item->ItemAttributes->Feature;
$title = $data->Items->Item->ItemAttributes->Title;
echo ' processing '.$counter++;
//connection to the database
$dbhandle1 = mysql_connect($hostname, $username, $password)
or die("Unable to connect to MySQL");
//select a database to work with
mysql_select_db("xxxxxxxx",$dbhandle1)
or die("Could not select db");
//execute the SQL query and return records
mysql_query('INSERT INTO `results`(`mpn`, `mfg`, `brand`, `asin`, `ean`, `upc`, `description`, `low_price_new`, `low_price_used`, `length`, `width`, `height`, `net_weight`, `gross_weight`, `shipped_weight`, `title`) VALUES ("'.$partnumber.'","'.$manufacturer.'","'.$brand.'","'.$asin.'","'.$ean.'","'.$upc.'","'.$description.'","'.$pricenew.'","'.$priceused.'","'.$length.'","'.$width.'","'.$height.'","'.$weight.'","'.$weight.'","'.$weight.'","'.$title.'")');
mysql_error();
mysql_close($dbhandle1);
//unset variables
unset($asin);
unset($weight);
unset($height);
unset($length);
unset($width);
unset($manufacturer);
unset($brand);
unset($partnumber);
unset($packagequantity);
unset($title);
unset($upc);
unset($data);
unset($keyword);
//flush
flush();
//force garbage collection
gc_enable();
gc_collect_cycles();
usleep(50000);
}
catch(Exception $e)
{
//echo $e->getMessage();
}
//print_r($result);
}
echo "<br><h3>Job Completed</h3><br><br><h1><a href='csv_amz.php'>Download Results As CSV File</a></h1><br><br><h1><a href='index.php'>Upload New File</a></h1>";
?>
My primary concern is to get this script to run through the entire list. When testing with 50-100 results, works without any problem.
As a secondary question, within the loop I have an echo with a counter. I would like to get that to display on screen in real time, instead of all at once when script finishes.

Related

Count all rows in database where column equals to `YES`

I have a database crm_data in which I have multiple tables. Now I want to calculate all rows in whole database where column_name value is equal to YES. To calculate all rows in database I am using this mysql query.
My SQL Query:
$sql = $db_con2->prepare("SELECT SUM(TABLE_ROWS) AS all_rows FROM
INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = 'crm_data'");
$sql->execute();
$all_rows = $sql->fetchAll();
if (count($all_rows) > 0) {
foreach ($all_rows as $all_rows) {
echo $all_rows['all_rows'];
}
} else {
$all_rows = '0';
}
Thanks
What I did I went back to basics and used SHOW TABLES then I use table query SELECT * FROM $company WHERE rental_status = 'YES'. and I got the results.
BTW thanks guys giving me your suggestion.
Try this, i hope it works for you
// connect to Database.
$links = mysqli_connect($db_host, $db_username, $db_password, $db_name );
if($links === false){
die("ERROR: Could not connect. " . mysqli_connect_error());
}
//get total
$n = 'YES';
$result = mysqli_query($links,"SELECT Count(*) As column_name FROM crm_data WHERE column_name='".$n."'");
$rows = mysqli_num_rows($result);
if($rows){
$gt = mysqli_fetch_assoc($result);
$total = $gt["column_name"];
}
echo $total;
Using PDO as requested.
// using PDO
$servername = "localhost";
$username = "user";
$password = "pass";
try {
$conn = new PDO("mysql:host=$servername;dbname=dbname", $username, $password);
// set the PDO error mode to exception
$conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
//echo "Connected successfully";
// call for the row count here.
$w='YES';
$rs = $conn->prepare("SELECT Count(*) As column_name FROM crm_data WHERE column_name='".$w."'");
$rs->execute();
$count = $rs->rowCount();
echo '<p>Total:'.$count.'</p>';
}
catch(PDOException $e)
{
echo "Connection failed: " . $e->getMessage();
}
You're not querying the data in your database you're now querying the actual structure of the database, which is concerning.
Nevertheless...
SELECT SUM(`COLUMN_NAME`) FROM `INFORMATION_SCHEMA`.`COLUMNS` where `COLUMN_NAME` = 'YES' AND `TABLE_SCHEMA` = 'crm_data';
Is the query you are looking for...

Why is my json data not being INSERTed into mySQL database?

I created a PHP handler to receive a JSON payload from a POST request and then insert it into a database in phpMyAdmin. I'm not sure why this is not working.
JSON:
payload = {
"version":"1.0",
"event":"video_recorded",
"data":{
"videoName":"vs1457013120534_862",
"audioCodec":"NellyMoser ASAO",
"videoCodec":"H.264",
"type":"FLV",
"orientation":"landscape",
"id":"0",
"dateTime":"2016-03-03 15:51:44",
"timeZone":"Europe/Bucharest",
"payload":"111.111.111.11",
"httpReferer":"http://site_from_where_video_was_recorded.com"
}
}
The PHP code I got from a tutorial online. The tutorial was from 2017 so I'm assuming everything is up to date, but yet it still does not work:
<?php
/* db variables */
$dbhost = 'localhost';
$dbname = 'name_db';
$dbuser = 'user_db';
$dbpass = 'pass_db';
/* grab the json */
$data = $_POST['payload'];
/* put json into php associative array */
$data_array = json_decode($data);
/* store in PHP variables */
$ip_address = $data_array['data']['payload'];
$vid_name = $data_array['data']['videoName'];
$date_time = $data_array['data']['dateTime'];
$time_zone = $data_array['data']['timeZone'];
/* connect to mysql db */
$con = mysql_connect($dbuser, $dbpass, $dbhost) or die('Could not connect: ' . mysql_error());
/* select the specific db */
mysql_select_db($dbname, $con);
/* insert the values into the db */
$sql = "INSERT INTO ip_and_videos(IpAddress, VideoName, DateTime, Timezone) VALUES('$ip_address','$vid_name','$date_time','$time_zone')";
if(!mysql_query($sql,$con))
{
die('Error : ' . mysql_error());
}
?>
I have the primary key set to an int and have it on auto increment. If I understand correctly I don't need to insert anything into that column because it will assign a number each time. Or do I still need to pass it when I INSERT the other variables?
This works for the array part, you get correct answer. So, your code is not bad, but you should check all errors (as stated by Bhavin in comment). And I'm retty sure you have a typo -> $vid_name = $data_array['data']['videName']; is NOT like $vid_name = $data_array['data']['videoName']; Thereforer, error_reporting will be very helpful, and after that, check the query if other errors (prepared statements ^^)
<?php
error_reporting(E_ALL);
ini_set('display_errors', 1);
$payload = '{
"version":"1.0",
"event":"video_recorded",
"data": {
"videoName":"vs1457013120534_862",
"audioCodec":"NellyMoser ASAO",
"videoCodec":"H.264",
"type":"FLV",
"orientation":"landscape",
"id":"0",
"dateTime":"2016-03-03 15:51:44",
"timeZone":"Europe/Bucharest",
"payload":"111.111.111.11",
"httpReferer":"http://site_from_where_video_was_recorded.com"
}
}';
$data_array = json_decode($payload, true);
/* store in PHP variables */
$ip_address = $data_array['data']['payload'];
$vid_name = $data_array['data']['videoName'];
$date_time = $data_array['data']['dateTime'];
$time_zone = $data_array['data']['timeZone'];
echo"[ $ip_address / $vid_name / $date_time / $time_zone ]";
// EDIT : added query
include"config.inc.php";
// connect to DB
$mysqli = mysqli_connect("$host", "$user", "$mdp", "$db");
if (mysqli_connect_errno()) { echo "Error connecting : " . mysqli_connect_error($mysqli); }
$query = " INSERT INTO ip_and_videos (`IpAddress`, `VideoName`, `DateTime`, `Timezone`) VALUES (?,?,?,?) ";
$stmt = $mysqli->prepare($query);
print_r($stmt->error_list);
$stmt->bind_param("ssss", $ip_address, $vid_name, $date_time, $time_zone );
if (!$stmt->execute()) { echo $stmt->error; } else { echo"true"; }
?>
Not sure if you had the same issue but this does not work for me...
$vid_name = $data_array['data']['videName'];
I had to use
$vid_name = $data_array->data->videoName;
Can't use stdClass as an array.
Should be better this way
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
/* db variables */
$dbhost = 'localhost';
$dbname = 'name_db';
$dbuser = 'user_db';
$dbpass = 'pass_db';
/* grab the json */
$data = $_POST['payload'];
/* put json into php associative array */
$alldata = json_decode($data,true);
print_r($alldata);
/* connect to mysql db */
$con = mysql_connect($dbuser, $dbpass, $dbhost) or die('Could not connect: ' . mysql_error());
/* select the specific db */
mysql_select_db($dbname, $con);
/* insert the values into the db */
foreach($alldata as $data_array) {
$ip_address = $data_array['data']['payload'];
$vid_name = $data_array['data']['videoName'];
$date_time = $data_array['data']['dateTime'];
$time_zone = $data_array['data']['timeZone'];
$sql = "INSERT INTO ip_and_videos(IpAddress, VideoName, DateTime, Timezone) VALUES('".$ip_address."','".$vid_name."','".$date_time."','".$time_zone."')";
mysql_query($sql);
echo mysql_errno($con) . ": " . mysql_error($con) . "\n";
}
?>
Hope this helps

I can't connect to db or pull data

I am using this same code `
php $postId = 41;
<!-- hidden items and variables. Elements that will not be revealed !-->
<span id="gameLength"><?php
// MySQL connect configuration
$dbname="my_db";
$host="localhost";
$user="guessthe";
$dbh=mysql_connect ($host,$user,"correctPassword?") or die ('I cannot connect to the database because: ' . mysql_error(). '');
mysql_select_db ("$dbname") or die('I cannot select the database because: ' . mysql_error());
$sql="SELECT * FROM games WHERE postId = $postId";
$result=mysql_query($sql);
$rows=mysql_fetch_array($result);
$gameId = $rows['id'];
$game100s = $rows['game100s'];
$gamesPlayedAllTime = $rows['gamesPlayed'];
$gamesPointsAllTime = $rows['gameScore'];
$gameLength = $rows['gameLength']; // get number of questions
$gameScore = $rows['gameScore'];
$gameType = $rows['gameType'];
$gametitle = $rows['gameSubTitle'];
echo $gameLength;
There is a value in the gameLength row! I can't get this code to pull any of the rows! Any idea what i'm doing wrong?
You're using MySQL, which is depcirated - and will be phased out. You should use MySQLi or PDO instead. Also, your $postId is defined outside a PHP-tag? Might just be a copy/paste mistake? Anyway, you can try the code below, which is in MySQLi:
<?php
$postId = 41;
?>
<!-- hidden items and variables. Elements that will not be revealed !-->
<span id="gameLength"><?php
// MySQL connect configuration
$dbname = "my_db";
$host = "localhost";
$user = "guessthe";
// Connecting to the database
$mysqli = new mysqli($host, $user, "correctPassword?", $dbname);
if ($mysqli->connect_errno) {
// If we are here, the connection failed
echo "Failed to connect to MySQL: (".$mysqli->connect_errno.") ".$mysqli->connect_error;
}
$sql ="SELECT * FROM games WHERE postId = $postId";
if ($result = $mysqli->query($sql)) {
// If the query was sucsessfull, we can get the rows
while ($row = $result->fetch_assoc()) {
$gameId = $row['id'];
$game100s = $row['game100s'];
$gamesPlayedAllTime = $row['gamesPlayed'];
$gamesPointsAllTime = $row['gameScore'];
$gameLength = $row['gameLength']; // get number of questions
$gameScore = $row['gameScore'];
$gameType = $row['gameType'];
$gametitle = $row['gameSubTitle'];
}
} else {
// If the query failed, do something here
}
echo $gameLength;
?>
I see some people commenting that you need to put the $postId variable inside quotes in the query, but when using double-quotes (") variables will be posted, so it's not really needed. Also note that things are case-sensitive, so if your results doesn't show, check for spelling-mistakes.
There are many errors in your code
Try this...
<?php
$postId = 41;
?>
<!-- hidden items and variables. Elements that will not be revealed !-->
<span id="gameLength">
<?php
// MySQL connect configuration
$host = "localhost";
$dbname = "my_db";
$user = "username";
$password = "password";
$dbh = mysql_connect ($host,$user,$password) or die ('I cannot connect to the database because: ' . mysql_error() . '');
mysql_select_db($dbname, $dbh) or die('I cannot select the database because: ' . mysql_error());
$sql = "SELECT * FROM games WHERE postId='$postId'";
$result = mysql_query($sql);
while($rows = mysql_fetch_array($result)){
$gameId = $rows['id'];
$game100s = $rows['game100s'];
$gamesPlayedAllTime = $rows['gamesPlayed'];
$gamesPointsAllTime = $rows['gameScore'];
$gameLength = $rows['gameLength']; // get number of questions
$gameScore = $rows['gameScore'];
$gameType = $rows['gameType'];
$gametitle = $rows['gameSubTitle'];
echo $gameLength;
}
?>
You need to fix this is your code and that should fix the error.
$sql="SELECT * FROM games WHERE postId ='".$postId."' ";
If you want all the records you can use a while loop. Here is some pseudo code.
while($row = mysql_fect_assoc($query)){
echo $row["THE THING YOU WANT"];
...
}

Mysql table using LIMIT, breaks randomly

When i run the following code it will return 100 users record from a table but when i increase the value of LIMIT from 100 to a greater number like 5000 then it will not return anything.i have total 6000 records in table. So how can i access different number of records like 2000, 3000 or even all 6000 records? kindly Guide what's wrong!
<?php
$db = mysql_connect("localhost","root","");
if (!$db) {
die('Could not connect to db: ' . mysql_error());}
mysql_select_db("distributedsms",$db);
$result = mysql_query("select * from users LIMIT 100", $db);
$json_response = array();
while ($row = mysql_fetch_array($result, MYSQL_ASSOC)) {
$row_array['user no'] = $row['sno'];
$row_array['mnc'] = $row['mnc'];
$row_array['mcc'] = $row['mcc'];
$row_array['lac'] = $row['lac'];
$row_array['cell id'] = $row['cell id'];
$row_array['lat'] = $row['lat'];
$row_array['lng'] = $row['lng'];
$row_array['address'] = $row['address'];//push the values in the array
array_push($json_response,$row_array);
$users = $json_response;}
$fp = fopen('users_data.json', 'w+');
fwrite($fp, json_encode($json_response));
fclose($fp);
echo json_encode($json_response);
?>
First, enable error reporting in your INI or Script:
<?php error_reporting(E_ALL);
ini_set('display_errors', 1);
This will help you if there is any error/warning in case of low memory allocation or similar.
To fetch all records, you shouldn't use LIMIT, remove LIMIT from your SQL.
i.e. select * from users
You should not use mysql_connect as it is deprecated. Use mysqli_connect instead. Also, I don't think you need to iterate so much, just choose what you want in your select statement.
UPDATE: another factor on why this might be happening can also be the returned information from the database, if you have apostrophes ' in your strings and you try to use json_encode, it will break, you would need to addslashes first.
Try the following, change your LIMIT as you wish, and let me know if you still have those errors:
<?php
$servername = "localhost";
$username = "username";
$password = "password";
$dbname = "myDB";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$sql = "SELECT sno,mnc,mcc,lac,cell_id,lat,lng,address FROM users LIMIT 100";
$result = $conn->query($sql);
if ($result->num_rows > 0) {
$response = array();
while ($row = $result->fetch_assoc()) {
foreach($row as $k => $str){
$row[$k] = addslashes($str);
}
array_push($response, $row);
}
echo json_encode($response);
} else {
echo "0 results";
}
$conn->close();
?>

Insert statement from php script to mysql: Database not reading insert statement

So everything up to the insert statement works perfectly. I know the database is connecting because I can select information from the database with the first two statements. I also know that the execute_statment3 works because no errors are being printed off and when it is put into the sql the statement is inserted the way it should be. Therefore the problem lies somewhere with the communication between the script and phpmyadmin. Please help I have been staring at this problem for two days and am going rather crazy.
<?php
session_start();
$hostname = 'localhost';
$username = '####';
$password = '####';
$connection = mysql_connect($hostname, $username, $password)
or die ('Connection error!!!');
$database = '####';
mysql_select_db($database);
$uid = $_SESSION['ID'];
$album = $_POST['albumname'];
$description = $_POST['description'];
$filename = $_FILES["upload_file"]["name"];
$filetype = $_FILES["upload_file"]["type"];
$filesize = $_FILES["upload_file"]["size"];
$file_on_server = $_FILES["upload_file"]["tmp_name"];
if ($filetype == "image/jpeg") {
$file_copy_name = date(m.d.y_H.i.s) . ".jpg";
copy($file_on_server, "uploads/" . $file_copy_name);
print "<br>";
print "<img src = \"uploads/$file_copy_name\">";
print "<br>";
$ret = system("pwd");
$picture = "uploads/$file_copy_name";
}
$execute_statement = "SELECT * FROM ImageAlbums WHERE Album = '$album'";
$results = mysql_query($execute_statement) or die ('Error executing SQL statement!!!');
while($item = mysql_fetch_array($results))
{
$album2 = $item['Album'];
}
if ($album2 == $album)
{
$execute_statement2 = "SELECT * FROM ImageAlbums WHERE Album = '$album'";
$results2 = mysql_query($execute_statement2) or die ('Error executing SQL statement2!!!');
while ($row2 = mysql_fetch_array($results2)) {
$AID = $row2["AlbumID"];
}
$execute_statement3 = "INSERT INTO Images (`ImageID`, `AlbumID`, `Description`, `Extensions`) VALUES ('NULL', '$AID', '$description', '$file_copy_name')";
($execute_statement3) or die ('Error executing SQL statement3!!!');
}
print "<br>";
print "<br>";
print $execute_statement3;
print "<br>";
print "<br>";
print $AID;
print "<br>";
print "<br>";
print $picture;
?>
I am using two databases for this script one of the databases is called ImageAlbums and has two columns called AlbumID and Album (AlbumID being a primary key). The second table is called Images and has four columns ImageID (primary key), AlbumID (foreign key), Description and Extensions.
You are not running the statement
($execute_statement3) or die ('Error executing SQL statement3!!!');
Try:
mysql_query($execute_statement3);
Also, make sure you escape all the variables.
make sure that the user you are connecting with in the php script has privileges for insert statements. you could be using a db user with only select privs...

Categories