How to perform that loop:
while ($row = mysql_fetch_array($result)) {
collectData($row['NAME']);
}
To not freeze my PHP after 30 seconds?
I think it can be done by taking whole Database into array or *.temp file to work with it, not to connect everytime I want to input/output something. But I have problems with files and arrays, so thats why I am asking: whats the better/best way to do it fast and painless?
<?php
//fiveMin - Database that data is taken form
//HighCharts - Database that data is transferred to
$fiveMin=mysql_connect($fiveMin_host,$fiveMin_user,$fiveMin_pass);
mysql_select_db($fiveMin_db,$fiveMin) or die (mysql_error());
$query="SELECT * FROM BetterShopItemStock";
$result = mysql_query($query);
while ($row = mysql_fetch_array($result)) {
collectData($row['NAME']);
}
function collectData($itemID) {
global $fiveMin_host, $fiveMin_user, $fiveMin_pass, $fiveMin_db, $week_host, $week_user, $week_pass, $week_db;
$fiveMin=mysql_connect($fiveMin_host,$fiveMin_user,$fiveMin_pass); //Load and store data from fiveMin Database
mysql_select_db($fiveMin_db,$fiveMin) or die (mysql_error());
$function_Query="SELECT AMT FROM BetterShopItemStock WHERE NAME = '$itemID'";
$function_Ask = mysql_query($function_Query);
$function_Result = mysql_fetch_row($function_Ask, 0);
$dataReadyToImport = "," . $function_Result[0];
#mysql_close($fiveMin);
$HighCharts=mysql_connect($week_host,$week_user,$week_pass); //Save stored data to weekly Database
mysql_select_db($week_db,$HighCharts) or die (mysql_error());
$function_Query="SELECT AMT FROM BetterShopItemStock WHERE NAME = '$itemID'";
$function_Ask = mysql_query($function_Query);
$function_Result = mysql_fetch_row($function_Ask, 0);
$storedData = $function_Result[0];
$dataReadyToImport = $storedData . $dataReadyToImport;
mysql_query("UPDATE BetterShopItemStock SET AMT='$dataReadyToImport' WHERE NAME='$itemID'");
#mysql_close($HighCharts);
}
?>
(1) You don't have to open and close a connection for every query - open one connection to each server and make multiple mysql_query() calls to them. That should speed you up quite a bit.
(2) Don't use "SELECT * FROM BetterShopItemStock" ... only select the row you need. (here, 'NAME'). And if you are already querying that table for 'NAME', then select 'AMT' at the same time, rather than making a second call to it.
(3) You can combine your select and update calls to HighCharts into a single query.
Altogether, everything should look something like this:
<?php
//fiveMin - Database that data is taken form
$fiveMin=mysql_connect($fiveMin_host,$fiveMin_user,$fiveMin_pass);
mysql_select_db($fiveMin_db,$fiveMin) or die (mysql_error());
//HighCharts - Database that data is transferred to
$HighCharts=mysql_connect($week_host,$week_user,$week_pass); //Save stored data to weekly Database
mysql_select_db($week_db,$HighCharts) or die (mysql_error());
$query="SELECT NAME,AMT FROM BetterShopItemStock";
$result = mysql_query($query,$fiveMin);
while ($row = mysql_fetch_row($result)) {
collectData($row[0],$row[1],$HighCharts);
}
function collectData($itemID, $itemAmt, $mysql) {
$id = mysql_real_escape_string($itemID);
$amt = mysql_real_escape_string($itemAmt);
$q = "UPDATE BetterShopItemStock SET ".
"AMT=CONCAT(AMT,',','$amt') WHERE NAME='$id'";
$r = mysql_query($q,$mysql);
return (!$r ? false : true);
}
//now close the databases
#mysql_close($HighCharts);
#mysql_close($fiveMin);
?>
(4) Lastly, if you are having execution timeout errors, look into PHP's set_time_limit to extend your execution time
Separate each one into classes and work with them after. It makes them run more efficiently and the querys do not intermesh:
<?php
class class1{
function fivemin(){
**code0**
}
function collectdata1-fivemin() {
class1::fivemin();
**code**
}
function collectdataHighcharts(){
class1::fivemin();
**code2**
}
}
$a = new class1;
$a->collectdata1-fivemin();
$a->collectdata1HighCharts();
?>
Related
I am trying to migrate ~55K records from a mySQL server over to MongoDB. I can't do this via any of the easily accessible methods like JSON/CSV importing because the data storage method (the way it is structured) will be very different. Because of this, I have created a script in php that is designed to do this.
The issue I have been running into with this is that over large sets of data (not reproducible using smaller data sets even when the smaller set includes problem entries) the queries will occasionally report no data despite the entry existing. It absolutely exists because when php accesses that specific entry directly or it is included in a smaller data set, it works just fine. For instance, in the import to a text file, i only received ~42k/54k records.
In the echo I am receiving through the url the php file is called through, I am showing that the query is called the correct number of times, but there are many records that are showing as not existing, thus the answering echo is blank. The code is included below:
//Makes a connection to the database
$conn = makeConnection();
$filename = '/home/dbserverdownload.txt';
$file = fopen($filename, 'a');
$sql = "SELECT * FROM maintable ORDER BY ID DESC LIMIT 1";
$resultID = mysqli_query($conn, $sql);
$ID = mysqli_fetch_object($resultID);
echo $ID->ID;
//loops through the database and appends the data to the file as it goes
for($var=2; $var <= $ID->ID; $var++){
$sql1 = "SELECT * FROM servertable WHERE ID = '$var'";
$result1 = mysqli_query($conn, $sql1);
$values = mysqli_fetch_object($result1);
$id = $values->ID;
$ip = $values->IP;
$port = $values->port;
$running = $values->running;
$afk = $values->afk;
$gamemode = $values->gamemode;
$maxplayers = $values->maxplayers;
$spawnprotection = $values->spawnprotection;
$whitelist = $values->whitelist;
$enablequery = $values->enablequery;
$enablercon = $values->enablercon;
$rconpassword = $values->rconpassword;
$motd = $values->motd;
$achievements = $values->announceplayerachievements;
$allowflight = $values->allowflight;
$spawnanimals = $values->spawnanimals;
$spawnmobs = $values->spawnmobs;
$forcegamemode = $values->forcegamemode;
$hardcore = $values->hardcore;
$pvp = $values->pvp;
$difficulty = $values->difficulty;
$generatorsettings = $values->generatorsettings;
$levelname = $values->levelname;
$levelseed = $values->levelseed;
$leveltype = $values->leveltype;
$autosave = $values->autosave;
if($ip == "148.57.44.10"){
//if the server is server1
$servername = "server1".$port;
} else if ($ip == "165.108.22.199"){
//if the server is server2
$servername = "server2".$port;
} else{
$servername = "";
}
//Adds all content that was already gained to the JSON string
$startingContent = "{\"_id\":\"$servername\",
\"ip\":\"$ip\",
\"port\":\"$port\",
\"running\":\"$running\",
\"afk\":\"$afk\",
\"gamemode\":\"$gamemode\",
\"maxplayers\":\"$maxplayers\",
\"spawnprotection\":\"$spawnprotection\",
\"whitelist\":\"$whitelist\",
\"enablequery\":\"$enablequery\",
\"enablercon\":\"$enablercon\",
\"rconpassword\":\"$rconpassword\",
\"motd\":\"$motd\",
\"announceplayerachievements\":\"$achievements\",
\"allowflight\":\"$allowflight\",
\"spawnanimals\":\"$spawnanimals\",
\"spawnmobs\":\"$spawnmobs\",
\"forcegamemode\":\"$forcegamemode\",
\"hardcore\":\"$hardcore\",
\"pvp\":\"$pvp\",
\"difficulty\":\"$difficulty\",
\"generatorsettings\":\"$generatorsettings\",
\"levelname\":\"$levelname\",
\"levelseed\":\"$levelseed\",
\"leveltype\":\"$leveltype\",
\"autosave\":\"$autosave\"
}";
echo $startingContent."<br/>";
//This is the JSON data that will be passed to mongo
if(strlen($ip)>6){
if (fwrite($file, $startingContent) === FALSE) {
echo "Cannot write to file ($filename) with $startingContent";
exit;
}
}
}
I have also tried this with a query that pulls a significant number(all, half, a quarter, etc) of the results in one chunk instead of tons of individual queries. The end result of that experiment was that a variable number of records were updated (usually a seemingly random number between 400 and 4000) each time it was run. Does anyone have any idea why this might be occurring? If not, should I just make my own program to iterate over the CSVs that I can export from mySQL?
First of all, replace your query $sql = "SELECT * FROM maintable ORDER BY ID DESC LIMIT 1";
with this $sql = "SELECT MAX(ID) as ID FROM maintable"; for better performance.
Your problem here is that you probably has a query that return false, then your application dies. Let's say, for example, your loop tries to query ID=3, but there is no row in your database with this ID. Then the next lines throws a exception, that you are not seeing. You could use set display_errors=On in you dev machine php.ini.
Use this to check if your have results to fetch:
....
for($var=2; $var <= $ID->ID; $var++){
$sql1 = "SELECT * FROM servertable WHERE ID = '$var'";
$result1 = mysqli_query($conn, $sql1);
if($result) {
$values = mysqli_fetch_object($result1);
$id = $values->ID;
$ip = $values->IP;
....
I am helping in some PHP design for a friends text game and have come to a stump.
I have scheduled a cron job to call the following page / following code, which is working correctly
<?php require("connect.php"); ?>
<?php
$sql = "SELECT id, name, health FROM users";
$query = mysql_query($sql) or die(mysql_error());
$row = mysql_fetch_object($query);
while($row = mysql_fetch_object($query)) {
$id = htmlspecialchars($row->id);
$name = htmlspecialchars($row->name);
$health = htmlspecialchars($row->health);
$sql = "SELECT * FROM property WHERE living='1' AND ownerid='$id'";
$query = mysql_query($sql) or die(mysql_error());
$row = mysql_fetch_object($query);
while($row = mysql_fetch_object($query)) {
$OwnerName = htmlspecialchars($row->ownername);
$OwnerID = htmlspecialchars($row->ownerid);
$RaidPropBonus = htmlspecialchars($row->raidperc);
$RaidPropMoney = htmlspecialchars($row->raidcash);
$PropertyLvl = htmlspecialchars($row->proplvl);
$Living = htmlspecialchars($row->living);
if($PropertyLvl == '5' && $Living == '1'){
if($health < '100'){
$result = mysql_query("UPDATE users SET health=$health + '1' WHERE id='$id'")
or die(mysql_error());
} else { }
} else { }
}
}
?>
Although this only works for ONE user only. I cannot understand why this is. Any other logged in / out accounts that have met the criteria have been ignored. I can maybe only think I am missing a loop? As the ID that is being met first is number 1 and it has stopped there?
Anybody advice at all maybe?
UPDATE - It seems correct I need to get a loop in there, but am so far failing to get this loop working correct. No matter where I seem to amend / add a loop it does not help. Please may somebody suggest anything?
UPDATE2 - As requested, updated with the new version of loop
For what I've understood, the loops should be made on the mysql_fetch_object that will get the each row from the query.
Take a look at the snippet
<?php
require("connect.php");
// here prepare the $userQuery (the one that fetches all users)
// then the first loop that will read each usew row
// AFAICT this should afect all script
while($userRow = mysql_fetch_object($userQuery))
{
// prepare data fetched from the $userQuery
// prepare the $propertyQuery (the one that fetches all properties of the user)
// then the second loop to read all user property rows
// and this will afect the updates
while($propertyRow = mysql_fetch_object($propertyQuery))
{
// prepare data fetched from $propertyQuery
// add logic here
}
}
?>
Also #Matthew Carpenter had a valid point, that mysql_* is deprecated, you should consider in using mysqli_*, or in my opinion take a look at PDO
i'm trying to make a long mysql query and process and update the row founded:
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
$result = mysql_query($query);
$total = mysql_num_rows($result);
echo $total;
while ($db_row = mysql_fetch_assoc($result))
{
//process row
}
but after 60 second give me a timeout request, i have try to insert these in my php code:
set_time_limit(400);
but it's the same, how i can do?
EDIT:
only the query:
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
takes 2-3 second to perform, so i think the problem is when in php i iterate all the result to insert to row or update it, so i think the problem is in the php, how i can change the timeout?
EDIT:
here is the complete code, i don't think is a problem here in the code...
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
$result = mysql_query($query);
$total = mysql_num_rows($result);
echo $total;
while ($db_row = mysql_fetch_assoc($result)) {
//print $db_row['id_show']."-".$db_row['actors']."<BR>";
$explode = explode("|", $db_row['actors']);
foreach ($explode as $value) {
if ($value != "") {
$checkactor = mysql_query(sprintf("SELECT id_actor,name FROM actors WHERE name = '%s'",mysql_real_escape_string($value))) or die(mysql_error());
if (mysql_num_rows($checkactor) != 0) {
$actorrow = mysql_fetch_row($checkactor);
$checkrole = mysql_query(sprintf("SELECT id_show,id_actor FROM actor_role WHERE id_show = %d AND id_actor = %d",$db_row['id_show'],$actorrow[0])) or die(mysql_error());
if (mysql_num_rows($checkrole) == 0) {
$insertactorrole = mysql_query(sprintf("INSERT INTO actor_role (id_show, id_actor) VALUES (%d, %d)",$db_row['id_show'],$actorrow[0])) or die(mysql_error());
}
} else {
$insertactor = mysql_query(sprintf("INSERT INTO actors (name) VALUES ('%s')",mysql_real_escape_string($value))) or die(mysql_error());
$insertactorrole = mysql_query(sprintf("INSERT INTO actor_role (id_show, id_actor, role) VALUES (%d, %d,'')",$db_row['id_show'],mysql_insert_id())) or die(mysql_error());
}
}
}
}
Should definitely try what #rid suggested, and to execute the query on the server and see the results/duration to debug - if the query is not a simple one, construct it as you would in your PHP script, and only echo the SQL command, don't have to execute it, and just copy that in to the server MySQL command line or whichever tool you use.
If you have shell access, use the top command after running the above script again, and see if the MySQL demon server is spiking in resources to see if it really is the cause.
Can you also try a simpler query in place of the longer one? Like just a simple SELECT count(*) FROM tvshows and see if that also takes a long time to return a value?
Hope these suggestions help.
There are so many problems with your code.
Don't store multiple values in a single column. Your actors column is pipe-delimited text. This is a big no-no.
Use JOINs instead of additional queries. You can (or could, if the above weren't true) get all of this data in a single query.
All of your code can be done in a single query on the server. As I see it, it takes no input from the user and produces no output. It just updates a table. Why do this in PHP? Learn about INSERT...SELECT....
Here are some resources to get you started (from Googling, but hopefully they'll be good enough):
http://www.sitepoint.com/understanding-sql-joins-mysql-database/
http://dev.mysql.com/doc/refman/5.1/en/join.html
http://dev.mysql.com/doc/refman/5.1/en/insert-select.html
What is Normalisation (or Normalization)?
Let me know if you have any further questions.
i have a problem my script has three mysql_query which should be used after each other , i am trying to create a script that reserve tickets by changing their status from sold = "No" to "Yes", the script count the number of tickets user has entered on html form which give the server side a variable with number of tickets called = $tickets.
hint : this is such a model so no need for mysql injection security
here is my code :
//get ticket status
$eventTicket = mysql_query("SELECT eventTickets FROM beventreservation WHERE eventId = '$eventId'") or die(mysql_error());
$ticketrow = mysql_fetch_array($eventTicket) or die(mysql_error());
//test... which is working !
echo $ticketrow['eventTickets'];
//get classId from classes
$selectClass = mysql_query("SELECT classId FROM quotaclasses WHERE className = '$classes' AND eventFK = '$eventId'") or die (mysql_error());
$classrow = mysql_fetch_array($selectClass) or die(mysql_error());
//this var is to define which class the user used
$choosedClass = $classrow['classId'];
//test ... which did not work !!!
echo $classrow['classId'];
if ($ticketrow['eventTickets'] == "Yes")
{
for($counter=1;$counter<$numberOfTickets;$counter++)
{
$bookTicket = mysql_query("UPDATE unites SET ticketSold = 'Yes' WHERE businessreservationIdFk = '$eventId' AND classIDfk ='$choosedClass'") or die(mysql_error());
echo "ticket ". $counter . " done !";
}
}
the script doesn't fetch this syntax, and there is no errors showed on my page !
$classrow = mysql_fetch_array($selectClass) or die(mysql_error());
also , i tried to echo the variable $tickets after this syntax , it did not showed up, is there a problem to fetch more than mysql_query on the same script page ? tell me where do i go wrong here please .
Don't call die() in conjunction with a mysql_fetch_*() call. If there are no rows returned, mysql_fetch_array() returns FALSE, which triggers your die() and kills your script even though there was no error. Since you have already don error checking on $selectClass in the mysql_query() call, you know it has succeeded.
// This query returned no rows, but was successful syntactically and functionally.
$selectClass = mysql_query("SELECT classId FROM quotaclasses WHERE className = '$classes' AND eventFK = '$eventId'") or die (mysql_error());
Instead, test if rows were returned:
if (mysql_num_rows($selectClass) > 0) {
// Fetch and do other stuff
$classrow = mysql_fetch_array($selectClass);
$choosedClass = $classrow['classId'];
// etc...
// etc...
}
else {
// Do whatever you need to do if no rows return
}
I have a webpage written in HTML. I have a dropdown list that is populated by a database utilizing a MySQL query:
<SELECT NAME = "Participant" STYLE = "WIDTH: 187" TITLE="Begin typing participant last name for fast searching." required>
<OPTION SELECTED VALUE = "">Select Participant...</OPTION>
<?PHP
$allParticipants = getall_participants();
foreach($allParticipants as &$value) {
$dt = date('Y-m-d');
$val = $value->get_id();
$optval = $dt.$val;
echo "<OPTION VALUE='",$optval,"'>";
echo $value->get_first_name()," ",$value->get_last_name();
echo "</OPTION>";
}
?>
</SELECT>
The getall_participants() looks like:
function getall_participants () {
connect();
$query = "SELECT * FROM dbParticipants ORDER BY last_name";
$result = mysql_query ($query);
$theParticipant = array();
while ($result_row = mysql_fetch_assoc($result)) {
$theParticipant = new Participant($result_row['last_name'],
$result_row['first_name'], $result_row['address']);
$theParticipants[] = $theParticipant;
}
mysql_close();
return $theParticipants;
}
And on this same page I have a textbox that is pre-filled-in by another database:
<?php
$dt = date('Y-m-d');
$participants = getall_dbParticipantEntry_byDate($dt);
foreach($participants as &$value) {
$a = $a.$value.", ";
}
echo "<INPUT TYPE='text' NAME='Participants' STYLE='WIDTH:50px;' TITLE='Participants' ";
echo "VALUE='[", $a.' ', "]'/>";
?>
That getall_dbParticipantEntry_byDate($date) looks like:
function getall_dbParticipantEntry_byDate($date) {
connect();
$query = 'SELECT * FROM dbParticipantEntry WHERE date = "'.$date.'"';
$result = mysql_query ($query);
$theParticipantEntry = array();
while ($result_row = mysql_fetch_assoc($result)) {
$theParticipantEntry = new ParticipantEntry($result_row['date'], $result_row['id'], $result_row['call_time'],
$result_row['result'], $result_row['notes']);
$theParticipantEntries[] = $theParticipantEntry->get_id();
}
mysql_close();
return $theParticipantEntries;
}
However, while both of these functions work fine individually, when they're both on the same webpage (like I meant them to be), only the one that comes first runs. I tested this by switching them in and out. They both complete their designated tasks, but only when alone on the page.
How can I get them both to run and populate their respective fields?
Thanks so much.
Try the following order:
Connect to mySQL server
Do task 1
Do task 2
Close Connection
For me it looks, like you have closed the mysqlconnection, before you do task2.
Edit:
Maybe you can do it like that?
function f1 ()
{
$res = mysql_connect(...);
// .. do some queries ..
mysql_query($sql, $res);
mysql_close($res )
}
function f2 ()
{
$res = mysql_connect(...);
// .. do some queries ..
mysql_query($sql, $res);
mysql_close($res )
}
Edit:
From php.net:
Be careful when using multiple links to connect to same database (with same username). Unless you specify explicitly in mysql_connect() to create a new link, it will return an already open link. If that would be closed by mysql_close(), it will also (obviously) close the other connection, since the link is the same.
Had lot of trouble figuring it out, since in <=4.3.6 there was a bug which didn't close the connection, but after the patch to >=4.3.7, all my application broke down because of a single script that did this.
You run them both on the same connection. You need to store the resource id returned from mysql_connect and pass this to each mysql method (each uses it's own relevant resource).
that said, I think it is time to:
Move to something more modern like Mysqli or PDO extensions. Much better API
Use some kind of abstraction on the connection managment, preferably one instance of a DB managment class per connection. Plenty of examples on the web, and it is way above the scope of this site to provide such instructions.