I am writing an app that wishes to randomly assign a number to users, then puts in into a MySql database. There are many people who use it at the same time and as such I dont want parallel uses to overwite each other.
My current code is the following:
$sql_get = "SELECT * FROM database";
$results = mysql_query($sql_get, $bd);
$list = array();
while($row = mysql_fetch_array($results))
{
if ($row['userId'] == "")
{
array_push($list, $row['number']);
}
}
$rand_nums = array_rand($list , 1);
$sql_update = "UPDATE database SET userId='". $userId ."' WHERE number=". $rand_nums;
$results = mysql_query($sql_update, $bd);
So basically, it gets the empty rows, puts them into a list, chooses a random empty row number and puts the data into the row. The current issue is that the get and the empty rows can happen at the same time for multiple users and may overwrite data written at the same time.
How can I structure this code (transaction or otherwise) to ensure concurrent use has no bad effects?
Thank you
You could do it all in one query:
UPDATE database AS d
SET d.userId = $userId
WHERE d.userId = ''
ORDER BY RAND()
LIMIT 1
Related
$unique = array();
$sql = "SELECT ID, TitleName, ArtistDisplayName, Mix FROM values_to_insert as A
WHERE A.ID = ";
//Get a single row from our data that needs to be inserted...
while($result = $conn->query(($sql. $count)))
{
//Get the $data of the single row query for inserting.
$data = mysqli_fetch_row($result);
$count++;
//SQL to get a match of the single row of $data we just fetched...
$get_match = "SELECT TitleName_ti, Artist_ti, RemixName_ti from titles as B
Where B.TitleName_ti = '$data[1]'
and B.Artist_ti = '$data[2]'
and B.RemixName_ti = '$data[3]'
LIMIT 1";
//If this query returns a match, then push this data to our $unique value array.
if(!$result = $conn->query($get_match))
{
//If this data has been pushed already, (since our data includes repeats), then don't
//put a repeat of the data into our unique array. Else, push the data.
if(!in_array($unique, $data))
{
echo 'Pushed to array: ' . $data[0] . "---" . $data[1] . "</br>";
array_push($unique, $data);
}
else
echo'Nothing pushed... </br>';
}
}
This has taken 5+ minutes and nothing has even printed to screen. I'm curious as to what is eating up so much time and possibly an alternative method or function for whatever it is taking all this time up. I guess some pointers in the right direction would be great.
This code basically gets all rows, one at a time, of table 'A'. Checks if there is a match in table 'B', and if there is, then I don't want that $data, but if there isn't, I then check whether or not the data itself is a repeat because my table 'A' has some repeat values.
Table A has 60,000 rows
Table B has 200,000 rows
Queries within queries are rarely a good idea
But there appear to be multiple issues with your script. It might be easier to just do the whole lot in SQL and push the results to the array each time. SQL can remove the duplicates:-
<?php
$unique = array();
$sql = "SELECT DISTINCT A.ID,
A.TitleName,
A.ArtistDisplayName,
A.Mix
FROM values_to_insert as A
LEFT OUTER JOIN titles as B
ON B.TitleName_ti = A.ID
and B.Artist_ti = A.TitleName
and B.RemixName_ti = A.ArtistDisplayName
WHERE B.TitleName_ti IS NULL
ORDER BY a.ID";
if($result = $conn->query(($sql)))
{
//Get the $data of the single row query for inserting.
while($data = mysqli_fetch_row($result))
{
array_push($unique, $data);
}
}
As to your original query.
You have a count (I presume it is initialised to 0, but if a character then that will do odd things), and get the records with that value. If the first id was 1,000,000,000 then you have done 1b queries before you ever find a record to process. You can just get all the rows in ID order anyway by removing the WHERE clause and ordering by ID.
You then just get a single record from a 2nd query where the details match, but only process them if no record is found. You do not use any of the values that are returned. You can do this by doing a LEFT OUTER JOIN to get matches, and checking that there was no match in the WHERE clause.
EDIT - as you have pointed out, the fields you appear to be using to match the records do not appear to logically match. I have used them as you did but I expect you really want to match B.TitleName_ti to A.TitleName, B.Artist_ti to A.ArtistDisplayName and B.RemixName_ti to A.Mix
I have a php script that displays records from a database. It's probably not the best script, as I'm very new to php.
I've added an additional column in my table and would like to keep a count in that column to show me how many times each of the records have been viewed.
Heres the part of the code I think i need to add the code to... if i need to post the entire page i will, but i just figured i could add the line to this part.
//Get the details from previous page
$SelectedCounty = $_POST["result"];
//set variable for next SEARCH
$option = '';
// Get the county names from database - no duplicates - Order A-Z
$query = "SELECT DISTINCT tradingCounty FROM offers ORDER BY tradingCounty ASC";
// execute the query, $result will hold all of the Counties in an array
$result = mysqli_query($con,$query);
while($row = mysqli_fetch_array($result)) {
$option .="<option>" . $row['tradingCounty'] . "</option>";
}
}
the new column name is 'views' and i just want to add 1 to it each time a record from the database is viewed.
any help greatly appreciated.
Add a new field views to the table.
When, user views the page, fire the SQL.
$query = "UPDATE offers SET views = views + 1";
mysqli_query($con,"update offers set views = views + 1");
If you have added the column, it probably has a NULL value. Either set the value to 0, by doing:
update offers
set views = 0;
Or use:
update offers
set views = coalesce(views, 0) + 1;
You can change your code with this rewritten code assuming that your Table has a column views (datatype int).
//Get the details from previous page
$SelectedCounty = $_POST["result"];
//set variable for next SEARCH
$option = '';
// Get the county names from database - no duplicates - Order A-Z
$query = "SELECT DISTINCT tradingCounty FROM offers ORDER BY tradingCounty ASC";
// execute the query, $result will hold all of the Counties in an array
$result = mysqli_query($con,$query);
if($result){
$query2 = "UPDATE offers SET views=views+1;
mysqli_query($con,$query2);
}
while($row = mysqli_fetch_array($result)) {
$option .="<option>" . $row['tradingCounty'] . "</option>";
}
Or if you need to track the view counts for individual records, you need to modify your code a bit. And probably you need to add one more field in the database for eg. id (datatype int) which can distinguish between different records.
Please clear your problem properly.
As far as i have analysed your code it brings out the following case.
There are different records for tradingConty, and whenever a user views that particular record(one of the tradingCounty record) by clicking that or any other action specified, the php script is set to increament the view count for that particular entry(we can get that by id) in the database.
If thats the scenario, we can easily generate a code accordingly.
I need to synchronize specific information between two databases (one mysql, the other a remote hosted SQL Server database) for thousands of rows. When I execute this php file it gets stuck/timeouts after several minutes I guess, so I wonder how I can fix this issue and maybe also optimize the way of "synchronizing" it.
What the code needs to do:
Basically I want to get for every row (= one account) in my database which gets updated - two specific pieces of information (= 2 SELECT queries) from another SQL Server database. Therefore I use a foreach loop which creates 2 SQL queries for each row and afterwards I update those information into 2 columns of this row. We talk about ~10k Rows which needs to run thru this foreach loop.
My idea which may help?
I have heard about things like PDO Transactions which should collect all those queries and sending them afterwards in a package of all SELECT queries, but I have no idea whether I use them correctly or whether they even help in such cases.
This is my current code, which is timing out after few minutes:
// DBH => MSSQL DB | DB => MySQL DB
$dbh->beginTransaction();
// Get all referral IDs which needs to be updated:
$listAccounts = "SELECT * FROM Gifting WHERE refsCompleted <= 100 ORDER BY idGifting ASC";
$ps_listAccounts = $db->prepare($listAccounts);
$ps_listAccounts->execute();
foreach($ps_listAccounts as $row) {
$refid=$row['refId'];
// Refsinserted
$refsInserted = "SELECT count(username) as done FROM accounts WHERE referral='$refid'";
$ps_refsInserted = $dbh->prepare($refsInserted);
$ps_refsInserted->execute();
$row = $ps_refsInserted->fetch();
$refsInserted = $row['done'];
// Refscompleted
$refsCompleted = "SELECT count(username) as done FROM accounts WHERE referral='$refid' AND finished=1";
$ps_refsCompleted = $dbh->prepare($refsCompleted);
$ps_refsCompleted->execute();
$row2 = $ps_refsCompleted->fetch();
$refsCompleted = $row2['done'];
// Update fields for local order db
$updateGifting = "UPDATE Gifting SET refsInserted = :refsInserted, refsCompleted = :refsCompleted WHERE refId = :refId";
$ps_updateGifting = $db->prepare($updateGifting);
$ps_updateGifting->bindParam(':refsInserted', $refsInserted);
$ps_updateGifting->bindParam(':refsCompleted', $refsCompleted);
$ps_updateGifting->bindParam(':refId', $refid);
$ps_updateGifting->execute();
echo "$refid: $refsInserted Refs inserted / $refsCompleted Refs completed<br>";
}
$dbh->commit();
You can do all of that in one query with a correlated sub-query:
UPDATE Gifting
SET
refsInserted=(SELECT COUNT(USERNAME)
FROM accounts
WHERE referral=Gifting.refId),
refsCompleted=(SELECT COUNT(USERNAME)
FROM accounts
WHERE referral=Gifting.refId
AND finished=1)
A correlated sub-query is essentially using a sub-query (query within a query) that references the parent query. So notice that in each of the sub-queries I am referencing the Gifting.refId column in the where clause of each sub-query. While this isn't the best for performance because each of those sub-queries still has to run independent of the other queries, it would perform much better (and likely as good as you are going to get) than what you have there.
Edit:
And just for reference. I don't know if a transaction will help here at all. Typically they are used when you have several queries that depend on each other and to give you a way to rollback if one fails. For example, banking transactions. You don't want the balance to deduct some amount until a purchase has been inserted. And if the purchase fails inserting for some reason, you want to rollback the change to the balance. So when inserting a purchase, you start a transaction, run the update balance query and the insert purchase query and only if both go in correctly and have been validated do you commit to save.
Edit2:
If I were doing this, without doing an export/import this is what I would do. This makes a few assumptions though. First is that you are using a mssql 2008 or newer and second is that the referral id is always a number. I'm also using a temp table that I insert numbers into because you can insert multiple rows easily with a single query and then run a single update query to update the gifting table. This temp table follows the structure CREATE TABLE tempTable (refId int, done int, total int).
//get list of referral accounts
//if you are using one column, only query for one column
$listAccounts = "SELECT DISTINCT refId FROM Gifting WHERE refsCompleted <= 100 ORDER BY idGifting ASC";
$ps_listAccounts = $db->prepare($listAccounts);
$ps_listAccounts->execute();
//loop over and get list of refIds from above.
$refIds = array();
foreach($ps_listAccounts as $row){
$refIds[] = $row['refId'];
}
if(count($refIds) > 0){
//implode into string for use in query below
$refIds = implode(',',$refIds);
//select out total count
$totalCount = "SELECT referral, COUNT(username) AS cnt FROM accounts WHERE referral IN ($refIds) GROUP BY referral";
$ps_totalCounts = $dbh->prepare($totalCount);
$ps_totalCounts->execute();
//add to array of counts
$counts = array();
//loop over total counts
foreach($ps_totalCounts as $row){
//if referral id not found, add it
if(!isset($counts[$row['referral']])){
$counts[$row['referral']] = array('total'=>0,'done'=>0);
}
//add to count
$counts[$row['referral']]['total'] += $row['cnt'];
}
$doneCount = "SELECT referral, COUNT(username) AS cnt FROM accounts WHERE finished=1 AND referral IN ($refIds) GROUP BY referral";
$ps_doneCounts = $dbh->prepare($doneCount);
$ps_doneCounts->execute();
//loop over total counts
foreach($ps_totalCounts as $row){
//if referral id not found, add it
if(!isset($counts[$row['referral']])){
$counts[$row['referral']] = array('total'=>0,'done'=>0);
}
//add to count
$counts[$row['referral']]['done'] += $row['cnt'];
}
//now loop over counts and generate insert queries to a temp table.
//I suggest using a temp table because you can insert multiple rows
//in one query and then the update is one query.
$sqlInsertList = array();
foreach($count as $refId=>$count){
$sqlInsertList[] = "({$refId}, {$count['done']}, {$count['total']})";
}
//clear out the temp table first so we are only inserting new rows
$truncSql = "TRUNCATE TABLE tempTable";
$ps_trunc = $db->prepare($truncSql);
$ps_trunc->execute();
//make insert sql with multiple insert rows
$insertSql = "INSERT INTO tempTable (refId, done, total) VALUES ".implode(',',$sqlInsertList);
//prepare sql for insert into mssql
$ps_insert = $db->prepare($insertSql);
$ps_insert->execute();
//sql to update existing rows
$updateSql = "UPDATE Gifting
SET refsInserted=(SELECT total FROM tempTable WHERE refId=Gifting.refId),
refsCompleted=(SELECT done FROM tempTable WHERE refId=Gifting.refId)
WHERE refId IN (SELECT refId FROM tempTable)
AND refsCompleted <= 100";
$ps_update = $db->prepare($updateSql);
$ps_update->execute();
} else {
echo "There were no reference ids found from \$dbh";
}
Currently I have a MySQL query I run in PHP and when going through the results I update the original table but a simple table with 500 rows takes 30 seconds to complete:
$sqlquery = mysql_query('SELECT id, special_data FROM stats_visits WHERE processed = 0');
while($row = mysql_fetch_assoc($sqlquery)){
$stat_id = $row['id'];
// Make use of special_data field for some operations
mysql_query('UPDATE stats_visits SET processed = 1 WHERE id = ' . $stat_id);
}
Is it because I am updating the table from which I am selecting? I've solved this by doing the following but because the table might have thousands of records in future I'm unsure how well the IN will hold up:
$statids = array();
$sqlquery = mysql_query('SELECT id, special_data FROM stats_visits WHERE processed = 0');
while($row = mysql_fetch_assoc($sqlquery)){
$statids[] = $row['id'];
// Make use of special_data field for some operations
}
mysql_query('UPDATE stats_visits SET processed = 1 WHERE id IN(' . implode(',', $statids) . ')');
If all you want is to update all the records to being processed = 1 Why not resolve it with a single query?:
UPDATE `stats_visits` SET `processed` = 1 WHERE `processed` = 0;
In the first version you're hitting the database once for every single update. This is normally a bad idea. The 2nd version is hitting the database once.
If you are concerned about how this will work out in the future, perhaps create a hybrid. Batch the updates up into groups of 100 and update them in blocks.
The easiest way would be to store all of the ids which need updating as an array then implode these into the query with the IN operator.
e.g.
$processedArray= array();
while($row = mysql_fetch_assoc($sqlquery)){
$stat_id = $row['id'];
//Your processing goes here
$processedArray[] = $stat_id; //Store an id if it needs updating as processed
}
mysql_query('UPDATE stats_visits SET processed = 1 WHERE id IN ('.implode(',', $processedArray).')');
Hey, I have a field called STATUS and it is either 1 to show or 0 to hide. My code is below. I am using an edit in place editor with jQuery. Everytime you update it creates a new ROW which I want, but I want only the new one to have STATUS = 1 and the others to 0. Any ideas on how I would do that?
<?php
include "../../inc/config.inc.php";
$temp = explode("_", $_REQUEST['element_id'] );
$field = $temp[0];
$id = $temp[1];
$textboxval = stripslashes(mysql_real_escape_string(preg_replace('/[\$]/',"",$_REQUEST["update_value"])));
$query = "INSERT INTO notes ($field,status,date,c_id) VALUES ('$textboxval','1',NOW(),'$id')";
mysql_query($query);
echo($_REQUEST['update_value']);
?>
I am not sure exactly what you mean - do you want to make all the entries except the new one have status = 0? If so, just issue an update before the insert:
UPDATE notes SET status = 0
However, I should also note that you have a potential SQL injection to worry about. By stripping slashes after applying "mysql real escape string", you are potentially allowing someone to put text in your SQL statement that will execute an arbitrary SQL statement.
Something like this, sorry for the post before, I mis read it the first time then went back:
<?php
include "../../inc/config.inc.php";
$temp = explode("_", $_REQUEST['element_id'] );
$field = $temp[0];
$id = $temp[1];
$textboxval = mysql_real_escape_stringstripslashes((preg_replace('/[\$]/',"",$_REQUEST["update_value"])));
// set older entries to 0 - to not show but show in history
$hide_notes = "UPDATE notes SET status = 0";
mysql_query($hide_notes);
// add new entry with status of 1 to show only latest note
$query = "INSERT INTO notes ($field,status,date,c_id) VALUES ('$textboxval','1',NOW(),'$id')";
mysql_query($query);
echo($_REQUEST['update_value']);
?>
i just ran in to a problem I didn't of the set up of my table doesn't allow me to show more than one client a time and i will be having numerous clients, my bad on planning ha
You really want to get the ID of the newly generated row and then trigger an UPDATE where you all rows where the ID is not the new row, e.g.
UPDATE notes SET status = 0 WHERE id != $newly_generated_id
If the ID column in your table is using AUTO_INCREMENT you can get its ID via "SELECT LAST_INSERT_ID()" and then use the return value in that statement in your UPDATE statement.
Pseudo code:
$insert = mysql_query("INSERT INTO ...");
$last_id = mysql_query("SELECT LAST_INSERT_ID()");
$update = mysql_quqery("UPDATE notes SET status = 0 WHERE id != $last_id");
The only caveat to this approach is where you might have a brief moment in time where 2 rows have status=1 (the time between your INSERT and the UPDATE). I would wrap all of this in a transaction to make the whole unit more atomic.