I am currently working on a simple auction site. I am storing bids in their own MySQL table called 'bids'. I am wondering what is the best way of ensuring that two of the same bids are not submitted at the exact same time.
My current strategy for verifying that the bid submitted is in fact the highest bid is to do the following (as an example):
$sql = "SELECT * FROM bids WHERE amount >= '".$bidamount."'";
$result = mysql_query($sql);
if(mysql_num_rows($result) == 0) {
$sql = "INSERT INTO bids SET amount = '".$bidamount."'";
mysql_query($sql);
$bidid = mysql_insert_id();
}
The problem with the above set of queries is that between the time the SELECT query is run and the INSERT query is run, another user could insert the same bid.
Is there some way to lock the table during the SELECT that would prevent this double-bidding from occurring? My main concern with locking tables for such a purpose would be performance problems when you have a lot of people bidding at once.
You may want to make conditional insert, like:
$amount = intval($amount);
$query = "
INSERT INTO
bids(amount)
SELECT
{$amount}
FROM
(SELECT 1) tmp_tbl
WHERE NOT EXISTS(
SELECT * FROM bids WHERE amount >= {$amount}
)
";
and check for affected (inserted) rows.
Related
I have a script that goes through all order history. It takes several minutes to print the results, but I noticed I perform several SQL statements that are similar enough I wonder if you could do another query on an existing SQL result.
For example:
-- first SQL request
SELECT * FROM orders
WHERE status = 'shipped'
Then, in a foreach loop, I want to find information from this result. My naive approach is to perform these three queries. Note the similarity to the query above.
-- grabs customer's LTD sales
SELECT SUM(total) FROM orders
WHERE user = :user
AND status = 'shipped'
-- grabs number of orders customer has made
SELECT COUNT(*) FROM orders
WHERE user = :user
AND status = 'shipped'
AND total != 0
-- grabs number of giveaways user has won
SELECT COUNT(*) FROM orders
WHERE user = :user
AND status = 'shipped'
AND total = 0
I end up querying the same table several times when the results I seek are subsets of the first query. I'd like to get information from the first query without performing more SQL calls. Some pseudocode:
$stmt1 = $db->prepare("
SELECT * FROM orders
WHERE status = 'shipped'
");
$stmt1->execute();
foreach($stmt1 as $var) {
$username = $var['username'];
$stmt2 = $stmt1->workOn("
SELECT SUM(total) FROM this
WHERE user = :user
");
$stmt2->execute(array(
':user' => $username
));
$lifesales = $stmt2->fetchColumn();
$stmt3 = $stmt1->workOn("
SELECT COUNT(*) FROM this
WHERE user = :user
AND total != 0
");
$stmt3->execute(array(
':user' => $username
));
$totalorders = $stmt3->fetchColumn();
$stmt4 = $stmt1->workOn("
SELECT COUNT(*) FROM this
WHERE user = :user
AND total = 0
");
$stmt4->execute(array(
':user' => $username
));
$totalgaws = $stmt4->fetchColumn();
echo "Username: ".$username;
echo "<br/>Lifetime Sales: ".$lifesales;
echo "<br/>Total Orders: ".$totalorders;
echo "<br/>Total Giveaways: ".$totalgaws;
echo "<br/><br/>";
}
Is something like this possible? Is it faster? My existing method is slow and ugly, I'd like a quicker way to do this.
We could do one pass through the table to get all three aggregates for all users:
SELECT s.user
, SUM(s.total) AS `ltd_sales`
, SUM(s.total <> 0) AS `cnt_prior_sales`
, SUM(s.total = 0) AS `cnt_giveaways`
FROM orders s
WHERE s.status = 'shipped'
GROUP
BY s.user
That's going to be expensive on large sets. But if we are needing that for all orders, for all users, that's likely going to be faster than doing separate correlated subqueries.
An index with leading column of user is going to allow MySQL to use the index for the GROUP BY operation. Including the status and total columns in the index will allow the query to be satisfied entirely from the index. (With the equality predicate on status column, we could also try an index with status as the leading column, followed by user column, then followed by total.
If we only need this result for a small subset of users e.g. we are fetching only the first 10 rows from the first query, then running a separate query is likely going to be faster. We'd just incorporate the condition WHERE s.user = :user into the query, as in the original code. But run just the one query rather than three separate queries.
We can combine that with the first query by making it into an inline view, wrapping it in parens and putting into the FROM clause as a row source
SELECT o.*
, t.ltd_sales
, t.cnt_prior_sale
, t.cnt_giveaways
FROM orders o
JOIN (
SELECT s.user
, SUM(s.total) AS `ltd_sales`
, SUM(s.total <> 0) AS `cnt_prior_sales`
, SUM(s.total = 0) AS `cnt_giveaways`
FROM orders s
WHERE s.status = 'shipped'
GROUP
BY s.user
) t
ON t.user = o.user
WHERE o.status = 'shipped'
I'm not sure about that column named "prior" sales... this is returning all shipped orders, without regard to comparing any dates (order date, fulfillment date, shipment date), which we would typically associate with a concept of what "prior" means.
FOLLOWUP
noticing that the question is modified, removing the condition "status = 'shipped'" from the count of all orders by the user...
I will note that we can move conditions from the WHERE clause into the conditional aggregates.
Not that all these results are needed by OP, but as a demonstration...
SELECT s.user
, SUM(IF(s.status='shipped',s.total,0)) AS `ltd_sales_shipped`
, SUM(IF(s.status<>'shipped',s.total,0)) AS `ltd_sales_not_shipped`
, SUM(s.status='shipped' AND s.total <> 0) AS `cnt_shipped_orders`
, SUM(s.status='canceled') AS `cnt_canceled`
, SUM(s.status='shipped' AND s.total = 0) AS `cnt_shipped_giveaways`
FROM orders s
GROUP
BY s.user
Once the results are returned from the database, you can not run an SQL on top of them. However you can store them in a temporary table, to reuse them.
https://dev.mysql.com/doc/refman/8.0/en/create-temporary-table.html
https://dev.mysql.com/doc/refman/8.0/en/create-table-select.html
https://dev.mysql.com/doc/refman/8.0/en/insert-select.html
You need to create a temporary table, and insert all the data from the select statement, and then you can run queries on that table. Not sure if it would help much in your case.
For your particular case you can do something like:
select user, (total = 0) as is_total_zero, count(*), sum(total)
from orders
where status = 'shipped'
group by user, total = 0
However you would have to do some additional summing to get the results of the second query which gives you the sums per user, as they would be divided into two different groups with a different is_total_zero value.
I need to synchronize specific information between two databases (one mysql, the other a remote hosted SQL Server database) for thousands of rows. When I execute this php file it gets stuck/timeouts after several minutes I guess, so I wonder how I can fix this issue and maybe also optimize the way of "synchronizing" it.
What the code needs to do:
Basically I want to get for every row (= one account) in my database which gets updated - two specific pieces of information (= 2 SELECT queries) from another SQL Server database. Therefore I use a foreach loop which creates 2 SQL queries for each row and afterwards I update those information into 2 columns of this row. We talk about ~10k Rows which needs to run thru this foreach loop.
My idea which may help?
I have heard about things like PDO Transactions which should collect all those queries and sending them afterwards in a package of all SELECT queries, but I have no idea whether I use them correctly or whether they even help in such cases.
This is my current code, which is timing out after few minutes:
// DBH => MSSQL DB | DB => MySQL DB
$dbh->beginTransaction();
// Get all referral IDs which needs to be updated:
$listAccounts = "SELECT * FROM Gifting WHERE refsCompleted <= 100 ORDER BY idGifting ASC";
$ps_listAccounts = $db->prepare($listAccounts);
$ps_listAccounts->execute();
foreach($ps_listAccounts as $row) {
$refid=$row['refId'];
// Refsinserted
$refsInserted = "SELECT count(username) as done FROM accounts WHERE referral='$refid'";
$ps_refsInserted = $dbh->prepare($refsInserted);
$ps_refsInserted->execute();
$row = $ps_refsInserted->fetch();
$refsInserted = $row['done'];
// Refscompleted
$refsCompleted = "SELECT count(username) as done FROM accounts WHERE referral='$refid' AND finished=1";
$ps_refsCompleted = $dbh->prepare($refsCompleted);
$ps_refsCompleted->execute();
$row2 = $ps_refsCompleted->fetch();
$refsCompleted = $row2['done'];
// Update fields for local order db
$updateGifting = "UPDATE Gifting SET refsInserted = :refsInserted, refsCompleted = :refsCompleted WHERE refId = :refId";
$ps_updateGifting = $db->prepare($updateGifting);
$ps_updateGifting->bindParam(':refsInserted', $refsInserted);
$ps_updateGifting->bindParam(':refsCompleted', $refsCompleted);
$ps_updateGifting->bindParam(':refId', $refid);
$ps_updateGifting->execute();
echo "$refid: $refsInserted Refs inserted / $refsCompleted Refs completed<br>";
}
$dbh->commit();
You can do all of that in one query with a correlated sub-query:
UPDATE Gifting
SET
refsInserted=(SELECT COUNT(USERNAME)
FROM accounts
WHERE referral=Gifting.refId),
refsCompleted=(SELECT COUNT(USERNAME)
FROM accounts
WHERE referral=Gifting.refId
AND finished=1)
A correlated sub-query is essentially using a sub-query (query within a query) that references the parent query. So notice that in each of the sub-queries I am referencing the Gifting.refId column in the where clause of each sub-query. While this isn't the best for performance because each of those sub-queries still has to run independent of the other queries, it would perform much better (and likely as good as you are going to get) than what you have there.
Edit:
And just for reference. I don't know if a transaction will help here at all. Typically they are used when you have several queries that depend on each other and to give you a way to rollback if one fails. For example, banking transactions. You don't want the balance to deduct some amount until a purchase has been inserted. And if the purchase fails inserting for some reason, you want to rollback the change to the balance. So when inserting a purchase, you start a transaction, run the update balance query and the insert purchase query and only if both go in correctly and have been validated do you commit to save.
Edit2:
If I were doing this, without doing an export/import this is what I would do. This makes a few assumptions though. First is that you are using a mssql 2008 or newer and second is that the referral id is always a number. I'm also using a temp table that I insert numbers into because you can insert multiple rows easily with a single query and then run a single update query to update the gifting table. This temp table follows the structure CREATE TABLE tempTable (refId int, done int, total int).
//get list of referral accounts
//if you are using one column, only query for one column
$listAccounts = "SELECT DISTINCT refId FROM Gifting WHERE refsCompleted <= 100 ORDER BY idGifting ASC";
$ps_listAccounts = $db->prepare($listAccounts);
$ps_listAccounts->execute();
//loop over and get list of refIds from above.
$refIds = array();
foreach($ps_listAccounts as $row){
$refIds[] = $row['refId'];
}
if(count($refIds) > 0){
//implode into string for use in query below
$refIds = implode(',',$refIds);
//select out total count
$totalCount = "SELECT referral, COUNT(username) AS cnt FROM accounts WHERE referral IN ($refIds) GROUP BY referral";
$ps_totalCounts = $dbh->prepare($totalCount);
$ps_totalCounts->execute();
//add to array of counts
$counts = array();
//loop over total counts
foreach($ps_totalCounts as $row){
//if referral id not found, add it
if(!isset($counts[$row['referral']])){
$counts[$row['referral']] = array('total'=>0,'done'=>0);
}
//add to count
$counts[$row['referral']]['total'] += $row['cnt'];
}
$doneCount = "SELECT referral, COUNT(username) AS cnt FROM accounts WHERE finished=1 AND referral IN ($refIds) GROUP BY referral";
$ps_doneCounts = $dbh->prepare($doneCount);
$ps_doneCounts->execute();
//loop over total counts
foreach($ps_totalCounts as $row){
//if referral id not found, add it
if(!isset($counts[$row['referral']])){
$counts[$row['referral']] = array('total'=>0,'done'=>0);
}
//add to count
$counts[$row['referral']]['done'] += $row['cnt'];
}
//now loop over counts and generate insert queries to a temp table.
//I suggest using a temp table because you can insert multiple rows
//in one query and then the update is one query.
$sqlInsertList = array();
foreach($count as $refId=>$count){
$sqlInsertList[] = "({$refId}, {$count['done']}, {$count['total']})";
}
//clear out the temp table first so we are only inserting new rows
$truncSql = "TRUNCATE TABLE tempTable";
$ps_trunc = $db->prepare($truncSql);
$ps_trunc->execute();
//make insert sql with multiple insert rows
$insertSql = "INSERT INTO tempTable (refId, done, total) VALUES ".implode(',',$sqlInsertList);
//prepare sql for insert into mssql
$ps_insert = $db->prepare($insertSql);
$ps_insert->execute();
//sql to update existing rows
$updateSql = "UPDATE Gifting
SET refsInserted=(SELECT total FROM tempTable WHERE refId=Gifting.refId),
refsCompleted=(SELECT done FROM tempTable WHERE refId=Gifting.refId)
WHERE refId IN (SELECT refId FROM tempTable)
AND refsCompleted <= 100";
$ps_update = $db->prepare($updateSql);
$ps_update->execute();
} else {
echo "There were no reference ids found from \$dbh";
}
I have created a web service using php and mysql to insert and return data for my app. One of the main complaints from users is the response is too slow, which I agree, as it takes anywhere from 5-10 seconds to complete task. Originally thought it was a hosting performance issue, but has not made a difference (godaddy versus AWS).
Looking at the SQL code in my php files, I think that is the issue. I'm looking to re-write to increase speed. I believe the issue is I use timestamps, and to return I always look for the max(timestamp).
Here is my input SQL statement:
$insert = mysql_query("INSERT INTO meals (user, date, meal, timeStamp, food) VALUES ('$user','$date','$meal','$timeStamp','$food')");
Here is my return SQL statement:
$query = mysql_query ("SELECT * FROM meals WHERE user = '$user' AND date = '$date' AND meal = '$meal' AND timeStamp = (SELECT MAX(`timeStamp`)FROM meals WHERE user = '$user' AND date = '$date' AND meal = '$meal')");
Would I be better off using an UPDATE or similar instead of using timestamps? Any other recommendations?
This is your query:
SELECT *
FROM meals
WHERE user = '$user' AND date = '$date' AND meal = '$meal' AND
timeStamp = (SELECT MAX(`timeStamp`)
FROM meals
WHERE user = '$user' AND date = '$date' AND meal = '$meal'
);
Assuming that you want only one row, the first simplification is:
SELECT *
FROM meals
WHERE user = '$user' AND date = '$date' AND meal = '$meal'
ORDER BY timestamp desc
LIMIT 1;
Next, indexes will help this query. In fact, an index on meals(user, date, meal, timestamp) would work for both the where and the order by. That should improve performance.
I wrote a product price/stock update script for Magento. I load the csv into an array and then iterate through it. The current code takes around 10 minutes to complete for 5,000 products, is there a faster way to do this? I've already bypassed Magento's API as that was extremely slow and switched to updating the database directly since its not many tables and its faster. Using timers to record the time, it takes about 10 minutes for the foreach loop and two minutes for the reindexALL
$con = mysql_connect("localhost","root","");
$selected = mysql_select_db("magento",$con);
$processes = Mage::getSingleton('index/indexer')->getProcessesCollection();
$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_MANUAL));
$processes->walk('save');
foreach($all_rows as $final)
{
$sql = mysql_query("SELECT entity_id from catalog_product_entity where sku = '".$final[ITEM]."'");
if ($row = mysql_fetch_array($sql)) {
//update price
$pricenew = $final['PRICE'] + ($final['PRICE']*.30);
mysql_query("UPDATE catalog_product_entity_decimal SET value = '$pricenew' where attribute_id = 75 AND entity_id = '".$row[entity_id]."' ");
//update retail price
$retailprice = $final['RETAIL'];
mysql_query("UPDATE catalog_product_entity_decimal SET value = '$retailprice' where attribute_id = 120 AND entity_id = '".$row[entity_id]."' ");
//update stock quantity and is in stock
$stockquantity = $final['QTY'];
$stockquantity = number_format($stockquantity, 4, '.', '');
mysql_query("UPDATE cataloginventory_stock_item SET qty = '$stockquantity', SET is_in_stock = 1 where product_id = '".$row[entity_id]."' ");
}
$processes->walk('reindexAll');
$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_REAL_TIME));
$processes->walk('save');
mysql_close($con);
If your table catalog_product_entity_decimal has index, that covers id (obviously it is) - then you have no other ways to speed it up. Since the slowest thing here is physical changing of the value.
Probably you can put a WHERE clause to to avoid of updating the price to the same value.
Other thoughts:
While most people look at performance optimizations for SELECT statements, UPDATE and DELETE statements are often overlooked. These can benefit from the principles of analyzing the Query Execution Plan (QEP). You can only run an EXPLAIN on a SELECT statement, however it’s possible to rewrite an UPDATE or DELETE statement to perform like a SELECT statement.
To optimize an UPDATE, look at the WHERE clause. If you are using the PRIMARY KEY, no further analysis is necessary. If you are not, it is of benefit to rewrite your UPDATE statement as a SELECT statement and obtain a QEP as previously detailed to ensure optimal indexes are used. For example:
UPDATE t
SET c1 = ‘x’, c2 = ‘y’, c3 = 100
WHERE c1 = ‘x’
AND d = CURDATE()
You can rewrite this UPDATE statement as a SELECT statement for using EXPLAIN:
EXPLAIN SELECT c1, c2, c3 FROM t WHERE c1 = ‘x’ AND d = CURDATE()
You should now apply the same principles as you would when optimizing SELECT statements.
Hey guys, I created a list for fixtures.
$result = mysql_query("SELECT date FROM ".TBL_FIXTURES." WHERE compname = '$comp_name' GROUP BY date");
$i = 1;
$d = "Start";
while ($row = mysql_fetch_assoc($result))
{
$odate = $row['date'];
$date=date("F j Y", $row['date']);
echo "<p>Fixture $i - $d to $date</p>";
}
As you can see from the query, the date is displayed from the fixtures table.
The way my system works is that when a fixture is "played", it is removed from this table. Therefore when the entire round of fixtures are complete, there wont be any dates for that round in this table. They will be in another table.
Is there anyway I can run an other query for dates at the same time, and display only dates from the fixtures table if there isnt a date in the results table?
"SELECT * FROM ".TBL_CONF_RESULTS."
WHERE compid = '$_GET[id]' && type2 = '2' ORDER BY date"
That would be the second query!
EDIT FROM HERE ONWARDS...
Is there anyway I can select the date from two tables and then only use one if there are matches. Then use the rows of dates (GROUPED BY) to populate my query? Is that possible?
It sounds like you want to UNION the two result sets, akin to the following:
SELECT f.date FROM tbl_fixtures f
WHERE f.compname = '$comp_name'
UNION SELECT r.date FROM tbl_conf_results r
WHERE r.compid = '$_GET[id]' AND r.type2 = '2'
GROUP BY date
This should select f.date and add rows from r.date that aren't already in the result set (at least this is the behaviour with T-SQL). Apparently it may not scale well, but there are many blogs on that (search: UNION T-SQL).
From the notes on this page:
//performs the query
$result = mysql_query(...);
$num_rows = mysql_num_rows($result);
//if query result is empty, returns NULL, otherwise,
//returns an array containing the selected fields and their values
if($num_rows == NULL)
{
// Do the other query
}
else
{
// Do your stuff as now
}
WHERE compid = '$_GET[id]' presents an oportunity for SQL Injection.
Are TBL_FIXTURES and TBL_CONF_RESULTS supposed to read $TBL_FIXTURES and $TBL_CONF_RESULTS?
ChrisF has the solution!
One other thing you might think about is whether it is necessary to do a delete and move to another table. A common way to solve this type of challenge is to include a status field for each record, then rather than just querying for "all" you query for all where status = "x". For example, 1 might be "staging", 2 might be "in use", 3 might be "used" or "archived" In your example, rather than deleting the field and "moving" the record to another table (which would also have to happen in the foreach loop, one would assume) you could simply update the status field to the next status.
So, you'd eliminate the need for an additional table, remove one additional database hit per record, and theoretically improve the performance of your application.
Seems like what you want is a UNION query.
$q1 = "SELECT DISTINCT date FROM ".TBL_FIXTURES." WHERE compname = '$comp_name'";
$q2 = "SELECT DISTINCT date FROM ".TBL_CONF_RESULTS.
"WHERE compid = '$_GET[id]' && type2 = '2'";
$q = "($q1) UNION DISTINCT ($q2) ORDER BY date";