Well, I'm afraid that I will not be able to post a minimum reproducible example, and for that I apologize. But, here goes nothing.
Ours is a weekly prepared meals service. I track order volume in many ways. Here is the structure of the relevant table:
So then I utilize the highlighted fields in many ways, such as indicating to delivery drivers if a customer is returning from the prior order being more than a month ago (last_order_w - prev_order_w > 4), for instance.
Lately I have been noticing that the data is not consistently updating properly. In the past 3 weeks, I would say it is an occurrence of 5%. If it were more consistent, I would be more confident in my ability to track down the issue, but I am not even sure how to provoke it, as I only really notice it after the fact.
The code that should cause the update is below:
<?php
//retrieve and iterate over IDs of orders placed since last synchronization.
$newOrders=array_map('reset',$dbh->query("select id from wp_posts where id > (select max(synced) from fitaf_weeks) and post_type='shop_order' and post_status='wc-processing'")->fetchAll(PDO::FETCH_NUM));
foreach($newOrders as $no){
//retrieve the metadata for the current order
$newMetas=array_map('reset',$dbh->query("select meta_key,meta_value from wp_postmeta where post_id=$no")->fetchAll(PDO::FETCH_GROUP|PDO::FETCH_UNIQUE));
//check if the current order is associated with an existing customer
$exist=$dbh->query("select * from fitaf_customers where id=".$newMetas['_customer_user'])->fetch();
//if not, gather the information we want to store from this post
$noExist=[$newMetas['_customer_user'],$newMetas['_shipping_first_name'],$newMetas['_shipping_last_name'],$newMetas['_shipping_address_1'],(strlen($newMetas['_shipping_address_2'])==0?NULL:$newMetas['_shipping_address_2']),$newMetas['_shipping_city'],$newMetas['_shipping_state'],$newMetas['_shipping_postcode'],$phone,$newMetas['_billing_email'],1,1,$no,$newMetas['_paid_date'],$week[3],$newMetas['_order_total']];
if($exist){
//if we found a record in the customer table, retrieve the data we want to modify
$oldO=$dbh->query("select last_order_id,last_order,last_order_w,lo,num_orders from fitaf_customers where id=".$newMetas['_customer_user'])->fetch(PDO::FETCH_GROUP|PDO::FETCH_ASSOC|PDO::FETCH_UNIQUE);
//make changes to the retrieved data, and make sure we are storing the most recently used delivery address and prepare the data points for the update command
$exists=[$phone,$newMetas['_shipping_first_name'],$newMetas['_shipping_last_name'],$newMetas['_shipping_postcode'],$newMetas['_shipping_address_1'],(strlen($newMetas['_shipping_address_2'])==0?NULL:$newMetas['_shipping_address_2']),$newMetas['_shipping_city'],$newMetas['_shipping_state'],$newMetas['_paid_date'],$no,$week[3],$oldO['last_order'],$oldO['last_order_id'],$oldO['last_order_w'],($oldO['num_orders']+1),($oldO['lo']+$newMetas['_order_total']),$newMetas['_customer_user']];
}
if(!$exist){
//if the customer did not exist, perform an insert
$dbh->prepare("insert into fitaf_customers(id,fname,lname,addr1,addr2,city,state,zip,phone,email,num_orders,num_weeks,last_order_id,last_order,last_order_w,lo) values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)")->execute($noExist);
}
else{
//if the customer did exist, update their data
$dbh->prepare("update fitaf_customers set phone=?,fname=?,lname=?,zip=?,addr1=?,addr2=?,city=?,`state`=?,last_order=?,last_order_id=?,last_order_w=?,prev_order=?,prev_order_id=?,prev_order_w=?,num_orders=?,lo=? where id=?")->execute($exists);
}
}
//finally retrieve the most recent post ID and update the field we check against when the syncornization script runs
$lastPlaced=$dbh->query('select max(id) from wp_posts where post_type="shop_order"')->fetch()[0];
$updateSync=$dbh-> query("update fitaf_weeks set synced=$lastPlaced order by id desc limit 1");
?>
Unfortunately I don't have any relevant error logs to show, however, as I documented the code for this post, I realized a potential shortcoming. I should be utilizing the data retrieved from the initial query of new posts, rather than a selecting the highest post id after performing this logic. However, I have timers running on my scripts, and this section hasn't taken over 3 seconds to run in a long time. So it seems unlikely, that the script, which runs on a cron every 5 minutes, is experiencing this unintended overlap?
While I have made the change to pop the highest ID off of $newOrders, and hope it solves the issue, I am still curious to see if anyone has any insights on what could cause this logic to fail at such a low occurrence.
It seems likely your problem comes from race conditions between multiple operations accessing your db.
First of all, your last few lines of code do SELECT MAX(ID) and then uses that value to update something. You Can't Do Thatâ„¢. If somebody else adds a row to that wp_posts table anytime after the entry you think is relevant, you'll use the wrong ID. I don't understand your app well enough to recommend a fix. But I do know this is a serious and notorious problem.
You have another possible race condition as well. Your logic is this:
SELECT something.
make a decision based on what you SELECTED.
INSERT or UPDATE based on that decision.
If some other operation, done by some other user of the db, intervenes between step 1 and step 3, your decision might be wrong.
You fix this with a db transaction. The ->beginTransaction() operation, well, begins the transaction. The ->commit() operation concludes it. And, the SELECT operation you use for step one should say SELECT ... FOR UPDATE.
Hi have a bunch of unique codes in a database which should only be used once.
Two users hit a script which assigns them at the same time and got the same codes!
The script is in Magento and the user can order multiple codes. The issue is if one customer orders 1000 codes the script grabs the top 1000 codes from the DB into an array and then runs through them setting them to "Used" and assigning them to an order. If a second user hits the same script at a similar time the script then grabs the top 1000 codes in the DB at that point in time which crosses over as the first script hasn't had a chance to finish assigning them.
This is unfortunate but has happened quite a few times!
My idea was to create a new table, once the user hits the script a row is made with "order_id" "code_type". Then in the same script a check is done so if a row is in this new table and the "code_type" matches that of which the user is ordering it will wait 60 seconds and check again until the previous codes are issued and the table is empty where it will then create a row and off it goes.
I am not sure if this is the best way or if two users hit at the same second again whether two rows will just be inserted and off we go with the same problem!
Any advice is much appreciated!
The correct answer depends on the database you use.
For example in MySQL with InnoDB the possible solution is a transaction with SELECT ... LOCK IN SHARE MODE.
Schematically it works this by firing following queries:
START TRANSACTION;
SELECT * FROM codes WHERE used = 0 LIMIT 1000 LOCK IN SHARE MODE;
// save ids
UPDATE codes SET used=1 WHERE id IN ( ...ids....);
COMMIT;
More information at http://dev.mysql.com/doc/refman/5.7/en/innodb-locking-reads.html
I am looking for a way to check if there are certain number of records within mysql table. For example: After POST request been before putting data to dabase, it checks first how many records there are. If lets say there are 24 records, then it will delete record with latest date based on timestamp and then inster new value from POST request. Has anyone got idea on how to do it? Looking forward fpr your answers. Below I attached simple code i wrote to insert data from post request into table.
<?php
include("connect.php");
$link=Connection();
$temp1=$_POST["temp1"];
$hum1=$_POST["hum1"];
$query = "INSERT INTO `tempLog` (`temperature`, `humidity`)
VALUES ('".$temp1."','".$hum1."')";
mysql_query($query,$link);
mysql_close($link);
header("Location: index.php");
?>
When you say delete with the latest date I have to assume you mean the oldest record? Your description doesnt tell me the name of you date field so Lets assume its onDate. You also didnt mention what your primary key is so lets assume that is just id. if you run the below query before inserting it will purge all the oldest records leaving only the newest 23 in the database.
delete from templog where id in (
select id from (
select #rownum:=#rownum+1 'rowid', t.id from templog t, (select #rownum:=0)r order by t.onDate
)v where v.rowid > 23
);
Of course you should test on data you don't mind losing.
It is best to do a cleanup purge each time instead of removing a single row before adding a new one because in the event of exceptions it will never clean itself down to the 24 rows you wish to truly have.
I also want to note that you may want to reconsider this method all together. Instead leave the data there, and only query the most recent 24 when displaying the log. Since you are going through the trouble of collecting the data you might as well keep it for future reporting. Then later down the road if your table gets to large run a simple daily purge query to delete anything older than a certain threshold.
Hope this helps.
I have a small PHP function on my website which basically does 3 things:
check if user is logged in
if yes, check if he has the right to do this action (DB Select)
if yes, do the related action (DB Insert/Update)
If I have several users connected at the same time on my website that try to access this specific function, is there any possibility of concurrency problem, like we can have in Java for example? I've seen some examples about semaphore or native PHP synchronization, but is it relevant for this case?
My PHP code is below:
if ( user is logged ) {
sql execution : "SELECT....."
if(sql select give no results){
sql execution : "INSERT....."
}else if(sql select give 1 result){
if(selected column from result is >= 1){
sql execution : "UPDATE....."
}
}else{
nothing here....
}
}else{
nothing important here...
}
Each user who accesses your website is running a dedicated PHP process. So, you do not need semaphores or anything like that. Taking care of the simultaneous access issues is your database's problem.
Not in PHP. But you might have users inserting or updating the same content.
You have to make shure this does not happen.
So if you have them update their user profile only the user can access. No collision will occur.
BUT if they are editing content like in a Content-Management System... they can overwrite each others edits. Then you have to implement some locking mechanism.
For example(there are a lot of ways...) if you write an update on the content keeping the current time and user.
Then the user has a lock on the content for maybe 10 min. You should show the (in this case) 10 min countdown in the frontend to the user. And a cancel button to unlock the content and ... you probably get the idea
If another person tries to load the content in those 10 min .. it gets an error. "user xy is already... lock expires at xx:xx"
Hope this helps.
In general, it is not safe to decide whether to INSERT or UPDATE based on a SELECT result, because a concurrent PHP process can INSERT the row after you executed your SELECT and saw no row in the table.
There are two solutions. Solution number one is to use REPLACE or INSERT ... ON DUPLICATE KEY UPDATE. These two query types are "atomic" from perspective of your script, and solve most cases. REPLACE tries to insert the row, but if it hits a duplicate key it replaces the conflicting existing row with the values you provide, INSERT ... ON DUPLICATE KEY UPDATE is a little bit more sophisticated, but is used in a similar situations. See the documentation here:
http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html
http://dev.mysql.com/doc/refman/5.0/en/replace.html
For example, if you have a table product_descriptions, and want to insert a product with ID = 5 and a certain description, but if a product with ID 5 already exists, you want to update the description, then you can just execute the following query (assuming there's a UNIQUE or PRIMARY key on ID):
REPLACE INTO product_description (ID, description) VALUES(5, 'some description')
It will insert a new row with ID 5 if it does not exist yet, or will update the existing row with ID 5 if it already exists, which is probably exactly what you want.
If it is not, then approach number two is to use locking, like so:
query('LOCK TABLE users WRITE')
if (num_rows('SELECT * FROM users WHERE ...')) {
query('UPDATE users ...');
}
else {
query('INSERT INTO users ...');
}
query('UNLOCK TABLES')
I have two tables called clients, they are exactly the same but within two different db's. Now the master always needs to update with the secondary one. And all data should always be the same, the script runs once per day. What would be the best to accomplish this.
I had the following solution but I think maybe theres a better way to do this
$sql = "SELECT * FROM client";
$res = mysql_query($conn,$sql);
while($row = mysql_fetch_object($res)){
$sql = "SELECT count(*) FROM clients WHERE id={$row->id}";
$res1 = mysql_query($connSecond,$sql);
if(mysql_num_rows($res1) > 0){
//Update second table
}else{
//Insert into second table
}
}
and then I need a solution to delete all old data in second table thats not in master.
Any advise help would be appreaciated
This is by no means an answer to your php code, but you should take a look # Mysql Triggers, you should be able to create triggers (on updates / inserts / deletes) and have a trigger (like a stored proceedure) update your table.
Going off the description you give, I would create a trigger that would check for changes to the 2ndary table, then write that change to the primary table, and delete that initial entry (if so required) form the 2ndary table.
Triggers are run per conditions that you define.
Hopefully this gives you insight into 'another' way of doing this task.
More references on triggers for mysql:
http://dev.mysql.com/doc/refman/5.0/en/triggers.html
http://www.mysqltutorial.org/create-the-first-trigger-in-mysql.aspx
You can use mysql INSERT ... SELECT like this (but first truncate the target table):
TRUNCATE TABLE database2.client;
INSERT INTO database2.client SELECT * FROM database1.client;
It will be way faster than doing it by PHP.
And to your notice:
As long as the mysql user has been given the right permissions to all databases and tables where data is pulled from or pushed to, this will work. Though the mysql_select_db function selects one database, the mysql statement may reference another if you use complete reference like databasename.tablename
Not exactly answering your question, but how about just using 1 table, instead of 2? You could use a fedarated table to access the other (if it's on a different mysql instance) or reference the table directly (like shamittomar's suggestion)
If both are on the same MySQL instance, you could easily use a view:
CREATE VIEW database2.client SELECT * FROM database1.client;
And that's it! No synchronizing, no cron jobs, no voodoo :)