Whats better; mysql scheduler or php script and crontab? - php

I have 1 mysql table where it has thousands of rows. I use this as a transaction history for my users. I also query this table on the same page for a sum of earnings per product. here are my two mysql calls;
$earnperproduct = mysqli_query($con,"SELECT product, SUM(amount) AS totalearn FROM wp_payout_history WHERE user=$userid GROUP BY product");
$result = mysqli_query($con,"SELECT * FROM wp_payout_history WHERE user=$userid ORDER BY date DESC");
my fear is that as the table grows, the $earnperproduct call will become too intensive and slow down page loading. Therefore instead of doing a sum command every time the page loads, i think it would be easier to update a summary (example wp_summary_table) whenever wp_payout_history is changed to replace past values with new SUM(AMOUNT)values per user and product; and thus query something like this;
$earnperproduct = mysqli_query($con,"SELECT * FROM wp_payout_summary_table WHERE user=$userid ORDER BY product DESC");
TL;DR
What is the best method to go about updating a table using the $earnedperproduct style call? would I be better using mysql event scheduler or a php script with a crontab? Is there any tutorials that can help me create either option for my needs?

Both the control mechanisms you mention use time as the trigger for an action. But your description says that you really want to trigger an action when the data changes. And in a relational database the best way to trigger an action when data is changed is with a ....trigger. Which makes your question a duplicate of this.
Arguably it may be more efficient to snapshot the transactions, then something like....
[INSERT INTO summary_table (user, total_amount, last_id) ]
SELECT
user, SUM(amount), MAX(id)
FROM (
SELECT a.user, a.total_amount AS amount, a.last_id
FROM summary_table a
WHERE a.user=$user_id
AND last_id=(SELECT MAX(b.last_id)
FROM summary_table b
WHERE b.user=$user_id)
UNION
SELECT h.user, h.amount, h.id
FROM wp_payout_history h
WHERE h.user=$user_id
AND h.id>(SELECT MAX(c.last_id)
FROM summary_table c
WHERE c.user=$user_id)
) ilv
GROUP BY user;
...then it doesn't really matter what you use to refresh the history - the query will always give you an up to date response. If you go down this route then add a dummy integer column in the summary table and add 0 as unaggregatedrows to the second SELECT and SUM(1) to the 4th SELECT to work out when it will be most efficient to update the summary table.

Related

Mysql fetch from last to first - [many records]

i want to fetch records from mysql starting from last to first LIMIT 20. my database have over 1M records. I am aware of order by. but from my understanding when using order by its taking forever to load 20 records i have no freaking idea. but i think mysql fetch all the records before ordering.
SELECT bookings.created_at, bookings.total_amount,
passengers.name, passengers.id_number, payments.amount,
passengers.ticket_no,bookings.phone,bookings.source,
bookings.destination,bookings.date_of_travel FROM bookings
INNER JOIN passengers ON bookings.booking_id = passengers.booking_id
INNER JOIN payments on payments.booking_id = bookings.booking_id
ORDER BY bookings.booking_id DESC LIMIT 10
I suppose if you execute the query without the order by the time would be satisfactory?
You might try to create an index in the column your are ordering:
create index idx_bookings_booking_id on bookings(booking_id)
You can try to find out complexity of the Query using
EXPLAIN SELECT bookings.created_at, bookings.total_amount,
passengers.name, passengers.id_number, payments.amount,
passengers.ticket_no,bookings.phone,bookings.source,
bookings.destination,bookings.date_of_travel FROM bookings
INNER JOIN passengers ON bookings.booking_id = passengers.booking_id
INNER JOIN payments on payments.booking_id = bookings.booking_id
ORDER BY bookings.booking_id DESC LIMIT 10
then check the proper index has been created on the table
SHOW INDEX FROM `db_name`.`table_name`;
if the index us not there create proper index on all the table
please add if anything is missing
The index lookup table needs to be able to reside in memory, if I'm not mistaken (filesort is much slower than in-mem lookup).
Use small index / column size
For a double in capacity use UNSIGNED columns if you need no negative values..
Tune sort_buffer_size and read_rnd_buffer_size (maybe better on connection level, not global)
See https://dev.mysql.com/doc/refman/5.7/en/order-by-optimization.html , particularly regarding using EXPLAIN and the maybe trying another execution plan strategy.
You seem to need another workaround like materialized views.
Tell me if this sounds like it:
Create another table like the booking table e.g. CREATE TABLE booking_short LIKE booking. Though you only need the booking_id column
And check your code for where exactly you create booking orders, e.g. where you first insert into booking. SELECT COUNT(*) FROM booking_short. If it is >20, delete the first record. Insert the new booking_id.
You can select the ID and join from there before joining for more details with the rest of the tables.
You won't need limit or sorting.
Of course, this needs heavy documentation to avoid maintenance problems.
Either that or https://stackoverflow.com/a/5912827/6288442

SQL select all files where a value in table A is the same as in table B (same database)

I'm building a sales system for a company, but I'm stuck with the following issue.
Every day I load .XML productfeed into a database called items. The rows in the productfeed are never in the same order, so sometimes the row with Referentie = 380083 is at the very top, and the other day that very same row is at the very bottum.
I also have to get all the instock value, but when I run the following query
SELECT `instock` FROM SomeTable WHERE `id` > 0
I get all values, but not in the same order as in the other table.
So I have to get the instock value of all rows where referentie in table A is the same as it is in table B.
I already have this query:
select * from `16-11-23 wed 09:37` where `referentie` LIKE '4210310AS'
and this query does the right job, but I have like 500 rows in the table.
So I need to find a way to automate the: LIKE '4210310AS' bit, so it selects all 500 values in one go.
Can somebody tell me how that can be done?
I'm not even sure I understand your problem...
Don't take this personally, but you seem to be concerned/confused by the ordering of the data in the tables which suggests to me your understanding of relational databases and SQL is lacking. I suggest you brush up on the basics.
Can't you just use the following query?
SELECT a.referentie
, b.instock
FROM tableA a
, tableB b
WHERE b.referentie = a.referentie

Select data from one table, arrange by a sum of data from another

A client is looking for a points system to be implemented on her website, I'm struggling to display the users based upon the amount of points collected, I hope somebody may be able to help me out here and point me in the right direction to getting this code to work properly.
I am selecting all data from ap_users and in the code I am also trying to select all data from ap_points although I do not require all the data from either tables, to be specific I only require:
ap_users:
user_id
first_name
last_name
display_img
hub_access
ap_points:
user_id
points_added
I thought that selecting ALL data may be the easiest route, will let you decide.
I am trying to select and display all users where hub_access = '1' and order by the total points_added by highest first. Points are added separately by rows and need to be added up (which is why I have the sum function).
$sql = "SELECT * FROM `ap_users`, `ap_points` WHERE `hub_access` = '1' ORDER BY sum(points_added) DESC";
I also tried configuring it to be specific tables like:
ap_users.hub_access and ORDER BY sum(ap_points.points_added) but these did not work either.
This current code is either showing no results or a single result with no errors displaying? I'm not sure whether I may need to use some kind of Group By function to connect the user_ids from both tables ?
SUM is an aggregating function. You should be grouping by user_id if you want the sum for each user_id.
Something like
SELECT *, sum(points_added) as sum_points FROM app_users
JOIN app_points ON app_users.user_id = app_points.user_id
WHERE `hub_access` = '1'
GROUP BY app_users.user_id
ORDER BY sum_points;
I have not tested that query, but that should give you an idea of the solution.

Order by votes - PHP

I have a voting script which pulls out the number of votes per user.
Everything is working, except I need to now display the number of votes per user in order of number of votes. Please see my database structure:
Entries:
UserID, FirstName, LastName, EmailAddress, TelephoneNumber, Image, Status
Voting:
item, vote, nvotes
The item field contains vt_img and then the UserID, so for example: vt_img4 and both vote & nvotes display the number of votes.
Any ideas how I can relate those together and display the users in order of the most voted at the top?
Thanks
You really need to change the structure of the voting table so that you can do a normal join. I would strongly suggest adding either a pure userID column, or at the very least not making it a concat of two other columns. Based on an ID you could then easily do something like this:
select
a.userID,
a.firstName,
b.votes
from
entries a
join voting b
on a.userID=b.userID
order by
b.votes desc
The other option is to consider (if it is a one to one relationship) simply merging the data into one table which would make it even easier again.
At the moment, this really is an XY problem, you are looking for a way to join two tables that aren't meant to be joined. While there are (horrible, ghastly, terrible) ways of doing it, I think the best solution is to do a little extra work and alter your database (we can certainly help with that so you don't lose any data) and then you will be able to both do what you want right now (easily) and all those other things you will want to do in the future (that you don't know about right now) will be oh so much easier.
Edit: It seems like this is a great opportunity to use a Trigger to insert the new row for you. A MySQL trigger is an action that the database will make when a certain predefined action takes place. In this case, you want to insert a new row into a table when you insert a row into your main table. The beauty is that you can use a reference to the data in the original table to do it:
CREATE TRIGGER Entries_Trigger AFTER insert ON Entries
FOR EACH ROW BEGIN
insert into Voting values(new.UserID,0,0);
END;
This will work in the following manner - When a row is inserted into your Entries table, the database will insert the row (creating the auto_increment ID and the like) then instantly call this trigger, which will then use that newly created UserID to insert into the second table (along with some zeroes for votes and nvotes).
Your database is badly designed. It should be:
Voting:
item, user_id, vote, nvotes
Placing the item id and the user id into the same column as a concatenated string with a delimiter is just asking for trouble. This isn't scalable at all. Look up the basics on Normalization.
You could try this:
SELECT *
FROM Entries e
JOIN Voting v ON (CONCAT('vt_img', e.UserID) = v.item)
ORDER BY nvotes DESC
but please notice that this query might be quite slow due to the fact that the join field for Entries table is built at query time.
You should consider changing your database structure so that Voting contains a UserID field in order to do a direct join.
I'm figuring the Entries table is where votes are cast (you're database schema doesn't make much sense to me, seems like you could work it a little better). If the votes are actually on the Votes table and that's connected to a user, then you should have UserID field in that table too. Either way the example will help.
Lets say you add UserID to the Votes table and this is where a user's votes are stored than this would be your query
SELECT Users.id, Votes.*,
SUM(Votes.nvotes) AS user_votes
FROM Users, Votes
WHERE Users.id = Votes.UserID
GROUP BY Votes.UserID
ORDER BY user_votes
USE ORDER BY in your query --
SELECT column_name(s)
FROM table_name
ORDER BY column_name(s) ASC|DESC

trick to get the sum of MySQL rows outside of MySQL

so I have the following query:
SELECT DISTINCT d.iID1 as 'id',
SUM(d.sum + d.count*r.lp)/sum(d.count) AS avgrat
FROM abcd i, abce r, abcf d
WHERE r.aID = 1 AND
d.iID1 <> r.rID AND d.iID2 = r.rID GROUP BY d.iID1
ORDER BY avgrat LIMIT 50;
the problem is....with millions of entries in the table, SUM() and GROUP BY would freeze up the query....is there a way to do exactly this that would execute instantaneously using MySQL and/or PHP hacks (perhaps do the summing with PHP....but how would I go about doing that...)
To answer the direct question: no, there is no way to do anything instantaneously.
If you have control over the table updates, or the application which adds the relevant records, then you could add logic which updates another table with the sum, count, and id with each update. Then a revised query targets the "sum table" and trivially calculates the averages.
One solution is to create a rollup table that holds your aggregate values
using a triggers on your source tables to keep it up to date.
You will need to decide if the overhead of the triggers is less then that of the query.
some important factors are:
The frequency of the source table updates
The run frequency of the aggregate query.

Categories