I need to do some calculation each time when a user click "View Statistics" link. The calculation is based on a query. It's too complicated for me to figure out how to re-group the query result I posted a question about this before . Now I'm thinking to create a view of MERGE type first. Then, every time when need to show statistics, I can just query from this view.
I have little knowledge about mysql. Not sure if this is the normal way to do the statistics calculation.
When does the view get updated? Does it keeps updating whenever a new entry related to this View is posted? Or, it updates only when being queried?
Should I delete the view every time after calculation?
You could use stored procedures as an alternative to views. This way you know that your data is freshly grabbed from your tables.
A view is not updated when data is inserted, it is simply a predefined query you can use.
If you have a need for complex calculations to create statistics, maybe create one or more triggers to help you to maintain a statistics table
Related
I am having an issue querying a couple of MySQL VIEWS.
The MySQL tables are generated hourly from a process that pulls data from a central Oracle database. Once the update is complete, a query automatically updates the VIEWS. (I do believe this is standard. I could be wrong).
At this point, I am using PHP to query said VIEWS to return data to the page. This is where my problem begins.
When the query begins, it can take well over 30 seconds to return 4 records. The VIEW contains no more than 30K records. But when I try to return a larger data set to the page, it now causes the server to freeze.
The query from my PHP script is very simple. It is looking like the following:
SELECT
DISCHARGE_ETA,
TERMINAL_DISCH,
IMPORT_RAMP,
ORIG_RAMP_FINAL_DEST_ETA,
POOL_LOCATION_FROM_IMP,
POOL_LOCATION_TO_IMP,
ROAD_HAULIER_IMP
// few more columns
FROM
view1
WHERE
" . $_SESSION['where'] . ";
I am using jQuery to send parameters over to the PHP script. PHP then builds the query by plugging all the parameters into the WHERE clause. The data is then returned via JSON.
The query that builds the actual VIEW in MySQL is a little more intense (currently not shown here).
My question is: are VIEWS the best route in retrieving the data I need?
I was thinking it would seem easier to query a TABLE instead of a VIEW. Perhaps the same hourly process that is updating the VIEWS could instead update a single TABLE. That way, I could query the TABLE instead of the VIEW.
Would this be the best bet? If not, are there any alternatives?
Edit
Here is the results from EXPLAIN
I have a database containing many tables : [followers, favorites, posts ...etc)
These tables define the different activities a user can achieve, he can send posts, add other people to favorites and follow others.
What I want to do.. is to extract data from these tables, and build a real-time news feed .
I have two options:
1- Creating a separate table for the notifications (so that I won't have to get data from multiple tables, then using a Javascript timer to return results every x seconds.
2- using XMPP server...that sends (or pushes) notifications every x second, without actually sending any ajax queries.
And for these two options, I don't know whether I should connect to these tables to get news feed, or just create a separate table for notifications.
I searched in the subject, but I didn't find something really helpful yet, Any links will be appreciated.
If your data is normalized, you should be able to pull all the data with one query (using JOINs), or you could try creating a View if you want to query from one table. It's always best to keep your data in the appropriate tables to avoid having to duplicate data.
The push notifications are easier on the server, since they're not receiving requests from each client. Depending on your load, you could probably get away with simple AJAX requests.
The request for news feed will be very frequently. So you must keep your code run fast and take as less resource (CPU-time, database query) as possible.
I suggest you to take the first option. It meets your requirement and is simple enough.
Because you have many tables, all of them will grow bigger day by day. Every time you connect them to get news feed will take a long time and increase the load of your database. In the other hand, you query sql will be complex.
Like #Curtis Mattoon said: avoid having to duplicate data, but sometime, we need spend more space for less time.
So I suggest to create a new table to store the notification data. You even can delete the old data from this table periodically.
At the same time, your sql and php code for news feed will be simple and run fast.
I want to write a top articles system.
I want to filter some content (articles/etc..) by number of views.
If I insert in database with views=views+1 every time when a user view that link, I think it's slowly and it's a bad practice.
An example of another site that does this is YouTube. It updates this table only at a certain interval, so the views aren't updated live. Is this a good practice to do this?
You could create a log (simple text file / xml / Json ………) to store the view count and do a job to parse the file and insert the result into the DB.
This job could run in time intervals or check the system for idle processor.
In my opinion, using the RAM to store sensitive data as this count (an important part of your system) seems kind of insecure.
Create a second table of views; you can also use this second table to filter "duplicate" views (backed too by cookies - removing duplicate views won't be a perfect operation).
As for being "slow" I've used that approach on many sites with many users. Storing and aggregating data is what databases do, so use their power.
Then you can use this second table to total the views, or periodically sum up the data and store the running total in the main table, clearing out the second table to keep space down. I usually keep all the data but demoralize for speed.
Google unfortunately didn't seem to have the answers I wanted. I currently own a small search engine website for specific content using PHP GET.
I want to add a latest searches page, meaning to have each search recorded, saved, and then displayed on another page, with the "most searched" at the top, or even the "latest search" at the top.
In short: Store my latest searches in a MySQL database (or anything that'll work), and display them on a page afterwards.
I'm guessing this would best be accomplished with MySQL, and then I'd like to output it in to PHP.
Any help is greatly appreciated.
Recent searches could be abused easily. All I have to do is to go onto your site and search for "your site sucks" or worse and they've essentially defaced your site. I'd really think about adding that feature.
In terms of building the most popular searches and scaling it nicely I'd recommend:
Log queries somewhere. Could be a MySQL db table but a logfile would be more sensible as it's a log.
Run a script/job periodically to extract/group data from the log
Have that periodic script job populate some table with the most popular searches
I like this approach because:
A backend script does all of the hard work - there's no GROUP BY, etc made by user requests
You can introduce filtering or any other logic to the backend script and it doesn't effect user requests
You don't ever need to put big volumes of data into the database
Create a database, create a table (for example recent_searches) and fields such as query (the query searched) and timestamp (unix timestamp that the query was made) said, then for your script your MySQL query will be something like:
SELECT * FROM `recent_searches` ORDER BY `timestamp` DESC LIMIT 0, 5
This should return the 5 most recent searches, with the most recent one appearing first.
Create table (something named like latest_searches) with fields query, searched_count, results_count.
Then after each search (if results_count>0), check, if this search query exists in that table. And update or insert new line into table.
And on some page you can just use data from this table.
It's pretty simple.
Ok, your question is not yet clear. But I'm guessing that you mean you want to READ the latest results first.
To achieve this, follow these steps:
When storing the results use an extra field to hold DATETIME. So your insert query will look like this:
Insert into Table (SearchItem, When) Values ($strSearchItem, Now() )
When retrieving, make sure you include an order by like this:
Select * from Table Order by When Desc
I hope this is what you meant to do :)
You simply store the link and name of the link/search in MySQL and then add a timestamp to record what time sb searched for them. Then you pull them out of the DB ordered by the timestamp and display them on the website with PHP.
Create a table with three rows: search link timestamp.
Then write a PHP script to insert rows when needed (this is done when the user actually searches)
Your main page where you want stuff to be displayed simply gets the data back out and puts them into a link container $nameOfWebsite
It's probably best to use a for/while loop to do step 3
You could additionally add sth like a counter to know what searches are the most popular / this would be another field in MySQL and you just keep updating it (increasing it by one, but limited to the IP)
I am writing a PHP/MySQL application (using CodeIgniter) that uses some jQuery functionality for dragging table rows. I have a table in which the user can drag rows to the desired order (kind of a queue for which I need to preserve the rank of each row). I've been trying to figure out how to (and whether I should) update the database each time the user drops a row, in order to simplify the UI and avoid a "Save" button.
I have the jQuery working and can send a serialized list back to the server onDrop, but is it good design practice to run an update query this often? The table will usually have 30-40 rows max, but if the user drags row 1 far down the list, then potentially all the rows would need to be updated to update the rank field.
I've been wondering whether to send a giant query to the server, to loop through the rows in PHP and update each row with its own Update query, to send a small serialized list to a stored procedure to let the server do all the work, or perhaps a better method I haven't considered. I've read that stored procedures in MySQL are not very efficient and use a separate process for each call. Any advice as to the right solution here? Thanks very much for your help!
Any question that includes "The table will usually have 30-40 rows max" ends with "Do whatever you want to it." I can't imagine an operation, however frequently it's performed, that would have any appreciable performance impact on a table that tiny.
The only real question is what the visitor will be doing while your request is going to and returning from the server. Will they be locked out of making other changes? If not, make sure you have a mechanism to ensure that the most recent change is the one that's really taken effect. (It's possible for requests to reach the server out of order, and you wouldn't want an outdated request to get saved as the final state.)