I created a view that is a join between two tables a.2_9 (18.483 rows) and b.2_10 (5700 rows)
it is a very simple join:
Create view v_2_9_join_2_10 AS
Select *
FROM 2_9 AS a
LEFT JOIN 2_10 b
ON a.2_9_tid = b.2_10_tid
AND a.2_9_region = b.2_10_region
AND a.n_2 = b.n_2
I to create the view and count the rows (18483) but for some reason it takes forever when I try to use the view, count or export or just browse and it is slowing down the database in PHP too much.
Any ideas of how I can look for the reason(s) of the slowing down or optimize the join?
To know why a query is being slow you can issue the query you used to create the view with the EXPLAIN keyword, this will show a little table with data that explains how MySQL executed the query so you can identify the bottlenecks there.
HERE you can find a little tutorial on how to understand the output of EXPLAIN.
Related
Recently I have develop an apps for processing student result. The algorithm is below:
Algorithm:
get list of all students from mysql
if found
get list of subject from mysql
get sum of marks from mysql for each student and subject
calculate pass,fail,grade point etc.
insert total marks into another table
//end subject list
error msg if no student
// end
this takes huge time.
now I like to speed up with jQuery.
For example,
load all student id and subject id into jQuery array. then send a request to PHP to process result using jQuery loop or any other way.
What you are saying does not make much sense. However, from your algorithm description above it looks like you are selecting students, subjects, marks in loops and hitting your DB for each student's subject's marks. Instead, use table joins (left join, inner join, etc.).
SELECT s.*, sb.*, SUM(m.value)
FROM students AS s
LEFT JOIN subjects as sb ON s.id = sb.student_id
INNER JOIN marks AS m ON sb.id = m.subject_id
GROUP BY s.id, sb.id
With this query you are replacing your 1000 queries with one in database calculation query. MySql will run the query in more optimised manner and require less resources.
It's pretty hard to say what is your real problem, but it sounds like first thing you should do is to avoid those loops. Maybe you could move part of this logic to the database using some stored procedures?
Having loads of database shots may affect performance significantly.
I've been working on this for some hours now and it's getting tiring. I want to get users posts from people I follow and myself, just like twitter does.
So there's a table users_posts that has all users' posts with column user_id to determine who made the post.
Another table users_followers that contains all followers ie. user_id follows following_id
So here's my query:
User ID = 2271
SELECT users_followers.following_id AS following_id, users_posts.id
FROM users_followers, users_posts
WHERE users_posts.user_id = users_followers.following_id
AND (users_followers.user_id =2271)
This query works but the problem is, it's kinda slow. Is there a faster way to do this?
Also, as you can see, I can only get posts from those I follow, and not myself.
Any help?
If I'm understanding your tables properly, I would do this with an explicit JOIN. I'm not sure how much that would speed things up versus the implicit JOIN you're using though.
SELECT following_id, p.id
FROM users_followers f
LEFT JOIN users_posts p
ON (p.user_id = f.following_id)
WHERE f.user_id = 2271
Chances are, adding an index or two to your tables would help.
Using the MySQL EXPLAIN command (just put it in front of your SELECT query) will show the indexes, temporary tables, and other resources that are used for a query. It should give you an idea when one query is faster or more efficient than another.
Your query is fine as written, provided you have properly indexed your tables. At a minimum you need an index on users_posts.user_id and users_followers.following_id.
A query can also be slowed by large numbers of records, even when it is fully indexed. In that case, I find phpmyadmin to be an invaluable tool. From the server page (maybe localhost) select the Status tab to see a wealth of information about how your mysql server is performing and suggestions for how to improve it.
I am building a web application for a customer and I have a problem with a particular SQL query.
The query is:
select order_header.order_no,
order_header.purchase_order_no,
order_header.entry_date,
order_header.delivery_date,
order_totals.total_quantity
from order_header,
order_totals
where order_header.order_no = order_totals.order_no
I have done some troubleshooting and this:
where order_header.order_no = order_totals.order_no
is the problem. The SQL query with this line takes 35 seconds (causes DataTables to even time out at times) and without it it is instant.
So, I know the problem but I'm not a DBA so don't know the solution.
It's not my database, so the solution I need to send to the DBA to sort so I can continue with my job. Something like
"Hey, would you mind doing A on B table so that C speeds up?"
I just don't know what actually needs to be done!
First add an index on both order_header.order_no and order_totals.order_no and check that both the columns are of the same type.
For other optimizations we should talk about the data.
Don't forget to update the statistics
Ask your DB to
add an index on the order_no column in both order_header and order_total
That should help with your current problem. Also, look up JOIN and change your query to use the JOIN syntax.
select order_header.order_no,
order_header.purchase_order_no,
order_header.entry_date,
order_header.delivery_date,
order_totals.total_quantity
from order_header
join order_totals ON order_header.order_no = order_totals.order_no
I have one big table.
Let's call it 'unicorns'.
And second table 'farmers'.
Structure:
farmers
id
name
... some more fields
unicorns
id
farmer_id
... some more fields
And there is query:
SELECT
f.id,
f.name,
COUNT(*)
FROM
farmers f
LEFT JOIN unicorns u
ON u.farmer_id = f.id
GROUP BY f.id
Of course tables are really named not this way and have more fields etc. But I have left only the most important things in my example.
The problem is that the system grows, and there are too many unicorns. Millions.
And this query (like-this) is executed on farmers list page.
And page is loading not so fast as before, because we join a multi-million table each time we load page.
Problem is:
We really need to display each farmer's unicorns count in the list.
We need to improve page load speed.
If you would need to optimize this, how would you achieve this result?
What would you recommend?
P.S.
Personally I think I need to exclude big-table-join from the main query, calculate unicorns counts separately and store them in the cache storage, recalculate them time after time. Maybe there is the best way, so I wanted to hear someone else's opinion.
I would just have an extra column on the farmer's table for NumberUnicorns and store it there. Then, via some database triggers, when a record is added to the unicorn table, it just updates the farmers table with the respective count from the new record. Also consider updating the count if unicorn records are deleted. Then, your query is immediately from the farmers table -- done.
I have a system where a user is part of a series of "runs", to each "run", can be added courses, teachers(users), classes and so on.
Each teacher(user) has chosen his/her classes & courses.
Here's a run-down of the tables I have that are relevant:
lam_run - The run in it self.
lam_run_course - Relational table that shows what runs has what courses
lam_teacher_course - Relational table that shows which teacher has which courses
lam_run_teacher - Relational table that shows what teachers are in what courses
What I want to do is show each teacher which runs that are relevant to them (based on which courses they have selected seen in lam_teacher_course) but in which they are not already participating.
Here's the MySQL code I have so far that does not work:
$query_relevant_runs = "
SELECT DISTINCT
lam_run_course.run_id
FROM
lam_teacher_course,
lam_run_course,
lam_run, lam_run_teacher
WHERE
lam_teacher_course.user_id = '1'
AND
lam_teacher_course.course_id = lam_run_course.course_id
AND
lam_run_teacher.user_id != '1'";
Instead this code shows all runs that are relevant, but it doesn't exclude the runs the user is already in..
What can I do to fix this?
Ps. Sorry for bad title, no idea what I should've called it :S
Here is a link to part of the databases (the relevant part): Link!
I think what you're looking for is:
LEFT JOIN `lam_run_teacher` `lam_run_teach_exclude`
ON `lam_run_teacher_exclude`.`user_id` = `lam_teacher_user`.`user_id`
...
WHERE `lam_run_teacher`.`user_id` IS NULL
The LEFT JOIN takes your current query, and appends the additional data to it. However, unlike the INNER JOIN you are using now (using the kinda-strange multiple-from syntax), the LEFT JOIN does not limit your resultset to just those where there is data for the righthand side. The righthand columns will be NULL. By filtering on that NULL, you can find all runs that are interesting, and for which there is not yet a relation to the teacher.
Does this help?
I'd recommend always using the normal join syntax (INNER JOIN target on target.id = source.id) - that way you're more aware of the idea that there are other kinds of join as well, and all your joins will look identical. It takes some getting used to, but definitely helps when your queries get more complex.
Also, in your cross-referencing tables, you can drop the primary key columns. If the only purpose of a table is to define a link between two tables, make the primary key consist of the two keys you've got. Unless you want to be able to related the same teacher to a run multiple times...
OK, took me way longer than it should have, but here's the complete thing:
SELECT
DISTINCT `lam_run_course`.run_id
FROM
`lam_run_course`
INNER JOIN
`lam_teacher_course`
ON `lam_teacher_course`.course_id = `lam_teacher_course`.course_id
LEFT JOIN
`lam_run_teacher` ON (`lam_run_teacher`.`run_id` = `lam_run_course`.`run_id` AND `lam_run_teacher`.`user_id` = 3)
WHERE
`lam_teacher_course`.user_id = 3
and `lam_run_teacher`.`run_id` IS NULL