Laravel / MYSQL (Eloquent) - Querying large table - php

I am making an online directory, this directory contains businesses, this is how the current table structure is set out:
1) "Business"
ID (PK)
Name
Phone_Number
Email
2) Tags
id (PK)
tag
3) Business_tags
id (PK)
business_id (FK)
tag_id (FK)
There are over 9k rows inside of the business table, and over 84,269 rows and there are over 29k rows inside the ("Business_tags") table (As a business can have multiple tags).
Inside the business model, is the following:
public function tags()
{
return $this->belongsToMany('App\Tags');
}
The issue is when I am trying to do a search, so for example, let's say that someone wants to search for a "Chinese" then it's takes more time than it probably should to return a value. For example, I am using:
$business = Business::where(function ($business) use ($request) {
$business->whereHas('tags', function ($tag) use ($request) {
});
})->paginate(20);
Searching takes on average: 35 seconds to display the results.
Here is the raw sql:
select * from `businesses` where (exists (select * from `tags` inner join `business_tags` on `tags`.`id` = `business_tags`.`tags_id` where `business_tags`.`business_id` = `businesses`.`id` and `name` in ('chinese')))
This takes on average: 52.4s to run inside Sequel pro (Using the raw SQL statement)
Any ideas how I can improve the performance of this query so that it's a lot faster? I want to have this functionality, but the user is not going to wait this long for a response!
EDIT:
1 PRIMARY businesses NULL ALL NULL NULL NULL NULL 8373 100.00 Using where
2 DEPENDENT SUBQUERY business_tags NULL ALL NULL NULL NULL NULL 30312 10.00 Using where
2 DEPENDENT SUBQUERY tags NULL eq_ref PRIMARY PRIMARY 4 halalhands.business_tags.tags_id 1 10.00 Using where

You're over-complicating this, and not using eloquent relationships correctly. You should be using JOINs instead:
$businesses = Business::join('business_tags', 'business_tags.business_id', '=', 'business.id')
->join('tags', function($join) {
$join->on('business_tags.tag_id', '=', 'tags.id')
->where('tags.name', '=', 'chinese');
})->get();
Or in raw SQL:
SELECT *
FROM `business`
INNER JOIN `business_tags` ON `business_tags`.`business_id` = `business`.`id`
INNER JOIN `tags` ON `business_tags`.`tag_id` = `tags`.`id` AND `tags`.`name` = 'chinese'
(Note that you could put that tags.name = 'chinese' part in the WHERE clause and yield the same effect)
Your current query does an exists subquery to get all the records from the pivot table that match the criteria, then passing that back to the main query. It's an extra step, and it's unnatural.
Eloquent relationships are NOT for complex queries like this, but are rather there to provide additional, related information about a record without having to write another query manually.
For instance, if you want to view a business, you might query with() phone numbers and addresses from other tables. You might want to list out their tags, or sync() them. But eloquent does not build and filter queries, that's what query builder is for.
Let me know if you need more explanation.

As a lot of others are also going to tell you.
Have you run EXPLAIN on your query?
Have you added indexes to your tables?
Because even with the amount of data you have mentioned the query should have been faster than what you have reported.
Also see if a JOIN can work here and if faster?(just a thought)

Related

Eloquent query to search over two tables

I am struggling with the following query, using Eloquent in Laravel 5.6.
I need to return all issues that have a tag_id of 5 assigned to them, where the project_id and item_id from the issues table matches the project_id and issue_id from my pivot table.
issues table:
issues_tags pivot table:
I have tried the following code, but it returns all issues from the issue table, however the expectation is 3 results.
Expected results
The results returned from the issues table should be ID 1, 4 and 5.
$issues = Issue::join('issues_tags', 'issues_tags.project_id', 'issues_tags.issue_id')->where('issues_tags.tag_id', 5)->select('issues.*')->get();
You need to specify the issues table instead of issues_tags on the join. A left join will also help reduce the results. Since you're joining on two different keys, you have to use a closure.
$issues = Issue::leftJoin('issues_tags', function($join) {
$join->on('issues.project_id', '=', 'issues_tags.project_id');
$join->on('issues.item_id', '=', 'issues_tags.issue_id');
})
->where('issues_tags.tag_id', 5)->select('issues.*')->get();
If the table is really supposed to match on project_id->project_id and issues.id -> issues_tags.issues_id, you can modify the 2nd join clause.

Laravel Querybuilder join on LIKE

I am attempting to join two tables using the Laravel's query builder however I seem to be having an issue getting the desired result using the query builder, I can however get it quite simply using a raw SQL statement. I simply want to return all mod rows that have the corrosponding value in the tag column in the tags table.
Schema
--------------
mods
--------------
id - int - (primary key)
name - varchar
--------------
tags
--------------
id - int
modid - int - (primary key of its parent mod)
tag - varchar
Working SQL query
SELECT * FROM mod JOIN tags ON tags.tag LIKE '%FPS%'
Query Builder
DB::table('mods')
->join('tags', function ($join) {
$join->on('tags.tag', 'like', '%FPS%');
})
->get();
Currently this is telling me: Unknown column '%FPS%' in 'on clause' but I am unsure how else to structure this. I intend on adding more orOn clauses as well as I will want to get results on multiple tags but firstly I just want to get a single tag working.
Appreciate any help.
SELECT * FROM mod JOIN tags ON tags.tag LIKE '%FPS%'
Your query builder is refusing to generate this query because it's nonsense!
To work correctly, a JOIN clause needs to compare two columns for equality -- one column on each side of the join table. A JOIN clause that doesn't do this is functionally "downgraded" to a WHERE clause. In the case of this query, the two tables end up cross joined.
What you probably want is:
SELECT * FROM mod
JOIN tags ON tags.modid = mod.id
WHERE tags.tag LIKE '%FPS%';
$join->on('tags.tag', 'like', '%FPS%');
Try by replacing
$join->where('tags.tag', 'like', '%FPS%');
This because the on method waiting a name of a field not a query value if you want it to deal with it in this way, you should use DB::raw('%FPS%').
Maybe you are trying to do something like the following:
DB::table('mods')
->select(DB::raw('mods.id as modid, mods.name, tags.id as tagid, tags.tag'))
->join('tags', function ($join) {
$join->on('tags.modid', '=', 'mods.id');
})
->where('tags.tag', 'like', '%FPS%')
->get();
If you want to see what is run in the database use
dd(DB::getQueryLog())
to see what queries were run.
Try this
Thank you

How get table with references other table in yii2?

i have multiple table that join together and i need one query and get all references too ! is that possible in yii2??
get them in hierarchy array ??
How ???
Is it possible to do not use join???
thanks for your help!!!!
If you created the model classes for each table using Gii and selected to create the relations in the generated models, you can do something like the following.
1) In your Countries model just change the method that declares the relationship with Airports like this:
public function getAirports() {
return $this->hasMany(Airports::className(), ['country_id' => 'id'])->with('airlines');
}
2) When you do the query for the countries and you need to have the related airports, airlines and flightbooked do it like this:
$countries = Countries::find()
->where('something = something_else')
->with('airports')
->with('flightbooked')
->all();
This way you will get all the related models populated with way less queries to the database than using lazy-loading.
I just wanted to give a small suggestion:
As you are maintaining the relations in the tables and if you have generated your code using Gii, then that will generate the joins for you. You can then access any column of any table easily.
But I think UNION may not be an alternative to JOIN.
Maybe u can use union all for this. with this operator, you can concatenate the result sets from multiple queries together, preserving all of the rows from each. Note that a UNION operator (without the ALL keyword) will eliminate any "duplicate" rows which exist in the resultset. The UNION ALL operator preserves all of the rows from each query (and will likely perform better since it doesn't have the overhead of performing the duplicate check and removal operation).
The number of columns and data type of each column must match in each of the queries. If one of the queries has more columns than the other, we sometimes include dummy expressions in the other query to make the columns and datatypes "match". Often, it's helpful to include an expression (an extra column) in the SELECT list of each query that returns a literal, to reveal which of the queries was the "source" of the row.
SELECT 'col1' AS source, col23, col343, col33, d FROM table1 WHERE ...
UNION ALL
SELECT 'col2', t2.fee, table2.fi, table2.fo, 'fum' FROM table2 JOIN table3 ON ...
UNION ALL
SELECT 'col3', '1', '2', buckle, my_shoe FROM table4
You can wrap a query like this in a set of parenthesis, and use it as an inline view (or "derived table", in MySQL lingo), so that you can perform aggregate operations on all of the rows. e.g:
select one.a
, SUM(one.b)
FROM (
SELECT 'q1' AS source, a, b, c, d FROM t1
UNION ALL
SELECT 'q2', t2.fee, t2.fi, t2.fo, 'fum' FROM t2
) one
GROUP BY one.a
ORDER BY one.a
But i think joining tables more suitable. Hope help you

SQL Join can't figure it out

I have a table called website that contains some data about websites. The columns of this table are: id, website, quick_url, user_id, status, etc.
Each website that is in the table was added by a user, which is is saved in the user_id column.
I have another table called blocks that has only 3 columns: id, user_id, website_id.
I want to get all the websites from the website table, that were not added by a given user_id, but also, only the websites that were not blocked by the given user_id. So, websites that were not added by a given user or blocked by him.
Here is what I've tried:
SELECT * FROM website LEFT OUTER JOIN blocks ON tbl_website.userid = blocks.user_id WHERE website.user_id = blocks.user_id AND blocks.user_id = NULL AND website,user_id != '177' LIMIT 500;
It doesn't give me the wanted results ...
First, I've tried to do it like this:
SELECT * FROM tbl_website WHERE id<>(SELECT website_id from tbl_website_blocks WHERE user_id = '177')
which makes much more sense for me than my previous query, but I get this error: Subquery returns more than 1 row
I guess you can't have a "loop in loop" in an SQL query.
I'm aware that I could do two queries, and filter the results, but I would like to do it as much as possible from the SQL language, so that I don't "overload" the server.
Any suggestions would be appreciated.
In your second query rewrite the condition on
WHERE id not in (SELECT website_id from.....)
with <> you can compare it with just one value but your select returns list of values, so you can use not in to get results that are different then the selected list of IDs
Instead of '<>', try 'Not In'
SELECT * FROM tbl_website
WHERE id Not In (SELECT website_id from tbl_website_blocks WHERE user_id = '177')
I should also add this query is not a Join.

Laravel 4: How to remove `limit 1` restriction from Find() in Eloquent?

I want to retrieve all the tasks details (task_title etc) belonging to one author_id (author2 in this case).
Tests table
author_id task_id
author2 task_1
author2 task_2
Tasks table
task_id task_title
task_1 task_title_1
task_2 task_title_2
Author table
author_id author_name
author_2 authorTwo
Model test.php
public function tasks()
{
return $this->belongsTo('Task','task_id');
}
TestsController.php
public function index()
{
$test=Test::find('author2')->tasks()->get();
return View::make('tests.index', compact('tests'));
}
and the query SQL:
select * from `tests` where `author_id` = 'author2' limit 1
select * from `tasks` where `tasks`.`task_id` = 'task1'
But actually in tasks table, there are more than one value related to author 2 (in this case task 1 and task2 ) but as the sql only illustrates task1.
How can I remove the limit 1 restriction to retrieve all the tasks belonging to author 2?
There are multiple issues with that. The main ones being:
You're using the wrong relationship for a many-many, look into belongsToMany()
It looks like you're trying to look up on a composite key which is not supported
Find is intended to fetch only one record, and does not limit the result set of a relationship. As shown by your SQL, you're not limiting your tasks by one, you're limiting tests. Use get() in conjunction with where() when you need to fetch multiple records.
Here is a link to the documentation to get you started - http://four.laravel.com/docs/eloquent
Test::where("author_id","=","author2")->get()
You should not be using find for anything that is not a primary key, by the way. This is the aim of find: it fetches one item as it assumes that the key it searches for is primary auto-increment (i.e. unique)

Categories