Got big table (~1.6m rows), which looks like:
------------------------------------
| id | text | image_id | order |
------------------------------------
| 2 | random | 12 | 1 |
------------------------------------
| 3 | random | 12 | 2 |
------------------------------------
| 5 | random | 12 | 1 |
------------------------------------
| 6 | random | 12 | 2 |
------------------------------------
| 8 | random | 17 | 1 |
------------------------------------
| 9 | random | 17 | 1 |
------------------------------------
The goal is to have:
------------------------------------
| id | text | image_id | order |
------------------------------------
| 2 | random | 12 | 1 |
------------------------------------
| 3 | random | 12 | 2 |
------------------------------------
| 8 | random | 17 | 1 |
------------------------------------
Many rows with different ids but other data is the same, so we need to keep only one row of each order number (111222333444 need to be 1234).
This query is working for small tables:
DELETE n1 FROM table n1, table n2 WHERE image_id = 12 AND n1.id > n2.id AND n1.order = n2.order
But for big tables query takes too long, so receive timeout.
DB Backend is Laravel & PHP, and we use chunks to query this tables.
Basically goal is to crawl through big table based on image_id and remove duplicates so for each image_id we got rows with order columns like: 1,3,4,5,6,7,8 etc.
Just use unique() method from laravel
$uniqueData = Model::unique(function ($item) {
return $item['image_id'].$item['order'];
});
$uniqueData->values()->all();
return $uniqueData;
Related
this is my database tables
Lists
Requests
products
Every request has one product and every list has many requests
now i want to get the min and max price/size for the products in list X , how can i achieve this with Laravel Elequent ?
Lists table
+----+----------+
| id | user_id |
+====+==========+
| 1 | 1 |
+----+----------+
| 2 | 1 |
+----+----------+
| 3 | 2 |
+----+----------+
Requests table
+----+-------------+----------+
| id | product_id | list_id |
+====+=============+==========+
| 1 | 1 | 1 |
+----+-------------+----------+
| 2 | 1 | 1 |
+----+-------------+----------+
| 3 | 2 | 2 |
+----+-------------+----------+
Products table
+----+-------+------+-------+
| id | price | size | other |
+====+=======+======+=======+
| 1 | 110 | 10 | test |
+----+-------+------+-------+
| 2 | 130 | 12 | test |
+----+-------+------+-------+
| 3 | 100 | 24 | test |
+----+-------+------+-------+
| 4 | 200 | 16 | test |
+----+-------+------+-------+
I want to get the min and max price/size for the products in list X
$maxSize = Product::whereHas('request.list', function($q) use($listId) {
$q->where('list.id', $listId);
})
->max('size');
$minSize = Product::whereHas('request.list', function($q) use($listId) {
$q->where('list.id', $listId);
})
->min('size');
Do the same for the min and max price.
Back in January, I received some very helpful information about consolidating multiple database queries into one "mega" query, it can be found HERE.
I have expanded on one of the sub queries, this one selects a random photo from the photos table for display with each and every project shown on the page. For starters, here is the table from the original post HERE:
+----------+------------+--------+
| photo_id | project_id | active |
|--------------------------------|
| 1 | 1 | 1 |
| 2 | 1 | 1 |
| 3 | 1 | 1 |
| 4 | 2 | 1 |
| 5 | 2 | 1 |
| 6 | 2 | 1 |
| 7 | 3 | 1 |
| 8 | 3 | 1 |
| 9 | 3 | 1 |
+----------+------------+--------+
This is the subquery recommended to me in the post mentioned HERE
(SELECT photo_id FROM Photos
where project_id = p.project_id ORDER BY RAND LIMIT 1) as random_photo,
I've added additional columns, filename, gallery number to which each image is associated, width of each image and height of each image.
+----------+------------+----------+--------+---------+-------+--------+
| photo_id | filename project_id | active | gallery | width | height |
|----------------------------------------------------------------------|
| 1 | pic1.jpg | 1 | 1 | 1 | 600 | 400 |
| 2 | pic2.jpg | 1 | 1 | 1 | 600 | 400 |
| 3 | pic3.jpg | 1 | 1 | 1 | 400 | 600 |
| 4 | pic4.jpg | 2 | 1 | 2 | 600 | 400 |
| 5 | pic5.jpg | 2 | 1 | 2 | 600 | 400 |
| 6 | pic6.jpg | 2 | 1 | 2 | 600 | 400 |
| 7 | pic7.jpg | 3 | 1 | 3 | 400 | 600 |
| 8 | pic8.jpg | 3 | 1 | 3 | 400 | 600 |
| 9 | pic9.jpg | 3 | 1 | 3 | 400 | 600 |
+----------+------------+----------+--------+---------+-------+--------+
I updated the query to select only images whose width was greater than the height so as to only select a landscape image:
(SELECT filename FROM photos
WHERE project_id = p.project_id AND width > height ORDER BY RAND() LIMIT 1) as random_photo,
And this works as desired.
My new hurdle is wanting to select both the filename AND the associated gallery number for the random image and this is where I'm stuck. In my original post (HERE) there is another subquery that selects a list of tags from another table and it uses SELECT GROUP_CONCAT so I tried that, in a number of ways but this is clearly an area I'm in way over my head.
I also tried using random_photo within a second subquery like this:
(SELECT gallery FROM photos
WHERE filename = random_photo) as gallery_number,
and it didn't work. Ideally, I'd like to retrieve a random photo filename and its associated gallery number in one shot. My apologies if I'm missing something obvious here, and thanks in advance for your help.
The simple but a bit awkward solution would be:
(SELECT Concat(gallery,'___',filename) FROM photos
WHERE project_id = p.project_id AND width > height ORDER BY RAND() LIMIT 1) as random_photo
If you use above solution you need to split the result at the first occurence of '___' in your php-code.
The more complicated solution would be selecting a random id and than joining the result to the photos-table, but it depends on the other parts of your 'mega-query' how to do this.
I am making a management system for a site that has a bunch of image galleries. The interface will allow the user to add, delete, or reorder images within each gallery.
I have a table for all the images across all galleries together, with 'id' auto-incremented, and default for sort_order set to 0.
+----+-----------+------------+------------+
| id |gallery_id | sort_order |
+----+-----------+------------+------------+
| 1 | 1 | 1 |
| 2 | 1 | 0 |
| 3 | 1 | 2 |
| 4 | 1 | 3 |
| 5 | 1 | 4 |
| 6 | 1 | 5 |
| 7 | 1 | 6 |
| 8 | 2 | 0 |
| 9 | 2 | 1 |
| 10 | 2 | 2 |
| 11 | 2 | 3 |
| 12 | 2 | 4 |
| 13 | 2 | 5 |
| 14 | 2 | 6 |
+----+-----------+------------+------------+
Here is the reorder query, using a serialized array via ajax:
if($_POST['item']) {
$order = 0;
foreach ($_POST['item'] as $id) {
$rearrange = $db->query
("UPDATE images SET sort_order = '".$order."' WHERE id = '".$id."'");
$order++;
It limits sort_order according to the length of the array that is passed, and then defaults to 0 for the next group its comes to. I can reorder, add, delete, and it always restricts changes to the gallery_id group I am editing.
(As an experiment, I inserted sort_order values ascending across all galleries, like this:
+----+-----------+------------+------------+
| id |gallery_id | sort_order |
+----+-----------+------------+------------+
| 1 | 1 | 0 |
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 3 | 1 | 3 |
| 4 | 1 | 4 |
| 5 | 1 | 5 |
| 6 | 1 | 6 |
| 7 | 1 | 7 |
| 8 | 2 | 8 |
| 9 | 2 | 9 |
| 10 | 2 | 10 |
| 11 | 2 | 11 |
| 12 | 2 | 12 |
| 13 | 2 | 13 |
| 14 | 2 | 14 |
+----+-----------+------------+------------+
And when I reordered rows using my interface, it limited the sort_order within gallery_id groups, exactly as shown in the first table).
I'm looking for caveats to this approach. Anyone done it this way?
UPDATE images
SET
sort_order = FIELD(id, A, B, C, D, E)
WHERE gallery_id = N
AND id IN(A, B, C, D, E)
I do it the same way, but I actually put the image inserted last by calculating the number of rows in each gallery. When a user wants to order he has to filter the list for a specific gallery first then he can sort the images in the gallery.
I cannot but point out the terrible security issues with your code. I really, really, really hope that you are not actually writing the queries like that as anybody can easily do whatever they want with your database.
http://xkcd.com/327/ comes to mind. If I submit an item like 0'; drop table images; your entire table is gone.
Would like to get the following as a result from the table structure below (MYSQL + PHP)
array[0][name]1,[desc]red,[title]hero,[desc]strong,[desc2]smells,[img][0]red1,[img][1]red2,[img][2]red3,ext[0].jpg,[ext][1].gif,[ext][2].png,[count][0]253,[count][1]211,[count][2]21,[count][3]121,[dist][0]5,[dist][1]5,[dist][2]12,[dist][3]2,[score][0]2,[score][1]3,[score][2]1,[score][3]5,[score][4]4,[val][0]5,[val][1]1,[val][2]4,[val][3]3,[val][4]4
The problem I have with a simple SELECT, JOIN and GROUP_CONCAT is that the values duplicate after selecting all the images.
I've tried various other ways for example selecting the data by row combined with a foreach loop in PHP, but I end up with lots of duplicates, and it looks very messy.
I also though about splitting it into multiple selects instead of using one, but I really would like to know if it can be done with one select.
Could someone help me with an MYSQL select? Thanks
game
+-----+----------+
| pid | name |
+-----+----------+
| 1 | red |
| 2 | green |
| 3 | blue |
+-----+----------+
detail
+-----+------+--------+-------+--------+
| id | pid | title | desc | desc 2 |
+-----+------+--------+-------+--------+
| 1 | 1 | hero |strong | smells |
| 2 | 2 | prince |nice | tall |
| 3 | 3 | dragon |big | green |
+-----+------+--------+-------+--------+
image
+-----+-----+-----+----+
| id | pid | img |ext |
+-----+-----+-----+----+
| 1 | 1 | red1|.jpg|
| 2 | 1 | red2|.gif|
| 3 | 1 | red3|.png|
+-----+-----+-----+----+
devmap
+-----+-----+-------+------+
| id | pid | count | dist |
+-----+-----+-------+------+
| 1 | 1 | 253 | 5 |
| 2 | 1 | 211 | 5 |
| 3 | 1 | 21 | 12 |
| 4 | 1 | 121 | 2 |
+-----+-----+-------+------+
stats
+-----+-----+-------+------+
| id | pid | scrore| val |
+-----+-----+-------+------+
| 1 | 1 | 2 | 5 |
| 2 | 1 | 3 | 1 |
| 3 | 1 | 1 | 4 |
| 4 | 1 | 5 | 3 |
| 5 | 1 | 4 | 3 |
+-----+-----+-------+------+
When you do a JOIN that involves more than a 1:1 mapping between tables you're going to have duplicate data, and there's no way to get around that in the query.
You can break it out into multiple selects, or you can loop through the result set and pare out whatever duplicate information you don't want.
This question already has an answer here:
Closed 10 years ago.
Possible Duplicate:
mysql with comma separated values
I have two tables with more lines, like this:
1. numberstable
---------------------------
| number_id | number_name |
---------------------------
| 1 | one |
| 2 | two |
| 3 | three |
| 4 | four |
.
.
.
---------------------------
2. testtable
-------------------------------
| test_id | numbers | sthelse |
-------------------------------
| 1 | 2.4.5.6 | text1 |
| 2 | 4.8.7.1 | text2 |
| 3 | 2.7.8.5 | text3 |
-------------------------------
First I would like to combine all three "numbers" rows from table "testtable" to get something like this: 1.2.4.5.6.7.8 and then exclude it in next query. This query is "SELECT number_id, number_name FROM numberstable ORDER BY number_name". After excluding I would like to show just numbers which aren't in use in "testtable" (9, 10, 11, ...).
How to do that?
If you are trying to relate the numberstable to the testable. I would think you would be much better served to add another table that would relate the two to where you had a schema like
1. numberstable
---------------------------
| number_id | number_name |
---------------------------
| 1 | one |
| 2 | two |
| 3 | three |
| 4 | four |
.
.
.
---------------------------
2. testtable
---------------------
| test_id | sthelse |
---------------------
| 1 | text1 |
| 2 | text2 |
| 3 | text3 |
--------------------
3. numbersintesttable
-----------------------
| test_id | number_id |
-----------------------
| 1 | 2 |
| 1 | 4 |
| 1 | 5 |
| 1 | 6 |
| 2 | 4 |
| 2 | 8 |
| 2 | 7 |
| 2 | 1 |
-----------------------
So the new table would be a many-to-many join table that you could use to get all your needed data in a single query by utilizing the type of joins you want (INNER, OUTER, etc.)