I have a database containing more than 100,000 values. The structure looks something like as follows:
id | countryid | webid | categoryid | title | dateaddedon
If I use basic RAND() considering there are so many ids it won't be able to return a random result. I end up seeing titles of same webid next to each other. I would rather want titles from different webids being displayed. Therefore I figured since there are only 4-5 different values of webid it might be a better option to randomize the output based on this. I am unable to figure out how to define which specific column values should be randomized when using mysql SELECT command.
I am current using following
SELECT * FROM table WHERE countryid='1' ORDER BY dateaddedon DESC, RAND(3)
I am currently using 3 as seed value. I am not sure what kind of impact does seed value have on RAND. I would highly appreciate if someone could explain that too.
If seed value is specified it produces a repeatable sequence of column values. Unless you require a repeatable value leave it out. Also if you should have the RAND() as the first clause in ORDER.
SELECT * FROM table WHERE countryid='1' ORDER BY RAND(),dateaddedon DESC
Related
I'm having some trouble coming up with an efficient solution to this problem. Maybe I am making it more complicated than needs to be. I have a table like this:
thing_id | user_id | order
1 1 0
2 1 1
3 1 2
The user may mess around with their things and it may happen that they change thing 1 to thing 3, and thing 3 to thing 1. In my case, it is not that the user is explicitly changing the order. Rather, they are modifying their bank of things, and they may change the thing in slot 1 to be the thing in slot 3, and vice versa. So if the user performs this operation, the table should look like this:
thing_id | user_id | order
3 1 0
2 1 1
1 1 2
What complicates this is that (thing_id, user_id) has a unique constraint, so doing sequential updates does not quite work. If I try to UPDATE tbl SET thing_id=3 WHERE thing_id=1, the unique constraint is broken.
The order column is purely for show, in order to make an alphabetized list. So I suppose I could use PHP to check the order and figure things out like that, but this introduces code that really has nothing to do with the important stuff. I'd like to find a solution that is purely/mostly SQL.
Also, along the same lines, if I were to insert a new row into the table, I would want the order value to be 3. Is there an efficient way to do this in SQL, without first having to SELECT MAX(order) WHERE user_id=1?
My comment seems to have gotten some traction, so I'm posting it as an answer... To avoid your problem, add a new column, without constraints, and just use that for user desired updates.
Why aren't you updating the order instead of the thingid?
UPDATE tbl
SET order = 2
WHERE thing_id=1;
Each row represents a "thing-user" pair. The data is the ordering that you want to use. You don't want to change the entity ("thing-user"). You want to change the data.
By the way, you'll then have to do some additional work to keep unique values in orders.
If you switched this around and put the unique constraint on user_id, order, then it would make sense to update the thing_id.
How can i get all of the records in a table that are out of
sequence so I know which account numbers I can reuse. I have a range
of account numbers from 50100 to 70100. I need to know which account
numbers are not stored in the table (not currently used) so I can use.
For instance say I have the following data in table:
Account Name
------ --------
50100 Test1
50105 Test2
50106 Test4
..
..
..
I should see the results:
50101
50102
50103
50104
because 50101-50104 are available account numbers since not currently in
table.
copied from http://bytes.com/topic/sql-server/answers/78426-get-all-unused-numbers-range
With respect to MYSQL and PHP.
EDITED
My range is 10000000-99999999.
My present way is using MySql query:
'SELECT FLOOR(10000000 + RAND() * 89999999) AS random_number FROM contacts WHERE "random_number" NOT IN (SELECT uid FROM contacts) LIMIT 1';
Thanks.
solution 1:
Generate a table with all possible accountnumbers in it. Then run a query similar to this:
SELECT id FROM allIDs WHERE id NOT IN (SELECT id FROM accounts)
Solution 2:
Get the whole id colummn into an array in php or java orso. Then run a for-loop to check if the number is in the array.
$ids = (array with all ids form the table)
for($i=50100;$i<=70100;$i++){
if(array_search($i, $ids) != -1){
$availableids[] = $i;
}
}
one way would be to create another table - fill it will all allowable numbers, then write a simple query to find the ones in the new table that are not in the original table.
Sort the accounts in the server, and find jumps in PHP while reading in the results. Any jump in the sorted sequence is "free for use", because they are ordered. You can sort with something like SELECT AccountNumber FROM Accounts SORT ASCENDING;.
To improve efficiency, store the free account numbers in another table, and use numbers from this second table until no more remain. This avoids making too many full reads (as in the first paragraph), which may be expensive. While you are at it, you may want to add a hook in the part of the code which deletes accounts, so they are immediately included in this second table, making the first step unnecessary.
I have a column called list which is used in my order by (in MYSQL queries) and within list is numbers: (e.g. 1 to 20)
This list is then output using MYSQL order by list ASC - However, when I update my list in backend using a Jquery drag drop UI list it is supposed to update the list frontend.
My problem is that my list order sometimes conflicts with other rows as there could be two or three rows with the value of 1 in list therefore when my order updates I would like to know how I can update other rows by +1 only if the rows are >= the order number given
I do not want to make the column primary as I am not aiming to make the list column unique, the reason for this is because there is more than one category - and in each category they all start at 1 - therefore if I make it unique it would cause errors because there was multiple 1's over different categories.
I asked a friend who said I could probably try PL/SQL using a trigger function but this is new grounds to me - I don't fully understand that language and was wondering if anyone could help me do what I am trying to using MYSQL or even PL/SQL.
This is what I have so far:
<?php
$project = mysql_real_escape_string(stripslashes($_POST['pid']));
$category = mysql_real_escape_string(stripslashes($_POST['cat']));
$order = mysql_real_escape_string(stripslashes($_POST['order']));
// need to do update the moved result (list row) and update all conflicting rows by 1
mysql_query("UPDATE `projects` SET `cat`='$category',`list`='$order' WHERE `id`='$project'")or die(mysql_query());
?>
Conclusion:
I am trying to update a none unique column to have unique values for that individual category. I am not sure how to update all the rows in that category by +1
#andrewsi is right, in particular I suggest order by list ASC, last_update DESC so in the same query where you update list you can timestamp last_update and therefore you will have not need to use triggers or any other updates.
In general, what andrewsi and Luis have suggested is true. Instead of (like andrewsi said) "do messy updates" you should really consider ordering by a second column.
However, I can maybe see your point for your approach. One similar situation I know it could apply is in a CMS where you let the backend user order items by changing the order number manually in textfields next to the items, e.g.
item 1 - [ 1 ]
item 2 - [ 3 ]
item 3 - [ 2 ]
... the number in the [] would then be the new order.
So, a quite messy solution would be (many steps, but if you do not have to worry about performance it might be OK for you, I don't know):
INSERT INTO projects (cat, list, timestamp_inserted) VALUES (:cat, :list, NOW())
and then as a second step
SELECT id, list FROM projects WHERE cat=:cat ORDER BY list ASC, timestamp_inserted DESC
and then loop through the array you get from the select and foreach row update (:i is the increasing index)
UPDATE projects SET list=:i WHERE id=:id
PS: you would have to add a column timestamp_inserted with a timestamp value.
PPS: to clearly state, I would not recommend this and never said it is best practice (for those considering to downvote because of this)
I have a table in MySQL that I'm accessing from PHP. For example, let's have a table named THINGS:
things.ID - int primary key
things.name - varchar
things.owner_ID - int for joining with another table
My select statement to get what I need might look like:
SELECT * FROM things WHERE owner_ID = 99;
Pretty straightforward. Now, I'd like users to be able to specify a completely arbitrary order for the items returned from this query. The list will be displayed, they can then click an "up" or "down" button next to a row and have it moved up or down the list, or possibly a drag-and-drop operation to move it to anywhere else. I'd like this order to be saved in the database (same or other table). The custom order would be unique for the set of rows for each owner_ID.
I've searched for ways to provide this ordering without luck. I've thought of a few ways to implement this, but help me fill in the final option:
Add an INT column and set it's value to whatever I need to get rows
returned in my order. This presents the problem of scanning
row-by-row to find the insertion point, and possibly needing to
update the preceding/following rows sort column.
Having a "next" and "previous" column, implementing a linked list.
Once I find my place, I'll just have to update max 2 rows to insert
the row. But this requires scanning for the location from row #1.
Some SQL/relational DB trick I'm unaware of...
I'm looking for an answer to #3 because it may be out there, who knows. Plus, I'd like to offload as much as I can on the database.
From what I've read you need a new table containing the ordering of each user, say it's called *user_orderings*.
This table should contain the user ID, the position of the thing and the ID of the thing. The (user_id, thing_id) should be the PK. This way you need to update this table every time but you can get the things for a user in the order he/she wants using ORDER BY on the user_orderings table and joining it with the things table. It should work.
The simplest expression of an ordered list is: 3,1,2,4. We can store this as a string in the parent table; so if our table is photos with the foreign key profile_id, we'd place our photo order in profiles.photo_order. We can then consider this field in our order by clause by utilizing the find_in_set() function. This requires either two queries or a join. I use two queries but the join is more interesting, so here it is:
select photos.photo_id, photos.caption
from photos
join profiles on profiles.profile_id = photos.profile_id
where photos.profile_id = 1
order by find_in_set(photos.photo_id, profiles.photo_order);
Note that you would probably not want to use find_in_set() in a where clause due to performance implications, but in an order by clause, there are few enough results to make this fast.
Here is the scenario 1.
I have a table called "items", inside the table has 2 columns, e. g. item_id and item_name.
I store my data in this way:
item_id | item_name
Ss001 | Shirt1
Sb002 | Shirt2
Tb001 | TShirt1
Tm002 | TShirt2
... etc, i store in this way:
first letter is the code for clothes, i.e S for shirt, T for tshirt
second letter is size, i.e s for small, m for medium and b for big
Lets say in my items table i got 10,000 items. I want to do fast retrieve, lets say I want to find a particular shirt, can I use:
Method1:
SELECT * from items WHERE item_id LIKE Sb99;
or should I do it like:
Method2:
SELECT * from items WHERE item_id LIKE S*;
*Store the result, then execute second search for the size, then third search for the id. Like the hash table concept.
What I want to achieve is, instead of search all the data, I want to minimize the search by search the clothes code first, follow by size code and then id code. Which one is better in term of speed in mysql. And which one is better in long run. I want to reduce the traffic and not to disturb the database so often.
Thanks guys for solving my first scenario. But another scenario comes in:
Scenario 2:
I am using PHP and MySQL. Continue from the preivous story. If my users table structure is like this:
user_id | username | items_collected
U0001 | Alex | Ss001;Tm002
U0002 | Daniel | Tb001;Sb002
U0003 | Michael | ...
U0004 | Thomas | ...
I store the items_collected in id form because one day each user can collect up to hundreds items, if I store as string, i.e. Shirt1, pants2, ..., it would required a very large amount of database spaces (imagine if we have 1000 users and some items name are very long).
Would it be easier to maintain if I store in id form?
And if lets say, I want to display the image, and the image's name is the item's name + jpg. How to do that? Is it something like this:
$result = Select items_collected from users where userid= $userid
Using php explode:
$itemsCollected = explode($result, ";");
After that, matching each item in the items table, so it would like:
shirt1, pants2 etc
Den using loop function, loop each value and add ".jpg" to display the image?
The first method will be faster - but IMO it's not the right way of doing it. I'm in agreement with tehvan about that.
I'd recommend keeping the item_id as is, but add two extra fields one for the code and one for the size, then you can do:
select * from items where item_code = 'S' and item_size = 'm'
With indexes the performance will be greatly increased, and you'll be able to easily match a range of sizes, or codes.
select * from items where item_code = 'S' and item_size IN ('m','s')
Migrate the db as follows:
alter table items add column item_code varchar(1) default '';
alter table items add column item_size varchar(1) default '';
update items set item_code = SUBSTRING(item_id, 1, 1);
update items set item_size = SUBSTRING(item_id, 2, 1);
The changes to the code should be equally simple to add. The long term benefit will be worth the effort.
For scenario 2 - that is not an efficient way of storing and retrieving data from a database. When used in this way the database is only acting as a storage engine, by encoding multiple data into fields you are precluding the relational part of the database from being useful.
What you should do in that circumstance is to have another table, call it 'items_collected'. The schema would be along the lines of
CREATE TABLE items_collected (
id int(11) NOT NULL auto_increment KEY,
userid int(11) NOT NULL,
item_code varchar(10) NOT NULL,
FOREIGN KEY (`userid`) REFERENCES `user`(`id`),
FOREIGN KEY (`itemcode`) REFERENCES `items`(`item_code`)
);
The foreign keys ensure that there is Referential integrity, it's essential to have referential integrity.
Then for the example you give you would have multiple records.
user_id | username | items_collected
U0001 | Alex | Ss001
U0001 | Alex | Tm002
U0002 | Daniel | Sb002
U0002 | Daniel | Tb001
U0003 | Michael | ...
U0004 | Thomas | ...
The first optimization would be splitting the id into three different fields:
one for type, one for size, one for the current id ending (whatever the ending means)
If you really want to keep the current structure, go for the result straight away (option 1).
If you want to speed up for results you should split up the column into multiple columns, one for each property.
Step 2 is to create an index for each column. Remember that mysql only uses one index per table per query. So if you really want speedy queries and your queries vary a lot with these properties, then you might want to create an index on (type,size,ending), (type,ending,size) etc.
For example a query with
select * from items where type = s and size = s and ending = 001
Can benefit from the index (type,size,ending) but:
select * from items where size = s and ending = 001
Can not, because the index will only be used in order, so it needs type, then size, then ending. This is why you might want multiple indexes if you really want fast searches.
One other note, generally it is not a good idea to use * in queries, but to select only the columns you need.
You need to have three columns for the model, size and id, and index them this way:
CREATE INDEX ix_1 ON (model, size, id)
CREATE INDEX ix_2 ON (size, id)
CREATE INDEX ix_3 ON (id, model)
Then you'll be able to search efficiently on any subset of the parameters:
model-size-id, model-size and model queries will use ix_1;
size-id and size queries will use ix_2;
model-id and id queries will use ix_3
Index on your column as it is now is equivalent to ix_1, and you can use this index to efficiently search on the appropriate conditions (model-size-id, model-size and model).
Actually, there is a certain access path called INDEX SKIN SCAN that may be used to search on non-first columns of a composite index, but MySQL does not support it AFAIK.
If you need to stick to your current design, you need to index the field and use queries like:
WHERE item_id LIKE #model || '%'
WHERE item_id LIKE #model || #size || '%'
WHERE item_id = #model || #size || #id
All these queries will use the index if any.
There is not need to put in into multiple queries.
I'm comfortable that you've designed your item_id to be searchable with a "Starts with" test. Indexes will solve that quickly for you.
I don't know MySQL, but in MSSQL having an index on a "Size" column that only has choices of S, M, L most probably won't achieve anything, the index won't be used because the values it contains are not sufficiently selective - i.e. its quicker to just go through all the data rather than "Find the first S entry in the index, now retrieve the data page for that row ..."
The exception is where the query is covered by the index - i.e. several parts of the WHERE clause (and indeed, all of them and also the SELECT columns) are included in the index. In this instance, however, the first field in the index (in MSSQL) needs to be selective. So put the column with the most distinct values first in the index.
Having said that if your application has a picklist for Size, Colour, etc. you should have those data attributes in separate columns in the record - and separate tables with lists of all the available Colours and Sizes, and then you can validate that the Colour / Size given to a Product is actually defined in the Colour / Size tables. Cuts down the Garbage-in / Garbage-out problem!
Your item_selected needs to be in a separate table so that it is "normalised". Don't store a delimited list in a single column, store it using individual rows in a separate table
Thus your USERS table will contain user_id & username
Your, new, items_collected table will contains user_id & item_id (and possibly also Date Purchased or Invoice Number)
You can then say "What did Alex buy" (your design has that) and also "Who bought Ss001" (which, in your design, would require ploughing through all the rows in your USERS table and splitting out the items_collected to find which ones contained Ss001 [1])
[1] Note that using LIKE wouldn't really be safe for that because you might have an item_id of "Ss001XXX" which would match WHERE items_collected LIKE '%Ss001%'