One of my db table has columns COLUMN1,COLUMN2,COLUMN3 which hold ids of users. This group of 3 should be unique. Mean stu1, stu2, stu3 record exists in above columns, it cannot have stu2, stu1, stu3 again. But it can have stu1, stu2, stu4
and and I have an array of students i.e [stu1, stu2, stu3, ....... stuN]
N could go upto 5000
What is the most efficient approach to find a combination from the array, which is not a duplicate.
I am using mysql and php (latest versions)
As #Strawberry noted in their comment, the design of the table (and the necessity of this query) seems suspicious. But assuming that it has to be this way, there's a couple of ways to go about this. A nice and clean trick would be to leverage the ordering capabilities of JSON_EXTRACT():
SELECT t2.ids, COUNT(1) occurrences
FROM (
SELECT
t.col1,
t.col2,
t.col3,
JSON_EXTRACT(JSON_OBJECT(col1, col1, col2, col2, col3, col3), '$.*') ids
FROM test t
) t2
GROUP BY t2.ids
Here's a working example. The idea here is that we create a JSON object containing the three ids (as both keys and values), and when we extract it again it will be ordered by key. Then we can just group by and count.
Related
I am using group by for like statement as i have database structure like this.
I want to get the count of workingzone groupby.but if i try to group by, then wrong output will appear as output will group by 99 and 99, as in figure.
My sql code is:
select count(organization),working_zone from `projects` where `district` = 12 and (`working_zone` = 99 or `working_zone` LIKE "99," or `working_zone` LIKE ",99") group by `organization`;
my desired result is:
count |working_zone
____6| 99
____3| 100
(99),(,99),(99,) should be grouped by doing sum and result should be 6.
You have an awful data structure -- although I wouldn't be surprised if the data is okay and you are really working off the result of a (reasonable) query. You should not be storing raw data in comma-delimited lists. Instead, use junction tables.
Why is having a separate row for each pair the SQLish way of storing data? Consider these reasons:
SQL has pretty based string functions (compared to other programming environments).
Data should be stored in its native type; don't store numbers as strings.
Foreign key relationships should be explicitly declared, and you can't declare a foreign key relationship using comma-delimited strings.
The primary SQL mechanism for optimizing queries are indexes, and comma-delimited lists preclude the use of indexes.
Sometimes, though, you are stuck with someone else's bad design decisions. If so, one solution uses a table of working zones:
select count(*), wz.working_zone
from projects p join
working_zones wz
on find_in_set(wz.working_zone, p.working_zone) > 0
where p.district; = 12 and
find_in_set(99, p.working_zone) > 0
group by wz.working_zone;
SELECT COUNT(organization),working_zone FROM table WHERE working_zone HAVING '99'
UNION ALL
SELECT COUNT(organization),SUBSTRING(working_zone,4) FROM table WHERE working_zone = '99,100'
Here is the problem: I want my data sorted in 2 differents way without calling two times the database.
Example:
SELECT data FROM costumers WHERE x ORDER BY budget, car, home
I need this data sorted in this way to do fast some operations.
In the meantime I need also the same data but:
ORDER BY home, budget, car
I can reorder the data in php or make a second query to my database but I hope there is a simplee and faster way to make it works with MySql.
(maybe the data could be fetch in two differents ways)
Thanks in advance :)
(first time I post, but not first time this website helps me)
you can do this sort in one query .
example if we have col1 and col2. col1 contain strings and col2 contains numbers
SELECT * FROM table
ORDER BY `col1` DESC, `col2` DESC
we assume cols values are
(col1, col2): (b,3), (a,4), (c,1), (a,1)
after we execute the query above it will return the records in this order
(col1, col2): (a,1), (a,4), (b,3), (c,1)
note: MySQL uses the first arg of order by as more impotent than second and so on. Even you can sort by triple or more in this way and even the date type could be ordered too.
I have page that display information from two different tables , and for that I have two queries.
There is no related info between these two tables.
Since both queries may contain a lot of information, I need create pagination.
BUT I don't want two separate paginations, I want only one that will contain results from query 1 and query 2 together.
How can I do that?
The only idea I have is to fetch all info of both queries into arrays, then combine the arrays into one, then create pagination that based on that array.
That of course would not help save resources.
You could use a union - the columns you're displaying must line up, so something like this should work:
select
col1 col1_alias,
col2 col2_alias,
...
from
table1
where
...
union
select
col1,
col2,
...
from
table2
where
...
order by col1_alias, col2_alias
limit 10
Basically the union will pull all the data together, and the order by and limit will apply to the whole result set.
The names of the columns don't need to match in the second select, but use column names from the first select for your order by (or create aliases, which is probably more readable depending on your dataset).
I think people were reluctant to help me last time because I didn't want to post up the columns, but I'm eager to solve this, so I'm putting aside my better judgement and laying it bare.
OK, I have two similar tables, and I'm using MySQL, and PHP to generate the tables. So far I have 3 tables:
CPE:
fastethernet00,subnet00,fastethernet01,subnet01,hsrp,vip,bgp,bgp_neighbor,remote_as,bgp_customer_net,next_hop,loopback,cpe,hostname,cpe_cust_index,int_next_hop_1,int_next_hop_2,voice,solution
Liverpool:
apn1,apn2,apn3,apn4,apn5,customer,vpi_vci,vlan1,cust_index,RADIUS,nexthop,atinterface1,atinterface2,feinterface1,feinterface2,spinterface1,spinterface2,ip_address_range1,ip_address_range2,handset_address_range1,handset_address_range2,handset_address_range3,handset_address_range4,handset_address_range5,handset_address_range6,handset_address_range7,handset_address_range8,handset_address_range9,handset_address_range10,handset_address_range11,handset_address_range12,handset_address_range13,handset_address_range14,handset_address_range15,handset_address_range16,DNS_Server1,DNS_Server2,OLVDMVPN,live
Greenwich:
apn_1,apn_2,apn_3,apn_4,apn_5,customer_,vpi_vci_,vlan_1,cust_index_,RADIUS_,nexthop_,atinterface_1,atinterface_2,feinterface_1,feinterface_2,spinterface_1,spinterface_2,ip_address_range_1,ip_address_range_2,handset_address_range_1,handset_address_range_2,handset_address_range_3,handset_address_range_4,handset_address_range_5,handset_address_range_6,handset_address_range_7,handset_address_range_8,handset_address_range_9,handset_address_range_10,handset_address_range_11,handset_address_range_12,handset_address_range_13,handset_address_range_14,handset_address_range_15,handset_address_range_16,DNS_Server_1,DNS_Server_2,OLVDMVPN_,live_
The first step I need to accomplish is linking Greenwich to Liverpool via apn_1/apn1. The same record won't necessarily be on both, and there may even be duplicates. The php I'm using to create this table is throwing up a syntax error around the UNION ALL:
mysql_query ("CREATE TABLE Both (
SELECT * FROM `Greenwich` left outer join Liverpool on Greenwich.apn_1 = Liverpool.apn1
UNION ALL
SELECT * FROM Greenwich
RIGHT OUTER JOIN Liverpool on Greenwich.apn_1 = Liverpool.apn1 where Greenwich.apn_1 is null)")
Can anyone see what's wrong with that?
Afterwards, I need to then link the resultant table to cpe_cust_index via cust_index/cust_index_, so ideally I think I need to merge these cells. If these cells are present in Greenwich and Liverpool they will always be the same.
Thanks in advance!
The answer is the 3rd way described here, using a "mutex (mutually exclusive) table
http://www.xaprb.com/blog/2006/05/26/how-to-write-full-outer-join-in-mysql/
This plus a filter give me exactly what I need. Now I just have to merge the index fields together, and apn 1 fields together.
Try naming all the columns that you want to select and alias then using AS so that they match. Union does not like column name mismatches.
Eg.
Select col1 as col1, col2 as col2 from table1
Union
Select col-1 as col1, col-2 as col2 from table2
I am writing a converter to transfer data from old systems to new systems. I am using php+mysql.
I have one table that contains millions records with duplicate entries. I want to transfer that data in a new table and remove all entries. I am using following queries and pseudo code to perform this task
select *
from table1
insert into table2
ON DUPLICATE KEY UPDATE customer_information = concat('$firstName',',','$lastName')
It takes ages to process one table :(
I am pondering that is it possible to use group by and get all grouped record automatically?
Other than going through each record and checking duplicate etc.?
For example
select *
from table1
group by firstName, lastName
insert into table 2 only one record and add all users'
first last name into column ALL_NAMES with comma
EDIT
There are different records for each customers with different information. Each row is called duplicated if first and last name of user is same. In new table, we will just add one customer and their bought product in different columns (we have only 4 products).
I don't know what you are trying to do with customer_information, but if you just want to transfer the non-duplicated set of data from one table to another, this will work:
INSERT IGNORE INTO table2(field1, field2, ... fieldx)
SELECT DISTINCT field1, field2, ... fieldx
FROM table1;
DISTINCT will take care of rows that are exact duplicates. But if you have rows that are only partial duplicates (like the same last and first names but a different email) then IGNORE can help. If you put a unique index on table2(lastname,firstname) then IGNORE will make sure that only the first record with lastnameX, firstnameY from table1 is inserted. Of course, you might not like which record of a pair of partial duplicates is chosen.
ETA
Now that you've updated your question, it appears that you want to put the values of multiple rows into one field. This is, generally speaking, a bad idea because when you denormalize your data this way you make it much less accessible. Also, if you are grouping by (lastname, firstname), there will not be names in allnames. Because of this, my example uses allemails instead. In any event, if you really need to do this, here's how:
INSERT INTO table2(lastname, firstname, allemails)
SELECT lastname, firstname, GROUP_CONCAT(email) as allemails
FROM table1
GROUP BY lastname, firstname;
If they are really duplicate rows (every field is the the same) then you can use:
select DISTINCT * from table1
instead of :
select * from table1