Is it possible somehow to return all the rows in a mysql table where one field has duplicate values in a comma separated string?
EG if we had fields
ID VALUE
'1', '123,123,678'
'2', '23,24,25'
I only want to return row 1 in this instance?
There is no way to do this from within SQL that's both practical and efficient.
As indicated by John Conde, you should normalize your database so that it looks like this:
ID VALUE
1 123
1 123
1 678
2 23
2 24
2 25
Then you can easily prevent the situation from arising by disallowing duplicate rows
(eg., by defining a UNIQUE index on the ID and VALUE columns). And if you can't / don't want to prevent it from happening, you can at least detect it much easier in pure SQL.
As it currently stands, I would get all rows into a PHP array and detect duplicate values from there. This isn't really efficient either, but at least it'll be more practical than trying to reach this in pure SQL.
$result = mysql_result('select ID, VALUE from table');
while($row = mysql_fetch_array($result, MYSQL_ASSOC))
{
$values_array = explode( ',' , $row['VALUE'] );
if(count($values_array) != count(array_unique($values_array)))
{
// There must be duplicate values in the array
echo 'Row with id ' .$row['ID'] .' has duplicate values';
}
}
You can do it like this, but normalizing your database would be much better.
(Oh, and don't use these mysql_... functions; you should use mysqli or PDO. This is just to show you how it works).
Assuming you just have just 3 values comma separated . then you can use this via mysql and get rid of php:
select a.id,a.v1 , a.v2,a.v3,a.value from (
SELECT id,value,SUBSTRING_INDEX(value, ',', 1)as v1,
SUBSTRING_INDEX(SUBSTRING_INDEX(value, ',', 2), ',', -1) as v2,
SUBSTRING_INDEX(value, ',', -1) as v3
FROM table1 )a
where v1 = v2 or v1 = v3 or v2 = v3
DEMO HERE
OTHER DEMO with extra values
OBS: have make extra v1 ,v2,v3 separated ,so you may use them in your work
Related
I need to insert a list of comment IDs into a column liked_comments.
For example if a user likes comments with IDs 72, 839, 37, then they will be stored in there.
$id = mysqli_real_escape_string($conn,$_POST['id']); // a number posted using ajax from another file
$username = mysqli_real_escape_string($conn,$_SESSION['username']);
if (!empty($_POST) && isset($_SESSION['username'])){
mysqli_query($conn,"UPDATE users SET liked_comments=liked_comments+$id WHERE username='$username'");
}
At the moment this just adds the IDs as though they were numbers rather than strings, but I need it to be 71,789,173 or 71 789 173 separated by a space or comma. Adding +' '+ or +','+ didn't seem to work.
You cannot concatenate using + in mysql, you have to use concat like
$Query = "UPDATE users SET liked_comments=concat(liked_comments, ',', '$id') WHERE username='$username'";
Also, Instead of storing comma separated values in column, you can move comments part to separated table, where each row contain PostID and UserID, that would be much neater and flexible.
The best way to do this would be to store arrays inside the database. For example; if we have this array:
$liked_comments_array = array(71, 789, 173);
You can store it in the array like so:
$liked_comments_array = serialize($liked_comments_array);
// now insert this into the database
Then when you fetch this array from the database, simply unserialize it like so:
$liked_array_comments = unserialize($field_from_database);
People enter values into a form and on one entry of the form I have multiple values.
Eg:
One Entry for Name
and Multiple entries for hobbies.
I could enter into the db by running a for loop but then that would be multiple entries for the same name for each different hobby.
How can I enter one name and all hobbies with 'space' into one DB field 'TWIG'. I could use arrays but it shows up as ARRAY but then its back to FOR loop.
for ($t=0; $t<=$_POST['tc']-1; $t++) {
echo "<BR> ts:".$_POST[$t]."<BR>";
$ths[]=$_POST[$t];
}
print_r ($ths);
$statement = $link->prepare("INSERT INTO quest(cnos, dte, twig)
VALUES(:q, :d, :t )");
$statement->execute(array(
":q" => htmlspecialchars ($_POST['iht']),
":d" => $_SERVER['REQUEST_TIME'],
":t"=> $ths
));
One possibility is to implode your hobbies / concatinate your string into one...
for ($t=0; $t<=$_POST['tc']-1; $t++) {
$ths = $_POST[$t] . " "; //Concatinate string, do no use array!
}
//Cut off last character " " to avoid ugly space at the end:
$ths = substr($ths, 0, strlen($ths) - 1);
However a more clean solution is to make a more clear database structure if you want for atomic values.
This is an 1:n relation (Each of your entries in table A relates to n instances in table B).
Here is an example that can be adapted into your schema very easy:
Table User(id[PK], name, age);
Table User_Hobbies: (user_id, hobby_descr);
--
INSERT INTO User(2, "Chris", 19);
INSERT INTO USER_User_Hobbies(2, "videogames");
INSERT INTO USER_User_Hobbies(2, "music");
--
Example query:
SELECT u.name, u.age, GROUP_CONCAT(uh.hobby_descr) AS hobbies
FROM User u
LEFT JOIN User_Hobbies uh ON u.id = uh.user_id
WHERE u.id = 123 /*Where is optional, if you want to filter for a specific user*/
GROUP BY u.id;
Possible result:
| name | age | hobbies |
chris 18 videogames, music
steve 22 computer, programming, php
Use the implode function for this as follows:
":t"=> implode(" ", $ths);
I am not sure, if I understand your question right, but if you want to insert an array as a comma-seperated string (or separated by whatever), why don't use the php implode function: http://php.net/manual/de/function.implode.php
For example:
$myHobbies = array("football", "basketball", "running");
$statement = $link->prepare("INSERT INTO quest(cnos, dte, twig)
VALUES(:q, :d, :t )");
$statement->execute(array(
":q" => htmlspecialchars ($_POST['iht']),
":d" => $_SERVER['REQUEST_TIME'],
":t"=> implode(" ", $myHobbies)
));
I think to use comma's or semicolons as separator, is better than whitespaces, since some of the hobbies could consists of two words (for example "pc gaming").
I think that better solution is to use json_encode and not implode, since it provides more robust structure.
":t" => json_encode($myHobbies)
You can use join function to store multiple value in same field in database may be it will works:-
":t"=>join(",", $ths);//if you want ,(coma) between two string
or
":t"=>join(" ", $ths);//if you want space between two string
I have a MySQL data table, in which I have more than 2 columns. First column has a unique value clinical trial value whereas second column has disease information. There are, in most of the cases, more than 2 disease names in one cell for a single id. I want to spilt those rows which cell contains two or more than two diseases. There is a pattern for searching also, i.e. small character is immediately followed by capital character., e.g. MalariaDengueTuberculosis like this. Suppose for these three diseases there is unique id, it should show like the following:
NCT-ID disease
4534343654 Maleria
4534343654 Dengue
4534343654 Tubercoulsosis
If you want to store one or more data in one String column, you could use JSON data formatting.
It is not very clear to me what you're trying to achieve. Since you're using PHP you can try to read the tables using PDO and print the results. For example:
$sql = 'SELECT diseases FROM diseases_info_table';
foreach($pdo->query($sql) as $row) {
$diseases = preg_split('/(?=[A-Z])/', $row['diseases']);
$sql2 = 'SELECT * FROM diseases WHERE disease_name IN ("' . implode('","', $diseases) . '")';
foreach ($pdo->query($sql2) as $row) {
print $row['NCT-ID '] . "\t";
print $row['disease'] . "\t";
}
}
But this way you're generating a lot of queries. If it is possible for you to rethink the database structure than I would recommend doing that.
To answer your question:
If you insist on multi data in one column you can use php explode:
$diseases = explode("|", $row['disease']); //Changing | to whatever separates your diseases.
$diseases is now an array of your diseases which you can do:
foreach ($diseases as $disease)
{
echo $disease;
}
However
Personally I would normalise your database now before trying to hack around solutions. Use an ID against your table and then have a diseases table to link to it. ie:
Main table
NCT-ID
4534343654
5768788544
3i33i3i078
Disease Table
disease_id nct_id disease
1 4534343654 Broken Wind
2 4534343654 Chronic Nosehair
3 4534343654 Corrugated Ankles
4 5768788544 Discrete Itching
5 3i33i3i078 Gastric Ejections
6 3i33i3i078 Bloaty Head
This allows multiple diseases against one nct-id. disease_id would be the primary key.
Let's say I have three values in PHP: "a", "b", "c". Doesn't matter whether in an array or comma separated string.
There is a table in database:
id | value
1 | a
2 | b
3 | d
My purpose is to find the values that are in php array but not in database table.
The given example will give "c".
Can I do it with only one query?
UPDATE
Received several good suggestions in answers about array_diff(), though in my case the DB table is really large and the array has not more than 5-6 items. So it would be better to perform 5-6 queries, I think.
If the PHP array is short, you can build a UNION ALL query to build your small table, then use NOT IN or LEFT JOIN query (whichever is faster) against the large table:
SELECT value
FROM (
SELECT 'a' AS value
UNION ALL
SELECT 'b'
UNION ALL
SELECT 'c'
) AS php_array_values
WHERE value NOT IN (
SELECT value
FROM that_large_table
);
Alternately, you can insert the php array values in a temporary table and use the IN or JOIN queries. Of course, this means you end up writing three extra queries:
CREATE TEMPORARY TABLE IF NOT EXISTS php_array_values (value VARCHAR(100));
DELETE FROM php_array_values;
INSERT INTO php_array_values VALUES ('a'), ('b'), ('c');
SELECT php_array_values.value
FROM php_array_values
LEFT JOIN that_large_table ON php_array_values.value = that_large_table.value
WHERE that_large_table.value IS NULL
how about this?
<?php
$a = array('a', 'b', 'c');
$values = implode("','", $a);
$sql = "SELECT DISTINCT `value` FROM `mytable` WHERE `value` IN ('$values')";
echo $sql;
perform the sql query. the result will be those 0 to 3 elements you already have. next, do an array_diff (which will not be heavy at all, since you'll have your initial small array, and the array of those in the db, which is even smaller).
$not_in_db = array_diff($a, $array_from_sql_result);
if what you have is a string with comma separated values, then you'll need to "explode" it first:
$s = "a,b,c";
$a = explode(",", $s);
You could select all the entries in the table and then perform an array_diff().
But this isn't one query, is one query and some post processing.
For this I would pull values from the table into an array and use array_diff
REF: http://www.php.net/manual/en/function.array-diff.php
Instead of pulling all the elements from the DB you can try using 'LIKE' statement, this will reduce the number of entries pulled from DB.
something like this :
PHP array values : a,b,c
DB values : a,b,d
select value from your_table_name where (value LIKE '%a%') OR (value LIKE '%b%') OR (value LIKE '%c%');
o/p of this will be : {a,b}
now use array_diff of php.
select max(*) as count from tablename
where field1 != $arr[0] && field2 != $arr[1] && field3 != $arr[2];
You can use and or or operators if you want. if the return count for this query is 0 then the array values are not exist in the database already.
I am writing an algorithm to generate combinations of items from a database. They need to be unique permutations (i.e. 145, 156 == 156, 145). The problem I am running into is how to keep track of previous combinations so that i do not end up with 145, 156 and 156, 145.
Currently I am adding them to an array with index of id1_id2... (sorted so id's are always be lowest to highest) and setting the value equal to 1 when a combo is generated so that i can check if $combos[$index] exists or not. If it does not exist, create it. (there are other criteria to weed out EVERY permutation, but they are irrelevant) Once these combinations are generated, they are being stored in a table in MySQL.
The problem I am running into is that with the test items i'm using (about 85) I cannot generate a combinations with more than 3 items (id1_id2_id3) without running out of memory as the number of combinations is MASSIVE and the $combos array takes up more than the 64M i am allotted in PHP memory.
Is there a way that I can do this a) without keeping track of previous combos or b) skipping the $combos array route and only adding a unique row to mysql and let mysql handle the duplicate checking.
Here is some pseudo code for reference:
$items = array(/*85 items*/);
foreach ($items as $item1){
generate(array($item1));
foreach($items as $item2){
generate(array($item1, $item2));
}
}
}
function generate($items_arary){
$temp_array = array();
foreach ($items_array as $item){
$temp_array[] = $item['id'];
}
sort($temp_array);
$index = implode("_", $temp_array);
if (!$combos[$index]){
$combos[$index] = 1;
/* some code to generate query to store to db */
}
}
the query ends up looking like this: (the database is truncated at beginning of script)
INSERT INTO `combos` (combo_id, more_info) VALUES ('id1_id2', 'Item Name');
In the process of writing this question, I thought of a possible solution: Making sure id3 > id2 > id1. Would this be a viable solution to remove the need for $combos?
The reason I asked about the before data structure is because you could do something like this:
$sql = "SELECT id FROM test_a";
$result = mysql_query($sql);
while ($row = mysql_fetch_array($result)) {
$item1 = $row['id'];
$sql2 = "SELECT id FROM test_a";
$result2 = mysql_query($sql2);
while ($row2 = mysql_fetch_array($result2)) {
$item2 = $row2['id'];
$combo1 = $item1 . "_" . $item2;
$combo2 = $item2 . "_" . $item1;
$sql3 = "SELECT * FROM combos WHERE combo_id = '$combo1' OR combo_id = '$combo2'";
$result3 = mysql_query($sql3);
if (mysql_num_rows($result3) == 0) {
$sql4 = "INSERT INTO combos (combo_id, more_info) VALUES ('$combo1','Item Name')";
$result4 = mysql_query($sql4);
}
}
}
When table test_a has the values 1,2,3, and 4 this script inserts:
1_1
1_2
1_3
1_4
2_2
2_3
2_4
3_3
3_4
4_4
This shouldn't have any memory problems. Although if you have a huge database you may run into a issue with php's time limit
Here is the same concept as my other answer but in an all SQL format.
INSERT INTO combos (combo_id, more_info)
SELECT CONCAT_WS("_",t1.id,t2.id), "item_name"
FROM test_a t1, test_a t2
WHERE NOT EXISTS (SELECT * FROM combos WHERE combo_id = CONCAT_WS("_",t1.id,t2.id))
AND NOT EXISTS (SELECT * FROM combos WHERE combo_id = CONCAT_WS("_",t2.id,t1.id))
Assuming you can get item_name from the db somewhere, this will probably be your fastest and least memory intensive solution. I am running a test on around 1000 ids at the moment. I'll update this when it finishes.
Yes. You can store and use the lexicographical index of the combination to reconstruct/iterate them, or Grey Codes if you need to iterate all of them.
Take a look at: "Algorithm 515: Generation of a Vector from the Lexicographical Index"; Buckles, B. P., and Lybanon, M. ACM Transactions on Mathematical Software, Vol. 3, No. 2, June 1977.
I've translated into C here, and describe more here.
If you don't need to enforce referential integrity automatically (which you're not if you use string concatenation), use one table for the 85 items, give them each an index (0-84), and use a second table to represent a given set of items, using a numeric datatype where each bit position in the number represents one item. (e.g. 000001101 represents items 0, 2, and 3)
For items more than 64 you may have to split them up into more than one field, or use a BLOB or a string (gack!).
If you use this as a primary key field, you can enforce non-duplicates.
In TSQL you can use a recursive CTE, Can''t remember where I got it, but pretty sweet. Note MYSQL doesn't use "With" option, so it won't work in MySQL
WITH Numbers(N) AS (
SELECT N
FROM ( VALUES(1), (2), (3), (4), (5), (6)) Numbers(N)),
Recur(N,Combination) AS (
SELECT N, CAST(N AS VARCHAR(20))
FROM Numbers
UNION ALL
SELECT n.N,CAST(r.Combination + ',' + CAST(n.N AS VARCHAR(10)) AS VARCHAR(20))
FROM Recur r
INNER JOIN Numbers n ON n.N > r.N)
select Combination
from RECUR
ORDER BY LEN(Combination),Combination;
to increase memory change
memory_limit = 512M in your php.ini
or
ini_set('memory_limit', '512M') in your php script
or
php_value memory_limit 512M in your .htaccess