I've been recently stuck on an issue I've been having involving sending a string of comma separated values into a stored procedure. My issue is that when I execute my stored procedure in PHP it uploads the values with quotes around it like so;
CALL `rankingInformation`('145', '5', '', '37,38,39,40,41')
Failing to add the quotes would make MySQL interpret them as extra parameters.
However it's mean't to be like in the WHERE IN on the query side it's meant to be formatted like so
'37', '38', '39', '40', '41'
Here is the query below, can anyone spot anything I can do? Here is what I've got up to now.
CREATE DEFINER = `root`#` localhost` PROCEDURE` rankingInformation`(IN` surveyId` INT, IN` filterCounting` INT, IN` survey_filter_id` INT, IN` question_limit` TEXT)
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT 'Gathers all the ranking information for a given ID'
BEGIN
DECLARE sfi int(2);
DECLARE ql TEXT;
IF(survey_filter_id = '') THEN
SET sfi = (SELECT sf2.survey_filter_id FROM survey_filters AS sf2 WHERE sf2.survey_id = 145 AND sf2.survey_filter_id IS NOT NULL LIMIT 1);
ELSE
SET sfi = survey_filter_id;
END IF;
SELECT
COUNT( * ) AS total, CONCAT(su.first_name, ' ', su.last_name) as full_name, sf.survey_filter_id, sf.survey_filter_name, qa.question_id, su.temp_group_1 AS department
FROM questions_answers AS qa
INNER JOIN survey_users AS su ON su.survey_users_id = qa.survey_users_id_answer
INNER JOIN survey_filters AS sf ON sf.survey_id = surveyId
WHERE qa.survey_id = surveyId
AND qa.question_id IN (splitAndTranslate(question_limit, ','))
AND sf.survey_filter_id = sfi
GROUP BY qa.survey_users_id_answer
HAVING total > filterCounting
ORDER BY total DESC;
END
splitAndTranslate
Here is a function I found which is mean't to do the job, I am not sure I am far away.
CREATE DEFINER=`root`#`localhost` FUNCTION `splitAndTranslate`(`str` TEXT, `delim` VARCHAR(1))
RETURNS text CHARSET utf8
LANGUAGE SQL
NOT DETERMINISTIC
CONTAINS SQL
SQL SECURITY DEFINER
COMMENT 'Fixes all Where IN issues'
BEGIN
DECLARE i INT DEFAULT 0; -- total number of delimiters
DECLARE ctr INT DEFAULT 0; -- counter for the loop
DECLARE str_len INT; -- string length,self explanatory
DECLARE out_str text DEFAULT ''; -- return string holder
DECLARE temp_str text DEFAULT ''; -- temporary string holder
DECLARE temp_val VARCHAR(255) DEFAULT ''; -- temporary string holder for query
-- get length
SET str_len=LENGTH(str);
SET i = (LENGTH(str)-LENGTH(REPLACE(str, delim, '')))/LENGTH(delim) + 1;
-- get total number delimeters and add 1
-- add 1 since total separated values are 1 more than the number of delimiters
-- start of while loop
WHILE(ctr<i) DO
-- add 1 to the counter, which will also be used to get the value of the string
SET ctr=ctr+1;
-- get value separated by delimiter using ctr as the index
SET temp_str = REPLACE(SUBSTRING(SUBSTRING_INDEX(str, delim, ctr), LENGTH(SUBSTRING_INDEX(str, delim,ctr - 1)) + 1), delim, '');
-- query real value and insert into temporary value holder, temp_str contains the exploded ID
#SELECT ImageFileName INTO temp_val FROM custombu_roomassets_images WHERE ImageID=temp_str;
-- concat real value into output string separated by delimiter
SET out_str=CONCAT(out_str, temp_val, ',');
END WHILE;
-- end of while loop
-- trim delimiter from end of string
SET out_str=TRIM(TRAILING delim FROM out_str);
RETURN(out_str); -- return
END
What did you do with FIND_IN_SET? Building on spencer7593's answer, it should work if you replace:
AND qa.question_id IN (splitAndTranslate(question_limit, ','))
with
AND FIND_IN_SET(qa.question_id, question_limit)>0
The commas within a string value are not interpreted as SQL text in the context of a SQL IN comparison. Your query is essentially of the form:
AND qa.question_id IN ('some,long,string,value')
And any comma characters within the string are just data; just characters that are part of the string. This is effectively the same as an equals comparison.
The MySQL FIND_IN_SET function might be a way for you to perform the comparison you want.
http://dev.mysql.com/doc/refman/5.5/en/string-functions.html#function_find-in-set
AND qa.question_id IN (splitAndTranslate(question_limit, ','))
Put these lines instead of the above line
AND (qa.question_id = question_limit
OR qa.question_id LIKE CONCAT(question_limit,',%')
OR qa.question_id LIKE CONCAT('%,',question_limit,',%')
OR qa.question_id LIKE CONCAT('%,',question_limit))
Then you don't need the splitAndTranslate Function anymore . .
Related
I have table invoices and there is column 'total' varchar(255). There are values like these: "500.00", "5'199.00", "129.60", "1.00" and others.
I need select records and filter by total column. For example, find records where total is not more than 180.
I tried this:
SELECT total from invoices WHERE invoices.total <= '180'
But in result there are :
125.25
100.50
1593.55 - not correct
4'799.00 - not correct
1.00
-99.00
2406.52 -not correct
How can I fix it and write correct filter for this column? Thanks!
You can use cast() function to convert it in float
SELECT total from invoices WHERE cast(invoices.total as decimal(16,2)) <= 180
Why are you storing numbers as strings? That is a fundamental problem with your data model, and you should fix it.
Sometimes, we are stuck with other people's really, really, really bad decisions. If that is the case, you can attempt to solve this with explicit conversion:
SELECT i.total
FROM invoices i
WHERE CAST(REPLACE(i.total, '''', '') as DECIMAL(20, 4)) <= 180;
Note that this will return an error if you have other unexpected characters in your totals.
If the string starts with a number, then contains non-numeric characters, you can use the CAST() function or convert it to a numeric implicitly by adding a 0:
SELECT CAST('1234abc' AS UNSIGNED); -- 1234
SELECT '1234abc'+0; -- 1234
To extract numbers out of an arbitrary string you could add a custom function like this:
DELIMITER $$
CREATE FUNCTION `ExtractNumber`(in_string VARCHAR(50))
RETURNS INT
NO SQL
BEGIN
DECLARE ctrNumber VARCHAR(50);
DECLARE finNumber VARCHAR(50) DEFAULT '';
DECLARE sChar VARCHAR(1);
DECLARE inti INTEGER DEFAULT 1;
IF LENGTH(in_string) > 0 THEN
WHILE(inti <= LENGTH(in_string)) DO
SET sChar = SUBSTRING(in_string, inti, 1);
SET ctrNumber = FIND_IN_SET(sChar, '0,1,2,3,4,5,6,7,8,9');
IF ctrNumber > 0 THEN
SET finNumber = CONCAT(finNumber, sChar);
END IF;
SET inti = inti + 1;
END WHILE;
RETURN CAST(finNumber AS UNSIGNED);
ELSE
RETURN 0;
END IF;
END$$
DELIMITER ;
Once the function is defined, you can use it in your query:
SELECT total from invoices WHERE ExtractNumber(invoices.total) <= 180
I have a table with a column 'search_text' type text.
In that field I have values:
1. 'MyBook MyBook PDF PDF',
2. 'Example 1 Example 2 Example 3'
3. 'John Snow John Snow'
I would like to distinct clean these fields.
Expected result:
1. 'MyBook PDF',
2. 'Example 1 2 3'
3. 'John Snow'
The approach I came up with goes as follows:
read the field for each record, split it by space (' '), put each text in array, do array_unique in PHP, then put the array back to string with join in PHP.
The thing is, this is a PHP based solution, I would like to have an MySQL solution for this. I have over 180.000 records I need to clean, I don't know what impact it would have to run this on PHP.
I have found a solution for MS SQL: Remove duplicate values in a cell SQL Server
Help greatly appreciated.
SQL of my test data:
CREATE TABLE IF NOT EXISTS `test` (
`id` int(10) unsigned NOT NULL,
`search_text` text COLLATE utf8_unicode_ci NOT NULL
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
INSERT INTO `test` (`id`, `search_text`) VALUES
(1, 'MyBook MyBook PDF PDF'),
(2, 'Example 1 Example 2 Example 3'),
(3, 'John Snow John Snow'),
(4, 'test test test test formula test test test formula test test test formula test test test formula test test test formula test test test formula '),
(5, '');
ALTER TABLE `test`
ADD PRIMARY KEY (`id`);
ALTER TABLE `test`
MODIFY `id` int(10) unsigned NOT NULL AUTO_INCREMENT,AUTO_INCREMENT=6;
Try this to sort by count :)
SELECT DISTINCT SUBSTRING_INDEX(SUBSTRING_INDEX(test.search_text, ' ', numbers.n), ' ', - 1) col_name
FROM (
SELECT 1 n
UNION ALL
SELECT 2
UNION ALL
SELECT 3
UNION ALL
SELECT 4
) numbers
INNER JOIN test ON CHAR_LENGTH(test.search_text) - CHAR_LENGTH(REPLACE(test.search_text, ' ', '')) >= numbers.n - 1
ORDER BY col_name;
You will need to write a MySQL function to do this for you. I would think that a PHP page will be just fine. 180,000 records isn't that many and it should (unless you are using a low spec server) run without putting much strain on anything else.
I wrote 2 for you that you might be able to make use of:
DROP PROCEDURE IF EXISTS explode;
DELIMITER //
CREATE PROCEDURE explode(str_string TEXT)
NOT DETERMINISTIC
BEGIN
DROP TABLE IF EXISTS explosion;
CREATE TABLE explosion (id INT AUTO_INCREMENT PRIMARY KEY NOT NULL, word VARCHAR(100));
SET #sql := CONCAT('INSERT INTO explosion (word) VALUES (', REPLACE(QUOTE(str_string), " ", '\'), (\''), ')');
PREPARE myStmt FROM #sql;
EXECUTE myStmt;
END //
DELIMITER ;
This procedure creates an "explode" function for use in MySQL. It uses a temporary table and explodes the words, separated by spaces into it
Then this function will read that table in, and put them into another temporary table with the duplicates removed:
DROP PROCEDURE IF EXISTS removeDuplicates;
DELIMITER //
CREATE PROCEDURE removeDuplicates(str TEXT)
BEGIN
DECLARE temp_word TEXT;
DECLARE last_word TEXT DEFAULT "";
DECLARE result TEXT;
DECLARE finished INT DEFAULT false;
DECLARE words_cursor CURSOR FOR
SELECT word FROM explosion;
DECLARE CONTINUE handler FOR NOT found
SET finished = true;
CALL explode(str);
DROP TABLE IF EXISTS temp_words;
CREATE TABLE temp_words (id INT AUTO_INCREMENT PRIMARY KEY NOT NULL, t VARCHAR(100));
OPEN words_cursor;
loop_words: LOOP
FETCH words_cursor INTO temp_word;
IF finished THEN
LEAVE loop_words;
END IF;
IF last_word = "" THEN
INSERT INTO temp_words (t) VALUES (temp_word);
SET last_word = temp_word;
ITERATE loop_words;
END IF;
IF last_word = temp_word THEN
SET last_word = temp_word;
ITERATE loop_words;
END IF;
INSERT INTO temp_words (t) VALUES (temp_word);
END LOOP loop_words;
CLOSE words_cursor;
END //
DELIMITER ;
So all you need to do is work out how to get the records in temp_words into your current database table.
I went for the PHP solution here:
$s = 'John Snow John Snow';
//remove duplicate values in string
$tmpArray = explode(" ", $s);
$tmpArray = array_unique($tmpArray);
$s = join(" ", $tmpArray);
Which is run before INSERT, and it does what I wanted.
I have a MySQL query and need to ask for a field which contains the ZIP code. Unfortunately some people enter also the city name or suburb into the ZIP field. I was looking for a query code which would return only the numbers from the field. Any code I already found on stackoverflow producec errors for me besides the following: How to remove all non-alpha numeric characters from a string?
This code however only deletes no alphanumeric characters. What would be the correct code to also remove all a,b,c characters.
Thank you for helping
UPDATE: The code posted by Syed Noman works for me in phpmyadmin.
However when I add this to my query in my php code I get a parsing error.
Here is the code which produces the error.
$query = "DROP FUNCTION IF EXISTS digits;
DELIMITER |
CREATE FUNCTION digits( str CHAR(32) ) RETURNS CHAR(32)
BEGIN
DECLARE i, len SMALLINT DEFAULT 1;
DECLARE ret CHAR(32) DEFAULT '';
DECLARE c CHAR(1);
IF str IS NULL
THEN
RETURN "";
END IF;
SET len = CHAR_LENGTH( str );
REPEAT
BEGIN
SET c = MID( str, i, 1 );
IF c BETWEEN '0' AND '9' THEN
SET ret=CONCAT(ret,c);
END IF;
SET i = i + 1;
END;
UNTIL i > len END REPEAT;
RETURN ret;
END |
DELIMITER ;
SELECT digits(`asdf`) FROM `12345` WHERE 1 ";
The error indicates a problem with the last ";"
USE REGEXP for getting only numbers
SELECT *
FROM your_table
WHERE zipcode REGEXP '^[0-9]+$';
or
SELECT *
FROM your_table
WHERE zipcode > 0
Hope it will help
expected output
Your zipcode column contains
123
145adb
adds142
157
237
output will be
123
157
237
Can anybody help me to get the sub_cat(csv) values which are not in id column from same table.
(sub_cat) comma separated values are the ids of same table, i need to get the values which are not in id column. Like 2,3,7 are present in id column whereas 20,24 are not. I need to get 20,24 only.
As I have elaborated in this post, I recommend not storing data in CSV format.
This gives trouble accessing and updating it.
I am not sure about this, but you might be able to simply use:
SELECT sub_cat FROM table_name WHERE id NOT IN (
SELECT sub_cat FROM table_name
)
However, I always prefer to store only one ID per row.
To do this purely in MySQL required writing a stored procedure; you'll need to change the database name test to the actual name of your database. It would have been a whole lot easier to do it with PHP - but where's the fun in that?
DELIMITER //
CREATE PROCEDURE test.check_subcats(
IN s_delimiter VARCHAR(30)
)
DETERMINISTIC
READS SQL DATA
BEGIN
DECLARE s_csv TEXT;
DECLARE i_subcat_index INT(10) unsigned DEFAULT 1;
DECLARE i_subcat_count INT(10) unsigned;
DECLARE l_category_done INT(10) DEFAULT FALSE;
DECLARE c_category CURSOR FOR SELECT category.sub_cat FROM category;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET l_category_done = TRUE;
-- create a temporary table to hold every csv value
CREATE TEMPORARY TABLE IF NOT EXISTS tmp_csv( cvalue VARCHAR(10) NOT NULL );
OPEN c_category;
l_category: LOOP
FETCH c_category INTO s_csv;
IF l_category_done THEN
LEAVE l_category;
ELSE
-- determine the number of sub-categories
SELECT (LENGTH(s_csv) - LENGTH(REPLACE(s_csv, s_delimiter, ''))) + 1 INTO i_subcat_count;
-- loop to store all csv values
WHILE i_subcat_index <= i_subcat_count DO
INSERT INTO tmp_csv ( cvalue ) (
SELECT REPLACE(SUBSTRING(
SUBSTRING_INDEX(s_csv, s_delimiter, i_subcat_index),
LENGTH(SUBSTRING_INDEX(s_csv, s_delimiter, i_subcat_index - 1)) + 1
), s_delimiter, '')
);
SET i_subcat_index = i_subcat_index + 1;
END WHILE;
END IF;
SET i_subcat_index = 1;
END LOOP;
CLOSE c_category;
SELECT DISTINCT tmp_csv.cvalue FROM tmp_csv WHERE tmp_csv.cvalue NOT IN ( SELECT category.id FROM category );
DROP TEMPORARY TABLE IF EXISTS tmp_csv;
END //
DELIMITER ;
I'm not 100% certain as to how robust it is but it was working with your data on my dev box.
You specify the delimiter for your CSV data when you call the procedure thus:
CALL `check_subcats`(',');
Essentially this loops through the category table to read sub_cat. It then splits the sub_cat value into chunks using the delimiter provided (much like PHPs explode() function) and writes every one of those values to a temporary table.
This then gives you a temporary table holding all your CSV data in individual bits and it's then a simple matter of selecting everything from that data that's NOT IN the category.id list.
Is it possible to use soundex to compare portion of a column with search term? For example if a user searches "fiftythree" - it will find "Nirve Sports Fifty-Three Cruiser in Gold Metallic". I tried combining soundex with locate function but got an error. Here is my php code:
$soundex = soundex($keyword);
$soundexPrefix = substr($soundex, 0, 2);
$sql = "SELECT name ".
"FROM products WHERE SOUNDEX(LOCATE('$keyword', name)) LIKE '%$soundexPrefix%'";
You'll need to break up each word and do the SOUNDEX comparison, which is exactly what this function I'm going to tell you about does.
Using the function
Example usage: SELECT p.name FROM products p WHERE soundex_match('fiftythree', p.name, ' ')
It takes 3 arguments:
needle: The word you are looking for
haysack: The string of words among which you are searching
splitChar: The whitespace charater that’ll split the string into
single words. Generally it is the space(‘ ‘)
If any word in haystack sounds similar to needle, the function will return 1 and 0 otherwise.
Creating the function in your database
So go into your database (phpMyAdmin or the command line) and execute this, you only need to do it this one time):
drop function if exists soundex_match;
delimiter $$
create function soundex_match (needle varchar(128), haystack text, splitChar varchar(1)) returns tinyint
deterministic
begin
declare spacePos int;
declare searchLen int default length(haystack);
declare curWord varchar(128) default '';
declare tempStr text default haystack;
declare tmp text default '';
declare soundx1 varchar(64) default soundex(needle);
declare soundx2 varchar(64) default '';
set spacePos = locate(splitChar, tempStr);
while searchLen > 0 do
if spacePos = 0 then
set tmp = tempStr;
select soundex(tmp) into soundx2;
if soundx1 = soundx2 then
return 1;
else
return 0;
end if;
end if;
if spacePos != 0 then
set tmp = substr(tempStr, 1, spacePos-1);
set soundx2 = soundex(tmp);
if soundx1 = soundx2 then
return 1;
end if;
set tempStr = substr(tempStr, spacePos+1);
set searchLen = length(tempStr);
end if;
set spacePos = locate(splitChar, tempStr);
end while;
return 0;
end
$$
delimiter ;
http://www.imranulhoque.com/mysql/mysql-function-soundex-match-multi-word-string/