What I've been trying to do is to select a row from a table while treating the varchar cells as int ones,
Here's a little explanation:
I have a table of phone numbers, some have "-" in them, some don't.
I wanted to select a number from the database, without including those "-" in the query.
So I used this preg_replace function:
$number = preg_replace("/[^0-9]/","",$number); //that leaves only the numbers in the variable
and then I run the following query:
"SELECT * FROM `contacts` WHERE `phone` = '{$number}'"
Now, of course it won't match sometimes since the number Im searching may have "-" in the database, so I tried to look for a solution,
on solution is just converting the cells into int's, but I'm not interested in doing that,
So after looking around, I found a MySQL function named CAST, used like : CAST(phone AS UNSIGNED)
I tried to mess with it, but it didn't seem to work.
Edit:
I kept looking around for a solution, and eventually used MySQL's REPLACE function for that.
"SELECT * FROM `contacts` WHERE REPLACE(phone,'-','') = '{$number}'"
Thank you all for your help.
MySQL doesn’t support extraction of regex matches.
You could try writing a stored function to handle it, but your best bet is to convert the data to ints so that all the numbers are uniform. I know you said you don't want to do that, but if you can, then it’s the best thing to do. Otherwise, you could do something like:
"SELECT * FROM `contacts` WHERE `phone` = '{$number}' OR `phone` = '{$number_with_dashes}'"
That is, search for the plain number OR the number with dashes.
1.
The easiest way to do it might be by using the REPLACE operator.
SELECT * FROM `contacts` WHERE REPLACE(REPLACE(`phone`, '-', ''), ' ', '') = '5550100';
What it does is simpy replacing all whitespaces and dashes with nothing, namely removing all spaces and dashes.
2.
Another alternative to solve the problem would be to use LIKE. If the phone numbers with a dash always are formatted the same way like 555-0100 and 555-0199 you can simple insert a %-sign instead of the dash. If your number may be formatted in different ways you can insert a %-between every character. It's not a beautiful solution but it does the trick.
SELECT * FROM `contacts` WHERE `phone` LIKE '555%0100';
or
SELECT * FROM `contacts` WHERE `phone` LIKE '5%5%5%0%1%0%0';
3.
You can use regular expressions. Since MySQL doesn't implement regex replace functions you need to use user defined functions. Have a look at https://launchpad.net/mysql-udf-regexp. It supports REGEXP_LIKE, REGEXP_SUBSTR, REGEXP_INSTR and REGEXP_REPLACE.
Edit: Removed my first answer and added some other alternatives.
I kept looking around for a solution, and eventually used MySQL's REPLACE function for that.
"SELECT * FROM `contacts` WHERE REPLACE(phone,'-','') = '{$number}'"
Related
I am running a very simple SELECT query in MySQL and it's not working.
SELECT string_name FROM table_name;
This is giving me required output. Like
This is string one.
This is string two.
This is string three.
and so on...
But if I am running a query like this
SELECT * FROM table_name WHERE string_name='This is string one'
It's not giving any output. I even tried TRIM function.
SELECT * FROM table_name WHERE TRIM(string_name)=TRIM('This is string one')
But it's still not giving any output.
Please suggest what I am missing here. Is it because of some formatting or am I doing any silly mistake. By the way, Strings are saved as VARCHAR in the database.
To reiterate from comments; sometimes "non-printing" control characters (like newlines) can make their way into data they were never intended to be a part of. You can test for this by checking CHAR_LENGTH of field values versus what you actually see. Obviously, on large amounts of data this can be difficult; but if you know of one problematic value already, you can use this method to confirm this is the problem on that row before attempting to identify the offending character.
Once this problem is confirmed, you can use queries with MySql's ASC() and substring functions to identify character codes until you find the character; it can be best to start from the end of the string and work back, as often the offending characters are at the end.
The character or characters identified in known problem rows are often the cause of other problem rows as well, so identifying the issue in one known row can actually help resolve all such problems.
Once the character code(s) are identified, queries like WHERE string_name LIKE CONCAT('%', CHAR(13), CHAR(10)) should work (in this case for traditional Windows newlines) to identify other similar problem rows. Obviously, adjust character codes and wildcards according to your circumstances.
If no row should ever have those characters anywhere, you should be able to clean up the data with an update like this:
UPDATE theTable SET theString = REPLACE(REPLACE(theString, CHAR(10), ''), CHAR(13), '') to remove the offending characters. Again, use the codes you've actually observed causing the problem; and you can convert them to spaces instead if circumstances are better handled that way, such as a newline between two words.
Have you tried using LIKE for debugging purposes?
SELECT * FROM table_name WHERE string_name LIKE 'This is string one'
/!\ Don't just switch from = to LIKE, read about why here
TLDR:
= is apparently 30x faster.
Use = wherever you can and LIKE wherever you must.
First of all, I must acknowledge the points made by #Uueerdo were actually the the main cause of this issue. Even I was somewhat sure that there are some hidden characters in the string causing all the issue but I was not sure how to find and fix that offending character.
Also, the approach suggested by #Uueerdo to check and replace the offending character using the ASCII code seems quite legit but as he himself mentioned that this process will take lot's of time and one have to manually check every string for that one offending character and then replace it.
Luckily after spending couple of hours on it, I came up with a much faster approach to fix the issue. For that, first of all I would like to share my use case.
My first query was for selecting all the strings from a database and printing the result on page.
$result = mysqli_query($conn, "SELECT * from table_name");
while($row = mysqli_fetch_array($result)){
$string_var = $row["string_name"];
echo $string_var;
echo "<br>";
}
The above code was working as expected and printing all the string_name from the table. Now, if I wanted to use the variable $string_var for another SELECT query in the same table, it was giving me 0 results.
$result = mysqli_query($conn, "SELECT * FROM table_name");
while($row = mysqli_fetch_array($result)){
$string_var = $row["string_name"];
echo "String Name : ".$string_var."";
$sec_result = ($conn, "SELECT * FROM table_name WHERE string_var='$string_name'");
if(mysqli_num_rows($sec_result) > 0){
echo "Has Results";
} else {
echo "No Results";
}
}
In this snippet, my second query $sec_result was always giving me No Results as output.
What I simply did to fix this issue.
$result = mysqli_query($conn, "SELECT * FROM table_name";
while ($row = mysqli_fetch_array($result)){
$string_var = $row["string_name"];
$row_id = $row["id"];
$update_row = mysqli_query($conn, "UPDATE table_name SET string_name='$string_var' WHERE id=$row_id");
}
This step updated all the strings from the table without any hidden/problem causing character.
I am not generalising this approach and I am not sure if this will work in every use case but it helped me fix my issue in less than a minute.
I request #Uueerdo and others with better understanding on this to post a more generic approach so that it can help others because I think many people who can't find a right approach in such conditions, end up using LIKE in place of = but that completely changes the core idea of the query.
I'm sure that there is a stupidly simple solution to this, but unfortunately my google-fu is too weak to find it.
I have a number of different tables for sizing, all following the same naming convention i.e size_001, size_002 etc. Within a loop I need to get the size entry that matches with the results already found.
Unfortunately there are no totally unique identifiers, as they repeat in each table (roman numerals for sizing). But they are unique in each individual table. So what I've tried so far looks a little bit like this:
SELECT * FROM CONCAT('size_00', '.$sizeTableID[$j].') WHERE sizeName LIKE '$sizeNames[$j]'"
Where $sizeTableId is a number from 1-9 and sizeName is a string e.g II or VI or, occasionally (because there's no consisitency), 2 etc
I've also tried ''$var'' inside the CONCAT and not using the CONCAT at all. Really I just need a way to join the database.size_00 and an integer variable.
If I understand correctly, this is actually simple:
$tablename = 'size00'.$sizeTableID[$j];
$sql = "SELECT * FROM $tablename WHERE sizeName LIKE '{$sizeNames[$j]}'";
and I think that solves it.
PHP is a bit quirky here.....
Try this one (when the variable is from an array/object, surround it with {})
$sql = "SELECT * FROM CONCAT('size_00', '{$sizeTableID[$j]}') WHERE sizeName LIKE '{$sizeNames[$j]}'";
I am using this basic mySQL query which works great:
$sql = "SELECT * FROM `clients` WHERE
MATCH(`LNAME`) AGAINST('$c') OR
MATCH(`FNAME`) AGAINST('$c') OR
MATCH(`MAIL`) AGAINST('$c') OR
MATCH(`TEL`) AGAINST('$c') "
where $c is the search query. Now this works with all single words/numbers but whenever I add 2 words no results are returned.
For example, if my database has aaaa bbbb in LNAME and I search for "aaaa bbbb" I get nothing back, however when I search for "aaaa" or "bbbb" it does work. I tried adding IN BOOLEAN MODE but it doesn't make a difference.
Could ayone explain to me how this works? $c is composed of letters, numbers and/or a #
thanks a lot.
First , you should use MATCH AGAINST like this:
$sql = "SELECT * FROM `clients` WHERE MATCH(`LNAME`,`FNAME`,`MAIL`,`TEL`) AGAINST('$c')"
Please notice:
Short words are ignored, the default minimum length is 4 characters.
You can change the min and max word length with the variables
ft_min_word_len and ft_max_word_len
and:
If a word is present in more than 50% of the rows it will have a
weight of zero. This has advantages on large datasets, but can make
testing difficult on small ones.
You can use LIKE and it probably will have better results.
Example of usage:
$sql = "SELECT * FROM `clients` WHERE `LNAME` LIKE '%$c%' OR `FNAME` LIKE '%$c%' OR ..."
I want to search like this: the user inputs e.g. "murrays", and the search result will show both records containing "murrays" and records containing "murray's". What should I do in my query.pl?
What do you think about using the SOUNDEX function and the SOUNDS LIKE operator ?
That way, you can simply do:
SELECT * from USERS WHERE name SOUNDS LIKE 'murrays'
I'm pretty sure it doesn't work for every case, and perhaps it is not the most efficient way to solve the problem, but it could fit your needs.
This won't help if you absolutely need to do these queries in SQL, but if you can set up a Lucene search index for it, you gain a lot of this kind of "fuzzy search" functionality. Note though that Lucene is quite a complex topic by itself.
What you could do is create an extra field in the database, which contains the data with all special characters stripped from it, and search there. A bit lame, I know. Looking forward to see smarter answers ;)
Quick and dirty:
SELECT * FROM myTable WHERE REPLACE(name, '\'', '') = 'murrays'
I would first build a search column which has the text without punctuation and then search on that. Otherwise you'll have have to have a series of regular expressions to search against or check individual records in PHP for matching: both of which are computational intensive operations.
Maybe something like this: (untested!)
SELECT * FROM users WHERE REPLACE(user_name, '\'', '') = "murrays"
If this is for single word searching, you could try using Soundex or Metaphone functions? These would handle sounds-like as well as spelling
Not sure if MySQL has these, but PHP does (which would require separate columns to hold these values).
Otherwise, Richy's no-punctuation extra column seems best.
You could try adding a replace to your query like this
replace(name, '''','')
to temporarily get rid of the apostrophes for the match.
select name from nametable where name = replace(name,'''','');
This query should be able to pick up "murrays" or "murray's".
var inputStr = "murrays";
inputStr = String.Replace("'", "\'", inputStr);
SELECT * FROM ATable WHERE Replace(AField, '\'', '') = inputStr OR AField = inputStr
strip user input and names in database from all non-letter characters.
Use levenstein distance or soundex to find murrays with murray or marrays. This is optional but your users would love that.
I'm having trouble with the sql below. Basically I have rows that contains strings according to the format: 129&c=cars. I only want the digits part, e.g. 129. The sql query is:
$result = mysql_query("SELECT * FROM " . $db_table . " WHERE id LIKE '" . $id . "%'");
Why doesn't % work? I can't use %...% because it catches too much.
I would actually recommend using regular expressions fo the matching, but unfortunately, there is no way to capture the matching part with mysql. You will have to do the extraction in php. If you have an array containing all the results called $array:
$array = preg_replace('/^(\d+).*/', '$1', $array);
You can use the MySQL 'regexp' stuff in the WHERE clause to reduce the amount of data retrieved to just the rows you want. The basic for of your query would look like:
SELECT * FROM table WHERE field REGEXP '^$id&'
where $id is inserted by PHP and the data you want is always at the start of the field and followed by a &. If not, adjust the regex to suit, of course.
MySQL's regex engine can't do capturing, unfortunately, so you'll still have to do some parsing in PHP as soulmerge showed above, but with the 'where regexp' stuff in MySQL, you'll only have to deal with rows you know contain the data you want, not the entire table.
Using a query like this:
SELECT *
FROM mytable
WHERE id >= '0' COLLATE UTF8_BIN
AND id < ':' COLLATE UTF8_BIN
will return all strings that start with a digit and make your expression sargable, i. e. and index on id can be used.
This will make your query run faster.