Complicated ORDER BY needed to sort irregular values - php

I am trying to develop a system to display products from a database on a webpage. Normally this is no problem, except that one manufacturer has several abnormal part numbers I need to sort by properly.
Normally, I could just use an invisible column to sort things out, but then that makes inserting new items in-between two older ones much more difficult.
For example, here are some of the part numbers:
1211
1225
14-302
14-303
2015
23157
3507
35280UP
42-3309
42-3312
4241
Now, the normal order by mfgr produces the above order.
What it SHOULD be is something more like this:
14-302
14-303
42-3309
42-3312
1211
1225
2015
3507
4241
23157
35280UP
What is going to be my best bet on sorting this properly? If they were just being made in a csv file and uploaded afterwards this wouldn't be a problem. But because of automatic database changes, the server will be modifying values in real time. So manually updating it is out of the question. This means that a back end will be required to insert new items, but by what method could I insert an item between another? I would prefer to not resort to something like decimals to give me X in between values (Like I have 1.00 and 2.00 and I want to put another between them so I make it 1.50).
Any help would be appreciated.
EDIT:
The way I would like to sort it is this:
if it has a hyphen, ie 14-302, it is sorted by the 14, and then any 14-xxx is sorted by the numbers after the hyphen.
then, just numbers would be sorted by their actual numner, 802 comes before 45768.
Then any number that has a letter(s) after it will be sorted by the number, the the letter so 123a comes before 123b but after 122. And 123b comes before 124c.
and lastly anything that begins with an M- will be sorted last, and by the numbers after the hyphen.

With your adjusted question the principle is still the same.
My first instinct was to go with extra field but you could use stored function for those.
You still need to divide your partnumber in 3 parts (I take it it's not more than that).
Part1 is an INTEGER and has the first number if existing else fill it with maxint (largest for part1).
Part2 is a CHAR and contains letters if existing.
Part3 is an INTEGER containing the second number if existing.
You can sort by calling a function for these values.
Here is the complete source:
For your convenience a link to a working SQL Fiddle.
It's really sloppy/fast programming but it works. Put together in little time and i'm sure there are points for improvement. You can tweak it yourself. Later you can delete the part1, part2 and part3 from the view. (but leave it in the order by) It's only to show how the sort is done.
DROP PROCEDURE IF EXISTS `uGetParts`//
DROP FUNCTION IF EXISTS `uExtractPart1`//
DROP FUNCTION IF EXISTS `uExtractPart2`//
DROP FUNCTION IF EXISTS `uExtractPart3`//
CREATE PROCEDURE `uGetParts`(
IN ins varchar(50),
OUT num1 int unsigned,
OUT num2 int unsigned,
OUT num3 int unsigned)
NO SQL
BEGIN
SET num1=0;
SET num2=0;
SET num3=0;
WHILE (num1<length(ins)) AND
(SUBSTRING(ins,num1+1,1) REGEXP('(^[0-9]+$)')=1) DO
SET num1=num1+1;
END WHILE;
SET num2=num1;
WHILE (num2<length(ins)) AND
(SUBSTRING(ins,num2+1,1) REGEXP('(^[0-9]+$)')=0) DO
SET num2=num2+1;
END WHILE;
SET num3=num2;
WHILE (num3<length(ins)) AND
(SUBSTRING(ins,num3+1,1) REGEXP('(^[0-9]+$)')=1) DO
SET num3=num3+1;
END WHILE;
END//
CREATE FUNCTION `uExtractPart1`(ins varchar(50))
RETURNS int unsigned NO SQL
BEGIN
DECLARE num1 INT default 0;
DECLARE num2 INT default 0;
DECLARE num3 INT default 0;
call uGetParts(ins,num1,num2,num3);
IF num1>0 THEN
RETURN CAST(SUBSTRING(ins,1,num1) AS UNSIGNED);
ELSE
RETURN ~0 >> 32;
END IF;
END//
CREATE FUNCTION `uExtractPart2`(ins varchar(50))
RETURNS varchar(50) NO SQL
BEGIN
DECLARE num1 INT default 0;
DECLARE num2 INT default 0;
DECLARE num3 INT default 0;
call uGetParts(ins,num1,num2,num3);
IF num2>num1 THEN
RETURN SUBSTRING(ins,num1+1,num2-num1);
ELSE
RETURN '';
END IF;
END//
CREATE FUNCTION `uExtractPart3`(ins varchar(50))
RETURNS int unsigned NO SQL
BEGIN
DECLARE num1 INT default 0;
DECLARE num2 INT default 0;
DECLARE num3 INT default 0;
call uGetParts(ins,num1,num2,num3);
IF num3>num2 THEN
RETURN CAST(SUBSTRING(ins,num2+1,num3-num2) AS UNSIGNED);
ELSE
RETURN 0;
END IF;
END//
You can call it like this:
SELECT
id,
TYPE,
uExtractPart1(TYPE) as part1,
uExtractPart2(TYPE) as part2,
uExtractPart3(TYPE) as part3
FROM Article
ORDER BY
uExtractPart1(TYPE),
uExtractPart2(TYPE),
uExtractPart3(TYPE)

Related

How to remove all non-numeric characters from a MySQL table entry

I'm trying to write a PHP program to update a MySQL table entry according to a phone number. The phone numbers in the database are entered without limitations and are typically formatted in the XXX-XXX-XXXX way, but sometimes have other characters due to typos. In order to ensure the query works every time, I want to remove all non-numeric characters from the entries so that I can compare the entries to phone numbers formatted like XXXXXXXXXX coming from a separate source.
I've done some research and found some solutions but am unsure how to incorporate them into the PHP script. I am fairly new to MySQL and most of the solutions provided user defined MySQL functions and I don't know how to put them into the PHP script and use them with the query I already have.
Here's one of the solutions I found:
CREATE FUNCTION [dbo].[CleanPhoneNumber] (#Temp VARCHAR(1000))
RETURNS VARCHAR(1000) AS BEGIN
DECLARE #KeepValues AS VARCHAR(50)
SET #KeepValues = '%[^0-9]%'
WHILE PATINDEX(#KeepValues, #Temp) > 0
SET #Temp = STUFF(#Temp, PATINDEX(#KeepValues, #Temp), 1, '')
RETURN #Temp
END
And this is the query I need the solution for:
$sql = "SELECT pid AS pid FROM patient_data " .
"WHERE pid = '$pID' AND phone_cell = '$phone_number';";
The query should return the data in the pid column for a single patient, so if the phone number is 1234567890 and the pid is 15, 15 should be returned. I have no output at the moment.
The example function definition is Transact-SQL (i.e. for Microsoft SQL Server), it's not valid MySQL syntax.
A function like this doesn't go "into" the PHP code. The function gets created on the MySQL database as a separate step, similar to creating a table. The PHP code can call (reference) the defined function just like it references builtin functions such as DATE_FORMAT or SUBSTR.
The SELECT statement follows the pattern of SQL that is vulnerable to SQL Injection. Any potentially unsafe values that are incorporated into SQL text must be properly escaped. A better pattern is to use prepared statements with bind placeholders.
As an example of a MySQL function:
DELIMITER $$
CREATE FUNCTION clean_phone_number(as_phone_string VARCHAR(1024))
RETURNS VARCHAR(1024)
DETERMINISTIC
BEGIN
DECLARE c CHAR(1) DEFAULT NULL;
DECLARE i INT DEFAULT 0;
DECLARE n INT DEFAULT 0;
DECLARE ls_digits VARCHAR(20) DEFAULT '0,1,2,3,4,5,6,7,8,9';
DECLARE ls_retval VARCHAR(1024) DEFAULT '';
IF ( as_phone_string IS NULL OR as_phone_string = '' ) THEN
RETURN as_phone_string;
END IF;
SET n := CHAR_LENGTH(as_phone_string);
WHILE ( i < n ) DO
SET i := i + 1;
SET c := SUBSTR(as_phone_string,i,1);
IF ( FIND_IN_SET(c,ls_digits) ) THEN
SET ls_retval := CONCAT(ls_retval,c);
END IF;
END WHILE;
RETURN ls_retval;
END$$
DELIMITER ;
We can execute these statements in the mysql command line client, connected as a user with sufficient privilege, to create the function.
This isn't necessarily the best way to write the function, but it does serve as a demonstration.
Once the function is created, we can reference it a SQL statement, for example:
SELECT t.foo
, clean_phone_number(t.foo)
FROM ( SELECT '1' AS foo
UNION ALL SELECT '1-888-TAXICAB'
UNION ALL SELECT '888-555-1212'
UNION ALL SELECT '+=_-()*&^%$##"''<>?/;:"abc...xyz'
UNION ALL SELECT ''
UNION ALL SELECT NULL
) t

SQL Updating Optional Parameters PHP

We want to change the way we pass values from PHP to stored procedures (T-SQL). I only have minor experience with PHP but I will attempt to explain the process from discussions with our web developer.
Current Process
Example test table
In order to update a record, such as Field3 in this example, we would pass all existing values back to the stored procedure.
EXEC dbo.UpdateTest #ID = 1, #Field1 = 'ABC', #Field2 = 'DEF', #Field3 = 'GHI', #Field4 = 'JKL'
Lets say to update Field3, you must click a button. This would navigate to a new page which would run the stored procedure to update the data. As the new page is unaware of the values it has to run a SELECT procedure to retrieve the values before running an UPDATE.
The script would then redirect the user back to the page which reloads the updated data and the changes are reflected on screen.
New Process
What we would like to do is only pass the fields we want to change.
EXEC dbo.UpdateTest #ID = 1, #Field2 = 'DEF', #Field3 = 'GHI'
Our solution is simple. First we set all of the updatable fields to optional (so NULL can be passed). We then check to see if the parameter is NULL (is not passed), if it is then we ignore it and if it isn't we update it.
UPDATE
dbo.Test
SET
Field1 = NULLIF(ISNULL(#Field1,Field1),'-999')
,Field2 = NULLIF(ISNULL(#Field2,Field2),'-999')
,Field3 = NULLIF(ISNULL(#Field3,Field3),'-999')
,Field4 = NULLIF(ISNULL(#Field4,Field4),'-999')
WHERE
ID = #ID
However we still want the procedure to update the database record to NULL if a NULL value is passed. The workaround for this was to assign an arbitrary value to equal NULL (in this case -999), so that the procedure will update NULL if the arbitrary value (-999) is passed.
This solution is rather messy and, in my eyes, an inefficient way of solving the problem. Are there any better solutions? What are we doing wrong?
A huge thanks in advance to any replies
Valdimir's method is great as far as passing a flag variable to identify when the value is passed or not passed and his notes about arbitrarily picking a value are right on, but I would guess that there are some arbitrary values you may never have to worry about. such as -999 for a integer when you don't allow for negative numbers, or '|||||||' for a null string. Of course this breaks down some when you do want to use negative numbers but then you could potentially play around with numbers too big for a data type such as BIGINT as a parameter default -9223372036854775808 for an int.... The issue really comes down to your business case of whether values can or can not be allowed.
However if you go a route like that, I would suggest 2 things. 1) don't pass the value from PHP to SQL instead make that the default value in SQL and test if the parameter is the default value. 2) Add a CHECK CONSTRAINT to the table to ensure the values are not used and cannot be represented in the table
So something like:
ALTER TABLE dbo.UpdateTest
CHECK CONSTRAINT chk_IsNotNullStandInValue (Field1 <> '|||||||||||||||||||' AND Field2 <> -999)
CREATE PROCEDURE dbo.UpdateTest
#ParamId numeric(10,0)
,#ParamField1 NVARCHAR(250) = '|||||||||||||||||||'
,#ParamField2 INT = -99999 --non negative INT
,#ParamField3 BIGINT = -9223372036854775808 --for an int that can be negative
AS
BEGIN
DECLARE #ParamField3Value INT
BEGIN TRY
IF ISNULL(#ParamField3,0) <> -9223372036854775808
BEGIN
SET #ParamField3Value = CAST(#ParamField3 AS INT)
END
END TRY
BEGIN CATCH
;THROW 51000, '#ParamField3 is not in range', 1
END CATCH
UPDATE dbo.Test
SET Field1 = IIF(#ParamField1 = '|||||||||||||||||||',Field1,#ParamField1)
,Field2 = IIF(#ParamField2 = -99999,Field2,#ParamField2)
,Field3 = IIF(#ParamField3 = -9223372036854775808, Field3, #ParamField3Value)
WHERE
ID = #ParamId
END
The real problem with this method is the numeric data field allowing for negative numbers as you really don't have an appropriate way of determining when the value should be null or not unless you can pick a number that will always be out of range. And I definitely realize how bad of an idea the BIGINT for INT example is because now your procedure will accept a numeric range that it shouldn't!
Another method/slight variation of Vladimir's suggestion is to flag when to make a field null rather than when to update. This will take a little getting used to for your PHP team to remember to use but because these flags can also be optional they don't have to be burdensome to always include something like:
CREATE PROCEDURE dbo.UpdateTest
#ParamId numeric(10,0)
,#ParamField1 NVARCHAR(250) = NULL
,#MakeField1Null BIT = 0
,#ParamField2 INT = NULL
,#MakeField2Null BIT = 0
,#ParamField3 INT = NULL
,#MakeField3Null BIT = 0
AS
BEGIN
UPDATE dbo.Test
SET Field1 = IIF(ISNULL(#MakeField1Null,0) = 1,NULL,ISNULL(#ParamField1,Field1))
,Field2 = IIF(ISNULL(#MakeField2Null,0) = 1,NULL,ISNULL(#ParamField2,Field2))
,Field3 = IIF(ISNULL(#MakeField3Null,0) = 1,NULL,ISNULL(#ParamField3,Field3))
WHERE
ID = #ParamId
END
Basically if you are using the stored procedure to Update a table and it has nullable fields, I don't think I would recommend having the paramaters be optional as it leads to business cases/situations that can be messy in the future especially concerning numeric data types!
Your approach where you use a magic number -999 for the NULL value has a problem, as any approach with magic numbers have. Why -999? Why not -999999? Are you sure that -999 can not be a normal value for the field? Even if it is not allowed for a user to enter -999 for this field now, are you sure that this rule will remain in place in few years when your application and database evolve? It is not about being efficient or not, but about being correct or not.
If your fields in the table were NOT NULL, then you could pass a NULL value to indicate that this field should not be updated. In this case it is OK to use a magic value NULL, because the table schema guarantees that the field can't be NULL. There is a chance that the table schema will change in the future, so NULL can become a valid value for a field.
Anyway, your current schema allows NULLs, so we should choose another approach. Have an explicit flag for each field that would tell the procedure whether the field should be updated or not.
Set #ParamUpdateFieldN to 1 when you want to change the value of this field. Procedure would use the value that is passed in the corresponding #ParamFieldN.
Set #ParamUpdateFieldN to 0 when you don't want to change the value of this field. Set #ParamFieldN to any value (for example, NULL) and the corresponding field in the table will not change.
CREATE PROCEDURE dbo.UpdateTest
-- Add the parameters for the stored procedure here
#ParamID numeric(10,0), -- not NULL
-- 1 means that the field should be updated
-- 0 means that the fleld should not change
#ParamUpdateField1 bit, -- not NULL
#ParamUpdateField2 bit, -- not NULL
#ParamUpdateField3 bit, -- not NULL
#ParamUpdateField4 bit, -- not NULL
#ParamField1 nvarchar(250), -- can be NULL
#ParamField2 nvarchar(250), -- can be NULL
#ParamField3 nvarchar(250), -- can be NULL
#ParamField4 nvarchar(250) -- can be NULL
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
SET XACT_ABORT ON;
BEGIN TRANSACTION;
BEGIN TRY
UPDATE dbo.Test
SET
Field1 = CASE WHEN #ParamUpdateField1 = 1 THEN #ParamField1 ELSE Field1 END
,Field2 = CASE WHEN #ParamUpdateField2 = 1 THEN #ParamField2 ELSE Field2 END
,Field3 = CASE WHEN #ParamUpdateField3 = 1 THEN #ParamField3 ELSE Field3 END
,Field4 = CASE WHEN #ParamUpdateField4 = 1 THEN #ParamField4 ELSE Field4 END
WHERE
ID = #ParamID
;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
-- TODO: process the error
ROLLBACK TRANSACTION;
END CATCH;
END
So, parameters of the procedure are not optional, but you use #ParamUpdateFieldN flags to indicate which parameters hold useful values and which parameters should be ignored.
EXEC dbo.UpdateTest #ID = 1, #Field1 = 'ABC', #Field2 = 'DEF', #Field3 = 'GHI', #Field4 = 'JKL'
and
EXEC dbo.UpdateTest #ID = 1, #Field2 = 'DEF', #Field3 = 'GHI'
Are both valid ways to make use of the same stored procedure with MsSql or Sybase. When you don't send the values, it is the same as sending a null. Unless you set a default in the stored procedure. In that case the default is used instead of the null.
Not enough reputation to just comment.
In my opinion your solution is good enough as long as the arbitrary value cannot be a normal value for any of the fields.
However, I'd consider passing and storing something else besides NULL (“N/A” for example) when a field should not have an “actual” value and it’s purposely updated from the client side.

Recursive count of employees under a manager using mysql

I need the list of all employees, following the hierarchy of the manager using MYSQL from the following table. In oracle or mssql it is easy job, but could not find any solution in MySQL. Can anyone help me out how to sort it out.
id name manager
1 John 6
2 Gill 7
3 Ben 2
4 Roy 8
5 Lenin 6
6 Nancy 7
7 Sam 0
8 Dolly 3
If you still can limit the maximal number of levels, here is a solution with a recursive procedure. Since recursive functions are not allowed in MySQL, we have here a function (manager_count), which wraps the results from the recursive procedure. Recursion depth is controlled by the max_sp_recursion_depth variable, which takes 255 as its maximum. Use as follows: SELECT *,manager_count(id) FROM my_table. It's not the optimal solution, since it doesn't take into account already counted branches of the hierarchy (a temporary table can actually serve as a cache).
DELIMITER //
DROP FUNCTION IF EXISTS manager_count//
CREATE FUNCTION manager_count(_manager INT) RETURNS INT
BEGIN
DECLARE _count INT DEFAULT 0;
SET max_sp_recursion_depth = 255;
# manager_count_helper does the job
CALL manager_count_helper(_manager, _count);
# subtract 1, because manager_count_helper will count this manager as well
RETURN _count - 1;
END//
DROP PROCEDURE IF EXISTS manager_count_helper//
CREATE PROCEDURE manager_count_helper(IN _manager INT, INOUT _count INT)
BEGIN
IF EXISTS (SELECT 1 FROM my_table WHERE id = _manager) THEN
BEGIN
DECLARE _next_manager INT DEFAULT 0;
DECLARE done BOOLEAN DEFAULT FALSE;
# cursor to loop through the employees
DECLARE _cursor CURSOR FOR SELECT id FROM my_table WHERE manager = _manager;
# if done, the done variable gets TRUE and it's time too leave
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
# count 1, because this guy should be counted as well
SET _count = _count + 1;
OPEN _cursor;
read_loop: LOOP
FETCH _cursor INTO _next_manager;
IF done THEN LEAVE read_loop;
END IF;
CALL manager_count_helper(_next_manager, _count);
END LOOP;
CLOSE _cursor;
END;
END IF;
END

Split MYSQL results to contain n sizes per slice

This question is basically similar to SoulieBaby's question here:Split MYSQL results into 4 arrays, except that I wanted to split the result to contain a specific length.
Say, I wanted the result of an array which has a length of 9 to be splitted and have the splitted array to contain 5 lengths. So first array will have 5 and second will have 4.
Is this possible?
Thanks so much for any help!
The referenced question wanted to always have 4 chunks, therefore the solution was to create chunks of size ceil(count($array) / 5).
This case is easier, the (maximum) size is constant but the number of chunks vary.
Therefore the answer is simply:
array_chunk($array, 5);
Try:
DELIMITER $$
CREATE PROCEDURE split (in data varchar(500),in cad char(1))
BEGIN
declare pos int default 0;
declare numero int default 0;
declare van int default 0;
declare a varchar(500) default '';
drop TEMPORARY table IF EXISTS tmp_split;
create TEMPORARY table tmp_split(dato varchar(500)) ENGINE=MEMORY;
set pos=LOCATE(cad,data);
while pos<>0 do
set numero=numero+1;
set pos=LOCATE(cad,data,pos+1);
end while;
set a=SUBSTRING_INDEX(data,cad,1);
while numero>van do
set data=SUBSTRING(data,LENGTH(a)+2);
insert into tmp_split values (a);
set a=SUBSTRING_INDEX(data,cad,1);
set van=van+1;
end while;
insert into tmp_split values (a);
select * from tmp_split;
END
call split('1,2,5,52,64,365,9714,253,6697,8,9,2,62',',');

Possible to order an SQL query that matches with a REGEX by the number of matches found?

I am using a sql query such as WHERE name REGEXP '[[:<:]]something[[:>:]]'.
Now this all works great but my results are not ordered by number of matches found which is what I am looking for. Any ideas on how to go about doing this or if it is even possible?
Thanks
Full Query is
SELECT `Item`.`id`, `Item`.`name`, `Item`.`short_bio`
FROM `items` AS `Item`
WHERE ((`Item`.`name` REGEXP '[[:<:]]hello[[:>:]]') OR
(`Item`.`name` REGEXP '[[:<:]]world[[:>:]]')
Now this query is generated based on user input, each space breaks the thing into a different part that is searched for. I would like to order the results based on the number of matches of all parts, this way the most relevant results are on the top.
How about something like this (don't know mysql, so it may need tweaking):
SELECT `Item`.`id`, `Item`.`name`, `Item`.`short_bio`
FROM `items` AS `Item`
WHERE ((`Item`.`name` REGEXP '[[:<:]]hello[[:>:]]') OR
(`Item`.`name` REGEXP '[[:<:]]world[[:>:]]')
ORDER BY (`Item`.`name` REGEXP '[[:<:]]hello[[:>:]]') +
(`Item`.`name` REGEXP '[[:<:]]world[[:>:]]') DESC
I found an UDF some time ago to do this. I'm really sorry I can't cite the source though.
DELIMITER //
CREATE DEFINER=`root`#`localhost` FUNCTION `substrCount`(s VARCHAR(255), ss VARCHAR(255)) RETURNS tinyint(3) unsigned
READS SQL DATA
BEGIN
DECLARE count TINYINT(3) UNSIGNED;
DECLARE offset TINYINT(3) UNSIGNED;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET s = NULL;
SET count = 0;
SET offset = 1;
REPEAT
IF NOT ISNULL(s) AND offset > 0 THEN
SET offset = LOCATE(ss, s, offset);
IF offset > 0 THEN
SET count = count + 1;
SET offset = offset + 1;
END IF;
END IF;
UNTIL ISNULL(s) OR offset = 0 END REPEAT;
RETURN count;
END
DELIMITER ;
There's also a nifty solution found here.
Regex matching operators in MySQL return either 1 or 0 depending on whether the match was found or not respectively (or null if either a pattern or string is null). No information about number of matches is available, so sorting is not possible either.

Categories