I have a few 10GB files with mysql info which I would like to filter for a specific table.
The queries look like this (though can have fewer or more line breaks):
SET INSERT_ID=2/\*!\*/;
at 858735202
121124 12:36:53 server id 1 end_log_pos 0 Query thread_id=9695754 exec_time=0 error_code=0
SET TIMESTAMP=1663753413/\*!\*/;
INSERT INTO `bank_accounts_daily`
(
`accounts_bank_md5` ,
`accounts_bank_payment_desc` ,
`accounts_bank_amount` ,
`accounts_bank_number` ,
`accounts_bank_sortcode` ,
`accounts_bank_currency` ,
`accounts_bank_date`,
`accounts_bank_code`
)
VALUES
(
'zxcvxzcvxzc4c9eeca78908296a2f007',
'NAMEJO M 1105294 BBP',
'278.50',
'645450441',
'20-55-19',
'1',
'26/55/2012',
'BBP'
)
/\*!\*/
I am using this, which works to retrieve every single statement:
preg_match_all('/(SET INSERT_ID=([0-9]+)\/\*\!\*\/\;)(.*?)(\/\*\!\*\/\;)(.*?)(\/\*\!\*\/)/s', $input, $output);
But when I attempt to expand it and add the extra pattern to match specifically the 'bank_accounts_daily' pattern it does not retrieve anything(regardless of backticks being escaped or not):
preg_match_all('/(SET INSERT_ID=([0-9]+)\/\*\!\*\/\;)(.*?)(\/\*\!\*\/\;)(.*?)(INSERT INTO \`bank_accounts_daily\`)(.*?)(\/\*\!\*\/)/s', $input, $output);
I do not understand why this is not working. I've tried variations without brackets, but nothing is working. Also - are there any potential problems with my approach that I am not seeing?
Try this regexp:
/(SET INSERT_ID=([0-9]+)\/\\\*\!\\\*\/\;)(.*?)(\/\\\*\!\\\*\/\;)(.*?)(INSERT INTO `bank_accounts_daily`)(.*?)(\/\\\*\!\\\*\/)/s
You weren't matching the backslashes in the /\*!\*/ markers. I don't see how the original regexp could have worked, since it also had this mistake.
Related
Using a combination of MS-SQL, PHP, and JSON, I have a column in my page called VendorAddress that outputs to the browser like this:
1111 THAT WAY DRIVE �TORRANCE
That strange black diamond with the questionmark.
I have narrowed it down to the fact that some users are copying and pasting from Excel, and there is an extra space between the street and city. I can prevent this from happening in the future. It's the ones that already been saved to the table that is causing a problem for me.
I have found there are over 300 records in the MS-SQL database that have this extra space.
What I need to do is write a query that would update the column by removing this extra space.
UPDATE [main].[dbo].[suppliers] SET [VendorAddress] = '1111 THAT WAY DRIVE TORRANCE' WHERE [PartnerCode] = '00011144';
This is the query I was using, going record for record for each PartnerCode that has the black diamond (336 total). I refuse to go record for record on this. I need to write a query that will do it for me.
I am not sure how to go about doing this. Everything I find online tells me to use TRIM, but it's only for trailing spaces. These are not trailing spaces. They are within the string.
I wish to write a query that would look at the whole table and remove the unnecessary spaces from within the string.
Edit
The special character cannot be seen in the database itself. The only time you see it is when PHP outputs it to the browser.
Basically, within the table, the field contains an extra space. The black diamond cannot be seen within the table.
It seems like you should be able to just employe the REPLACE function:
UPDATE [main].[dbo].[suppliers]
SET [VendorAddress] = REPLACE(VendorAddress, '�', '')
This command seems to work:
SELECT REPLACE('1111 THAT WAY DRIVE �TORRANCE', '�', '')
Have you tried updating it away? Here's some code for capturing the glyph if copy/paste won't work.
DECLARE #Glyph CHAR(1);
SELECT #Glyph = SUBSTRING([VendorAddress]
, -- Put numberic position of Glyph in string here
,1)
FROM [main].[dbo].[suppliers]
WHERE [PartnerCode] = '00011144'; -- Record with erroneous Glyph
SELECT #Glyph AS [Questionable Character]
, [VendorAddress] AS [Before]
, REPLACE([VendorAddress],#Glyph,'') AS [After]
WHERE [PartnerCode] = '00011144'; -- Use same record as above.
/* Uncomment out this bit when you know it works...
UPDATE [main].[dbo].[suppliers]
SET [VendorAddress] = REPLACE([VendorAddress],#Glyph,'')
WHERE [VendorAddress] LIKE '%' + #Glyph + '%';
*/
To replace double spaces with a single space use the following:
UPDATE [main].[dbo].[suppliers]
SET [VendorAddress] = replace([VendorAddress], ' ',' ')
WHERE [VendorAddress] LIKE '% %'
Repeat it until you get 0 rows affected.
* EDIT *
Otherwise can you execute this query to find out the code of that character?
select distinct cast([VendorAddress] AS varbinary) FROM [main].[dbo].[suppliers] WHERE [VendorAddress] LIKE '1111 THAT WAY DRIVE%TORRANCE'
$itemList = DB::table('items')
->orderBy('id', 'desc')
->where('item_status', 2)
->where(function($query) use($queryArr)
{
foreach($queryArr as $uID){
$query->whereRaw("tags LIKE '%$uID%'");
}
})->paginate(21);
I have been facing this issue since a long time. Problem when you do a LIKE search is it grabs the data of WOMEN when it's just eg MEN
Mainly because MEN is inside Women
I also tried the following but failed(This sort of grab a word) men without women data
$query->where('tags', 'LIKE', '%'.$uID.'%');
SELECT 'a word a' REGEXP '[[:<:]]word[[:>:]]';
How do i use that Word boundary query in laravel query builder
Tried this and still failed $query->whereRaw("tags LIKE REGEXP '[[:<:]]Men[[:>:]]'");
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'REGEXP '[[:<:]]Men[[:>:]]')' at line 1 (SQL: select count(*) as aggregate from items where item_status = ? and (tags LIKE REGEXP '[[:<:]]Men[[:>:]]')) (Bindings: array ( 0 => 2, ))
I also understand some asked why not just created a proper way of handling these item's category. Well i think for now using Full Text Search is fine for me at most when it come to scaling i will use elastic search. true?
UPDATE
Apologies for not giving an example of tags
Bag,Shoes,Men,Wedges
Bag,Shoes,Men
Men,Shoes,Bag
If values are separated by commas, try to use following
WHERE `tags` REGEXP "(^|,)men(,|$)"
This will require to have comma or end of string around the word men.
Your REGEXP solution is throwing the syntax error because of the extra LIKE keyword. The correct syntax is simply WHERE column REGEXP 'pattern'.
If you find this hard to remember, try using RLIKE instead, which is a synonym for MySQL's REGEXP. There is less chance you will accidentally write WHERE column LIKE RLIKE ... in your query because it is more obviously redundant.
The pattern itself should work fine. Personally, I dislike using those word boundary classes in MySQL; since you know your tags are comma delimited and space padded, [ ,]Man[ ,] would function just as well. The word boundary class breaks at punctuation other than underscores, so you could run into trouble if you have tags that are hyphenated, for instance.
If you want to use multiple word tags with spaces, either of the previous patterns is buggy. In that case I would try to stick with commas as your delimiter, get rid of the whitespace padding and use anchors instead, as suggested in one of the other answers: (^|,)Man($|,)
In your query you said : Find anything that contains the tag 'MEN' => '%MEN%', that's why it shows: WOMEN AND MEN, so to answer to your question you should use starts with instead of contains, like this :
$query->where('tags', 'LIKE', $uID.'%');
Ok I know my database is not normalized but in my case, It is not possible to normalize the data as each user has different access level with access rights, everytime I modify something and I need to delete the rows and insert back so I went for comma-separated values, here the issue is if I remove a particular group, regex doesn't work, it throws me an error..am using php and the error goes like this
FUNCTION database_name.REGEXP_REPLACE does not exist
Table structure
allowed_group_ids
+----------------+
12345,34345,55454
My query, say for example $delete_id is 12345 or say 55454, I only pass one ID at a time and the id's has no space in between and it's a text field
UPDATE tbl_scripts SET allowed_group_ids = TRIM(BOTH ','
FROM REGEXP_REPLACE(allowed_group_ids, '(,(\s)?)?$detele_id', ''))
WHERE system_id = {$_SESSION['system_id']}
So what's wrong in here?
There is no such thing as regexp_replace in MySQL.
You could perform this query using regular replace:
UPDATE tbl_scripts SET allowed_group_ids =
trim(BOTH ',' FROM
replace(
replace(allowed_group_ids, '$delete_id',','),
',,',
','
)
)
WHERE system_id = {$_SESSION['system_id']}
This first removes the ID, and removes any doubled-up commas, then removes any commas at the start/end of the string. This should be sufficient but you can add another replace to remove whitespace if necessary.
Update: SQL Fiddle shows the query in action
Using Drupal 7, and I'm trying to get results from the database using the LIKE command but it doesn't recognize my wildcards. I'm not sure if this is even a Drupal issue, or if I'm doing something wrong. Anyways here's an example of the data I'm trying to match, along with my patters
Data to Match
a:2:{i:1;s:2:"17";i:2;s:1:"3";}
My like Queries
$pattern1 = 'a:2:{i:1;s:2:"17";i:2;s:1:"%";}'//works
$pattern2 = 'a:2:{i:1;s:1:"%";i:2;s:1:"3";}'//fails
$result = db_query(
"
SELECT pa.nid, pa.model, pa.combination
FROM {$Product_Adjustments} pa
WHERE pa.combination LIKE :pattern
",
array(
':pattern' => $pattern1
)
);
Additionally, I've tried the '_' wildcard, but that doesn't bring anything up either
Are you sure the pattern is correct? Notice pattern 1, the first string is 2 long, and in pattern 2 you're looking for one that's only 1 long. Are you sure that's right? Are the lengths of the individual pieces of that serialized data predictable enough to even query this way? It seems unlikely, and you'll probably have to store some normalized data instead.
Am wondering whether and how i can retrieve a portion of text data from the text field in MSQL DB. What SQL function can i use in my query statement.
Eg.I have a text field containing data that looks like below;
(firstname1,lastname1,23,,district1,state1,province1),(firstname2,lastname2,23,,district2,state2,province2),
(firstname3,lastname3,23,,district3,state3,province3),...etc
how would i select for example a piece of data starting from
"lastname1" to ")" the bracket closing the first set.
am trying to figure out how i can do this but still failing.
The thing is i don't want to retrieve all the data in the text field...so am wondering whether the MySQL query statements can help...
though the regular expressions would be of great use in this case as i think, but i don't know how to structure them, am a new Bi...please help me at-least on this case.
thanks in advance.
You can also do it directly in MYSQL. Something like this would work:
SELECT SUBSTR( YOUR_COLUMN, INSTR( YOUR_COLUMN, 'START_CHAR_GOES_HERE' ) , INSTR( YOUR_COLUMN, 'END_CHAR_GOES_HERE' ) - INSTR( YOUR_COLUMN, 'START_CHAR_GOES_HERE' ) )
FROM YOUR_TABLE
reference:
http://dev.mysql.com/doc/refman/5.5/en/string-functions.html#function_substr
http://dev.mysql.com/doc/refman/5.5/en/string-functions.html#function_instr
use locate(), substr() and char_length() in mysql.
Also read this.