I have a to match a field in MySQL, for which I thought I could use a regular expression, but it appears that MySQL doesn't have the functionality I need to do the job. Here's the scenario:
I have a variable in PHP called $url. Let's say this variable is set as the string "/article/my-article/page/2". I also have a table of URLs in MySQL from which I would like to pull content. The URLs stored in my table, however, include wildcards.
Previously, I had this set up so that the value stored in the table looked like this: "/article/%/page/%".
With that configuration, I could just run:
SELECT * FROM urls WHERE '$url' LIKE url
And this would match, which is the desired functionality.
What I'd like to do now, is allow a more advanced wildcard, such that instead of "/article/%/page/%", my MySQL data could be "/article/{{slug}}/page/{{page_no}}".
I want to create a SQL query that will match this data, using the same $url input. LIKE is no longer the correct comparison, since I'm not using the built-in "%" wildcard, but rather {{.*}}. Any ideas how to accomplish this?
There is a library of user defined functions that gives you preg_replace in MySQL:
http://www.mysqludf.org/lib_mysqludf_preg/
It sounds like what you want to do is have the new syntax in the database where the URLs have placeholders you would pass to your php-based (sprintf) variable replacement code, but still be able to do the original comparisons to match the URL.
If I understand correctly you want to take a new URL format
/article/{{slug}}/page/{{page_no}}
and match it against something like
/article/my-article/page/2
The preg plugin sagi mentioned can do the substitution you need, which will turn one of your newly formatted URLs into the original format you used to determine the match using the LIKE syntax. The following query:
SELECT PREG_REPLACE('/({{.*?}})/', '%', `url`) FROM urls;
Would turn the new url (/article/{{slug}}/page/{{page_no}}) into what it was originally
/article/%/page/%
which can then be fed back through your original query, something like this:
SELECT * FROM urls
WHERE '/article/my-article/page/2' LIKE preg_replace('/({{.*?}})/', '%', `url`);
Some binary distributions like MAMP, XAMMP etc have the plugin already installed, but it isn't installed on a lot of systems like Macports / Ubuntu. Here are a couple of articles about installing the preg plugin. Hope it helps.
http://quickshiftin.com/blog/2011/04/installing-mysql-preg-plugin-osx-macports/
http://quickshiftin.com/blog/2011/12/installing-the-mysql-preg-plugin-on-ubuntu-with-apt-get/
The user sagi above mentions http://www.mysqludf.org/lib_mysqludf_preg/ but as this answer is very old as are most of the tutorials, I wanted to expand on this for the sake of newcomers to this question.
Firstly, the library is really great and speaking from experience I can say it seems to have been maintained and is still working flawlessly in 2015.
To get it installed and working, I could only find some very dated tutorials so thought I would share what I did that worked for me installing latest stable release (v1.1) on Ubuntu 14.04:
apt-get update
apt-get install libpcre3-dev libmysqlclient-dev build-essential libmysqld-dev libpcre3-dev
wget https://github.com/mysqludf/lib_mysqludf_preg/archive/lib_mysqludf_preg-1.1.tar.gz
tar -xzf lib_mysqludf_preg-1.1.tar.gz
cd lib_mysqludf_preg-1.1
./configure
make install
make installdb
service mysql restart
You should now have all the following functions available to you:
lib_mysqludf_preg_info
preg_capture
preg_check
preg_replace
preg_rlike
preg_position
Starting from mysql 5.5, you can use RLIKE :
either store url in REGEXP-style, and query with
SELECT * FROM urls WHERE '$url' RLIKE url;
or keep it in LIKE-style, and make a replacement (% by .* , _ by .):
SELECT * FROM urls WHERE '$url' RLIKE REPLACE(REPLACE(url, '%', '.*'), '_', '.');
To be complete, you would have to do other replacements to escape chars that are regexp-significant : ? \ () [] ... (see php function preg_quote)
I've found a solution. It's not ideal, but I'll put it in here in case a pure SQL solution never surfaces. I can leave the url field in MySQL equal to "/articles/%/page/%" and add another field called variables that stores a list of variable names to be used. That way I can use my original query, then replace the "%" characters in the return value with "{{variable}}" in PHP with sprintf after I've retrieved the data.
Still, I'd like to see this solved in SQL if possible, since I think the concept is valuable.
Related
A client of mine has used Velvet Blues Update URLs plugin (https://en-gb.wordpress.org/plugins/velvet-blues-update-urls/) on their site and have broken pretty much all of their links...
So now the links look like: www.site.compost-name/ instead of www.site.com/post-name/.
This is a live site, so I need to be very careful how I fix it so I don't make matters worse.
So I know I need some kind of Regex to check for the site url followed by either letters or numbers without the /.
I can view a lot f the broken links by running a query like: SELECT * FROM wp_postmeta WHERE meta_value REGEXP 'site.com[0-9 a-z A-Z]'.
And typically when updating something in the DB I would run a query like: UPDATE wp_postmeta SET meta_value = replace(meta_value,'http://www.oldurl','http://www.newurl');.
But from what I have found from my research is that MySQL doesn't support a kind of preg_replace functionality (How to do a regular expression replace in MySQL?).
I guess one approach might be to add a filter to wp_content and fix it before it is loaded on the page, but this only masks the problem and doesn't necessarily fix it.
I'm a little stumped on this one because there are thousands of broken links so there is a lot of scope for a search and replace query to go awry and potentially make things worse, plus my regex knowledge is limited.
You can use NOT LIKE operand for this.
For example,
UPDATE wp_postmeta
SET meta_value=REPLACE(meta_value,'site.com','site.com/')
WHERE meta_value LIKE '%site.com%'
AND
meta_value NOT LIKE '%site.com/%'
Of course, just in case, you need to run SELECT query first and review the results. Because it is not ideal approach and may cause some rare issues.
sed is your best friend for things like this.
sed 's/oldurl/newurl/g' -r /project/root will update every plain text instance in your entire project.
I have a database that stores static paths of image cache files we keep. Something like:
http://subdomain.domain.com/img/cache/24295839.jpg
is stored hundreds of thousands of times. We recently switched to SSL and need to create some type of script to change all of the http to https in the database.
The field is:
cached_file_url
I don't know where to start with this. Any help would be appreciated.
UPDATE yourtable
SET
cached_file_url = CONCAT('https://',
SUBSTRING_INDEX(cached_file_url, 'http://', -1))
Please see fiddle here.
You should be able to do this in one simple query.
UPDATE table
SET cached_file_url = REPLACE(cached_file_url, 'http://', 'https://')
WHERE cached_file_url LIKE 'http://%'
You should familiarize yourself with MySQL string functions (or at least know where to look when you have questions like this).
http://dev.mysql.com/doc/refman/5.5/en/string-functions.html
You can do a lot of the sorts of string manipulations you might commonly do in a programming language within MySQL itself.
...or any clause in my case.
I'm trying to create a mirgration script for a project I'm currently working on, and in order to do that I need to parse a group of SQL scripts, extract the SELECT, FROM, WHERE, whatever blocks out of it and do the necesary changes.
Is there a library that does that sort of thing? If not, could someone please point me to the right direction?
The application is in PHP, using CodeIgniter 2.1.0 and the DB is in MSSQL 2008
There's an older library for PEAR that should do exactly what you want: http://pear.php.net/package/SQL_Parser/.
Alternatively, you could just use regular expressions which wouldn't be as robust.
I've been figuring out how to let apostrophe's cross URI's.
I'm building a site that allows users to "create photo albums". I have a link that when clicked, it will load and display all the contents of a certain album. I'm using codeigniter so this page is called this way:
http://www.fourthdraft.com/index.php/admin/manageAlbumContents/dan's/91
admin = controller
managealbums = function
dan's (album name) = variable
As you know, codeigniter does not allow apostrophe(') in uri's. My problems are:
If I htmlspecialchars/htmlentities
the album name it becomes &#xx;
Those new characters also not
allowed
If I url encode it becomes %xx. percent is allowed but codeigniter
urldecodes it before processing so
it just reverts back to apostrophe
I've tried making my own preg_replace ( ' => '~apos~' ) but i
just find it inefficient, too much
lines to run and tedious since I
have an 80% done website and the
strings I have to replace are
everywhere.
I've also considered using base64_encode. It takes more space
but it does the job. Then again, the
encoded version contains '=' which
is also disallowed
As much as possible I do not want to just add apostrophe in the allowed characters list in codeigniter's config file. I believe they don't have it there for a reason. At the same time, I'm running out of options.
The reason for wanting to allow apostrophe's is because in this context, it's bound to be used. For example, what if someone decided to put 'dan's birthday party' as an album name? It's bound to happen. and i'm pretty sure my users would complain. Even if I manage to convince them otherwise, what will i replace that with? dan_s birthday party? looks wrong. Also, if facebook can do it I should too. At the very least, if facebook did it, then that means there's a way.
If you guys have any suggestions, fire away. Otherwise I'm wondering if it's ok (and safe) to just allow apostrophe in the allowed URI characters. I know it's VERY dangerous for mysql which i use a lot but I just remembered codeigniter's query binding variables automatically escapes characters. I'm wondering if that would suffice and keep me safe.
Otherwise, please please please give me a good idea. I'm drained out
I like to believe that the days of mysql_query("SELECT * FROM table WHERE x={$_GET['val']}") are over. That being said, it's OK with any decent database library as long as you use parameter binding. So go ahead and use urlencode.
I have an old drupal site that I'd like to upgrade, but I need to move all the site data files (like jpgs, gifs, etc.) from /files to /sites/default/files.
I'd like to use a PHP script or just a MySQL command to find any instance of /files/* and change it to /sites/default/files/* (without messing up the string in the * part of the name, of course!).
Is this pretty easy to do? Any pointers on a function I could use?
MySQL does have some built-in string replacement functions. How about something like this?
UPDATE table SET field = REPLACE(field,'/files/','/sites/default/files/');
There's other functions you can use for more complex replacements (ie. regular expressions) if you need as well.
I'm pretty sure that it's just a case of changing the 'files' path in the Drupal configuration.
If you're just changing the files table, you can do an UPDATE with SQL, like zombat said. If you have a significant number of other instances of the paths ( IE - full HTML node bodies and the like ) your best bet would be to export the DB to a text file (can do it with mysqldump or the export feature of PHPMyAdmin) and then just update the strings there - either with a suitable text editor, a command-line tool like sed or a bunch of interns.