A user can define the format of an identifier in my system, and this is stored in the d/b as a regex string (for example, "/^\d{6}$/", or a more complicated example of "/^[A-Z]{2}\d{8}$/").
Can anyone suggest how I can calculate the maximum length of the string that the given regex can match (thanks #Ulver)?
Many thanks for reading!
This answer assumes 5 things:
The expressions are simple, as per your examples.
You do not have * or + operators in your expression.
You do not have patterns of the type foo{n, }, where n is some positive, integer value.
Each expression starts with ^ and ends with $.
I am also assuming that each term is followed by the amount of times you expect to match it.
To calculate the amount of characters they match, you could go through the expression and look for 2 patterns:
{n}, which translates to match exactly n times. In this case, extract n.
{n, m}, which translates to match at least n times, and at most m times. In this case, extract m.
Once that you will have all the n and m values, you would simply add them together.
Some more details on the assumptions:
As expressions get more complicated, you will need to keep track of various characters. For instance, ^[A-Z]{2}$ means match 2 upper case letters. Thus, the length of what is matched will be 2. On the other hand, foo{2} means fooo. But afooo and foooobar will also be matched. Thus, you have no control over the lenght of the pattern. also (abc){2} means match abc twice, thus, in this case, you would need to multiply the value of n (the value in the braces) with the length of what ever lies within the brackets which precede it, if any. Of course, you could have nested values.
The * and + operator denote 0 or more, and 1 or more respectively. Thus, there is, theoretically, no limit on the length of whatever it is matched.
Similar to point 2, {n,} means match at least n times. Thus, there is no upper limit.
Similar to point 1, without the ^ and $ anchor, an expression can match any string. The expression foo can match afoo, foobar, foooooooooooooooooooooooo and so on.
I took this assumption for reasons similar to point 1. You could enhance your application to look for [] pairs and count them as 1 character, but I think you could have other caveats.
Related
I am using some data which gives paths for google maps either as a path or a set of two latitudes and longitudes. I have stored both values as a BLOB in a mySql database, but I need to detect the values which are not paths when they come out in the result. In an attempt to do this, I have saved them in the BLOB in the following format:
array(lat,lng+lat,lng)
I am using preg_match to find these results, but i havent managed to get any to work. Here are the regex codes I have tried:
^[a]{1}[r]{2}[a]{1}[y]{1}[\(]{1}[1-9\.\,\+]{1*}[\)]{1}^
^[a]{1}[r]{2}[a]{1}[y]{1}[\(]{1}(\-?\d+(\.\d+)?),(\-?\d+(\.\d+)?)\+(\-?\d+(\.\d+)?),(\-?\d+(\.\d+)?)[\)]{1}^
Regex confuses me sometimes (as it is doing now). Can anyone help me out?
Edit:
The lat can be 2 digits followed by a decimal point and 8 more digits and the lng can be 3 digits can be 3 digits follwed by a decimal point and 8 more digits. Both can be positive or negative.
Here are some example lat lngs:
51.51160000,-0.12766000
-53.36442000,132.27519000
51.50628000,0.12699000
-51.50628000,-0.12699000
So a full match would look like:
array(51.51160000,-0.12766000+-53.36442000,132.27519000)
Further Edit
I am using the preg_match() php function to match the regex.
Here are some pointers for writing regex:
If you have a single possibility for a character, for example, the a in array, you can indeed write it as [a]; however, you can also write it as just a.
If you are looking to match exactly one of something, you can indeed write it as a{1}, however, you can also write it as just a.
Applying this lots, your example of ^[a]{1}[r]{2}[a]{1}[y]{1}[\(]{1}[1-9\.\,\+]{1*}[\)]{1}^ reduces to ^array\([1-9\.\,\+]{1*}\)^ - that's certainly an improvement!
Next, numbers may also include 0's, as well as 1-9. In fact, \d - any digit - is usually used instead of 1-9.
You are using ^ as the delimiter - usually that is /; I didn't recognize it at first. I'm not sure what you can use for the delimiter, so, just in case, I'll change it to the usual /.This makes the above regex /array\([\d\.\,\+]{1*}\)/.
To match one or more of a character or character set, use +, rather than {1*}. This makes your query /array\([\d\.\,\+]+\)/
Then, to collect the resulting numbers (assuming you want only the part between the brackets, put it in (non-escaped) brackets, thus: /array\(([\d\.\,\+]+)\)/ - you would then need to split them, first by +, then by ,. Alternatively, if there are exactly two lat,lng pairs, you might want: /array\(([\d\.]+),([\d\.]+)\+([\d\.]+),([\d\.]+)\)/ - this will return 4 values, one for each number; the additional stuff (+, ,) will already be removed, because it is not in (unescaped) brackets ().
Edit: If you want negative lats and longs (and why wouldn't you?) you will need \-? (a "literal -", rather than part of a range) in the appropriate places; the ? makes it optional (i.e. 0 or 1 dashes). For example, /array\((\-?[\d\.]+),(\-?[\d\.]+)\+(\-?[\d\.]+),(\-?[\d\.]+)\)/
You might also want to check out http://regexpal.com - you can put in a regex and a set of strings, and it will highlight what matches/doesn't match. You will need to exclude the delimiter / or ^.
Note that this is a little fast and loose; it would also match array(5,0+0,1...........). You can nail it down a little more, for example, by using (\-?\d*\.\d+)\) instead of (\-?[\d\.]+)\) for the numbers; that will match (0 or 1 literal -) followed by (0 or more digits) followed by (exactly one literal dot) followed by (1 or more digits).
This is the regex I made:
array\((-*\d+\.\d+),(-*\d+\.\d+)\+(-*\d+\.\d+),(-*\d+\.\d+)\)
This also breaks the four numbers into groups so you can get the individual numbers.
You will note the repeated pattern of
(-*\d+\.\d+)
Explanation:
-* means 0 or more matches of the - sign ( so - sign is optional)
\d+ means 1 or more matches of a number
\. means a literal period (decimal)
\d+ means 1 or more matches of a number
The whole thing is wrapped in brackets to make it a captured group.
I have a really huge numberrange to check against lots of regular expressions.
In order to improve the number of comparisons, I want to remove those regexp, that are "smaller" than the starting number range.
I could not find any information wether a simple "<" smaller would work on recognizing if I compare it like strings.
ie:
if($regexp > $number)
array_push($sorted_regex, $regexp);
Regexp would include following special characters: . [abc] [a-b] * +
Numbers against to check are always given in a range as starting number and ending number.
So what I wanna do is just check those, that could possibly match any number in that range.
It's not possible to compare pattern strings and expect you compared potential pattern results. You may need to sit down and optimize order of your patterns manually, moving all these which match most often to the top of your list.
I'm making a date matching regex, and it's all going pretty well, I've got this so far:
"/(?:[0-3])?[0-9]-(?:[0-1])?[0-9]-(?:20)[0-1][0-9]/"
It will (hopefully) match single or double digit days and months, and double or quadruple digit years in the 21st century. A few trials and errors have gotten me this far.
But, I've got two simple questions regarding these results:
(?: ) what is a simple explanation for this? Apparently it's a non-matching group. But then...
What is the trailing ? for? e.g. (? )?
[Edited (again) to improve formatting and fix the intro.]
This is a comment and an answer.
The answer part... I do agree with alex' earlier answer.
(?: ), in contrast to ( ), is used to avoid capturing text, generally so as to have fewer back references thrown in with those you do want or to improve speed performance.
The ? following the (?: ) -- or when following anything except * + ? or {} -- means that the preceding item may or may not be found within a legitimate match. Eg, /z34?/ will match z3 as well as z34 but it won't match z35 or z etc.
The comment part... I made what might considered to be improvements to the regex you were working on:
(?:^|\s)(0?[1-9]|[1-2][0-9]|30|31)-(0?[1-9]|10|11|12)-((?:20)?[0-9][0-9])(?:\s|$)
-- First, it avoids things like 0-0-2011
-- Second, it avoids things like 233443-4-201154564
-- Third, it includes things like 1-1-2022
-- Forth, it includes things like 1-1-11
-- Fifth, it avoids things like 34-4-11
-- Sixth, it allows you to capture the day, month, and year so you can refer to these more easily in code.. code that would, for example, do a further check (is the second captured group 2 and is either the first captured group 29 and this a leap year or else the first captured group is <29) in order to see if a feb 29 date qualified or not.
Finally, note that you'll still get dates that won't exist, eg, 31-6-11. If you want to avoid these, then try:
(?:^|\s)(?:(?:(0?[1-9]|[1-2][0-9]|30|31)-(0?[13578]|10|12))|(?:(0?[1-9]|[1-2][0-9]|30)-(0?[469]|11))|(?:(0?[1-9]|[1-2][0-9])-(0?2)))-((?:20)?[0-9][0-9])(?:\s|$)
Also, I assumed the dates would be preceded and followed by a space (or beg/end of line), but you may want ot adjust that (eg, to allow punctuations).
A commenter elsewhere referenced this resource which you might find useful:
http://rubular.com/
It is a non capturing group. You can not back reference it. Usually used to declutter backreferences and/or increase performance.
It means the previous capturing group is optional.
Subpatterns
Subpatterns are delimited by parentheses (round brackets), which can be nested. Marking part of a pattern as a subpattern does two things:
It localizes a set of alternatives. For example, the pattern
cat(aract|erpillar|) matches one of the words "cat", "cataract", or
"caterpillar". Without the parentheses, it would match "cataract",
"erpillar" or the empty string.
It sets up the subpattern as a capturing subpattern (as defined
above). When the whole pattern matches, that portion of the subject
string that matched the subpattern is passed back to the caller via
the ovector argument of pcre_exec(). Opening parentheses are counted
from left to right (starting from 1) to obtain the numbers of the
capturing subpatterns.
For example, if the string "the red king" is matched against the pattern the ((red|white) (king|queen)) the captured substrings are "red king", "red", and "king", and are numbered 1, 2, and 3.
The fact that plain parentheses fulfill two functions is not always helpful. There are often times when a grouping subpattern is required without a capturing requirement. If an opening parenthesis is followed by "?:", the subpattern does not do any capturing, and is not counted when computing the number of any subsequent capturing subpatterns. For example, if the string "the white queen" is matched against the pattern the ((?:red|white) (king|queen)) the captured substrings are "white queen" and "queen", and are numbered 1 and 2. The maximum number of captured substrings is 65535. It may not be possible to compile such large patterns, however, depending on the configuration options of libpcre.
As a convenient shorthand, if any option settings are required at the start of a non-capturing subpattern, the option letters may appear between the "?" and the ":". Thus the two patterns
(?i:saturday|sunday)
(?:(?i)saturday|sunday)
match exactly the same set of strings. Because alternative branches are tried from left to right, and options are not reset until the end of the subpattern is reached, an option setting in one branch does affect subsequent branches, so the above patterns match "SUNDAY" as well as "Saturday".
It is possible to name a subpattern using the syntax (?Ppattern). This subpattern will then be indexed in the matches array by its normal numeric position and also by name. PHP 5.2.2 introduced two alternative syntaxes (?pattern) and (?'name'pattern).
Sometimes it is necessary to have multiple matching, but alternating subgroups in a regular expression. Normally, each of these would be given their own backreference number even though only one of them would ever possibly match. To overcome this, the (?| syntax allows having duplicate numbers. Consider the following regex matched against the string Sunday:
(?:(Sat)ur|(Sun))day
Here Sun is stored in backreference 2, while backreference 1 is empty. Matching yields Sat in backreference 1 while backreference 2 does not exist. Changing the pattern to use the (?| fixes this problem:
(?|(Sat)ur|(Sun))day
Using this pattern, both Sun and Sat would be stored in backreference 1.
Reference : http://php.net/manual/en/regexp.reference.subpatterns.php
I'm trying to devise a regex pattern (in PHP) which will allow for any alternation of two subpatterns. So if pattern A matches a group of three letters, and B matches a group of 2 numerals, all of these would be OK:
aaa
aaa66bbb
66
67abc
12abc34def56ghi78jkl
I don't mind which subpattern starts or ends the sequence, just that after the first match, the subpatterns must alternate. I'm totally stumped by this - any advice will be gratefully received!
Here's a general solution:
^(?:[a-z]{3}(?![a-z]{3})|[0-9]{2}(?![0-9]{2}))+$
It's a simple alternation--three letters or two digits--but the negative lookaheads ensure that the same alternative is never matched twice in a row. Here's a slightly more elegant solution just for PHP:
/^(?:([a-z]{3})(?!(?1))|([0-9]{2})(?!(?2)))+$/
Instead of typing the same subpatterns multiple times, you can put them capturing groups and use (?1), (?2), etc. to apply them again wherever else you want--in this case, in the lookaheads.
"/^(?:$A(?:$B$A)*$B?|$B(?:$A$B)*$A?)\$/"
will match either pattern A followed by however many alternating pattern B's and pattern A's, and maybe a final B...or a B followed by however many A-B pairs plus an A if it's there.
I've made this a string (and escaped the final $) cause you're going to have some interpolation to do. Make sure $A and $B are in some kind of grouping (like parentheses) if you want the ?'s to match the right thing. In your examples, $A might be '([a-zA-Z]{3})' and $B might be '(\d\d)'.
Note, if you want to match some number of the same letter or digit, or instances of the same set of letters or digits, you'll need to do some magic with backreferences -- probably named ones, since any numbered backreference will depend on the number of capture groups before the one you want (or between the one you want and where you are), but that number gets complicated if the subpatterns have parentheses in them.
Take a look at this (and check conditional subpatterns). I've personally never used them but seems to be what you're looking for.
/\b(?:(([a-z])\2\2)(?:(([0-9])\4)\1)*(?:([0-9])\5)?|(([0-9])\7)(?:(([a-z])\9\9)\6)*(?:([a-z])\10\10)?)\b/
or if you want to allow any non digit char in the group of three:
/\b(?:((\D)\2\2)(?:((\d)\4)\1)*(?:(\d)\5)?|((\d)\7)(?:((\D)\9\9)\6)*(?:(\D)\10\10)?)\b/
This will match any pattern that consists of two alternating groups one group consists of 3 times the same char and the other of 2 times the same digit.
This Regex will match
aaa
11
bbb22
33ccc
ddd44ddd
55eee55
fff66fff66
77ggg77ggg
But not
aaa11bbb
Im looking for function (PHP will be the best), which returns true whether exists string matches both regexpA and regexpB.
Example 1:
$regexpA = '[0-9]+';
$regexpB = '[0-9]{2,3}';
hasRegularsIntersection($regexpA,$regexpB) returns TRUE because '12' matches both regexps
Example 2:
$regexpA = '[0-9]+';
$regexpB = '[a-z]+';
hasRegularsIntersection($regexpA,$regexpB) returns FALSE because numbers never matches literals.
Thanks for any suggestions how to solve this.
Henry
For regular expressions that are actually regular (i.e. don't use irregular features like back references) you can do the following:
Transform the regexen into finite automata (the algorithm for that can be found here(chapter 9) for example).
Build the intersection of the automata (You have a state for each state in the cartesian product of the states of the two automata. You then transition between the states according to the original automata's transition rules. E.g. if you're in state x1y2, you get the input a, the first automaton has a transition x1->x4 for input x and the second automaton has y2->y3, you transition into the state x4y3).
Check whether there's a path from the start state to the end state in the new automaton. If there is, the two regexen intersect, otherwise they don't.
Theory.
Java library.
Usage:
/**
* #return true if the two regexes will never both match a given string
*/
public boolean isRegexOrthogonal( String regex1, String regex2 ) {
Automaton automaton1 = new RegExp(regex1).toAutomaton();
Automaton automaton2 = new RegExp(regex2).toAutomaton();
return automaton1.intersection(automaton2).isEmpty();
}
A regular expression specifies a finite state machine that can recognize a potentially infinite set of strings. The set of strings may be infinite but the number of states must be finite, so you can examine the states one by one.
Taking your second example: In the first expression, to get from state 0 to state 1, the string must start with a digit. In the second expression, to get from state 0 to state 1, the string must start with a letter. So you know that there is no string that will get you from state 0 to state 1 in BOTH expressions. You have the answer.
Taking the first example: You know that if the string starts with a digit you can get from state 0 to state 1 with either regular expression. So now you can eliminate state 0 for each, and just answer the question for each of the two (now one state smaller) finite-state-machines.
This looks a lot like the well-known "halting problem", which as you know is unsolvable in the general case for a Turing machine or equivalent. But in fact the halting problem IS solvable for a finite-state machine, simply because there are a finite number of states.
I believe you could solve this with a non-deterministic FSM. If your regex had only one transition from each state to the next, a deterministic FSM could solve it. But a regular expression means that for instance if you are in state 2, then if the caracter is a digit you go to state 3, else if the character is a letter you go to state 4.
So here's what I would do:
Solve it for the subset of FSM's that have only one transition from one state to the next. For instance a regex that matches both "Bob" and "bob", and a second regex that matches only "bob" and "boB".
See if you can implement the solution in a finite state machine. I think this should be possible. The input to a state is a pair representing a transition for one FSM and a transition for the second one. For instance: State 0: If (r1, r2) is (("B" or "b"), "b") then State 1. State 1: If (r1, r2) is (("o"), ("o")) then state 2. etc.
Now for the more general case, where for instance state two goes back to state two or an earlier state; for example, regex 1 recognizes only "meet" but regex 2 recognizes "meeeet" with an unlimited number of e's. You would have to reduce them to regex 1 recognizing "t" and regex 2 recognizing "t". I think this may be solvable by a non-deterministic FSM.
That's my intuition anyway.
I don't think it's NP-complete, just because my intuition tells me you should be able to shorten each regex by one state with each step.
It is possible. I encountered it once with Pellet OWL reasoner while learning semantic web technologies.
Here is an example that shows how regular expressions can be parsed into a tree structure. You could then (in theory) parse your two regular expressions to trees and see if one tree is a subset of the other tree, ie. if one tree can be found in within other tree's nodes.
If it is found, then the other regular expression will match (not only, but also) a subset of what the first regular expression will match.
It is not much of a solution, but maybe it'll help you.