I've been looking for a simple regex for URLs, does anybody have one handy that works well? I didn't find one with the zend framework validation classes and have seen several implementations.
Use the filter_var() function to validate whether a string is URL or not:
var_dump(filter_var('example.com', FILTER_VALIDATE_URL));
It is bad practice to use regular expressions when not necessary.
EDIT: Be careful, this solution is not unicode-safe and not XSS-safe. If you need a complex validation, maybe it's better to look somewhere else.
I used this on a few projects, I don't believe I've run into issues, but I'm sure it's not exhaustive:
$text = preg_replace(
'#((https?|ftp)://(\S*?\.\S*?))([\s)\[\]{},;"\':<]|\.\s|$)#i',
"'$3$4'",
$text
);
Most of the random junk at the end is to deal with situations like http://domain.example. in a sentence (to avoid matching the trailing period). I'm sure it could be cleaned up but since it worked. I've more or less just copied it over from project to project.
As per the PHP manual - parse_url should not be used to validate a URL.
Unfortunately, it seems that filter_var('example.com', FILTER_VALIDATE_URL) does not perform any better.
Both parse_url() and filter_var() will pass malformed URLs such as http://...
Therefore in this case - regex is the better method.
As per John Gruber (Daring Fireball):
Regex:
(?i)\b((?:https?://|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'\".,<>?«»“”‘’]))
using in preg_match():
preg_match("/(?i)\b((?:https?://|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'\".,<>?«»“”‘’]))/", $url)
Here is the extended regex pattern (with comments):
(?xi)
\b
( # Capture 1: entire matched URL
(?:
https?:// # http or https protocol
| # or
www\d{0,3}[.] # "www.", "www1.", "www2." … "www999."
| # or
[a-z0-9.\-]+[.][a-z]{2,4}/ # looks like domain name followed by a slash
)
(?: # One or more:
[^\s()<>]+ # Run of non-space, non-()<>
| # or
\(([^\s()<>]+|(\([^\s()<>]+\)))*\) # balanced parens, up to 2 levels
)+
(?: # End with:
\(([^\s()<>]+|(\([^\s()<>]+\)))*\) # balanced parens, up to 2 levels
| # or
[^\s`!()\[\]{};:'".,<>?«»“”‘’] # not a space or one of these punct chars
)
)
For more details please look at:
http://daringfireball.net/2010/07/improved_regex_for_matching_urls
Just in case you want to know if the url really exists:
function url_exist($url){//se passar a URL existe
$c=curl_init();
curl_setopt($c,CURLOPT_URL,$url);
curl_setopt($c,CURLOPT_HEADER,1);//get the header
curl_setopt($c,CURLOPT_NOBODY,1);//and *only* get the header
curl_setopt($c,CURLOPT_RETURNTRANSFER,1);//get the response as a string from curl_exec(), rather than echoing it
curl_setopt($c,CURLOPT_FRESH_CONNECT,1);//don't use a cached version of the url
if(!curl_exec($c)){
//echo $url.' inexists';
return false;
}else{
//echo $url.' exists';
return true;
}
//$httpcode=curl_getinfo($c,CURLINFO_HTTP_CODE);
//return ($httpcode<400);
}
I don't think that using regular expressions is a smart thing to do in this case. It is impossible to match all of the possibilities and even if you did, there is still a chance that url simply doesn't exist.
Here is a very simple way to test if url actually exists and is readable :
if (preg_match("#^https?://.+#", $link) and #fopen($link,"r")) echo "OK";
(if there is no preg_match then this would also validate all filenames on your server)
I've used this one with good success - I don't remember where I got it from
$pattern = "/\b(?:(?:https?|ftp):\/\/|www\.)[-a-z0-9+&##\/%?=~_|!:,.;]*[-a-z0-9+&##\/%=~_|]/i";
The best URL Regex that worked for me:
function valid_URL($url){
return preg_match('%^(?:(?:https?|ftp)://)(?:\S+(?::\S*)?#|\d{1,3}(?:\.\d{1,3}){3}|(?:(?:[a-z\d\x{00a1}-\x{ffff}]+-?)*[a-z\d\x{00a1}-\x{ffff}]+)(?:\.(?:[a-z\d\x{00a1}-\x{ffff}]+-?)*[a-z\d\x{00a1}-\x{ffff}]+)*(?:\.[a-z\x{00a1}-\x{ffff}]{2,6}))(?::\d+)?(?:[^\s]*)?$%iu', $url);
}
Examples:
valid_URL('https://twitter.com'); // true
valid_URL('http://twitter.com'); // true
valid_URL('http://twitter.co'); // true
valid_URL('http://t.co'); // true
valid_URL('http://twitter.c'); // false
valid_URL('htt://twitter.com'); // false
valid_URL('http://example.com/?a=1&b=2&c=3'); // true
valid_URL('http://127.0.0.1'); // true
valid_URL(''); // false
valid_URL(1); // false
Source: http://urlregex.com/
function validateURL($URL) {
$pattern_1 = "/^(http|https|ftp):\/\/(([A-Z0-9][A-Z0-9_-]*)(\.[A-Z0-9][A-Z0-9_-]*)+.(com|org|net|dk|at|us|tv|info|uk|co.uk|biz|se)$)(:(\d+))?\/?/i";
$pattern_2 = "/^(www)((\.[A-Z0-9][A-Z0-9_-]*)+.(com|org|net|dk|at|us|tv|info|uk|co.uk|biz|se)$)(:(\d+))?\/?/i";
if(preg_match($pattern_1, $URL) || preg_match($pattern_2, $URL)){
return true;
} else{
return false;
}
}
Edit:
As incidence pointed out this code has been DEPRECATED with the release of PHP 5.3.0 (2009-06-30) and should be used accordingly.
Just my two cents but I've developed this function and have been using it for a while with success. It's well documented and separated so you can easily change it.
// Checks if string is a URL
// #param string $url
// #return bool
function isURL($url = NULL) {
if($url==NULL) return false;
$protocol = '(http://|https://)';
$allowed = '([a-z0-9]([-a-z0-9]*[a-z0-9]+)?)';
$regex = "^". $protocol . // must include the protocol
'(' . $allowed . '{1,63}\.)+'. // 1 or several sub domains with a max of 63 chars
'[a-z]' . '{2,6}'; // followed by a TLD
if(eregi($regex, $url)==true) return true;
else return false;
}
And there is your answer =) Try to break it, you can't!!!
function link_validate_url($text) {
$LINK_DOMAINS = 'aero|arpa|asia|biz|com|cat|coop|edu|gov|info|int|jobs|mil|museum|name|nato|net|org|pro|travel|mobi|local';
$LINK_ICHARS_DOMAIN = (string) html_entity_decode(implode("", array( // #TODO completing letters ...
"æ", // æ
"Æ", // Æ
"À", // À
"à", // à
"Á", // Á
"á", // á
"Â", // Â
"â", // â
"å", // å
"Å", // Å
"ä", // ä
"Ä", // Ä
"Ç", // Ç
"ç", // ç
"Ð", // Ð
"ð", // ð
"È", // È
"è", // è
"É", // É
"é", // é
"Ê", // Ê
"ê", // ê
"Ë", // Ë
"ë", // ë
"Î", // Î
"î", // î
"Ï", // Ï
"ï", // ï
"ø", // ø
"Ø", // Ø
"ö", // ö
"Ö", // Ö
"Ô", // Ô
"ô", // ô
"Õ", // Õ
"õ", // õ
"Œ", // Œ
"œ", // œ
"ü", // ü
"Ü", // Ü
"Ù", // Ù
"ù", // ù
"Û", // Û
"û", // û
"Ÿ", // Ÿ
"ÿ", // ÿ
"Ñ", // Ñ
"ñ", // ñ
"þ", // þ
"Þ", // Þ
"ý", // ý
"Ý", // Ý
"¿", // ¿
)), ENT_QUOTES, 'UTF-8');
$LINK_ICHARS = $LINK_ICHARS_DOMAIN . (string) html_entity_decode(implode("", array(
"ß", // ß
)), ENT_QUOTES, 'UTF-8');
$allowed_protocols = array('http', 'https', 'ftp', 'news', 'nntp', 'telnet', 'mailto', 'irc', 'ssh', 'sftp', 'webcal');
// Starting a parenthesis group with (?: means that it is grouped, but is not captured
$protocol = '((?:'. implode("|", $allowed_protocols) .'):\/\/)';
$authentication = "(?:(?:(?:[\w\.\-\+!$&'\(\)*\+,;=" . $LINK_ICHARS . "]|%[0-9a-f]{2})+(?::(?:[\w". $LINK_ICHARS ."\.\-\+%!$&'\(\)*\+,;=]|%[0-9a-f]{2})*)?)?#)";
$domain = '(?:(?:[a-z0-9' . $LINK_ICHARS_DOMAIN . ']([a-z0-9'. $LINK_ICHARS_DOMAIN . '\-_\[\]])*)(\.(([a-z0-9' . $LINK_ICHARS_DOMAIN . '\-_\[\]])+\.)*('. $LINK_DOMAINS .'|[a-z]{2}))?)';
$ipv4 = '(?:[0-9]{1,3}(\.[0-9]{1,3}){3})';
$ipv6 = '(?:[0-9a-fA-F]{1,4}(\:[0-9a-fA-F]{1,4}){7})';
$port = '(?::([0-9]{1,5}))';
// Pattern specific to external links.
$external_pattern = '/^'. $protocol .'?'. $authentication .'?('. $domain .'|'. $ipv4 .'|'. $ipv6 .' |localhost)'. $port .'?';
// Pattern specific to internal links.
$internal_pattern = "/^(?:[a-z0-9". $LINK_ICHARS ."_\-+\[\]]+)";
$internal_pattern_file = "/^(?:[a-z0-9". $LINK_ICHARS ."_\-+\[\]\.]+)$/i";
$directories = "(?:\/[a-z0-9". $LINK_ICHARS ."_\-\.~+%=&,$'#!():;*#\[\]]*)*";
// Yes, four backslashes == a single backslash.
$query = "(?:\/?\?([?a-z0-9". $LINK_ICHARS ."+_|\-\.~\/\\\\%=&,$'():;*#\[\]{} ]*))";
$anchor = "(?:#[a-z0-9". $LINK_ICHARS ."_\-\.~+%=&,$'():;*#\[\]\/\?]*)";
// The rest of the path for a standard URL.
$end = $directories .'?'. $query .'?'. $anchor .'?'.'$/i';
$message_id = '[^#].*#'. $domain;
$newsgroup_name = '(?:[0-9a-z+-]*\.)*[0-9a-z+-]*';
$news_pattern = '/^news:('. $newsgroup_name .'|'. $message_id .')$/i';
$user = '[a-zA-Z0-9'. $LINK_ICHARS .'_\-\.\+\^!#\$%&*+\/\=\?\`\|\{\}~\'\[\]]+';
$email_pattern = '/^mailto:'. $user .'#'.'(?:'. $domain .'|'. $ipv4 .'|'. $ipv6 .'|localhost)'. $query .'?$/';
if (strpos($text, '<front>') === 0) {
return false;
}
if (in_array('mailto', $allowed_protocols) && preg_match($email_pattern, $text)) {
return false;
}
if (in_array('news', $allowed_protocols) && preg_match($news_pattern, $text)) {
return false;
}
if (preg_match($internal_pattern . $end, $text)) {
return false;
}
if (preg_match($external_pattern . $end, $text)) {
return false;
}
if (preg_match($internal_pattern_file, $text)) {
return false;
}
return true;
}
function is_valid_url ($url="") {
if ($url=="") {
$url=$this->url;
}
$url = #parse_url($url);
if ( ! $url) {
return false;
}
$url = array_map('trim', $url);
$url['port'] = (!isset($url['port'])) ? 80 : (int)$url['port'];
$path = (isset($url['path'])) ? $url['path'] : '';
if ($path == '') {
$path = '/';
}
$path .= ( isset ( $url['query'] ) ) ? "?$url[query]" : '';
if ( isset ( $url['host'] ) AND $url['host'] != gethostbyname ( $url['host'] ) ) {
if ( PHP_VERSION >= 5 ) {
$headers = get_headers("$url[scheme]://$url[host]:$url[port]$path");
}
else {
$fp = fsockopen($url['host'], $url['port'], $errno, $errstr, 30);
if ( ! $fp ) {
return false;
}
fputs($fp, "HEAD $path HTTP/1.1\r\nHost: $url[host]\r\n\r\n");
$headers = fread ( $fp, 128 );
fclose ( $fp );
}
$headers = ( is_array ( $headers ) ) ? implode ( "\n", $headers ) : $headers;
return ( bool ) preg_match ( '#^HTTP/.*\s+[(200|301|302)]+\s#i', $headers );
}
return false;
}
Inspired in this .NET StackOverflow question and in this referenced article from that question there is this URI validator (URI means it validates both URL and URN).
if( ! preg_match( "/^([a-z][a-z0-9+.-]*):(?:\\/\\/((?:(?=((?:[a-z0-9-._~!$&'()*+,;=:]|%[0-9A-F]{2})*))(\\3)#)?(?=(\\[[0-9A-F:.]{2,}\\]|(?:[a-z0-9-._~!$&'()*+,;=]|%[0-9A-F]{2})*))\\5(?::(?=(\\d*))\\6)?)(\\/(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\8)?|(\\/?(?!\\/)(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\10)?)(?:\\?(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\11)?(?:#(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\12)?$/i", $uri ) )
{
throw new \RuntimeException( "URI has not a valid format." );
}
I have successfully unit-tested this function inside a ValueObject I made named Uri and tested by UriTest.
UriTest.php (Contains valid and invalid cases for both URLs and URNs)
<?php
declare( strict_types = 1 );
namespace XaviMontero\ThrasherPortage\Tests\Tour;
use XaviMontero\ThrasherPortage\Tour\Uri;
class UriTest extends \PHPUnit_Framework_TestCase
{
private $sut;
public function testCreationIsOfProperClassWhenUriIsValid()
{
$sut = new Uri( 'http://example.com' );
$this->assertInstanceOf( 'XaviMontero\\ThrasherPortage\\Tour\\Uri', $sut );
}
/**
* #dataProvider urlIsValidProvider
* #dataProvider urnIsValidProvider
*/
public function testGetUriAsStringWhenUriIsValid( string $uri )
{
$sut = new Uri( $uri );
$actual = $sut->getUriAsString();
$this->assertInternalType( 'string', $actual );
$this->assertEquals( $uri, $actual );
}
public function urlIsValidProvider()
{
return
[
[ 'http://example-server' ],
[ 'http://example.com' ],
[ 'http://example.com/' ],
[ 'http://subdomain.example.com/path/?parameter1=value1¶meter2=value2' ],
[ 'random-protocol://example.com' ],
[ 'http://example.com:80' ],
[ 'http://example.com?no-path-separator' ],
[ 'http://example.com/pa%20th/' ],
[ 'ftp://example.org/resource.txt' ],
[ 'file://../../../relative/path/needs/protocol/resource.txt' ],
[ 'http://example.com/#one-fragment' ],
[ 'http://example.edu:8080#one-fragment' ],
];
}
public function urnIsValidProvider()
{
return
[
[ 'urn:isbn:0-486-27557-4' ],
[ 'urn:example:mammal:monotreme:echidna' ],
[ 'urn:mpeg:mpeg7:schema:2001' ],
[ 'urn:uuid:6e8bc430-9c3a-11d9-9669-0800200c9a66' ],
[ 'rare-urn:uuid:6e8bc430-9c3a-11d9-9669-0800200c9a66' ],
[ 'urn:FOO:a123,456' ]
];
}
/**
* #dataProvider urlIsNotValidProvider
* #dataProvider urnIsNotValidProvider
*/
public function testCreationThrowsExceptionWhenUriIsNotValid( string $uri )
{
$this->expectException( 'RuntimeException' );
$this->sut = new Uri( $uri );
}
public function urlIsNotValidProvider()
{
return
[
[ 'only-text' ],
[ 'http//missing.colon.example.com/path/?parameter1=value1¶meter2=value2' ],
[ 'missing.protocol.example.com/path/' ],
[ 'http://example.com\\bad-separator' ],
[ 'http://example.com|bad-separator' ],
[ 'ht tp://example.com' ],
[ 'http://exampl e.com' ],
[ 'http://example.com/pa th/' ],
[ '../../../relative/path/needs/protocol/resource.txt' ],
[ 'http://example.com/#two-fragments#not-allowed' ],
[ 'http://example.edu:portMustBeANumber#one-fragment' ],
];
}
public function urnIsNotValidProvider()
{
return
[
[ 'urn:mpeg:mpeg7:sch ema:2001' ],
[ 'urn|mpeg:mpeg7:schema:2001' ],
[ 'urn?mpeg:mpeg7:schema:2001' ],
[ 'urn%mpeg:mpeg7:schema:2001' ],
[ 'urn#mpeg:mpeg7:schema:2001' ],
];
}
}
Uri.php (Value Object)
<?php
declare( strict_types = 1 );
namespace XaviMontero\ThrasherPortage\Tour;
class Uri
{
/** #var string */
private $uri;
public function __construct( string $uri )
{
$this->assertUriIsCorrect( $uri );
$this->uri = $uri;
}
public function getUriAsString()
{
return $this->uri;
}
private function assertUriIsCorrect( string $uri )
{
// https://stackoverflow.com/questions/30847/regex-to-validate-uris
// http://snipplr.com/view/6889/regular-expressions-for-uri-validationparsing/
if( ! preg_match( "/^([a-z][a-z0-9+.-]*):(?:\\/\\/((?:(?=((?:[a-z0-9-._~!$&'()*+,;=:]|%[0-9A-F]{2})*))(\\3)#)?(?=(\\[[0-9A-F:.]{2,}\\]|(?:[a-z0-9-._~!$&'()*+,;=]|%[0-9A-F]{2})*))\\5(?::(?=(\\d*))\\6)?)(\\/(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\8)?|(\\/?(?!\\/)(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\10)?)(?:\\?(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\11)?(?:#(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\12)?$/i", $uri ) )
{
throw new \RuntimeException( "URI has not a valid format." );
}
}
}
Running UnitTests
There are 65 assertions in 46 tests. Caution: there are 2 data-providers for valid and 2 more for invalid expressions. One is for URLs and the other for URNs. If you are using a version of PhpUnit of v5.6* or earlier then you need to join the two data providers into a single one.
xavi#bromo:~/custom_www/hello-trip/mutant-migrant$ vendor/bin/phpunit
PHPUnit 5.7.3 by Sebastian Bergmann and contributors.
.............................................. 46 / 46 (100%)
Time: 82 ms, Memory: 4.00MB
OK (46 tests, 65 assertions)
Code coverage
There's is 100% of code-coverage in this sample URI checker.
"/(http(s?):\/\/)([a-z0-9\-]+\.)+[a-z]{2,4}(\.[a-z]{2,4})*(\/[^ ]+)*/i"
(http(s?)://) means http:// or https://
([a-z0-9-]+.)+ =>
2.0[a-z0-9-] means any a-z character or any 0-9 or (-)sign)
2.1 (+) means the character can be one or more ex: a1w,
a9-,c559s, f)
2.2 \. is (.)sign
2.3. the (+) sign after ([a-z0-9\-]+\.) mean do 2.1,2.2,2.3
at least 1 time
ex: abc.defgh0.ig, aa.b.ced.f.gh. also in case www.yyy.com
3.[a-z]{2,4} mean a-z at least 2 character but not more than
4 characters for check that there will not be
the case
ex: https://www.google.co.kr.asdsdagfsdfsf
4.(\.[a-z]{2,4})*(\/[^ ]+)* mean
4.1 \.[a-z]{2,4} means like number 3 but start with
(.)sign
4.2 * means (\.[a-z]{2,4})can be use or not use never mind
4.3 \/ means \
4.4 [^ ] means any character except blank
4.5 (+) means do 4.3,4.4,4.5 at least 1 times
4.6 (*) after (\/[^ ]+) mean use 4.3 - 4.5 or not use
no problem
use for case https://stackoverflow.com/posts/51441301/edit
5. when you use regex write in "/ /" so it come
"/(http(s?)://)([a-z0-9-]+.)+[a-z]{2,4}(.[a-z]{2,4})(/[^ ]+)/i"
6. almost forgot: letter i on the back mean ignore case of
Big letter or small letter ex: A same as a, SoRRy same
as sorry.
Note : Sorry for bad English. My country not use it well.
OK, so this is a little bit more complex then a simple regex, but it allows for different types of urls.
Examples:
google.com
www.microsoft.com/
http://www.yahoo.com/
https://www.bandcamp.com/artist/#!someone-special!
All which should be marked as valid.
function is_valid_url($url) {
// First check: is the url just a domain name? (allow a slash at the end)
$_domain_regex = "|^[A-Za-z0-9-]+(\.[A-Za-z0-9-]+)*(\.[A-Za-z]{2,})/?$|";
if (preg_match($_domain_regex, $url)) {
return true;
}
// Second: Check if it's a url with a scheme and all
$_regex = '#^([a-z][\w-]+:(?:/{1,3}|[a-z0-9%])|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))$#';
if (preg_match($_regex, $url, $matches)) {
// pull out the domain name, and make sure that the domain is valid.
$_parts = parse_url($url);
if (!in_array($_parts['scheme'], array( 'http', 'https' )))
return false;
// Check the domain using the regex, stops domains like "-example.com" passing through
if (!preg_match($_domain_regex, $_parts['host']))
return false;
// This domain looks pretty valid. Only way to check it now is to download it!
return true;
}
return false;
}
Note that there is a in_array check for the protocols that you want to allow (currently only http and https are in that list).
var_dump(is_valid_url('google.com')); // true
var_dump(is_valid_url('google.com/')); // true
var_dump(is_valid_url('http://google.com')); // true
var_dump(is_valid_url('http://google.com/')); // true
var_dump(is_valid_url('https://google.com')); // true
For anyone developing with WordPress, just use
esc_url_raw($url) === $url
to validate a URL (here's WordPress' documentation on esc_url_raw). It handles URLs much better than filter_var($url, FILTER_VALIDATE_URL) because it is unicode and XSS-safe. (Here is a good article mentioning all the problems with filter_var).
Peter's Regex doesn't look right to me for many reasons. It allows all kinds of special characters in the domain name and doesn't test for much.
Frankie's function looks good to me and you can build a good regex from the components if you don't want a function, like so:
^(http://|https://)(([a-z0-9]([-a-z0-9]*[a-z0-9]+)?){1,63}\.)+[a-z]{2,6}
Untested but I think that should work.
Also, Owen's answer doesn't look 100% either. I took the domain part of the regex and tested it on a Regex tester tool http://erik.eae.net/playground/regexp/regexp.html
I put the following line:
(\S*?\.\S*?)
in the "regexp" section
and the following line:
-hello.com
under the "sample text" section.
The result allowed the minus character through. Because \S means any non-space character.
Note the regex from Frankie handles the minus because it has this part for the first character:
[a-z0-9]
Which won't allow the minus or any other special character.
Here is the way I did it. But I want to mentoin that I am not so shure about the regex. But It should work thou :)
$pattern = "#((http|https)://(\S*?\.\S*?))(\s|\;|\)|\]|\[|\{|\}|,|”|\"|'|:|\<|$|\.\s)#i";
$text = preg_replace_callback($pattern,function($m){
return "$m[1]$m[4]";
},
$text);
This way you won't need the eval marker on your pattern.
Hope it helps :)
Here's a simple class for URL Validation using RegEx and then cross-references the domain against popular RBL (Realtime Blackhole Lists) servers:
Install:
require 'URLValidation.php';
Usage:
require 'URLValidation.php';
$urlVal = new UrlValidation(); //Create Object Instance
Add a URL as the parameter of the domain() method and check the the return.
$urlArray = ['http://www.bokranzr.com/test.php?test=foo&test=dfdf', 'https://en-gb.facebook.com', 'https://www.google.com'];
foreach ($urlArray as $k=>$v) {
echo var_dump($urlVal->domain($v)) . ' URL: ' . $v . '<br>';
}
Output:
bool(false) URL: http://www.bokranzr.com/test.php?test=foo&test=dfdf
bool(true) URL: https://en-gb.facebook.com
bool(true) URL: https://www.google.com
As you can see above, www.bokranzr.com is listed as malicious website via an RBL so the domain was returned as false.
I've found this to be the most useful for matching a URL..
^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-]*)*\/?$
There is a PHP native function for that:
$url = 'http://www.yoururl.co.uk/sub1/sub2/?param=1¶m2/';
if ( ! filter_var( $url, FILTER_VALIDATE_URL ) ) {
// Wrong
}
else {
// Valid
}
Returns the filtered data, or FALSE if the filter fails.
Check it here
Related
Using PHP5 (cgi) to output template files from the filesystem and having issues spitting out raw HTML.
private function fetch($name) {
$path = $this->j->config['template_path'] . $name . '.html';
if (!file_exists($path)) {
dbgerror('Could not find the template "' . $name . '" in ' . $path);
}
$f = fopen($path, 'r');
$t = fread($f, filesize($path));
fclose($f);
if (substr($t, 0, 3) == b'\xef\xbb\xbf') {
$t = substr($t, 3);
}
return $t;
}
Even though I've added the BOM fix I'm still having problems with Firefox accepting it. You can see a live copy here: http://ircb.in/jisti/ (and the template file I threw at http://ircb.in/jisti/home.html if you want to check it out)
Any idea how to fix this? o_o
you would use the following code to remove utf8 bom
//Remove UTF8 Bom
function remove_utf8_bom($text)
{
$bom = pack('H*','EFBBBF');
$text = preg_replace("/^$bom/", '', $text);
return $text;
}
try:
// -------- read the file-content ----
$str = file_get_contents($source_file);
// -------- remove the utf-8 BOM ----
$str = str_replace("\xEF\xBB\xBF",'',$str);
// -------- get the Object from JSON ----
$obj = json_decode($str);
:)
Another way to remove the BOM which is Unicode code point U+FEFF
$str = preg_replace('/\x{FEFF}/u', '', $file);
b'\xef\xbb\xbf' stands for the literal string "\xef\xbb\xbf". If you want to check for a BOM, you need to use double quotes, so the \x sequences are actually interpreted into bytes:
"\xef\xbb\xbf"
Your files also seem to contain a lot more garbage than just a single leading BOM:
$ curl http://ircb.in/jisti/ | xxd
0000000: efbb bfef bbbf efbb bfef bbbf efbb bfef ................
0000010: bbbf efbb bf3c 2144 4f43 5459 5045 2068 .....<!DOCTYPE h
0000020: 746d 6c3e 0a3c 6874 6d6c 3e0a 3c68 6561 tml>.<html>.<hea
...
if anybody using csv import then below code useful
$header = fgetcsv($handle);
foreach($header as $key=> $val) {
$bom = pack('H*','EFBBBF');
$val = preg_replace("/^$bom/", '', $val);
$header[$key] = $val;
}
This global funtion resolve for UTF-8 system base charset. Tanks!
function prepareCharset($str) {
// set default encode
mb_internal_encoding('UTF-8');
// pre filter
if (empty($str)) {
return $str;
}
// get charset
$charset = mb_detect_encoding($str, array('ISO-8859-1', 'UTF-8', 'ASCII'));
if (stristr($charset, 'utf') || stristr($charset, 'iso')) {
$str = iconv('ISO-8859-1', 'UTF-8//TRANSLIT', utf8_decode($str));
} else {
$str = mb_convert_encoding($str, 'UTF-8', 'UTF-8');
}
// remove BOM
$str = urldecode(str_replace("%C2%81", '', urlencode($str)));
// prepare string
return $str;
}
An extra method to do the same job:
function remove_utf8_bom_head($text) {
if(substr(bin2hex($text), 0, 6) === 'efbbbf') {
$text = substr($text, 3);
}
return $text;
}
The other methods I found cannot work in my case.
Hope it helps in some special case.
A solution without pack function:
$a = "1";
var_dump($a); // string(4) "1"
function deleteBom($text)
{
return preg_replace("/^\xEF\xBB\xBF/", '', $text);
}
var_dump(deleteBom($a)); // string(1) "1"
I'm not so fond of using preg_replace or preg_match for simple tasks. What about this alternative method of detecting and removing the BOM?
function remove_utf8_bom(string $text): string
{
$bomStart = mb_substr($text, 0, 1);
return ($bomStart == pack('H*','EFBBBF')) ?
mb_substr($text, 1) :
$text;
}
If you are reading some API using file_get_contents and got an inexplicable NULL from json_decode, check the value of json_last_error(): sometimes the value returned from file_get_contents will have an extraneous BOM that is almost invisible when you inspect the string, but will make json_last_error() to return JSON_ERROR_SYNTAX (4).
>>> $json = file_get_contents("http://api-guiaserv.seade.gov.br/v1/orgao/all");
=> "\t{"orgao":[{"Nome":"Tribunal de Justi\u00e7a","ID_Orgao":"59","Condicao":"1"}, ...]}"
>>> json_decode($json);
=> null
>>>
In this case, check the first 3 bytes - echoing them is not very useful because the BOM is invisible on most settings:
>>> substr($json, 0, 3)
=> " "
>>> substr($json, 0, 3) == pack('H*','EFBBBF');
=> true
>>>
If the line above returns TRUE for you, then a simple test may fix the problem:
>>> json_decode($json[0] == "{" ? $json : substr($json, 3))
=> {#204
+"orgao": [
{#203
+"Nome": "Tribunal de Justiça",
+"ID_Orgao": "59",
+"Condicao": "1",
},
],
...
}
When working with faulty software it happens that the BOM part gets multiplied with every saving.
So I am using this to get rid of it.
function remove_utf8_bom($text) {
$bom = pack('H*','EFBBBF');
while (preg_match("/^$bom/", $text)) {
$text = preg_replace("/^$bom/", '', $text);
}
return $text;
}
How about this:
function removeUTF8BomHeader($data) {
if (substr($data, 0, 3) == pack('CCC', 0xef, 0xbb, 0xbf)) {
$data = substr($data, 3);
}
return $data;
}
tested a lot and it works perfect without any issue
Using PHP5 (cgi) to output template files from the filesystem and having issues spitting out raw HTML.
private function fetch($name) {
$path = $this->j->config['template_path'] . $name . '.html';
if (!file_exists($path)) {
dbgerror('Could not find the template "' . $name . '" in ' . $path);
}
$f = fopen($path, 'r');
$t = fread($f, filesize($path));
fclose($f);
if (substr($t, 0, 3) == b'\xef\xbb\xbf') {
$t = substr($t, 3);
}
return $t;
}
Even though I've added the BOM fix I'm still having problems with Firefox accepting it. You can see a live copy here: http://ircb.in/jisti/ (and the template file I threw at http://ircb.in/jisti/home.html if you want to check it out)
Any idea how to fix this? o_o
you would use the following code to remove utf8 bom
//Remove UTF8 Bom
function remove_utf8_bom($text)
{
$bom = pack('H*','EFBBBF');
$text = preg_replace("/^$bom/", '', $text);
return $text;
}
try:
// -------- read the file-content ----
$str = file_get_contents($source_file);
// -------- remove the utf-8 BOM ----
$str = str_replace("\xEF\xBB\xBF",'',$str);
// -------- get the Object from JSON ----
$obj = json_decode($str);
:)
Another way to remove the BOM which is Unicode code point U+FEFF
$str = preg_replace('/\x{FEFF}/u', '', $file);
b'\xef\xbb\xbf' stands for the literal string "\xef\xbb\xbf". If you want to check for a BOM, you need to use double quotes, so the \x sequences are actually interpreted into bytes:
"\xef\xbb\xbf"
Your files also seem to contain a lot more garbage than just a single leading BOM:
$ curl http://ircb.in/jisti/ | xxd
0000000: efbb bfef bbbf efbb bfef bbbf efbb bfef ................
0000010: bbbf efbb bf3c 2144 4f43 5459 5045 2068 .....<!DOCTYPE h
0000020: 746d 6c3e 0a3c 6874 6d6c 3e0a 3c68 6561 tml>.<html>.<hea
...
if anybody using csv import then below code useful
$header = fgetcsv($handle);
foreach($header as $key=> $val) {
$bom = pack('H*','EFBBBF');
$val = preg_replace("/^$bom/", '', $val);
$header[$key] = $val;
}
This global funtion resolve for UTF-8 system base charset. Tanks!
function prepareCharset($str) {
// set default encode
mb_internal_encoding('UTF-8');
// pre filter
if (empty($str)) {
return $str;
}
// get charset
$charset = mb_detect_encoding($str, array('ISO-8859-1', 'UTF-8', 'ASCII'));
if (stristr($charset, 'utf') || stristr($charset, 'iso')) {
$str = iconv('ISO-8859-1', 'UTF-8//TRANSLIT', utf8_decode($str));
} else {
$str = mb_convert_encoding($str, 'UTF-8', 'UTF-8');
}
// remove BOM
$str = urldecode(str_replace("%C2%81", '', urlencode($str)));
// prepare string
return $str;
}
An extra method to do the same job:
function remove_utf8_bom_head($text) {
if(substr(bin2hex($text), 0, 6) === 'efbbbf') {
$text = substr($text, 3);
}
return $text;
}
The other methods I found cannot work in my case.
Hope it helps in some special case.
A solution without pack function:
$a = "1";
var_dump($a); // string(4) "1"
function deleteBom($text)
{
return preg_replace("/^\xEF\xBB\xBF/", '', $text);
}
var_dump(deleteBom($a)); // string(1) "1"
I'm not so fond of using preg_replace or preg_match for simple tasks. What about this alternative method of detecting and removing the BOM?
function remove_utf8_bom(string $text): string
{
$bomStart = mb_substr($text, 0, 1);
return ($bomStart == pack('H*','EFBBBF')) ?
mb_substr($text, 1) :
$text;
}
If you are reading some API using file_get_contents and got an inexplicable NULL from json_decode, check the value of json_last_error(): sometimes the value returned from file_get_contents will have an extraneous BOM that is almost invisible when you inspect the string, but will make json_last_error() to return JSON_ERROR_SYNTAX (4).
>>> $json = file_get_contents("http://api-guiaserv.seade.gov.br/v1/orgao/all");
=> "\t{"orgao":[{"Nome":"Tribunal de Justi\u00e7a","ID_Orgao":"59","Condicao":"1"}, ...]}"
>>> json_decode($json);
=> null
>>>
In this case, check the first 3 bytes - echoing them is not very useful because the BOM is invisible on most settings:
>>> substr($json, 0, 3)
=> " "
>>> substr($json, 0, 3) == pack('H*','EFBBBF');
=> true
>>>
If the line above returns TRUE for you, then a simple test may fix the problem:
>>> json_decode($json[0] == "{" ? $json : substr($json, 3))
=> {#204
+"orgao": [
{#203
+"Nome": "Tribunal de Justiça",
+"ID_Orgao": "59",
+"Condicao": "1",
},
],
...
}
When working with faulty software it happens that the BOM part gets multiplied with every saving.
So I am using this to get rid of it.
function remove_utf8_bom($text) {
$bom = pack('H*','EFBBBF');
while (preg_match("/^$bom/", $text)) {
$text = preg_replace("/^$bom/", '', $text);
}
return $text;
}
How about this:
function removeUTF8BomHeader($data) {
if (substr($data, 0, 3) == pack('CCC', 0xef, 0xbb, 0xbf)) {
$data = substr($data, 3);
}
return $data;
}
tested a lot and it works perfect without any issue
I've been looking for a simple regex for URLs, does anybody have one handy that works well? I didn't find one with the zend framework validation classes and have seen several implementations.
Use the filter_var() function to validate whether a string is URL or not:
var_dump(filter_var('example.com', FILTER_VALIDATE_URL));
It is bad practice to use regular expressions when not necessary.
EDIT: Be careful, this solution is not unicode-safe and not XSS-safe. If you need a complex validation, maybe it's better to look somewhere else.
I used this on a few projects, I don't believe I've run into issues, but I'm sure it's not exhaustive:
$text = preg_replace(
'#((https?|ftp)://(\S*?\.\S*?))([\s)\[\]{},;"\':<]|\.\s|$)#i',
"'$3$4'",
$text
);
Most of the random junk at the end is to deal with situations like http://domain.example. in a sentence (to avoid matching the trailing period). I'm sure it could be cleaned up but since it worked. I've more or less just copied it over from project to project.
As per the PHP manual - parse_url should not be used to validate a URL.
Unfortunately, it seems that filter_var('example.com', FILTER_VALIDATE_URL) does not perform any better.
Both parse_url() and filter_var() will pass malformed URLs such as http://...
Therefore in this case - regex is the better method.
As per John Gruber (Daring Fireball):
Regex:
(?i)\b((?:https?://|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'\".,<>?«»“”‘’]))
using in preg_match():
preg_match("/(?i)\b((?:https?://|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))+(?:\(([^\s()<>]+|(\([^\s()<>]+\)))*\)|[^\s`!()\[\]{};:'\".,<>?«»“”‘’]))/", $url)
Here is the extended regex pattern (with comments):
(?xi)
\b
( # Capture 1: entire matched URL
(?:
https?:// # http or https protocol
| # or
www\d{0,3}[.] # "www.", "www1.", "www2." … "www999."
| # or
[a-z0-9.\-]+[.][a-z]{2,4}/ # looks like domain name followed by a slash
)
(?: # One or more:
[^\s()<>]+ # Run of non-space, non-()<>
| # or
\(([^\s()<>]+|(\([^\s()<>]+\)))*\) # balanced parens, up to 2 levels
)+
(?: # End with:
\(([^\s()<>]+|(\([^\s()<>]+\)))*\) # balanced parens, up to 2 levels
| # or
[^\s`!()\[\]{};:'".,<>?«»“”‘’] # not a space or one of these punct chars
)
)
For more details please look at:
http://daringfireball.net/2010/07/improved_regex_for_matching_urls
Just in case you want to know if the url really exists:
function url_exist($url){//se passar a URL existe
$c=curl_init();
curl_setopt($c,CURLOPT_URL,$url);
curl_setopt($c,CURLOPT_HEADER,1);//get the header
curl_setopt($c,CURLOPT_NOBODY,1);//and *only* get the header
curl_setopt($c,CURLOPT_RETURNTRANSFER,1);//get the response as a string from curl_exec(), rather than echoing it
curl_setopt($c,CURLOPT_FRESH_CONNECT,1);//don't use a cached version of the url
if(!curl_exec($c)){
//echo $url.' inexists';
return false;
}else{
//echo $url.' exists';
return true;
}
//$httpcode=curl_getinfo($c,CURLINFO_HTTP_CODE);
//return ($httpcode<400);
}
I don't think that using regular expressions is a smart thing to do in this case. It is impossible to match all of the possibilities and even if you did, there is still a chance that url simply doesn't exist.
Here is a very simple way to test if url actually exists and is readable :
if (preg_match("#^https?://.+#", $link) and #fopen($link,"r")) echo "OK";
(if there is no preg_match then this would also validate all filenames on your server)
I've used this one with good success - I don't remember where I got it from
$pattern = "/\b(?:(?:https?|ftp):\/\/|www\.)[-a-z0-9+&##\/%?=~_|!:,.;]*[-a-z0-9+&##\/%=~_|]/i";
The best URL Regex that worked for me:
function valid_URL($url){
return preg_match('%^(?:(?:https?|ftp)://)(?:\S+(?::\S*)?#|\d{1,3}(?:\.\d{1,3}){3}|(?:(?:[a-z\d\x{00a1}-\x{ffff}]+-?)*[a-z\d\x{00a1}-\x{ffff}]+)(?:\.(?:[a-z\d\x{00a1}-\x{ffff}]+-?)*[a-z\d\x{00a1}-\x{ffff}]+)*(?:\.[a-z\x{00a1}-\x{ffff}]{2,6}))(?::\d+)?(?:[^\s]*)?$%iu', $url);
}
Examples:
valid_URL('https://twitter.com'); // true
valid_URL('http://twitter.com'); // true
valid_URL('http://twitter.co'); // true
valid_URL('http://t.co'); // true
valid_URL('http://twitter.c'); // false
valid_URL('htt://twitter.com'); // false
valid_URL('http://example.com/?a=1&b=2&c=3'); // true
valid_URL('http://127.0.0.1'); // true
valid_URL(''); // false
valid_URL(1); // false
Source: http://urlregex.com/
function validateURL($URL) {
$pattern_1 = "/^(http|https|ftp):\/\/(([A-Z0-9][A-Z0-9_-]*)(\.[A-Z0-9][A-Z0-9_-]*)+.(com|org|net|dk|at|us|tv|info|uk|co.uk|biz|se)$)(:(\d+))?\/?/i";
$pattern_2 = "/^(www)((\.[A-Z0-9][A-Z0-9_-]*)+.(com|org|net|dk|at|us|tv|info|uk|co.uk|biz|se)$)(:(\d+))?\/?/i";
if(preg_match($pattern_1, $URL) || preg_match($pattern_2, $URL)){
return true;
} else{
return false;
}
}
Edit:
As incidence pointed out this code has been DEPRECATED with the release of PHP 5.3.0 (2009-06-30) and should be used accordingly.
Just my two cents but I've developed this function and have been using it for a while with success. It's well documented and separated so you can easily change it.
// Checks if string is a URL
// #param string $url
// #return bool
function isURL($url = NULL) {
if($url==NULL) return false;
$protocol = '(http://|https://)';
$allowed = '([a-z0-9]([-a-z0-9]*[a-z0-9]+)?)';
$regex = "^". $protocol . // must include the protocol
'(' . $allowed . '{1,63}\.)+'. // 1 or several sub domains with a max of 63 chars
'[a-z]' . '{2,6}'; // followed by a TLD
if(eregi($regex, $url)==true) return true;
else return false;
}
And there is your answer =) Try to break it, you can't!!!
function link_validate_url($text) {
$LINK_DOMAINS = 'aero|arpa|asia|biz|com|cat|coop|edu|gov|info|int|jobs|mil|museum|name|nato|net|org|pro|travel|mobi|local';
$LINK_ICHARS_DOMAIN = (string) html_entity_decode(implode("", array( // #TODO completing letters ...
"æ", // æ
"Æ", // Æ
"À", // À
"à", // à
"Á", // Á
"á", // á
"Â", // Â
"â", // â
"å", // å
"Å", // Å
"ä", // ä
"Ä", // Ä
"Ç", // Ç
"ç", // ç
"Ð", // Ð
"ð", // ð
"È", // È
"è", // è
"É", // É
"é", // é
"Ê", // Ê
"ê", // ê
"Ë", // Ë
"ë", // ë
"Î", // Î
"î", // î
"Ï", // Ï
"ï", // ï
"ø", // ø
"Ø", // Ø
"ö", // ö
"Ö", // Ö
"Ô", // Ô
"ô", // ô
"Õ", // Õ
"õ", // õ
"Œ", // Œ
"œ", // œ
"ü", // ü
"Ü", // Ü
"Ù", // Ù
"ù", // ù
"Û", // Û
"û", // û
"Ÿ", // Ÿ
"ÿ", // ÿ
"Ñ", // Ñ
"ñ", // ñ
"þ", // þ
"Þ", // Þ
"ý", // ý
"Ý", // Ý
"¿", // ¿
)), ENT_QUOTES, 'UTF-8');
$LINK_ICHARS = $LINK_ICHARS_DOMAIN . (string) html_entity_decode(implode("", array(
"ß", // ß
)), ENT_QUOTES, 'UTF-8');
$allowed_protocols = array('http', 'https', 'ftp', 'news', 'nntp', 'telnet', 'mailto', 'irc', 'ssh', 'sftp', 'webcal');
// Starting a parenthesis group with (?: means that it is grouped, but is not captured
$protocol = '((?:'. implode("|", $allowed_protocols) .'):\/\/)';
$authentication = "(?:(?:(?:[\w\.\-\+!$&'\(\)*\+,;=" . $LINK_ICHARS . "]|%[0-9a-f]{2})+(?::(?:[\w". $LINK_ICHARS ."\.\-\+%!$&'\(\)*\+,;=]|%[0-9a-f]{2})*)?)?#)";
$domain = '(?:(?:[a-z0-9' . $LINK_ICHARS_DOMAIN . ']([a-z0-9'. $LINK_ICHARS_DOMAIN . '\-_\[\]])*)(\.(([a-z0-9' . $LINK_ICHARS_DOMAIN . '\-_\[\]])+\.)*('. $LINK_DOMAINS .'|[a-z]{2}))?)';
$ipv4 = '(?:[0-9]{1,3}(\.[0-9]{1,3}){3})';
$ipv6 = '(?:[0-9a-fA-F]{1,4}(\:[0-9a-fA-F]{1,4}){7})';
$port = '(?::([0-9]{1,5}))';
// Pattern specific to external links.
$external_pattern = '/^'. $protocol .'?'. $authentication .'?('. $domain .'|'. $ipv4 .'|'. $ipv6 .' |localhost)'. $port .'?';
// Pattern specific to internal links.
$internal_pattern = "/^(?:[a-z0-9". $LINK_ICHARS ."_\-+\[\]]+)";
$internal_pattern_file = "/^(?:[a-z0-9". $LINK_ICHARS ."_\-+\[\]\.]+)$/i";
$directories = "(?:\/[a-z0-9". $LINK_ICHARS ."_\-\.~+%=&,$'#!():;*#\[\]]*)*";
// Yes, four backslashes == a single backslash.
$query = "(?:\/?\?([?a-z0-9". $LINK_ICHARS ."+_|\-\.~\/\\\\%=&,$'():;*#\[\]{} ]*))";
$anchor = "(?:#[a-z0-9". $LINK_ICHARS ."_\-\.~+%=&,$'():;*#\[\]\/\?]*)";
// The rest of the path for a standard URL.
$end = $directories .'?'. $query .'?'. $anchor .'?'.'$/i';
$message_id = '[^#].*#'. $domain;
$newsgroup_name = '(?:[0-9a-z+-]*\.)*[0-9a-z+-]*';
$news_pattern = '/^news:('. $newsgroup_name .'|'. $message_id .')$/i';
$user = '[a-zA-Z0-9'. $LINK_ICHARS .'_\-\.\+\^!#\$%&*+\/\=\?\`\|\{\}~\'\[\]]+';
$email_pattern = '/^mailto:'. $user .'#'.'(?:'. $domain .'|'. $ipv4 .'|'. $ipv6 .'|localhost)'. $query .'?$/';
if (strpos($text, '<front>') === 0) {
return false;
}
if (in_array('mailto', $allowed_protocols) && preg_match($email_pattern, $text)) {
return false;
}
if (in_array('news', $allowed_protocols) && preg_match($news_pattern, $text)) {
return false;
}
if (preg_match($internal_pattern . $end, $text)) {
return false;
}
if (preg_match($external_pattern . $end, $text)) {
return false;
}
if (preg_match($internal_pattern_file, $text)) {
return false;
}
return true;
}
function is_valid_url ($url="") {
if ($url=="") {
$url=$this->url;
}
$url = #parse_url($url);
if ( ! $url) {
return false;
}
$url = array_map('trim', $url);
$url['port'] = (!isset($url['port'])) ? 80 : (int)$url['port'];
$path = (isset($url['path'])) ? $url['path'] : '';
if ($path == '') {
$path = '/';
}
$path .= ( isset ( $url['query'] ) ) ? "?$url[query]" : '';
if ( isset ( $url['host'] ) AND $url['host'] != gethostbyname ( $url['host'] ) ) {
if ( PHP_VERSION >= 5 ) {
$headers = get_headers("$url[scheme]://$url[host]:$url[port]$path");
}
else {
$fp = fsockopen($url['host'], $url['port'], $errno, $errstr, 30);
if ( ! $fp ) {
return false;
}
fputs($fp, "HEAD $path HTTP/1.1\r\nHost: $url[host]\r\n\r\n");
$headers = fread ( $fp, 128 );
fclose ( $fp );
}
$headers = ( is_array ( $headers ) ) ? implode ( "\n", $headers ) : $headers;
return ( bool ) preg_match ( '#^HTTP/.*\s+[(200|301|302)]+\s#i', $headers );
}
return false;
}
Inspired in this .NET StackOverflow question and in this referenced article from that question there is this URI validator (URI means it validates both URL and URN).
if( ! preg_match( "/^([a-z][a-z0-9+.-]*):(?:\\/\\/((?:(?=((?:[a-z0-9-._~!$&'()*+,;=:]|%[0-9A-F]{2})*))(\\3)#)?(?=(\\[[0-9A-F:.]{2,}\\]|(?:[a-z0-9-._~!$&'()*+,;=]|%[0-9A-F]{2})*))\\5(?::(?=(\\d*))\\6)?)(\\/(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\8)?|(\\/?(?!\\/)(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\10)?)(?:\\?(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\11)?(?:#(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\12)?$/i", $uri ) )
{
throw new \RuntimeException( "URI has not a valid format." );
}
I have successfully unit-tested this function inside a ValueObject I made named Uri and tested by UriTest.
UriTest.php (Contains valid and invalid cases for both URLs and URNs)
<?php
declare( strict_types = 1 );
namespace XaviMontero\ThrasherPortage\Tests\Tour;
use XaviMontero\ThrasherPortage\Tour\Uri;
class UriTest extends \PHPUnit_Framework_TestCase
{
private $sut;
public function testCreationIsOfProperClassWhenUriIsValid()
{
$sut = new Uri( 'http://example.com' );
$this->assertInstanceOf( 'XaviMontero\\ThrasherPortage\\Tour\\Uri', $sut );
}
/**
* #dataProvider urlIsValidProvider
* #dataProvider urnIsValidProvider
*/
public function testGetUriAsStringWhenUriIsValid( string $uri )
{
$sut = new Uri( $uri );
$actual = $sut->getUriAsString();
$this->assertInternalType( 'string', $actual );
$this->assertEquals( $uri, $actual );
}
public function urlIsValidProvider()
{
return
[
[ 'http://example-server' ],
[ 'http://example.com' ],
[ 'http://example.com/' ],
[ 'http://subdomain.example.com/path/?parameter1=value1¶meter2=value2' ],
[ 'random-protocol://example.com' ],
[ 'http://example.com:80' ],
[ 'http://example.com?no-path-separator' ],
[ 'http://example.com/pa%20th/' ],
[ 'ftp://example.org/resource.txt' ],
[ 'file://../../../relative/path/needs/protocol/resource.txt' ],
[ 'http://example.com/#one-fragment' ],
[ 'http://example.edu:8080#one-fragment' ],
];
}
public function urnIsValidProvider()
{
return
[
[ 'urn:isbn:0-486-27557-4' ],
[ 'urn:example:mammal:monotreme:echidna' ],
[ 'urn:mpeg:mpeg7:schema:2001' ],
[ 'urn:uuid:6e8bc430-9c3a-11d9-9669-0800200c9a66' ],
[ 'rare-urn:uuid:6e8bc430-9c3a-11d9-9669-0800200c9a66' ],
[ 'urn:FOO:a123,456' ]
];
}
/**
* #dataProvider urlIsNotValidProvider
* #dataProvider urnIsNotValidProvider
*/
public function testCreationThrowsExceptionWhenUriIsNotValid( string $uri )
{
$this->expectException( 'RuntimeException' );
$this->sut = new Uri( $uri );
}
public function urlIsNotValidProvider()
{
return
[
[ 'only-text' ],
[ 'http//missing.colon.example.com/path/?parameter1=value1¶meter2=value2' ],
[ 'missing.protocol.example.com/path/' ],
[ 'http://example.com\\bad-separator' ],
[ 'http://example.com|bad-separator' ],
[ 'ht tp://example.com' ],
[ 'http://exampl e.com' ],
[ 'http://example.com/pa th/' ],
[ '../../../relative/path/needs/protocol/resource.txt' ],
[ 'http://example.com/#two-fragments#not-allowed' ],
[ 'http://example.edu:portMustBeANumber#one-fragment' ],
];
}
public function urnIsNotValidProvider()
{
return
[
[ 'urn:mpeg:mpeg7:sch ema:2001' ],
[ 'urn|mpeg:mpeg7:schema:2001' ],
[ 'urn?mpeg:mpeg7:schema:2001' ],
[ 'urn%mpeg:mpeg7:schema:2001' ],
[ 'urn#mpeg:mpeg7:schema:2001' ],
];
}
}
Uri.php (Value Object)
<?php
declare( strict_types = 1 );
namespace XaviMontero\ThrasherPortage\Tour;
class Uri
{
/** #var string */
private $uri;
public function __construct( string $uri )
{
$this->assertUriIsCorrect( $uri );
$this->uri = $uri;
}
public function getUriAsString()
{
return $this->uri;
}
private function assertUriIsCorrect( string $uri )
{
// https://stackoverflow.com/questions/30847/regex-to-validate-uris
// http://snipplr.com/view/6889/regular-expressions-for-uri-validationparsing/
if( ! preg_match( "/^([a-z][a-z0-9+.-]*):(?:\\/\\/((?:(?=((?:[a-z0-9-._~!$&'()*+,;=:]|%[0-9A-F]{2})*))(\\3)#)?(?=(\\[[0-9A-F:.]{2,}\\]|(?:[a-z0-9-._~!$&'()*+,;=]|%[0-9A-F]{2})*))\\5(?::(?=(\\d*))\\6)?)(\\/(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\8)?|(\\/?(?!\\/)(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/]|%[0-9A-F]{2})*))\\10)?)(?:\\?(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\11)?(?:#(?=((?:[a-z0-9-._~!$&'()*+,;=:#\\/?]|%[0-9A-F]{2})*))\\12)?$/i", $uri ) )
{
throw new \RuntimeException( "URI has not a valid format." );
}
}
}
Running UnitTests
There are 65 assertions in 46 tests. Caution: there are 2 data-providers for valid and 2 more for invalid expressions. One is for URLs and the other for URNs. If you are using a version of PhpUnit of v5.6* or earlier then you need to join the two data providers into a single one.
xavi#bromo:~/custom_www/hello-trip/mutant-migrant$ vendor/bin/phpunit
PHPUnit 5.7.3 by Sebastian Bergmann and contributors.
.............................................. 46 / 46 (100%)
Time: 82 ms, Memory: 4.00MB
OK (46 tests, 65 assertions)
Code coverage
There's is 100% of code-coverage in this sample URI checker.
"/(http(s?):\/\/)([a-z0-9\-]+\.)+[a-z]{2,4}(\.[a-z]{2,4})*(\/[^ ]+)*/i"
(http(s?)://) means http:// or https://
([a-z0-9-]+.)+ =>
2.0[a-z0-9-] means any a-z character or any 0-9 or (-)sign)
2.1 (+) means the character can be one or more ex: a1w,
a9-,c559s, f)
2.2 \. is (.)sign
2.3. the (+) sign after ([a-z0-9\-]+\.) mean do 2.1,2.2,2.3
at least 1 time
ex: abc.defgh0.ig, aa.b.ced.f.gh. also in case www.yyy.com
3.[a-z]{2,4} mean a-z at least 2 character but not more than
4 characters for check that there will not be
the case
ex: https://www.google.co.kr.asdsdagfsdfsf
4.(\.[a-z]{2,4})*(\/[^ ]+)* mean
4.1 \.[a-z]{2,4} means like number 3 but start with
(.)sign
4.2 * means (\.[a-z]{2,4})can be use or not use never mind
4.3 \/ means \
4.4 [^ ] means any character except blank
4.5 (+) means do 4.3,4.4,4.5 at least 1 times
4.6 (*) after (\/[^ ]+) mean use 4.3 - 4.5 or not use
no problem
use for case https://stackoverflow.com/posts/51441301/edit
5. when you use regex write in "/ /" so it come
"/(http(s?)://)([a-z0-9-]+.)+[a-z]{2,4}(.[a-z]{2,4})(/[^ ]+)/i"
6. almost forgot: letter i on the back mean ignore case of
Big letter or small letter ex: A same as a, SoRRy same
as sorry.
Note : Sorry for bad English. My country not use it well.
OK, so this is a little bit more complex then a simple regex, but it allows for different types of urls.
Examples:
google.com
www.microsoft.com/
http://www.yahoo.com/
https://www.bandcamp.com/artist/#!someone-special!
All which should be marked as valid.
function is_valid_url($url) {
// First check: is the url just a domain name? (allow a slash at the end)
$_domain_regex = "|^[A-Za-z0-9-]+(\.[A-Za-z0-9-]+)*(\.[A-Za-z]{2,})/?$|";
if (preg_match($_domain_regex, $url)) {
return true;
}
// Second: Check if it's a url with a scheme and all
$_regex = '#^([a-z][\w-]+:(?:/{1,3}|[a-z0-9%])|www\d{0,3}[.]|[a-z0-9.\-]+[.][a-z]{2,4}/)(?:[^\s()<>]+|\(([^\s()<>]+|(\([^\s()<>]+\)))*\))$#';
if (preg_match($_regex, $url, $matches)) {
// pull out the domain name, and make sure that the domain is valid.
$_parts = parse_url($url);
if (!in_array($_parts['scheme'], array( 'http', 'https' )))
return false;
// Check the domain using the regex, stops domains like "-example.com" passing through
if (!preg_match($_domain_regex, $_parts['host']))
return false;
// This domain looks pretty valid. Only way to check it now is to download it!
return true;
}
return false;
}
Note that there is a in_array check for the protocols that you want to allow (currently only http and https are in that list).
var_dump(is_valid_url('google.com')); // true
var_dump(is_valid_url('google.com/')); // true
var_dump(is_valid_url('http://google.com')); // true
var_dump(is_valid_url('http://google.com/')); // true
var_dump(is_valid_url('https://google.com')); // true
For anyone developing with WordPress, just use
esc_url_raw($url) === $url
to validate a URL (here's WordPress' documentation on esc_url_raw). It handles URLs much better than filter_var($url, FILTER_VALIDATE_URL) because it is unicode and XSS-safe. (Here is a good article mentioning all the problems with filter_var).
Peter's Regex doesn't look right to me for many reasons. It allows all kinds of special characters in the domain name and doesn't test for much.
Frankie's function looks good to me and you can build a good regex from the components if you don't want a function, like so:
^(http://|https://)(([a-z0-9]([-a-z0-9]*[a-z0-9]+)?){1,63}\.)+[a-z]{2,6}
Untested but I think that should work.
Also, Owen's answer doesn't look 100% either. I took the domain part of the regex and tested it on a Regex tester tool http://erik.eae.net/playground/regexp/regexp.html
I put the following line:
(\S*?\.\S*?)
in the "regexp" section
and the following line:
-hello.com
under the "sample text" section.
The result allowed the minus character through. Because \S means any non-space character.
Note the regex from Frankie handles the minus because it has this part for the first character:
[a-z0-9]
Which won't allow the minus or any other special character.
Here is the way I did it. But I want to mentoin that I am not so shure about the regex. But It should work thou :)
$pattern = "#((http|https)://(\S*?\.\S*?))(\s|\;|\)|\]|\[|\{|\}|,|”|\"|'|:|\<|$|\.\s)#i";
$text = preg_replace_callback($pattern,function($m){
return "$m[1]$m[4]";
},
$text);
This way you won't need the eval marker on your pattern.
Hope it helps :)
Here's a simple class for URL Validation using RegEx and then cross-references the domain against popular RBL (Realtime Blackhole Lists) servers:
Install:
require 'URLValidation.php';
Usage:
require 'URLValidation.php';
$urlVal = new UrlValidation(); //Create Object Instance
Add a URL as the parameter of the domain() method and check the the return.
$urlArray = ['http://www.bokranzr.com/test.php?test=foo&test=dfdf', 'https://en-gb.facebook.com', 'https://www.google.com'];
foreach ($urlArray as $k=>$v) {
echo var_dump($urlVal->domain($v)) . ' URL: ' . $v . '<br>';
}
Output:
bool(false) URL: http://www.bokranzr.com/test.php?test=foo&test=dfdf
bool(true) URL: https://en-gb.facebook.com
bool(true) URL: https://www.google.com
As you can see above, www.bokranzr.com is listed as malicious website via an RBL so the domain was returned as false.
I've found this to be the most useful for matching a URL..
^(https?:\/\/)?([\da-z\.-]+)\.([a-z\.]{2,6})([\/\w \.-]*)*\/?$
There is a PHP native function for that:
$url = 'http://www.yoururl.co.uk/sub1/sub2/?param=1¶m2/';
if ( ! filter_var( $url, FILTER_VALIDATE_URL ) ) {
// Wrong
}
else {
// Valid
}
Returns the filtered data, or FALSE if the filter fails.
Check it here
I'm using CakePHP 1.3.7 and ran into a very specific issue.
The Sanitize core class method used in my application is the one of version 1.2. When I want to save particular data, it gives me a warning :
Warning: array_merge(): Argument #2 is not an array in
/usr/share/php/cake/libs/sanitize.php on line 113
but it does save, and with the right encoding/format.
Here's the method who causes this warning (version 1.2, which is NOT on line 113, but I'll come to that later)
function html($string, $remove = false) {
if ($remove) {
$string = strip_tags($string);
} else {
$patterns = array("/\&/", "/%/", "/</", "/>/", '/"/', "/'/", "/\(/", "/\)/", "/\+/", "/-/");
$replacements = array("&", "%", "<", ">", """, "'", "(", ")", "+", "-");
$string = preg_replace($patterns, $replacements, $string);
}
return $string;
}
And here's how this method is called
$value = Sanitize::html($value,true);
Now as you can see, array_merge() is not called in this method, but if I replace the html() method by the 1.3 version
function html($string, $options = array()) {
static $defaultCharset = false;
if ($defaultCharset === false) {
$defaultCharset = Configure::read('App.encoding');
if ($defaultCharset === null) {
$defaultCharset = 'UTF-8';
}
}
$default = array(
'remove' => false,
'charset' => $defaultCharset,
'quotes' => ENT_QUOTES
);
$options = array_merge($default, $options);
if ($options['remove']) {
$string = strip_tags($string);
}
return htmlentities($string, $options['quotes'], $options['charset']);
}
array_merge() falls exactly on line 113.
If I now call html() this way
$value = Sanitize::html($value,array('remove' => true));
I don't get the warning anymore. However, my data doesn't save with the right encoding/format anymore.
Here's an example of text I need to save (it is french and needs UTF-8 encoding)
L'envoi d'une communication & à la fenêtre
I can't overcome this doing
$value = Sanitize::html($value,array('remove' => true, 'quotes' => ENT_HTML401));
because I'm using PHP 5.3.6 thus I can't use the constant ENT_HTML401
If I use another constant like ENT_NOQUOTES, it ignores the quotes (obviously) but not the french accents and other special chars, which is intented to work this way but I want to save the text exactly like I quoted (or at least read it).
I'm guessing I wouldn't need to use htmlentities, but I think it is safer to and updating the core method is the only way I found to not get the warning. I also suppose I should not really modify these files other than for updating them?
So, briefly, I want to :
Get rid of the warning
Save/read data in the right format
I might have forgotten some infos, thanks
I ended up updating the html() method of the Sanitize class to match version 1.3 as follow
function html($string, $options = array()) {
static $defaultCharset = false;
if ($defaultCharset === false) {
$defaultCharset = Configure::read('App.encoding');
if ($defaultCharset === null) {
$defaultCharset = 'UTF-8';
}
}
$default = array(
'remove' => false,
'charset' => $defaultCharset,
'quotes' => ENT_QUOTES
);
$options = array_merge($default, $options);
if ($options['remove']) {
$string = strip_tags($string);
}
return htmlentities($string, $options['quotes'], $options['charset']);
}
I call it like this
$value = Sanitize::html($value, array('remove'=>true,'quotes'=>ENT_NOQUOTES));
And I simply decode the text fields this way whenever I read their value from database
$data['Model']['field'] = html_entity_decode($data['Model']['field'], ENT_NOQUOTES, "UTF-8");
EDIT : I had to undo what I described above because the way data was encoded in the 1.3 version of the function made it so we had to decode the data in the whole application when reading it.
Also, I am NOT using CakePHP 1.3.7 (got confused with cake console); I'm using 1.2.4 so updating the function was not appropriate afterall.
I kept the version 1.2 and this time I simply changed the second parameter to an array as follow and it seemed to do the trick as I'm not getting the warning anymore.
function html($string, $options = array()) {
if ($options['remove']) {
$string = strip_tags($string);
} else {
$patterns = array("/\&/", "/%/", "/</", "/>/", '/"/', "/'/", "/\(/", "/\)/", "/\+/", "/-/");
$replacements = array("&", "%", "<", ">", """, "'", "(", ")", "+", "-");
$string = preg_replace($patterns, $replacements, $string);
}
return $string;
}
I need to write a function to parse variables which contain domain names. It's best I explain this with an example, the variable could contain any of these things:
here.example.com
example.com
example.org
here.example.org
But when passed through my function all of these must return either example.com or example.co.uk, the root domain name basically. I'm sure I've done this before but I've been searching Google for about 20 minutes and can't find anything. Any help would be appreciated.
EDIT: Ignore the .co.uk, presume that all domains going through this function have a 3 letter TLD.
Stackoverflow Question Archive:
How to get domain name from url?
Check if domain equals value?
How do I get the base url?
print get_domain("http://somedomain.co.uk"); // outputs 'somedomain.co.uk'
function get_domain($url)
{
$pieces = parse_url($url);
$domain = isset($pieces['host']) ? $pieces['host'] : '';
if (preg_match('/(?P<domain>[a-z0-9][a-z0-9\-]{1,63}\.[a-z\.]{2,6})$/i', $domain, $regs)) {
return $regs['domain'];
}
return false;
}
If you want a fast simple solution, without external calls and checking against predefined arrays. Works for new domains like "www.domain.gallery" also, unlike the most popular answer.
function get_domain($host){
$myhost = strtolower(trim($host));
$count = substr_count($myhost, '.');
if($count === 2){
if(strlen(explode('.', $myhost)[1]) > 3) $myhost = explode('.', $myhost, 2)[1];
} else if($count > 2){
$myhost = get_domain(explode('.', $myhost, 2)[1]);
}
return $myhost;
}
domain.com -> domain.com
sub.domain.com -> domain.com
www.domain.com -> domain.com
www.sub.sub.domain.com -> domain.com
domain.co.uk -> domain.co.uk
sub.domain.co.uk -> domain.co.uk
www.domain.co.uk -> domain.co.uk
www.sub.sub.domain.co.uk -> domain.co.uk
domain.photography -> domain.photography
www.domain.photography -> domain.photography
www.sub.domain.photography -> domain.photography
I would do something like the following:
// hierarchical array of top level domains
$tlds = array(
'com' => true,
'uk' => array(
'co' => true,
// …
),
// …
);
$domain = 'here.example.co.uk';
// split domain
$parts = explode('.', $domain);
$tmp = $tlds;
// travers the tree in reverse order, from right to left
foreach (array_reverse($parts) as $key => $part) {
if (isset($tmp[$part])) {
$tmp = $tmp[$part];
} else {
break;
}
}
// build the result
var_dump(implode('.', array_slice($parts, - $key - 1)));
I ended up using the database Mozilla has.
Here's my code:
fetch_mozilla_tlds.php contains caching algorhythm. This line is important:
$mozillaTlds = file('http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/effective_tld_names.dat?raw=1');
The main file used inside the application is this:
function isTopLevelDomain($domain)
{
$domainParts = explode('.', $domain);
if (count($domainParts) == 1) {
return false;
}
$previousDomainParts = $domainParts;
array_shift($previousDomainParts);
$tld = implode('.', $previousDomainParts);
return isDomainExtension($tld);
}
function isDomainExtension($domain)
{
$tlds = getTLDs();
/**
* direct hit
*/
if (in_array($domain, $tlds)) {
return true;
}
if (in_array('!'. $domain, $tlds)) {
return false;
}
$domainParts = explode('.', $domain);
if (count($domainParts) == 1) {
return false;
}
$previousDomainParts = $domainParts;
array_shift($previousDomainParts);
array_unshift($previousDomainParts, '*');
$wildcardDomain = implode('.', $previousDomainParts);
return in_array($wildcardDomain, $tlds);
}
function getTLDs()
{
static $mozillaTlds = array();
if (empty($mozillaTlds)) {
require 'fetch_mozilla_tlds.php';
/* #var $mozillaTlds array */
}
return $mozillaTlds;
}
UPDATE:
The database has evolved and is now available at its own website - http://publicsuffix.org/
Almost certainly, what you're looking for is this:
https://github.com/Synchro/regdom-php
It's a PHP library that utilizes the (as nearly as is practical) full list of various TLD's that's collected at publicsuffix.org/list/ , and wraps it up in a spiffy little function.
Once the library is included, it's as easy as:
$registeredDomain = getRegisteredDomain( $domain );
$full_domain = $_SERVER['SERVER_NAME'];
$just_domain = preg_replace("/^(.*\.)?([^.]*\..*)$/", "$2", $_SERVER['HTTP_HOST']);
There are two ways to extract subdomain from a host:
The first method that is more accurate is to use a database of tlds (like public_suffix_list.dat) and match domain with it. This is a little heavy in some cases. There are some PHP classes for using it like php-domain-parser and TLDExtract.
The second way is not as accurate as the first one, but is very fast and it can give the correct answer in many case, I wrote this function for it:
function get_domaininfo($url) {
// regex can be replaced with parse_url
preg_match("/^(https|http|ftp):\/\/(.*?)\//", "$url/" , $matches);
$parts = explode(".", $matches[2]);
$tld = array_pop($parts);
$host = array_pop($parts);
if ( strlen($tld) == 2 && strlen($host) <= 3 ) {
$tld = "$host.$tld";
$host = array_pop($parts);
}
return array(
'protocol' => $matches[1],
'subdomain' => implode(".", $parts),
'domain' => "$host.$tld",
'host'=>$host,'tld'=>$tld
);
}
Example:
print_r(get_domaininfo('https://mysubdomain.domain.co.uk/index.php'));
Returns:
Array
(
[protocol] => https
[subdomain] => mysubdomain
[domain] => domain.co.uk
[host] => domain
[tld] => co.uk
)
This is a short way of accomplishing that:
$host = $_SERVER['HTTP_HOST'];
preg_match("/[^\.\/]+\.[^\.\/]+$/", $host, $matches);
echo "domain name is: {$matches[0]}\n";
Check this my simple solution!
function getHost($a){
$tld = preg_replace('/.*\.([a-zA-Z]+)$/','$1',$a);
return trim(preg_replace('/.*(([\.\/][a-zA-Z]{2,}){'.((substr_count($a, '.') <= 2 && mb_strlen( $tld) != 2) ? '2,3' : '3,4').'})/im','$1',$a),'./');
}
echo getHost('https://webmail.google.com.br')."<br>";
echo getHost('https://google.com.br')."<br>";
echo getHost('https://webmail.google.net.br')."<br>";
echo getHost('https://webmail.google.net')."<br>";
echo getHost('https://google.net')."<br>";
echo getHost('webmail.google.com.br')."<br>";
#output
google.com.br
google.com.br
google.net.br
google.net
google.net
google.com.br
As a variant to Jonathan Sampson
function get_domain($url) {
if ( !preg_match("/^http/", $url) )
$url = 'http://' . $url;
if ( $url[strlen($url)-1] != '/' )
$url .= '/';
$pieces = parse_url($url);
$domain = isset($pieces['host']) ? $pieces['host'] : '';
if ( preg_match('/(?P<domain>[a-z0-9][a-z0-9\-]{1,63}\.[a-z\.]{2,6})$/i', $domain, $regs) ) {
$res = preg_replace('/^www\./', '', $regs['domain'] );
return $res;
}
return false;
}
This script generates a Perl file containing a single function, get_domain from the ETLD file. So say you have hostnames like img1, img2, img3, ... in .photobucket.com. For each of those get_domain $host would return photobucket.com. Note that this isn't the fastest function on earth, so in my main log parser that's using this, I keep a hash of host to domain mappings and only run this for hosts that aren't in the hash yet.
#!/bin/bash
cat << 'EOT' > suffixes.pl
#!/bin/perl
sub get_domain {
$_ = shift;
EOT
wget -O - http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/effective_tld_names.dat?raw=1 \
| iconv -c -f UTF-8 -t ASCII//TRANSLIT \
| egrep -v '/|^$' \
| sed -e 's/^\!//' -e "s/\"/'/g" \
| awk '{ print length($0),$0 | "sort -rn"}' | cut -d" " -f2- \
| while read SUFF; do
STAR=`echo $SUFF | cut -b1`
if [ "$STAR" = '*' ]; then
SUFF=`echo $SUFF | cut -b3-`
echo " return \"\$1\.\$2\.$SUFF\" if /([a-zA-Z0-9\-]+)\.([a-zA-Z0-9\-]+)\.$SUFF\$/;"
else
echo " return \"\$1\.$SUFF\" if /([a-zA-Z0-9\-]+)\.$SUFF\$/;"
fi
done >> suffixes.pl
cat << 'EOT' >> suffixes.pl
}
1;
EOT
This isn't foolproof and should only really be used if you know the domain isn't going to be anything obscure, but it's easier to read than most of the other options:
$justDomain = $_SERVER['SERVER_NAME'];
switch(substr_count($justDomain, '.')) {
case 1:
// 2 parts. Must not be a subdomain. Do nothing.
break;
case 2:
// 3 parts. Either a subdomain or a 2-part suffix
// If the 2nd part is over 3 chars's, assume it to be the main domain part which means we have a subdomain.
// This isn't foolproof, but should be ok for most domains.
// Something like domainname.parliament.nz would cause problems, though. As would www.abc.com
$parts = explode('.', $justDomain);
if(strlen($parts[1]) > 3) {
unset($parts[0]);
$justDomain = implode('.', $parts);
}
break;
default:
// 4+ parts. Must be a subdomain.
$parts = explode('.', $justDomain, 2);
$justDomain = $parts[1];
break;
}
// $justDomain should now exclude any subdomain part.
//For short domain like t.co (twitter) the function should be :
function get_domain($url)
{
$pieces = parse_url($url);
$domain = isset($pieces['host']) ? $pieces['host'] : '';
if (preg_match('/(?P<domain>[a-z0-9][a-z0-9\-]{0,63}\.[a-z\.]{2,6})$/i', $domain, $regs)) {
return $regs['domain'];
}
return false;
}
Regex could help you out there. Try something like this:
([^.]+(.com|.co.uk))$
I think your problem is that you haven't clearly defined what exactly you want the function to do. From your examples, you certainly don't want it to just blindly return the last two, or last three, components of the name, but just knowing what it shouldn't do isn't enough.
Here's my guess at what you really want: there are certain second-level domain names, like co.uk., that you'd like to be treated as a single TLD (top-level domain) for purposes of this function. In that case I'd suggest enumerating all such cases and putting them as keys into an associative array with dummy values, along with all the normal top-level domains like com., net., info., etc. Then whenever you get a new domain name, extract the last two components and see if the resulting string is in your array as a key. If not, extract just the last component and make sure that's in your array. (If even that isn't, it's not a valid domain name) Either way, whatever key you do find in the array, take that plus one more component off the end of the domain name, and you'll have your base domain.
You could, perhaps, make things a bit simpler by writing a function, instead of using an associative array, to tell whether the last two components should be treated as a single "effective TLD." The function would probably look at the next-to-last component and, if it's shorter than 3 characters, decide that it should be treated as part of the TLD.
To do it well, you'll need a list of the second level domains and top level domains and build an appropriate regular expression list. A good list of second level domains is available at https://wiki.mozilla.org/TLD_List. Another test case apart from the aforementioned CentralNic .uk.com variants is The Vatican: their website is technically at http://va : and that's a difficult one to match on!
Building on Jonathan's answer:
function main_domain($domain) {
if (preg_match('/([a-z0-9][a-z0-9\-]{1,63})\.([a-z]{3}|[a-z]{2}\.[a-z]{2})$/i', $domain, $regs)) {
return $regs;
}
return false;
}
His expression might be a bit better, but this interface seems more like what you're describing.
Ah - if you just want to handle three character top level domains - then this code works:
<?php
// let's test the code works: these should all return
// example.com , example.net or example.org
$domains=Array('here.example.com',
'example.com',
'example.org',
'here.example.org',
'example.com/ignorethis',
'example.net/',
'http://here.example.org/longtest?string=here');
foreach ($domains as $domain) {
testdomain($domain);
}
function testdomain($url) {
if (preg_match('/^((.+)\.)?([A-Za-z][0-9A-Za-z\-]{1,63})\.([A-Za-z]{3})(\/.*)?$/',$url,$matches)) {
print 'Domain is: '.$matches[3].'.'.$matches[4].'<br>'."\n";
} else {
print 'Domain not found in '.$url.'<br>'."\n";
}
}
?>
$matches[1]/$matches[2] will contain any subdomain and/or protocol, $matches[3] contains the domain name, $matches[4] the top level domain and $matches[5] contains any other URL path information.
To match most common top level domains you could try changing it to:
if (preg_match('/^((.+)\.)?([A-Za-z][0-9A-Za-z\-]{1,63})\.([A-Za-z]{2,6})(\/.*)?$/',$url,$matches)) {
Or to get it coping with everything:
if (preg_match('/^((.+)\.)?([A-Za-z][0-9A-Za-z\-]{1,63})\.(co\.uk|me\.uk|org\.uk|com|org|net|int|eu)(\/.*)?$/',$url,$matches)) {
etc etc
Based on http://www.cafewebmaster.com/find-top-level-domain-international-urls-php
function find_tld($url){
$purl = parse_url($url);
$host = strtolower($purl['host']);
$valid_tlds = ".ab.ca .bc.ca .mb.ca .nb.ca .nf.ca .nl.ca .ns.ca .nt.ca .nu.ca .on.ca .pe.ca .qc.ca .sk.ca .yk.ca .com.cd .net.cd .org.cd .com.ch .net.ch .org.ch .gov.ch .co.ck .ac.cn .com.cn .edu.cn .gov.cn .net.cn .org.cn .ah.cn .bj.cn .cq.cn .fj.cn .gd.cn .gs.cn .gz.cn .gx.cn .ha.cn .hb.cn .he.cn .hi.cn .hl.cn .hn.cn .jl.cn .js.cn .jx.cn .ln.cn .nm.cn .nx.cn .qh.cn .sc.cn .sd.cn .sh.cn .sn.cn .sx.cn .tj.cn .xj.cn .xz.cn .yn.cn .zj.cn .com.co .edu.co .org.co .gov.co .mil.co .net.co .nom.co .com.cu .edu.cu .org.cu .net.cu .gov.cu .inf.cu .gov.cx .edu.do .gov.do .gob.do .com.do .org.do .sld.do .web.do .net.do .mil.do .art.do .com.dz .org.dz .net.dz .gov.dz .edu.dz .asso.dz .pol.dz .art.dz .com.ec .info.ec .net.ec .fin.ec .med.ec .pro.ec .org.ec .edu.ec .gov.ec .mil.ec .com.ee .org.ee .fie.ee .pri.ee .eun.eg .edu.eg .sci.eg .gov.eg .com.eg .org.eg .net.eg .mil.eg .com.es .nom.es .org.es .gob.es .edu.es .com.et .gov.et .org.et .edu.et .net.et .biz.et .name.et .info.et .co.fk .org.fk .gov.fk .ac.fk .nom.fk .net.fk .tm.fr .asso.fr .nom.fr .prd.fr .presse.fr .com.fr .gouv.fr .com.ge .edu.ge .gov.ge .org.ge .mil.ge .net.ge .pvt.ge .co.gg .net.gg .org.gg .com.gi .ltd.gi .gov.gi .mod.gi .edu.gi .org.gi .com.gn .ac.gn .gov.gn .org.gn .net.gn .com.gr .edu.gr .net.gr .org.gr .gov.gr .com.hk .edu.hk .gov.hk .idv.hk .net.hk .org.hk .com.hn .edu.hn .org.hn .net.hn .mil.hn .gob.hn .iz.hr .from.hr .name.hr .com.hr .com.ht .net.ht .firm.ht .shop.ht .info.ht .pro.ht .adult.ht .org.ht .art.ht .pol.ht .rel.ht .asso.ht .perso.ht .coop.ht .med.ht .edu.ht .gouv.ht .gov.ie .co.in .firm.in .net.in .org.in .gen.in .ind.in .nic.in .ac.in .edu.in .res.in .gov.in .mil.in .ac.ir .co.ir .gov.ir .net.ir .org.ir .sch.ir .gov.it .co.je .net.je .org.je .edu.jm .gov.jm .com.jm .net.jm .com.jo .org.jo .net.jo .edu.jo .gov.jo .mil.jo .co.kr .or.kr .com.kw .edu.kw .gov.kw .net.kw .org.kw .mil.kw .edu.ky .gov.ky .com.ky .org.ky .net.ky .org.kz .edu.kz .net.kz .gov.kz .mil.kz .com.kz .com.li .net.li .org.li .gov.li .gov.lk .sch.lk .net.lk .int.lk .com.lk .org.lk .edu.lk .ngo.lk .soc.lk .web.lk .ltd.lk .assn.lk .grp.lk .hotel.lk .com.lr .edu.lr .gov.lr .org.lr .net.lr .org.ls .co.ls .gov.lt .mil.lt .gov.lu .mil.lu .org.lu .net.lu .com.lv .edu.lv .gov.lv .org.lv .mil.lv .id.lv .net.lv .asn.lv .conf.lv .com.ly .net.ly .gov.ly .plc.ly .edu.ly .sch.ly .med.ly .org.ly .id.ly .co.ma .net.ma .gov.ma .org.ma .tm.mc .asso.mc .org.mg .nom.mg .gov.mg .prd.mg .tm.mg .com.mg .edu.mg .mil.mg .com.mk .org.mk .com.mo .net.mo .org.mo .edu.mo .gov.mo .org.mt .com.mt .gov.mt .edu.mt .net.mt .com.mu .co.mu .aero.mv .biz.mv .com.mv .coop.mv .edu.mv .gov.mv .info.mv .int.mv .mil.mv .museum.mv .name.mv .net.mv .org.mv .pro.mv .com.mx .net.mx .org.mx .edu.mx .gob.mx .com.my .net.my .org.my .gov.my .edu.my .mil.my .name.my .edu.ng .com.ng .gov.ng .org.ng .net.ng .gob.ni .com.ni .edu.ni .org.ni .nom.ni .net.ni .gov.nr .edu.nr .biz.nr .info.nr .com.nr .net.nr .ac.nz .co.nz .cri.nz .gen.nz .geek.nz .govt.nz .iwi.nz .maori.nz .mil.nz .net.nz .org.nz .school.nz .com.pf .org.pf .edu.pf .com.pg .net.pg .com.ph .gov.ph .com.pk .net.pk .edu.pk .org.pk .fam.pk .biz.pk .web.pk .gov.pk .gob.pk .gok.pk .gon.pk .gop.pk .gos.pk .com.pl .biz.pl .net.pl .art.pl .edu.pl .org.pl .ngo.pl .gov.pl .info.pl .mil.pl .waw.pl .warszawa.pl .wroc.pl .wroclaw.pl .krakow.pl .poznan.pl .lodz.pl .gda.pl .gdansk.pl .slupsk.pl .szczecin.pl .lublin.pl .bialystok.pl .olsztyn.pl .torun.pl .biz.pr .com.pr .edu.pr .gov.pr .info.pr .isla.pr .name.pr .net.pr .org.pr .pro.pr .edu.ps .gov.ps .sec.ps .plo.ps .com.ps .org.ps .net.ps .com.pt .edu.pt .gov.pt .int.pt .net.pt .nome.pt .org.pt .publ.pt .net.py .org.py .gov.py .edu.py .com.py .com.ru .net.ru .org.ru .pp.ru .msk.ru .int.ru .ac.ru .gov.rw .net.rw .edu.rw .ac.rw .com.rw .co.rw .int.rw .mil.rw .gouv.rw .com.sa .edu.sa .sch.sa .med.sa .gov.sa .net.sa .org.sa .pub.sa .com.sb .gov.sb .net.sb .edu.sb .com.sc .gov.sc .net.sc .org.sc .edu.sc .com.sd .net.sd .org.sd .edu.sd .med.sd .tv.sd .gov.sd .info.sd .org.se .pp.se .tm.se .parti.se .press.se .ab.se .c.se .d.se .e.se .f.se .g.se .h.se .i.se .k.se .m.se .n.se .o.se .s.se .t.se .u.se .w.se .x.se .y.se .z.se .ac.se .bd.se .com.sg .net.sg .org.sg .gov.sg .edu.sg .per.sg .idn.sg .edu.sv .com.sv .gob.sv .org.sv .red.sv .gov.sy .com.sy .net.sy .ac.th .co.th .in.th .go.th .mi.th .or.th .net.th .ac.tj .biz.tj .com.tj .co.tj .edu.tj .int.tj .name.tj .net.tj .org.tj .web.tj .gov.tj .go.tj .mil.tj .com.tn .intl.tn .gov.tn .org.tn .ind.tn .nat.tn .tourism.tn .info.tn .ens.tn .fin.tn .net.tn .gov.to .gov.tp .com.tr .info.tr .biz.tr .net.tr .org.tr .web.tr .gen.tr .av.tr .dr.tr .bbs.tr .name.tr .tel.tr .gov.tr .bel.tr .pol.tr .mil.tr .k12.tr .edu.tr .co.tt .com.tt .org.tt .net.tt .biz.tt .info.tt .pro.tt .name.tt .edu.tt .gov.tt .gov.tv .edu.tw .gov.tw .mil.tw .com.tw .net.tw .org.tw .idv.tw .game.tw .ebiz.tw .club.tw .co.tz .ac.tz .go.tz .or.tz .ne.tz .com.ua .gov.ua .net.ua .edu.ua .org.ua .cherkassy.ua .ck.ua .chernigov.ua .cn.ua .chernovtsy.ua .cv.ua .crimea.ua .dnepropetrovsk.ua .dp.ua .donetsk.ua .dn.ua .if.ua .kharkov.ua .kh.ua .kherson.ua .ks.ua .khmelnitskiy.ua .km.ua .kiev.ua .kv.ua .kirovograd.ua .kr.ua .lugansk.ua .lg.ua .lutsk.ua .lviv.ua .nikolaev.ua .mk.ua .odessa.ua .od.ua .poltava.ua .pl.ua .rovno.ua .rv.ua .sebastopol.ua .sumy.ua .ternopil.ua .te.ua .uzhgorod.ua .vinnica.ua .vn.ua .zaporizhzhe.ua .zp.ua .zhitomir.ua .zt.ua .co.ug .ac.ug .sc.ug .go.ug .ne.ug .or.ug .ac.uk .co.uk .gov.uk .ltd.uk .me.uk .mil.uk .mod.uk .net.uk .nic.uk .nhs.uk .org.uk .plc.uk .police.uk .bl.uk .icnet.uk .jet.uk .nel.uk .nls.uk .parliament.uk .sch.uk .ak.us .al.us .ar.us .az.us .ca.us .co.us .ct.us .dc.us .de.us .dni.us .fed.us .fl.us .ga.us .hi.us .ia.us .id.us .il.us .in.us .isa.us .kids.us .ks.us .ky.us .la.us .ma.us .md.us .me.us .mi.us .mn.us .mo.us .ms.us .mt.us .nc.us .nd.us .ne.us .nh.us .nj.us .nm.us .nsn.us .nv.us .ny.us .oh.us .ok.us .or.us .pa.us .ri.us .sc.us .sd.us .tn.us .tx.us .ut.us .vt.us .va.us .wa.us .wi.us .wv.us .wy.us .edu.uy .gub.uy .org.uy .com.uy .net.uy .mil.uy .com.ve .net.ve .org.ve .info.ve .co.ve .web.ve .com.vi .org.vi .edu.vi .gov.vi .com.vn .net.vn .org.vn .edu.vn .gov.vn .int.vn .ac.vn .biz.vn .info.vn .name.vn .pro.vn .health.vn .com.ye .net.ye .ac.yu .co.yu .org.yu .edu.yu .ac.za .city.za .co.za .edu.za .gov.za .law.za .mil.za .nom.za .org.za .school.za .alt.za .net.za .ngo.za .tm.za .web.za .co.zm .org.zm .gov.zm .sch.zm .ac.zm .co.zw .org.zw .gov.zw .ac.zw .com.ac .edu.ac .gov.ac .net.ac .mil.ac .org.ac .nom.ad .net.ae .co.ae .gov.ae .ac.ae .sch.ae .org.ae .mil.ae .pro.ae .name.ae .com.ag .org.ag .net.ag .co.ag .nom.ag .off.ai .com.ai .net.ai .org.ai .gov.al .edu.al .org.al .com.al .net.al .com.am .net.am .org.am .com.ar .net.ar .org.ar .e164.arpa .ip6.arpa .uri.arpa .urn.arpa .gv.at .ac.at .co.at .or.at .com.au .net.au .asn.au .org.au .id.au .csiro.au .gov.au .edu.au .com.aw .com.az .net.az .org.az .com.bb .edu.bb .gov.bb .net.bb .org.bb .com.bd .edu.bd .net.bd .gov.bd .org.bd .mil.be .ac.be .gov.bf .com.bm .edu.bm .org.bm .gov.bm .net.bm .com.bn .edu.bn .org.bn .net.bn .com.bo .org.bo .net.bo .gov.bo .gob.bo .edu.bo .tv.bo .mil.bo .int.bo .agr.br .am.br .art.br .edu.br .com.br .coop.br .esp.br .far.br .fm.br .g12.br .gov.br .imb.br .ind.br .inf.br .mil.br .net.br .org.br .psi.br .rec.br .srv.br .tmp.br .tur.br .tv.br .etc.br .adm.br .adv.br .arq.br .ato.br .bio.br .bmd.br .cim.br .cng.br .cnt.br .ecn.br .eng.br .eti.br .fnd.br .fot.br .fst.br .ggf.br .jor.br .lel.br .mat.br .med.br .mus.br .not.br .ntr.br .odo.br .ppg.br .pro.br .psc.br .qsl.br .slg.br .trd.br .vet.br .zlg.br .dpn.br .nom.br .com.bs .net.bs .org.bs .com.bt .edu.bt .gov.bt .net.bt .org.bt .co.bw .org.bw .gov.by .mil.by .ac.cr .co.cr .ed.cr .fi.cr .go.cr .or.cr .sa.cr .com.cy .biz.cy .info.cy .ltd.cy .pro.cy .net.cy .org.cy .name.cy .tm.cy .ac.cy .ekloges.cy .press.cy .parliament.cy .com.dm .net.dm .org.dm .edu.dm .gov.dm .biz.fj .com.fj .info.fj .name.fj .net.fj .org.fj .pro.fj .ac.fj .gov.fj .mil.fj .school.fj .com.gh .edu.gh .gov.gh .org.gh .mil.gh .co.hu .info.hu .org.hu .priv.hu .sport.hu .tm.hu .2000.hu .agrar.hu .bolt.hu .casino.hu .city.hu .erotica.hu .erotika.hu .film.hu .forum.hu .games.hu .hotel.hu .ingatlan.hu .jogasz.hu .konyvelo.hu .lakas.hu .media.hu .news.hu .reklam.hu .sex.hu .shop.hu .suli.hu .szex.hu .tozsde.hu .utazas.hu .video.hu .ac.id .co.id .or.id .go.id .ac.il .co.il .org.il .net.il .k12.il .gov.il .muni.il .idf.il .co.im .net.im .gov.im .org.im .nic.im .ac.im .org.jm .ac.jp .ad.jp .co.jp .ed.jp .go.jp .gr.jp .lg.jp .ne.jp .or.jp .hokkaido.jp .aomori.jp .iwate.jp .miyagi.jp .akita.jp .yamagata.jp .fukushima.jp .ibaraki.jp .tochigi.jp .gunma.jp .saitama.jp .chiba.jp .tokyo.jp .kanagawa.jp .niigata.jp .toyama.jp .ishikawa.jp .fukui.jp .yamanashi.jp .nagano.jp .gifu.jp .shizuoka.jp .aichi.jp .mie.jp .shiga.jp .kyoto.jp .osaka.jp .hyogo.jp .nara.jp .wakayama.jp .tottori.jp .shimane.jp .okayama.jp .hiroshima.jp .yamaguchi.jp .tokushima.jp .kagawa.jp .ehime.jp .kochi.jp .fukuoka.jp .saga.jp .nagasaki.jp .kumamoto.jp .oita.jp .miyazaki.jp .kagoshima.jp .okinawa.jp .sapporo.jp .sendai.jp .yokohama.jp .kawasaki.jp .nagoya.jp .kobe.jp .kitakyushu.jp .per.kh .com.kh .edu.kh .gov.kh .mil.kh .net.kh .org.kh .net.lb .org.lb .gov.lb .edu.lb .com.lb .com.lc .org.lc .edu.lc .gov.lc .army.mil .navy.mil .weather.mobi .music.mobi .ac.mw .co.mw .com.mw .coop.mw .edu.mw .gov.mw .int.mw .museum.mw .net.mw .org.mw .mil.no .stat.no .kommune.no .herad.no .priv.no .vgs.no .fhs.no .museum.no .fylkesbibl.no .folkebibl.no .idrett.no .com.np .org.np .edu.np .net.np .gov.np .mil.np .org.nr .com.om .co.om .edu.om .ac.com .sch.om .gov.om .net.om .org.om .mil.om .museum.om .biz.om .pro.om .med.om .com.pa .ac.pa .sld.pa .gob.pa .edu.pa .org.pa .net.pa .abo.pa .ing.pa .med.pa .nom.pa .com.pe .org.pe .net.pe .edu.pe .mil.pe .gob.pe .nom.pe .law.pro .med.pro .cpa.pro .vatican.va .ac .ad .ae .aero .af .ag .ai .al .am .an .ao .aq .ar .arpa .as .at .au .aw .az .ba .bb .bd .be .bf .bg .bh .bi .biz .bj .bm .bn .bo .br .bs .bt .bv .bw .by .bz .ca .cat .cc .cd .cf .cg .ch .ci .ck .cl .cm .cn .co .com .coop .cr .cu .cv .cx .cy .cz .de .dj .dk .dm .do .dz .ec .edu .ee .eg .er .es .et .eu .fi .fj .fk .fm .fo .fr .ga .gb .gd .ge .gf .gg .gh .gi .gl .gm .gov .gp .gq .gr .gs .gt .gu .gw .gy .hk .hm .hn .hr .ht .hu .id .ie .il .im .in .info .int .io .iq .ir .is .it .je .jm .jo .jobs .jp .ke .kg .kh .ki .km .kn .kr .kw .ky .kz .la .lb .lc .li .lk .lr .ls .lt .lu .lv .ly .ma .mc .md .mg .mh .mil .mk .ml .mm .mn .mo .mobi .mp .mq .mr .ms .mt .mu .museum .mv .mw .na .name .nc .ne .net .nf .ng .ni .nl .no .np .nr .nu .nz .om .org .pa .pe .pf .pg .ph .pk .pl .pm .pn .post .pr .pro .ps .pt .pw .py .qa .re .ro .ru .rw .sa .sb .sc .sd .se .sg .sh .si .sj .sk .sl .sm .sn .so .sr .st .su .sv .sy .sz .tc .td .tf .tg .th .tj .tk .tl .tm .tn .to .tp .tr .travel .tt .tv .tw .tz .ua .ug .uk .um .us .uy .uz .va .vc .ve .vg .vi .vn .vuwf .ye .yt .yu .za .zm .zw .ca .cd .ch .cn .cu .cx .dm .dz .ec .ee .es .fr .ge .gg .gi .gr .hk .hn .hr .ht .hu .ie .in .ir .it .je .jo .jp .kr .ky .li .lk .lt .lu .lv .ly .ma .mc .mg .mk .mo .mt .mu .nl .no .nr .nr .pf .ph .pk .pl .pr .ps .pt .ro .ru .rw .sc .sd .se .sg .tj .to .to .tt .tv .tw .tw .tw .tw .ua .ug .us .vi .vn";
$tld_regex = '#(.*?)([^.]+)('.str_replace(array('.',' '),array('\\.','|'),$valid_tlds).')$#';
//remove the extension
preg_match($tld_regex,$host,$matches);
if(!empty($matches) && sizeof($matches) > 2){
$extension = array_pop($matches);
$tld = array_pop($matches);
return $tld.$extension;
}else{ //change to "false" if you prefer
return $host;
}
}
Here is what I am using:
It works great without needing any arrays for tld's
$split = array_reverse(explode(".", $_SERVER['HTTP_HOST']));
$domain = $split[1].".".$split[0];
if(function_exists('gethostbyname'))
{
if(gethostbyname($domain) != $_SERVER['SERVER_ADDR'] && isset($split[2]))
{
$domain = $split[2].".".$split[1].".".$split[0];
}
}
It is not possible without using a TLD list to compare with as their exist many cases like
http://www.db.de/ or http://bbc.co.uk/
But even with that you won't have success in every case because of SLD's like http://big.uk.com/ or http://www.uk.com/
If you need a complete list you can use the public suffix list:
http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/effective_tld_names.dat?raw=1
Feel free to use my function. It won't use regex and it is fast:
http://www.programmierer-forum.de/domainnamen-ermitteln-t244185.htm#3471878
Here is how you strip the TLD from any URL - I wrote the code to work on my site: http://internet-portal.me/ - This is a working solution that is used on my site.
$host is the URL that has to be parsed. This code is a simple solution and reliable
compared to everything else I have seen, It works on any URL that I have tried!!!
see this code parsing the page you are looking at right now!
http://internet-portal.me/domain/?dns=https://stackoverflow.com/questions/1201194/php-getting-domain-name-from-subdomain/6320437#6320437
================================================================================
$host = filter_var($_GET['dns']);
$host = $host . '/'; // needed if URL does not have trailing slash
// Strip www, http, https header ;
$host = str_replace( 'http://www.' , '' , $host );
$host = str_replace( 'https://www.' , '' , $host );
$host = str_replace( 'http://' , '' , $host );
$host = str_replace( 'https://' , '' , $host );
$pos = strpos($host, '/'); // find any sub directories
$host = substr( $host, 0, $pos ); //strip directories
$hostArray = explode (".", $host); // count parts of TLD
$size = count ($hostArray) -1; // really only need to know if not a single level TLD
$tld = $hostArray[$size]; // do we need to parse the TLD any further -
// remove subdomains?
if ($size > 1) {
if ($tld == "aero" or $tld == "asia" or $tld == "biz" or $tld == "cat" or
$tld == "com" or $tld == "coop" or $tld == "edu" or $tld == "gov" or
$tld == "info" or $tld == "int" or $tld == "jobs" or $tld == "me" or
$tld == "mil" or $tld == "mobi" or $tld == "museum" or $tld == "name" or
$tld == "net" or $tld == "org" or $tld == "pro" or $tld == "tel" or
$tld == "travel" or $tld == "tv" or $tld == "ws" or $tld == "XXX") {
$host = $hostArray[$size -1].".".$hostArray[$size]; // parse to 2 level TLD
} else {
// parse to 3 level TLD
$host = $hostArray[$size -2].".".$hostArray[$size -1].".".$hostArray[$size] ;
}
}
No need for listing all the countries TLD, they are all 2 letters, besides the special ones listed by IANA
https://gist.github.com/pocesar/5366899
and the tests are here http://codepad.viper-7.com/QfueI0
Comprehensive test suit along with working code. The only caveat is that it won't work with unicode domain names, but that's another level of data extraction.
From the list, I'm testing against:
$urls = array(
'www.example.com' => 'example.com',
'example.com' => 'example.com',
'example.com.br' => 'example.com.br',
'www.example.com.br' => 'example.com.br',
'www.example.gov.br' => 'example.gov.br',
'localhost' => 'localhost',
'www.localhost' => 'localhost',
'subdomain.localhost' => 'localhost',
'www.subdomain.example.com' => 'example.com',
'subdomain.example.com' => 'example.com',
'subdomain.example.com.br' => 'example.com.br',
'www.subdomain.example.com.br' => 'example.com.br',
'www.subdomain.example.biz.br' => 'example.biz.br',
'subdomain.example.biz.br' => 'example.biz.br',
'subdomain.example.net' => 'example.net',
'www.subdomain.example.net' => 'example.net',
'www.subdomain.example.co.kr' => 'example.co.kr',
'subdomain.example.co.kr' => 'example.co.kr',
'example.co.kr' => 'example.co.kr',
'example.jobs' => 'example.jobs',
'www.example.jobs' => 'example.jobs',
'subdomain.example.jobs' => 'example.jobs',
'insane.subdomain.example.jobs' => 'example.jobs',
'insane.subdomain.example.com.br' => 'example.com.br',
'www.doubleinsane.subdomain.example.com.br' => 'example.com.br',
'www.subdomain.example.jobs' => 'example.jobs',
'test' => 'test',
'www.test' => 'test',
'subdomain.test' => 'test',
'www.detran.sp.gov.br' => 'sp.gov.br',
'www.mp.sp.gov.br' => 'sp.gov.br',
'ny.library.museum' => 'library.museum',
'www.ny.library.museum' => 'library.museum',
'ny.ny.library.museum' => 'library.museum',
'www.library.museum' => 'library.museum',
'info.abril.com.br' => 'abril.com.br',
'127.0.0.1' => '127.0.0.1',
'::1' => '::1',
);
As already said Public Suffix List is only one way to parse domain correctly. I recomend TLDExtract package, here is sample code:
$extract = new LayerShifter\TLDExtract\Extract();
$result = $extract->parse('here.example.com');
$result->getSubdomain(); // will return (string) 'here'
$result->getHostname(); // will return (string) 'example'
$result->getSuffix(); // will return (string) 'com'
This is to get domain.tld in any case
public static function get_domain_with_extension($url)
{
$pieces = parse_url($url);
$domain = isset($pieces['host']) ? $pieces['host'] : $pieces['path'];
if (preg_match('/(?P<domain>[a-z0-9][a-z0-9\-]{1,63}\.[a-z\.]{2,6})$/i', $domain, $regs)) {
return $regs['domain'];
}
return false;
}
NO NEED FOR REGEX. There exists native parse_url:
echo parse_url($your_url)['host'];