I am getting a weird result for the client IP in PHP in some cases.
Result in Most Cases (Expected Result) :
192.123.132.123
Erroneous Result Type 1:
for="192.123.132.123"
Erroneous Result Type 2:
for="192.123.132.123:1232"
Code for getting the IP:
<?php
function getIP(){
$ip = isset($_SERVER['REMOTE_ADDR']) ? $_SERVER['REMOTE_ADDR'] : '-';
$proxy = false;
if (!empty($_SERVER['HTTP_VIA']) || !empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
$proxy = true;
} elseif (!empty($_SERVER['REMOTE_HOST'])) {
$aProxyHosts = array('proxy','cache','inktomi');
foreach ($aProxyHosts as $proxyName) {
if (strpos($_SERVER['REMOTE_HOST'], $proxyName) !== false) {
$proxy = true;
break;
}
}
}
// Has the viewer come via an HTTP proxy?
if ($proxy) {
// Try to find the "real" IP address the viewer has come from
$aHeaders = array('HTTP_FORWARDED','HTTP_FORWARDED_FOR','HTTP_X_FORWARDED','HTTP_X_FORWARDED_FOR','HTTP_CLIENT_IP');
foreach ($aHeaders as $header) {
if (!empty($_SERVER[$header])) {
$ip = $_SERVER[$header];
break;
}
}
}
if (!empty($ip)) {
// The "remote IP" may be a list, ensure that
// only the last item is used in that case
$ip = explode(',', $ip);
$ip = trim($ip[count($ip) - 1]);
}
return $ip;
}
?>
I know that I can clean the result to get the correct value (IP) but I am puzzled at why is this happening in the first place.
PS: 192.123.132.123 is an arbitrary IP used to explain the issue.
You're reading arbitrary HTTP headers... not all of them contain purely the IP, some are in the form of for=... and some include the port as well.
Using any HTTP header instead $_SERVER['REMOTE_ADDR'] means you're allowing anyone to mask/fake their IP address by simply sending an HTTP header. You should be perfectly aware of where such headers may be set, which usually means you know they're set by a proxy you control. In this case you obviously don't know where those headers are coming from, so you should not use them.
If you decide to use an HTTP header, you should know which one exactly you want to read and what format it's in. If its format is for=..., then parse that format correctly.
Related
I'm calling through Axios a PHP script checking whether a URL passed to it as a parameter can be embedded in an iframe. That PHP script starts with opening the URL with $_GET[].
Strangely, a page with cross-origin-opener-policy: same-origin (like https://twitter.com/) can be opened with $_GET[], whereas a page with Referrer Policy: strict-origin-when-cross-origin (like https://calia.order.liven.com.au/) cannot.
I don't understand why, and it's annoying because for the pages that cannot be opened with $_GET[] I'm unable to perform my checks on them - the script just fails (meaning I get no response and the Axios call runs the catch() block).
So basically there are 3 types of pages: (1) those who allow iframe embeddability, (2) those who don't, and (3) the annoying ones who not only don't but also can't even be opened to perform this check.
Is there a way to open any page with PHP, and if not, what can I do to prevent my script from failing after several seconds?
PHP script:
$source = $_GET['url'];
$response = true;
try {
$headers = get_headers($source, 1);
$headers = array_change_key_case($headers, CASE_LOWER);
if (isset($headers['content-security-policy'])) {
$response = false;
}
else if (isset($headers['x-frame-options']) &&
$headers['x-frame-options'] == 'DENY' ||
$headers['x-frame-options'] == 'SAMEORIGIN'
) {
$response = false;
}
} catch (Exception $ex) {
$response = $ex;
}
echo $response;
EDIT: below is the console error.
Access to XMLHttpRequest at 'https://path.to.cdn/iframeHeaderChecker?url=https://calia.order.liven.com.au/' from origin 'http://localhost:3000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
CustomLink.vue?b495:61 Error: Network Error
at createError (createError.js?2d83:16)
at XMLHttpRequest.handleError (xhr.js?b50d:84)
VM4758:1 GET https://path.to.cdn/iframeHeaderChecker?url=https://calia.order.com.au/ net::ERR_FAILED
The error you have shown is coming from Javascript, not from PHP. get_headers() returns false on failure, it will not throw an exception - the catch() never happens. get_headers() just makes an http request, like your browser, or curl, and the only reason that would fail is if the URL is malformed, or the remote site is down, etc.
It is the access from http://localhost:3000 to https://path.to.cdn/iframeHeaderChecker with Javascript that has been blocked, not PHP access to the URLs you are passing as parameters in $_GET['url'].
What you're seeing is a standard CORS error when you try to access a different domain than the one the Javascript is running on. CORS means Javascript running on one host cannot make http requests to another host, unless that other host explicitly allows it. In this case, the Javascript running at http://localhost:3000 is making an http request to a remote site https://path.to.cdn/. That's a cross-origin request (localhost !== path.to.cdn), and the server/script receiving that request on path.to.cdn is not returning any specific CORS headers allowing that request, so the request is blocked.
Note though that if the request is classed as "simple", it will actually run. So your PHP is working already, always, but bcs the right headers aren't returned, the result is blocked from being displayed in your browser. This can lead to confusion bcs for eg you might notice a delay while it gets the headers from a slow site, whereas it is super fast for a fast site. Or maybe you have logging which you see is working all the time, despite nothing showing up in your browser.
My understanding is that https://path.to.cdn/iframeHeaderChecker is your PHP script, some of the code of which you have shown in your question? If so, you have 2 choices:
Update iframeHeaderChecker to return the appropriate CORS headers, so that your cross-origin JS request is allowed. As a quick, insecure hack to allow access from anyone and anywhere (not a good idea for the long term!) you could add:
header("Access-Control-Allow-Origin: *");
But it would be better to update that to more specifically restrict access to only your app, and not everyone else. You'll have to evaluate the best way to do that depending on the specifics of your application and infrastructure. There many questions here on SO about CORS/PHP/AJAX to check for reference. You could also configure this at the web server level, rather than the application level, eg here's how to configure Apache to return those headers.
If iframeHeaderChecker is part of the same application as the Javascript calling it, is it also available locally, on http://localhost:3000? If so, update your JS to use the local version, not the remote one on path.to.cdn, and you avoid the whole problem!
This is just my rough guess about what wrong with your code can be.
I noticed you do:
a comparison of values from $headers but without
ensuring they have the same CAPITAL CASE as the values you compare against. Applied: strtoupper().
check with isset() but not test if key_exist before
Applied: key_exist()
check with isset() but perhaps you should use !empty() instead of isset()
compare result:
$value = "";
var_dump(isset($value)); // (bool) true
var_dump(!empty($value)); // (bool) false
$value = "something";
var_dump(isset($value)); // (bool) true
var_dump(!empty($value)); // (bool) true
unset($value);
var_dump(isset($value)); // (bool) false
var_dump(!empty($value)); // (bool) false
The code with applied changes:
<?php
error_reporting(E_ALL);
declare(strict_types=1);
header('Access-Control-Allow-Origin: *');
ob_start();
try {
$response = true;
if (!key_exists('url', $_GET)) {
$msg = '$_GET does not have a key "url"';
throw new \RuntimeException($msg);
}
$source = $_GET['url'];
if ($source !== filter_var($source, \FILTER_SANITIZE_URL)) {
$msg = 'Passed url is invaid, url: ' . $source;
throw new \RuntimeException($msg);
}
if (filter_var($source, \FILTER_VALIDATE_URL) === FALSE) {
$msg = 'Passed url is invaid, url: ' . $source;
throw new \RuntimeException($msg);
}
$headers = get_headers($source, 1);
if (!is_array($headers)) {
$msg = 'Headers should be array but it is: ' . gettype($headers);
throw new \RuntimeException($msg);
}
$headers = array_change_key_case($headers, \CASE_LOWER);
if ( key_exists('content-security-policy', $headers) &&
isset($headers['content-security-policy'])
) {
$response = false;
}
elseif ( key_exists('x-frame-options', $headers) &&
(
strtoupper($headers['x-frame-options']) == 'DENY' ||
strtoupper($headers['x-frame-options']) == 'SAMEORIGIN'
)
) {
$response = false;
}
} catch (Exception $ex) {
$response = "Error: " . $ex->getMessage() . ' at: ' . $ex->getFile() . ':' . $ex->getLine();
}
$phpOutput = ob_get_clean();
if (!empty($phpOutput)) {
$response .= \PHP_EOL . 'PHP Output: ' . $phpOutput;
}
echo $response;
Using Throwable instead of Exception will also catch Errors in PHP7.
Keep in mind that:
$response = true;
echo $response; // prints "1"
but
$response = false;
echo $response; // prints ""
so for the $response = false you'll get an empty string, not 0
if you want to have 0 for false and 1 for true then change the $response = true; to $response = 1; for true and $response = false; to $response = 0; for false everywhere.
I hope that somehow helps
I've installed a Tor relay and Nginx and created my .onion on my Linux server.
In torrc HiddenServicePort 80 127.0.0.1:8747
In nginx's default: listen 8747
I've modified TorDNSExitList's PHP Pear Net_DNS to use Net_DNS2. When I echo out the $ip, $myip, $myport I get:
ip = 127.0.0.1
my ip = 127.0.0.1
port = 8747
Thus it is picking the IP address as the local machine and not the Tor exit node's IP address. Is there another why to test if the page is access via the Tor network?
(I've also tried this suggestion)
The solution is to check for 127.0.0.1 IP address, seeing that torrc points to 127.0.0.1. This works when accessing the website via the .onion path. But the full check still needs to be done as the website can be access via the full URL, e.g. http:// [IP Address]:[Port] - using a "normal" or Tor browser. My changes to the function below:
<?php include("Net/DNS2.php");
// torel_check ($ip, $port, $destip) queries the Tor DNS Exit List server.
// The result of the query is one of the following:
// -1 : DNS lookup failed to get a response, or other error occurred.
// 0 : $ip does not appear to be a Tor exit.
// 1 : $ip is a known Tor exit for the provided destination IP / port.
function revaddr ($ip) {
list($a, $b, $c, $d) = split("[.]", $ip);
return("${d}.${c}.${b}.${a}");
}
function torel_qh ($ip, $port, $destip) {
$rsrcip = revaddr ($ip);
$rdstip = revaddr ($destip);
return("${rsrcip}.${port}.${rdstip}.ip-port.exitlist.torproject.org");
}
function torel_check ($ip, $port, $destip) {
try{
if($ip == "127.0.0.1") {
//TX: Access via .onion path
// is Tor exit
return (1);
}
//TX: Access web site directly
$ndr = new Net_DNS2_Resolver();
$qh = torel_qh($ip, $port, $destip);
// uncomment these two lines to query the server directly...
//$ns = "exitlist-ns.torproject.org";
//$ndr->nameservers( array($ns) );
// tune DNS params accordingly. this is just my preference.
$ndr->retrans = 2;
$ndr->retry = 3;
$ndr->usevc = 0;
// perform DNS query
// TX: Old Net_DNS check $ndr->search($qh)
if (! $pkt = $ndr->query($qh)) {
if (strcmp($ndr->errorstring, "NXDOMAIN") == 0) {
// response but no answer. does not appear to be Tor exit.
return (0);
}
// search failed: no response or other problem...
return(-1);
}
if (! isset($pkt->answer[0])) {
// response but no answer section. does not appear to be Tor exit.
// (this should only happen when authority sections are provided without answer)
return(0);
}
// is Tor exit
return(1);
} catch(Net_DNS2_Exception $e) {
return (-1);
}
}
// get client request parameters from Apache or equiv server:
$ip = $myip = $myport = 0;
if (isset ($_SERVER["REMOTE_ADDR"])) { $ip = $_SERVER["REMOTE_ADDR"]; }
if (isset ($_SERVER["SERVER_ADDR"])) { $myip = $_SERVER["SERVER_ADDR"]; }
if (isset ($_SERVER["SERVER_PORT"])) { $myport = $_SERVER["SERVER_PORT"]; }
$istor = torel_check($ip, $myport, $myip);
TX: is my comments
I'm looking for good code in PHP for Banning some spammers IP's My server is giving me error 500 if I'm using .htaccess
This will do the work
$getip = $_SERVER["REMOTE_ADDR"];
$banned_ip = array();
$banned_ip[] = '194.9.94.*';
$banned_ip[] = '77.105.2.*';
foreach($banned_ip as $banned)
{
$blacked=str_replace('*', '', $banned);
$len=strlen($blacked);
if ($getip==$blacked || substr($getip, 0, $len)==$blacked)
{
$_banned_ip=true;
}
}
if($_banned_ip==true){
echo 'THIS IP IS BANNED!';
exit;
}
The simplest way would be to have a database that keeps a list of the banned ip addresses, if you want to do it on the PHP end rather than directly in the server.
for($i = 0;$i < count($listOfIps);$i++) {
if($listOfIps[$i] == filteredIP($_SERVER['REMOTED_ADDR'])) { //filteredIP is not a native function, it's just a representation of however you want to filter the ip addresses which are sent to you
$banned = true;
}
}
if($banned):
//redirect user or kill script
else:
//render page
endif;
However, there may be better solutions based on the page or application specifics, but this is the best solution I can think of based on your question
I use a javascript API from (http://www.iplocationtools.com) to give me the location data from a visitors IP. For some reason, their API won't give me the actual IP of the visitor, just the other info, which is why I have to use PHP and CodeIgniter to give me the IP instead.
So I use CodeIgniter/PHP to get the IP of a visitor and add it to the database along with location data from above by using PHP's ip2long([the ip code igniter gives me])
I'm confused when my database table looks like this: http://pulse.media.mit.edu/images/1.png
Which is wrong? I'm tempted to believe CodeIgniter is wrong since it gives me the same IP so many times. Age and sex are self reported and I doubt one person is making up all this information.
At the end of the day, all we really need is the users IP and location, preferably from the same source, so we don't compound errors.
Anybody have a better idea on how to do this?
EDIT: Here is the code that I'm using to get the IP address from CodeIgniter
$data['ip_address'] = ip2long($this->input->ip_address());
$this->pulse_model->voter_info($data);
Then the voter_info function just inserts it into the database where it's stored as an INT(11).
And here is the function ip_address:
function ip_address()
{
if ($this->ip_address !== FALSE)
{
return $this->ip_address;
}
if (config_item('proxy_ips') != '' && $this->server('HTTP_X_FORWARDED_FOR') && $this->server('REMOTE_ADDR'))
{
$proxies = preg_split('/[\s,]/', config_item('proxy_ips'), -1, PREG_SPLIT_NO_EMPTY);
$proxies = is_array($proxies) ? $proxies : array($proxies);
$this->ip_address = in_array($_SERVER['REMOTE_ADDR'], $proxies) ? $_SERVER['HTTP_X_FORWARDED_FOR'] : $_SERVER['REMOTE_ADDR'];
}
elseif ($this->server('REMOTE_ADDR') AND $this->server('HTTP_CLIENT_IP'))
{
$this->ip_address = $_SERVER['HTTP_CLIENT_IP'];
}
elseif ($this->server('REMOTE_ADDR'))
{
$this->ip_address = $_SERVER['REMOTE_ADDR'];
}
elseif ($this->server('HTTP_CLIENT_IP'))
{
$this->ip_address = $_SERVER['HTTP_CLIENT_IP'];
}
elseif ($this->server('HTTP_X_FORWARDED_FOR'))
{
$this->ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
}
if ($this->ip_address === FALSE)
{
$this->ip_address = '0.0.0.0';
return $this->ip_address;
}
if (strpos($this->ip_address, ',') !== FALSE)
{
$x = explode(',', $this->ip_address);
$this->ip_address = trim(end($x));
}
if ( ! $this->valid_ip($this->ip_address))
{
$this->ip_address = '0.0.0.0';
}
return $this->ip_address;
}
$_SERVER['REMOTE_ADDR'] is the PHP code to return the IP address of the person viewing the page.
Old versions of ip2long() will return -1 if the IPv4 address is invalid. You may want to use inet_pton instead and expand the field used to hold it to 128 bits.
Based on your code, it looks like $this->input->ip_address() has the possibility of returning '0.0.0.0' if the IP is not valid or could not be determined. However, your comments also state that you need to record the ip address even if the above method returns '0.0.0.0'.
First, I'd recommend checking to see if $this->input->ip_address() and $this->valid_ip() are working as expected. Is $this->valid_ip() returning false for IP's that should be considered valid?
Second, I'd update your code to always fall back to $_SERVER['REMOTE_ADDR'] if $this->input->ip_address() returns '0.0.0.0'.
$ip_address = $this->input->ip_address();
if($ip_address == '0.0.0.0') {
$ip_address = $_SERVER['REMOTE_ADDR'];
}
$data['ip_address'] = ip2long($ip_address);
$this->pulse_model->voter_info($data);
Or if you wanted, you could not use $this->input->ip_address() and do as #rockerest suggests and just use $_SERVER['REMOTE_ADDR'] to being with.
$data['ip_address'] = ip2long($_SERVER['REMOTE_ADDR']);
$this->pulse_model->voter_info($data);
I'm going to block all bots except the big search engines. One of my blocking methods will be to check for "language": Accept-Language: If it has no Accept-Language the bot's IP address will be blocked until 2037. Googlebot does not have Accept-Language, I want to verify it with DNS lookup
<?php
gethostbyaddr($_SERVER['REMOTE_ADDR']);
?>
Is it ok to use gethostbyaddr, can someone pass my "gethostbyaddr protection"?
function detectSearchBot($ip, $agent, &$hostname)
{
$hostname = $ip;
// check HTTP_USER_AGENT what not to touch gethostbyaddr in vain
if (preg_match('/(?:google|yandex)bot/iu', $agent)) {
// success - return host, fail - return ip or false
$hostname = gethostbyaddr($ip);
// https://support.google.com/webmasters/answer/80553
if ($hostname !== false && $hostname != $ip) {
// detect google and yandex search bots
if (preg_match('/\.((?:google(?:bot)?|yandex)\.(?:com|ru))$/iu', $hostname)) {
// success - return ip, fail - return hostname
$ip = gethostbyname($hostname);
if ($ip != $hostname) {
return true;
}
}
}
}
return false;
}
In my project, I use this function to identify Google and Yandex search bots.
The result of the detectSearchBot function is caching.
The algorithm is based on Google’s recommendation - https://support.google.com/webmasters/answer/80553
In addition to Cristian's answer:
function is_valid_google_ip($ip) {
$hostname = gethostbyaddr($ip); //"crawl-66-249-66-1.googlebot.com"
return preg_match('/\.googlebot|google\.com$/i', $hostname);
}
function is_valid_google_request($ip=null,$agent=null){
if(is_null($ip)){
$ip=$_SERVER['REMOTE_ADDR'];
}
if(is_null($agent)){
$agent=$_SERVER['HTTP_USER_AGENT'];
}
$is_valid_request=false;
if (strpos($agent, 'Google')!==false && is_valid_google_ip($ip)){
$is_valid_request=true;
}
return $is_valid_request;
}
Note
Sometimes when using $_SERVER['HTTP_X_FORWARDED_FOR'] OR $_SERVER['REMOTE_ADDR'] more than 1 IP address is returned, for example '155.240.132.261, 196.250.25.120'. When this string is passed as an argument for gethostbyaddr() PHP gives the following error:
Warning: Address is not a valid IPv4 or IPv6 address in...
To work around this I use the following code to extract the first IP address from the string and discard the rest. (If you wish to use the other IPs they will be in the other elements of the $ips array).
if (strstr($remoteIP, ', ')) {
$ips = explode(', ', $remoteIP);
$remoteIP = $ips[0];
}
https://www.php.net/manual/en/function.gethostbyaddr.php
The recommended way by Google is to do a reverse dns lookup (gethostbyaddr) in order to get the associated host name AND then resolve that name to an IP (gethostbyname) and compare it to the remote_addr (because reverse lookups can be faked, too).
But beware, end lokups take time and can severely slow down your webpage (maybe check for user agent first).
Google also publishes a machine readable file containing the IP addresses of their crawlers, see the link below.
See:
https://developers.google.com/search/docs/advanced/crawling/verifying-googlebot
https://webmasters.googleblog.com/2006/09/how-to-verify-googlebot.html
//The function
function is_google() {
return strpos($_SERVER['HTTP_USER_AGENT'],"Googlebot");
}
How to verify Googlebot.
If you have a site that has thousands of pages then going for reverse DNS will be costly, So I think the best method is to hard code ips list. (Php code example)
function googleBotIPsList(){
return "ips"; //hard coded IPs here.
}
Also you can make another function which gets the latest ips. Now upto you how frequently you call this function.
function getLatestGoogleBotIPsList(){
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_URL,"https://developers.google.com/static/search/apis/ipranges/googlebot.json");
$result=curl_exec($ch);
curl_close($ch);
$result = (json_decode($result, true));
$ips='';
for($i=0;$i<count($result['prefixes']);$i++) {
$ips .= ($result['prefixes'][$i]['ipv6Prefix'] ? $result['prefixes'][$i]['ipv6Prefix'] : $result['prefixes'][$i]['ipv4Prefix']).',';
}
return rtrim($ips,',');
}
Then use strpos to check from the hardcoded list
if(strpos(googleBotIPsList(),zen_get_ip_address()) !==false){
// Insert into your table etc.
}