Any idea where my code is going wrong...I am trying to connect through a proxy with the curl function in php...I assuming the proxy worked bc I tried a few from this list http://hidemyass.com/proxy-list/search-234921 but cant seem to get any to function correctly...
Thoughts?
function my_fetch($url,$user_agent='Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)')
{
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt ($ch, CURLOPT_HEADER, 0);
curl_setopt ($ch, CURLOPT_REFERER, 'http://www.google.com/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXY, '75.74.244.122:1523');
$data = curl_exec();
curl_close($ch);
return $result;
}
It doesn't look like the proxy you are using is working:
jasonfunk#jasonfunk-laptop:$ telnet 75.74.244.122 1523
Trying 75.74.244.122...
telnet: Unable to connect to remote host: Connection refused
You can try multiple proxy by using random one by one using this script
Get random proxy
function get_random_proxy(){
srand ((double)microtime()*1000000);
$f_contents = file ("proxy.txt");
$line = $f_contents[array_rand ($f_contents)];
return $line;
}
call curl function using one proxy randomly
function get_curl_proxy($url){
$proxy_ip = get_random_proxy();
$agent = "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.4 (KHTML, like Gecko) Chrome/4.0.233.0 Safari/532.4";
$referer = "http://www.google.com/";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy_ip);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_MAXREDIRS, 2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
For further reference see this
http://altafphp.blogspot.in/2012/06/using-proxies-with-curl-in-php.html
Related
I have to download large videos with curl. Cause the site where I download from put 403 blocks, And I can pass thru with only Curl. Thats my php curl function code. I need this with the download but it shouldnt eat up my memory..
function fakeip()
{
//return long2ip( mt_rand(0, 65537) * mt_rand(0, 65535) );
return "127.0.0.1";
}
function curlx($feed,$coo=null,$ref=null)
{
$ch = curl_init();
$timeout = 0;
curl_setopt ($ch, CURLOPT_URL, $feed);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("REMOTE_ADDR: ".fakeip(),"X-Client-IP: ".fakeip(),"Client-IP: ".fakeip(),"HTTP_X_FORWARDED_FOR: ".fakeip(),"X-Forwarded-For: ".fakeip()));
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0');
//curl_setopt($ch, CURLOPT_INTERFACE, "88.150.225.55");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
if(!empty($coo))
{
curl_setopt($ch, CURLOPT_COOKIEFILE, $coo);
curl_setopt($ch, CURLOPT_COOKIEJAR, $coo);
}
if(empty($ref))
{
curl_setopt($ch, CURLOPT_REFERER,$feed);
}
else
{
curl_setopt($ch, CURLOPT_REFERER,$ref);
}
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$veri= curl_exec($ch);
curl_close($ch);
return str_replace(array("\n", "\t", "\r",), null, $veri);
}
Hy,
I have a bit of a problem here. One client of mine asked to create a login script for a website built in ASP. As much as I can take and use Response Cookies, I am not able to take and use Request Cookies. I tried, but they are not saved in my cookie files. As such, I cannot login into the very website. Its address is http://www.itraceuk.co.uk/Default.asp?secure=signin
Here is the code I am currently using:
$ch4 = curl_init();
curl_setopt ($ch4, CURLOPT_URL, $posturl);
curl_setopt($ch4, CURLOPT_HEADER, true);
curl_setopt ($ch4, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($ch4, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt ($ch4, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.101 Safari/537.36");
curl_setopt ($ch4, CURLOPT_TIMEOUT, 60);
curl_setopt ($ch4, CURLOPT_FOLLOWLOCATION, true);
curl_setopt ($ch4, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch4, CURLOPT_COOKIESESSION, 1);
curl_setopt ($ch4, CURLOPT_COOKIEJAR, $cookie2);
curl_setopt ($ch4, CURLOPT_COOKIEFILE, $cookie2);
curl_setopt ($ch4, CURLOPT_REFERER, 'http://www.itraceuk.co.uk/Default.asp?secure=signin');
$result4 = curl_exec ($ch4);
curl_close($ch4);
<?php
$posturl = 'http://www.itraceuk.co.uk/Default.asp?secure=signin';
$cookie = __DIR__ . '/cookie.txt'; // file and folder must be writeable
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $posturl);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729)");
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.itraceuk.co.uk/Default.asp?secure=signin');
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);
var_dump(file_get_contents($cookie));
Set cookie folder/file permissions to writeable by PHP and Server.
Reuse the cookie data after the login.
Curl Post to the login form with correct credentials, that logs you in and sets the cookie (curl CURLOPT_POSTFIELDS). Then reuse this cookie for the second curl request (this code here).
For a complete example of a HTTPS cURL login, please see: https://stackoverflow.com/a/10307956/1163786
I want to get content of some url through the proxy. I had taken proxy ip and port from http://letushide.com/filter/http,all,all/list_of_free_HTTP_proxy_servers
but I am using following code and it return blank page.
$url = "http://google.co.in";
$agent = "Mozilla/5.0 (X11; U; Linux i686; en-US)
AppleWebKit/532.4 (KHTML, like Gecko)
Chrome/4.0.233.0 Safari/532.4";
$referer = "http://www.google.com/";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXY, '190.122.XXX.XXX:8080');
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_MAXREDIRS, 2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
$data = curl_exec($ch);
curl_close($ch);
echo $data;
The below function returns different result than that actually we render the page directly.
What would be the issue?
function file_get_contents_curl($url) {
$ch = curl_init();
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1)Gecko/20061204 Firefox/2.0.0.1";
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //Set curl to return the data instead of printing it to the browser.
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt ($ch, CURLOPT_TIMEOUT, 10);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
My guess is that your accessing it with one browser, but your setting the $useragent to another. The external site might be returning different data depending on the useragent
The Following gives me a 407 error, Can anyone see why or where I'm going wrong?
I am behind a proxy.
I have spent a while on this.
I'm using the correct credentials.
Thanks
function pushMeTo($widgeturl,$text,$signature) {
$agent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12";
if (!function_exists("curl_init")) die("pushMeTo needs CURL module, please install CURL on your php.");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $widgeturl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'user:pass');
curl_setopt($ch, CURLOPT_PROXY, "IP");
curl_setopt($ch, CURLOPT_PROXYPORT, 8080);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
$page = curl_exec($ch); //or die("Curl exe failed");
print_r(curl_error($ch));
print_r($page);
preg_match("/form action=\"(.*?)\"/", $page, $form_action);
preg_match("/textarea name=\"(.*?)\"/", $page, $message_field);
preg_match("/input type=\"text\" name=\"(.*?)\"/", $page, $signature_field);
//print_r($message_field);
$ch = curl_init();
$strpost = $message_field[1].'=' . urlencode($text) . '&'.$signature_field[1].'=' . urlencode($signature);
curl_setopt($ch, CURLOPT_POSTFIELDS, $strpost );
curl_setopt($ch, CURLOPT_URL, $form_action[1]);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
$page = curl_exec($ch);
}
?>
<?
$url = "MY URL";
pushmeTo ($url,"This Is A Test","ServiceDesk");
?>
The answer was to add:
curl_setopt($curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
After that everything worked!
Try adding:
curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP'); // just a guess
If this does not work for you then check if mod_proxy apache module is loaded or not.
If thats also loaded , then try recompiling curl module using latest version
Hope it helps you in debugging.
Check the bottom, where you use the function. Should have a capitol M.