I am using cURL to access a facebook page. Locally it works perfect, but when I upload it to my dev server, it breaks and returns an empty string. I've checked and cURL is installed on the server. Here's the code I use to access facebook:
$header = array();
$header[] = 'Accept: text/json';
$header[] = 'Cache-Control: max-age=0';
$header[] = 'Connection: keep-alive';
$header[] = 'Keep-Alive: 300';
$header[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';
$header[] = 'Accept-Language: en-us,en;q=0.5';
$header[] = 'Pragma: ';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://facebook.com/feeds/page.php?format=json&id=135137236003');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729)');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$result = curl_exec($ch);
curl_close ($ch);
Any help is appreciated!
Change the accept header to */* or application/json as facebook is sending the response header as application/json.
And change this url
http://facebook.com/feeds/page.php?format=json&id=135137236003
to
http://www.facebook.com/feeds/page.php?format=json&id=135137236003
as facebook is redirecting the non-www request to www requests. Though it works for you as put follow location, but it saves one reound trip
Related
function get_data($url,$proxy=Null){
$agents = array(
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:7.0.1) Gecko/20100101 Firefox/7.0.1',
'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.9) Gecko/20100508 SeaMonkey/2.0.4',
'Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)',
'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_7; da-dk) AppleWebKit/533.21.1 (KHTML, like Gecko) Version/5.0.5 Safari/533.21.1'
);
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: ";
$curl = curl_init();
curl_setopt($curl, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($curl, CURLOPT_HTTPHEADER, $header);
curl_setopt($curl,CURLOPT_USERAGENT,$agents[array_rand($agents)]);
curl_setopt($curl, CURLOPT_REFERER, "http://google.com/");
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, TRUE); ///** Follow Redirect
$html1 = curl_exec($curl);
curl_close($curl);
return $html1;
}
Above is my function and i am trying to get a page from proxy site
echo get_data('http://www.hostfast.info/browse.php?u=lZpnCp2dHRM0%2BnBp1Ljfmr8I%2BA%3D%3D&b=5');
But this is not working ....its giving me home page of that site and if i am trying new search its also not working... i am new to CURL ... but i think there is some thing to do with cookies ... how can i fix this
thx
To save cookie in cURL with PHP:
curl_setopt($curl, CURLOPT_COOKIEFILE, "yourcookiefile.txt");
curl_setopt($curl, CURLOPT_COOKIEJAR, "yourcookiefile.txt");
define('POSTURL', 'http://hostfast.info/includes/process.php?action=update');
define('POSTVARS', 'u=google.com/complete/search?output=toolbar&q=love'); // POST VARIABLES TO BE SENT
$ch = curl_init(POSTURL);
curl_setopt($ch, CURLOPT_POST ,1);
curl_setopt($ch, CURLOPT_POSTFIELDS ,POSTVARS);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION ,1);
curl_setopt($ch, CURLOPT_HEADER ,0); // DO NOT RETURN HTTP HEADERS
curl_setopt($ch, CURLOPT_RETURNTRANSFER ,1); // RETURN THE CONTENTS OF THE CALL
curl_setopt($ch, CURLOPT_COOKIEFILE, "yourcookiefile.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "yourcookiefile.txt");
$Rec_Data = curl_exec($ch);
curl_close($ch);
echo $Rec_Data;
This works .. ;)
I am trying to read the following ebay webpage to PHP variable for processing:
http://offer.ebay.co.uk/ws/eBayISAPI.dll?ViewBidsLogin&_trksid=p2047675.l2564&rt=nc&item=321069150620
It shows fine in any modern browser without need to log in.
When I am trying to read the page to PHP variable with the following code:
$url="http://offer.ebay.co.uk/ws/eBayISAPI.dll?ViewBidsLogin&_trksid=p2047675.l2564&rt=nc&item=321069150620";
$header = array();
$header[] = 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5';
$header[] = 'Cache-Control: max-age=0';
$header[] = 'Connection: keep-alive';
$header[] = 'Keep-Alive: 300';
$header[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';
$header[] = 'Accept-Language: en-us,en;q=0.5';
$header[] = 'Pragma: ';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729)');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$string = curl_exec($ch);
curl_close ($ch);
echo $string;
I am getting the following page http://www.talumets.com/tmp/error.jpg that asks me to enter numbers from photos to continue. Sometimes my code works, but 95% time it asks me to enter numbers. I have also tried $string= file_get_contents( $url ), but the same problem. Any idea how to bypass this?
Thanks,
Tom
What you are seeing is ebay`s captcha protection against script such as yours. I dont think there is good way to bypass that.
You could try to limit your requests per second ratio and hope you will not trigger captcha
Ideal solution (if you dont want to use api) would be use of multiple servers with only few requests per second each.
I use this API to grab lyrics: http://www.chartlyrics.com/api.aspx
Unfortunately text is not html indented. How do I parse it so that in web browser I have correct spacing / indentation?
if ($_GET['get_lyrics'] == 1 AND $_GET['song_name'] AND $_GET['song_author']) {
//via http://api.chartlyrics.com/
$url = "http://api.chartlyrics.com/apiv1.asmx/SearchLyricDirect?artist=".$_GET['song_author']."&song=".$_GET['song_name'];
$url = str_replace(" ", '%20', $url);
//
$header = array();
$header[] = 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5';
$header[] = 'Cache-Control: max-age=0';
$header[] = 'Connection: keep-alive';
$header[] = 'Keep-Alive: 300';
$header[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';
$header[] = 'Accept-Language: en-us,en;q=0.5';
$header[] = 'Pragma: ';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729)');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$result = curl_exec($ch);
curl_close ($ch);
//
if ($result) {
$xml = new SimpleXMLElement($result);
print_r(json_encode($xml->Lyric));
}
}
I am not sure what are you trying to do, but try nl2br($text) to add html line endings and then use CSS to style it the way you want, other option is to put it in <pre></pre> tags and style the element.
You could use a pre instead of a div.
Or parse the text :
Use a regex to get the different paragraphs and use p around them.
Or just replace "\n" in your text with br
use nl2br
I got a problem with downloading video from videoslasher.com. I wrote a script to get source link and it is working fine.
Example link:
http://storage2.videoslasher.com/free/K/K1/K1Z6TBW88QCM.flv?h=XeahiwUFe_yHiduYx5EzBg&e=1377200810
But when i am downloading with script i get 403 (forbidden).
Code downloading video:
$headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
$headers[] = 'Accept-Language: pl,en-us;q=0.7,en;q=0.3';
$headers[] = 'Accept-Encoding: gzip, deflate';
$headers[] = 'Connection: keep-alive';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.videoslasher.com/static/player/flowplayer.commercial-3.2.7.swf');
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt( $ch, CURLOPT_HTTPHEADER, $headers );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE,'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_BUFFERSIZE, 4096000);
echo curl_exec($ch);
curl_close($ch);
Does anyone know what could be the problem?
I have a Facebook page feed that I want to access:
http://facebook.com/feeds/page.php?format=json&id=123456789 (Not a real ID)
When I put the URL in the browser, it works just fine, but when I try to access it using file_get_contents, Facebook sends me to a page that says I am using an unsupported browser. This data is public though so I shouldn't need an access token to obtain it. Is there an extra step I need to take in order to access this data? I also tried using cURL with no success.
Any help is appreciated. Thanks.
You can use curl and mimic a browser, see this thread on how to do it.
$header = array();
$header[] = 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5';
$header[] = 'Cache-Control: max-age=0';
$header[] = 'Connection: keep-alive';
$header[] = 'Keep-Alive: 300';
$header[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';
$header[] = 'Accept-Language: en-us,en;q=0.5';
$header[] = 'Pragma: ';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'YOUR URL HERE');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11 (.NET CLR 3.5.30729)');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_TIMEOUT, 20);
$result = curl_exec($ch);
curl_close ($ch);
echo $result;
Don't use this url, it is really really old method to access it. Use their relatively new Graph API