I am using curl to hit a url and get contents from it and below is the code
$url = 'http://uber.com';
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_URL => $url,
CURLOPT_HEADER => false,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_USERAGENT => 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.16 (KHTML, like Gecko) \ Chrome/24.0.1304.0 Safari/537.16',
CURLOPT_CONNECTTIMEOUT => 180,
CURLOPT_TIMEOUT => 180 + 60,
CURLOPT_COOKIE => 'lang=en;'
));
$response = curl_exec($ch);
var_dump($response);
It gives me the content when I execute the script on local server but while executing this on staging and production server, situated in US, it give 403-forbidden error. I also tested the same from another server in US and it works fine.
Any help is appreciated.
Related
I am Using Whatsapp Media Api to get Image send from user to business number using webhooks and i also get the Image ID and url and Image as well in Postman but when I use that curl request made from Postman it alwasy shows facebook error:
Sorry, something went wrong.
We're working on it and we'll get it fixed as soon as we can.
But its working fine in Postman what am i doing wrong?
<?php
$ua = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.16 (KHTML, like Gecko) \
Chrome/24.0.1304.0 Safari/537.16';
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'Image_URL_from_Api',
CURLOPT_USERAGENT => $ua,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_HTTPHEADER => array(
'Authorization: Bearer My_Token'
),
));
$response = curl_exec($curl);
curl_close($curl);
echo $response;
OK, I've got the answer I needed to give the user_agent value which is in postman it is automatically putting CURLOPT_USERAGENT => 'PostmanRuntime/7.26.10' in the header but for curl, I've added CURLOPT_USERAGENT => 'curl/7.64.1', and it worked.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent
I am using the following code to run curl.But for one url, I am getting 502 Bad Gateway error.
<?php
//$proxy = '127.0.0.1:80';
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => '<requesturl>',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_SSL_VERIFYPEER=>false,
CURLOPT_SSL_VERIFYHOST=>false,
//CURLOPT_PROXY=>$proxy,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_HTTPHEADER => array(
'Cookie: PHPSESSID=e03338f51c56ada6870d530080127581'
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
print_r($response);
print_r($err);
curl_close($curl);
?>
I have removed https and put http and checked but for that also not working.
My PHP version is 5.6.
Thanks,
Rekha
I have found the answer.I have added user agent and it worked fine.
curl_setopt($ch,CURLOPT_USERAGENT,'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13');
For dynamically get user agent in php,
$userAgent = $_SERVER['HTTP_USER_AGENT'];
This will help someone.
Thanks,
Rekha
I've got a problem with my PHP script using CURL. I want to download file from another server (Probably with hotlink protection). The URL is like this: example.com/script.extension?id=12345&type=xxx&another=qwerty
When i hit that link from my website, It sends 'Referer:' in headers, so the download is not starting. (without the header it works)
I have to modify the file's name, so i've used CURL to download that to my server. When I'm doing that with that script, the file is created, but it is empty:
function my_function($the_url) {
$fp = fopen ('tmp_file.extension', 'w+');
$user_agent='Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 Firefox/8.0';
$options = array(
CURLOPT_VERBOSE => 1,
CURLOPT_URL => $the_url,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_POST => false,
CURLOPT_USERAGENT => $user_agent,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HEADER => false,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_ENCODING => "",
CURLOPT_AUTOREFERER => false,
CURLOPT_REFERER => "http://example.com/",
CURLOPT_CONNECTTIMEOUT => 120,
CURLOPT_TIMEOUT => 120,
CURLOPT_MAXREDIRS => 10,
);
$ch = curl_init();
curl_setopt_array($ch, $options);
$my_exec = curl_exec($ch);
fwrite($fp, $my_exec);
curl_close($ch);
fclose($fp);
}
But it works well with file which is hosted on my site (eg. mysite.com/file.zip)
Also, when I run command through the shell:
$ curl -L -o file.tmp --referer http://example.com/ full url here
It downloaded the file properly.
What's the reason I cannot do it with my PHP function?
This is the php code that I am using.
I tested this code on localhost (xampp) and everything ok. but when I upload this code to my host (1and1 hosting) This code will not work. Please help me to find out why?
Thank you very much!
$url = "https://www.packagetrackr.com/track/1ZX799390355046642";
$page = get_web_page($url);
echo $page;
function get_web_page($url)
{
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true, // don't return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_USERAGENT => "Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_2_1 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5", // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
CURLOPT_SSL_VERIFYPEER => false, // Disabled SSL Cert checks
CURLOPT_REFERER => $url // referent
);
$ch = curl_init( $url );
curl_setopt_array( $ch, $options );
$content = curl_exec( $ch );
curl_close( $ch );
return $content;
}
you can try turn on allow_url_fopen in your php.ini file
or
php_value allow_url_fopen On in .htaccess
There is a website http://www.tremorgames.com and I want to do a cURL login. I have a code which I use on my server http://shalva97.x10.mx/tremorlogin/ but it just does not work, but after a lot of testing I decided to install Apache on my PC and tested the exactly same code and it worked.
Here is the code :
<?php
$url="http://www.tremorgames.com/index.php";
$postdata = "loginuser=shalva&loginpassword=shalva&Submit=&saveme=1";
$ch = curl_init();
$f = fopen('request.txt', 'w');
curl_setopt_array($ch, array(
CURLOPT_URL => $url,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_VERBOSE => 1,
CURLOPT_STDERR => $f,
CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0',
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $postdata,
CURLOPT_HEADER => 1,
CURLOPT_HTTPHEADER => array("Accept-Language: en-US;q=0.6,en;q=0.4", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Connection: keep-alive"),
CURLOPT_COOKIEFILE => "cookies.txt", //Save cookies
CURLOPT_COOKIEJAR => "cookies.txt", //Cookies located
CURLOPT_REFERER => 'http://www.tremorgames.com/index.php',
CURLOPT_ENCODING => 'gzip,deflate'
));
echo curl_exec($ch);
fclose($f);
curl_close($ch);
?>
Executing this same code on both my PC and server, I get different headers, on PC it receives multiple "Set-Cookie" header, but on server it is only one.
Why it does not work on the server? Why it works on my PC?
well after a lot of thinking i found out that it is not only headers is sent but the ip and my location. The site does not allow logins from other countries thats why it didnot loged in, the code is correct and it was all about server location. i hope this will help others who also try to do same thing.