I am trying to open a https url but it is not working.It works for other https websites but for this it is not working.I don't know the reason why it is happening like that.Below is the code i have written.
Please help.
<?php
error_reporting(E_ALL);
$url = "https://sgate.bmwgroup.com/login_common.fcc";
$ch = curl_init($url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 200);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch,CURLOPT_SSL_VERIFYHOST,1);
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
curl_setopt($ch,CURLOPT_FOLLOWLOCATION,TRUE);
curl_setopt ( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)" );
$curl_scraped_page= curl_exec($ch);
if(!$curl_scraped_page)
{
echo "first learn";
}
if(curl_errno($ch)) {
echo 'Curl error: ' . curl_error($ch);
}
echo $curl_scraped_page;
curl_close($ch);
?>
good news that page is working, bad news that site is using javascript, and while he doesn't get his scripts, the page is not showing. instead of echo $curl_scraped_page;
try to use:
var_dump(htmlspecialchars($curl_scraped_page));
Related
I am trying to display a https url but it is not working.It works for other https websites like facebook,microsoft,google but for this it is not working.I don't know the reason why it is happening like that.Below is the code i have written.
Please help.
<?php
error_reporting(E_ALL);
$url = "https://sgate.bmwgroup.com/login_common.fcc";
$ch = curl_init($url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 200);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch,CURLOPT_SSL_VERIFYHOST,1);
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
curl_setopt($ch,CURLOPT_FOLLOWLOCATION,TRUE);
curl_setopt ( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6 (.NET CLR 3.5.30729)" );
$curl_scraped_page= curl_exec($ch);
if(!$curl_scraped_page)
{
echo "first learn";
}
if(curl_errno($ch)) {
echo 'Curl error: ' . curl_error($ch);
}
echo $curl_scraped_page;
curl_close($ch);
?>
I'm creating a script that is scraping the site www.piratebay.se. The script was working OK two-three days ago but now I'm having problems with it.
This is my code:
$URL = 'http://thepiratebay.se';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
curl_setopt($ch, CURLOPT_COOKIE, "language=pt_BR; c[thepiratebay.se][/][language]=pt_BR");
$fonte = curl_exec ($ch);
curl_close ($ch);
echo $fonte;
The response of this code is not clean HTML, but looks like this instead:
��[s۸N>��k�9��-ىmI7��$�8�.v��͕���$h���y�G�Sg:ӷ>�5����ʱ�aor&���.v)���������) d�w��8w�l����c�u""1����F*G��ِ�2$�6�C�}��z(bw�� 4Ƒz6�S��t4�K��x�6u���~�T���ACJb��T^3�USPI:Mf��n�'��4��� ��XE�QQ&�c5�`'β�T Y]D�Q�nBfS�}a�%� ���R) �Zn��̙ ��8IB�a����L�
I already tried to use user agent on .htaccess, PHP and cURL but to no success.
Add this:
curl_setopt($ch, CURLOPT_ENCODING , "gzip");
Tested on my local environment, works fine with it.
Hi i need to send a curl request from my web app to an external webpage
actually i use this code
public function execute()
{
$this->_prepare();
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $this->get_url());
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9) Gecko/2008052906 Firefox/3.0");
//curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $this->get_post_request_array());
if(curl_exec($ch) === false)
{
echo 'Curl error: ' . curl_error($ch);
}
curl_close ($ch);
}
But this return a page with some js issues, so i need to open the destination webpage into a new window.
It's possibile do that with curl or i need to use another execute function?
thanks
I am trying to use php cURL to fetch amazon web page but get
HTTP/1.1 503 Service Temporarily Unavailable instead. Is Amazon blocking cURL?
http://www.amazon.com/gp/offer-listing/B003B7Q5YY/
<?php
function get_html_content($url) {
// fake user agent
$userAgent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $userAgent);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch,CURLOPT_COOKIEFILE,'cookies.txt');
curl_setopt($ch,CURLOPT_COOKIEJAR,'cookies.txt');
$string = curl_exec($ch);
curl_close($ch);
return $string;
}
echo get_html_content("http://www.amazon.com/gp/offer-listing/B003B7Q5YY");
?>
I use simple
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $offers_page);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 GTB5');
$html = curl_exec($ch);
curl_close($ch);
but i have another problem. if you send a lot of queries to amazon - they start send 500 page to you.
I have a chat service that i would like to make an announcement bot that runs on cron to post daily updates to this chat. The url is http://www6.cbox.ws/box/?boxid=524970&boxtag=7xpsk7&sec=form I have tried various curl examples online but none seem to get the job done. My latest attempt which was a failure.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www6.cbox.ws/box/?boxid=&boxtag=&sec=profile&n=andysmith&k=0000000000000000000000000000000000000000");
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_REFERER, 'http://www6.cbox.ws/box/?boxid=&boxtag=&sec=profile&n=andysmith&logpword=iloveJD');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cbox.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cbox.txt');
curl_exec($ch);
curl_close($ch);
I just need it to login and post a message.
try with
$result=curl_exec($ch);
//print $result;
if($result === false)
{
echo '<br/>Curl error: '.curl_error($ch);
curl_close($ch);
exit;
}
and see the error
Note: if u r setting 'CURLOPT_REFERER' , u need to set
curl_setopt($ch, CURLOPT_HEADER, true);