i am trying to grab data from here : http://mediaforest.biz/mobile/nowplaying.aspx
in the page you select a station and post it then you get new page with data. but i cant grab it, i get the same page again.
i used this code:
<?php
header ('Content-type: text/html; charset=utf-8');
$url = "http://mediaforest.biz/mobile/nowplaying.aspx";
$referer = "";
// headers
$header[] = "Host: ".parse_url($url, PHP_URL_HOST);
$header[] = "User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; he; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3";
$header[] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$header[] = "Accept-Language: he,en-us;q=0.7,en;q=0.3";
$header[] = "Accept-Encoding: gzip,deflate";
$header[] = "Accept-Charset: windows-1255,utf-8;q=0.7,*;q=0.7";
$header[] = "Keep-Alive: 115";
$header[] = "Connection: keep-alive";
$cookie="cookie.txt";
$fp=fopen($cookie,"w+");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_REFERER,$referer);
curl_setopt($ch, CURLOPT_TIMEOUT, 900);
curl_setopt($ch, CURLOPT_FAILONERROR, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_HEADER,true);
curl_setopt($ch, CURLOPT_COOKIEFILE,$cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR,$cookie);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
$content=curl_exec($ch);
echo $content;
if(stristr($content,"__EVENTTARGET")){
$array1=explode('__EVENTTARGET" value="',$content);
$content1=$array1[1];
$array2=explode('">
<input type="hidden" name="__EVENTARGUMENT"',$content1);
$content2=$array2[0];
$EVENTTARGET=urlencode($content2);
}
if(stristr($content,"__EVENTARGUMENT")){
$array1=explode('__EVENTARGUMENT" value="',$content);
$content1=$array1[1];
$array2=explode('">
<script language',$content1);
$content2=$array2[0];
$EVENTARGUMENT=urlencode($content2);
}
if(stristr($content,"formNowPlaying")){
$array1=explode('method="post" action="',$content);
$content1=$array1[1];
$array2=explode('">
<input type="hidden" name="__EVENTTARGET"',$content1);
$content2=$array2[0];
$nexturl=$content2;
}
//echo $EVENTTARGET." ".$EVENTARGUMENT." ".$nexturl;
$url = "http://mediaforest.biz/mobile/".$nexturl;
$fields = "EVENTTARGET=".$EVENTTARGET."&__EVENTARGUMENT=".$EVENTARGUMENT."&MyChannels=0&ViewChannel_Button=Show";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_REFERER,$referer);
curl_setopt($ch, CURLOPT_TIMEOUT, 900);
curl_setopt($ch, CURLOPT_FAILONERROR, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_HEADER,true);
curl_setopt($ch, CURLOPT_COOKIEFILE,$cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR,$cookie);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
$content_stage2=curl_exec($ch);
echo $content_stage2;
?>
If you're getting data from the first request, try closing afterwards (for each request).
$content=curl_exec($ch);
curl_close($ch);
echo $content;
and
$content_stage2=curl_exec($ch);
curl_close($ch);
echo $content_stage2;
I don't have much experience with cURL, and I didn't look into it, but at first glance it appears that this:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
needs to be this:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
Could be wrong though.
Related
Sorry for my English. Tell me how to pass non-standard headers in cURL
$headers = array("Accept: application/json, text/plain, /*",
"Captcha: NDRMR9",
"Referer: https://avtokod.mos.ru/AutoHistory/");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://avtokod.mos.ru/AutoHistory/GetByLicensePlate");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query(array('licensePlate' => $licensePlate, 'sts' => $sts)));
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:61.0) Gecko/20100101 Firefox/61.0');
curl_setopt($ch, CURLOPT_COOKIEFILE, $_SERVER["DOCUMENT_ROOT"]."/cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, $_SERVER["DOCUMENT_ROOT"]."/cookie.txt");
$response = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
echo "<pre>";
echo $response;
print_r($info['request_header']);
But I get an error
{"code":1,"message":"Необходимо ввести капчу"}
I. e. I didn't send a captcha.
I can see that in the debugger the Captcha parameter is passed in the header
open image
What am I doing wrong?
I am trying to login into Instagram using cookies. I am using CURL its give me in response a blank page
$cookie = "mid=-MIDVALUE-;csrftoken=-TOKEN-;sessionid=-SESSIONID-3%3AvOZRX4fPL9K5L66ztOV9D28F8j2A3CF9%3A%7B%22_auth_user_id%22%3A5607163585%2C%22_auth_user_backend%22%3A%22accounts.backends.CaseInsensitiveModelBackend%22%2C%22_auth_user_hash%22%3A%22%22%2C%22_platform%22%3A4%2C%22_token_ver%22%3A2%2C%22_token%22%3A%225607163585%3AgdHBjbbOmDw5KHQyXJ7ipnXifcf6LiDP%3Abb2dd9683364a2a3d8fd01a240effb73b3ed19ab12f3f33e9b864e5ee1e6ec38%22%2C%22last_refreshed%22%3A1518249956.7502748966%7D";
$url = "https://www.instagram.com/accounts/login/";
$ch = #curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
$head[] = "Connection: keep-alive";
$head[] = "Keep-Alive: 300";
$head[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$head[] = "Accept-Language: en-us,en;q=0.5";
curl_setopt($ch, CURLOPT_USERAGENT, 'Opera/9.80 (J2ME/MIDP; Opera Mini/9.80 (S60; SymbOS; Opera Mobi/23.348; U; en) Presto/2.5.25 Version/10.54');
curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_HTTPHEADER, $head);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 60);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$page = curl_exec($ch);
curl_close($ch);
I was getting this cookies from java just need to use cookies to login
i know this code is not much batter that is because i don't know much about how Instagram work..
hello im trying to get a page using curl but always getting error 403. Is there any way to get this web page?
My source code is like this
<?php
set_time_limit(0);
error_reporting(0);
$LOGINURL = "https://sso.orange.fr/espace-client/desimlockage";
$ch = curl_init();
$headers[] = "Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
$headers[] = "Accept-Language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4";
$headers[] = "Connection: Keep-Alive";
$headers[] = "Host:iz.orange.fr";
$headers[] = "Referer:http://www.sosh.fr/espace-client-accueil";
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, 'User-Agent: curl/7.39.0');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_URL, $LOGINURL);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
$content = curl_exec($ch);
echo $content;
?>
comment curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); and try once. hope it'll help.
I use the following general code to log into other https sites and pull records using forms, but it doesn't seem to work for www.voip.ms. I've created a testing account so if anyone wants to take a crack at it and tell me what I did wrong. (Warning, the site only gives your IP address 4 tries until it bans it)
<?php
ini_set('max_execution_time', 300);
$username="meahmatt#aol.com";
$password="testaccount";
$url="https://www.voip.ms/m/login.php";
$cookie="cookie.txt";
$postdata = "col_email".$username."&col_password=".$password."&action=login&form1=";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt ($ch, CURLOPT_REFERER, $url);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt ($ch, CURLOPT_POST, 1);
$result = curl_exec ($ch);
curl_close($ch);
echo $result;
?>
I've also tried setting CURLOPT_SSL_VERIFYPEER, TRUE with no change
I had the same problem recently trying to use the twitter api using curl_exec from godaddy.
The magic was to disable both peer and host verification in the options :
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // required as godaddy fails
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); // required as godaddy fails
The error was a certificate verification problem. I have no problem using this exact script on non-godaddy servers.
CURLE_SSL_CACERT (60)
Peer certificate cannot be authenticated with known CA certificates.
The full request looks like this :
$url = "https://api.twitter.com/1.1/statuses/user_timeline.json?..."
$headers = array(
"Authorization: Bearer ".$bearer."",
);
$ch = curl_init(); // setup a curl
curl_setopt($ch, CURLOPT_URL, $url); // set url to send to
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); // set custom headers
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // return data reather than echo
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // required as godaddy fails
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); // required as godaddy fails
// $info = curl_getinfo($ch); // debug info
// var_dump($info); // dump curl info
$result = curl_exec($ch); // run the curl
curl_close($ch); // stop curling
// Check for errors and display the error message
if($errno = curl_errno($ch)) { echo "curlerror::$errno::"; }
Also notice that curl_getinfo and curl_errno were invaluable at finding the problem.
tl;dr , friends don't let friends use godaddy.
You could try this function if you like. It's helped me out a few times.
If you still have trouble try fiddler2 (fiddler2.com) to check for all of the headers and attempt to replicate them in PHP
ini_set('max_execution_time', 300);
$fields['col_email'] = "meahmatt#aol.com";
$fields['col_password'] = "testaccount";
$fields['action'] = "login";
$fields['form1'] = "";
$url = "https://www.voip.ms/m/login.php";
$html = get_html($url,$url,$fields);
function get_html($url,$ref='',$fields=array(),$cookie='cookie.txt'){
// $proxyAddress = '127.0.0.1:8888';
$ch = curl_init();
touch($cookie);
$header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,";
$header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$header[] = "Cache-Control: max-age=0";
$header[] = "Connection: keep-alive";
$header[] = "Keep-Alive: 300";
$header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$header[] = "Accept-Language: en-us,en;q=0.5";
$header[] = "Pragma: "; //browsers keep this blank.
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows;U;Windows NT 5.0;en-US;rv:1.4) Gecko/20030624 Netscape/7.1 (ax)');
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
if($proxyAddress != ''){
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt($ch, CURLOPT_PROXY, $proxyAddress);
}
if(count($fields)>0){
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
curl_setopt ($ch, CURLOPT_POST, 1);
}
curl_setopt($ch, CURLOPT_REFERER, $ref);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
$result = curl_exec ($ch);
if(!$result){
echo "cURL error number:" .curl_errno($ch);
echo "cURL error:" . curl_error($ch);
exit;
}
curl_close ($ch);
return($result);
}
Now I am trying to login to my account using PHP and CURL, but still I am not sure why is not working.
The page I'm trying to login to is: http://lardi-trans.com/index.jsp
After the execution of the script, I get the same page without authorization.
Thanks in advance!
<?php
$url = "http://lardi-trans.com/log/index.jsp";
$referer = "http://lardi-trans.com/index.jsp";
$user_agent = "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:17.0) Gecko/20100101 Firefox/17.0";
$post = "action=/log/login.jsp&log=MYLOGIN&passwd=MYPASSWORD&enter=enter&onlog=%D0%92%D0%BE%D0%B9%D1%82%D0%B8";
$header [] = "Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$header [] = "Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3";
$header [] = "Accept-Encoding:gzip,deflate";
$header [] = "Accept-Language:en-US,en;q=0.5";
$user_cookie = "cookies.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url );
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 3);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_ENCODING, "identity");
curl_setopt($ch, CURLOPT_COOKIEFILE, $user_cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $user_cookie);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$page = curl_exec($ch);
curl_close($ch);
echo $page;
?>
You should set your $url variable to the value of the login form's action - http://lardi-trans.com/log/login.jsp