Hello I am trying to query the site URL which will return a dynamic json file containing the following information...
{"input_address":"1BEgj5JUUsUz91bR9F6q6YoUPgtAGWAZg9","destination":"1A8JiWcwvpY7tAopUkSnGuEYHmzGYfZPiq","fee_percent":0}
I need to extract the input_address from that json file. This is probably easy although I am very new to php, here is what i have located in xx.php...
<?php
$root_url = 'https://blockchain.info/api/receive?method=create&address=1A8JiWcwvpY7tAopUkSnGuEYHmzGYfZPiq';
$agent = 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)':
$ch = curl_init();
curl_setopt($ch, CURL_SSL_VERIFYPEER, false);
curl_setopt($ch, CURL_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, 'Content-type:application/json');
curl_setopt($ch, CURLOPT_URL, $root_url);
curl_setopt($ch, CURLOPY_USERAGENT, $agent);
$result=curl_exec($ch);
curl_close($ch);
$gg = json_decode($result);
var_dump(json_decode($result));
$hh = var_dump($gg['input_address']);
//$zz = json_decode($result);
echo 'blah:'. $gg;
echo 'hh:'. $hh;
?>
When I go to xx.php it is returning a blank page and I thought it should return a page that echos the contents of the entire json file? Any help is appreciated.
Related
i am trying to get web page content with curl from some websites but they return 400 bad request ( file_get_contents return empty ) here's the function i am using :
function file_get_contents_curl($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
Put error_reporting(E_ALL); line at the top file where you are calling this function.
It will generate the cause of an error.
Crawling http://www.mfinante.ro/infocodfiscal.html?cod=299 is not working.
It's getting redirected to some other location. But why?
<?php
$url = 'http://www.mfinante.ro/infocodfiscal.html?cod=299';
$agent= 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)';
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_ENCODING ,"");
curl_setopt($curl, CURLOPT_USERAGENT, $agent);
$html = curl_exec($curl);
$redirectURL = curl_getinfo($curl,CURLINFO_EFFECTIVE_URL );
curl_close($curl);
echo $html;
?>
I'm unable to understand why this happening.
You could use htmlspecialchars() to get the source code of the response
echo htmlspecialchars($html);
It's likely that there is a javascript or meta redirect in there somewhere. My JS is so poor i can't really help you with that.
If you can find that, you can build a regular expression to find the URL and then fetch it's contents.
If you visit URL:
https://selfsolve.apple.com/agreementWarrantyDynamic.do?caller=sp&sn=990002316140324
then it will redirect and results will be shown at URL:
https://selfsolve.apple.com/wcResults.do
I'm trying with PHP cURL to get this results but the page is empty. Its not redirecting.
Here is my code which I tried:
<?php
$url ='https://selfsolve.apple.com/agreementWarrantyDynamic.do?caller=sp&sn=990002316140324';
$http_headers = array(
'Accept: /*',
'Connection: keep-alive'
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $http_headers);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) . '/applecookie.txt');
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, true);
$retValue = curl_exec($ch);
$response = json_decode(curl_exec($ch));
$ee = curl_getinfo($ch);
print_r($ee);
print_r($retValue);
?>
How to make it work?
==== (Possible) Issue: Your PHP configuration has safe_mode or open_basedir enabled.
CURLOPT_FOLLOWLOCATION (integer) This constant is not available when
open_basedir or safe_mode are enabled.(http://php.net/manual/en/curl.constants.php)
==== (Possible) Issue: The remote service isn't responding as you expect. Break it down into individual parts and log the output, or check Google Chrome (or similar) for the redirect:
A-ha! Chrome shows that there is no redirect!
In PHP this might look something like the below. This code will cycle through the redirect chain manually and give you chance to inspect responses along the way.:
(see code below)
==== Issue: You are executing the request twice (you probably noticed this!):
$retValue = curl_exec($ch);
$response = json_decode(curl_exec($ch));
==== Issue: You are expecting to json_decode a HTML response. This will not work (and can't be expected to).
IN SHORT
It looks like there is a redirect in JavaScript that this page is using, as opposed to normal header redirects. You might have to rethink your approach as you'll probably struggle to extract this information from the page, and it's certainly going to be subject to change. (It's actually submitting a form to the next URL so you'll have to work out where the data is from -- again, check the Chrome log).
(footnote) And the code that will help you spot this in PHP (for this URL it returns 200 straight away -- there is no redirect!):
<?php
$url = 'https://selfsolve.apple.com/agreementWarrantyDynamic.do?caller=sp&sn=990002316140324';
$http_headers = array(
'Accept: */*',
'Connection: keep-alive',
'Accept-Encoding:gzip, deflate, sdch',
'Accept-Language:en-US,en;q=0.8,es;q=0.6'
);
$finished = false;
$count = 0;//we don't want to redirect forever!
$currentUrl = $url;
while( $finished == false && $count < 10 ) {
$count++;
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $http_headers);
curl_setopt($ch, CURLOPT_URL, $currentUrl);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
// not while we're testing: //curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, true);
$retValue = curl_exec($ch);
$info = curl_getinfo($ch);
$responseCode = $info['http_code'];
if($responseCode > 300 && $responseCode < 303) {
echo "\n redirecting ($responseCode) to ".$info['redirect_url'];
$currentUrl = $info['redirect_url'];
} else {
$finished = true;
echo "\n finished ($responseCode) content length:".strlen($retValue);
}
}
//now try the whole thing
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $http_headers);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, true);
$retValue = curl_exec($ch);
$info = curl_getinfo($ch);
echo "\nWhole request: finished ($responseCode) content length:".strlen($retValue). " total redirects:".$info['redirect_count'];
echo "\n\n";
Output:
finished (200) content length:4833
Whole request: finished (200) content length:4833 total redirects:0
I have page name customer.php in this page, I have to call gift.php with query string. And gift.php page has a form set to auto submit.
I have try this using CURL but it gives 404 error. And using php file_get_contents this one also redirect to 404. And tried using jQuery Ajax, I haven't got any result, But Its working with iframe, but I belive it is not the best option
<iframe src="gift.php?name=John"></iframe>
Could someone please help me to do this using curl...
this code not working
$url =urlencode('gift.php?name=john');
$header = array("Accept: application/json");
$ch = curl_init();
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_ENCODING, "gzip");
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
$retValue = curl_exec($ch);
$response = json_decode(curl_exec($ch));
$ee = curl_getinfo($ch);
print_r($ee);
print_r($retValue);
For curl don't use $url =urlencode('gift.php?name=john'); and $url must contain domainname. For example $url = 'http://domain.com/gift.php?name=' . urlencode('john');
I'm trying to submit a form to a .aspx page with curl and then do something with the response. The problem is that my code works when I'm submiting it from my local xampp server but when submited from webserver I get "HTTP Error 400. The request URL is invalid."
I tried removing CURLOPT_POST option, found it somewhere on SO. I also tried urlencoding but then I get nothing.
$url = "http://www.somepage.com/locations/default.aspx#location_page_map";
$kv[]='search=92627';
$kv[]='__VIEWSTATE';
$kv[]='__EVENTTARGET';
$query_string = join("&", $kv);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLOPT_POST, count($kv));
curl_setopt($ch, CURLOPT_POSTFIELDS, $query_string);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
$output = curl_exec($ch);
var_dump($output);
curl_close($ch);
You can actually leave out the __VIEWSTATE and __EVENTTARGET there most likely something todo with ASP's form value persistence, also you can remove the #location_page_map as thats just to focus the page on the map section, so will not impact the results from the service/site your trying to scrape. You then use http_build_query() to turn the array into a string for curl.
<?php
//$url = "http://www.myfitfoods.com/locations/default.aspx#location_page_map";
$url = "http://www.somepage.com/locations/default.aspx#location_page_map";
$kv['search'] = '92627';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLOPT_POST, count($kv));
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($kv));
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)');
$output = curl_exec($ch);
var_dump($output);
curl_close($ch);
You haven't defined your $kv array propertly. Curl will take an array, but it has to be in key=>value format. All you've provided is 3 values. e.g. you'd actually be passing
=search%3D62627&=__VIEWSTATE&=__EVENTTARGET
^--no key ^---no key ^--- no key
Try:
$kv = array(
'search' => 92627,
'x' => '__VIEWSTATE',
'y' => '__EVENTTARGET'
)
curl_setopt($ch, CURL_POSTFIELDS, $kv);
or similar instead.