A similar question has been posted at but i could not find the solution there
Curl error Could not resolve host: saved_report.xml; No data record of requested type"
<?php
$url="http://en.wikipedia.org/wiki/Pakistan";
$ch = curl_init(urlencode($url));
echo $ch;
// used to spoof that coming from a real browser so we don't get blocked by some sites
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 4);
curl_setopt($ch, CURLOPT_TIMEOUT, 8);
curl_setopt($ch, CURLOPT_LOW_SPEED_TIME, 10);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$content = curl_exec($ch);
$info = curl_getinfo($ch);
if ($content === false || $info['http_code'] != 200) {
$content = "No cURL data returned for $url [". $info['http_code']. "]";
if (curl_error($ch))
$content .= "\n". curl_error($ch);
}
else {
// 'OK' status; format $output data if necessary here:
echo "...";
}
echo $content;
curl_close($ch);
?>
when i paste the same address in browser i am able to access the webpage. but when i run this script i get the error message. Can anyone please help me.
Thanks
Remove the urlencode call.
remove the urlencode($url) it should be:
$ch = curl_init($url);
Well.
If you remove urlencode() with instantiating your $ch-var, you go just fine. urlencode() is definitely wrong here.
Good:
$ch = curl_init($url);
Bad:
$ch = curl_init(urlencode($url));
$ch = curl_init($url);
instead of
$ch = curl_init(urlencode($url));
Related
I have been working on setting up the frase api. and created the following curl snippet.
<?php
$url = 'http://api.frase.io/api/v1/process_url';
//The data you want to send via POST
$fields = ['url' => 'https://firstsiteguide.com/best-gaming-blogs/', 'token' => "dd528796a9924dae9962bc5bd7ccdb20"];
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST, 1);
curl_setopt($ch,CURLOPT_POSTFIELDS,http_build_query($fields));
curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT ,3);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0');
curl_setopt($ch,CURLOPT_TIMEOUT, 20);
$response = curl_exec($ch);
if (curl_errno($ch)) {
$error_msg = curl_error($ch);
echo "<br/>CURL ERROR: ". $error_msg ."<br/>";
}else{
print "curl response is:" . $response ;
}
curl_close ($ch);
?>
I am not sure why, But I am receiving the following error for the same
The requested URL returned error: 400 Bad Request
Can help me identify what part of code I am missing or doing wrong. Thank you so much in advance.
You're passing the token as a body parameter instead of a header. The body parameter needs to be sent as a JSON encoded string as mentioned on the API documentation page. Also, you need to remove or at least increase the cURL timeout value as it takes time to fetch, process and return a value from the API end. Note that the API will return the response in JSON format.
So, the complete code should be as:
<?php
$url = 'http://api.frase.io/api/v1/process_url'; //The endpoint url you want to send data via POST
$headers = ['token: dd528796a9924dae9962bc5bd7ccdb20']; // add this line, headers
$fields = ['url' => 'https://firstsiteguide.com/best-gaming-blogs/']; // modify this line, body
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); // add this line
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($fields)); // modify this
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0');
$response = curl_exec($ch);
if (curl_errno($ch))
{
$error_msg = curl_error($ch);
echo "<br/>CURL ERROR: " . $error_msg . "<br/>";
}
else
{
print($response);
// $values = json_decode($response, true); // this is an array with all the values
}
curl_close($ch);
?>
Using Wikiepdia API link to get some basic informations about some world known characters.
Example : (About Dave Longaberger)
This would show as following
Now my question
I'd like to parse the xml to get such basic informations between <extract></extract> to show it.
Here is my idea but failed (I/O warning : failed to load external entity)
<?PHP
$url = 'http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles=Dave Longaberger&format=xml&exintro=1';
$xml = simplexml_load_file($url);
// get extract
$text=$xml->pages[0]->extract;
// show title
echo $text;
?>
Another idea but also failed (failed to open stream: HTTP request failed!)
<?PHP
function get_url_contents($url){
$crl = curl_init();
$timeout = 5;
curl_setopt ($crl, CURLOPT_URL,$url);
curl_setopt ($crl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($crl, CURLOPT_CONNECTTIMEOUT, $timeout);
$ret = curl_exec($crl);
curl_close($crl);
return $ret;
}
$url = "http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles=Dave Longaberger&format=xml&exintro=1";
$text = file_get_contents($url);
echo $text;
?>
so any idea how to do it. ~ Thanks
Update (after added urlencode or rawurlencode still not working)
$name = "Dave Longaberger";
$name = urlencode($name);
$url = 'http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles='.$name.'&format=xml&exintro=1';
$text = file_get_contents($url);
Also not working
$url = 'http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles=Dave Longaberger&format=xml&exintro=1';
$url = urlencode($url);
$text = file_get_contents($url);
nor
$url = 'http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles='.rawurlencode('Dave Longaberger').'&format=xml&exintro=1';
$text = file_get_contents($url);
Well so i really don't know looks like it is impossible by somehow.
Set the User Agent Header in your curl request, wikipedia replies with error 403 forbidden otherwise.
<?PHP
$url = "http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles=Dave+Longaberger&format=xml&exintro=1";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
$xml = curl_exec($ch);
curl_close($ch);
echo $xml;
?>
Alternatively:
ini_set("user_agent","Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
$url = "http://en.wikipedia.org/w/api.php?action=query&prop=extracts&titles=Dave+Longaberger&format=xml&exintro=1";
$xml = simplexml_load_file($url);
$extracts = $xml->xpath("/api/query/pages/page/extract");
var_dump($extracts);
Look at the note in this php man page
http://php.net/manual/en/function.file-get-contents.php
If you're opening a URI with special characters, such as spaces, you need to encode the URI with urlencode().
I am using this code to get page from internet, but I get result status 0:
$url='http://www.jiwlp.com';
$this->url = $url;
if (isset($this->url)) {
// start cURL instance
$this->ch = curl_init ();
// this tells cUrl to return the data
curl_setopt ($this->ch, CURLOPT_RETURNTRANSFER, 1);
// set the url to download
curl_setopt ($this->ch, CURLOPT_URL, $this->url);
// follow redirects if any
curl_setopt($this->ch,CURLOPT_FOLLOWLOCATION, true);
// tell cURL if the data is binary data or not
curl_setopt($this->ch, CURLOPT_BINARYTRANSFER, $this->binary);
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($this->ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($this->ch, CURLOPT_VERBOSE, 1);
curl_setopt($this->ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($this->ch, CURLOPT_TIMEOUT, 5);
// grabs the webpage from the internet
$this->html = curl_exec($this->ch);
$this->status = curl_getinfo($this->ch, CURLINFO_HTTP_CODE);
print_r(curl_getinfo($this->ch)); // closes the connection
curl_close ($this->ch);
}
What am I doing wrong?
this version works for me, removed the oo
$url = 'http://www.jiwlp.com';
if(isset($url)){
$ch = curl_init();
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_FOLLOWLOCATION,true);
curl_setopt($ch,CURLOPT_BINARYTRANSFER,$binary);
$useragent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER,true);
curl_setopt($ch,CURLOPT_SSL_VERIFYHOST,false);
curl_setopt($ch,CURLOPT_VERBOSE,1);
curl_setopt($ch,CURLOPT_USERAGENT,$useragent);
curl_setopt($ch,CURLOPT_TIMEOUT,5);
curl_exec($ch);
$status = curl_getinfo($ch,CURLINFO_HTTP_CODE);
print_r(curl_getinfo($ch));
curl_close($ch);
}
The response status is already stored in $this->status, I'm assuming you are referring to HTTP response status codes, so instead of
// grabs the webpage from the internet
$this->html = curl_exec($this->ch);
$this->status = curl_getinfo($this->ch, CURLINFO_HTTP_CODE);
print_r(curl_getinfo($this->ch));
Try printing out $this->status instead.
// grabs the webpage from the internet
$this->html = curl_exec($this->ch);
$this->status = curl_getinfo($this->ch, CURLINFO_HTTP_CODE);
print_r($this->status);
check CURL in php disabled function.
if CURL_EXEC is disabled in the webserver, php wont give error instead will [http_code] => 0 header_size] => 0 .......
run phpinfo() from a php page is one of a way to get the list of disabled php functions.
I use this function to get http site/link status:
<?php
function get_link_status($url, $timeout = 10)
{
$ch = curl_init();
// set cURL options
$opts = array(CURLOPT_RETURNTRANSFER => true, // do not output to browser
CURLOPT_URL => $url, // set URL
CURLOPT_NOBODY => true, // do a HEAD request only
CURLOPT_TIMEOUT => $timeout); // set timeout
curl_setopt_array($ch, $opts);
curl_exec($ch); // do it!
$status = curl_getinfo($ch, CURLINFO_HTTP_CODE); // find HTTP status
curl_close($ch); // close handle
echo $status; //or return $status;
//example of check
if ($status == '301') { echo 'This is redirected';}
}
get_link_status('http://example.com');
?>
I have a web service which you post data to and it returns you a txt.gz file. I'm trying to use cURL to post up the information, however I'm not sure how to be ready and handle the file which comes back at me to download.
I'm getting a successful response; but the file is 0 KB, and obviously isn't downloading. Any help would be greatly appreciated!
Here is currently what I'm doing:
$url = 'http://www.mywonderfulurl.com';
$fields = array('id'=>'123');
$fields_string = 'id=123';
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
$ch = curl_init();
// set user agent
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch, CURLOPT_TIMEOUT, 250);
curl_setopt($ch, CURLOPT_FILE, 'download.txt.gz');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
if (file_put_contents('download.txt.gz', curl_exec($ch)) === false) {
// Handle unable to write to file error.
echo('you failed');
exit;
}
echo curl_getinfo( $ch, CURLINFO_HTTP_CODE );
//close connection
curl_close($ch);
You are never writing to the file.
You need to write the result to file.
fwrite($fp, $result);
You are probably better off using file_put_contents, then you don't need to open/close the file.
http://php.net/manual/en/function.file-put-contents.php
if ($file_put_contents('download.txt.gz', curl_exec($ch)) === false) {
// Handle unable to write to file error.
}
But maybe using the $result in between to check the request was actually ok before saving it to file.
You should check the return from curl_exec()
$result = curl_exec($ch);
if ($result === false) {
echo 'Curl error: ' . curl_error($ch);
}
Writing false to a file would result in an empty file, so this could be whats happening.
i am fetching somesite page..
but it display nothing
and url address change.
example i have typed
http://localhost/sushant/EXAMPLE_ROUGH/curl.php
in curl page my coding is=
$fp = fopen("cookie.txt", "w");
fclose($fp);
$agent= 'Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9) Gecko/2008052906 Firefox/3.0';
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
// 2. set the options, including the url
curl_setopt($ch, CURLOPT_URL, "http://www.fnacspectacles.com/place-spectacle/manifestation/Grand-spectacle-LE-ROI-LION-ROI4.htm");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
// 3. execute and fetch the resulting HTML output
if(curl_exec($ch) === false)
{
echo 'Curl error: ' . curl_error($ch);
}
else
echo $output = curl_exec($ch);
// 4. free up the curl handle
curl_close($ch);
but it canege url like this..
http://localhost/aide.do?sht=_aide_cookies_
object not found.
how can solve these problem help me
It looks like you're both trying to save cookies to cookies.txt, and read them from there. What you would normally do is that the first url you visit, you have curl save the cookies to a file. Then, for subseqent requests, you supply that file.
I'm not sure of the php aspects, but from the curl aspects it looks like you're trying to read a cookie file that doesn't exist yet.
edit: oh, and if you're only doing one request, you shouldn't even need cookies.
Seems like there is javascript in the output, which is causing the redirect.
So for testing purpose, instead of using:
echo $output = curl_exec($ch);
Use:
$output = curl_exec($ch);
echo strip_tags($output);
Update:
The code below will put the contents into contents.htm .. everything u need for pasring should be in there and in the output variable.
if(curl_exec($ch) === false)
{
echo 'Curl error: ' . curl_error($ch);
}
else{
$output = curl_exec($ch);
$fp2 = fopen("content.htm" , "w");
fwrite($fp2 , $output);
fclose($fp2);
}