I tried to open an external url. It works fine in my local server. But when i moved to live server it showing time out error.
When i replaced the url by a url in the same domain ,it works fine.
allow_url_fopen is ON in the server.
<?php
if ($fp = fopen('https://www.google.com/', 'r')) {
$content = '';
// keep reading until there's nothing left
while ($line = fread($fp, 1024)) {
$content .= $line;
}
echo $content;
echo 'do something with the content here';
// ...
} else {
echo 'an error occured when trying to open the specified url';
}
?>
Updated
$curl_handle=curl_init();
curl_setopt($curl_handle,CURLOPT_URL,'https://www.google.co.in/');
curl_setopt($curl_handle,CURLOPT_CONNECTTIMEOUT,2);
curl_setopt($curl_handle,CURLOPT_RETURNTRANSFER,1);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
if (empty($buffer)){
print "Nothing returned from url..<p>";
}
else{
print $buffer;
}
I tried cURL too. It returns "Nothing returned from url..". But it works fine in my local and demo server.
You should use curl to get data from third party URL.
please check the below link and example :
What is cURL in PHP?
Example :
$ch = curl_init();
$curlConfig = array(
CURLOPT_URL => "http://www.example.com/yourscript.php",
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_POSTFIELDS => array(
'field1' => 'some date',
'field2' => 'some other data',
)
);
curl_setopt_array($ch, $curlConfig);
$result = curl_exec($ch);
curl_close($ch);
Related
I want to grab this number (28/28) with PHP from a tracking software (https://aircrasher.livefpv.com/live/scoring/) for drone races. But I can't. I think it's a websocket, but I don't know how to retrieve information from it. I tried curl, I tried the "ultimate-web-scraper" from GitHub (https://github.com/cubiclesoft/ultimate-web-scraper). I tried a code example from another post from here (PHP simple web socket client) but I'm not able to get this number. Is it imposible to get this number (with php)?
This is what I tried so far with the help of "ultimate-web-scraper" (https://kleesit.de/beta/livetime/webscraper/test.php):
<?php
require_once "support/web_browser.php";
require_once "support/tag_filter.php";
// Retrieve the standard HTML parsing array for later use.
$htmloptions = TagFilter::GetHTMLOptions();
// Retrieve a URL (emulating Firefox by default).
$url = "https://aircrasher.livefpv.com/live/scoring/";
$web = new WebBrowser();
$result = $web->Process($url);
// Check for connectivity and response errors.
if (!$result["success"])
{
echo "Error retrieving URL. " . $result["error"] . "\n";
exit();
}
if ($result["response"]["code"] != 200)
{
echo "Error retrieving URL. Server returned: " . $result["response"]["code"] . " " . $result["response"]["meaning"] . "\n";
exit();
}
// Get the final URL after redirects.
$baseurl = $result["url"];
// Use TagFilter to parse the content.
$html = TagFilter::Explode($result["body"], $htmloptions);
// Find all anchor tags inside a div with a specific class.
// A useful CSS selector cheat sheet: https://gist.github.com/magicznyleszek/809a69dd05e1d5f12d01
echo "All the URLs:\n";
$result2 = $html->Find("div.someclass a[href]");
if (!$result2["success"])
{
echo "Error parsing/finding URLs. " . $result2["error"] . "\n";
exit();
}
foreach ($result2["ids"] as $id)
{
// Faster direct access.
echo "\t" . $html->nodes[$id]["attrs"]["href"] . "\n";
echo "\t" . HTTP::ConvertRelativeToAbsoluteURL($baseurl, $html->nodes[$id]["attrs"]["href"]) . "\n";
}
// Find all table rows that have 'th' tags.
// The 'tr' tag IDs are returned.
$result2 = $html->Filter($html->Find("tr"), "th");
if (!$result2["success"])
{
echo "Error parsing/finding table rows. " . $result2["error"] . "\n";
exit();
}
foreach ($result2["ids"] as $id)
{
echo "\t" . $html->GetOuterHTML($id) . "\n\n";
}
?>
This is what I tried with CURL and the help of a stackoverflow post (https://kleesit.de/beta/livetime/test2.php):
<?php
// random 7 chars digit/alphabet for "t" param
$random = substr(md5(mt_rand()), 0, 7);
$socket_server='https://aircrasher.livefpv.com/live/scoring/';
// create curl resource
$ch = curl_init();
//N°1 GET request
curl_setopt_array(
$ch,
[
CURLOPT_URL => $socket_server,
CURLOPT_RETURNTRANSFER => TRUE,
// since it is TRUE by default we should disable it
// on our localhost, but from real https server it will work with TRUE
CURLOPT_SSL_VERIFYPEER => FALSE
]);
// $output contains the output string
$output = curl_exec($ch);
$output=substr($output, 1);
echo $socket_server;
// server response in my case was looking like that:
// '{"sid":"4liJK2jWEwmTykwjAAAR","upgrades":["websocket"],"pingInterval":25000,"pingTimeout":20000,"maxPayload":1000000}'
var_dump($output);
$decod=json_decode($output);
// setting ping Interval accordingly with server response
$pingInterval = $decod->pingInterval;
//N°2 POST request
$socket_server_with_sid = $socket_server.'&sid='.$decod->sid;
curl_setopt_array(
$ch,
[
CURLOPT_URL => $socket_server_with_sid,
CURLOPT_POST => TRUE,
CURLOPT_TIMEOUT_MS => $pingInterval,
// 4 => Engine.IO "message" packet type
// 0 => Socket.IO "CONNECT" packet type
CURLOPT_POSTFIELDS => '40'
]);
$output = curl_exec($ch);
// ok
var_dump($output);
// Pervious 2 requests are called "hand shake" now we can send a message
// N°3 socket.emit
if ($output==='ok') {
curl_setopt_array(
$ch,
[
CURLOPT_URL => $socket_server_with_sid,
CURLOPT_TIMEOUT_MS => $pingInterval,
CURLOPT_POST => TRUE,
// 4 => Engine.IO "message" packet type
// 2 => Engine.IO "EVENT" packet type
CURLOPT_POSTFIELDS => '42["chat message","0devhost message 10"]'
]);
$output = curl_exec($ch);
// ok
echo $output.'<br/>';
echo $socket_server_with_sid;
}
// close curl resource to free up system resources
curl_close($ch);
?>
I wrote a script on my web server. when I run my code using the URL (http://www.eurekabd.com/RMS/corn.php) and found ok. Then I made a corn job scheduler in cPanel by this command /usr/local/bin/php /home/eurekabd/public_html/RMS/corn.php. When cron job run this command it sends me mail that
Warning: file_get_contents(shakil.txt): failed to open stream: No such file or directory in /home/eurekabd/public_html/RMS/corn.php on line 12
<?php
$response = get_web_page("http://144.48.2.11/Test_PHP/live_check.php");
//$response = get_web_page("http://localhost/phpScriptTesting/request.php");
$txtFile = "shakil.txt";
echo $response;
/*if(! filesize( $txtFile ) )
{
echo " Not EMPTY FILE ";
}*/
echo "Server is =".$response;
if ((trim(file_get_contents('shakil.txt')) != false) && ($response == "OK")) {
echo "-->I have some data to send.";
//http://144.48.2.11/Test_PHP/index_2.php/?data={1239{fnc:15,sts:[0,8,25,526,19][0103044396424b7f0c][01030caf854366a45b436695684366376d][01030c9ba63b447bb33b7249523a1d4641][01030c00003f8000003f8000003f80d6f8][010304999a3f1924ba][x][x][x][x][x][0b03080113045403ff03f057ea][x]}})
$file_content = file_get_contents("shakil.txt");
$file_content_separated_by_spaces = explode(";", $file_content);
//print_r ($file_content_separated_by_spaces)."</br>";
$loop_size = sizeof($file_content_separated_by_spaces)-2;
echo $loop_size;
for ($x = 0; $x <= $loop_size; $x++) {
$dataFormatforServer = str_replace(' ', '_', $file_content_separated_by_spaces[$x]);
$textData = "http://144.48.2.11/Test_PHP/index_3.php/?data=".$dataFormatforServer;
//$textData = "http://localhost/phpScriptTesting/test.php/?data=".$dataFormatforServer;
echo $textData."</br>";
$sendData = get_web_page($textData);
//echo $sendData."</br>";
}
$file_to_delete = 'shakil.txt';
unlink($file_to_delete);
$my_file = 'shakil.txt';
$handle = fopen($my_file, 'w') or die('Cannot open file: '.$my_file);
}
else{
echo "I don't Have Any Data to send.";
}
function get_web_page($url) {
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => false, // don't return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
CURLOPT_ENCODING => "", // handle compressed
CURLOPT_USERAGENT => "test", // name of client
CURLOPT_AUTOREFERER => true, // set referrer on redirect
CURLOPT_CONNECTTIMEOUT => 300, // time-out on connect
CURLOPT_TIMEOUT => 300, // time-out on response
);
$ch = curl_init($url);
curl_setopt_array($ch, $options);
$response = curl_getinfo($ch, CURLINFO_HTTP_CODE);
$content = curl_exec($ch);
curl_close($ch);
return $content;
}
?>
you may add/change php.ini attributes for that..
allow_url_fopen = On
It may resolve the issue..
I am trying to retrieve the html from file get contents in php then save it to a php file so I can include it into my homepage.
Unfortunately my script isn't saving the data into the file. I also need to verwrite this data on a daily basis as it will be setup with a cron job.
Can anyone tell me where I am going wrong please? I am just learning php :-)
<?php
$richSnippets = file_get_contents('http://website.com/data');
$filename = 'reviews.txt';
$handle = fopen($filename,"x+");
$somecontent = echo $richSnippets;
fwrite($handle,$somecontent);
echo "Success";
fclose($handle);
?>
A couple of things,
http://website.com/data gets a 404 error, it doesn't exist.
Change your code to
$site = 'http://www.google.com';
$homepage = file_get_contents($site);
$filename = 'reviews.txt';
$handle = fopen($filename,"w");
fwrite($handle,$homepage);
echo "Success";
fclose($handle);
Remove $somecontent = echo $richSnippets; it doesn't do anything.
if you have the proper permissions it should work.
Be sure that your pointing to an existing webpage.
Edit
When cURL is enabled you can use the following function
function get_web_page( $url ){
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => false, // don't return headers
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_USERAGENT => "spider", // who am i
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
);
$ch = curl_init( $url );
curl_setopt_array( $ch, $options );
$content = curl_exec( $ch );
curl_close( $ch );
return $content;
}
Now change
$homepage = file_get_contents($site);
in to
$homepage = get_web_page($site);
You should use / instead of ****
$homepage = file_get_contents('http://website.com/data');
Also this part
$somecontent = echo $richSnippets;
I don't see $richSnippets above... it's probably not declared?
You probably want to do this:
fwrite($handle,$homepage);
When i'm trying to invoke the YQL via cURL i'm getting the following error.
HTTP Version Not Supported
Description: The web server "engine1.yql.vip.bf1.yahoo.com" is using an unsupported version of the HTTP protocol.
Following is the code used
// URL
$URL = "https://query.yahooapis.com/v1/public/yql?q=select * from html where url=\"http://www.infibeam.com/Books/search?q=9788179917558\" and xpath=\"//span[#class='infiPrice amount price']/text()\"&format=json";
// set url
curl_setopt($ch, CURLOPT_URL, $URL);
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
echo $output;
?>
Invoking the same URL from thr browser works fine
https://query.yahooapis.com/v1/public/yql?q=select * from html where
url="http://www.infibeam.com/Books/search?q=9788179917558" and
xpath="//span[#class='infiPrice amount price']/text()"&format=json
Can someone please point me what is wrong in the code?
The problem is probably caused because the url you feed to cURL is not valid. You need to prepare / encode the individual values of the query strings for use in a url.
You can do that using urlencode():
$q = urlencode("select * from html where url=\"http://www.infibeam.com/Books/search?q=9788179917558\" and xpath=\"//span[#class='infiPrice amount price']/text()\"");
$URL = "https://query.yahooapis.com/v1/public/yql?q={$q}&format=json";
In this case I have only encoded the value of q as the format does not contain characters that you cannot use in a url, but normally you'd do that for any value you don't know or control.
Okay I gottacha .. The problem was with the https. Used the following snippet for debug
if (false === ($data = curl_exec($ch))) {
die("Eek! Curl error! " . curl_error($ch));
}
Added below code to accept SSL certificates by default.
$options = array(CURLOPT_URL => $URL,
CURLOPT_HEADER => "Content-Type:text/xml",
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_RETURNTRANSFER => TRUE
);
Complete code is here
<?php
// create curl resource
$ch = curl_init();
// URL
$q = urlencode("select * from html where url=\"http://www.infibeam.com/Books/search?q=9788179917558\" and xpath=\"//span[#class='infiPrice amount price']/text()\"");
$URL = "https://query.yahooapis.com/v1/public/yql?q={$q}&format=json";
echo "URL is ".$URL;
$ch = curl_init();
//Define curl options in an array
$options = array(CURLOPT_URL => $URL,
CURLOPT_HEADER => "Content-Type:text/xml",
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_RETURNTRANSFER => TRUE
);
//Set options against curl object
curl_setopt_array($ch, $options);
//Assign execution of curl object to a variable
$data = curl_exec($ch);
echo($data);
//Pass results to the SimpleXMLElement function
//$xml = new SimpleXMLElement($data);
echo($data);
if (false === ($data = curl_exec($ch))) {
die("Eek! Curl error! " . curl_error($ch));
}
if (200 !== (int)curl_getinfo($ch, CURLINFO_HTTP_CODE)) {
die("Oh dear, no 200 OK?!");
}
//Close curl object
curl_close($ch);
?>
We're trying to debug some cURL errors on the server, and I would like to see the STDERR log. Currently, all we can see for our error is "error code: 7" and that we can't connect to target server. We have contacted the host and made special rule to open the port we need and we're even ignoring the certificate for the time being.
Still, we can't connect. I need to debug this, but I can't see any pertinent information on my end.
The lines mentioning "VERBOSE" and "STDERR" are the most important, I think. Nothing is written to $curl_log. What am I doing wrong? Following the manuals logic, this should be correct...
PHP in use:
<?php
$curl = curl_init();
$curl_log = fopen("curl.txt", 'w');
$url = "http://www.google.com";
curl_setopt_array($curl, array(
CURLOPT_URL => $url, // Our destination URL
CURLOPT_VERBOSE => 1, // Logs verbose output to STDERR
CURLOPT_STDERR => $curl_log, // Output STDERR log to file
CURLOPT_SSL_VERIFYPEER => 0, // Do not verify certificate
CURLOPT_FAILONERROR => 0, // true to fail silently for http requests > 400
CURLOPT_RETURNTRANSFER => 1 // Return data received from server
));
$output = fread($curl_log, 2048);
echo $output; // This returns nothing!
fclose($curl_log);
$response = curl_exec($curl);
//...restofscript...
?>
From PHP manual: http://php.net/manual/en/function.curl-setopt.php
CURLOPT_VERBOSE TRUE to output verbose information. Writes output to STDERR
CURLOPT_STDERR An alternative location to output errors to instead of STDERR.
It is not a permission issue either, I have set file and script permissions to 777 on server side and my local client is windows and has never cared about permission settings (it's only for dev anyway).
You are making couple mistakes in your example:
1) you have to call curl_exec() prior to reading from the "verbose log", because curl_setopt() doesn't perform any action, so nothing can be logged prior to the curl_exec().
2) you are opening $curl_log = fopen("curl.txt", 'w'); only for write, so nothing could be read, even after you write to the file and rewind the internal file pointer.
So the correct shortened code should look like:
<?php
$curl = curl_init();
$curl_log = fopen("curl.txt", 'rw'); // open file for READ and write
$url = "http://www.google.com";
curl_setopt_array($curl, array(
CURLOPT_URL => $url,
CURLOPT_VERBOSE => 1,
CURLOPT_STDERR => $curl_log,
CURLOPT_RETURNTRANSFER => 1
));
$response = curl_exec($curl);
rewind($curl_log);
$output= fread($curl_log, 2048);
echo "<pre>". print_r($output, 1). "</pre>";
fclose($curl_log);
// ...
?>
NOTE: verbose log could be longer than 2048 bytes, so you could "fclose" the $curl_log after curl_exec() and then read the whole file with for example file_get_contents().
In that case, the point 2) should not be considered as mistake :-)
A bit late to the party, but this page still pops up high in Google, so let's go.
It seems that CURLOPT_VERBOSE doesn't log anything if CURLINFO_HEADER_OUT is also set to TRUE.
This is a know bug in PHP (#65348), and due to reasons they decided not to fix it.
Putting al above answers together, I use this function to make a Curl Post Request with loggin to a file option:
function CURLPostRequest($url, array $post = NULL, array $options = array(), $log_file = NULL){
$defaults = array(
CURLOPT_POST => 1,
CURLOPT_HEADER => 0,
CURLOPT_URL => $url,
CURLOPT_FRESH_CONNECT => 1,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FORBID_REUSE => 1,
CURLOPT_TIMEOUT => 4,
CURLOPT_POSTFIELDS => http_build_query($post)
);
if (is_resource($log_file)){
$defaults[CURLOPT_VERBOSE]=1;
$defaults[CURLOPT_STDERR]=$log_file;
$defaults[CURLINFO_HEADER_OUT]=1;
}
$ch = curl_init();
curl_setopt_array($ch, ($options + $defaults));
if( ! $result = curl_exec($ch)){
throw new Exception(curl_error($ch));
}
if (is_resource($log_file)){
$info = curl_getinfo($ch);
if (isset($info['request_header'])){
fwrite($log_file, PHP_EOL.PHP_EOL.'* POST Content'.PHP_EOL.PHP_EOL);
fwrite($log_file, print_r($info['request_header'],true));
fwrite($log_file, http_build_query($post));
}
fwrite($log_file, PHP_EOL.PHP_EOL.'* Response Content'.PHP_EOL.PHP_EOL);
fwrite($log_file, $result.PHP_EOL.PHP_EOL);
}
curl_close($ch);
return $result;
}
Hope this help to someone.
I needed to close the file before being able to read it, this worked for me:
$filename = 'curl.txt';
$curl_log = fopen($filename, 'w'); // open file for write (rw, a, etc didn't help)
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, $curl_log);
$result = curl_exec($ch);
fclose($curl_log);
$curl_log = fopen($filename, 'r'); // open file for read
$output= fread($curl_log, filesize($filename));
echo $output;
(PHP 5.6.0, Apache/2.2.15)
From php manual for function curl_setopt:
CURLOPT_FILE The file that the transfer should be written to. The default is STDOUT (the browser window).
You should put
$output = fread($curl_log, 2048);
echo $output; // This returns nothing!
fclose($curl_log);
after $response = curl_exec($curl); otherwise, file is closed during curl is executing.