PHP's fopen() can't get url, but curl can - php

A while back, I wrote a little utility function that takes inPath and outPath, opens both and copies from one to the other using fread() and fwrite(). allow_url_fopen is enabled.
Well, I've got a url that I'm trying to get the contents of, and fopen() doesn't get any data, but if I use curl to do the same, it works.
The url in question is: http://www.deltagroup.com/Feeds/images.php?lid=116582497&id=1
fopen version:
$in = #fopen( $inPath, "rb" );
$out = #fopen( $outPath, "wb" );
if( !$in || !$out )
{
echo 0;
}
while( $chunk = fread( $in, 8192 ) )
{
fwrite( $out, $chunk, 8192 );
}
fclose( $in );
fclose( $out );
if( file_exists($outPath) )
{
echo 1;
}
else
{
echo 0;
}
curl version:
$opt = "curl -o " . $outPath . " " . $inPath;
$res = `$opt`;
if( file_exists($outPath) )
{
echo 1;
}
else
{
echo 0;
}
Any idea why this would happen?

Even using php's curl, I was unable to download the file- until I added a curlopt_useragent string. Nothing in the response indicated that it was required (no errors, nothing other than an HTTP 200).
Final code:
$out = #fopen( $outPath, "wb" );
if( !$out )
{
echo 0;
}
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $inPath );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt( $ch, CURLOPT_FILE, $out );
curl_setopt( $ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13');
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 15 );
curl_setopt( $ch, CURLOPT_TIMEOUT, 18000 );
$data = curl_exec( $ch );
curl_close( $ch );
fclose( $out );
if( file_exists($outPath) )
{
echo 1;
}
else
{
echo 0;
}

Related

What is going wrong with this API?

I am coding a website for my friend whom plays Habbo Hotel which is a virtual game. He linked me to some API.
http://habboemotion.com/guide/habinfo & http://habboemotion.com/guide/habboapi
I have been using this code to show the data from the api.
<?php
$info = habbo( "Tyler", "com" );
if( $info ) {
foreach( $info->user AS $name ) {
echo $name->motto;
}
} else {
echo "Habbo not found";
}
?>
Why is nothing appearing? It just appears to be a blank screen.
As the previous person said, make sure to include your habbo() function.
I changed the habbo() function to remove gzip compression and gzinflate(). That seemed to fix the blank page issue. However it seems to take a few seconds to load the page and is on the slow side.
It would also appear that $user->motto isn't allowed. As such, I've replaced it with $friends->motto.
Hope this helps! I am still very new to APIs.
<?php
error_reporting(E_ALL); // Debugging
ini_set('display_errors', 1); // Debugging
function habbo( $name, $hotel ) {
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, "https://www.habbo." . $hotel . "/api/public/users" );
curl_setopt( $ch, CURLOPT_HEADER, false );
curl_setopt( $ch, CURLOPT_HTTPHEADER, array( 'Accept-Encoding: identity' ) ); // Changed to "identity"
curl_setopt( $ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT'] );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 0 );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, 0 );
$response = curl_exec( $ch ); // Added
//$get = gzinflate( substr( curl_exec( $ch ), 10, -8 ) );
//preg_match( "/setCookie\((.*)\);/", $get, $get );
//$get = explode( ",", str_replace( array( "'", " " ), "", $get[1] ) );
//curl_setopt( $ch, CURLOPT_HTTPHEADER, array( "Cookie:" . $get[0] . "=" . $get[1] ) );
curl_setopt( $ch, CURLOPT_URL, "http://www.habbo." . $hotel . "/api/public/users?name=" . $name );
$id = json_decode( curl_exec( $ch ) );
if( isset( $id ) && $id->profileVisible == 1 ) {
curl_setopt( $ch, CURLOPT_URL, "http://www.habbo." . $hotel . "/api/public/users/" . $id->uniqueId . "/profile" );
$info = json_decode( curl_exec( $ch ) );
} else
$info = false;
curl_close( $ch );
return $info;
}
Here is the function call:
$info = habbo( "Tyler", "com" );
if( $info ) {
foreach( $info->friends AS $friend ) {
echo $friend->motto . "<br />";
}
} else {
echo "habbo not found or homepage hidden";
}
?>

Select HTML content using PHP

I want to get the paragraphs under this tag:
I tried to:
<?php
$doc = new DOMDocument();
$doc->loadHTMLFile("https://sabq.org/xMQjz2");
$elements = $doc->getElementsByTagName('p');
if (!is_null($elements)) {
foreach ($elements as $element) {
$nodes = $element->childNodes;
foreach ($nodes as $node) {
echo $node->textContent. "\n";
}
}
}
?>
And I got the paragraphs I wanted along with unwanted ones, and they were duplicated.
EDIT:
I changed the URL, hope it works
The link that you have provided throws an error when accessing it so what I did, I found a function that could get the contents of the webpage using curl instead of the DOMDocument class which you were using.
I used preg_match and regex to extract the specific element that you were looking for.
Here's the code:
<?php
//opened url
$content = get_fcontent("https://sabq.org/%D8%B4%D8%A7%D9%87%D8%AF-%D8%A3%D9%84%D9%81-%D8%B5%D9%81%D8%AD%D8%A9-%D8%AA%D8%B1%D9%88%D9%8A-%D9%82%D8%B5%D8%B5-%D8%A7%D9%84%D8%AD%D8%B1%D9%85%D9%8A%D9%86-%D9%85%D9%86%D8%B0-%D8%A7%D9%86%D8%B7%D9%84%D8%A7%D9%82-%D8%A7%D9%84%D8%B9%D9%87%D8%AF-%D8%A7%D9%84%D8%B3%D8%B9%D9%88%D8%AF%D9%8A");
//extract specific html tag and its innerHTML
preg_match('/<p .*? ng\-bind\-html\=\"getContent\(material\.content\)\" .*?>.*?<\/p>/m', $content[0], $matches);
//display the wanted element
echo $matches[0];
//getting contents using curl because threw error: failed to open stream
function get_fcontent( $url, $javascript_loop = 0, $timeout = 5 ) {
$url = str_replace( "&", "&", urldecode(trim($url)) );
$cookie = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init();
curl_setopt( $ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1" );
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_COOKIEJAR, $cookie );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_ENCODING, "" );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false ); # required for https urls
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, $timeout );
curl_setopt( $ch, CURLOPT_TIMEOUT, $timeout );
curl_setopt( $ch, CURLOPT_MAXREDIRS, 10 );
$content = curl_exec( $ch );
$response = curl_getinfo( $ch );
curl_close ( $ch );
if ($response['http_code'] == 301 || $response['http_code'] == 302) {
ini_set("user_agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.7.3) Gecko/20041001 Firefox/0.10.1");
if ( $headers = get_headers($response['url']) ) {
foreach( $headers as $value ) {
if ( substr( strtolower($value), 0, 9 ) == "location:" )
return get_url( trim( substr( $value, 9, strlen($value) ) ) );
}
}
}
if ( ( preg_match("/>[[:space:]]+window\.location\.replace\('(.*)'\)/i", $content, $value) || preg_match("/>[[:space:]]+window\.location\=\"(.*)\"/i", $content, $value) ) && $javascript_loop < 5) {
return get_url( $value[1], $javascript_loop+1 );
} else {
return array( $content, $response );
}
}
?>
For testing, I created a local file called test.html:
<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body>
<p>This should not be showing.</p>
<p ng-bind-html="getContent(material.content)" id="dev-content" class="details-text">This is a test.</p>
</body>
</html>
I used the local url http://localhost/example/test.html instead of the link you provided for testing purposes.
And from the local file I created for testing, I got the following result:
<p ng-bind-html="getContent(material.content)" id="dev-content" class="details-text">This is a test.</p>
Here's the result that I got from the original url:
<p ng-bind-html="getContent(material.content)" id="dev-content" class="details-text"></p>
I hope this helps!

Unable to download remote mp3 file to server using PHP

I am trying to download a pronunciation file (approx. 8kb) to server using a server-side PHP. Taking cue from a number of threads discussing this issue, I tried the following:
$numwrd = str_word_count($wrd);
if($numwrd == 1){
$html = file_get_html("http://www.dictionaryapi.com/api/v1/references/spanish/xml/" . rawurlencode($wrd) . "?key=" . rawurlencode('6d4d41f9-c28f-4544-9bb3-1b4708d1a4d1'));
$sn = $html->find('sound');
if($sn[0] != ""){
$foldername = findsub($sn[0]->plaintext);
$filename = explode(".", $sn[0], 2)[0];
$audiofn = $foldername . $filename . '.mp3';
$soundurl = 'http://media.merriam-webster.com/audio/prons/es/me/mp3/' . $foldername . '/' . $filename . '.mp3';
$path = 'amit.mp3';
$headers = getHeaders($soundurl);
if ($headers['http_code'] === 200 and $headers['download_content_length'] < 1024*1024) {
if (download($url, $path)){
return $audiofn . " " . $soundurl;
}
}
}
else { return "not found"; }
}
else { return "not found"; }
function getHeaders($url)
{
$ch = curl_init($url);
curl_setopt( $ch, CURLOPT_NOBODY, true );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, false );
curl_setopt( $ch, CURLOPT_HEADER, false );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_MAXREDIRS, 3 );
curl_exec( $ch );
$headers = curl_getinfo( $ch );
curl_close( $ch );
return $headers;
}
function download($url, $path)
{
# open file to write
$fp = fopen ($path, 'w+');
# start curl
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
# set return transfer to false
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, false );
curl_setopt( $ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
# increase timeout to download big file
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 10 );
# write data to local file
curl_setopt( $ch, CURLOPT_FILE, $fp );
# execute curl
curl_exec( $ch );
# close curl
curl_close( $ch );
# close local file
fclose( $fp );
if (filesize($path) > 0) return true;
}
This didn't work so I tried again with file_get_contents. This method however only creates the file but with zero bytes. The values in $foldername, $filename, $audiofn, and $soundurl are evaluating correctly and all these variables have been tested. I can manually download the file by browsing to the URL, right clicking in the browser, and clicking download file as.... What could be wrecking my PHP?
P.S.: I just tried a modified function using cURLand this failed too:
function down($url, $target){//feeding it $soundurl and $path values
set_time_limit(0);
$file = fopen(dirname(__FILE__) . $target, 'w+');
$curl = curl_init($url);
curl_setopt_array($curl, [
CURLOPT_URL => $url,
CURLOPT_BINARYTRANSFER => 1,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FILE => $file,
CURLOPT_TIMEOUT => 50,
CURLOPT_USERAGENT => 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)'
]);
$response = curl_exec($curl);
if($response === false) {
throw new \Exception('Curl error: ' . curl_error($curl));
}
$response;
}
Finally got it to work! This is what fixed it (line 7):
$filename = explode(".", $sn[0]->plaintext, 2)[0];
The reason I had to add the ->plaintext attribute is because without it, the value being returned to $filename was an xml tag instead of the text inside of that tag. Since the following line of code takes this value as input, this was corrupting the URL being called for download:
$soundurl = 'http://media.merriam-webster.com/audio/prons/es/me/mp3/' . $foldername . '/' . $filename . '.mp3';
Now the file downloads successfully because the URL is being formed correctly.

Download image from remote website and show to use

What is the quickest most memory efficient way to download a remote image and then display that to the user? - Note I don't want to save it, I just want to pass it on to the user.
i.e.
a user goes to www.website.com/image1.jpg
In the background I would use a getFile script similar to this to retrieve the file, but how do I display this to the user and is this a memory efficient way of doing it?
function getUrl($url, $method='', $vars='', $fh = '') {
$ch = curl_init();
if ($method == 'post') {
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $vars);
}
if ($fh != '')
{
curl_setopt($ch, CURLOPT_FILE, $fh);
}
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies/cookies.txt');
$buffer = curl_exec($ch);
curl_close($ch);
return $buffer;
}
Here it is:
function proxyImage( $fromHost, $path, $bufsize=4096 ) {
$conn = fsockopen($fromHost,80);
fwrite( $conn, "GET {$path} HTTP/1.0\r\n" );
fwrite( $conn, "Host: {$fromHost}\r\n" );
fwrite( $conn, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20100101 Firefox/8.0\r\n" );
fwrite( $conn, "Connection: close\r\n" );
fwrite( $conn, "\r\n" );
$answer = '';
while( !feof($conn) and !$bodystarted ) {
$portion = fread( $conn, $bufsize );
if( strpos($portion,"\r\n\r\n")!==false ) $bodystarted = true;
$answer .= $portion;
}
list( $headers, $bodypart ) = explode( "\r\n\r\n", $answer, 2 );
foreach( explode("\r\n",$headers) as $h )
header($h);
echo $bodypart;
while( !feof($conn) )
echo fread( $conn, $bufsize );
fclose($conn);
}
proxyImage( 'www.newyorker.com', '/online/blogs/photobooth/NASAEarth-01.jpg' );

how to know if a certain resource exist?

For instance, I want to know if these files exists.
http://www.stackoverflow.com/favicon.ico
http://www.stackoverflow.com/reset.css
and then download it ( if exixts obviously ).
<?php
if( ( $file = file_get_contents( 'http://www.stackoverflow.com/favicon.ico' ) ) ) {
echo "file exists.";
file_put_contents( 'favicon.ico', $file );
}
else {
echo "File does not exist.";
}
Try this:
<?php
$ch = curl_init();
$url = 'YOUR_URL_HERE'; // the url you want to check
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 20 );
curl_setopt( $ch, CURLOPT_USERAGENT, $_SERVER[ 'HTTP_USER_AGENT' ] );
// make "HEAD" request
curl_setopt( $ch, CURLOPT_HEADER, true );
curl_setopt( $ch, CURLOPT_NOBODY, true );
$res = curl_exec( $ch );
$res = explode( ' ', substr( $res, 0, strpos( $res, "\n" ) ) );
// if 404, file does not exist
if( $res[ 1 ] != 404 ) {
$file = file_get_contents( $url );
} else {
// This url does not exist
$file = '';
}
curl_close( $ch );
?>
Hope this helps.

Categories