Comparing HTTP_REFERER to a value read from file fails - php

I'm attempting to pass a key via the $_SERVER['HTTP_REFERER'] (set via cURL) from one server to another, to then process some POST data and send an XML file back. I have the key stored in identical files on both servers (rotating them via cron job SCP) but when I read the key on the receiving end and compare to the referer it fails every time and throws the 418 back to me.
If I use if($refkey = "mykeyhere") instead of if($refkey = $key) it works properly, but obviously hard coding they key isn't going to work here - I need to be able to read it from a file.
I've tried using strstr(), doing a cast with (string)$key however gettype() on both returns string so it should work. The key is being sent properly in $_SERVER['HTTP_REFERER'] and being read in properly from the file on the receiving end, I've echo'd both and they're identical. Does anyone know why the comparison would fail, am I doing something stupidly, and obviously wrong? Thanks.
cURL request build code:
$file = "key";
$fh = fopen($file, 'r');
$key = fread($fh, filesize($file));
fclose($fh);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://example.com/xmlreq.php");
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FORBID_REUSE, 1);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_REFERER, $key);
curl_setopt($ch, CURLOPT_POSTFIELDS, "greeting=hello" );
curl_exec($ch);
curl_close($ch);
receiving code (xmlreq.php):
if($_SERVER['REQUEST_METHOD'] === "POST" ){
$file = "key";
$fh = fopen($file, "r");
$key = fread($fh, filesize($file));
fclose($fh);
$refkey = $_SERVER['HTTP_REFERER'];
if($refkey == $key){
print_r("hello, world!"); // it should run this
}else
header("HTTP/1.0 418 I'm a teapot"); // but runs this instead
}else
header("HTTP/1.0 404 Not Found");

As a first troubleshooting step, on both sides, replace $key with trim($key). Alternatively, on the receiving side, change your if condition to if (trim($refkey) == trim($key)) {
The file may contain a newline/carriage-return after the key, and it may NOT send that extra character in the referrer variable. The receiving side may be trying to compare the referrer with the one from the file, and the file version might have the newline included.

Related

Curl and php download file from url not working

I'm trying do retrieve and download a file (image) from a remote location.
Inside the php.ini the allow_url_fopen is enabled, but i can't download the image.
Code i'm using is described below
$local_file = "test.jpg";
$remote_file = "http://somehost:6346/cgi-bin/CGIProxy.fcgi?cmd=snapPicture2&usr=xxxx&pwd=xxxx";
$ch = curl_init();
$fp = fopen ($local_file, 'w+');
$ch = curl_init($remote_file);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_ENCODING, "");
curl_exec($ch);
curl_close($ch);
fclose($fp);
with any other url that contains a real jpg file, it's working perfectly, i suppose that the issue is that the url use some special characters that doesn't like to curl.
If i try to execute the php snippet above,page load for almost 1 minute,and it seems that no error are displayed,the image test.jpg is created, but it's empty.
Do you have any suggestion?
Thanks!
Try this
$local_file = "test.jpg";
$remote_file = "http://somehost:6346/cgi-bin/CGIProxy.fcgi?cmd=snapPicture2&usr=xxxx&pwd=xxxx";
function getPage($url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
function saveToFile($base, $decode=false, $output_file)
{
$ifp = fopen($output_file, "wb");
if ($decode){
fwrite($ifp, base64_decode($base));
}else{
fwrite($ifp, $base);
}
fclose($ifp);
return($output_file);
}
$remote_page = getPage($remote_file);
$saved_file = saveToFile($remote_page , false, $local_file);
when debugging issues like this, set CURLOPT_VERBOSE, it will probably reveal why the page loaded for almost 1 minute, with no apparent output.
i suppose that the issue is that the url use some special characters - this is fully possible, for example your username and password, they're supposed to be urlencoded. urlencoding is binary safe, meaning you can have any special characters you'd like, you just need to encode it properly. use urlencode() or http_build_query() for that, eg
$remote_file = "http://somehost:6346/cgi-bin/CGIProxy.fcgi?" . http_build_query ( array (
'cmd' => 'snapPicture2',
'usr' => 'username',
'pwd' => 'password'
) );
now http_build_query will properly urlencode any special characters in your username and password (for example, if your username is an email address, the # becomes %40).
if that doesn't fix it, what does CURLOPT_VERBOSE say?
also, final note, here you're sending the download request with credentials in a GET request. that's very unusual, the vast majority of websites want you to login with a POST request, and there are good security-related reasons for that, are you sure your website allows sending credentials in GET parameters? the vast majority of websites doesn't allow it... (and the best way to find out, is to record a browser logging in, does the browser use GET parameters, or POST parameters?)

Curl/PHP challenge transferring image and base64_encode/decode

I am transfering data from one linux-box to another. In generel this works fine, BUT I am having trouble when it comes to transferring images. I have tested all kinds of stuff. I hope some one may be able to help me out.
$filename = "/home/user/image.jpg";
$handle = fopen($filename, "r");
$data = fread($handle, filesize($filename));
$data = base64_encode($data);
# Transfer reading
#$arrIn['changeCharset'] = "true";
$arrIn['postFields'] = "action=test&data=$base64";
$test = curlServerPost($arrIn);
Here is my CURL-function:
function curlServerPost($arrIn)
{
$postFields = $arrIn['postFields'];
$url = "$GLOBALS[remoteSite]"; // Where you want to post data
$ch = curl_init(); // Initiate cURL
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true); // Tell cURL you want to post something
curl_setopt($ch, CURLOPT_POSTFIELDS, $postFields); // Define what you want to post
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // Return the output in string format
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Accepter alle certifikater, se: http://unitstep.net/blog/2009/05/05/using-curl-in-php-to-access-https-ssltls-protected-sites/
$res = curl_exec($ch); // Execute
curl_close($ch); // Close cURL handle
return($res);
}
Finally, here is the receiver (server) part
$data = $_POST[data];
$data = base64_decode($data);
$stmt = $GLOBALS[pdo]->prepare("INSERT INTO cameraImages (cameraImagesId) VALUES ('')");
$stmt->execute();
$cameraImagesId = $GLOBALS[pdo]->lastInsertId();
$stmt = $GLOBALS[pdo]->prepare("UPDATE cameraImages SET cameraImagesFile='$data' WHERE cameraImagesId='$cameraImagesId'");
$stmt->execute();
Some last remarks:
- If I don't base_decode on server-side I receive (but completely wrong format. Errors in image)
If I base_decode om server-side. Nothing is received.
I would like to don't encode/decode at all. If I do that. Only a small part of the image is stored (corrupted image)
My datafield is LONGBLOB (mysql)
PHP on server side is: PHP 5.3.3
PHP on client side is: PHP 7.0.27-0+deb9u1
I have tried all kinds of stuff. Followed all kinds of tutorials. It just won't work for me :-/
So if anyone can come up with ideas I am more than willing to test and try :)
Loooking forward to hear from you.
Succes. I just needed
urlencode($data);
And removed base64_encode($data); / base64_decode($data);
I have dropped putting images in database since many ppl don't recommend it.

Collecting file with PHP CURL after validating request downloads an empty file

I am doing a system where one of my sites goes to the other to get documents.
On the first site I am using Curl to make a request to get the file wanted:
I am using the solution from Download file from URL using CURL :
function collect_file($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, false);
curl_setopt($ch, CURLOPT_REFERER, "http://example.com");
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HEADER, 0);
$result = curl_exec($ch);
curl_close($ch);
echo $result;
return($result);
}
function write_to_file($text,$new_filename){
$fp = fopen($new_filename, 'w');
fwrite($fp, $text);
fclose($fp);
}
$curlUrl = 'http://site2.com/file-depository/14R4NP8JkoIHwIyjnexSUmyJibdpHs5ZpFs3NLFCxcs54kNhHj';
$new_file_name = "testfile-new.png";
$temp_file_contents = collect_file($curlUrl);
write_to_file($temp_file_contents,$new_file_name);
I am testing downloading an image. If i use a direct URL into $curlUrl , for instance http://site2.com/file-depository/image.png it works perfect.
What I am doing is that the URL http://site2.com/file-depository/14R4NP8JkoIHwIyjnexSUmyJibdpHs5ZpFs3NLFCxcs54kNhHj is then parsed and checked against a database to match the document requested, once there is a document matched I need to provide this document to the Curl response.
I have tried many ways to read the file but everytime i am getting a file on the other end but it is only 1kb in size (45 expected) and when trying to open it i get an error unkown file type etc.
On the second site, once the URL is validated here is what I have:
$file = readfile('some-image.png');
echo $file;
I am guessing there is part of the information which belongs to the file missing but can't figure it out, any pointers appreciated!
I have replaced
function write_to_file($text,$new_filename){
$fp = fopen($new_filename, 'w');
fwrite($fp, $text);
fclose($fp);
}
by file_put_contents($new_file_name,trim($temp_file_contents));
Please note the trim(), the issue was that I was apparently collecting some empty space in front of the file content.

Save copy of a photo from Google Places API Place Photos using curl

I'm trying to grab a photo from Google Place Photos using curl and save it on my server.
The request format as per the Google API documentation is like this:
https://maps.googleapis.com/maps/api/place/photo?maxwidth=400&photoreference=CoQBegAAAFg5U0y-iQEtUVMfqw4KpXYe60QwJC-wl59NZlcaxSQZNgAhGrjmUKD2NkXatfQF1QRap-PQCx3kMfsKQCcxtkZqQ&sensor=true&key=AddYourOwnKeyHere
So I tried this function:
function download_image1($image_url, $image_file){
$fp = fopen ($image_file, 'w+');
$ch = curl_init($image_url);
// curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // enable if you want
curl_setopt($ch, CURLOPT_FILE, $fp); // output to file
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 1000); // some large value to allow curl to run for a long time
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0');
// curl_setopt($ch, CURLOPT_VERBOSE, true); // Enable this line to see debug prints
curl_exec($ch);
curl_close($ch); // closing curl handle
fclose($fp); // closing file handle
}
download_image1($photo, "test.jpg");
..where $photo holds the request url.
This is not working, it saves an empty image with header errors, it probably is because the request is not the actual url of the photo. Also, in the request url, it's not possible to know which image extension I'm going to get (jpg, png, gif, etc) so that's another problem.
Any help on how to save the photos appreciated.
EDIT: I get the header errors "Can't read file header" in my image viewer software when I try to open the image. The script itself doesn't show any errors.
I found a solution here:
http://kyleyu.com/?q=node/356
It gives a very useful function to return the actual URL after redirection:
function get_furl($url)
{
$furl = false;
// First check response headers
$headers = get_headers($url);
// Test for 301 or 302
if(preg_match('/^HTTP\/\d\.\d\s+(301|302)/',$headers[0]))
{
foreach($headers as $value)
{
if(substr(strtolower($value), 0, 9) == "location:")
{
$furl = trim(substr($value, 9, strlen($value)));
}
}
}
// Set final URL
$furl = ($furl) ? $furl : $url;
return $furl;
}
So you pass the Google Place Photo request uRL to this function and it returns the actual URL of the photo after the redirection which then can be used with CURL. It also explains that sometimes, the curl option curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); doesn't always work.
https://stackoverflow.com/a/23540352/2979237
We can minimize the above code to change as adding 1 as the second parameter of get_header() function like $headers = get_headers($url, 1);. that will return the associate values.

check if file exist on remote website without knowing extension

I need to find a file name _template on more than one remote server.
The file will have one of two extensions:
_template.htm
or
_template.php
I can find a file using cURL on a remote server which is easy but cant wrap my head around how to find one or the other.
The code is in a loop and it will search on about 20 different sites for this file, if it finds it just needs to say: Found it if it doesn't find any of the two files it needs to say: None Found.
Doing cURL twice takes too long to load.
Searching through the ftp array also takes too long.
My current code below that doesnt work, in know why it doesnt work thanks guys i would like to know how i can make it work: (This is in a loop)
$template_file = glob("/_template.{htm,php}", GLOB_BRACE);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $client_link.$template_file);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
$data = curl_exec($ch);
curl_close($ch);
preg_match_all("/HTTP\/1\.[1|0]\s(\d{3})/",$data,$matches);
$code = end($matches[1]);
if(!$data)
{
echo "No Files Found<br>";
}
else
{
if($code == 200)
{
$filefound = 'Found'.$template_file;
}
elseif($code == 404)
{
$filefound = '404 File not Found';
}
}
The problem is with the glob function - that function matches paths on the local filesystem. Add a print_r($template_file);' after theglob` and you'll see that it doesn't match anything (or it only matches things on your local system).
What you need to do is build the URLs one-by-one:
foreach(array("htm", "php") as $ext)
{
// build the URL string
$url = "http://example.com/_template." . $ext;
// now do whatever you need to with the URL...
}

Categories