unicode characters in image URL - 404 - php

I am trying to open an image that has Latin characters in its name (113_Atlético Madrid).
I saved it by encoding its name with the PHP function rawurlencode(), so now its new name is 113_Atl%C3%A9tico%20Madrid. But when I am trying to open it by this URL for example mysite.com/images/113_Atl%C3%A9tico%20Madrid.png I got 404 error.
How I can fix this issue?
PHP code:
if(isset($_FILES['Team'])){
$avatar = $_FILES['Team'];
$model->avatar = "{$id}_".rawurlencode($model->name).".png";
if(!is_file(getcwd()."/images/avatars/competitions/{$model->avatar}")){
move_uploaded_file($avatar['tmp_name']['avatar'], getcwd()."/images/avatars/teams/{$model->avatar}");
}
}

%-encoding is for URLs. Filenames are not URLs. You use the form:
http://example.org/images/113_Atl%C3%A9tico%20Madrid.png
in the URL, and the web server will decode that to a filename something like:
/var/www/example-site/data/images/113_Atlético Madrid.png
You should use rawurlencode() when you're preparing the filename to go in a URL, but you shouldn't use it to prepare the filename for disc storage.
There is an additional problem here in that storing non-ASCII filenames on disc is something that is unreliable across platforms. Especially if you run on a Windows server, the PHP file APIs like move_uploaded_file() can very likely use an encoding that you didn't want, and you might end up with a filename like 113_Atlético Madrid.png.
There isn't necessarily an easy fix to this, but you could use any form of encoding, even %-encoding. So if you stuck with your current rawurlencode() for making filenames:
/var/www/example-site/data/images/113_Atl%C3%A9tico%20Madrid.png
that would be OK but you would then have to use double-rawurlencode to generate the matching URL:
http://example.org/images/113_Atl%25C3%25A9tico%2520Madrid.png
But in any case, it's very risky to include potentially-user-supplied arbitrary strings as part of a filename. You may be open to directory traversal attacks, where the name contains a string like /../../ to access the filesystem outside of the target directory. (And these attacks commonly escalate for execute-arbitrary-code attacks for PHP apps which are typically deployed with weak permissioning.) You would be much better off using an entirely synthetic name, as suggested (+1) by #MatthewBrown.
(Note this still isn't the end of security problems with allowing user file uploads, which it turns out is a very difficult feature to get right. There are still issues with content-sniffing and plugins that can allow image files to be re-interpreted as other types of file, resulting in cross-site scripting issues. To prevent all possibility of this it is best to only serve user-supplied files from a separate hostname, so that XSS against that host doesn't get you XSS against the main site.)

If you do not need to preserve the name of the file (and often there are good reasons not to) then it might be best to simply rename the entirely. The current timestamp is a reasonable choice.
if(isset($_FILES['Team'])){
$avatar = $_FILES['Team'];
$date = new DateTime();
$model->avatar = "{$id}_".$date->format('Y-m-d-H-i-sP').".png";
if(!is_file(getcwd()."/images/avatars/competitions/{$model->avatar}")){
move_uploaded_file($avatar['tmp_name']['avatar'], getcwd()."/images/avatars/teams/{$model->avatar}");
}
}
After all, what the file was called before it was uploaded shouldn't be that important and much more importantly if two users have a picture called "me.png" there is much less chance of a conflict.
If you are married to the idea of encoding the file name then I can only point you to other answers:
How do I use filesystem functions in PHP, using UTF-8 strings?
PHP - FTP filename encoding issue
PHP - Upload utf-8 filename

Related

File truncated when reading

I am writing some json results in files in PHP on shared hosting (fwrite).
Then I read those files to extract json results (file_get_contents).
It happens some times (maybe one out of more than one thousand) that when I read this file it appears truncated: I can only read a multiple of the first 32768 bytes of the file.
I added some code to copy/paste the file I am reading in case the json string is not valid, and I then get 2 different files: the original one was correctly written as it contains a valid json string and the copied one contains only the beginning of the original one and has a size of x*32768 bytes.
Would you have any idea of what could be the problem and how to solve this? (I don't know how to investigate further)
Thank you
Without example code it is impossible to give a 'fix my code' answer, but when doing file write/read sort of programming, you should follow a simple process (which, from the description, is missing one fairly critical step!)
First, write to a TEMP file (you are writing to a file, but it is important here to write to a TEMP file - otherwise, you could have race conditions....... ;);
an easy way to do that in php
$yourData = "whateverYourDataIs....";
$goodfilename = 'whateverYourGoodFileNameIsForYourData.json';
$tempfilename = 'tempfile' . time(); // MANY ways to do this (lots of SO posts on it - just get a unique name every time you write ('unique' may not be needed if you only occasionally do a write, but it is a good safety measure to avoid collisions and time() works for many programs.)
// Now, use $tempfilename in your fwrite.
$fwrite = fwrite($tempfilename,$yourData);
if ($fwrite === false) {
// the write failed, so do whatever 'error' function you may need
// since it failed, there should be no file, but not a bad idea to attempt to delete it
unlink ($tempfile);
}
else {
// the write succeeded, so let's do a 'sanity check' on the file to make sure it is good JSON (this is a 'paranoid' check, but "better safe than sorry", right?)
if(json_decode($tempfile)){
// we know the file is good JSON, so now RENAME (this is really fast, so collisions are almost impossible) NOTE: see http://php.net/manual/en/function.rename.php comments for some potential challenges and workarounds if you have trouble with rename.
rename($tempfilename,$goodfilename);
}
// Now, the GOOD file will contain your new data - and those read issues are gone! (though never say 'never' - it may be possible, but very unlikely!)
}
This may/not be your issue directly and you will have to suit this to fit your code, but as a safety factor - and a good way to avoid collisions, it should give you ~100% read success, which I believe is what you are after!)
If this doesn't help, then some direct code will be needed to provide a more complete answer.
As suggested by #UlrichEckhardt comment, it was due to read / write concurrency problem. I was trying to read a file that was being writen. I solved this by just waiting before trying to read the file again

PHP - file_put_contents file manipulation

I'm trying to write a PHP file on a server and to bypass the extension in the end.
This is the PHP file - 1.php:
<?php
file_put_contents("folder\\".$GET['file'].".PNG",$_GET['content']);
?>
I'm trying to bypass the PNG extension and to write a PHP file.
like this:
1.php?file=attack.php%00&content=blabla
but it's not working
I tried:
Null char (%00,%u0000)
Long filename
CRLF chars
space char
?,&,|,>,<,(,),{,},[,],\,!,~,:,; chars
backspace char
../
php protocol
php://filter/write=convert.base64-decode/resource=1.php
(will not work because the folder in the begging)
Anyone have any idea?
Thanks!
There are several fundamental problems here;
This code is very unsafe, I could set get as ../../1.php and overwrite this file to do whatever I want. It appears that you're doing some security testing however, so I guess that may be the problem
php is not a protocal, it's a language so php://anything should not work.
folder\\ doesn't make sense, what is this supposed to be/do?
That said though, for educational purposes prepending ../../ should allow you to escape out of the folder/ directory.
For example if this is in /home/Zak/mytest/ with the expectation of a directory within that called folder designated to store these PNG files, then a file of ../../zak_homedir should put a file at /home/Zak/zak_homedir.PNG due to relative path resolution.

preventing null byte attack with image files

According to what I read online, to prevent null byte attacks I should use the following on all user input:
$data = str_replace(chr(0), '', $data);
Makes sense to me. However, how do you do this on images the user has uploaded via form? I don't have much experience dealing with images like this.
I'm assuming you can't just do it like:
$_FILES['pic']['tmp_name'] = str_replace(chr(0), '', $_FILES['pic']['tmp_name']);
As mentioned in comments, PHP is no longer generally vulnerable to this attack. Attempts to open files with names containing null bytes will now fail, instead of opening an unexpected file.
Even in versions of PHP that were vulnerable to this attack, no filtering was necessary for uploaded files. The temporary file name used for uploaded files is generated internally by PHP, and will not contain null bytes or any other "surprising" special characters such as spaces.

Using ftp_get() when there are "spaces" in the file path and filename

I need to download a file via PHP ftp_get(), but the foolish provider is using directories and file names contaning whitespace.. The file path I'm dealing with is similar to /product info/more stuff/inventory and stuff.csv
The spaces in the path and in the filename itself is making it difficult to retrieve anything. I already tried the following without success:
$path = "/product\ info/more\ stuff/inventory\ and\ stuff.csv";
$path = "/product%20info/more%20stuff/inventory%20and%20stuff.csv";
$path = '"/product info/more stuff/inventory and stuff.csv"';
Thanks again for taking the time to help me out.
Your third attempt, quoting the complete path, was already the recommended approach. Though it very much depends on the actual server implementation.
FTP per RFC859 is comprised of a terminal session and a data transfer channel. Basically the FTP server provides a mini-shell on the command port. As such, typical shell string escaping rules do apply. URL encoding can be ruled out here.
I'd advise first to use single quotes however. Preferrably use escapeshellarg() to apply them. And try ftp_nb_get() while at it.
$path = "/foo foo/bar bar/baz baz.csv";
ftp_nb_get($con, "save.csv", escapeshellarg($path), 2);
If that doesn't work, further debugging is necessary. While all ftp_* function arguments are left unprocessed, you could as well try to send a ftp_raw request. This won't actually activate the data channel reading, but might return a more concrete error response.
print_r(ftp_raw($con, "RETR '/path to/some file.csv'\r\n"));
And I'm just gonna say it, if you're still getting a file not found error then; it's entirely possible that the file really doesn't exist at the presumed location. In that case manually traverse the directory structure with ftp_nlist and ftp_rawlist with var_dump (in case of extra trailing spaces for subdirs).
Alternatively just use PHPs ftp:// stream wrapper (which also supports PASV mode). Whose implementation is distinct from that of the ext/ftp functions. Here funnily enough, URL encoding is again the correct approach, but quoting still necessary (ftp_fopen_wrapper.c does not quote itself):
= file_get_contents("ftp://user:pw#example.org/'path%20to/file%20and.csv'");
// Inline quotes may likely trip up some FTP server implementations..
A much better alternative though is just using cURL.
// You'll have to use the long-winded PHP curl functions of course.
print curl("ftp://.../file with spaces.csv")->exec();
Last option is just resorting to calling a Unixland client. (If not wget, than a plain ftp client.)
$url = escapeshellarg("ftp://user:pw#ftp.example.org/$path");
$file = `wget $url`;
If you still can't retrieve any files, you'll have to look for an alternative FTP client in PHP for further debugging. Guess who wrote one.
To get a list of files or folders with spaces in the path.
ftp_chdir($conn, $path);
$children = ftp_rawlist($conn,'-a .');
Source

completely deleting a file from server

I want to delete a file by using PHP. I have used the unlink() function, but I was wondering about the security of unlink. Is the file completely deleted from the server? I want to make sure that there is no way to get the file back and the file is completely removed from the server.
open the file in binary mode for writing, write 1's over the entire file, close the file, and then unlink it. overwrites any data within the file so it cannot be recovered.
Personally i would say use 1's instead of 0's as 1's are actual data and will always write, where as 0's may not write, depending on several factors.
Edit: After some thought, and reading of comments, i would go with a hybrid approach, depending on "how deleted" you want the file to be, if you simply wish to make it so the data cannot be recovered, overwrite the entire files length with 1's as this is fast, and destroys the data, the problem with this, is it leaves a set length of uniform data on the disk which infers a file USED to be there and gives away the files length, giving vital pieces of forensic information. Simply writing random data will not avoid this also, as if all the drive sectors around this file are untouched, this will also leave a forensic trace.
The best solution factoring in forensic deletion, obfuscation and plausible deniability (again, this is overkill, but im adding it for the sake of adding it), overwrite the entire length of the file with 1's and then, for HALF the length of the file in bytes, write from mt_rand in random length sizes, from random starting points, leaving the impression that many files of varying lengths used to be in this area, thus creating a false trail. (again, this is completely overkill and is generally only needed by serial killers and the CIA, but im adding it for the sake of doing so).
the US government used to recommend a seven step wipe, for disks.
1) all '1's
2) all '0's
3) the pattern '01'
4) the pattern '10'
5) a random pattern
6) all '1'
7) a random pattern,
re the code sample, using a language like PHP is wrong for this type of wipe as your relaying on the OS really wipeing the file and not doing something cleaver like only wipeing it the last time or just unlinking it, however...
(untested)
$filename = "/usr/local/something.txt";
$size = filesize($filename);
$pat1 = chr(0);
$pat2 = chr(255);
$pat3 = chr(170);
$pat4 = chr(85);
$mask = str_repeat($pat1, $size);
file_put_contents($filename, $mask);
$mask = str_repeat($pat2, $size);
file_put_contents($filename, $mask);
$mask = str_repeat($pat3, $size);
file_put_contents($filename, $mask);
$mask = str_repeat($pat4, $size);
file_put_contents($filename, $mask);
This might not answer HOW to perfectly delete a file "with PHP", but it answers your question: "Is the file completely deleted from the server ?"
In some cases, No! (on UNIX/POSIX OS).
According to the highest voted comment on the offical PHP unlink() manual page, the unlink function does not really delete the file, it's deleting the system link to the file's content ! As files can have several files names (!) [symlinks?] the file will only be deleted when ALL file names are unlinked. So, if your file has 2 names, then unlink() will not really delete the file unless you unlink() both file names. Dear linux guys, please correct me here if necessary.
This might be why the function is called unLINK() and not delete() !!!
Here a full quote of the excellent comment:
Deleted a large file but seeing no increase in free space or decrease of disk usage? Using UNIX or other POSIX OS? The unlink() is not about removing file, it's about removing a file name. The manpage says: `unlink - delete a name and possibly the file it refers to''. Most of the time a file has just one name -- removing it will also remove (free, deallocate) thebody' of file (with one caveat, see below). That's the simple, usual case.
However, it's perfectly fine for a file to have several names (see the link() function), in the same or different directories. All the names will refer to the file body and keep it alive', so to say. Only when all the names are removed, the body of file actually is freed. The caveat: A file's body may *also* bekept alive' (still using diskspace) by a process holding the file open. The body will not be deallocated (will not free disk space) as long as the process holds it open. In fact, there's a fancy way of resurrecting a file removed by a mistake but still held open by a process...
Have a look on unlink()'s sister function link() here.
The (imo) best way to delete a file via PHP:
The way to go to really delete a file with PHP (in linux) is to use the exec() function, which executes real bash commands (doing things with linux bash feel correct btw). In this case, the file test.jpg would be deleted by doing:
exec("rm test.jpg);
More info on how to use rm (remove) correctly can be found for example here. Please note: PHP needs the right to delete the file!
UPDATE: Unfortunatly, the linux rm command ALSO does not really delete the file if it has two names/links. Look here for more info.
I'll have a deeper research on that and give feedback...
It is possible that because of some fragmentation on the disk some parts of file will stay, even if the file is totally overwritten.
The other way is to run (by shell_exec()) external program, that is system specific. Here is an example (for Windows), however I have not tested it.
You should do multiple passes of overwriting to deminish traces. For instance using the US DoD 5220-22.M : "Overwrite all addressable locations with a character, its complement, then a random character and verify" (from killdisk site)
Here's what the EFF recommends to permanently remove a file http://ssd.eff.org/tech/deletion.
In my embedded Ubuntu device, I use: echo exec('rm /usr/share/subdirectory/subdirectory/filename'); This works for me.
if you use rm -f (--force) then linux will
ignore nonexistent files and arguments, never prompt
rm -d will
remove empty directories
If you enter rm --help at the prompt you get the help screen. The last lines read:
Note that if you use rm to remove a file, it might be possible to recover some of its contents, given sufficient expertise and/or time. For greater assurance that the contents are truly unrecoverable, consider using shred.
Since my system is a "closed" system then I'm not concerned about violating security issues. My logic being that one must have the system password to SSH into the OS and the only user interface is via web pages.
#Sliq's comments are still true to date. You need to decide for your case.

Categories