I'm using crypt() which in the particular case uses an md5 hash with 12 character salt.
Here is an example of the string crypt() returns modified from php.net, crypt documentation.
$1$rasmusle$rISCgZzpwk3UhDidwX/in0
Here is the salt which also includes the encoding type.
$1$rasmusle$
Here it the encoding type. ( MD5 in this case )
$1$
and finally the hash value.
rISCgZzpwk3UhDidwX/in0
You can not have forward slashes in file names as this will be interpreted as a folder.
Should I simply remove all the forward slashes and are there other issue with the characters set that crypt() uses.
It looks like you want to prevent / allow access to the image for specific users. If that is the case I would do the following:
Store the images outside of the document root. This makes sure the images cannot simply be directly requested.
Store the images original name in the database and also store the sha1_file() hash in the same record. This adds the benefit if not having duplicate images on your server. Although images are small it prevents cluttering of the system.
When somebody requests a "private" image they will request it through a PHP file which will check whether the user has the privileges to access the file and if so serves the file (from the database).
With the above method you will have the most control over who can request the images and your users will thank you for that.
Note: that you cannot simply store all images in the same folder, because all filesystems have limits as to how many files can be stored in a single directory
A simple example of a PHP script that serves an image would look something like the following:
<?php
// always set the header and change it according to the type of the image
header("Content-type: image/jpeg");
echo file_get_contents('/path/to/the/image.jpg');
/$1$/ - Is an algorithm that used to create a hash
You can just use md5 md5_file/ sha1 sha1_file functions that would create hash without that additional information. Unless you want to use different algorithms at the same time.
Run a URLEncode method over your hash, and it should replace all of the '/' with %2F... I know this isn't a perfect fix, because i think things like apache server still block any web requests with '%2F' in the url. Just my 2 cents on the matter
ALWAYS normalize user provided data, including file names, unless you want to be hacked by uploading file with name containig NULL to fool PHP. Specify allowed characters (i.e A-Za-z0-9 and convert all other to i.e. underscore. Or use sha1/md5 to create hash from filename and store file under that name.
EDIT
This will replace all characters except for A-Z, a-z, 0-9 with underscore _:
$normalizedName = preg_replace('/[^A-Za-z0-9]/', '_', $userProvidedName);
Related
I am trying a capture the flag game but I can only enter 50 characters in the url, I have found the name of the file but the name of the file is over 50 chars long (hence why its a challenge), the filename consists of [a-z][0-9] with a .key at the end of the file. I can execute almost all php functions via the url (as long as it doesn't go over 50 chars)
The name is unique with the .key at the end which is why I essentially wanted to use readfile(*.key);
Any suggestions?
note:
CTF is a game penetration testers play to practice their hacking skills...
I have cracked most of the web app along with source code but this is my final hurdle...
the source code its self restricts the url length to 50 characters (you know chars...strlen...)
I dont need to extend the url in the ini file it this is apart of the game.
Use
readfile(md5('th3key').'.key');
It happens that the md5 function will return that string...
I can't seem to find a reference. I am assuming the PHP function file_exists uses system calls on linux and that these are safe for any string that does not contain a \0 character, but I would like to be sure.
Does anyone have (preferably non-anecdotal) information regarding this? Is is vulnerable to injection if I don't check the strings first?
I guess you need to, because the user may enter something like :
../../../somewhere_else/some_file and access a file that he is not allowed to access .
I suggest that you generate the absolute path of the file independently in your php code and just get the file name from user by basename()
or exclude any input containing ../ like :
$escaped_input = str_replace("../","",$input);
It depends on what you're trying to protect against.
file_exists doesn't do any writing to disk, which means that the worst that can happen is that someone gains some information about your file system or the existence of files that you have.
In practice however, if you're doing something later on with the same file that was previously checked with file_exists, such as includeing it, you may wish to perform more stringent checks.
I'm assuming that you may be passing arbitrary values, possibly sourced from user input, into this function.
If that is the case, it somewhat depends on why you actually need to use file_exists in the first place. In general, for any filesystem function that the user can pass values directly into, I'd try to filter out the string as much as possible. This is really just being pedantic and on the safe side, and may be unnecessary in practice.
So, for example, if you only ever need to check the existence of a file in a single directory, you should probably strip out directory delimiters of all sorts.
From personal experience, I've only ever passed user input into a file_exists call for mapping to a controller file, in which case, I'd just strip out any non-alphanumeric + underscore character.
UPDATE: reading your comments recently added, no there aren't special characters as this isn't executed in a shell. Even \0 should be fine, at least on newer PHP versions (I believe older ones would cut the string before the \0 when sent to underlying filesystem calls).
I am developing a site framework in php (codeigniter) and want to introduce image versioning on image uploads so that I can take advantage of image caching. The easiest approach would just be to md5 the image and use that as the file name but I don't like this approach for the following reasons:
1)Not SEO friendly on the image names
2)md5 hashes seem unnecessarily long - and therefore larger database
field required.
So I am considering using an approach such as the following:
Start the filename with the entered name of the image with underscores instead of spaces then add a randomly generated integer, say 8 digits long. This will mean I have to check for an existing image by that name and then regenerate the integer if one exists (however unlikely that is).
Presumably I will also have to unique filename for every image size, so I guess the solution here would be to add a prefix representing the file size.
Now I want to get this right first time since it will be a pain to change once the framework is deployed so I am really just looking for input on
a)Whether my concerns are justified (particularly does the filename do
anything for SEO and does the length of a random string of numbers
affect it)
b)Whether there is anything else I should be concerned about or check
for with my proposed approach.
c)Is there an easier approach, perhaps a hashing algorithm which
produces much shorter results.
d) Is there already a ci lib out there that does this?
Thank you for your input and advice
This answers a few of your questions:
Replacing spaces with underscores is not enough to have a clean filename as you'd need to check for more weird characters, but you can use sanitize_filename() method in CI's security library: http://ellislab.com/codeigniter/user-guide/libraries/security.html
If you do want to preserve the original filename, your approach sounds good to me. Though, 8-digit integer at the end of filename can be replaced by '-1’, ‘-2’, ‘-3' by simple incremental loop checking if the file with that ending exists or not.
File Upload library is something you can check out - http://ellislab.com/codeigniter/user-guide/libraries/file_uploading.html. It is flexible and can be configured to keep the original filenames. Getting sanitize_filename() from Security lib to work along should do exactly what you need.
In all my CI applications I always use encrypted filename (this optional feature is provided by CI file upload class). At the same time I can configure the library to not overwrite already existing file by adding a number to it (if no encryption is used) or by just giving it another encrypted name (when encryption option is on). I do like it this way as it keeps the filenames consistent clean (although long and not SEO-friendly, however ALT tag gives it more exposure to search engines).
This is a little out of the blue and it's mostly curiosity. I hope it's not a waste pf time and space.
I was writing a little script to validate accounts with a link so I decided to send an email with a link to the php script and in the link I would put two variables to get with the _GET array. A key and the email. Then I would just search the database with that email and key and change it's activated status to true... No prob. Easy enough even though it may not be very elegant..
I used a script for the generation of the key that I used elsewhere in the site for generating a new password (to reset it for instance) but sometimes it didn't work and after a lot of tries I noticed (and I felt stupid then) that the array my password generation function drew from was this:
'0123456789_!##$%&*()-=+abcdfghjkmnpqrstvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
So naturally I deleted the & character that is used for separating variables in the url... Then in another try I noticed that the link in the email was not recognized whole and stopped after the '#' character as well which I then remembered is used for references in an html so I deleted that as well. In the end I decided to leave only alphanumeric characters to be sure but I am curious; Are ther any more characters that are not 'valid' for url's using utilizing _GET and is there any way to use those characters anyway (maybe ulr encode or somwething)
There are plenty of characters that are invalid. Use urlencode to convert them to URL safe encodings. (Always run that function over any data you are inserting into a URL).
You have to use urlencode() before sending the values to $_GET.
You could use url_encode and url_decode but I would stay away from & # ? these are normal URL characters.
Also when it comes to passwords : dont stress about an algorithm, use sha1 crypt or something along those lines with a salt. These algorithms will be much stronger than your homemade ones.
What would be the best way to compare files and or directories. Lets say I want to store files on a sever or collective of servers like a cloud based system. My users are in collaboration with one another in many cases and some not. Either way I can have upwards of a hundred people or more with the same exact file. Only key difference is they likey renamed the file or whatever. But essentially same exact data all around. Now other thing is there is no specific file type. There's pdf, doc, docx, txt, videos, audio files, etc.. but the issue boils down to the same files over and over. What i want to do is cut it down. Remove the hundreds of dupes and with the help of a database store things like the file name the user provided so I can in turn store the single file left how and where I want while still providing the info they used essentially.
Now i know I can do something with md5 or sha1 or sha2 or something equivalent that will essentially give me a unique value I can use for such comparisons. But i am not exactly sure how or where to begin with that. Such as how with php can I get the sha or md5 of a file? When i look up stuff for those I get methods for strings but not files..
Overall I am here looking to bounce ideas around to figuring this out not so much as a direct means.. any help would be great.
$filePath = '/var/www/site/public/uploads/foo.txt'
$data = file_get_contents($filePath);
$key = sha1($data); //or $key = sha1_file($filePath);
Save this $key in a column of table also mark that column as UNIQUE so no to same file can be stored by default.
Use sha1 instead of md5 since many version control system like git use sha1 hash itself to identify uniqueness of file
When a file is uploaded:
Compute the hash (SHA1, etc.)
Rename the file to that hash and store it (unless a file with that hash already exists [you already have it])
Store the hash in your database.
When a file is requested:
Get the hash from your database
Return the file based on the hash
Use HTTP headers so the user's browser provides it to them with the filename they used
To get the md5 hash of a file at $path...
$hash = md5(file_get_contents($path));
Hope this partially answers your question.
There are many ways you can accomplish such a system. But if I'd have to write one from scratch, this is most likely how I would do it :
have three database tables (in pseudocode) :
table users {
id integer ## PK
username string
password string ## sha1
...
}
table user_files {
user_id integer ## Composite INDEX
file_id integer ##
filename string
}
table files {
id integer ## PK
uniq_id string ## basically 'yyyMMddhhmmssRRRR' INDEX
sha_hash string ## sha1
md5_hash string ## md5
}
Where files.sha_hash is the result of computing the sha1 of the file, files.md5_hash is the result of computing the md5 of the same file, as double security check, and files.filename the actual file name. On the server, the file would be stored and renamed to files.uniq_id to make sure there is no name collision, where the last RRRR chars represents a random string (cycle RRRR until uniq_id is unique in the database)
Note : PHP provides sha1_file and md5_file. Use these when computing files!
When a user stores a file, process the file (describe in step 1) and save it appropriately. To avoid having too many files in the same folder on the server, you may decompose files.uniq_id and separate each files into yyyy/MM sub folders.
Next, associate user_files.file_id = files.id and user_files.user_id = users.id and set user_files.filename to the uploaded file name (see next step).
If a user uploads another file, process the result as in 2, but check whether the result match another files.sha_hash, files.md5_hash. At this point, if we have a match, it doesn't matter what name the file has, it's very likely the exact same file, so associate the found user_files.file_id = files.id and user_files.user_id = users.id and set user_files.filename to the uploaded file name.
Note : this will cause to have 1 physical file and 2 virtual files on your server.
If a user rename a file without modifying it, simply rename user_files.filename matching the file he/she wants to rename.
If a user deletes a file, check how many user_files.file_id matches and only if 1 match is found, delete the physical file and the files entry. Otherwise, simply remove the user_files association.
If a user modifies the file with or without renaming it, perform a delete (step 5) and another save (step 3)
You can use :
md5(file_get_contents($filename));
To generate a hash for a file.
With that in mind, two entirely different files will produce the exact same md5 hash (Same problem with the other hashes, although you can have much less collisions by using a better hash method than md5). To compare two files you need to do it byte by byte, but you don't want to analyze every byte of every file on the hard disk to find a match.
What you need to do is store the hash for every file in your database in an a column, which should also be an index.
Then you can select all files with the same hash as the new file from your database.
That will give you a small list of files. Say you have 100,000 files on the disc. You might get a list of a few files that match the hash. Most of the time the lists will be short. Then you can loop through those files byte by byte to see if it's a match. Searching through a list of the ~10 files that have the same hash will save you from searching through all 100,000 files, but you still need to do the byte by byte comparison, because those 10 files could all be very different.
Is it necessary? Hard disk is very cheap these days so who cares for the duplicates? I would imagine that are not that big?
MD5 et al. are not unique. Just a quick way of saying that two files are not the same. It is possible for two files to have the same MD5 value but contain different data.