I'm building a site were users can upload images and then "use" them. What I would like is some thoughts and ideas about how to manage temporary uploads.
For example, a user uploads an image but decides not to do anything with it and just leaves the site. I have then either uploaded the file to the server, or loaded it to the server memory, but how do I know when the image can be removed? First, I thought of just having a temporary upload folder which is emptied periodically, but it feels like there must be something better?
BTW I'm using cakePHP and MySQL. Although images are stored on the server, only the location is stored in the dbb.
Save the information about file to MySQL, and save the last time the image was viewed - can be done via some script that would be altered everytime the image is being used.. and check the database for images not used for 30 days, delete them..
You could try to define a "session" in some way and give the user some information about it. For example, in SO, there is a popup when you started an answer but try to leave the site (and your answer would be lost). You could do the same and delete the uploaded image if the user proceeds. Of course, you can still use a timeout or some other rules (maximum image folder size etc.).
I'm not sure what does "temporary upload" mean in your app. The file is either uploaded or not, and under the ownership of a user. If a user doesn't want to do anything at the moment, you have no other choice but to leave the file where it is.
What you can do is put a warning somewhere on your image management page about unused images, but removing them yourself seems like a bad practice (at least from the user perspective).
As a user,When I upload the image to a server(assuming I want to use it later) and leave the site, I don't expect it to be deleted if I am a registered user.
I would prefer it to be there in my acct until I come back.I would suggest thinking in those lines and implementing a solution to save the users' images if possible.
Check the last accessed/modified time of file to see it if has been used.
Related
i build a security script (not upload viruses s.e.) to upload images. I have now a question due to security of output the images on website.
Why this is not a security way to output the image:
<img src="/pathdir/imagerealname.jpg">
Why its better to make a script like photo.php and load original image as other name?
The URL that you use for your image isn't a problem. You can use whatever you want. The issue comes when you allow users to store files on disk that they name.
While it is possible to allow a user to upload a file directly to something like /users/username.jpg, you must take extra caution to ensure that they don't set up their naming in such a way that the file lands in the wrong place, or becomes executable.
For example, maybe someone's username is ../../../etc/init.d/somethingevil. You wouldn't want them putting scripts there. Or, maybe someone's username becomes evil.php, and then they can just call /users/evil.php and execute code on your servers.
Again, it is possible to set the file name to something the user provided, but then you have to do a bunch of checking to ensure it's fine.
There are also issues of usernames changing. You would need to protect against someone changing to a username that someone else previously had.
Now, back to your PHP script. What you could do is always write files to the hash of the username. For example, on disk you might have /var/www/images/users/B858CB282617FB0956D960215C8E84D1CCF909C6.jpg. And, you might have a rewrite rule that sets up all requests for /images/users/(.*) to go to /user-image.php. /user-image.php then hashes this username and actually outputs the contents of the image on disk. This also enables you to do things in your PHP script like resize that user image on-demand, via libvips or similar.
I have a simple image upload form. When someone uploads an image, it is for a football pool, so there always is a $poolid that goes with the image they upload.
Right now, I am naming the uploaded image using the poolid. So for example, if someone uploads an image, it might get named P0714TYER7EN.png.
All the app will ever do is, when it outputs the football pool's page, it will check to see if an image exists for that pool and if so, it will show it. It checks like this:
if (file_exists("uploads/".$poolid.".png")) { //code to show it }
My first thought when planning this was to add a field called "image" in my MYSQL database's table for all the pool information (called pools) and I would store a value of either the image name (P0714TYER7EN.png) or empty if there wasn't one uploaded. Then I would check that field in the database to determine if an image exists or not.
But I realized I don't really need to store anything in the database because I can simply use the PHP file_exists check above to know if there is an image or not.
In other words, it would seem redundant to have a field in the database.
Everything works doing it this way (i.e. NOT having a field in the database) but I'm wondering if this is bad practice for any reason?
If anyone feels that I should absolutely still have a field in the database, please share your thoughts. I just want to do it the proper way.
Thank you.
The approach could depend a lot on what exactly you're trying to do. Seems like the options you would have is:
File System Only
Benefits would be the speed of accessing static files of an image and use of it in your HTML directly which makes it a more simple solution. Also if you're comfortable with using these functions it will be faster to finish.
Drawbacks would be that you're limited to using file_exists and similar. Any code to manage files this way has to be very specific and static. You also can not search or perform operations efficiently on this. In general relying on the file system alone is not a best practice from my experience.
Database Only
Benefits, you can use Blob type as a column with meta data like owner, uploader, timestamp, etc. in the same row. This makes checking for existing files faster as well as any searching or other operations fast and efficient.
Drawbacks, you can't serve files statically using a CDN or even a cookie-less subdomain or other strategies for page performance. You also have to use PHP and MySQL to generate then serve any images via code rather than just referring to the image file directly.
Hybrid
Benefits, basically the same benefits as both above. You can have your metadata in MySQL with a MD5 hash and location of the file available as well. Your PHP then renders the page with a direct link to the file rather than processing the Blob to an image. You could use this in conjunction with a CDN by prefixing or storing the CDN location as well.
Drawbacks, if you manually changed names of files on the server you'd have to rely on a function matching hashes to detect this, though this would also affect a File System Only that needs to detect a duplicate file potentially.
TLDR; the Hybrid approach is what you'll see most software use like WordPress or others and I believe would be considered a best practice while file system only is a bit of a hack.
Note: Database only could be a best approach in specific situations where you want database clustering and replication of images directly in your database rather than to a file system (especially if the file system is restricted access or unable to be modified for any reason, then you have full flexibility on the DB).
You can also use the blob datatypes from mysql. There you can save the image as binary data next to the data about the football pool.
So when you want to load an football pool you simple fire an sql statement and check if it returns a result, if so load the image from the database and display the data, otherwise throw an error.
If you have very frequent access you can simply put the images into a seperate table and load the image independent of the data about the football pool. Additional set some cache headers on the image and put it in a seperate file, this way you could simply save the primary key of the images in football table. Then you want to display the web page you simply load another document, pass it the primary key of the image, there the image will be loaded, or if the browser has it in cache, will load it from cache without querying the database.
This way you also have a better consistency of data and images.
Your uploading an image to specific folder and that too with poolid which will be unique. It should work just fine.
Problem :
The code you have written works great. But the problem is, for the first time if the image loaded is .png and second time loaded file in jpeg or jpg then file exists wont check that and hence it may fail.
Caution :
If you have already taken a caution to check that the image uploaded must and should be png than the file_exists will work great.
Alternate Solution :
In case if your not checking for the image type to be .png then I highly advice you to take a boolean image column in your table by is_image_uploaded or something which can be set once you upload the file every time.
This makes sure that in case next time you wan to upload the image then you can directly go and check in your database table and see that if is_image_uploaded column is set or not. If not set then upload or else ignore or do whatever you want
I have a PHP script where a user can upload an image. This image is stored in the temporary directory and is returned to the user. The user can then use a javascript interface to crop the image. (x1,y2)(x2,y2) is sent to the script which is used to crop the image. It is then returned to the user for another preview and\or crop. Once the user is sufficiently satisfied he will click "save". The temp file is copied over to the original and the temp deleted. These are not per-user images, but rather images of equipment. Any user in the organization can replace any image of equipment. This approach is good but there are a few issues:
1) Let's say the user uploads an image for preview but then closes the browser window. I will be left with a temporary file. This can become an issue. Sure I can have a CRON clean them up but in theory I can have a ton of temporary files (this is ugly). The cron can also delete the user's temp file during an edit.
2) To deal with number 1 I can always have a temporary file per piece of equipment, such as equip1.temp and equip1.jpg. All uploads are stored in equip1.temp, all commits are transfered to equip1.jpg. If two users are trying to upload pictures of the same piece of equipment at the same time this could mess them up (highly unlikely + not an issue, but still ugly)
3) I can always pass the image back and forth (user "uploads" image and it get's echoed back as an <img src="base64....." />. The resulting edits + original base64 string are sent back to PHP for processing). This solution relieves the temp file issue but I noticed it takes several seconds to send high res images back and forth.
How would you deal with this situation?
I had a similar issue like this. If I recall correctly (its been a while), I ended up creating a table in a DB to store file names and session keys/time. Each time the script loaded, if there was a dead session in the database, the corresponding session and image/file was deleted.
I don't know if that's a good solution or not, but it solved the multiple user access problem for me.
I wouldn't recommend #3 due to the reasons you mentioned.
I suggest you do this instead:
User uploads file to a random temporary name. equip1.jpg gets stored as equip1_fc8293ae82f72cf7.jpg. Be sure you script will juggle both file names around. It will allow two users to upload the same equipment, with the last one to upload being the winner, but no conflict along the way.
Everytime your cropper works with the temp image, you should "touch" it to update the modified time.
Let the user finish their edits, move the temp file in place of the final image name.
Have a cron, or a section of your uploader script, that deletes abandoned temp files that have a mtime older than an hour or so. You suggest this is messy, because of the potential of lots of temp files, but do you expect a lot of images to be abandoned? Garbage collection is a very standard method for this problem.
I have a quick question if anyone could help. I am building a CMS for a client where they can log in, and change content (including pictures via upload file form) that are all stored in a database.
My question.. I have been researching, and everywhere says I need to store the image files outside the root folder. Is this necessary in my case if only a few people will be uploading files, inside an admin panel, where they must first log in to the site? I will have already taken steps client side by making sure of file type, size, extension etc... then changing the name of the file before adding it to my DB... Is this secure enough, or am I asking for trouble down the road?
Thanks
Its generally a good idea to store uploaded content someplace where it cant directly be addressed by a browser. You dont want someone uploading a .php file (or some other format you forgot to check for) and then being able execute it by pulling up the direct url. Rather, you'd have a wrapper script that delivered the file.
So yes, its a good idea, but not 'necessary' (by the dictionary definition of the word). You can certainly choose not to do so if in your judgement the admin area is otherwise secure.
That said, in the scenario you describe, as long as its only admin users who can upload images, I dont think its a huge deal either way.
btw, if you are not already, verify the images by their file headers or content, not file extension.
I have no idea how the big websites save the pictures on their servers. Could any one tell me how do they save the pictures that are uploaded by the users in their database?
I was thinking, maybe they would just save the file(the picture) in some path and just save that path in the databse is that right?
But I want to do it this way. Is this right? For example, a website named www.photos.com. When a user uploads a picture I would create a folder of the user name and save those pictures in that folder.
I believe we can create a directory using php file concepts. So when a new user uploads his picture or file, I want to create a directory with his name.
Example: if user name is john, I would create a directory like this on photos.com www.photos.com/john/ and then save all his pictures to this directory when he uploads a picture. Is this the right way to do this?
I have no one here that has good knowledge of saving the files to servers so please let me know how to do this? I want to do it the correct and secure way.
All big websites don't save pictures to the database they store them in the disk.
They save a reference to the picture's position in a table. And then link from there.
Why? Performance.
Pulling heavy content from a database is a huge performance bottleneck. And databases don't scale horizontally that well, so it would mean even a bigger problem. All big sites use static content farms to deal with static content such as images. That's servers who won't care less about your identity.
How do they keep the pictures really private you might ask? They don't.
The picture's link is, in itself, the address and the password. Let's take Facebook, for example. If I store a private picture on my account you should not be able to open it. But, as long as you have the correct address you can.
This picture is private. Notice the filename
10400121_87110566301_7482172_n.jpg
(facebook changes the url from time to time so the link may be broken)
It's non sequential. The only way to get the picture is to know it's address.
Based on a previous user photo you can't guess the next one.
It has a huge entropy so even if you start taking random wild guesses you'll have an extensive amount of failures and, if you do get to a picture, you won't be able to, from there, realize the owners identity which, in itself, is protection in anonymity.
Edit (why you should not store images in a "username" folder:
After your edit it became clear that you do intent to put files on disk and not on the database. This edit covers the new scenario.
Even though your logic (create a folder per user) seams more organized it creates problems when you start having many users and many pictures. Imagine that your servers have 1T disk space. And lets also imagine that 1T is more or less accurate with the load the server can handle.
Now you have 11 users, assume they start uploading at the same time and each will upload more than 100GB of files. When they reach 91GB each the server is full and you must start storing images on a different server. If that user/folder structure is followed you would have to select one of the users and migrate all of his data to a different server. Also, it makes a hard-limit on a user who can't upload more than 1T in files.
Should I store all files in the same folder, then?
No, big-sites generally store files in sequential folders (/000001/, /000002/, etc) having an x defined number of files per folder. This is mainly for file-system performance issues.
More on how many files in a directory is too many?
It is usually a bad idea to store images in your database (if your site is popular). Database is, traditionally, one of main bottlenecks in most any application out there. No need to load it more than necessary. If images are in the filesystem, many http servers (nginx, for example) will serve them most efficiently.
The biggest social network in Russia, Vkontakte does exactly this: store images in the filesystem.
Another big social network implemented a sophisticated scalable blob storage. But it's not available to the public, AFAIK.
Summary of this answer: don't store blobs in the database.
is this the right way to do
Yes.
The only thing I'd suggest to use not name but id.
www.photos.com/albums/1234/ would be okay for starter.
Image management may best be achieved by physically uploading images to the server and then recording file location and image details in a database. Subsequently, a Search Form could be configured to permit the user to do a text search, part number search, or other queries. A PHP script could be written to produce a valid HTML image tag based on data found in the table.
uploading images into a MySQL™ BLOB field is such a bad idea such image data is generally problematic if the images are much larger than thumbnails. If the images are large, you can end up having to copy/paste one SQL INSERT statement at a time (into phpMyAdmin). If the images are large and the SQL INSERT statement is broken into two lines by your text editor, you'll never be able to restore the image.