I am creating a social network where users upload their profile image.
This image will be used in their profile page in 150 / 150 px dimension.
In the home page i.e user Activity feed I need the same image to be in 75 / 75 px.
What would be the best practice to do this
Resize image on fly (timthumb).
Resize and Save image in the server.
While uploading a photo create required set of thumbnails and save as a [image_name]thumb[size_name].jpg or so:
uploaded: file.jpg
medium: file_thumb_150x150.jpg
small: file_thumb_75x75.jpg
Naming convention is up to you, but in a fast way you get easy access to the data you need. No need to use server to generate it on the fly or scale in a browser.
I've been working on this problem for a little while and have come across 3 main ways of doing this:
Generate thumbnail images at the point of upload as a background process.
Generate images on demand through the main application
Generate images on demand using the URL as an API
Each approach has its pros and cons.
This approach is the most restrictive, you have to know all the uses and sizes of thumbnails up front so that you can generate them immediately after upload. The main advantage is that the images can be served efficiently using a server like nginx and are just like any other static resources.
Django has a library called sorl-thumbnail which provides a template tag for generating thumbnails of all kinds as and when they are needed. It uses a fast key/value store to keep track of what thumbnails have been generated, and invalidates stale generated images automatically if it detects the source image has been changed. The template tag then returns the URL for the generated image, which can be served directly from nginx without going through a scripting layer. More flexible than 1, but you can't (for example) generate an image URL using JavaScript and expect it to exist, it has to be done by the website's backend code or templates.
Fully dynamic and flexible, you can get whatever version of the image you want just by tweaking the URL, Amazon uses this method as do all those placeholder image generation websites. Can generate URLs in JavaScript and do whatever fancy things you want. The website itself doesn't need any knowledge of the thumbnailing layer short of maybe a few helper methods to generate the URLs for you. BUT, this is obviously the most resource-intensive way of doing things and you need to make sure your architecture can handle the load. You need to use every trick in the book to invalidate caches in a timely manner, avoid unnecessary hits on the resizing script etc.
I'm a big fan of the 3rd way of doing things, I like image generation to be completely isolated from my main website functionality and I like the considerable flexibility, but you need to know what you're doing when it comes to configuring your servers to handle it.
I tell you what I do. I allways store the full size image, but renaming it ussing the db ID with leading zeros. On the first use I create the thumbnail and store it in other folder, using it in next calls.
If server space and bandwidth is an issue you should consider using an cdn.
Amazon has a good service,
Related
I'm currently rewriting a website that need a lot of different sizes for each images. In the past I was doing it by creating the thumbnails images for all sizes on the upload. But now I have a doubt about is performance. This is because now I have to change my design and half of my images are not of the right size. So I think of 2 solutions :
Keep doing this and add a button on the backend to re-generate all the images. The problem is that I always need to know every sizes needed by every part of the site.
Only upload the real size image, and when displaying it, put in the SRC tag something like sr="thumbs.php?img=my-image-path/image.jpg&width=120&height=120". Then create the thumb and display it. Also my script would check if the thumb already exists, if it does it doesn't need to recrate it so just display it. Each 5 Days launch a script with a crontask to delete all the thumbs (to be sure to only use the usefull ones).
I think that the second solution is better but I'm a little concern by the fact that I need to call php everytime an image is shown, even if it's already created, it's php that give it to display...
Thanks for your advises
Based on the original question and subsequent comments, it would sound like on-demand generation would be suitable for you, as it doesn't sound like you will have a demanding environment in terms of absolutely minimizing the amount of download time to the end client.
It seems you already have a grasp around the option to give your <img> tags a src value that is a PHP script, with that script either serving up a cached thumbnail if it exists, or generating it on the fly, caching it, and then serving it up, so let me give you another option.
Generally speaking, utilizing PHP to serve up static resources is not a great idea as you begin to scale your site as
This would require the additional overhead of invoking PHP to serve these sorts of requests, something much more optimized with the basic web server like Apache, Nginx, etc. This means your site is going to be able to handle less traffic per server because it is using extra memory, CPU, etc. in order to serve up this static content.
It makes it hard to move those static resources into a single repository outside of the server for serving up content (such as CDN). This means you have to duplicate your files on each and every web server you have powering a site.
As such, my suggestion would be to still serve up the images as static image files via the webserver, but generate thumbnails on the fly if they are missing. To achieve this you can simply create a custom redirect rule or 404 handler on the web server, such that requests in your thumbnail directory which do not match an existing thumbnail image could be redirected to a PHP script to automatically generate the thumbnail and serve up the image (without the browser even knowing it). Future requests against this thumbnail would be served up as a static image.
This scales quite nicely as, if in the future you have the need to move your static images to a single server (or CDN), you can just use an origin-pull mechanism to try to get the content from your main servers, which will auto-generate them via the same mechanism I just mentioned.
Use the second option, if you don't have too much storage and first if you don't have too much CPU.
Or you can combine these: generate and store the image at the first open of the php thumbnails generator and nex time just give back the cached image.
With this solution you'll have only the necessary images and if you want you can delete sometimes the older ones.
I am making a web application that needs to show 3 types of thumbnails to a user. No I might end up with a lot of thumbnail files on the server for a lot of users.
This makes me think is generating thumbnails on the fly is a better option than storing them?
Speed vs Storage vs Logic - Which one to go for?
Does anyone here ever faced such a dilemma - let me know!
I am using CodeIgniter and its inbuilt Image Library for generating thumbnails.
I would go with: generate when needed, store afterwards.
Link to the image using a URL like /img/42/400x300.jpg. Through rewrite rules, you can fire up a PHP script should the image not exist. That script can then generate the requested image in the requested size and store it in the public web folder, where the web server can serve it directly the next time.
That gives you the best of both worlds: the image is not generated until needed, it is only generated once and it even makes it very flexible to work with different image sizes on the fly.
If you're worried about storage space, you can add a regular clean-up job which removes old images or perhaps analyses your access log files and removes images which where not accessed for some time.
My comment as an answer: (why not :)
My personal thoughts on this are, if you're anticipating a lot of users go with storage as the the load of creating dynamic thumbnails for every one of these users for every page load is going to hurt the server, maybe create it dynamically the first time it's ever viewed and then store it.
You may also take advantage of browser caching to save load and bandwidth. (marginal but every little helps)
I'm planning to develop an area where the users can upload pictures. I know how to upload a picture on the server using PHP but the problem is what is the best practice to develop a performing system.
The idea is to display in different pages thumbs and I would like to know if it's a better idea to save two different images (thumb + original) on the server or if it's better to save just the original and create all the thumbs on the fly. Thumb + original means more space on the server, whereas the option "thumbs on the fly" means most likely a server overload.
I found couple of good scripts to resize and cropping on the fly but not sure if it's a good idea to use especially if the web site has few thousands visitor per day (or may be more in the future just to be optimistic/pessimistic).
Absolutely generate and save the thumbnails on disk. Storage is cheap.
You can generate some thumbnails and save them on disk but in the long term that's problematic due to different devices needing different sizes, different formats, etc.
If you are already saving the uploaded images on S3, Azure Storage, or Google Cloud I recommend to use some on the fly image processing service like imglab or cloudinary.
With these services you can generate many different types of cropping, and serving them in different (modern) formats like WebP or AVIF so you don't need to generate them before hand. SEO will be improved wit this option too.
Additionaly images will be behind a global CDN so users will get the images in a fast way independent or their location.
I'm using an image resizer to output images in my website. Is it possible to use a CDN in the current situation?
The image resizer takes the file path of the image and outputs the desired image. Now even if I use a CDN, the image resizer script is hosted on my server. So every image request is going through my server. Will using a CDN benefit me in any way?
The cached object on CDNs are keyed on the request URI, so you should benefit from a CDN provided you application isn't generating any randomness in the URLs. If your image request looks like this
/resizer/200x100/thing.jpg
# ...or...
/resizer/thing.jpg?size=200x100
Then the CDN will cache it for all subsequent requests.
If your sever and script is quicker enough then I would use your server code. This means you can play around with the script if you need to add custom functions. If you have any serious problems or want much more options which a CDN may provide then switch.
Short answer: No. If your server resizes images on-the-fly, then it still serves as a bottleneck, and the benefits of the CDN are essentially lost.
Three solutions you might be comfortable with:
Have the image resizer script run once upon image upload, and create all necessary image sizes, then store everything in a logical manner on the CDN. (Reasonable solution. Downside: Very rigid, adding new image sizes requires considerable work).
Have the image resizer script run (ie: resize a single image and upload to the CDN) upon request, but only if the image does not exist on the CDN already. (You can either store a list of created images in a database or, preferably, if at all possible use the object notation default image technique) (Cool solution. Downside: Old browsers don't like the object tag, non-standard, albeit valid, code).
Switch CDNs, use a more mature CDN service that allows you to manipulate media files via API. ex: http://cloudinary.com/ (Smooth sailing solution. Downside: Not as cheap as the non-intelligent CDNs out there, but in most cases you should hardly feel it, and it will save you a ton of coding).
Hope this helps, I'd love to hear what solution you chose.
I'm currently working on a portfolio site wherein I would have to load a large quantity of photos on the same page.
These images are loaded dynamically through PHP, and at the moment I have opted to save thumbnail versions of these images beforehand and load these.
My concern, however, is that this might not be an optimal solution should the site have a large number of users: I would basically have duplicates of each image multiplied by the number of images users have uploaded.
My question for you is whether or not there are better solutions to this problem? A way to load a page as fast as possible without compromising too much space?
Thanks a ton.
Loading a webpage full of lots of images will be slow. The reason for this is because of the bandwidth needed to transfer these images.
You have a couple of options
Load full images in tiled mode. These images will be full images, just resized to fit in a "thumbnail" view. The advantage of this is that you are only saving 1 image, but that image is full sized, and will take a long time to load.
Load the thumbnails as you said you are doing. The advantage to this is performance, but you need to store two copies of each image. Also, depending on how you tackle the thumbnail creation, you may require users to upload two copies of the image to provide their own thumbnail... which could stink.
Load thumbnails, but dynamically generate them on upload. You are essentially keeping two copies of the image on disk, but you are dynamically creating it through some php image modification API. This will load faster, but still eats up disk space. It also minimizes user/administrative requirements to provide a thumbnail.
Load thumbnails on-demand, as the page is requested. This approach would take some testing, as I've never tried it. Basically, you would invoke the php image modification API (or better yet out-source to a native solution!) to create a one-time-use (or cached) thumbnail to be used. You might say "OMG, that'll take so long!". I think this approach might actually be usable if you apply an appropriate caching mechanism so you aren't constantly recreating the same thumbnails. It will keep down on bandwidth, and since the limiting factor here is the network connection, it might be faster then just sending the full images (since the limiting factor of creating the thumbnails would now be CPU/Memory/Hard Disk).
I think #4 is an interesting concept, and might be worth exploring.
What i think is :
A. Take Advantage of Cache (System Cache , Cache In Headers , HTTP Cache .. all Cache )
B. Don't generate Thumb all the time
C. Use a Job Queing System such as Gearman or beanstalkd to generate thumbs so that you don't have to do it instantly
D. Use Imagick is more efficient
E. Paginate
F. Example only generate thumb when the original file has been modified
$file = "a.jpg" ;
$thumbFile = "a.thumb.jpg" ;
$createThumb = true;
if(is_file($thumbFile))
{
if((filemtime($file) - 10) < filemtime($thumbFile));
{
$createThumb = false;
}
}
if($createThumb === true)
{
$thumb = new Imagick();
$thumb->readImage($file);
$thumb->thumbnailImage(50, null);
$thumb->writeImage($thumbFile);
$thumb->destroy();
}
Consider using a sprite or a collage of all the images, so that only one larger image is loaded, saving bandwidth and decreasing page load time.
Also, as suggested already, pagination and and async loading can improve it some.
References:
http://css-tricks.com/css-sprites/
http://en.wikipedia.org/wiki/Sprite_(computer_graphics)#Sprites_by_CSS
Yes, caching the thumbnails is a good idea, and will work out fine when done right. This kind of thing is a great place for horizontal scaling.
You should look into using a
CDN. (Or
possibly implementing something similar.) The caching system will
generate images of commonly-requested sizes, and groom those off if
they become infrequently requested at a later time.
You could also scale horizontally and transplant this service to another server or
vserver in your network or cloud.
It is a disk-intensive and bandwidth-intensive thing, image delivery. Not as bad as video!
Generate thumbnail of large images on-the-fly. (Some hint)
Load images asynchronously (Try JAIL)
Paginate
Caching is also an option, works from the second time the site is loaded though.