images folder contains about 14000 jpg images.
I want to load them using loading = 'lazy' but there is a performance problem on a client side (Chrome).
Each scroll firstly shows empty rectangles and then the images are loading, but very slowly.
On youtube home page I'm scrolling fluently, i.e. without waiting for images, and as I can see - there is no limit to scroll down.
How to solve this?
$arr = glob("../images/*.jpg");
$ht = '';
foreach($arr as $el){
$ht .= "<img class='bimg' src = '" . $el . "' width = 151.11' height = '86.73' alt = 'img' loading = 'lazy'>\n";
}
echo $ht;
Do you really need to show 14000 images? Is a user realistically going to scroll through 14000 images? Will they even scroll through 500? I'm unfamiliar with the lazy attribute, but I assume the more you have the more work the browser has to do which is why you're seeing a performance issues. What was the last site you visited with even a <ul> of 14000 items in a row? Even large <table> elements are paginated.
I would guess you could render less images at once (even 100 is a lot to lazy load at once depending on the size) and once the user gets to the 50 mark, make a request for 100 more, etc. Paginate your requests.
The other thing you could look at would be the common data-lazy=http://example.com/path/source.jpg. You can read more about that here. Even if you don't use a framework there are plenty of other resource on how to load an image based on the window scroll event.
Here is a fairly simple answer: https://stackoverflow.com/a/5871121/3404054
My 2 cents is that you don't need 14k images at once, it seems unrealistic to use that many images. Adjust your logic to use what you need, when you need it.
14000 in one page request? I can't even imagine for 1k of images to be loaded in a browser, how much more if it is already 14k.
From your code:
foreach($arr as $el){
$ht .= "<img class='bimg' src = '" . $el . "' width = 151.11' height = '86.73' alt = 'img' loading = 'lazy'>\n";
}
I have few suggestions:
You're loading all images src='".$el."' in a browser. Usually, lazyload does not fetch the image in src attribute. It sometimes stores the location via data attribute and then the lazyload plugin will fetch it dynamically.
Check your network tab to check the performance of your page.
For 14k of images, I would suggest that you fetch data by batch. Example, load 100 out of 1000, and so on. You might do it via ajax.
Related
I'm building a web based system, which will host loads and loads of highres images, and they will be available for sale. Of course I will never display the highres image, instead when browsing people will only see a low resolution, watermarked image. Currently the workflow is as follows:
PHP script handles the highres image upload, when image is uploaded, it's automatically re-sized to a low res image and to a thumbnail image as well and both of the files are saved on the server, (no watermark is added).
When people are browsing, the page displays the thumbnail of the image, on click, it enlarges and displays the lowres image with watermark as well. At the time being I apply the watermark on the fly whenever the lowres image is opened.
My question is, what is the correct way:
1) Should I save a 2nd copy of the lowres image with thumbnail, only when it's access for the first time? I mean if somebody access the image, I add the watermark on the fly, then display the image & store it on the server. Next time the same image is accessed if a watermarked copy exist just display the wm copy, otherwise apply watermark on the fly. (in case watermark.png is changed, just delete the watermarked images and they will be recreated as accessed).
2) Should I keep applying watermarks on the fly like I'm doing now.
My biggest question is how big is the difference between a PHP file_exists(), and adding a watermark to an image, something like:
$image = new Imagick();
$image->readImage($workfolder.$event . DIRECTORY_SEPARATOR . $cat . DIRECTORY_SEPARATOR .$mit);
$watermark = new Imagick();
$watermark->readImage($workfolder.$event . DIRECTORY_SEPARATOR . "hires" . DIRECTORY_SEPARATOR ."WATERMARK.PNG");
$image->compositeImage($watermark, imagick::COMPOSITE_OVER, 0, 0);
All lowres images are 1024x1024, JPG with a quality setting of 45%, and all unnecessary filters removed, so the file size of a lowres image is about 40Kb-80Kb.
It is somehow related to this question, just the scale and the scenarios is a bit different.
I'm on a dedicated server (Xeon E3-1245v2) cpu, 32 GB ram, 2 TB storage), the site does not have a big traffic overall, but it has HUGE spikes from time to time. When images are released we get a few thousand hits per hours with people browsing trough the images, downloading, purchasing, etc. So while on normal usage I'm sure that generating on the fly is the right approach, I'm a bit worried about the spike period.
Need to mention that I'm using ImageMagick library for image processing, not GD.
Thanks for your input.
UPDATE
None of the answers where a full complete solution, but that is good since I never looked for that. It was a hard decision which one to accept and whom to accord the bounty.
#Ambroise-Maupate solution is good, but yet it's relay on the PHP to do the job.
#Hugo Delsing propose to use the web server for serving cached files, lowering the calls to PHP script, which will mean less resources used, on the other hand it's not really storage friendly.
I will use a mixed-merge solution of the 2 answers, relaying on a CRON job to remove the garbage.
Thanks for the directions.
Personally I would create a static/cookieless subdomain in a CDN kinda way to handle these kind of images. The main reasons are:
Images are only created once
Only accessed images are created
Once created, an image is served from cache and is a lot faster.
The first step would be to create a website on a subdomain that points to an empty folder. Use the settings for IIS/Apache or whatever to disable sessions for this new website. Also set some long caching headers on the site, because the content shouldn't change
The second step would be to create an .htaccess file containing the following.
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*) /create.php?path=$1 [L]
This will make sure that if somebody would access an existing image, it will show the image directly without PHP interfering. Every non-existing request will be handled by the create.php script, which is the next thing you should add.
<?php
function NotFound()
{
if (!headers_sent()) {
$protocol = (isset($_SERVER['SERVER_PROTOCOL']) ? $_SERVER['SERVER_PROTOCOL'] : 'HTTP/1.0');
header($protocol . ' 404 Not Found');
echo '<h1>Not Found</h1>';
exit;
}
}
$p = $_GET['path'];
//has path
if (strlen($p)<=1)
NotFound();
$clean = explode('?', $p);
$clean = explode('#', $clean[0]);
$params = explode('/', substr($clean[0], 1)); //drop first /
//I use a check for two, because I dont allow images in the root folder
//I also use the path to determine how it should look
//EG: thumb/125/90/imagecode.jpg
if (count($params)<2)
NotFound();
$type = $params[0];
//I use the type to handle different methods. For this example I only used the full sized image
//You could use the same to handle thumbnails or cropped/watermarked
switch ($type) {
//case "crop":if (Crop($params)) return; else break;
//case "thumb":if (Thumb($params)) return; else break;
case "image":if (Image($params)) return; else break;
}
NotFound();
?>
<?php
/*
Just some example to show how you could create a responds
Since you already know how to create thumbs, I'm not going into details
Array
(
[0] => image
[1] => imagecode.JPG
)
*/
function Image($params) {
$tmp = explode('.', $params[1]);
if (count($tmp)!=2)
return false;
$code = $tmp[0];
//WARNING!! SQL INJECTION
//USE PROPER DB METHODS TO GET REALPATH, THIS IS JUST EXAMPLE
$query = "SELECT realpath FROM images WHERE Code='".$code."'";
//exec query here to $row
$realpath = $row['realpath'];
$f = file_get_contents($realpath);
if (strlen($f)<=0)
return false;
//create folder structure
#mkdir($params[0]);
//if you had more folders, continue creating the structure
//#mkdir($params[0].'/'.$params[1]);
//store the image, so a second request won't access this script
file_put_contents($params[0].'/'.$params[1], $f);
//you could directly optimize the image for web to make it even better
//optimizeImage($params[0].'/'.$params[1]);
//now serve the file to the browser, because even the first request needs to show the image
$finfo = finfo_open(FILEINFO_MIME_TYPE);
header('Content-Type: '.finfo_file($finfo, $params[0].'/'.$params[1]));
echo $f;
return true;
}
?>
I would suggest you to create watermarked images on-the-fly and to cache them at the same time as everybody suggested.
Then you could create a garbage-collector PHP script that will be executed every days (using cron). This script will browse your cache folder to read every image access time. This can done using fileatime() PHP method. Then when a cached wm image has not been accessed within 24 or 48 hours, just delete it.
With this method, you can handle spike periods as images are cached at the first request. AND you will save your HDD space as your garbage-collector script will delete unused images for you.
This method will only work if your server partition has atime updates enabled.
See http://php.net/manual/en/function.fileatime.php
For most scenarios, lazily applying the watermark would probably make most sense (generate the watermarked image on the fly when requested then cache the result) however if you have big spikes in demand you are creating a mechanism to DOS yourself: create the watermarked version on upload.
Considering your HDD storage capacity and Pikes.
I would only create a watermarked image if it is viewed.(so yes on the fly) In that way you dont use to much space with a bunch a files that are or might not be viewed.
I would not watermark thumbnails i would rather make a filter that fake watermark and protect from being saved. That filter would apply to all thumbnails without creating a second image.
In this way all your thumbbails are watermarked (Fake with onther element on top).
Then if one of these thumbnails is viewed it generate a watermarked image (only once) since after its generated you load the new watermarked image.
This would be the most efficient way to deal with your HDD storage and Pikes.
The other option would be to upgrade your hosting services. Godaddy offer unlimited storage and bandwith for about 50$ a year.
I can't get the cycle2 pager to work and I've checked the various questions and answers on this topic. I'm beginning to think it may have to do with my setup: the user clicks one of several buttons and that brings up one or more thumbnail images from a mysql database. Each thumbnail is related via the database to one or more larger images that I'd like to play in a slideshow. An ajax call sends the info about which thumb was clicked and a server page retrieves the larger images from a directory and displays them all in a div on the calling page as follows:
<?php
$thumb_path = $_POST['thumb_path'];
ob_start();
echo basename($thumb_path, ".png");
$thumb_name = ob_get_contents();
ob_end_clean();
$dir = "pathTo/$thumb_name/*.png";
$images = glob( $dir );
foreach( $images as $image ):
echo "<img src='" . $image . "' />";
endforeach;
?>
So far, so good as all the larger images are displayed (statically, all at once) in the div. The problem comes when I try to wrap the php script above in any of the cycle2 pagers (I'd like to use the one that automatically creates thumbnails for navigating the larger images, but none work). What happens is that all the larger images are loaded on top of each other without any navigation controls (thumbnails, dots, numbers etc. as the case may be). The larger images do play if I set the
data-cycle-timeout
parameter to something other than zero, but I'd like to set it to zero and let the users navigate the larger images themselves.
I'm a total noob at web programming and it's taken a while to get this stuff to work, chiefly by studying code snippets I've found on the web. I hope any solution doesn't involve a major redesign.
Thanks in advance for any help.
A site I am working on has a lot of images that are pulled from a database. The dimensions of the images are not consistent and I am trying to display them in uniformly sized boxes (divs). I do not know the dimensions of any of the images but I can retrieve them with:
document.getElementById( myImage ).width
document.getElementById( myImage ).height
After this I do my tests to see how to resize images to fit the uniform boxes. Finally I set the effects with:
document.getElementById( myImage ).width = theNewWidth
document.getElementById( myImage ).height = theNewHeight
This function is only called once per image by using onload="resizingFunction( imgId );" in the img tag. It takes about 1-2 seconds for every image in the database to complete this function and the function is never run for any of those images again. Despite never running again, the site runs significantly slower if I use this function. After googling I tried adding:
document.getElementById( myImage ).removeAttribute("width")
document.getElementById( myImage ).removeAttribute("height")
Before setting the new width and height. This did improve the speed but it is still slower than if I had not resized the images. Again, just for clarification, each image is resized one time after it has been loaded but for some reason this still slows down the site.
Images are created by being PHP echoed into JavaScript. This is necessary because they need information from the database (PHP), and the JavaScript places them inside the correct box (div). Here is the creation of image code:
echo "\t\t\tdocument.getElementById('gBox".$i."').innerHTML = '<img onload=\"image_applyToGrid(".$i.");\" id=\"img".$i."\" style=\"left:0; top:0;\" src=\"'+gBoxes[".$i."].imgPath+'\"/>';\n";
Here is the image resizing function that images call once onload:
function image_applyToGrid(inId) {
inIdImage = document.getElementById("img"+inId);
var imgW = inIdImage.width;
var imgH = inIdImage.height;
if (imgW > imgH) {
var proportions = imgW/imgH;
imgH = gridUnit;
imgW = gridUnit*proportions;
inIdImage.style.left = -((imgW-gridUnit)>>1)+"px";
}
else {
var proportions = imgH/imgW;
imgW = gridUnit;
imgH = gridUnit*proportions;
inIdImage.style.top = -((imgH-gridUnit)>>1)+"px";
}
inIdImage.removeAttribute("width");
inIdImage.removeAttribute("height");
inIdImage.width = imgW;
inIdImage.height = imgH;
}
Resizing images with Javascript is generally not an ideal approach. You are consuming all of the bandwidth to send the full size images across the web and then scaling them down. A better way would be to pull the images from your image store and resize them server side. Then store the result in a server side cache so you can provide all of your client requests with the optimized images. No need to over think the concept of cache here, in this case it could be as simple as a directory or a new column in your database. (FWIW, I prefer not to store binary data in databases but that's probably another discussion)
See: http://www.white-hat-web-design.co.uk/blog/resizing-images-with-php/
The images are probably much too large. I would suggest preparing the images for web use. Google "optimizing images for web" for some ideas on how to do this.
It's bad practice to take full-size images and load the entire image at different dimensions. Consider having thumbnails generated and placed on the page.
If you have a 2500x2500 image that you're just setting the height and width to 500x500, the entire 2500x2500 image is still having to be loaded.
I'm building a site that depends on bookmarklets. These bookmarklets pull the URL and a couple of other elements. However, I need to select 1 image from the page the user bookmarks. Currently I'm trying to use the PHP Simple HTML DOM Parser http://simplehtmldom.sourceforge.net/
It pulls the HTML as expected, and returns the tags as expected. However, I want to take this a step further and only return images with a min width of 40px. I know about the function getimagesize() but from what I understand, this is resource heavy. Is there a better method available to pre-process the image and achieve the results I'm looking for?
Thanks!
First check if the image HTML tag has a width attribute. If it's above 40, skip over it. As Matthew mentioned, it will get false positives where people sized down a large image to 40px wide, but that's no big deal; the point of this step is to quickly weed out the first dozen or so images that are obviously too big.
Once the script catches an image that SAYS it's under 40px wide, check the header information to deduce a general width based on the size of the file. This is faster than getimagesize because you don't have to download the image to get the info.
function get_image_kb($path) {
$headers = get_headers($path);
$len = explode(" ",$headers[6]);
return $len[1];
}
$imageKb = get_image_kb('test1.jpg');
// I'm going to gander 40x80 is about 2000kb
$cutoffSize = 2000;
if ($imageKb < $cutoffSize) {
// this is the one!
}
else {
// it was a phoney, keep scraping
}
Setting it at 2000kb will also let through images that are 100x30, which isn't good.
However, at this point, you've weeded out most of the huge 800kb files that would really slow you down, and because we know it's under 2kb, it's not too taxing to test this one with getimagesize() to get an accurate width.
You can tweak the process depending on how picky you are for the 40px mark, as usual higher accuracy takes more time, and vice versa.
I've developed an image-scraping mechanism in PHP+JS that allows a user to share URLs and get a rendered preview (very much like Facebook's previewer when you share links). However, the whole process sometimes gets slow or sometimes fetches wrong images, so in general, I'd like to know how to improve it, especially its speed and accuracy. Stuff like parsing the DOM faster or getting image sizes faster. Here's the process I'm using, for those who want to know more:
A. Get the HTML of the page using PHP (I actually use one of CakePHP's classes, which in turn use fwrite and fread to fetch the HTML. I wonder if cURL would be significantly better).
B. Parse the HTML using DOMDocument to get the img tags, while also filtering out any "image" that is not a png, jpg, or gif (you know, sometimes people place tracking scripts inside img tags).
$DOM = new DOMDocument();
#$DOM->loadHTML($html); //$html here is a string returned from step A
$images = $DOM->getElementsByTagName('img');
$imagesSRCs = array();
foreach ($images as $image) {
$src = trim($image->getAttribute('src'));
if (!preg_match('/\.(jpeg|jpg|png|gif)/', $src)) {
continue;
}
$src = urldecode($src);
$src = url_to_absolute($url, $src); //custom function; $url is the link shared
$imagesSRCs[] = $src;
}
$imagesSRCs = array_unique($imagesSRCs); // eliminates copies of a same image
C. Send an array with all those image tags to a page which processes using Javascript (specifically, JQuery). This processing consists mostly in discarding images that are less than 80pixels (so I dont get blank gifs, hundreds of tiny icons, etc.). Because it must calculate each image size, I decided to use JS instead of PHP's getimagesize() because it was insanely slow. Thus, as the images get loaded by the browser, it does the following:
$('.fetchedThumb').load(function() {
$smallestDim = Math.min(this.width, this.height);
if ($smallestDim < 80) {
$(this).parent().parent().remove(); //removes container divs and below
}
});
Rather than downloading the content like this, why not create a server-side component that uses something like wkhtmltoimage or PhantomJS to render an image of the page, and then just scale the image down to a preview size.
This is exactly why I made jQueryScrape
It's a very lightweight jQuery plugin + PHP proxy that lets you scrape remote pages asynchronously, and it's blazing fast. That demo I linked above goes to around 8 different sites and pulls in tons of content, usually in less than 2 seconds.
The biggest bottleneck when scraping with PHP is that PHP will try to download all referenced content (meaning images) as soon as you try to parse anything server side. To avoid this, the proxy in jQueryScrape actually breaks image tags on the server before sending it to the client (by changing all img tags to span tags.)
The jQuery plugin then provides a span2img method that converts those span tags back to images, so the downloading of images is left to the browser and happens as the content is rendered. You can at that point use the result as a normal jQuery object for parsing and rendering selections of the remote content. See the github page for basic usage.