Website I'm developing is showing products on sale and a lot of them are being updated and imported from XML (imported/updated 2x a day).
The problem is with getting product images. Some of them do not display even if url is provided.
Some XML contain from tens of items to hundreds of them.
The code loops through each item, gets all the data and then the image.
Example code regarding getting the image :
if (#file_get_contents($i->image)) {
$name = uniqid(rand(), true) . '.jpg';
$img = $name;
$url = $i->image;
$destinationPath = public_path() . '/img/upload/Items/'.$name;
#file_put_contents($destinationPath, #file_get_contents($url));
} else {
// do something else if there is no image
}
Is my approach wrong here ?
Remote server can't keep up with file requests ? And before each request need some pause ?
If items are updated 2x daily I shouldn't bother with saving images locally and give img src the external url from XML ?
Better use cURL() ?
If approach is accepted what improvement in needs ?
Thank you.
Related
When the gallery is loaded not all images are showing up. It is random. Meaning that sometimes the pages shows image 5 and the next it does not. It only happens when a lot of images need to get loaded.
I work with codeigniter and the images are stored outside of the htdocs folder. So therefore I work with a img controller to load in the images.
I only recently noticed this issue because it is the first time I need to load in 500+ images. So I presume it has something to do with the number of images the site needs to process.
IMG controller
public function jpg($foldername, $file)
{
if(($foldername != null) && ($file != null))
{
$filename = basename($file);
$file_extension = strtolower(pathinfo($filename, PATHINFO_EXTENSION));
switch($file_extension)
{
case "gif": $contentType="image/gif"; break;
case "png": $contentType="image/png"; break;
case "jpeg": $contentType="image/jpeg"; break;
case "jpg": $contentType="image/jpeg"; break;
default:
}
$path = DOCROOT . GALLERYS . $foldername . '/' . $file;
header('Content-type: ' . $contentType);
readfile($path);
}
}
Example of the code beeing used. It is used in a foreach but as it is kinda complex I will just show this.
$imageLocation = 'img/jpg/' . $folderName . '/' . $img['image_name'];
$id = $selected_gallery . $counter;
<img src="<?php echo base_url($imageLocation); ?>" alt="<?php echo $selected_gallery . ' ' . $counter; ?>">
I am looking for a solution so I can be sure it is all loaded in correctly.
I've worked on an image gallery many years ago, and the order the images are loaded is absolutely crucial to the gallery, and the browser doesn't know the order you need, so it loads them semi-randomly, or at least seemingly randomly. It may be that the issue you are seeing is just that it hasn't gotten to downloading your image yet, rather than never downloading it.
What you need is to pre-load your images in the order your user is likely to view them. If it's just a flat table that scrolls down your page is a little different than if you are showing them 1 image at a time, but it's a minimal difference.
This will have to be done in the browser, so you're looking for JavaScript, rather than a PHP solution. Your PHP will need to supply the image paths to the browser in the order the images should be displayed, though, and it'll likely have to be in a JavaScript array.
Going off the related Question here I'm going to suggest something similar to this:
<script type="text/javascript">
// created by your PHP code, or could be POST/GET using AJAX, but would be slightly delayed
var imagePathArray = [
"..\images\image1.png",
"..\images\image12.gif",
"..\images\image31.svg",
"..\images\image3.jpg"
];
var imageArray = [];
function PreloadImages(){
for (var i = 0; i < imagePathArray.length; i++){
var img = new Image();
img.src = imagePathArray[i];
imageArray.push(img);
}
}
</script>
...
<body onload="PreloadImages()">
...
(You can substitute you favorite loop, that's not what's important.)
What this does is start loading the images in the order you want and keeps a reference to the image, so the browser doesn't clear it's cache after a few seconds. You can try it without the imageArray variable if you want, since the disk cache should still have the image to reference, but a comment seemed to suggest that the browser would release it from memory and your pre-load would have been for nothing. It's been probably 10 years since I've worked on this project, so I don't remember how it all works, not to mention that browsers work significantly different since then.
For the slide show, you really only need to pre-load the images 1-5 places around the currently viewed image. Here, you can change the signature of the PreloadImages() to include a start and stop index for the array and change the start and stop indexes for the loop. You might also want to keep track of what's already pre-loaded so you don't load an image already loaded.
I have a question about the application generate QR code image.
I have an application when clients click a button there will generate a QR code image, my way is store in the project library, then print <img> with the url to the screen. then clients can see it.
But I have a doubt, if there are multi clients using the QR code at the same time, whether there will get a mix?
my code is bellow:
function generate_qrcode($url){
$filename = 'hante_qrcode.png';
$errorCorrectionLevel = 'L';
$matrixPointSize = 4;
//generate QR code image
$o = QRcode::png($url, $filename, $errorCorrectionLevel, $matrixPointSize, 2);
echo "<pre>";
print_r($o);
print_r('<img src="hante_qrcode.png">');
}
if there get mix, how to solve this problem?
But I have a doubt, if there are multi clients using the QR code at the same time, whether there will get a mix?
yes
how to solve this problem?
there are two ways to solve this problem
you can provide unique name for every files like using timestamp using time() function or with user ID. cause as per you are passing parameters while generating qr code you need to store the file. without saving file also possible but in that case you can't configure pixel size and frame size. you can refer this for PHP QR code-Examples
don't store image on server and find some js to generate qr code directly from client side.
having a one demo for that check if you can use it
var qrcode = new QRCode("qrcode");
qrcode.makeCode('https://stackoverflow.com');
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<script src="https://cdn.rawgit.com/davidshimjs/qrcodejs/gh-pages/qrcode.min.js"></script>
<div id="qrcode"></div>
Of course it will be overwritten.
Solution 1
Create unique filename for every image. This way you can save your images for use later. Another benefit of this, you don't have to create image again for same url.
$filename = md5($url) . ".png";
if(!file_exists($filename)){
$o = QRcode::png($url, $filename, ...);
}
echo '<img src="'.$filename.'">';
Solution 2
If you don't want to save images for disk space reasons you can serve image directly. In your code, user sends request to index.php and fetch image address as response. After then browser makes another request to get image. You can return image rather than returning html.
// image.php
// Still we want to give uniqe filename because we can get another request while one request is processing
$filename = md5(microtime) . "_qr.png";
$o = QRcode::png($url, $filename, ...);
$image = file_get_contents($filename);
// remove the file after stored in a variable
unlink($filename);
header('Content-Type: image/jpeg');
header('Content-Length: ' . filesize($image));
echo $image;
// index.html
<img src="image.php?url=someurl">
I'm building a web based system, which will host loads and loads of highres images, and they will be available for sale. Of course I will never display the highres image, instead when browsing people will only see a low resolution, watermarked image. Currently the workflow is as follows:
PHP script handles the highres image upload, when image is uploaded, it's automatically re-sized to a low res image and to a thumbnail image as well and both of the files are saved on the server, (no watermark is added).
When people are browsing, the page displays the thumbnail of the image, on click, it enlarges and displays the lowres image with watermark as well. At the time being I apply the watermark on the fly whenever the lowres image is opened.
My question is, what is the correct way:
1) Should I save a 2nd copy of the lowres image with thumbnail, only when it's access for the first time? I mean if somebody access the image, I add the watermark on the fly, then display the image & store it on the server. Next time the same image is accessed if a watermarked copy exist just display the wm copy, otherwise apply watermark on the fly. (in case watermark.png is changed, just delete the watermarked images and they will be recreated as accessed).
2) Should I keep applying watermarks on the fly like I'm doing now.
My biggest question is how big is the difference between a PHP file_exists(), and adding a watermark to an image, something like:
$image = new Imagick();
$image->readImage($workfolder.$event . DIRECTORY_SEPARATOR . $cat . DIRECTORY_SEPARATOR .$mit);
$watermark = new Imagick();
$watermark->readImage($workfolder.$event . DIRECTORY_SEPARATOR . "hires" . DIRECTORY_SEPARATOR ."WATERMARK.PNG");
$image->compositeImage($watermark, imagick::COMPOSITE_OVER, 0, 0);
All lowres images are 1024x1024, JPG with a quality setting of 45%, and all unnecessary filters removed, so the file size of a lowres image is about 40Kb-80Kb.
It is somehow related to this question, just the scale and the scenarios is a bit different.
I'm on a dedicated server (Xeon E3-1245v2) cpu, 32 GB ram, 2 TB storage), the site does not have a big traffic overall, but it has HUGE spikes from time to time. When images are released we get a few thousand hits per hours with people browsing trough the images, downloading, purchasing, etc. So while on normal usage I'm sure that generating on the fly is the right approach, I'm a bit worried about the spike period.
Need to mention that I'm using ImageMagick library for image processing, not GD.
Thanks for your input.
UPDATE
None of the answers where a full complete solution, but that is good since I never looked for that. It was a hard decision which one to accept and whom to accord the bounty.
#Ambroise-Maupate solution is good, but yet it's relay on the PHP to do the job.
#Hugo Delsing propose to use the web server for serving cached files, lowering the calls to PHP script, which will mean less resources used, on the other hand it's not really storage friendly.
I will use a mixed-merge solution of the 2 answers, relaying on a CRON job to remove the garbage.
Thanks for the directions.
Personally I would create a static/cookieless subdomain in a CDN kinda way to handle these kind of images. The main reasons are:
Images are only created once
Only accessed images are created
Once created, an image is served from cache and is a lot faster.
The first step would be to create a website on a subdomain that points to an empty folder. Use the settings for IIS/Apache or whatever to disable sessions for this new website. Also set some long caching headers on the site, because the content shouldn't change
The second step would be to create an .htaccess file containing the following.
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*) /create.php?path=$1 [L]
This will make sure that if somebody would access an existing image, it will show the image directly without PHP interfering. Every non-existing request will be handled by the create.php script, which is the next thing you should add.
<?php
function NotFound()
{
if (!headers_sent()) {
$protocol = (isset($_SERVER['SERVER_PROTOCOL']) ? $_SERVER['SERVER_PROTOCOL'] : 'HTTP/1.0');
header($protocol . ' 404 Not Found');
echo '<h1>Not Found</h1>';
exit;
}
}
$p = $_GET['path'];
//has path
if (strlen($p)<=1)
NotFound();
$clean = explode('?', $p);
$clean = explode('#', $clean[0]);
$params = explode('/', substr($clean[0], 1)); //drop first /
//I use a check for two, because I dont allow images in the root folder
//I also use the path to determine how it should look
//EG: thumb/125/90/imagecode.jpg
if (count($params)<2)
NotFound();
$type = $params[0];
//I use the type to handle different methods. For this example I only used the full sized image
//You could use the same to handle thumbnails or cropped/watermarked
switch ($type) {
//case "crop":if (Crop($params)) return; else break;
//case "thumb":if (Thumb($params)) return; else break;
case "image":if (Image($params)) return; else break;
}
NotFound();
?>
<?php
/*
Just some example to show how you could create a responds
Since you already know how to create thumbs, I'm not going into details
Array
(
[0] => image
[1] => imagecode.JPG
)
*/
function Image($params) {
$tmp = explode('.', $params[1]);
if (count($tmp)!=2)
return false;
$code = $tmp[0];
//WARNING!! SQL INJECTION
//USE PROPER DB METHODS TO GET REALPATH, THIS IS JUST EXAMPLE
$query = "SELECT realpath FROM images WHERE Code='".$code."'";
//exec query here to $row
$realpath = $row['realpath'];
$f = file_get_contents($realpath);
if (strlen($f)<=0)
return false;
//create folder structure
#mkdir($params[0]);
//if you had more folders, continue creating the structure
//#mkdir($params[0].'/'.$params[1]);
//store the image, so a second request won't access this script
file_put_contents($params[0].'/'.$params[1], $f);
//you could directly optimize the image for web to make it even better
//optimizeImage($params[0].'/'.$params[1]);
//now serve the file to the browser, because even the first request needs to show the image
$finfo = finfo_open(FILEINFO_MIME_TYPE);
header('Content-Type: '.finfo_file($finfo, $params[0].'/'.$params[1]));
echo $f;
return true;
}
?>
I would suggest you to create watermarked images on-the-fly and to cache them at the same time as everybody suggested.
Then you could create a garbage-collector PHP script that will be executed every days (using cron). This script will browse your cache folder to read every image access time. This can done using fileatime() PHP method. Then when a cached wm image has not been accessed within 24 or 48 hours, just delete it.
With this method, you can handle spike periods as images are cached at the first request. AND you will save your HDD space as your garbage-collector script will delete unused images for you.
This method will only work if your server partition has atime updates enabled.
See http://php.net/manual/en/function.fileatime.php
For most scenarios, lazily applying the watermark would probably make most sense (generate the watermarked image on the fly when requested then cache the result) however if you have big spikes in demand you are creating a mechanism to DOS yourself: create the watermarked version on upload.
Considering your HDD storage capacity and Pikes.
I would only create a watermarked image if it is viewed.(so yes on the fly) In that way you dont use to much space with a bunch a files that are or might not be viewed.
I would not watermark thumbnails i would rather make a filter that fake watermark and protect from being saved. That filter would apply to all thumbnails without creating a second image.
In this way all your thumbbails are watermarked (Fake with onther element on top).
Then if one of these thumbnails is viewed it generate a watermarked image (only once) since after its generated you load the new watermarked image.
This would be the most efficient way to deal with your HDD storage and Pikes.
The other option would be to upgrade your hosting services. Godaddy offer unlimited storage and bandwith for about 50$ a year.
I can't get the cycle2 pager to work and I've checked the various questions and answers on this topic. I'm beginning to think it may have to do with my setup: the user clicks one of several buttons and that brings up one or more thumbnail images from a mysql database. Each thumbnail is related via the database to one or more larger images that I'd like to play in a slideshow. An ajax call sends the info about which thumb was clicked and a server page retrieves the larger images from a directory and displays them all in a div on the calling page as follows:
<?php
$thumb_path = $_POST['thumb_path'];
ob_start();
echo basename($thumb_path, ".png");
$thumb_name = ob_get_contents();
ob_end_clean();
$dir = "pathTo/$thumb_name/*.png";
$images = glob( $dir );
foreach( $images as $image ):
echo "<img src='" . $image . "' />";
endforeach;
?>
So far, so good as all the larger images are displayed (statically, all at once) in the div. The problem comes when I try to wrap the php script above in any of the cycle2 pagers (I'd like to use the one that automatically creates thumbnails for navigating the larger images, but none work). What happens is that all the larger images are loaded on top of each other without any navigation controls (thumbnails, dots, numbers etc. as the case may be). The larger images do play if I set the
data-cycle-timeout
parameter to something other than zero, but I'd like to set it to zero and let the users navigate the larger images themselves.
I'm a total noob at web programming and it's taken a while to get this stuff to work, chiefly by studying code snippets I've found on the web. I hope any solution doesn't involve a major redesign.
Thanks in advance for any help.
I'm doing a check like this:
// ADDITIONAL PRODUCT IMAGE
$add1 = #get_headers("https://{image_server_link}/DIx.jpg_RB2000,2000,255,255,255,127/-/article/" . $product_number . "_1.jpg");
if(strpos($add1[0], "404") === FALSE) {
// Image found, so let's print it out
}
It's inside a CSV generator, and because of the csv contains 2000+ products the get_headers slows down generation from 3 to 4 MB/S to 10-15KB/s. As our CSV is 6.9 MB this will take ages.
Is their any other option to check if an additional image exists which is faster?
By the way, the images are hosted on a seperate image-server with varnish caching, so it's not possible to do a is_file function or something..
You could make a php script on the other side that gives you a list of available images or checks for availability of multiple images at once (a web service).
Edit:
Or just use the index page of the image directory.