Idea is to store user profile photos to file server. I have done with uploading part to static folder, but I want to make It a bit dynamic.
File name is generating in following: $userid . '-' . round(microtime(true)) . '.jpg';
I want to store images based on $userid, 1000 per folder.
So it have to check if $userid <= 1000 and folder not exists, create new folder named 000001-100000.
For example if $userid = 1001 it have to check if folder not exists and create new folder with name 002001-002000 and so on.
How could I achieve it dynamically? There could be over 100 000 users, so checking in following not so best idea I think:
if (!file_exists('images/000001-001000')) && $userid <= 1000 {
mkdir('images/000001-001000', 0777, true);
}
if (!file_exists('images/001001-002000')) && $userid > 1000 && $userid <= 2000 {
mkdir('images/001001-002000', 0777, true);
}
p.s. this is not duplicated as marked, I need to store 1000 photos per folder, not to create specific folder for each user.
$count = floor($userid / 1000);
$begin = ($count * 1000) + 1;
$end = ($count + 1) * 1000;
$strBegin = $begin;
$strEnd = $end;
if($begin==1){
$strBegin = "0001";
}
if(is_dir('images/'.$strBegin.'-'.$strEnd)==false){
mkdir('images/'.$strBegin.'-'.$strEnd);
}
Try this I have created recursive method to check value. Then create directory according it
function get_thousand_value($userid,$value_min)
{
if($userid <= $value_min){
$return_val = $value_min;
}
else
{
$value_min += 1000;
$return_val =get_thousand_value($userid,$value_min);
}
return $return_val;
}
$value_min = 1000;
$last_val = get_thousand_value($userid,$value_min);
$start_val = ($last_val == 1000 ? '0001' : ($last_val - 999));
$dir_name= 'images/'.$start_val."-".$last_val;
if (!file_exists($dir_name)) {
mkdir($dir_name, 0777, true);
}
Just generate folders once. If you can't predict which folder is last, you may generate 5-50 folder more and check it with cron; do I need to generate more folders or not?
Related
Im trying to implement a solution for database user limit exceed problem by using a different database user every 3 minutes.
I created 4 users and the script works fine with 3 users but at the forth one, it returns the first username again and keep returning it a while.
<?php
function GetUser()
{
$file = $_SERVER['DOCUMENT_ROOT'] .'/dbUsers.txt';
$users = array("user1", "user2", "user3","user4");
$user = "";
$userIndex = 0;
if (file_exists($file)) {
if (filemtime($file) < time()-100) //0.5*3600
{
$userIndex = file_get_contents($file);
if($userIndex >4)
$userIndex =0;
file_put_contents($file, $userIndex + 1);
}
}
else
{
file_put_contents($file, $userIndex);
}
$user = $users[$userIndex];
return $user;
}
?>
Thanks in advance.
Why using a file and file time. You said "every 30 minutes"
function GetUser()
{
# static: define once on scan/compile time
static $users = array("user1","user2","user3","user4");
$index = intval( time() / (30*60) ) % count($users);
return $users[$index];
}
Notes: time() returns the number of seconds since epoch (1971-01-01 0:00:00 UTC). Then I divided it by 30 minutes (30*60). The modulo operator % iterates through the user list.
At Prestashop when I remove some products, the images remains at their directory /img/p/ so at the moment my images directory it's close to being 2GB.
Found this script to delete useless and BBDD unlinked images on Prestashop but I don't know why It's only detecting the orphan images but not removing them. Maybe the unlink function is disabled on the php configuration? Is it possible? Or the code is wrong? Is there another way to do this kind of cleanup?
<?php
// root path of the shop
$shop_root='/home/myweb/public_html/';
// limit number of image files to check, set to 10 for testing
$limit=1000000000;
include $shop_root . '/config/settings.inc.php';
$pdo = new PDO( 'mysql:host='._DB_SERVER_.';dbname='._DB_NAME_, _DB_USER_, _DB_PASSWD_ );
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$r=$pdo->query('select count(1) cnt from ps_image')->fetch();
echo 'count images database: '.$r['cnt'] . "<Br />";
// reset some counters
$cnt_files=0;
$cnt_checked=0;
$cnt_not_found=0;
$cnt_found=0;
for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
$path=$shop_root.'img/p/'.$ii;
delImage($path);
for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
{
$path=$shop_root.'img/p/'.$ii.'/'.$jj;
delImage($path);
for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
{
$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
delImage($path);
for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
{
$path=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
delImage($path);
}
}
}
}
echo 'files: '.$cnt_files.' checked: '.$cnt_checked.' not_found: '.$cnt_not_found.' found: '.$cnt_found;
function delImage($imageDir)
{
global $limit, $pdo, $cnt_files, $cnt_checked, $cnt_not_found, $cnt_found;
if ($handle = #opendir($imageDir)) { //# is wriiten to avoid warning message and is handled in else condition
echo $imageDir."<BR />";
while ($cnt_files != $limit && false !== ($entry = readdir($handle))) {
if ($entry != "." && $entry != "..") {
$cnt_files++;
$pi = explode('-',$entry);
if($pi[0]>0 && !empty($pi[1])) {
$cnt_checked++;
if(!checkExistsDb($pdo,$pi[0])) {
$cnt_not_found++;
echo 'rm '.$imageDir.'/'.$entry."<BR />";
unlink($imageDir.'/'.$entry);
} else {
$cnt_found++;
}
}
}
}
closedir($handle);
}
else
{
echo $imageDir." doesn't exist".'<BR />';
}
}
function checkExistsDb($pdo, $id_image) {
$r=$pdo->query($q='select \'ok\' ok, (select id_image from ps_image where id_image = '.(int)$id_image.') id_image');
$row=$r->fetch();
if($row['ok']!='ok') die( 'Problem with query, please correct');
return $row['id_image']?true:false;
}
?>
Here's an output piece:
/home/myweb/public_html/img/p/9/9/7
/home/myweb/public_html/img/p/9/9/7/0 doesn't exist
/home/myweb/public_html/img/p/9/9/7/1 doesn't exist
/home/myweb/public_html/img/p/9/9/7/2 doesn't exist
/home/myweb/public_html/img/p/9/9/7/3 doesn't exist
/home/myweb/public_html/img/p/9/9/7/4 doesn't exist
/home/myweb/public_html/img/p/9/9/7/5 doesn't exist
/home/myweb/public_html/img/p/9/9/7/6 doesn't exist
/home/myweb/public_html/img/p/9/9/7/7 doesn't exist
/home/myweb/public_html/img/p/9/9/7/8 doesn't exist
/home/myweb/public_html/img/p/9/9/7/9 doesn't exist
/home/myweb/public_html/img/p/9/9/8
/home/myweb/public_html/img/p/9/9/8/0 doesn't exist
/home/myweb/public_html/img/p/9/9/8/1 doesn't exist
/home/myweb/public_html/img/p/9/9/8/2 doesn't exist
/home/myweb/public_html/img/p/9/9/8/3 doesn't exist
/home/myweb/public_html/img/p/9/9/8/4 doesn't exist
/home/myweb/public_html/img/p/9/9/8/5 doesn't exist
/home/myweb/public_html/img/p/9/9/8/6 doesn't exist
/home/myweb/public_html/img/p/9/9/8/7 doesn't exist
/home/myweb/public_html/img/p/9/9/8/8 doesn't exist
/home/myweb/public_html/img/p/9/9/8/9 doesn't exist
/home/myweb/public_html/img/p/9/9/9
/home/myweb/public_html/img/p/9/9/9/0 doesn't exist
/home/myweb/public_html/img/p/9/9/9/1 doesn't exist
/home/myweb/public_html/img/p/9/9/9/2 doesn't exist
/home/myweb/public_html/img/p/9/9/9/3 doesn't exist
/home/myweb/public_html/img/p/9/9/9/4 doesn't exist
/home/myweb/public_html/img/p/9/9/9/5 doesn't exist
/home/myweb/public_html/img/p/9/9/9/6 doesn't exist
/home/myweb/public_html/img/p/9/9/9/7 doesn't exist
/home/myweb/public_html/img/p/9/9/9/8 doesn't exist
/home/myweb/public_html/img/p/9/9/9/9 doesn't exist
files: 29013 checked: 22290 not_found: 0 found: 22290
Try this. I spent a lot of time to develop this after many failures from different solutions. For me it is the safest method.
<?php
####PUT THIS FILE INTO YOUR MAIN SHOP FOLDER####
// root path of the shop, almost no one needs to change something here.
$shop_root = $_SERVER['DOCUMENT_ROOT']."/"; // need to have slash / at the end
$image_folder = 'img/p/'; // also needs slash at the ennd
$scan_dir = $shop_root.$image_folder;
include_once($shop_root.'config/config.inc.php');
include $shop_root . 'config/settings.inc.php';
#---------------------------------------------#
$last_id = (int)Db::getInstance()->getValue('
SELECT id_image FROM '._DB_PREFIX_.'image ORDER BY id_image DESC
');
$counted_images = Db::getInstance()->executeS('
SELECT count(*) as qnt FROM '._DB_PREFIX_.'image
');
$counted_images = (int)$counted_images[0]['qnt'];
echo 'There was '.$last_id.' images in database but only '.$counted_images.' is used right now. Lets check how many of them are eating up our storage without no reason.<br>';
//$limit = 150; // for testing
$limit = $last_id; // for production
$removed_images = 0;
for ($i=1; $i <= $limit; $i++) {
if (!imageExistsInDB($i)){
$imageDir = str_split($i);
$imageDir = implode('/', $imageDir);
$path = $scan_dir.$imageDir;
deleteImagesFromPath($path);
}
}
function deleteImagesFromPath($path) {
global $removed_images;
$images = glob($path . '/*.{jpg,png,gif,jpeg}', GLOB_BRACE);
if ($images){
foreach ($images as $file) {
if (is_file($file)) {
unlink($file);
}
}
$removed_images++;
echo 'Deleted images from folder ' . $path . '/' ."<br/>";
}
}
function imageExistsInDB($id_image){
return Db::getInstance()->getValue('
SELECT id_image FROM '._DB_PREFIX_.'image WHERE id_image = '.(int)$id_image
);
}
echo '--------------------------------------<br>';
if ($removed_images > 0)
echo 'Hurray! We removed '.$removed_images.' product images!';
else
echo 'Everything is ok with Your images. I did not removed any of them or I made it before. Good Job Presta!';
Check the files permissions. If php user has not permissions to delete files of other users the function unlink will not work.
It will never work because you remove the root of your path:
you remove: /home/myweb/public_html/img/p/9/9/7
and after you want to remove: /home/myweb/public_html/img/p/9/9/7/0 but /home/myweb/public_html/img/p/9/9/7 doesn't exist :)
To work do this:
for($ii=1; ($ii<=9) && ($cnt_files != $limit); $ii++)
{
$pathRacine=$shop_root.'img/p/'.$ii;
//delImage($path);
for($jj=0; ($jj<=9) && ($cnt_files != $limit); $jj++)
{
$path1=$shop_root.'img/p/'.$ii.'/'.$jj;
//delImage($path);
for($kk=0; ($kk<=9) && ($cnt_files != $limit); $kk++)
{
$path2=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk;
//delImage($path);
for($ll=0; ($ll<=9) && ($cnt_files != $limit); $ll++)
{
$path3=$shop_root.'img/p/'.$ii.'/'.$jj.'/'.$kk.'/'.$ll;
delImage($path3);
}
delImage($path2);
}
delImage($path1);
}
delImage($pathRacine);
}
And do not use the same variable name everywhere
For your information, when you delete the product Prestashop automatically delete all the images associated with the product.
I have also checked your script and it worked fine for me by changing $shop_root and adding configuration file.
require(dirname(__FILE__).'/config/config.inc.php');
// root path of the shop
$shop_root = $_SERVER['DOCUMENT_ROOT']."/";
You need to check folder permission and check why it's not allowing to unlink images.
if(!empty($_POST['newdir'])) {
$dir_base= preg_replace("/[^a-zA-ZÀ-ÿ0-9\s]/",'',$_POST['newdir']);
$dir_path=$pattern.$dir_base;
$dir_path_lwr = strtolower($dir_path);
$old_dirs = glob($pattern.'*', GLOB_ONLYDIR);
$old_dirs_lwr = array_map('strtolower', $old_dirs);
$i = 1;
$cond = true;
while($cond) {
if(in_array($dir_path_lwr, $old_dirs_lwr)) {
$i++;
$new_base=$dir_base.' '.$i.'';
$new_path= $pattern.$new_base;
$dir_path_lwr = strtolower($new_path);
}
elseif(!in_array($dir_path_lwr, $old_dirs_lwr)) {
mkdir($dir_path, 0755);
$cond = false;
}
}
}
EDIT Have changed the above to the most recent version. For testing, I am now simply echoing out the $cond var rather than creating directories all over the place. The 'Not In' argument works fine - when a duplicate name IS found, the page takes about 10 to 12 seconds to load, before white screening. There's something wrong with my while loop i think...
What I'm trying to do is:
Submit a new directory name to the script
Clean and escape that name
Check it against existing directories in that location ($pattern)
If there is no conflict, create the directory (works)
If there is a conflict, append a number (eg, My Directory 2) and then create the directory (doesn't work)
So, basically how to check against existing file names, and create a uniquely name directory dynamically. All help appreciated :)
Fixed it - i was confusing the vars within the loop.
Posting for future reference:
if(!empty($_POST['newdir'])) {
$dir_base= preg_replace("/[^a-zA-ZÀ-ÿ0-9\s]/",'',$_POST['newdir']);
$dir_path=$pattern.$dir_base;
$dir_path_lwr = strtolower($dir_path);
$old_dirs = glob($pattern.'*', GLOB_ONLYDIR);
$old_dirs_lwr = array_map('strtolower', $old_dirs);
$i = 1;
$cond = true;
while($cond) {
if(in_array($dir_path_lwr, $old_dirs_lwr)) {
$i++;
$new_base=$dir_base.' '.$i.'';
$dir_path= $pattern.$new_base;
$dir_path_lwr = strtolower($dir_path);
}
elseif(!in_array($dir_path_lwr, $old_dirs_lwr)) {
mkdir($dir_path, 0755);
$cond = false;
}
}
}
I have a file uploader and I want the filenames to auto increment number. I don't feel the need to use a database to do this and I want to keep the code relatively clean, I'm pretty new in file upload and management in PHP so I'm not exactly sure what to do. Could anyone direct me in the right path?
Here is my current code, it just uses an md5 of a bunch of seeds.
<?php
if(isset($_FILES['imagedata']['tmp_name']))
{
// Directory related to the location of your gyazo script
$newName = 'images/' . substr(md5(rand() . time()), 0, 20) . '.png';
$tf = fopen($newName, 'w');
fclose($tf);
move_uploaded_file($_FILES['imagedata']['tmp_name'], $newName);
// Website
echo 'http://davidknag.com/' . $newName;
}
?>
<?php
if(isset($_FILES['imagedata']['tmp_name'])) {
// Directory related to the location of your gyazo script
$fileCount = count (glob ('images/*.png'));
$newName = 'images/' . ( $fileCount + 1) . '.png';
$tf = fopen($newName, 'w');
fclose($tf);
move_uploaded_file($_FILES['imagedata']['tmp_name'], $newName);
// Website
echo 'http://davidknag.com/' . $newName;
}
It just counts all .png files in the directory, increments that number by 1 and uses that as its filename.
Note that if you're storing a very large amount of files (say 10.000s), it's faster to use Joseph Lusts' method, but otherwise this will work jus tfine.
You can just have a basic text file in the given folder. Store the number in there. Read it out and increment it as needed.
It would be easiest to make a function like getNextNumber() that did the above and then you could use it as needed. You could also do this in a $_SERVER[] variable, but it would need to be reloaded from the file on server restart.
<?PHP
// a basic example
function getNextNumber() {
$count = (int)file_get_contents('yourFile.txt');
$count+=1;
file_put_contents('yourFile.txt',$count);
return $count;
}
?>
Note that if you are using this a great deal, you'll need a more advanced sequence generator since this will perform 2 file IO's on each call.
You can try the code below. It creates a file with .png extension and unique name in outdir/
$filename = uniqFile('outdir', '.png');
move_uploaded_file($_FILES['imagedata']['tmp_name'], $filename);
function uniqFile($dir, $ext)
{
if (substr($dir, -1, 1) != '/')
{
$dir .= '/';
}
for ($i=1; $i<999999; $i++)
{
if (!is_file($dir . $i . $ext))
{
return $i . $ext;
}
}
return false;
}
A little late in the game but this pair of functions does the trick and follows the familiar format of the filename followed by "(n)" and then the file extension:
incrementFileName() returns the updated filename incremented by 1 with input filename and destination directory. splitLast() is a modification of explode to only split on the last occurrence of some substring.
function incrementFileName($name,$path){
if (!array_search($name,scandir($path))) {
return $name;
} else {
$ext=splitLast($name,".")[1];
$baseFileName=splitLast(splitLast($name,".")[0],"(")[0];
$num=intval(splitLast(splitLast($name,"(")[1],")")[0])+1;
return incrementFileName($baseFileName."(".$num.").".$ext,$path);
}
}
function splitLast($string,$delim) {
$parts = explode($delim, $string);
if (!$parts || count($parts) === 1) {
$before=$string;
$after="";
} else {
$after = array_pop($parts);
$before=implode($delim, $parts);
}
return array($before,$after);
}
When handling upload, set your filename with it:
$fileName = incrementFileName($_FILES['file']['name'], $path);
This will return someFileName(1).jpg or someFileName(2).jpg etc.
function enc($length = "string") {
if(!is_numeric($length) || $length > 255 || $length < 1){
$length = rand("3","6");
}
// $randomID = substr(uniqid(sha1(crypt(md5("".time("ysia", true)."".rand())))), 0, $length);
$randomID = genUnique($length);
$count = 0;
while(glob("$randomID.*") || fetch("select * from `short` where `short` = '$randomID'") || fetch("select * from `images` where `name` = '$randomID'") || glob("img/$randomID.*") || is_numeric($randomID)){
if($count > 20){
$length++;
}
$randomID = genUnique($length);
$count++;
}
return $randomID;
}
this code is pretty old (not even using mysqli), but i figured i'd include it first
<?php
include_once "functions.php";
if(!isset($_REQUEST['api'])){
notfound("");
}
$con = connect();
$key = $_REQUEST['api'];
$ver = $_REQUEST['version'];
if($ver != "10-26-2016" || $key == "zoidberg")
{
die("Please upgrade your in4.us.exe by logging in and clicking download.");
}
if($key == "nokey"){
die("You need to keep the exe with the ini file to pair your api key. Copy ini file to same directory or redownload.");
}
$key = mysql_real_escape_string($key);
$findkey = fetch(" SELECT * from `users` where `key` = '$key' ");
if(!is_array($findkey)){
die("No user with that API Key found. Configure the INI File using your api key on in4.us");
}
$user = $findkey['username'];
if(isset($_FILES['imagedata']['tmp_name'])){
$newName = enc();
$tf = fopen("img/".$newName.".png", 'w');
fclose($tf);
move_uploaded_file($_FILES['imagedata']['tmp_name'], "img/".$newName.".png");
$domain = $_SERVER['HTTP_HOST'];
date_default_timezone_set('America/New_York');
$mysqldate = date("Y-m-d H:i:s");
$qry = mysql_query("INSERT INTO `images` (`name`, `added`, `dateadded`) VALUES ('$newName', '$user', '$mysqldate');");
if(!qry){
die('Invalid query: ' . mysql_error());
}
echo "http://$domain/$newName.png";
disconnect($con);
}else{
notfound("");
}
?>
I'm currently using two functions to delete from each folder after 1 minute but as they basically do the same thing (just different folders called). I was wondering if they could be merged into one?
function DeleteFromFolder1() {
$captchaFolder = 'folder1/';
$fileTypes = '*.jpg';
$expire_time = 1;
foreach(glob($captchaFolder . $fileTypes) as $Filename) {
$FileCreationTime = filectime($Filename);
$FileAge = time() - $FileCreationTime;
if($FileAge > ($expire_time * 60))
{
unlink($Filename);
}
}
}
function DeleteFromFolder2() {
$captchaFolder = 'folder2/';
$fileTypes = '*.jpg';
$expire_time = 1;
foreach(glob($captchaFolder . $fileTypes) as $Filename) {
$FileCreationTime = filectime($Filename);
$FileAge = time() - $FileCreationTime;
if($FileAge > ($expire_time * 60))
{
to ($Filename);
}
}
}
Pass the folder name as an argument.
function DeleteFromFolder($captchaFolder) {
$fileTypes = '*.jpg';
$expire_time = 1;
foreach(glob($captchaFolder . $fileTypes) as $Filename) {
$FileCreationTime = filectime($Filename);
$FileAge = time() - $FileCreationTime;
if($FileAge > ($expire_time * 60))
{
unlink($Filename);
}
}
}
Thanks for your answers everyone but I have now sorted it by adding:
unlink(path/to/temp/image.jpg);
to my results page which deletes the uploaded image once the thumb is created and removed the function associated with it.
Once again thanks for your answers :)
function DeleteFromFolder($captchaFolder) {
$fileTypes = '*.jpg';
// and so on
}
If you like, you can add two helper functions
function DeleteFromFolder1() { return DeleteFromFolder('folder1/'); }
function DeleteFromFolder2() { return DeleteFromFolder('folder2/'); }
To try to do this without editing your structure you can pass a variable into your main function. You can do something like this
function DeleteFromFolder1($dir=NULL) {
if($dir == NULL)
$captchaFolder = 'folder1/';
else
$captchaFolder = $dir;
$fileTypes = '*.jpg';
$expire_time = 1;
foreach(glob($captchaFolder . $fileTypes) as $Filename) {
$FileCreationTime = filectime($Filename);
$FileAge = time() - $FileCreationTime;
if($FileAge > ($expire_time * 60))
{
unlink($Filename);
}
}
}
function DeleteFromFolder2() {
DeleteFromFolder1("folder2/");
}
}
That should work without making any major changes to your current code base.
EDIT (Adding more of a description for some clarity)
I was assuming that your code was already implemented in some way. If that is the case, a rather clunky solution is like mine above (this will allow the smallest number of edits to be made). Otherwise, you could consolidate this function to just the first one and it will work fine. DeleteFromFolder2() is merely a redirect function.
The function takes an argument $file which is null if not declared when calling the function. If $file == NULL, then it will delete folder1 by default, otherwise, it will attempt to delete the folder specified. I hope that clears things up a bit!
Good luck!
Dennis M.