I am currently in the process of writing a mobile app with the help of phonegap. One of the few features that I would like this app to have is the ability to capture an image and upload it to a remote server...
I currently have the image capturing and uploading/emailing portion working fine with a compiled apk... but in my php, I am currently naming the images "image[insert random number from 10 to 20]... The problem here is that the numbers can be repeated and the images can be overwritten... I have read and thought about just using rand() and selecting a random number from 0 to getrandmax(), but i feel that I might have the same chance of a file overwriting... I need the image to be uploaded to the server with a unique name every-time, no matter what... so the php script would check to see what the server already has and write/upload the image with a unique name...
any ideas other than "rand()"?
I was also thinking about maybe naming each image... img + date + time + random 5 characters, which would include letters and numbers... so if an image were taken using the app at 4:37 am on March 20, 2013, the image would be named something like "img_03-20-13_4-37am_e4r29.jpg" when uploaded to the server... I think that might work... (unless theres a better way) but i am fairly new to php and wouldn't understand how to write something like that...
my php is as follows...
print_r($_FILES);
$new_image_name = "image".rand(10, 20).".jpg";
move_uploaded_file($_FILES["file"]["tmp_name"], "/home/virtual/domain.com/public_html/upload/".$new_image_name);
Any help is appreciated...
Thanks in advance!
Also, Please let me know if there is any further info I may be leaving out...
You may want to consider the PHP's uniqid() function.
This way the code you suggested would look like the following:
$new_image_name = 'image_' . date('Y-m-d-H-i-s') . '_' . uniqid() . '.jpg';
// do some checks to make sure the file you have is an image and if you can trust it
move_uploaded_file($_FILES["file"]["tmp_name"], "/home/virtual/domain.com/public_html/upload/".$new_image_name);
Also keep in mind that your server's random functions are not really random. Try random.org if you need something indeed random. Random random random.
UPD: In order to use random.org from within your code, you'll have to do some API requests to their servers. The documentation on that is available here: www.random.org/clients/http/.
The example of the call would be: random.org/integers/?num=1&min=1&max=1000000000&col=1&base=10&format=plain&rnd=new. Note that you can change the min, max and the other parameters, as described in the documentation.
In PHP you can do a GET request to a remote server using the file_get_contents() function, the cURL library, or even sockets. If you're using a shared hosting, the outgoing connections should be available and enabled for your account.
$random_int = file_get_contents('http://www.random.org/integers/?num=1&min=1&max=1000000000&col=1&base=10&format=plain&rnd=new');
var_dump($random_int);
You should use tempnam() to generate a unique file name:
// $baseDirectory Defines where the uploaded file will go to
// $prefix The first part of your file name, e.g. "image"
$destinationFileName = tempnam($baseDirectory, $prefix);
The extension of your new file should be done after moving the uploaded file, i.e.:
// Assuming $_FILES['file']['error'] == 0 (no errors)
if (move_uploaded_file($_FILES['file']['tmp_name'], $destinationFileName)) {
// use extension from uploaded file
$fileExtension = '.' . pathinfo($_FILES['file']['name'], PATHINFO_EXTENSION);
// or fix the extension yourself
// $fileExtension = ".jpg";
rename($destinationFileName, $destinationFileName . $fileExtension);
} else {
// tempnam() created a new file, but moving the uploaded file failed
unlink($destinationFileName); // remove temporary file
}
Have you considered using md5_file ?
That way all of your files will have unique name and you would not have to worry about duplicate names. But please note that this will return same string if the contents are the same.
Also here is another method:
do {
$filename = DIR_UPLOAD_PATH . '/' . make_string(10) . '-' . make_string(10) . '-' . make_string(10) . '-' . make_string(10);
} while(is_file($filename));
return $filename;
/**
* Make random string
*
* #param integer $length
* #param string $allowed_chars
* #return string
*/
function make_string($length = 10, $allowed_chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890') {
$allowed_chars_len = strlen($allowed_chars);
if($allowed_chars_len == 1) {
return str_pad('', $length, $allowed_chars);
} else {
$result = '';
while(strlen($result) < $length) {
$result .= substr($allowed_chars, rand(0, $allowed_chars_len), 1);
} // while
return $result;
} // if
} // make_string
Function will create a unique name before uploading image.
// Upload file with unique name
if ( ! function_exists('getUniqueFilename'))
{
function getUniqueFilename($file)
{
if(is_array($file) and $file['name'] != '')
{
// getting file extension
$fnarr = explode(".", $file['name']);
$file_extension = strtolower($fnarr[count($fnarr)-1]);
// getting unique file name
$file_name = substr(md5($file['name'].time()), 5, 15).".".$file_extension;
return $file_name;
} // ends for is_array check
else
{
return '';
} // else ends
} // ends
}
Related
I have a <form> with a multiple file input that stores those files into the server. The names of the files must follow a numeric standard and that's why I rename them before storing. Each post has an ID an can have more than one file so the pattern is: post-id + dash and a number + file extension, this way, if the same post has two file with the same extensions, the dash + number will avoid them to be overwriten. So, before I store the files I run a loop to find the proper number to the name. The problem is, the function to verify the existence of the file seems not to be returning true when it should. The code:
$counter = 0;
do{
$nomeArquivo = $post->id . "-{$counter}" . "." . $arq->extension();
$counter++;
//these commented are other ways of verification I tried
//}while(Storage::exists($nomeArquivo));
//}while(file_exists("/storage/" . $nomeArquivo));
}while(is_file("/storage/" . $post->id . "-{$counter}"));
Storage::putFileAs('/public', $arq, $nomeArquivo);
This code above runs inside a foreach($files as $arq) where $files are the files from the form input.
Reference: https://laravel.com/docs/5.4/filesystem and https://laravel.com/api/5.4/Illuminate/Filesystem/Filesystem.html
Use the File Facade to check for the existence of a file:
File::exists($myfile)
http://laravel-recipes.com/recipes/123/determining-if-a-file-exists
If name of the file need to be unique only then,
You can give Unique Name like this:
$file = $request->file('file');
$extension = $file->getClientOriginalExtension();
$destination ='/files/';
$filename = uniqid() . '.' . $extension;
$file->move($destination, $filename);
Now save above file name in your database. Hope it may help you
As user #fubar pointed in the comments, I was referencing the wrong path to verify the existence of the file. I change the while condition to while(\File::exists(public_path() . '/storage/' . $nomeArquivo)) and it worked well. Thanks
I want to rename all files in a folder with random numbers or characters.
This my code:
$dir = opendir('2009111');
$i = 1;
// loop through all the files in the directory
while ( false !== ( $file = readdir($dir) ) ) {
// do the rename based on the current iteration
$newName = rand() . (pathinfo($file, PATHINFO_EXTENSION));
rename($file, $newName);
// increase for the next loop
$i++;
}
// close the directory handle
closedir($dir);
but I get this error:
Warning: rename(4 (2).jpg,8243.jpg): The system cannot find the file specified
You're looping through files in the directory 2009111/, but then you refer to them without the directory prefix in rename().
Something like this should work better (though see the warning about data loss below):
$oldName = '2009111/' . $file;
$newName = '2009111/' . rand() . (pathinfo($file, PATHINFO_EXTENSION));
rename($oldName, $newName);
Of course, you may want to put the directory name in a variable or make other similar tweaks. I'm still not clear on why you're trying to do this, and depending on your goals there may be better ways of reaching them.
Warning! The approach you are using could cause data loss! A $newName could be generated that is the same name as an existing file, and rename() overwrites target files.
You should probably make sure $newName doesn't exist before you rename().
I want to download about 200 images from an URL. For instance: Fetch www.web.com/images/001.png and download, fetch www.web.com/images/002.png and download and finish in 200.png.
I've read Grab/download images from multiple pages using php preg_match_all & cURL but I don't know how to modify the PHP to do that thing.
I'll be very appreciated if you can help me. Thank you so much
The simplest way that comes to mind, is to create a for loop, which counts from 1 to 200 and for every count, it then does a request for the image and saves it to the disk (I assume you want to save the image to your disk). Working example can be found at the bottom.
Let's start by settings some variables:
$baseUrl = 'http://www.web.com/';
$localDirectory = 'downloaded_images/';
$maxImageNumber = 200;
The first variable $baseUrl defines where the images will be loaded from. The second defines your local directory in which the images will be saved. Please make sure that this directory exists before running the code, because it will not be automatically generated. The last variable, $maxImageNumber stores the largest image number, which we will need for the for loop.
After setting the variables, we can write the for loop. Separated by semicolons ; there are three parts in the brackets. First one being the starting point of the number we will be counting upwards. The second makes sure we don't go over our limit and the last one just states that the number will be counted up, using the ++ shorthand.
for($imageNumber = 1; $imageNumber <= $maxImageNumber; $imageNumber++) {
// code goes here
}
Inside this for loop we can now generate our file name and download/save the image to the disk. The first line uses the function str_pad() to add leading 0s to the image file (just like in your example) and then adds the extension .png to the name. This allows us to reuse the file name for loading as well as for saving. Second line loads the image with the function file_get_contents() by combining the base and the image file name. In the last line we use file_put_contents() to save the $fileData, we loaded just before, to the disk (using the local directory and the image file name).
$imageFileName = str_pad($imageNumber, 3, '0', STR_PAD_LEFT) . '.png';
$fileData = file_get_contents($baseUrl . $imageFileName);
file_put_contents($localDirectory . $imageFileName, $fileData);
The complete code should look like this:
<?php
$baseUrl = 'http://www.web.com/images/';
$localDirectory = 'downloaded_images/';
$maxImageNumber = 10;
for($imageNumber = 1; $imageNumber <= $maxImageNumber; $imageNumber++) {
$imageFileName = str_pad($imageNumber, 3, '0', STR_PAD_LEFT) . '.png';
$fileData = file_get_contents($baseUrl . $imageFileName);
file_put_contents($localDirectory . $imageFileName, $fileData);
}
In the unlikely event that this all should not work, there are a few things you can try
check that the $localDirectory exists and is writable.
if the file_get_contents() should not work, you can use curl instead
make sure the computer this code is executed on can access the remote server
Thank you so much but imagine that pics are in www.web.com/image_0001_big.png and www.web.com/image_002_big.png and so on... (last will be web.com/image_0200_big.png)
This doesn't work (i don't know why)
<?php
$baseUrl = 'http://www.web.com/images';
$localDirectory = 'downloaded_images/';
$maxImageNumber = 200;
$input='image_';
for($imageNumber = 001; $imageNumber <= $maxImageNumber; $imageNumber++) {
$imageFileName = str_pad($input, $imageNumber, 4, '_big', STR_PAD_LEFT) . '.png';
$fileData = file_get_contents($baseUrl . $imageFileName);
file_put_contents($localDirectory . $imageFileName, $fileData);
}
Thanks. You're amazing! Thanks for the help! I'm learning PHP and I'm a beginner yet:(
My error:
Warning: str_pad() expects at most 4 parameters, 5 given in C:\xampp\htdocs\auto.php on line 9
Warning: file_get_contents(http://www.web.com/images/.png): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in C:\xampp\htdocs\auto.php on line 10
I've found a semi-solution:_
<?php
$baseUrl = 'http://web.com/images';
$localDirectory = 'downloaded_images/';
$maxImageNumber = 400;
$input='image0';
$input2='big';
$extension ='.png';
for($imageNumber = 100; $imageNumber <= $maxImageNumber; $imageNumber++) {
$fileData = file_get_contents($baseUrl . $input . $imageNumber . $input2 . $extension);
file_put_contents($localDirectory . $imageNumber, $fileData);
}
?>
The problem is when... 0001 0009 0010 !! I want to tell php "hey, always 4 digits"
I have a project that needs to create files using the fwrite in php. What I want to do is to make it generic, I want to make each file unique and dont overwrite on the others.
I am creating a project that will record the text from a php form and save it as html, so I want to output to have generated-file1.html and generated-file2.html, etc.. Thank you.
This will give you a count of the number of html files in a given directory
$filecount = count(glob("/Path/to/your/files/*.html"));
and then your new filename will be something like:
$generated_file_name = "generated-file".($filecount+1).".html";
and then fwrite using $generated_file_name
Although I've had to do a similar thing recently and used uniq instead. Like this:
$generated_file_name = md5(uniqid(mt_rand(), true)).".html";
I would suggest using the time as the first part of the filename (as that should then result in files being listed in chronological/alphabetic order, and then borrow from #TomcatExodus to improve the chances of the filename being unique (incase of two submissions being simultaneous).
<?php
$data = $_POST;
$md5 = md5( $data );
$time = time();
$filename_prefix = 'generated_file';
$filename_extn = 'htm';
$filename = $filename_prefix.'-'.$time.'-'.$md5.'.'.$filename_extn;
if( file_exists( $filename ) ){
# EXTREMELY UNLIKELY, unless two forms with the same content and at the same time are submitted
$filename = $filename_prefix.'-'.$time.'-'.$md5.'-'.uniqid().'.'.$filename_extn;
# IMPROBABLE that this will clash now...
}
if( file_exists( $filename ) ){
# Handle the Error Condition
}else{
file_put_contents( $filename , 'Whatever the File Content Should Be...' );
}
This would produce filenames like:
generated_file-1300080525-46ea0d5b246d2841744c26f72a86fc29.htm
generated_file-1300092315-5d350416626ab6bd2868aa84fe10f70c.htm
generated_file-1300109456-77eae508ae79df1ba5e2b2ada645e2ee.htm
If you want to make absolutely sure that you will not overwrite an existing file you could append a uniqid() to the filename. If you want it to be sequential you'll have to read existing files from your filesystem and calculate the next increment which can result in an IO overhead.
I'd go with the uniqid() method :)
If your implementation should result in unique form results every time (therefore unique files) you could hash form data into a filename, giving you unique paths, as well as the opportunity to quickly sort out duplicates;
// capture all posted form data into an array
// validate and sanitize as necessary
$data = $_POST;
// hash data for filename
$fname = md5(serialize($data));
$fpath = 'path/to/dir/' . $fname . '.html';
if(!file_exists($fpath)){
//write data to $fpath
}
Do something like this:
$i = 0;
while (file_exists("file-".$i.".html")) {
$i++;
}
$file = fopen("file-".$i.".html");
When a user uploads an image to my site, the image goes through this process;
user uploads pic
store pic metadata in db, giving the image a unique id
async image processing (thumbnail creation, cropping, etc)
all images are stored in the same uploads folder
So far the site is pretty small, and there are only ~200,000 images in the uploads directory. I realise I'm nowhere near the physical limit of files within a directory, but this approach clearly won't scale, so I was wondering if anyone had any advice on upload / storage strategies for handling large volumes of image uploads.
EDIT:
Creating username (or more specifically, userid) subfolders would seem to be a good solution. With a bit more digging, I've found some great info right here; How to store images in your filesystem
However, would this userid dir approach scale well if a CDN is bought into the equation?
I've answered a similar question before but I can't find it, maybe the OP deleted his question...
Anyway, Adams solution seems to be the best so far, yet it isn't bulletproof since images/c/cf/ (or any other dir/subdir pair) could still contain up to 16^30 unique hashes and at least 3 times more files if we count image extensions, a lot more than any regular file system can handle.
AFAIK, SourceForge.net also uses this system for project repositories, for instance the "fatfree" project would be placed at projects/f/fa/fatfree/, however I believe they limit project names to 8 chars.
I would store the image hash in the database along with a DATE / DATETIME / TIMESTAMP field indicating when the image was uploaded / processed and then place the image in a structure like this:
images/
2010/ - Year
04/ - Month
19/ - Day
231c2ee287d639adda1cdb44c189ae93.png - Image Hash
Or:
images/
2010/ - Year
0419/ - Month & Day (12 * 31 = 372)
231c2ee287d639adda1cdb44c189ae93.png - Image Hash
Besides being more descriptive, this structure is enough to host hundreds of thousands (depending on your file system limits) of images per day for several thousand years, this is the way Wordpress and others do it, and I think they got it right on this one.
Duplicated images could be easily queried on the database and you'd just have to create symlinks.
Of course, if this is not enough for you, you can always add more subdirs (hours, minutes, ...).
Personally I wouldn't use user IDs unless you don't have that info available in your database, because:
Disclosure of usernames in the URL
Usernames are volatile (you may be able to rename folders, but still...)
A user can hypothetically upload a large number of images
Serves no purpose (?)
Regarding the CDN I don't see any reason this scheme (or any other) wouldn't work...
MediaWiki generates the MD5 sum of the name of the uploaded file, and uses the first two letters of the MD5 (say, "c" and "f" of the sum "cf1e66b77918167a6b6b972c12b1c00d") to create this directory structure:
images/c/cf/Whatever_filename.png
You could also use the image ID for a predictable upper limit on the number of files per directory. Maybe take floor(image unique ID / 1000) to determine the parent directory, for 1000 images per directory.
Yes, yes I know this is an ancient topic. But the problem to store large amount of images and how the underlying folder structure should be organized. So I present my way to handle it in the hope this might help some people.
The idea using md5 hash is the best way to handle massive image storage. Keeping in mind that different values might have the same hash I strongly suggest to add also the user id or nicname to the path to make it unique. Yep that's all what's needed. If someone has different users with the same database id - well, there is something wrong ;) So root_path/md5_hash/user_id is everything you need to do it properly.
Using DATE / DATETIME / TIMESTAMP is not the optimal solution by the way IMO. You end up with big clusters of image folders on a buisy day and nearly empty ones on less frequented ones. Not sure this leads to performance problems but there is something like data aesthetics and a consistent data distribution is always superior.
So I clearly go for the hash solution.
I wrote the following function to make it easy to generate such hash based storage paths. Feel free to use it if you like it.
/**
* Generates directory path using $user_id md5 hash for massive image storing
* #author Hexodus
* #param string $user_id numeric user id
* #param string $user_root_raw root directory string
* #return null|string
*/
function getUserImagePath($user_id = null, $user_root_raw = "images/users", $padding_length = 16,
$split_length = 3, $hash_length = 12, $hide_leftover = true)
{
// our db user_id should be nummeric
if (!is_numeric($user_id))
return null;
// clean trailing slashes
$user_root_rtrim = rtrim( $user_root_raw, '/\\' );
$user_root_ltrim = ltrim( $user_root_rtrim, '/\\' );
$user_root = $user_root_ltrim;
$user_id_padded = str_pad($user_id, $padding_length, "0", STR_PAD_LEFT); //pad it with zeros
$user_hash = md5($user_id); // build md5 hash
$user_hash_partial = $hash_length >=1 && $hash_length < 32
? substr($user_hash, 0, $hash_length) : $user_hash;
$user_hash_leftover = $user_hash_partial <= 32 ? substr($user_hash, $hash_length, 32) : null;
$user_hash_splitted = str_split($user_hash_partial, $split_length); //split in chunks
$user_hash_imploded = implode($user_hash_splitted,"/"); //glue aray chunks with slashes
if ($hide_leftover || !$user_hash_leftover)
$user_image_path = "{$user_root}/{$user_hash_imploded}/{$user_id_padded}"; //build final path
else
$user_image_path = "{$user_root}/{$user_hash_imploded}/{$user_hash_leftover}/{$user_id_padded}"; //build final path plus leftover
return $user_image_path;
}
Function test calls:
$user_id = "1394";
$user_root = "images/users";
$user_hash = md5($user_id);
$path_sample_basic = getUserImagePath($user_id);
$path_sample_advanced = getUserImagePath($user_id, "images/users", 8, 4, 12, false);
echo "<pre>hash: {$user_hash}</pre>";
echo "<pre>basic:<br>{$path_sample_basic}</pre>";
echo "<pre>customized:<br>{$path_sample_advanced}</pre>";
echo "<br><br>";
The resulting output - colorized for your convenience ;):
Have you thought about using something like Amazon S3 to store the files? I run a photo hosting company and after quickly reaching limits on our own server, we switched over to AmazonS3. The beauty of S3 is that there are no limits like inodes and what not, you just keep throwing files at it.
Also: If you don't like S3, you can always try and break it down into subfolders as much as you can:
/userid/year/month/day/photoid.jpg
You can convert a username to md5 and set a folder from 2-3 first letters of md5 converted username for the avatars and for images you can convert and playing with time , random strings , ids and names
8648b8f3ce06a7cc57cf6fb931c91c55 - devcline
Also a first letter of the username or id for the next folder or inverse
It will look like
Structure:
stream/img/86/8b8f3ce06a7cc57cf6fb931c91c55.png //simplest
stream/img/d/2/0bbb630d63262dd66d2fdde8661a410075.png //first letter and id folders
stream/img/864/d/8b8f3ce06a7cc57cf6fb931c91c55.png // with first letter of the nick
stream/img/864/2/8b8f3ce06a7cc57cf6fb931c91c55.png //with unique id
stream/img/2864/8b8f3ce06a7cc57cf6fb931c91c55.png //with unique id in 3 letters
stream/img/864/2_8b8f3ce06a7cc57cf6fb931c91c55.png //with unique id in picture name
Code
$username = substr($username_md5, 1); // to cut first letter from the md5 converted nick
$username_first = $username[0]; // the first letter
$username_md5 = md5($username); // md5 for username
$randomname = uniqid($userid).md5(time()); //for generate a random name based on ID
you can try also with base64
$image_encode = strtr(base64_encode($imagename), '+/=', '-_,');
$image_decode = base64_decode(strtr($imagename, '-_,', '+/='));
Steam And dokuwiki use this structure.
You might consider the open source http://danga.com/mogilefs/ as it is perfect for what you're doing. It'll take you from thinking about folders to namespaces (which could be users) and let it store you images for you. The best part is you don't have to care how the data is stored. It makes it completely redundant and you can even set controls around how redundant thumbnails are as well.
I got soultion im using for a long time. It's quite old code, and can be further optimised, but it still serves good as it is.
It's a immutable function creating directory structure based on:
Number that identifies image (FILE ID):
it's recommended that this numer is unique for base directory, like primary key for database table, but it's not required.
The base directory
The maximum desired number of files and first level subdirectories. This promised can be kept only if every FILE ID is unique.
Example of usage:
Using explicitly FILE ID:
$fileName = 'my_image_05464hdfgf.jpg';
$fileId = 65347;
$baseDir = '/home/my_site/www/images/';
$baseURL = 'http://my_site.com/images/';
$clusteredDir = \DirCluster::getClusterDir( $fileId );
$targetDir = $baseDir . $clusteredDir;
$targetPath = $targetDir . $fileName;
$targetURL = $baseURL . $clusteredDir . $fileName;
Using file name, number = crc32( filename )
$fileName = 'my_image_05464hdfgf.jpg';
$baseDir = '/home/my_site/www/images/';
$baseURL = 'http://my_site.com/images/';
$clusteredDir = \DirCluster::getClusterDir( $fileName );
$targetDir = $baseDir . $clusteredDir;
$targetURL = $baseURL . $clusteredDir . $fileName;
Code:
class DirCluster {
/**
* #param mixed $fileId - numeric FILE ID or file name
* #param int $maxFiles - max files in one dir
* #param int $maxDirs - max 1st lvl subdirs in one dir
* #param boolean $createDirs - create dirs?
* #param string $path - base path used when creatign dirs
* #return boolean|string
*/
public static function getClusterDir($fileId, $maxFiles = 100, $maxDirs = 10,
$createDirs = false, $path = "") {
// Value for return
$rt = '';
// If $fileId is not numerci - lets create crc32
if (!is_numeric($fileId)) {
$fileId = crc32($fileId);
}
if ($fileId < 0) {
$fileId = abs($fileId);
}
if ($createDirs) {
if (!file_exists($path))
{
// Check out the rights - 0775 may be not the best for you
if (!mkdir($path, 0775)) {
return false;
}
#chmod($path, 0775);
}
}
if ( $fileId <= 0 || $fileId <= $maxFiles ) {
return $rt;
}
// Rest from dividing
$restId = $fileId%$maxFiles;
$formattedFileId = $fileId - $restId;
// How many directories is needed to place file
$howMuchDirs = $formattedFileId / $maxFiles;
while ($howMuchDirs > $maxDirs)
{
$r = $howMuchDirs%$maxDirs;
$howMuchDirs -= $r;
$howMuchDirs = $howMuchDirs/$maxDirs;
$rt .= $r . '/'; // DIRECTORY_SEPARATOR = /
if ($createDirs)
{
$prt = $path.$rt;
if (!file_exists($prt))
{
mkdir($prt);
#chmod($prt, 0775);
}
}
}
$rt .= $howMuchDirs-1;
if ($createDirs)
{
$prt = $path.$rt;
if (!file_exists($prt))
{
mkdir($prt);
#chmod($prt, 0775);
}
}
$rt .= '/'; // DIRECTORY_SEPARATOR
return $rt;
}
}