I have 1500 files that are named with an incorrectly dateformat. I would like to rename them. Are there a tool that can do that? Otherwise a piece of php code.
File names are:
ddmmyyyy.xls (e.g. 15012010 for 15.th Jan 2010)
and I would like:
yyyymmdd.xls (e.g. 20100115.xls)
Any clue on how this can be done for 1500 files in one go?
BR. Anders
UPDATE:
Also tried the MP3TAG, that is suggested in one of the answers. It is a free tool and also did the job. It took a while to figure out how to use it. If you wanne try do this:
add xls (or other format) to the list of editable files in configuration
choose folder to load files AND mark files in the pane you want to edit
I clicked the "Convert - Quick" button. It is also possible to save schemaes for future use but I could not figure out how.
after clicking "convert - quick" choose "using regex" (only regex option)
And then you just add the info to process the renaming. In my case:
field: _FILENAME
from: ([0-9]{2})([0-9]{2})([0-9]{4})
to: $3-$2-$1
Now all files named 15012010.xls (ddmmyyyy.xls) will be named 2010-01-15.xls
Here's a start (untested, but you should get the idea).
$files = glob('your/folder/*.xls');
foreach($files as $file) {
preg_match_all('/^(\d{2})(\d{2})(\d{4})\.xls$/', basename($file), $matches);
if ( ! $matches) continue;
$year = $matches[0][3];
$month = $matches[0][2];
$day = $matches[0][1];
$newFilename = $year . $month . $day . '.xls';
rename ( $file, dirname($file) . '/' . $newFilename );
}
If you have a Linux machine with the files... you can use bash to do:
for f in *.xls; do
mv $f "$(echo $f | cut -c4-8)$(echo $f | cut -c3,4)$(echo $f | cut -c1,2).xls"
done
A tool that can perform filename pattern conversion is Mp3tag.
Choose convert and then filename - filename.
I'm sure there's other tools out there too!
(This answer isn't really in the StackOverflow spirit but I think the OP isn't necessarily looking for an automated solution...)
Based on alex function, but this one correctly adds the .xls extension.
foreach (glob('/path/to/your/*.xls') as $file)
{
rename($file, dirname($file) . '/' . substr(basename($file), 4, 4) . substr(basename($file), 2, 2) . substr(basename($file), 0, 2) . '.xls');
}
if you have bash
#!/bin/bash
shopt -s nullglob
for xls in [0-9]*.xls
do
day=${xls:0:2}
mth=${xls:3:2}
yr=${xls:4:4}
echo mv "$xls" "${yr}${mth}${day}.xls"
done
no need external tools.
File names are: ddmmyyyy.xls (e.g.
15012010 for 15.th Jan 2010)
and I would like: yyyymmdd.xls (e.g.
20100115.xls)
Use this script.
# Script RenameYYYYMMDD.txt
var str dir, list, file, newname, dd, mm
lf -r -n "*.xls" $dir > $list
while ($list <> "")
do
lex "1" $list > $file ; stex -p "^/^l[" $file > $newname ; chex "2]" $newname > $dd
chex "2]" $newname > $mm ; sin "^.^l" ($mm+$dd) $newname > null
system rename ("\""+$file+"\"") $newname
done
This script is in biterscripting ( http://www.biterscripting.com ). Test the script first in a test folder.
To test, save the script code in file "C:/Scripts/RenameYYYYMMDD.txt", and enter the following command.
script "C:/Scripts/RenameYYYYMMDD.txt" dir("C:/path/to/test folder")
This command will rename all files ddmmyyyy.xls under directory "C:/path/to/test folder" to yyyymmdd.xls.
Related
I have a directory with 100 of xlxs files. Now what I want to do is to convert all these files into PDF all at one time or some at one time. The conversion process is working fine at the moment with foreach and cron. But it can process or convert files one at a time which increase waiting time at the user end who is waiting for PDF files.
I am thinking about parallel processing at this time but don't know how to implement this.
Here is my current code
$files = glob("/var/www/html/conversions/xlxs_files/*");
if(!empty($files)){
$now = time();
$i = 1;
foreach ($files as $file) {
if (is_file($file) && $i <= 8) {
echo $i.'-----'.basename($file).'----'.date('m/d/Y H:i:s',#filemtime($file));
echo '<br>';
$path_parts = pathinfo(basename($file));
$xlsx_file_name = basename($file);
$pdf_file_name = $path_parts['filename'].'.pdf';
echo '<br>';
try{
$result = ConvertApi::convert('pdf', ['File' => $common_path.'xlxs_files/'.$xlsx_file_name],'xlsx');
echo $log = 'conversion start for '.basename($file).' on '. date('d-M-Y h:i:s');
echo '<br>';
$result->getFile()->save($common_path.'pdf_files/'.$pdf_file_name);
echo $log = 'conversion start for '.basename($file).' on '. date('d-M-Y h:i:s');
echo '<br>';
mail('amit.webethics#gmail.com','test','test');
unlink($common_path.'xlxs_files/'.$xlsx_file_name);
}catch(Exception $e){
$log_file_data = createAlogFile();
$log = 'There is an error with your file .'. $xlsx_file_name.' -- '.$e->getMessage();
file_put_contents($log_file_data, $log . "\n", FILE_APPEND);
continue;
}
$i++;
}
}
}else{
echo 'nothing to process';
}
Any help will be highly appreciated. Thanks
Q : I am thinking about parallel processing at this time but don't know how to implement this.
Fact #1:this is not a kind of a true-[PARALLEL] orchestration of the flow of processing.
Fact #2:a standard GNU parallel (all details kindly read in man parallel) will help you maximise the performance of your processing pipeline, given the list of all files to convert and tweaking other parameters as the amounts of CPU/cores used and RAM-resources you may reserve/allocate to perform this batch conversion as fast as possible.
ls _files_to_convert.mask_ | parallel --jobs _nCores_ \
--load 99% \
--block _RAMblock_ \
... \
--dry-run \
_converting_process_
might serve as an immediate apetiser for what the GNU parallel is capable of.
All credits and thanks are to go to Ole Tange.
You could start multiple PHP scripts at a time. How to do that in detail answer is here: https://unix.stackexchange.com/a/216475/91593
I would go for this solution:
N=4
(
for thing in a b c d e f g; do
((i=i%N)); ((i++==0)) && wait
task "$thing" &
done
)
Another way is to try to use PHP for that. There is in depth answer to this question: https://stackoverflow.com/a/36440644/625521
I need to deploy a PHP application written by CodeIgniter to client's web server (CentOS 5 or 6). As PHP is the scripting language, it does not need to compile to binary code for deployment. It has chances that client will modify the PHP program by themselves without a notice to me. If client has modified the program that made the application out of order, we need to take extra man power to find their modification and fix it.
So I would like to made something that can easy to let me know any files (php, css, html, etc.) of the application has been modification after my deployment. Is there any method suggested by anyone?
Thank you,
Use filemtime()
int filemtime ( string $filename )
This PHP function returns the time when the data blocks of a file were being written to, that is, the time when the content of the file was changed.
<?php
// outputs e.g. somefile.txt was last modified: December 12 2014 09:16:23.
$filename = 'somefile.txt';
if (file_exists($filename)) {
echo "$filename was last modified: " . date ("F d Y H:i:s.", filemtime($filename));
}
?>
To get the last modification time of a directory, you can use this:
<pre>
$getLastModDir = filemtime("/path/to/directory/.");
</pre>
Take note on the last dot which is needed to see the directory as a file and to actually get a last modification date of it.
This comes in handy when you want just one 'last updated' message on the frontpage of your website and still taking all files of your website into account.
To get the modification date of some remote file, you can use the fine function by notepad at codewalker dot com (with improvements by dma05 at web dot de and madsen at lillesvin dot net).
But you can achieve the same result more easily now with stream_get_meta_data (PHP>4.3.0).
However a problem may arise if some redirection occurs. In such a case, the server HTTP response contains no Last-Modified header, but there is a Location header indicating where to find the file. The function below takes care of any redirections, even multiple redirections, so that you reach the real file of which you want the last modification date.
<?php
// get remote file last modification date (returns unix timestamp)
function GetRemoteLastModified( $uri )
{
// default
$unixtime = 0;
$fp = fopen( $uri, "r" );
if( !$fp ) {return;}
$MetaData = stream_get_meta_data( $fp );
foreach( $MetaData['wrapper_data'] as $response )
{
// case: redirection
if( substr( strtolower($response), 0, 10 ) == 'location: ' )
{
$newUri = substr( $response, 10 );
fclose( $fp );
return GetRemoteLastModified( $newUri );
}
// case: last-modified
elseif( substr( strtolower($response), 0, 15 ) == 'last-modified: ' )
{
$unixtime = strtotime( substr($response, 15) );
break;
}
}
fclose( $fp );
return $unixtime;
}
?>
I'm working on a command-line PHP project and want to be able to recreate the PHAR file that is my deployment artifact. The challenge is that I can't create two PHAR's that have identical sha1sums and were created more than 1 second apart from each other. I would like to be able to exactly recreate my PHAR file if the input files are the same (i.e. came from the same git commit).
The following code snippet demonstrates the problem:
#!/usr/bin/php
<?php
$hashes = array();
$file_names = array('file1.phar','file2.phar');
foreach ($file_names as $name) {
if (file_exists($name)) {
unlink($name);
}
$phar = new Phar($name);
$phar->addFromString('cli.php', "cli\n");
$hashes[]=sha1_file($name);
// remove the sleep and the PHAR's are identical.
sleep(1);
}
if ($hashes[0]==$hashes[1]) {
echo "match\n";
} else {
echo "do not match\n";
}
As far as I can tell, the "modification time" field for each file in the PHAR manifest is always set to the current time, and there seems to be no way or overriding that. Even touch("phar://file1.phar/cli.php", 1413387555) gives the error:
touch(): Can not call touch() for a non-standard stream
I ran the above code in PHP 5.5.9 on ubuntu trusty and PHP 5.3 on RHEL5 and both versions behave the same way and fail to create identical PHAR files.
I'm trying to do this in order to follow the advice in the book Continuous Deployment by Jez Humble and David Farley
Any help is appreciated.
The Phar class currently does not allow users to alter or even access the modifiction time. I thought of storing your string into a temporary file and using touch to alter the mtime, but that does not seem to have any effect. So you'll have to manually change the timestamps in the created files and then regenerate the archive signature. Here's how to do it with current PHP versions:
<?php
$filename = "file1.phar";
$archive = file_get_contents($filename);
# Search for the start of the archive header
# See http://php.net/manual/de/phar.fileformat.phar.php
# This isn't the only valid way to write a PHAR archive, but it is what the Phar class
# currently does, so you should be fine (The docs say that the end-of-PHP-tag is optional)
$magic = "__HALT_COMPILER(); ?" . ">";
$end_of_code = strpos($archive, $magic) + strlen($magic);
$data_pos = $end_of_code;
# Skip that header
$data = unpack("Vmanifest_length/Vnumber_of_files/vapi_version/Vglobal_flags/Valias_length", substr($archive, $end_of_code, 18));
$data_pos += 18 + $data["alias_length"];
$metadata = unpack("Vlength", substr($archive, $data_pos, 4));
$data_pos += 4 + $metadata["length"];
for($i=0; $i<$data["number_of_files"]; $i++) {
# Now $data_pos points to the first file
# Files are explained here: http://php.net/manual/de/phar.fileformat.manifestfile.php
$filename_data = unpack("Vfilename_length", substr($archive, $data_pos, 4));
$data_pos += 4 + $filename_data["filename_length"];
$file_data = unpack("Vuncompressed_size/Vtimestamp/Vcompressed_size/VCRC32/Vflags/Vmetadata_length", substr($archive, $data_pos, 24));
# Change the timestamp to zeros (You can also use some other time here using pack("V", time()) instead of the zeros)
$archive = substr($archive, 0, $data_pos + 4) . "\0\0\0\0" . substr($archive, $data_pos + 8);
# Skip to the next file (it's _all_ the headers first, then file data)
$data_pos += 24 + $file_data["metadata_length"];
}
# Regenerate the file's signature
$sig_data = unpack("Vsigflags/C4magic", substr($archive, strlen($archive) - 8));
if($sig_data["magic1"] == ord("G") && $sig_data["magic2"] == ord("B") && $sig_data["magic3"] == ord("M") && $sig_data["magic4"] == ord("B")) {
if($sig_data["sigflags"] == 1) {
# MD5
$sig_pos = strlen($archive) - 8 - 16;
$archive = substr($archive, 0, $sig_pos) . pack("H32", md5(substr($archive, 0, $sig_pos))) . substr($archive, $sig_pos + 16);
}
else {
# SHA1
$sig_pos = strlen($archive) - 8 - 20;
$archive = substr($archive, 0, $sig_pos) . pack("H40", sha1(substr($archive, 0, $sig_pos))) . substr($archive, $sig_pos + 20);
}
# Note: The manual talks about SHA256/SHA512 support, but the according flags aren't documented yet. Currently,
# PHAR uses SHA1 by default, so there's nothing to worry about. You still might have to add those sometime.
}
file_put_contents($filename, $archive);
I've written this ad-hoc for my local PHP 5.5.9 version and your example above. The script will work for files created similar to your example code from above. The documentation hints to some valid deviations from this format. There are comments at the according lines in the code; you might have to add something there if you want to support general Phar files.
Allright, I have a overview of sites where I was/am working on. And for every site I have a php file. And in that php file I use this code to get the newest and oldest date of a file what is note a picture in the directory.
$date = "test/*.*";
$files = array_filter(glob($date), function($file) {
$ext = substr($file, strrpos($file, '.'));
return !in_array($ext, array('.jpg', '.bit', '.png', '.jpeg', '.gif', '.bmp'));
});
$latest = count($files)-1 ;
array_multisort(
array_map( 'filemtime', $files ),
SORT_NUMERIC,
SORT_ASC,
$files
);
$newestfile = date ("d F Y ", filemtime($files[0]));
$oldestfile = date ("d F Y ", filemtime($files[$latest]));
if($newestfile == $oldestfile) {
echo date ("d F Y ", filemtime($files[0]));
} else {
echo date ("d F Y ", filemtime($files[0]));
echo " - " ;
echo date ("d F Y .", filemtime($files[$latest]));
}
the output of this code would be like: 16 January 2013 - 25 October 2013 .
In my overview page I use a code to include all the php files (of the websites I've made) to that page. (btw. the php files are not big pages. just with a picture and a bit of text.)
$listy = glob("sites/*.php");
print_r ($listy) ;
array_multisort(
array_map( 'filemtime', $listy ),
SORT_NUMERIC,
SORT_DESC,
$listy
);
if (empty($listy)) {
include('includes/emptycontent.php');
} else {
foreach ($listy as $filename) {
include $filename;
}
}
the output of this array would be like:
Array (
[0] => sites/test.php
[1] => sites/test2.php
[2] => sites/test3.php
So far so good, no problems in that.
Now, I want to sort the included files not on time of the included file, like I did in the code above. But i want it to sort on the latest date of the files in the directory like in the php file. so actually I want to combine those to codes to one.
so I have a php file called test. and I want the date of the latest file in the directory also called test. Those have always the same name.
What I thought was to use the output of the second code and then get rid of the "sites/" and the ".php". Those names must be in a array I think. and then for each name get the newestfile and sort them from newest to oldest.
I think like this I get the sites on which I was working recently at the top and the older ones at the bottom of the page.
Maybe my approach is totally wrong but I have no idea how to do that in code.
Look at this bit:
array_map( 'filemtime', $listy ),
Here, you’re effectively converting a list of files to a list of modification dates. How about converting that into another function. One that finds a most recent file inside a directory:
array_map(function ($filename) {
// $filename = sites/test1.php
$dir = substr($filename, strlen("sites/"), - strlen(".php")); // cut those!
// or:
$dir = basename($filename, '.php');
// put the code listing files inside $dir
// then sort it (you did it in the first part)
// and then `return` the most or the least recent one
}, $listy),
HTH.
Which is faster between glob() and opendir(), for reading around 1-2K file(s)?
http://code2design.com/forums/glob_vs_opendir
Obviously opendir() should be (and is) quicker as it opens the directory handler and lets you iterate. Because glob() has to parse the first argument it's going to take some more time (plus glob handles recursive directories so it'll scan subdirs, which will add to the execution time.
glob and opendir do different things. glob finds pathnames matching a pattern and returns these in an array, while opendir returns a directory handle only. To get the same results as with glob you have to call additional functions, which you have to take into account when benchmarking, especially if this includes pattern matching.
Bill Karwin has written an article about this recently. See:
http://www.phparch.com/2010/04/28/putting-glob-to-the-test/
Not sure whether that is perfect comparison but glob() allows you to incorporate the shell-like patterns as well where as opendir is directly there for the directories there by making it faster.
another question that can be answered with a bit of testing. i had a convenient folder with 412 things in it, but the results shouldn't vary much, i imagine:
igor47#whisker ~/test $ ls /media/music | wc -l
412
igor47#whisker ~/test $ time php opendir.php
414 files total
real 0m0.023s
user 0m0.000s
sys 0m0.020s
igor47#whisker ~/test $ time php glob.php
411 files total
real 0m0.023s
user 0m0.010s
sys 0m0.010s
Okay,
Long story short:
if you want full filenames+paths, sorted, glob is practically unbeatable.
if you want full filenames+paths unsorted, use glob with GLOB_NOSORT.
if you want only the names, and no sorting, use opendir + loop.
That's it.
Some more thoughts:
You can do tests to compose the exact same result with different methods only to find they have approximately the same time cost. Merely for fetching the information you'll have no real winner. However, consider these:
Dealing with a huge file list, glob will sort faster - it uses the filesystem's sort method which will always be superior. (It knows what it sorts while PHP doesn't, PHP sorts a hashed array of arbitrary strings, it's simply not fair to compare them.)
You'll probably want to filter your list by some extensions or filename masks for which glob is really efficient. You have fnmatch() of course, but calling it every time will never be faster than a system-level filter trained for this very job.
On the other hand, glob returns a significantly bigger amount of text (each name with full path) so with a lot of files you may run into memory allocation limits. For a zillion files, glob is not your friend.
OpenDir is more Faster...
<?php
$path = "/var/Upload/gallery/TEST/";
$filenm = "IMG20200706075415";
function microtime_float()
{
list($usec, $sec) = explode(" ", microtime());
return ((float)$usec + (float)$sec);
}
echo "<br> <i>T1:</i>".$t1 = microtime_float();
echo "<br><br> <b><i>Glob :</i></b>";
foreach( glob($path.$filenm.".*") as $file )
{
echo "<br>".$file;
}
echo "<br> <i>T2:</i> ".$t2 = microtime_float();
echo "<br><br> <b><i>OpenDir :</b></i>";
function resolve($name)
{
// reads informations over the path
$info = pathinfo($name);
if (!empty($info['extension']))
{
// if the file already contains an extension returns it
return $name;
}
$filename = $info['filename'];
$len = strlen($filename);
// open the folder
$dh = opendir($info['dirname']);
if (!$dh)
{
return false;
}
// scan each file in the folder
while (($file = readdir($dh)) !== false)
{
if (strncmp($file, $filename, $len) === 0)
{
if (strlen($name) > $len)
{
// if name contains a directory part
$name = substr($name, 0, strlen($name) - $len) . $file;
}
else
{
// if the name is at the path root
$name = $file;
}
closedir($dh);
return $name;
}
}
// file not found
closedir($dh);
return false;
}
$file = resolve($path.$filenm);
echo "<br>".$file;
echo "<br> <i>T3:</i> ".$t3 = microtime_float();
echo "<br><br> <b>glob time:</b> ". $gt= ($t2 - $t1) ."<br><b>opendir time:</b>". $ot = ($t3 - $t2) ;
echo "<u>". (( $ot < $gt ) ? "<br><br>OpenDir is ".($gt-$ot)." more Faster" : "<br><br>Glob is ".($ot-$gt)." moreFaster ") . "</u>";
?>
Output:
T1:1620133029.7558
Glob :
/var/Upload/gallery/TEST/IMG20200706075415.jpg
T2: 1620133029.7929
OpenDir :
/var/Upload/gallery/TEST/IMG20200706075415.jpg
T3: 1620133029.793
glob time:0.037137985229492
opendir time:5.9843063354492E-5
OpenDir is 0.037078142166138 more Faster