cakephp 3 lock file property - php

I have an application that create many php process to read file.
Sometimes process read the same file so I need to lock file when a process read It.
The application need to read only one times a file and after delete It.
Sometimes two process read the same file and this is a big problem.
I have create many process that every one loop through a folder, read all file, and delete it.
I don't want that two process read the same file, but the problem is that the process works simultaneously.
code of every process :
public function readAllXmlAndSave()
$xml_files = glob(TMP . 'xml/*.xml');
foreach($xml_files as $fileXml){
$explStr = explode('/', $fileXml);
$filename = $explStr[count($explStr) - 1];
$path = TMP . '/xml/' . $filename;
$file = new File($path, false);
if($file->exists()){
$string = $file->read();
//some work
$file->close();
$file->delete();
}
}
}
I have create a simple script to understand how lock file works into cakephp 3 but seems that property lock doesn't work as I aspected.
code
$path = TMP . '/xml/test.xml'; //inside there is "HELLO"
$file = new File($path, false);
$string = $file->read();
echo($string);
$file->lock = true;
$string2 = $file->read();
echo($string2);
$string3 = $file->read();
echo($string3);
I aspected that only the first echo work not the other.. Why I get always:
HELLO
HELLO
HELLO
and not only the first echo?
HELLO

Related

Generation of zip file in php is slow

I have a script which put images, floor plans and video into a zip file, it can reach 500mb easily but most of the time average is 150mb.
The generation of the zip file is extremely slow and i can't figurate why. Is there any tips to improve my script?
It took me 10 min to create the zip file in the server just for 100mb.
if( !empty( $files ) ){
$random_nbr = mt_rand(1,5646866662);
$path = 'webroot/img/tmp/' . $random_nbr;
if (!file_exists(\Cake\Core\Configure::read('pathTo') . 'webroot/img/tmp')) {
mkdir(\Cake\Core\Configure::read('pathTo') . 'webroot/img/tmp', 0777, true);
}
$destination = \Cake\Core\Configure::read('pathTo') . $path . '_media.zip';
$media_url = \Cake\Core\Configure::read('websiteUrl') . '/img/tmp/' . $random_nbr . '_media.zip';
$zip = new ZipArchive();
$zip->open( $destination, ZIPARCHIVE::CREATE | ZIPARCHIVE::OVERWRITE );
// Photos
if (isset($files['photos'])):
foreach( $files['photos'] as $f ){
$context = stream_context_create(array('http' => array('header'=>'Connection: close\r\n')));
// Original
$parsed_file = $f['original_file'];
$download_file = file_get_contents($parsed_file, false,$context);
$zip->addFromString('photos/original/' . basename($parsed_file), $download_file);
// Web with or without a watermark
$web = $this->Images->state_image(1270, $f['id'], 0, '');
$web = $web->response('jpg');
$zip->addFromString('photos/web/' . $f['name'], $web);
// High Res Web with or without a watermark
$web = $this->Images->state_image(2000, $f['id'], 0, '');
$web = $web->response('jpg');
$zip->addFromString('photos/high_res_web/' . $f['name'], $web);
}
endif;
// Floor Plan
if (isset($files['floorplan'])):
foreach( $files['floorplan'] as $f ){
$parsed_file = $f['original_file'];
$context = stream_context_create(array('http' => array('header'=>'Connection: close\r\n')));
$download_file = file_get_contents($parsed_file, false,$context);
$zip->addFromString('floorplan/' . basename($parsed_file), $download_file);
}
endif;
// Video
if (isset($files['video'])):
foreach( $files['video'] as $f ){
$parsed_file = $f['original_file'];
$context = stream_context_create(array('http' => array('header'=>'Connection: close\r\n')));
$download_file = file_get_contents($parsed_file, false,$context);
$zip->addFromString('floorplan/' . basename($parsed_file), $download_file);
}
endif;
$zip->close();
echo $media_url;
die();
}
Outside of dedicated hardware, there probably is not much you'll be able to do to speed up the actual zipping process. You could try exec()'ing the system zip utility rather than using PHP to do it, but that may not change things.
What you can do though (if the host allows it) is background the process and provide a status page so users can see how long until their file is ready. I've done this in the past for similar problems.
What I did was have a table in the database that would store information about the zip file to be created, and a list of all the files to be added to the zip file. Then I'd exec() off a background script with the ID of the newly created DB record.
The background process would read the DB for all the details and begin creating the zip file. Periodically it would update the DB with a % complete. When finished it'd update the DB with the file system path to the newly generated zip file.
Then I had another page for the end user that displayed a progress bar. The page would periodically make an Ajax request to the server to get the new % complete for the file and update the bar accordingly. When the file was complete it would change to a download link for them to begin downloading the file.
There was another cron job process that would periodically go through and delete all the temp files older than 5 days. If users needed the file again they had to have it re-generated.

How to save file in subfolder using php

I have a script with a mysql query which saves a file called invoice.xml every day automatically by running a cron job. In case no data is found a no_orders.txt is saved.
I would like this file not be saved to the same folder as the script.php file is in but to a subfolder called invoices.
The renaming of the old invoice.xml is done with the following code
// rename old file
$nowshort = date("Y-m-d");
if(file_exists('invoice.xml')) {
rename('invoice.xml','invoice_'.$nowshort.'.xml');
}
The saving is done with the following code:
if($xml1 !='') {
$File = "invoice.xml";
$Handle = fopen($File, 'w');
fwrite($Handle, $xml1);
print "Data Written - ".$nowMysql;
fclose($Handle);
#print $xml;
die();
} else {
print "No new orders - ".$nowMysql;
$File = "no_orders_".$nowshort.".txt";
$Handle = fopen($File, 'w');
fclose($Handle);
die();
}
Could I please get assistance how to save this file to a subfolder. Also the renaming of the existing file would need to be within the subfolder then. I have already tried with possibilities like ../invoice/invoice.xml but unfortunately without any success.
Thank you
Just give the path of file 'invoice.xml' to $File.
Otherwise create some $Dir object which will point to Folder named 'invoice', then use accordingly
Use __DIR__ magic constant to retrieve your script.php directory, then you can append /invoice/invoice.xml .
Example if path to your script php something like this:
/var/www/path/to/script.php
$currentDir = __DIR__; //this wil return /var/www/path/to
$invoicePath = $currentDir.'/invoice/invoice.xml';

On creating zip file by php I get two files instead of one

I'm struggling around with a simple PHP functionality: Creating a ZIP Archive with some files in.
The problem is, it does not create only one file called filename.zip but two files called filename.zip.a07600 and filename.zip.b07600. Pls. see the following screenshot:
The two files are perfect in size and I even can rename each of them to filename.zip and extract it without any problems.
Can anybody tell me what is going wrong???
function zipFilesAndDownload_Defect($archive_file_name, $archiveDir, $file_path = array(), $files_array = array()) {
// Archive File Name
$archive_file = $archiveDir."/".$archive_file_name;
// Time-to-live
$archiveTTL = 86400; // 1 day
// Delete old zip file
#unlink($archive_file);
// Create the object
$zip = new ZipArchive();
// Create the file and throw the error if unsuccessful
if ($zip->open($archive_file, ZIPARCHIVE::CREATE) !== TRUE) {
$response->res = "Cannot open '$archive_file'";
return $response;
}
// Add each file of $file_name array to archive
$i = 0;
foreach($files_array as $value){
$expl = explode("/", $value);
$file = $expl[(count($expl)-1)];
$path_file = $file_path[$i] . "/" . $file;
$size = round((filesize ($path_file) / 1024), 0);
if(file_exists($path_file)){
$zip->addFile($path_file, $file);
}
$i++;
}
$zip->close();
// Then send the headers to redirect to the ZIP file
header("HTTP/1.1 303 See Other"); // 303 is technically correct for this type of redirect
header("Location: $archive_file");
exit;
}
The code which calls the function is a file with a switch-case... it is called itself by an ajax-call:
case "zdl":
$files_array = array();
$file_path = array();
foreach ($dbh->query("select GUID, DIRECTORY, BASENAME, ELEMENTID from SMDMS where ELEMENTID = ".$osguid." and PROJECTID = ".$osproject.";") as $subrow) {
$archive_file_name = $subrow['ELEMENTID'].".zip";
$archiveDir = "../".$subrow['DIRECTORY'];
$files_array[] = $archiveDir.DIR_SEPARATOR.$subrow['BASENAME'];
$file_path[] = $archiveDir;
}
zipFilesAndDownload_Defect($archive_file_name, $archiveDir, $file_path, $files_array);
break;
One more code... I tried to rename the latest 123456.zip.a01234 file to 123456.zip and then unlink the old 123456.zip.a01234 (and all prior added .a01234 files) with this function:
function zip_file_exists($pathfile){
$arr = array();
$dir = dirname($pathfile);
$renamed = 0;
foreach(glob($pathfile.'.*') as $file) {
$path_parts = pathinfo($file);
$dirname = $path_parts['dirname'];
$basename = $path_parts['basename'];
$extension = $path_parts['extension'];
$filename = $path_parts['filename'];
if($renamed == 0){
$old_name = $file;
$new_name = str_replace(".".$extension, "", $file);
#copy($old_name, $new_name);
#unlink($old_name);
$renamed = 1;
//file_put_contents($dir."/test.txt", "old_name: ".$old_name." - new_name: ".$new_name." - dirname: ".$dirname." - basename: ".$basename." - extension: ".$extension." - filename: ".$filename." - test: ".$test);
}else{
#unlink($file);
}
}
}
In short: copy works, rename didn't work and "unlink"-doesn't work at all... I'm out of ideas now... :(
ONE MORE TRY: I placed the output of $zip->getStatusString() in a variable and wrote it to a log file... the log entry it produced is: Renaming temporary file failed: No such file or directory.
But as you can see in the graphic above the file 43051221.zip.a07200 is located in the directory where the zip-lib opens it temporarily.
Thank you in advance for your help!
So, after struggling around for days... It was so simple:
Actually I work ONLY on *nix Servers so in my scripts I created the folders dynamically with 0777 Perms. I didn't know that IIS doesn't accept this permissions format at all!
So I remoted to the server, right clicked on the folder Documents (the hierarchically most upper folder of all dynamically added files and folders) and gave full control to all users I found.
Now it works perfect!!! The only thing that would be interesting now is: is this dangerous of any reason???
Thanks for your good will answers...
My suspicion is that your script is hitting the PHP script timeout. PHP zip creates a temporary file to zip in to where the filename is yourfilename.zip.some_random_number. This file is renamed to yourfilename.zip when the zip file is closed. If the script times out it will probably just get left there.
Try reducing the number of files to zip, or increasing the script timeout with set_time_limit()
http://php.net/manual/en/function.set-time-limit.php

Check if file exists in .tar using PHP

In my program I need to read .png files from a .tar file.
I am using pear Archive_Tar class (http://pear.php.net/package/Archive_Tar/redirected)
Everything is fine if the file im looking for exists, but if it is not in the .tar file then the function timouts after 30 seconds. In the class documentation it states that it should return null if it does not find the file...
$tar = new Archive_Tar('path/to/mytar.tar');
$filePath = 'path/to/my/image/image.png';
$file = $tar->extractInString($filePath); // This works fine if the $filePath is correct
// if the path to the file does not exists
// the script will timeout after 30 seconds
var_dump($file);
return;
Any suggestions on solving this or any other library that I could use to solve my problem?
The listContent method will return an array of all files (and other information about them) present in the specified archive. So if you check if the file you wish to extract is present in that array first, you can avoid the delay that you are experiencing.
The below code isn't optimised - for multiple calls to extract different files for example the $files array should only be populated once - but is a good way forward.
include "Archive/Tar.php";
$tar = new Archive_Tar('mytar.tar');
$filePath = 'path/to/my/image/image.png';
$contents = $tar->listContent();
$files = array();
foreach ($contents as $entry) {
$files[] = $entry['filename'];
}
$exists = in_array($filePath, $files);
if ($exists) {
$fileContent = $tar->extractInString($filePath);
var_dump($fileContent);
} else {
echo "File $filePath does not exist in archive.\n";
}

PHP, delete path from TXT file

I have, problem.. I display images from dir in ARRAY with button 'delete' - action delete.php..
If I click 'delete' file delete.php should delete image from dir and path from TXT file..
Below PHP code delete only file from dir, I don't know how I can delete PATH from TXT files - I need this script..
TXT file looking that:
../../gallery/glowna//thumb_1300625269.jpg|
../../gallery/glowna//thumb_1300625300.jpg|
../../gallery/glowna/thumb_1300626725.jpg
And delete.php
<?php
$plik=$_POST['usun'];
$nowa = substr($plik, 6, 20);
unlink('../../gallery/glowna/'.$_POST['usun']);
unlink('../../gallery/glowna/'.$nowa);
header("location:usun.php");
?>
I trying use below code, but something is wrong, because TXT file are cleaning ALL:
$txt = "../../dynamic_ajax.txt";
$img = "../../gallery/glowna/".$_POST['usun'];
$file = file_get_contents($txt, true);
$file2 = explode('|', $file);
$search=array_search($img, $file2);
unset($search);
$separator = implode("|", $file2);
file_put_contents($txt, $separator);
Ok think I understand what you mean. This is something I jotted down, you might want to clean up the code a bit.
$q = 'thumb_1300625300.jpg';
$files = file_get_contents('files.txt');
$arr = explode('|', $files);
foreach ($arr as &$file) {
if (strpos($file, $q) !== false) {
$file = '';
break;
}
}
$files = implode('|', $arr);
$files = str_ireplace('||', '|', $files);
file_put_contents('files.txt', $files);
Pretty simple code.
Opens up the file and splits it by |
Then it loops through the arrray looking for the path that matches the image and makes it empty and then skips the loop
Then you implode the string and then remove the double | because we removed an element
A couple of caveats. This script only looks for one instance of the path. If you have multiple, then let the loop run its course and remove the break. You also need to modify str_ireplace('||', '|', $files); so that it will look for multiple |
What about this?
$file = file_get_contents($txt, true);
$file2 = explode('|', $file);
$new_array = Array();
foreach ($file2 as $path) {
if (/* path should be preserved */) {
$new_array[] = $path;
}
}
$new_contents = implode("|", $new_array);
file_put_contents($txt, $new_contents);
But be aware that a little while after you put this on a public server, your TXT file will be gone. Imagine this:
1st process (thread) opens the file for writing (truncating it to 0 characters).
2nd process reads the empty file.
1st p. writes good file.
2nd process writes empty file.
You could get around that by using some lock mechanism, but consider other options. If you have only paths in that file, why not having a special folder for your images? Then just list that folder and you know which files are present. If you want to save some metadata with the images, database is your friend.

Categories