I have a 100.000 rows txt file and I need to read it in order to insert most part of it into my DB.
I'd like to use this plugin, as I found it very easy to use:
http://www.bram.us/projects/js_bramus/jsprogressbarhandler/#download
My problem is: I read the txt file with PHP, but I don't understant how to update the progress bar!
I was thinking something like this
echo '$("#progressbar").progressbar({ value: '.($k++).' });';
where $k goes from 0 to 100, but, WHERE do I have to put it??
I've found this method:
http://spidgorny.blogspot.it/2012/02/progress-bar-for-lengthy-php-process.html
I think that this could help with some editing to the code.
You can't possibly mix the php and the javascript:
The PHP will run and generate an HTML/JS file
The HTML/JS file will be sent to the client
The client will run the JS: $("#progressbar").progressbar({ value: XX });
So k will be static.
--
If you really want to do something like that easily, you could use an intermediary DB table with three columns: txt_file, position, length
And often update this table while the PHP script is running over a txt file.
Client side, in Javascript, you can make an ajax request using jQuery for example every 5 or 10seconds, which is going to call an other PHP page, and this PHP page will only return the corresponding row from the intermediary table. Once you have the result you can update the progressbar.
--
It's the simplest solution to implement for you, but it's still really dirty, and the parsing of the txt file better have to be really long !
There is no direct method to achieve this thing. PHP script executes first and then the output is sent to the client viewing the web page that is why you cannot show live status of your PHP script's processing to the client.
You will have to use a combination of AJAX and Database :
Create a table to track the progress of the loading of the text file. Whenever a user (client) sends a request to the page, keep on updating the table with the progress. Use Session id as the index on the table so that it would be easy to track the progress per client. Now use AJAX requests to get the progress from the table and present it to the client with your progress bar.
Related
I'm working on an app that creates QR codes and renders them onto multiple graphics for a user.
The Problem:
I wrote a script to import users to create from a CSV. I'm needing to create over 100 users (each including the process above). Right now it takes roughly 1 minute to complete for each new user to complete the processing.. then spits out all my error/success messages at once.
My Question:
Rather than the browser slowly loading the result view (currently stays on a white page until complete) as my script is processing, is their a somewhat easy way to display the live progress and errors as they happen? Something like a progress bar updated as each user is created/fails. I'm guessing it will require AJAX?
When dealing with websites, remember the golden rule.
PHP MUST DIE.
Noobs assume this is people rubbishing PHP. It isn't. It's the HTTP request cycle.
Request In > PHP > Response Out > PHP process dies.
This is only the case when dealing with web servers and browsers, not CLI PHP. But the point is that you may end up getting Apache timeouts if your script takes as long as you say.
One solution could be to set up a cron that checks for a file and if it finds it, processes it, dumping a line number in a text file that your browser could check, which means you could fetch progress:
<?php
if (file_exists('/some/csv/to/process.csv')) {
// open file
// get row to work on
// process row
// update progress file with next line number
}
Meanwhile, you could set up a script that does this:
<?php
$progress = file_get_contents('/path/to/progress.txt');
header('Content-Type: application/json');
echo json_encode(['progress' => $progress]);
And then get the progress using AJAX inside a setInterval function:
$.get('/path/to/progress/json/page', function(data){
console.log(data);
});
Just an idea, may or may not suit you but give it a try!
I have implemented a SAAS scenario with my Windows server: a user could upload a file in a website, then Fetch.exe is an application coded in C# hosted in the server, Fetch.exe takes the uploaded file as an input, executes and generates an output to download for the user.
So in my php, I use exec to wrap Fetch.exe:
exec("Fetch.exe " . $inputFile . " > " . $outputFile)
Uploading and executing (ie, Fetch.exe) may take more than several seconds, and I want to show the user that it is processing and everything is going fine.
I have found some threads that discuss how to show a progress bar for the uploading. Whereas, does anyone know what I could do to show the approximate progress of Fetch.exe? Do I have to split it into smaller applications and use several exec?
You could supply Fetch.exe with an randomly generated ID from php, for example from the uniqueid function. Fetch.exe will create a file called <uniqueid>.txt with a progress percentage. From the browser, you could call another script with that unique ID to get the contents of that .txt file. In order, it would be something like this:
User uploads the file to PHP
PHP:
handles the uploaded file
creates a uniqueID
starts Fetch.exe with the file and the uniqueID
returns a page with the uniqueID embedded
The following happens in parallel:
Fetch.exe creates a textfile called /progress/uniqueid.txt with the uniqueid as name. It logs the progress into it.
The browser does an AJAX call to http://example.com/progress/uniqueid.txt and shows the progress to the user
And finally, when the progress reaches 100% the browser downloads the file. The only thing you might want to add is the pruning of the progress files after a while. Say you delete all files older than 10 minutes every hour.
Your PHP program needs a way to know the state of the subprocess (the Fetch.exe application), so, Fetch.exe needs to send info about the processing state, the most natural way to do this is through the standard output (the standard output is the information that provides a program when you run it from cmd).
Knowing this, you can run and keep reading a subprocess output from php using popen().
And secod, you can use the PHP ob_flush() and flush() with the onmessage javascript event to establish the comunication from your client page with your running php script, here you can find a good tutorial on how do this.
Let me describe what I've made ar first:
I have to import large ammount of data from different xml's to my database and because it last a lot I had to put a progress bar and I did it like this: I split the whole import into tiny little AJAX requests and I import little data at a time (when an ajax request completes the progress bar increases a bit). This whole idea is great but the data just keeps getting bigger and bigger and I can't optimize the code anymore (it's as optimized as it gets).
The problem is that everytime I do a AJAX call I lose a lot of time with things specific to the framework (model initializations and stuff), with the browser handling the url and so on. So I was wondering if I could use the flush function from php.
But I've been reading that the flush function doesn't work great on all browsers (which is weird cause it's a server-side function). If I would use the flush function I would just write <script>increase_progressbar</script> or whatever I want and I could do it.
So, any opinions on the flush function? I've been testing it on little scripts but I want to know if someone really used it with big scripts. Also, I can listen to any other suggestion of doing what I want to do :)
I wont give you direct advise, but I will tell you how I did it in one of my projects. In my case I need to upload an Excel files and then parse them. The data exceeding 3000 rows and I had to check all columns of each row for some data. When I parse it directly after the upload, the parser often crashes somewhere and it was really not safe.
So how I did it? The upload process has been split in 2 parts:
Upload physically the file (regular upload field and submit). When the button is clicked some CSS and JS "magic" hide the form and one nice loading bar appears on the screen. When the upload has been done the page just refreshes and the form appear again for the next file
Start parsing the data on the background using php-cli as #Dragon suggest with exec().
In the database I had a table which stores information about the files and there is a boolean field called "parsed". When the parser finishes the job, the last task is to update that field to true.
So here is the whole process from user point of view:
select a file and upload it.
wait until the file has been uploaded on the server. Till then a message and loading bar appear indicating that something is working. The upload form has been hidden with CSS and JS, so preventing user to upload another file.
When it's over the page has been refreshed (because I did normal _POST submit) the form appear on the screen again as well as a list of recently uploaded files (this I've stored this in the session).
In each of the nodes of that list I had an indicator (an icon). In the first time it's a spinner (ajax spinning wheel).
On a regular basis (30 sec or 1 min) I've checked the file table through Ajax call and reading the parsed field. If the background process has been over, the field was set to true and with some JS and CSS I've changed the icon to "Done". Otherwise the spinner remain.
In my project I doesn't have requirement to show extra details about the imports, but you can always go wild with other extra data.
Hope this help you with your project.
How can you bring in a path as a variable from php? I have hundreds of mp3 file names stored in a My SQL database and I want to call the name, load it into a variable and then have it replace the URL in the call to the sound file. I am all set with the PHP and database side of things, but I have been frustrated with the Flash part. I think it would just involve passing a variable from php to flash. The web pages we are building would eventually have 10 - 15 files per page and each one would have a different sound file and a different image that you could click to trigger the sound. First click would start the file and the second would stop the sound. The images and sound files are all stored in the database.
I found this code on another post and it is basically what I want to do. Still missing the button part, but if I can figure out the variable from PHP I think it will open up a bunch of new options.
var soundRequest:URLRequest = "path/to/file.mp3"; //the path would be a variable passed from the database to php and then to the actionscript
var s:Sound = new Sound(soundRequest);
var sChannel = s.play(0, int.MAX_VALUE); //Causes it to repeat by the highest possible number to flash.
//Above starts the sound immediatly (Streaming);
//Now to wait for completion instead, pretend we didnt start it before.
s.addEventLister(Event.SOUND_COMPLETE, onSComplete, false, 0, true);
function onSComplete(e:Event):void
{
var sChannel = s.play(0, int.MAX_VALUE); //Causes it to repeat by the highest possible
}
If you have a click to trigger on the page then you should use Javascript to Flash communication on the page. One flash file and a communication to flash which file to play. With anything flash these days you've probably heard of SWFobject and with Javascript you've probably heard of jQuery. Well what you need is very well documented in the jQuery SWFObject
plugin.
There is a standalone example here.
http://jquery.thewikies.com/swfobject/example_flashInteract.html
It would make sense to send all the file names from PHP to AS3 and store them in an Array, after that everything can be handled on the client side.
You need to check for Flash / PHP communication , there are loads of tutorials available on the net. Here's an example.
Output mySQL data as XML with PHP
The basic idea is to call a PHP script which should return your mp3 information as XML or JSON. I personally favor JSON but you will need to download a library.
as3 corelib
After the data has been retrieved , you can create ValueObjects
Populating Value Objects with web service XML
You should end up with an Array of ValueObjects which can then assign to your various images.
You could create a specific class for each image that would take as parameter a ValueObject.
For more info on AS3 , go here
Is there a way to make a progress bar (using ajax and jquery) for any script runnig in the server? I use a php script to elaborate some datas and I want to output a percent info to the client browser of the script progress. I tried to use session var but with poor results.
You can poll your script progress somehow (datas processed are marked with 1 datas that not are marked with 0) you can count the progress percentage.
Note: my tip was an outline only. I don't know how your script crunching your data and the data can or cannot be marked like i said above.
What kind of data processing happens in the background?