I have a php script which processes XML file uploaded by user . When file is big (like 50MB) it can take several minutes. On small files it works how it's expected. But I faced with one problem with large ones.
So, my file looks like:
if(file_exist($filename)) {
return array(false, "File with this name already exist");
} else {
/*
Processing of file
*/
return array(true, "")
}
When user upload file with javascript uploader, I make ajax request to this script. And the thing is that for large file it execute twice. For the first time it go to the "processing" section, at the second time it returns error that file with this name already exist (which is true, actually).
I add logging function in the beginning of the script
$logger = Zend_Registry::get('logger');
$logger->log('assign file function called', 7);
and I see that script has been called twice.
But in Firebug I see only 1 ajax request to the script. In Apache access.log I see only one request. Apache error.log is empty.
Any idea what it can be? Probably any configuration for long-time-executed scripts?
UPD Javascript for calling script
$("#save_file_btn").click(function(){
var filename = $("input#selected_file").val();
if(!filename.length) {
customAlert("Choose file at first");
return false;
}
$.ajax({
url: '/otms/publisher/assign-file-to-publisher',
beforeSend: function(xhr){
$("#add_form").html('<div class="loader"></div>');
},
dataType: 'json',
data: {filename: filename},
success: function(data) {
if(data.success) {
//Process added ONIX file
processOnixFile(data.fileId, function(){
getFileList();
$("#add_form").html('<div class="text-center">File has been successfully uploaded</div>');
setTimeout('$("#add_file_modal").modal("hide")', 2500);
});
} else {
var output = '';
$.each(data.error, function(index,value){
output += '<div class="bold">'+value.title+'</div>';
output += '<ul>';
$.each(value.errorList, function(i, errorMsg){
output += '<li>'+errorMsg+'</li>';
});
output += '</ul>';
});
$("#add_form").html('<div class="red">'+output+'</div>');
}
}
});
});
I checked and this is the only place were any request is made to script 'assign-file-to-publisher' (where I have problem).
UPD 2.
What I found out is that script called twice only in Firefox browser. In Safari request doesn't return anything. In Chrome it returns the following status:
(failed) net::ERR_EMPTY_RESPONSE
I still do not see any error messages in my application logs or apache logs.
As I mentioned script works with big file and it can take up to several minutes to process it.
In my configuration:
max_execution_time = 300
while Chrome return error on 1:40 time.
Related
The task is to implement the following:
1. On the page there is a form for downloading a file, the file should be loaded asynchronously, without rebooting with the help of AJAX and displaying the upload progress
2. If the file is successfully downloaded, send a command to the server, which should check the queue, and if the queue is empty, inform the client that the processing of the file has begun, then start the processing itself
3. If processing has begun, then show the progress of this processing
What did you manage to implement
File upload was successfully implemented, after a successful upload, the server issues a line with the status successfully, and the full path of the downloaded file
Upload JS snippet
$.ajax({
url: 'api/upload/load',
type: 'post',
data: fd,
contentType: false,
processData: false,
xhr: function() {
var xhr = new window.XMLHttpRequest();
xhr.upload.addEventListener("progress", function(evt) {
if (evt.lengthComputable) {
var percent = Math.ceil((evt.loaded / evt.total) * 100);
console.log(percent) // Upload progress
}
}, false);
return xhr;
},
success: function(data){
var response = JSON.parse(data);
if(response.status == 'success'){
//if upload success, run command
startConversion(response.path)
}
}
});
Reponse from api/upload/load
{"status":"success","path":"<path>"}
In the AJAX script, I do a check: if the response.status == 'success', then we call the startConversion function, which takes the full path of the newly loaded file as an argument. This function sends another AJAX request to the server, where the queue check is in progress.
startConversion JS Snippet
function startConversion(path){
$.ajax({
url: '/api/upload/conversion',
type: 'POST',
data: { path: path },
success: function(data){
var response = JSON.parse(data);
// if response.status == 'start', run checkConversion function
}
})
Code /api/upload/conversion
$queue = $this->model->getQueue();
if($queue < 5){
// If queue is empty, notify browser
$output = array('progress' => 'start');
echo json_encode($output);
// Then start shell_ecxec
$cmd = 'some_command';
shell_exec('ffmpeg.exe -i ' . $path . ' ' . $cmd . ' 1>log.txt 2>&1 ');
} else {
$output = array('progress' => 'waiting');
echo json_encode($output);
}
Here, for some reason, a message is already not being displayed stating that processing has begun (echo before $cmd)
Full link
And here the most interesting begins. It would seem that it would be possible in the method to success call the check on the status of the processing (start, queue or an error) and if the processing began to call the third function, which, again AJAX, polls the server returns the progress of the processing
But in reality, the following happens: as soon as the second function sends the request, nothing is returned in response, but processing is in progress, and in the console it shows the status of the request - PENDING. As soon as the server completes processing, a response is issued from the server that the queue is empty and you can start processing the progress, although the processing has already been completed
There are suggestions that further execution of the script is blocked by the command shell_exec() until the end of her work
Also, it is not clear to me why the response from the server is not issued when the echo is clearly registered in it, and why the browser is waiting for the complete completion of the work shel_exec() because after it, the code does not have one. What should be done, that is, as if according to the logic of things and by the code, the server should give an answer in the form of a JSON string, the browser should show that the request has completed with the status of 200, and the server, in the meantime, should start the conversion, but for some reason this does not happen...
I'm in the process of creating a lapcounter for a slot car racetrack.
Whenever a car passes a sensor, it sends some data through a microcontroller to the computer which writes the data to a file. Now I would like to detect whenever that file has changed, meaning that a car has passed the sensor, and store that data onto a database (mysql), then present it on the screen in real time. The filecheck would have to be done continuously, and as rapidly as possible..
Also a timer that counts in hundreds of a seconds would be nice to present.. But that's another thing, and has nothing to do with this question...
There may be better ways of doing this, but I'm pretty sure it could be possible with some ajax, php and mysql on a local server that is directly connected to the microcontroller.
Correct me if I'm wrong, but Facebook, Stackoverflow, and instant messages/chats uses some similar future I would assume..
The Logic for file Change can be on php end 'check_the_file_change.php'
The below is the smaple JS code, which will send the request every 1 second to a php file
'check_the_file_change.php'
$(document).ready(function(){
function checkFileChange(){
setTimeout(function () {
$.ajax({
url :'check_the_file_change.php',
type : 'POST',
data :{
user_secret : 'XXXX'
},
success : function(respsone){
if(response == "true"){
//file changed
}else{
//file not chnaged
}
},
complete : function(){
checkFileChange(); //net file check after 1 sec from the completion of the previous request
}
});
},1000);
}
}
});
ON PHP end
header("Access-Control-Allow-Origin: *"); // use this in case the ajax call is coming from other domain
$filename = 'somefile.txt';
if (file_exists($filename)) {
if( time() - filemtime($filename) > 60){
echo "false";
}else{ //file is modified 1 minute ago
echo "true";
}
}
I'm using a head request as follows in order to check if a given file exist or not.
$.ajax({
url:'http://www.example.com/somefile.ext',
type:'HEAD',
error: function()
{
//file not exists
},
success: function()
{
//file exists
}
});
However, now I'm putting the files in a folder which is restricted by .htaccess through the following rule
deny from all
But now it looks like jquery is always returning 404 Not Found as a result of trying to access a resricted folder.
I can't give up on restricting this folder as the files are being uploaded by users and I can't be sure they are safe.
How can I check if a file exist in this folder ? given that this file exist on the same domain ?
Use php to look for the file, and handle it via jQuery...
jQuery
$.ajax({
url: "somephpfile.php?file=filename.txt",
success: function(file_exists) {
if(file_exists) {
alert("File Exists");
}
}
});
php
<?php
if(file_exists($_GET["file"])) {
echo "1";
} else {
echo "0";
}
Make sure you have some security in the php logic to make sure it can't be used to hack your server!
you cant access the files behind .htaccess unless the user is already logged in and has access to the files. You can make a public page (php) that takes a file name as a parameter, and then do server side checks to see if the file exists, and return a JSON object or true/false.. something like this:
$.ajax({
url:'http://www.example.com/checkfile.php?locn=somefile.ext',
error: function()
{
//this happens when jquery cant contact your php script or your server threw an error... 404's and 500's... just show a generic "cant connect to server" or something.
},
success: function(data)
{
//this means you got data back from the server as a 200 response.
//it does not say whether or not the file exists.
//to do that, you must examine the JSON response. perhaps like this:
if (data == true)
{
//the file exists
}
else
{
//the file does not exist
}
}
});
your checkfile.php might look something like this (pseudo-code):
$fileloc = $_GET["locn"];
if (file_exists($locn))
{
echo "true";
}
else
{
echo "false";
}
i'm trying to read the filesize of a file while uploading it this way
1 . start an $.ajax(); request to start a server to server downloading of file (i'm using php fopen + stream_copy_to_stream );
2 . start a second $a.ajax(); request each tot time and try to read the current filesize till the end of download process;
unfortunely this don't work as expected.
both the requests are made correctly but the filesize is read only after the file is fully transfered. so i get all the alert message at the end of the process instead of meanwhile
i guess i'm doing something that can't be done? or i'm just missing something?
pseudo code:
var uploadStart = 0;
function startUploadProgressBar(file) {
var file = file;
uploadStart = setInterval(function() {
$.ajax({
data: 'uploadProgress=1&file=' + file,
success: function(json) {
alert(json.filesize)
}
});
},
1000);
}
$('#submit').click(function() {
var file = somevar;
startUploadProgressBar(file);
$.ajax({
success: function(json) {
clearInterval(uploadStart);
}
});
return false;
});
Just for uploading files via ajax, you can consider using the jQuery forms plugin. It supports progress bars, as far as I know.
If you want to stick to your method, read through the comments of the function's manual page in the PHP manual. There are alternatives that can be modified outputting states of the transfer.
Why not just have the PHP page return the size of the file itself using filesize()? That way you don't need to poll repeatedly.
I'm doing a long poll method chatroom. But it seems that, when a long poll occurs and I refresh the page in chrome OR i try to send another async request everything times out (i.e i cant load my domain again until i close/reopen the browser).
My client side code is:
$(document).ready(function() {
setTimeout(
function () {
longPollForMessages();
},
500
);
});
function longPollForMessages()
{
$.ajax({
url: url,
dataType: 'json',
success: function(data) {
$('#chat_messages').append('<div>'+data.messages+'</div>');
longPollForMessages();
}
});
}
And my serverside:
while(true) {
$messages = $db->getMessages();
if (!$messages || sizeof($messages)==0) {
sleep(1);
} else {
echo '{"message":'.json_encode($messages).'}';
die();
}
}
Any ideas? Assume no syntax errors.
I can see you have already answered your own question, but I recently had a similar problem and found another way to handle it is to disable the setTimeout on ajax call, then restart it on success. That way you aren't pinging your server for information when it isn't ready to give it.
I figured it out from this question: stackoverflow.com/questions/4457178/… - php locks a given session until the page is done loading so the second ajax call wasn't able to go through. You have to release the lock by calling session_write_close();