Big XML file import to database PHP - php

Question

I am facing a problem that somehow I don't see the solution to it. I have a XML file that needs to be importet to custom DB structure, when user uploads / imports the file the ajax post is waiting untill the file import is finished, but this could take 5 hours or more I don't know. What is the best way to handle this UI issue.
I was thinkg about thread uplaod, to split the file in multiple parts and upload each with it's own thread (pthreads, having problems with instalation on centos 7 / PHP 7)
Or if there is any other way that I could import the file in the background and whenerever the user refreshes the page there would be a status log output so that user would know when the import is finished and if successful.

Answer 1

You would want to run them using a background job ( a detached process ) this way the end user gets a confirmation message right away, and then send an email when the long running task is complete. Then they don't have to wait for it to finish. As I mentioned in the comments I have a class I wrote on my git hub for this
https://github.com/ArtisticPhoenix/MISC/blob/master/BgProcess.php
But it passes the args as a path because it's setup for Code Igniter, so you would have to change that or split the arguments up within your code.
Anyway the basics is similar to running a cron job, This varies in the implantation depending on the OS of the server. But on Linux the command is like this
php -f "path/to/phpfile.php" "{args}" > /dev/null &
The > /dev/null & part sends the output to null ( throws it away ) and the & runs it as a non-blocking process meaning the script starting the command can continue on. So using an example as this
.. other code before starting background job ..
exec( 'php -f "path/to/phpfile/xmlProcessor.php" "testXML/2" > /dev/null &');
.. code to tell user job is started .. this runs right after the call without waiting for that process to finish.
Then in xmlProcessor.php you would have this
<?php
$args = explode('/', $argv[1]);
$file = $ags[0];
$user_id = $args[1];
... code to process xml
... email user confirmation of completion
http://php.net/manual/en/reserved.variables.argv.php
As I said typically would call it this way,
exec( 'php -f "path/to/phpfile/xmlProcessor.php" "testXML" "2" > /dev/null &');
And access them using
$argv[1] // = testXML
$argv[2] // = 2
But because I use this with CI, it does it's routing for me to a special controller and handles all that. The nice thing about my class is that it should find the PHP executable under most cases, and it has windows compatibility built in ( which was a pain in the ...)
Using that class you would just call it like this
$command = new BgProcess( "path/to/phpfile/xmlProcessor.php", "testXML", 2);
echo $command;
Would output 'php -f "path/to/phpfile/xmlProcessor.php" "testXML/2" > /dev/null &' after starting the process ( the return is just for debugging )
Basically your running a separate background job with PHP via the command line.