Large CSV download script timing out - php

Question

I have a a script that can upload the contents of a CSV file and download the links to a local directory, the CSV file i need to upload to it is about 4056 lines long and 4056 FTP downloads, the script works fine but the web server times out when i use it.
even tried set_time_limit(0);
is there a way i could session the script and loop the process so it could do it's job for a few hours without interruptions.
<?php
/**
* Please change your upload directory according to your needs. Make sure you include the trailing slash!
*
* Windows C:\tmp\
* Linux /tmp/
*/
$uploaddir = '/tmp/';
if(isset($_FILES['userfile']['name'])){
// Read uploaded file
$lines = file($_FILES['userfile']['tmp_name']);
echo "Reading file ... <br/>";
$linecount = 0;
foreach($lines as $line ){
echo ++$linecount . ". FTP Url is : " .$line . "<br/>";
echo " Downloading " . $line . "<br/>";
$parsed_url_values = parse_url($line);
//TODO perhaps do a validation of the ftp url??
if($parsed_url_values['scheme'] == 'ftp'){
// set up basic connection
$conn_id = ftp_connect($parsed_url_values['host']);
// login with username and password
$login_result = ftp_login($conn_id, $parsed_url_values['user'], $parsed_url_values['pass']);
ftp_pasv($conn_id, true);
$path = rtrim($parsed_url_values['path'], '_');
$filename = basename($path);
if (ftp_get($conn_id, $uploaddir . $filename, $path , FTP_BINARY)) {
echo " Successfully downloaded the file " .$line . "<br/>";
} else {
echo " Could not save the file to " . $line . ". Please verify the url is correct and the file exists.<br/>";
}
} else {
echo " Sorry. This script was made for FTP downloads only.";
}
}
}
?>
<form enctype="multipart/form-data" action="" method="post">
<input type="hidden" name="MAX_FILE_SIZE" value="30000" />
Select the file to upload : <input name="userfile" type="file" />
<input type="submit" value="Upload" />
</form>

Answer 1

You need to increase the execution time of your script by using max_execution_time. You can use this example, place it on top of your script:
<?php
ini_set('max_execution_time', 300); //300 seconds = 5 minutes
.. The rest of your script
Hope this will help you!

Answer 2

In my practice I often use step by step loading if i can't load data via cron or php cli.
Something like that (not tested):
<?php
$newFileName = '/path/to/file.csv';
$line = isset($_REQUEST['csvline']) ? (int) $_REQUEST['csvline'] : 0;
$handle = fopen($newFileName, "r");
//10 seconds for each step
$stepTime = 10;
$startTime = time();
$lineCounter = 0;
while (($fileop = fgetcsv($handle)) !== false)
{
//already loaded lines
$lineCounter++;
if ($lineCounter <= $line) continue;
//here goes your script logic
//stops when time goes out
if (time() - $startTime >= $stepTime) break;
}
//next you need to query this script again
//using js maybe (window.location or ajax call)
//so you can see loading progress or any other useful information
//like
if (!feof($handle)) {
echo "<script>window.location = '/your.php?csvline={$lineCounter}'<script>";
}
//you even can start loading from last line before script fails
But i'm sure that using cron is the best solution for your problem.