how can insert 1000000 row from textarea into database in laravel? - php

Question

how can insert 1000000 row from textarea into database in laravel 8 ???????
i write this code and just can insert 30000 row and then browser give me HTTP ERROR 500
i set max_execution_time to 300 in php.ini
this is my code
please help me . thanks
public function mobile_store(Request $request)
{
$data = $request->validate([
'mobile' => ['required', 'string', 'unique:mobiles,mobile'],
]);
$textAr = collect(explode("\r\n", $data['mobile']));
$ALL = $textAr->unique();
$Filter = $ALL->filter()->all();
$counter_unique = count($Filter);
$counter = count($textAr);
$insert_data = collect();
foreach ($Filter as $line) {
if (strlen($line) >= 10) {
$final = '+98' . substr($line, -10);
}
$insert_data->push([
'mobile' => $final,
'created_at' => Carbon::now(),
'updated_at' => Carbon::now(),
]);
}
foreach ($insert_data->chunk(5000) as $chunk) {
Mobile::insert($chunk->toArray());
}
return redirect()->back()->with('success', "There were $counter_unique rows in the list and $counter non-duplicate rows were entered");
}

Answer 1

Don't store all of the data first, just use 1 foreach and every 5000 records store the data and then reset the array and do the next batch of records.
$insert_data = collect();
$totalRecords = 0;
$batchCount = 0;
foreach ($Filter as $line) {
if (strlen($line) >= 10) {
$final = '+98' . substr($line, -10);
}
$insert_data->push([
'mobile' => $final,
'created_at' => Carbon::now(),
'updated_at' => Carbon::now(),
]);
if ( $batchCount++ == 5000 ) {
// Insert data
Mobile::insert($insert_data->toArray());
// Reset batch collection
$insert_data = collect();
// Reset counter of current batch
$batchCount = 0;
}
// Count of all records
$totalRecords++;
}
// Insert remaining records
if( $insert_data->count() > 0 )
Mobile::insert($insert_data->toArray());
}