Laravel 8 - Process millions of records in database - php

Question

There are millions of records in a database table.
purchase
id (increments), device_id (int), app_id (int), status(boolean), receipt (string)
My framework laravel 8 and I use mysql
I have created a command
app/Console/Commands/ProcessPurchase.php
<?php
namespace App\Console\Commands;
use App\Models\Purchase;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Queue;
use App\Jobs\ProcessPurchaseJob;
class ProcessPurchase extends Command
{
/**
* The name and signature of the console command.
*
* #var string
*/
protected $signature = 'process:purchase';
/**
* The console command description.
*
* #var string
*/
protected $description = 'Command description';
/**
* Create a new command instance.
*
* #return void
*/
public function __construct()
{
parent::__construct();
}
/**
* Execute the console command.
*
* #return int
*/
public function handle()
{
Log::info('process:purchase command start successfully.');
$purchases = Purchase::getUnequalizedPurchases(250);
if(count($purchases) > 0) {
foreach ($purchases as $purchase) {
//push job to the queue
Queue::push((new ProcessPurchaseJob($purchase, $purchase->os)));
}
} else {
Log::info('process:purchase command end successfully.');
}
}
}
and created a job
app/Console/Commands/ProcessPurchaseJob.php
<?php
namespace App\Jobs;
use App\Models\Purchase;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;
class ProcessPurchaseJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
private $purchase, $os;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct(Purchase $purchase, string $os)
{
$this->purchase = $purchase;
$this->os = $os;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
//getting data from the third-party endpoint
if($purchaseData = getPurchaseDataFromMarket($this->purchase->receipt, $this->os)) {
$this->purchase->expire_date = $purchaseData['expire_date'];
$this->purchase->status = $purchaseData['status'];
try {
$this->purchase->save();
Log::info('purchase job runned successfully. receipt : ' . $this->purchase->receipt);
} catch (\Exception $exception) {
Log::error($exception->getMessage());
}
} else {
Log::error('purchase request error. receipt : ' . $this->purchase->receipt);
}
}
}
and added to app/http/Console/Kernel.php
$schedule->command('process:purchase')
->everyFiveMinutes();
to start cron, execute this command.
./vendor/bin/sail artisan schedule:work
to do the jobs in queue, execute this command
./vendor/bin/sail artisan queue:work --queue=high,default
I should send a request to third-party HTTP endpoint and update per every record.
This process take 1.5 secs about per a record. So, for 10.000.000 records, it takes 173.6 days.
I should can process all these data in a short time. (like 12 hours etc.)
What should i do ?