How to save scraped data through command in database - php

Question

I have this in the command but not saving the database.
$client = new Client(HttpClient::create(['timeout' => 60]));
$crawler = $client->request('GET', 'https://www.bbc.com/news/');
$model = new Scraper();
$crawler->filter('.gs-c-promo-heading')->each(function ($node) {
$model->title = $node->text();
});
$crawler->filter('.gs-c-promo-summary')->each(function ($node) {
$model->text = $node->text();
});
$crawler->filter('.gs-c-timestamp')->each(function ($node) {
$model->time = $node->text();
});
$crawler->filter('.gs-c-section-link')->each(function ($node) {
$model->country = $node->text();
});
$model->save();
$this->info('done success');
Giving this type of error
$ php artisan scraper:start
ErrorException
Creating default object from empty value
at E:\wamp64\www\Laravel7Projects\system\system\app\Console\Commands\ScrapCommand.php:49
45| $crawler = $client->request('GET', 'https://www.bbc.com/news/');
46|
47| $crawler->filter('.gs-c-promo-heading')->each(function ($node) {
48|
> 49| $model->title = $node->text();
51| });
52| $crawler->filter('.gs-c-promo-summary')->each(function ($node) {
53|
1 E:\wamp64\www\Laravel7Projects\system\system\app\Console\Commands\ScrapCommand.php:49
Illuminate\Foundation\Bootstrap\HandleExceptions::handleError("Creating default object from empty value", "E:\wamp64\www\Laravel
7Projects\system\system\app\Console\Commands\ScrapCommand.php", [Object(Symfony\Component\DomCrawler\Crawler), Object(stdClass)])
2 E:\wamp64\www\Laravel7Projects\system\system\vendor\symfony\dom-crawler\Crawler.php:352
App\Console\Commands\ScrapCommand::App\Console\Commands\{closure}(Object(Symfony\Component\DomCrawler\Crawler))
but with this mthod is working which is no need because I want to save all attributes in one new class
$crawler->filter('.gs-c-promo-heading')->each(function ($node) {
$model = new Scraper();
$model->title = $node->text();
$model->save();
});
I want to save every attribute in the database but getting error
With this logic only saving first row where as i want to save all loop in the database
$client = new Client(HttpClient::create(['timeout' => 60]));
$crawler = $client->request('GET', 'https://www.bbc.com/news/');
$model = new Scraper();
$crawler->filter('.gs-c-promo-heading')->each(function ($node) use ($model) {
$model->title = $node->text();
});
$model->save();
Edit: Only titles are saving in the database whereas others not saving with this method because we are looping only titles in foreach
And map is not working whereas each in working you can check below code
$titles = $crawler->filter('.gs-c-promo-heading')->each(function($node) {
return $node->text();
});
$texts = $crawler->filter('.gs-c-promo-summary')->each(function($node) {
return $node->text();
});
$times = $crawler->filter('.gs-c-timestamp')->each(function($node) {
return $node->text();
});
$countries = $crawler->filter('.gs-c-section-link')->each(function($node) {
return $node->text();
});
$dataArray = [];
foreach ($titles as $key => $item) {
$newModelData = [
'title' => $titles[$key],
'text' => $texts[$key],
'time' => $times[$key],
'country' => $countries[$key]
];
$dataArray[] = $newModelData;
}
Scraper::insert($dataArray);

Answer 1

When you are using each() you don't have access to $model variable (outside of the scope).
You should try use() to introduce $model variable inside of local scope of an anonymous function
$crawler->filter('.gs-c-promo-heading')->each(function ($node) use($model) {
$model->title = $node->text();
});
Edit: If you want to save multiple items at the same time you could pluck necessary values and then create arrays of data according to your model attributes.
$titles = $crawler->filter('.gs-c-promo-heading')->map(function($node) {
return $node->text();
});
$texts = $crawler->filter('.gs-c-promo-heading')->map(function($node) {
return $node->text();
});
// same for times, countries
$dataArray = [];
foreach ($titles as $key => $item) {
$newModelData = [
'title' => $titles[$key],
'text' => $texts[$key],
'time' => $times[$key],
'country' => $countries[$key]
];
$dataArray[] = $newModelData;
}
Model::createMany($dataArray);