In my web application I have a function that is executed more than 100 times in a minute, and I'm trying to find a better performing approach.
public static function removeDuplicateFeeds($id, $feeds, $todo)
{
$actionsHistory = ActionsHistory::whereAccountId($id)->whereActionName($todo)->get();
if (!empty($actionsHistory)) {
foreach ($actionsHistory as $history) {
foreach ($feeds as $key => $feed) {
if ($history->action_name == $feed['pk']) {
unset($feeds[$key]);
}
}
}
}
}
I want to remove all the elements from $feeds that are in $actionsHistory as well.
UPDATE:
in this test code first index of $feeds array as "pk":"7853740779" is stored on my database and after remove duplicate this item should be removed, but i have all of $feeds items into $filteredFeeds too
$userAccountSource = InstagramAccount::with('user', 'schedule')->get();
$feeds = [
json_decode('{"pk":"7853740779","username":"teachkidss","full_name":"..."}'),
json_decode('{"pk":"7853740709","username":"teachkidss","full_name":"..."}'),
json_decode('{"pk":"7853740009","username":"teachkidss","full_name":"..."}')
];
$filteredFeeds = AnalyzeInstagramPageController::removeDuplicateFeeds($userAccountSource[0]->id, $feeds, 'like');
public function removeDuplicateFeeds($id, $feeds, $todo)
{
$feeds = collect($feeds); // If $feeds is not already a collection
$actionsHistory = ActionsHistory::whereAccountId($id)
->whereActionName($todo)
->whereIn('action_name', $feeds->pluck('pk')) // retrieves only duplicate records
->select('action_name') // reducing the select improves performance
->get(); // Should return a eloquent collection instance
if (!empty($actionsHistory)) {
return $feeds->whereNotIn('pk', $actionsHistory->pluck('action_name'));
}
return $feeds;
}
Without knowing the number of feed or database records you'll have to test these for performance against your dataset and see if these are more performant.
public static function removeDuplicateFeeds($id, $feeds, $todo)
{
$feeds = collect($feeds); // If $feeds is not already a collection
$actionsHistory = ActionsHistory::whereAccountId($id)
->whereActionName($todo)
->select('action_name') // reducing the select improves performance
->get(); // Should return an eloquent collection instance
if (!empty($actionsHistory)) {
return $feeds->whereNotIn('pk', $actionsHistory->pluck('action_name'));
}
return $feeds;
}
or if your database query returns significantly more records than you have feeds you could try leveraging mysql's faster query speed instead of using php's slower array_filter/foreach speed.
public static function removeDuplicateFeeds($id, $feeds, $todo)
{
$feeds = collect($feeds); // If $feeds is not already a collection
$actionsHistory = ActionsHistory::whereAccountId($id)
->whereActionName($todo)
->whereIn('action_name', $feeds->pluck('pk')) // retrieves only duplicate records
->select('action_name') // reducing the select improves performance
->get(); // Should return a eloquent collection instance
if (!empty($actionsHistory)) {
return $feeds->whereNotIn('pk', $actionsHistory->pluck('action_name'));
}
return $feeds;
}
If either of these works, it would be good to know how much faster this was for you. Let us know. Good luck.
You may try array_unique function from PHP. Source.
You can use build in laravel unique() function from collection, see more here :
https://laravel.com/docs/5.6/collections#method-unique
doing that you cove will maintain clean and elegant.
Related
How to transform data (from Repository/Entity to PersonData) in proper way in Symfony?
Code like below is useful for me. I using this metod often and make work with arrays of PersonData objects. But I'm aware that it brakes SOLID principles (Repository class filter/transform data but should only get data from db).
If I separate filling PersonData my other services will be grow some extra lines of code because they need to do every time what actual findActive() is doing.
// src/Repository
/**
* #return \App\ValueObject\PersonData[]
*/
public function findActive(): array
{
$entities = $this->findBy(['active_status' => true]);
if (! $entities) {
return [];
}
$data = [];
foreach ($entities as $entity) {
$data[$entity->getId()] = new PersonData(
$entity->getName(),
$entity->getScore(),
$entity->getValue()
);
}
return $data;
}
How can I improve the performance of a query with the following characteristics (this is a simulation to exemplify):
public function getData()
{
$itens = Item::orderBy('id_item', 'DESC')->get();
foreach($itens as $item){
$item->games = $item->games()->get();
foreach( $item->games as $player){
$item->players = $player->players()->get();
}
}
return response()->json(['success'=>true, 'itens'=>$itens]);
}
In my real case, the foreach returns a lot of data with performance problems;
The Waiting (TTFB), that is, the time the browser is waiting for the
first byte of a response, takes almost 27s
class Item extends Model{
// ...
public function games(){
return this->hasMany(Game::class);
}
public function players(){
return this->hasManyThrough(Player::class ,Game::class);
}
}
public function getData()
{
$itens = Item::orderBy('id_item', 'DESC')->with('games','players')->get();
return response()->json(['success'=>true, 'itens'=>$itens]);
}
That will result in 3 queries and the output will be like:
[
// item 0 ...
[
// item data
[
// games
],
[
// players
]
],
// other items
]
Here you go
public function getData()
{
$items = Item::with('games.players')->orderBy('id_item', 'DESC')->get();
return response()->json(['success'=>true, 'items'=>$items]);
}
Code based solutions:
Looping on array is around 2 times cheaper than looping on List.
A 'for' loop is much faster than 'foreach'
a other solutio is:
Filtering unnecessary parts for running the loop
I'm Trying to create single array that contains all the ID of parent and child from the database.
But all I was getting is single data.
My ideal output is:
array('160', '161', '162', '163', '164');
what am I getting is only
array('160');
Here is what I've done so far.
public function arrayId(array $elements) {
$where_in = array();
foreach($elements as $element){
if($element->isArray) {
$elems = $this->my_model->select_where('tbl_policies', array('parent_id' => $element->id));
$this->arrayId($elems);
}
$where_in[] = $element->id;
}
return $where_in;
}
$id = 160; //for instance
$elements = $this->my_model->select_where('tbl_policies', array('id' => $id));
$where_in = $this->arrayId($elements);
die(print_r($where_in));
and the data I'm fetching here:
tbl_policies
It's kinda difficult for me to construct questions. So please if something is not clear, do comment below, I'll try my best to make it more understandable. Thanks in advance.
I understand, that you want to delete a parent with all its children and grandchildren. But you do it not directly and sequentially rather want to collect all ids of the records to be deleted. You should go following steps:
Parent-Id (example 160) is already known. Add this to your list.
Write a recursive function such as getChildrenIds(parentId).
Within this function you should iterate over children. And if a child has the flag "isArray" (according to your application logic) then you should call getChildrenIds(currentChildId)
I have written following function. It should work.
public function getChildrenIds( int $parentId, array &$idList) {
$idList[] = $parentId;
$records = $this->my_model->select_where('tbl_policies', array('parent_id' => $parentId));
foreach($records as $r){
if($r->isArray)
$this->getChildrenIds($r->id, $idList);
else
$idList[] = $r->id;
}
return;
}
public function CollectIds(){
$id = 160; //for instance
$where_in = array();
$this->getChildrenIds($id, $where_in);
}
Please notice, that $where_in passed by reference to the recursive function getChildrenIds() and filled there.
I cant orderBy points. Points is accessor.
Controller:
$volunteers = $this->volunteerFilter();
$volunteers = $volunteers->orderBy('points')->paginate(10);
Volunteers Model:
public function siteActivities()
{
return $this->belongsToMany(VolunteerEvent::class, 'volunteer_event_user', 'volunteer_id', 'volunteer_event_id')
->withPivot('data', 'point', 'point_reason');
}
public function getPointsAttribute(){
$totalPoint = 0;
$volunteerPoints = $this->siteActivities->pluck('pivot.point', 'id')->toArray() ?? [];
foreach ($volunteerPoints as $item) {
$totalPoint += $item;
}
return $totalPoint;
}
But I try to sortyByDesc('points') in view it works but doesn't work true. Because paginate(10) is limit(10). So it doesn't sort for all data, sort only 10 data.
Then I try to use datatable/yajra. It works very well but I have much data. so the problem came out
Error code: Out of Memory
You could aggregate the column directly in the query
$volunteers = $this->volunteerFilter();
$volunteers = $volunteers->selectRaw('SUM(pivot.points) AS points)')->orderByDesc('points')->paginate(10);
I want to copy a record with all his relations.
I'm trying with:
$o = Doctrine::getTable('Table')->Find(x);
$copy = $object->copy();
$relations = $o->getRelations();
foreach ($relations as $name => $relation) {
$copy->$relation = $object->$relation->copy();
}
$copy->save();
This code doesn't works, but I think it's on the way.
I never could get the deep copy function to operate correctly.
I manually coded a deep copy function for one of my models like this
public function copyAndSave ()
{
$filters = array('id', 'created');
$survey = $this->copy();
$survey->Survey_Entries = new Doctrine_Collection("Survey_Model_Entry");
$survey->Assignment_Assignments = new Doctrine_Collection("Assignment_Model_Assignment");
$survey->Survey_Questions = new Doctrine_Collection("Survey_Model_Question");
$survey->save();
foreach ($this->Survey_Questions as $question)
{
$answers = $question->Survey_Answers;
$newQuestion = $question->copy();
$newQuestion->survey_surveys_id = $survey->id;
$newQuestion->save();
$newAnswers = new Doctrine_Collection("Survey_Model_Answer");
foreach($answers as $answer)
{
$answer = $answer->copy();
$answer->save();
$answer->survey_questions_id = $newQuestion->id;
$newAnswers->add($answer);
}
$newQuestion->Survey_Answers = $newAnswers;
$survey->Survey_Questions->add($newQuestion);
}
return $survey->save();
}
You can read about copy() here. It takes an optional parameter $deep:
$deep
whether to duplicates the objects targeted by the relations
So
$copy = $object->copy(true);
should do it.
Sorry if I'm resurrecting this thread...
I found myself in search of a solution recently where I needed to copy a record and retain the references of the original. A deep copy $record->copy(true) copies the references, which was no good for me. This was my solution:
$record = Doctrine_Core::getTable('Foo')->find(1);
$copy = $record->copy();
foreach($record->getTable()->getRelations() as $relation) {
if ($relation instanceof Doctrine_Relation_Association) {
$ids = array();
foreach ($relation->fetchRelatedFor($record) as $r) {
$ids[] = $r->getId();
}
$copy->link($relation->getAlias(), $ids);
}
}
if ($copy->isValid()) {
$copy->save();
}
Hope this helps :)
This is how i done, but some fix is needed.
$table = $entidade->getTable();
$relations = $table->getRelations();
foreach($relations as $relation => $data) {
try {
$entity->loadReference($relation);
} catch(Exception $e) {
die($e->getMessage());
}
}
I am using Symfony1.4.1 and that uses Doctrine 1.2.1 (I think).
I have been trying to make a function that did all the above myself, when I found one that already exists.
Try this in any function and look at the results:
$tmp=$this->toArray(TRUE);
var_dump($tmp);
$this->refreshRelated();
$tmp=$this->toArray();
var_dump($tmp);
$tmp=$this->toArray(TRUE);
var_dump($tmp);
exit();
I am going to try two different things:
A/ put $this->refreshRelated() into the constructor of all my model objects.
B/ write a function that takes an array depicting the object graph that I want populated. Calling the function refereshRelatedGraph($objectGraphArray). With the right structure of the array (having all the appropriate relation names at each level), I could control which relations get populated and which don't. One use for this is to populate only children, not parent relations. The other is for when a ERD/Schema/ObjectGraph has an element that is 'owned' by more than one object (many to many, other special circumstances that I have), I could control which side of the relationships get pre(non lazy) loaded.