Merging duplicate arrays with count - php

Question

I'm working with large sets of data and i need to merge the arrays when they are duplicate. If they get merged I need a count added to the array.
array:3721 [▼
0 => array:3 [▼
"subscriber" => "gmail.com."
"code" => 554
"status" => 50
]
1 => array:3 [▼
"subscriber" => "apied.be"
"code" => 550
"status" => 51
]
2 => array:3 [▼
"subscriber" => "beton-dobbelaere.be"
"code" => 550
"status" => 50
]
3 => array:3 [▼
"subscriber" => "live.be"
"code" => 550
"status" => 51
]
4 => array:3 [▼
"subscriber" => "hotmail.be"
"code" => 550
"status" => 51
]
5 => array:3 [▼
"subscriber" => "telenet.be"
"code" => 550
"status" => 50
]
6 => array:3 [▼
"subscriber" => "telenet.be"
"code" => 550
"status" => 55
]
7 => array:3 [▼
"subscriber" => "telenet.be"
"code" => 550
"status" => 51
]
8 => array:3 [▼
"subscriber" => "telenet.be"
"code" => 550
"status" => 51
]
This should look something like:
array:3721 [▼
0 => array:3 [▼
"subscriber" => "gmail.com."
"code" => 554
"status" => 50
"amount" => 1
]
1 => array:3 [▼
"subscriber" => "apied.be"
"code" => 550
"status" => 51
"amount" => 1
]
2 => array:3 [▼
"subscriber" => "beton-dobbelaere.be"
"code" => 550
"status" => 50
"amount" => 1
]
3 => array:3 [▼
"subscriber" => "live.be"
"code" => 550
"status" => 51
"amount" => 1
]
4 => array:3 [▼
"subscriber" => "hotmail.be"
"code" => 550
"status" => 51
"amount" => 1
]
5 => array:3 [▼
"subscriber" => "telenet.be"
"code" => 550
"status" => 50
"amount" => 1
]
6 => array:3 [▼
"subscriber" => "telenet.be"
"code" => 550
"status" => 55
"amount" => 1
]
7 => array:3 [▼
"subscriber" => "telenet.be"
"code" => 550
"status" => 51
"amount" => 2
]
When I merged this example using
array_unique($hardbounces, SORT_REGULAR);
I was left with around 534 results instead of 3721 this is great, only I need to know the amount as well and it has to be somewhat efficient because the result sets can be very big (much bigger).
After it need to be sorted as well to domains and amounts.
I'm using laravel 5.1 if necessary I could turn the array into a collection so helper functions are available

Answer 1

I'm not sure how efficient this code is with really many items (BTW, you did not mention the order of magnitude of the numbers) , but I think it will be more efficient than converting arrays into Laravel collections
$arr = [
0 => [
"subscriber" => "gmail.com.",
"code" => 554,
"status" => 50,
],
1 => [
"subscriber" => "apied.be",
"code" => 550,
"status" => 51,
],
2 => [
"subscriber" => "beton-dobbelaere.be",
"code" => 550,
"status" => 50,
],
3 => [
"subscriber" => "live.be",
"code" => 550,
"status" => 51,
],
4 => [
"subscriber" => "hotmail.be",
"code" => 550,
"status" => 51,
],
5 => [
"subscriber" => "telenet.be",
"code" => 550,
"status" => 50,
],
6 => [
"subscriber" => "telenet.be",
"code" => 550,
"status" => 55,
],
7 => [
"subscriber" => "telenet.be",
"code" => 550,
"status" => 51,
],
8 => [
"subscriber" => "telenet.be",
"code" => 550,
"status" => 51,
],
];
$res = [];
foreach($arr as $element) {
if(empty($res[$element['subscriber']])) {
$res[$element['subscriber']] = [$element, 'count' => 1];
} else {
$res[$element['subscriber']]['count']++;
}
}
var_dump($res);

Answer 2

Try
<?php
$input = array(
0 => array(
"subscriber" => "gmail.com.",
"code" => 554,
"status" => 50),
1 => array(
"subscriber" => "apied.be",
"code" => 550,
"status" => 51),
2 => array(
"subscriber" => "beton-dobbelaere.be",
"code" => 550,
"status" => 50),
3 => array(
"subscriber" => "live.be",
"code" => 550,
"status" => 51),
4 => array(
"subscriber" => "hotmail.be",
"code" => 550,
"status" => 51),
5 => array(
"subscriber" => "telenet.be",
"code" => 550,
"status" => 50),
6 => array(
"subscriber" => "telenet.be",
"code" => 550,
"status" => 55),
7 => array(
"subscriber" => "telenet.be",
"code" => 550,
"status" => 51),
8 => array(
"subscriber" => "telenet.be",
"code" => 550,
"status" => 51)
);
/**
*#param array $counted The array already counted or NULL
*#param array $new The array to count or to merge with the counted $counted
*/
function merge_xor_count(array $counted = NULL, array $new){
if($counted === NULL){
$counted = array();
}
foreach($new as $keyNew => $valueNew){
$matches = false;
foreach($counted as $keyOut => $valueOut){
if ($valueOut['subscriber'] == $valueNew['subscriber'] && $valueOut['code'] == $valueNew['code'] &&
$valueOut['status'] == $valueNew['status']){
$matches = $keyOut;
}
}
if($matches !== false){
$counted[$matches]['amount']++;
}
else{
if(!isset($valueNew['amount'])) $valueNew['amount'] = 1;
$counted[] = $valueNew;
}
}
return $counted;
}
$output = merge_xor_count(NULL, $input);
print_r ($output)."\n";
$output = merge_xor_count($output, $input);
print_r ($output)."\n";
?>

Answer 3

I managed to fix it myself with a foreach loop it runs faster then i expected (0.3179 seconds)
$merged = [];
foreach($hardbounces as &$hardbounce){
if(empty($merged)){
$merged[] = $hardbounce;
}else{
$i = 0;
foreach($merged as $key => $merge){
$i++;
if($hardbounce['subscriber'] == $merge['subscriber'] && $hardbounce['code'] == $merge['code'] && $hardbounce['status'] == $merge['status']){
$merged[$key]['amount']++;
break;
}
if(count($merged) == $i){
$merged[] = $hardbounce;
}
}
}
}
I loop through the bounces, if merged is empty (first iteration) it just adds the hardbounce.
From that point on I will loop through the new array and check if there is a duplicate. When this happens we just add one to the amount and break out of the foreach. Otherwise it will eventually still add duplicates.
After i check if we made it to the last iteration, which means if it still hasn't break-ed we should add the hardbounce because it doesn't exist yet.
To be clear I made sure the amount 1 already exists before this loop runs, instead of adding it during this loop.