PHP/MySQL - Analyzing common sets across multiple sets - php

Question

Let's say I have two tables, people and families.
families has two fields - id and name. The name field contains the family surname.
people has three fields - id, family_id and name - The family_id is the id of the family that that person belongs to. The name field is that person's first name.
It's basically a one to many relationship with one family having many people.
I want to get a lists of name sets, ordered by the highest occurrence of the largest set of names across families.
That probably doesn't make much sense...
To explain what I want further, we can score each set of names. The 'score' is the array size * number of occurrences across families.
For example, let's say two names, 'John' and 'Jane' both existed in three families - That set's 'score' would be 2*3 = 6.
How could I get an array of sets of names, and the set's 'score', ordered by each set's score?
Sample Result Set (I've put it in a table layout, but this could be a multi-dimensional array in PHP) - Note this is just randomly thought up and doesn't reflect any statistical name data.
names | occurrences | score
Ben, Lucy | 4 | 8
Jane, John | 3 | 6
James, Rosie, Jack | 2 | 6
Charlie, Jane | 2 | 4
Just to clarify, I'm not interested in sets where:
The number of occurrences is 1 (obviously, just one family).
The set size is 1 (just a common name).
I hope I have explained my somewhat complex problem - if anyone needs clarification please say.

Answer 1

OK, got it:
<?php
require_once('query.lib.php');
$db=new database(DB_TYPE,DB_HOST,DB_USER,DB_PASS,DB_MISC);
$qry=new query('set names utf8',$db);
//Base query, this filters out names that are in just one family
$sql='select name, cast(group_concat(family order by family) as char) as famlist, count(*) as num from people group by name having num>0 order by num desc';
$qry=new query($sql,$db);
//$qry->result is something like
/*
Array
(
[name] => Array
(
[0] => cathy
[1] => george
[2] => jack
[3] => john
[4] => jane
[5] => winston
[6] => peter
)
[famlist] => Array
(
[0] => 2,4,5,6,8
[1] => 2,3,4,5,8
[2] => 1,3,5,7,8
[3] => 1,2,3,6,7
[4] => 2,4,7,8
[5] => 1,2,6,8
[6] => 1,3,6
)
[num] => Array
(
[0] => 5
[1] => 5
[2] => 5
[3] => 5
[4] => 4
[5] => 4
[6] => 3
)
)
$qry->rows=7
*/
//Initialize
$names=$qry->result['name'];
$rows=$qry->rows;
$lists=array();
for ($i=0;$i<$rows;$i++) $lists[$i]=explode(',',$qry->result['famlist'][$i]);
//Walk the list and populate pairs - this filters out pairs, that are specific to only one family
$tuples=array();
for ($i=0;$i<$rows;$i++) {
for ($j=$i+1;$j<$rows;$j++) {
$isec=array_intersect($lists[$i],$lists[$j]);
if (sizeof($isec)>1) {
//Every tuple consists of the name-list, the family list, the length and the latest used name
$tuples[]=array($names[$i].'/'.$names[$j],$isec,2,$j);
}
}
}
//Now walk the tuples again rolling forward, until there is nothing left to do
//We do not use a for loop just for style
$i=0;
while ($i<sizeof($tuples)) {
$tuple=$tuples[$i];
//Try to combine this tuple with all later names
for ($j=$tuple[3]+1;$j<$rows;$j++) {
$isec=array_intersect($tuple[1],$lists[$j]);
if (sizeof($isec)>0) $tuples[]=array($tuple[0].'/'.$names[$j],$isec,$tuple[2]+1,$j);
}
$i++;
}
//We have all the tuples, now we just need to extract the info and prepare to sort - some dirty trick here!
$final=array();
while (sizeof($tuples)>0) {
$tuple=array_pop($tuples);
//name list is in $tuple[0]
$list=$tuple[0];
//count is sizeof($tuple[1])
$count=sizeof($tuple[1]);
//length is in $tuple[2]
$final[]=$tuple[2]*$count."\t$count\t$list";
}
//Sorting and output is all that is left
rsort($final);
print_r($final);
?>
I am sorry I just realized I use a query lib that I can't source in here, but from the comment you will easily be able to create the arrays as in the section "Initialize".
Basically what I do is starting with the pairs I keep an array of the families all the names in the current name list belong to, then intersect it with all not-yet tried names.

Answer 2

Will this work?
SELECT
f.name AS 'surname',
GROUP_CONCAT(DISTINCT p.name ORDER BY p.name) AS 'names',
COUNT(DISTINCT p.name) AS 'distinct_names',
COUNT(p.id) AS 'occurrences',
COUNT(DISTINCT p.name) * COUNT(p.id) AS 'score'
FROM
families f
LEFT JOIN people p ON ( f.id = p.family_id )
GROUP BY
f.id
ORDER BY
f.name