PHP MySQL hierarchy data optimization - php

I have a table employees, which contains columns : employee_id, name, employee_manager_id.
employee_manager_id references to employee_id. It's a hierarchal data.
I have this output using PHP but couldn't achieve it using only one mySQL query. As of now, I need to process the data in PHP using recursive function so i can achieve this kind output.
Sample Array Output
0 => (
employee_id => 2,
name => Jerald,
employee_manager_id => 1,
depth => 1
),
1 => (
employee_id => 3,
name => Mark,
employee_manager_id => 2,
depth => 2
),
2 => (
employee_id => 6,
name => Cyrus,
employee_manager_id => 3,
depth => 3
),
3 => (
employee_id => 4,
name => Gerby,
employee_manager_id => 2,
depth => 2
)
As of now, this is my recursive function in PHP to achieve the output above.
function get_employees_by_hierarchy( $_employee_id = 0, $_depth = 0, $_org_array = array() ) {
if ( $this->org_depth < $_depth ) {
$this->org_depth = $_depth;
}
$_depth++;
$_query = "SELECT * FROM employees WHERE ";
if ( !$_employee_id ) {
$_query .= "employee_manager_id IS NULL OR employee_manager_id = 0";
}
else {
$_query .= "employee_manager_id = " . $this->dbh->quoteSmart( $_employee_id );
}
$_result = $this->query( $_query );
while ( $_row = $_result->fetchRow() ) {
$_row['depth'] = $_depth;
array_push( $_org_array, $_row );
$_org_array = $this->get_employees_by_hierarchy(
$_row['employee_id'],
$_depth,
$_org_array
);
}
return $_org_array;
}
My question is, is there anyway so I can achieve the array output i want using just one mysql query?
If not possible in mysql query, is there anymore to optimize in my current code?
Any help would greatly be appreciated.
Thanks

You can try a nested set a.k.a. celko tree but insert and delete is very expensive. There is also closures and path enumeration (materialized path) but I'm not an expert. MySql doesn't support recursive queries.

I don't think you can get the depth with your current model without doing any processing on the results, but you don't need to make multiple queries.
Assuming $employees is the list of employees indexed by employee_id, you could do something like this:
function set_employee_depth(&$employees, $id) {
if (!isset($employees[$id]['depth'])) {
$employee_manager_id = (int) $employees[$id]['employee_manager_id'];
if (!$employee_manager_id) {
$employees[$id]['depth'] = 0;
} elseif ($employee_manager_id !== $id) {
$employees[$id]['depth'] = 1 + set_employee_depth($employees, $employee_manager_id);
} else {
throw new \Exception('Employee cannot be its own manager!');
}
}
return $employees[$id]['depth'];
}
foreach ($employees as $id => $employee) {
set_employee_depth($employees, $id);
}

So, your table is composed of 3 columns (employee_id, name, employee_manager_id). employee_manager_id is a self reference to employee_id. You want to construct an array with all records, adding an extra field called depth which represents the distance of said employee to the "big boss", with only one query to the database. Is that correct? I'm also assuming that the database structure can't be changed.
If these assumptions are correct, this is a basic HIERARCHICAL/TREE data structure and thus, you have a couple of ways you can tackle this problem.
First Script
The first script runs the results array sequentially, finding the main node / trunk (the big boss) first and then adding it's children, then it's grandchildren and so on. Each time a node is "sorted", it will be removed from the cycle until no nodes are left. It assumes that:
There are no Orphan records (employees with invalid managers_ids)
There are no Circular References, either simple (A is manager of B and B is manager of A) or complex (*A manager of B, B manager of C and C manager of A)
Each path (from the main node to last node) can have an infinite number of nodes
$results are produced by running a simple query SELECT * FROM employees ORDER BY employee_manager_id
Code:
$finalArray = array();
$limit = count($results);
while (count($results) > 0) {
$results[0]['cnt'] = isset($results[0]['cnt']) ? $results[0]['cnt']++ : 0; // set num of times each element was already visited
if ($results[0]['cnt'] === $limit) { //prevent an infinite cycle
break;
}
$manId = $results[0]['manager_id'];
if ($manId === null) {
$results[0]['depth'] = 0;
} else if ( ($key = searchForId($manId, $finalArray)) !== null ) {
$results[0]['depth'] = $finalArray[$key]['depth'] + 1; //use the depth of parent to calculate its own
} else {
$results[] = $results[0]; //parent was not visited yet so we add it to the end of array
array_shift($results);
continue;
}
unset($results[0]['cnt']);
$finalArray[] = array_shift($results);
}
function searchForId($id, $array) {
foreach ($array as $key => $val) {
if ($val['id'] === $id) {
return $key;
}
}
return null;
}
This script is pretty straightforward. It only runs one query to the DB. In best case scenario, it will only traverse the array once. In worse case scenario, it will visit each element count(array) - 1, which can be slow with big arrays. However, since the results are pre-sorted, best case scenario will probably be more common.
Second Script
The second script builds an actual tree of elements. It's a bit more complex but achieves similar results. Also, the Depth is calculated dynamically.
class Employee {
public $id;
public $name;
public $manager;
public function __construct($id, $name, Employee $manager = null) {
$this->id = $id;
$this->name = $name;
$this->manager = $manager;
}
public function setManager(Employee $manager) {
$this->manager = $manager;
}
public function getDepth() {
if ($this->manager === null) {
return 0;
} else {
return $this->manager->getDepth() + 1;
}
}
}
$finalArray = array();
$paths = array();
foreach ($results as $r) {
$finalArray[(int) $r['id']] = new Employee((int)$r['id'], $r['name']);
if ($r['manager_id'] !== null) {
$paths[(int) $r['id']] = (int) $r['manager_id'];
}
}
foreach ($paths as $k => $v) {
if (isset($finalArray[$k]) && isset($finalArray[$v])) {
$finalArray[$k]->setManager($finalArray[$v]);
}
}

Here is a link for answer
Here is full code for making tree structure for hierarchy management using php and mysql.

Related

CakePHP 3.8 convert array to Cake\ORM\Entity

I am currently involved in a CakePHP project and do not know how I can pass a modified query/array to a paginator.
Here is my controller:
public function index($fooElement = '')
{
$query = $this->Properties->find()->where(['fooElement' => $fooElement]);
//The fooFunction needs an array cause for an internal call of cakes HASH::NEST function
$data= $this->FooModel->_fooFunction($query->enableHydration(false)->toList();
//Error: Not a paginable object
$data = $this->paginate($data)
$this->set(compact('fooElement', 'data'));
$this->set('_serialize', ['data']);
if (empty($fooElement)) {
$this->render('otherView');
}
}
EDIT: Here is the fooFunction:
public function _fooFunction($data)
{
$out = [];
$cache = [];
$nested = Hash::nest($data, ['idPath' => '{n}.id', 'parentPath' => '{n}.parent_id']);
$out = $this->_setOrderAndLevel($nested);
return $out;
}
protected function _setOrderAndLevel($items, $level = 0, $number = 0)
{
$out = [];
$items = Hash::sort($items, '{n}.orderidx');
foreach ($items as $item) {
$item['level'] = $level;
if (!empty($item['children'])) {
$children = $item['children'];
unset($item['children']);
$out[] = $item;
$out = array_merge($out, $this->_setOrderAndLevel($children, $level + 1));
} else {
$out[] = $item;
}
}
return ($out);
}
The _fooFunction takes the casted database query, makes some adjustments, adds two new properties and returns a nested Array. It maps id with parent_id in order to get children and a level description. The level description will be used for indentations in the view to display a hierarchical order.
IMPORTANT NOTICE: I am already beware of TreeBehavior in CakePHP but the problem is that our database has no left/right fields and I am not able to add them. Within this project I have to choose this way.
However $data contains exactly what I want but I need to transform it into a compatible object for pagination.
EDIT: Thanks to ndm I could build a paginable object with the necessary constraints. The last problem I still have in front of me is to merge all children and possible sub-children. A parent can have nth children and also a children can sometimes have nth sub-children. Therefore I solved this with a recursive call of my _setOrderAndLevel function within the fooFunction.
This is the current structure:
array(
[0] = fooEntity(
id = 1,
orderidx = 1,
parentId = null,
level = 0,
children(
id = 2,
orderidx = 2,
parentId = 1,
level = 1
children(
id = 3,
orderidx = 3,
parentId = 2,
level = 2
........
But it should be this:
array(
[0] = fooEntity(
id = 1,
orderidx = 1,
parentId = null
level = 0
[1] = fooEntity(
id = 2,
orderidx = 2,
parentId = 1,
level = 1
[2] = fooEntity(
id = 3,
orderidx = 3,
parentId = 2,
level = 2
........
I tried to build a second result formatter but it does not work:
...
return $results
->nest('id', 'parent_id', 'children')
->map($decorate);
})
->formatResults(function (\Cake\Collection\CollectionInterface $results) {
return $results->map(function ($data) {
call_user_func_array('array_merge', $data);
});
});
Maybe a "combine->" call could be the solution but I am not sure.
Any help is welcome
Generally if you need to format the results in some way, you should most likely use a result formatter, in order to be able to keep query object intact, and rom looking at the resulting format that your function produces, that is what you should use in this case, a result formatter.
If you need the ordering you could do that on SQL level already, and for nesting the results you could use the result collection's nest() method, ie you could ditch using the Hash class:
$query = $this->Properties
->find()
->where(['fooElement' => $fooElement])
->order(['orderidx' => 'ASC'])
->formatResults(function (\Cake\Collection\CollectionInterface $results) {
$fold = function ($rows, $level = 0) use (&$fold) {
$folded = [];
foreach ($rows as $row) {
$row['level'] = $level;
$children = $row['children'] ?: null;
unset($row['children']);
$folded[] = $row;
if ($children) {
$folded = array_merge(
$folded,
$fold($children, $level ++)
);
}
}
return $folded;
};
$nested = $results->nest('id', 'parent_id', 'children');
$folded = $fold($nested);
return collection($folded);
});
Note that you must return an instance of \Cake\Collection\CollectionInterface from the result formatter. The docs say that returning an(y) iterator would be enough, but as soon as there are additional formatters appended that expect a collection, things would break.
See also
Cookbook > Database Access & ORM > Query Builder > Adding Calculated Fields
Cookbook > Collections > Working with Tree Data

Codeigniter $query->result() loop to associative array

I'm having trouble getting this loop to return all of the rows that I know mysql is generating in this query. The objects in my javascript console are returning as I would like them to; however, I'm only getting the last row from each unique $row->student_id. I can tell that each unique row->student_id is being overwritten by the previous, but I'm not sure how else to do this logic.
This is part of a Codeigniter model. My controller is performing a json_encode function on this function, then I'm pulling this data into my view with jQuery $.ajax wired to a handlebars.js template. I'm aware that I could use ember for data bindings, but that is beyond my expertise at this time.
$query = $this->db->get();
if ($query->num_rows() > 0) {
$prev = null;
foreach ($query->result() as $row) {
if ($prev != $row->student_id) {
$data[$row->student_id] = array(
'first' => $row->last_name,
'last' => $row->first_name);
$data[$row->student_id]['dorc'] = array($row->debit_or_credit);
$prev = $row->student_id;
} else {
$data[$row->student_id]['dorc'][] += $row->debit_or_credit;
}
}
return $data;
} else {
return array();
}`
Your second if condition is not correct. You should replace it by this :
foreach ($query->result() as $row) {
if (!isset($data[$row->student_id])) { // Creates the row if not exists
$data[$row->student_id] = array(
'first' => $row->last_name,
'last' => $row->first_name);
}
if (!isset($data[$row->student_id]['dorc'])) { // Create the dorc key if not exists
$data[$row->student_id]['dorc'] = array($row->debit_or_credit);
} else { // if it exists, push the debit_or_credit row value
$data[$row->student_id]['dorc'][] = $row->debit_or_credit;
}
}

Parent child Relationship in mysql

I am having a table like the following,need to display as Parent and child format
--------------------------------------------------------
id role_name role_id parent_id
--------------------------------------------------------
1 NSM 1 0
2 MR 5 2
3 ASM 4 3
4 ZSM 3 4
5 RSM 2 1
---------------------------------------------------------
the result is like to be the following
NSM
---RSM
-----ZSM
-----NSM
-----MR
NSM->ROOT
RSM->FIRST CHILD
ZSM->SECOND CHILD
NSM->THIRD CHILD
MR->LEAF
// Fetch all the roles
$result = mysql_query("select * from roles");
$roles = array();
while( $role = mysql_fetch_assoc($result) ) {
$roles[] = $role;
}
// Function that builds a tree
function build_tree($roles, $parent_id=0) {
$tree = array();
foreach ($roles as $role) {
if ($role['parent_id'] == $parent_id) {
$tree[] = array(
'role' => $role,
'children' => build_tree($roles, $role['parent_id'])
);
}
}
return $tree;
}
// Function that walks and outputs the tree
function print_tree($tree) {
if (count($tree) > 0) {
print("<ul>");
foreach($node in $tree) {
print("<li>");
htmlspecialchars($node['role']['role_name']);
print_tree($node['children']);
print("</li>");
}
print("</ul>");
}
}
SQL Results are always flat - you'll not be able to return a hierarchy view of that data in a query.
Instead, I would suggest using whichever client components you are using to show that (is it a tree? what exactly?) that knows how to go thru a flat list and build a hierarchy out of that.
If you want to print a view like that in a console (why would you ever want to do that?), you could do like this:
$data = array();
$query = mysql_query("SELECT * FROM table ORDER BY parent_id");
while($array = mysql_fetch_assoc($query))
{
$data[$array['parent_id']][] = $array;
}
function output_hierarchy($id, $prepend)
{
$current = $data[$id];
foreach($current as $item)
{
print $prepend . " " . $item['role_name'];
if(count($data[$item['id']]) > 0)
{
output_hierarchy($item['id'], $prepend . "--");
}
}
}
output_hierarchy(0, '');
If you want to use this on your website, you can easily adapt it. Code should be self-explanatory.

Merging data structures

I have a set of structured data, that I'm trying to merge together, based on a set of conditions.
There is a set of rows, returned from the db. Each row has a course id, and a discipline id. There's an equal amount of disciplines in each course, but some disciplines are repeated in both courses.
I want to build a data structure where if the discipline is in both courses, then it only appears once on a line in a new data structure, and nothing else, but if there are two unmatched disciplines, then they are both included in a new course.
The code I'm using so far, is filtering and removing the keys that are duplicated, and adding them to a new array. This works fine.
However, because I can't control the order in which the data comes (don't ask) I'm having troubles making sure that each line has either a discipline that appears in both courses, or one of each.
I think I need some techniques to help deal with this, and was wondering if anyone had come across this before. I want to avoid making many consecutive loops, if possible.
Thanks.
edit: messy and horrible code below:
function buildDisciplineMap(){
$sql = "SELECT [idcurso]
,[idversao]
,[ordem]
,[bloco]
,[obsbloco]
,[iddisciplina] as idd
,cast([iddisciplina] as TEXT) as iddisciplina
,[descdisciplina]
,[iddepartamento]
,[descdepartamento]
,[ects]
,[horas]
,[idregente]
,[regente]
,[idregente1]
,[regente1]
,[idregente2]
,[regente2]
,[idregente3]
,[regente3]
,cast([objectivos] as TEXT) as objectivos
,cast([programa] as TEXT) as programa
,[descdisciplina_en]
,cast([objectivos_en] as TEXT) as objectivos_en
,cast([programa_en] as TEXT) as programa_en
FROM [proffile2].[dbo].[vw_site_CursosDisciplinas_FEG]
where idcurso = '3512 GE' or idcurso = '3513 ECON' order by idcurso desc ";
$discs = $this->returnObject($sql);
$map = new stdClass();
// find blocos, and titles
foreach ($discs as $key => $value) {
if (isset($map->bloco[$value->bloco])) {
// block already exists
} else {
#echo "making new block";
$map->bloco[$value->bloco] = new stdClass();
}
if (strlen($value->obsbloco)>1) {
$map->bloco[$value->bloco]->title = $value->obsbloco;
}
}
foreach ($map->bloco as $keybloco => $value) {
$map->bloco[$keybloco]->lines = array();
$processed_ids = array();
foreach ($discs as $kd => $vd) {
if ($vd->bloco == $keybloco) {
// check if this discipline occurs more than once in this block
foreach ($discs as $kdd => $vdd) {
if ($vdd->iddisciplina == $vd->iddisciplina && $kdd != $kd && !in_array($kd,$processed_ids) && !in_array($kdd,$processed_ids)) {
// this discipline is for both courses
$details = array();
$details['both'] = $vd;
$map->bloco[$keybloco]->lines[] = $details;
array_push($processed_ids, $kd, $kdd);
unset($discs[$kdd]);
unset($discs[$kd]);
break;
}
}
}
}
}
$processed_ids = array();
foreach ($discs as $key => $value) {
echo $value->idcurso."\n";
}
foreach ($discs as $kd => $vd) {
$bloco = $vd->bloco;
$lastidx =sizeof($map->bloco[$bloco]->lines)-1;
$line = $map->bloco[$bloco]->lines[$lastidx];
echo sizeof($map->bloco[$bloco]->lines);
#pr($line);
if (isset($line['both'])) {
echo "omog - \n ";
$map->bloco[$bloco]->lines[][$vd->idcurso] = $vd;
unset($discs[$kd]);
continue;
}
#pr($line['both']->idcurso);
foreach ($map->bloco[$bloco]->lines as $k => $v) {
echo $k."-";
#echo $v['3513 ECON']->idcurso;
}
if ($line[$vd->idcurso]) {
echo 'add';
$map->bloco[$bloco]->lines[][$vd->idcurso] = $vd;
} else {
echo 'fill';
$map->bloco[$bloco]->lines[sizeof($map->bloco[$bloco]->lines)-1][$vd->idcurso] = $vd;
}
}
echo sizeof($discs);
return $map;
}
You said "don't ask", but I've got to: why can't you control the order of the rows? Aren't you the one writing the database query?
If you think fixing the order of the rows will help you parse and build a new data structure better, why not make sorting the rows the first step in your process? Or, is the data set too large?
You might find some of PHP's array set manipulations to be of use. i.e. array_diff, array_intersect_key etc.

how to loop over looped elements in Php?

I'm creating a tree-structure of categories with parentid's which can be called from children in such a way:
ID | Name | ParentID
1 1 0
2 2 1
3 3 2
4 4 1
Resulting in this:
1 = 1
2 = 1 -> 2
3 = 1 -> 2 -> 3
4 = 1 -> 4
which means 3 is a child of 2, which is a child of 1.
when trying to get this idea (with the -> to show what relations are set) I only get to the second grade (1 -> 2) but not to the third (1->2->3) because of the looping function I use for it.
//put all ID's in an array
while ($row2 = $connector->fetchArray($result2)){
$id = $row2['ID'];
$parents[$id] = $row2['name'];
}
// show the tree-structure
while ($row = $connector->fetchArray($result)){
if($row['parentid']!=0)echo $parents[$row['parentid']].' -> ';
echo $row['name'].' - ';
echo '<br>';
}
I'd like two things to change:
have the code automatically generate a tree sized as necessary.
in the while-loops i have to select the $result twice (once as $result, once as $result2) to make it work. these $result's have exactly the same database-query:SELECT ID,name,parentid FROM categories
to fetch results from. I'd like to only declare this once.
Thanks for all the good answers. I've gone with the easiest, less-code-to-implement approach:
$result = $connector->query('SELECT ID,name,parentid FROM categories');
// Get an array containing the results.
$parents = array();
while ($row = $connector->fetchArray($result)){
$id = $row['ID'];
$parents[$id] = array('ID' => $row['ID'],'name' => $row['name'],'parentid' => $row['parentid']);
}
foreach ($parents as $id => $row){
$pid=$id;
$arrTmp= array();
do { // iterate through all parents until top is reached
$arrTmp[]=$pid;
$pid = $parents[$pid]['parentid'];
}while ($pid != 0);
$arrTmp = array_reverse($arrTmp);
foreach($arrTmp as $id){
echo $parents[$id]['name'].' -> ';
}
echo '<br>';
}
Rather than have PHP organize the items into a tree, why not ask the database to do it for you? I found this article on hierarchical data to be very good and the examples are almost identical to yours.
EDIT
The SQL for getting the full tree using the Adjacency Model is not ideal. As the article explains it requires rather a lot of joins for even a small hierarchy. Is it not possible for you to use the Nested Set approach? The SQL stays the same regardless of the size of the hierarchy and INSERT and DELETE shouldn't be very difficult either.
If you really want to do hierachies with parent ids(suitable only for small number of items/hierachies)
I modified your code a little bit(I did not test it so there may be some syntax errors):
//put all recordsets in an array to save second query
while ($row2 = $connector->fetchArray($result2)){
$id = $row2['ID'];
$parents[$id] = array('name' => $row2['name'],'parent' => $row2['parentid']);
}
// show the tree-structure
foreach ($parents as $id => $row){
$pid = $row['parentid'];
while ($pid != 0){ // iterate through all parents until top is reached
echo $parents[$pid]['name'].' -> ';
$pid = $parents[$pid]['parentid'];
}
echo $parents[$id]['name'].' - ';
echo '<br>';
}
To answer your comment:
$parents = array();
$parents[2] = array('ID'=>2,'name'=>'General','parentid'=>0);
$parents[3] = array('ID'=>3,'name'=>'Gadgets','parentid'=>2);
$parents[4] = array('ID'=>4,'name'=>'iPhone','parentid'=>3);
foreach ($parents as $id => $row){
$pid=$id;
$arrTmp= array();
do { // iterate through all parents until top is reached
$arrTmp[]=$pid;
$pid = $parents[$pid]['parentid'];
}while ($pid != 0);
$arrTmp = array_reverse($arrTmp);
foreach($arrTmp as $id){
echo $parents[$id]['name'].' -> ';
}
echo '<br>';
}
Prints out:
General ->
General -> Gadgets ->
General -> Gadgets -> iPhone ->
Maybe easier with OOP. Just sort the query by parentId
Note: The listChildren method and the printout at the bottom is just there to show it is listed correctly. I did not interpret the question that the display was important.
class Element {
public $id;
public $name;
public $parent = null;
public $children = array();
public function __construct($id, $name)
{
$this->id = $id;
$this->name = $name;
}
public function addChild($element)
{
$this->children[$element->id] = $element;
$element->setParent($this);
}
public function setParent($element)
{
$this->parent = $element;
}
public function hasChildren()
{
return !empty($this->children);
}
public function listChildren()
{
if (empty($this->children)) {
return null;
}
$out = array();
foreach ($this->children as $child) {
$data = $child->id . ':' . $child->name;
$subChildren = $child->listChildren();
if ($subChildren !== null) {
$data .= '[' . $subChildren . ']';
}
$out[] = $data;
}
return implode(',', $out);
}
}
$elements = array();
$noParents = array();
while ($row = $connector->fetchArray($result)) {
$elements[$row['id']] = $element = new Element($row['id'], $row['name']);
if (isset($elements[$row['parent']])) {
$elements[$row['parent']]->addChild($element);
} else {
$noParents[] = $element;
}
}
foreach ($noParents as $element) {
if ($element->hasChildren()) {
echo "Element {$element->id} has children {$element->listChildren()}.\n";
} else {
echo "Element {$element->id} has no children.\n";
}
}
If you are using PostgreSQL as the database, you can use the connectby() function to create the record set:
SELECT *
FROM connectby('tableName', 'id', 'parent_id')
AS t(keyid text, parent_keyid text, level int);
I love this function, and use all the time in my code. It can do some very powerful things, very quickly, and you don't have maintain the left/right values like the (adjacency model).

Categories