Optimizing multiple queries in MySQL and PHP - php

Questions
How should I do the query(ies) to get this results?
Should I use a different structure for database tables?
Details
I want to get results from 3 tables:
+------------------------------+-------------------+
| courses | id | <-------+
| | name | |
| | | |
+------------------------------+-------------------+ |
| sections | id | <-------|----------+
| | course_id | <- FK(courses.id) |
| | name | |
+------------------------------+-------------------| |
| resources | id | |
| | section_id | <- FK(sections.id)-+
| | name |
+------------------------------+-------------------+
I want to store results in a PHP Array like this:
Array
(
[courses] => Array
(
[id] => 1
[name] => course 1
[sections] => Array
(
[0] => Array
(
[id] => 1
[course_id] => 1
[name] => course 1 section 1
[resources] => Array
(
[0] => Array
(
[id] => 1
[section_id] => 1
[name] => resource 1
)
)
)
)
)
)
EDIT
What I did:
$cources = DB::query(Database::SELECT,
'select * from courses')->execute($db,false)[0]; // Get all courses as array
foreach($courses as &$course) {
$sections = DB::query(Database::SELECT,
'select * from sections where course_id = '.$courses['id']);
$course['sections'] = $sections;
foreach($course['sections'] as &&section) {
$resources = DB::query(...); // Get array of resources
$section['resources'] = $resources;
}
}

The database structure is normalized - this is correct and should not be changed.
However, SQL returns de-normalized or "flattened" data for an N+ join: only a set of homogenous records can be returned in a single result-set. (Some databases, like SQL Server, allow returning structure by supporting XML generation.)
To get the desired array structure in PHP will require:
Separate queries/result-sets (as shown in the post): ick!
There will about one query/object. While the theoretical bounds might be similar, the practical implementation will be much less efficient and the overhead will be much more than for single query. Remember that every query incurs (at the very least) a round-trip penalty - as such, this is not scalable although it will likely work just fine for smaller sets of data or for "time insensitive" operations.
Re-normalize the resulting structure:
This is very trivial to do with support of a "Group By" operation, as found in C#/LINQ. I am not sure how this would be approached [easily] in PHP1. This isn't perfect either, but assuming that hashing is used for the grouping, this should be able to scale fairly well - it will definitely be better than #1.
Instead of the above, consider writing the query in such a way that the "flat" result can be used within the current problem/scope, if possible. That is, analyze how the array is to be used - then write the queries around that problem. This is often a better approach that can scale very well.
1 Related to re-normalizing the data, YMMV:
SQL result to PHP multidimensional array
PHP array to multidimensional array
Group a multidimensional array by a particular value?

You can try something like this
SELECT * FROM (
select c.id, c.name from courses c
union
select s.id, r.name,s.course_ID from sections s
union
select r.id, r.name,r.section_ID from resources r
)

You cant get multi dimensional result from mysql. The query for getting the elements should be like this:
select courses.id as coursesId,courses.name as coursesName,sections.id as sectionsId,sections.name as sectionsName,resources.id as resourcesId, resources.name as resourcesName
from courses
left join sections on courses.id=sections.course_id
left join resources on sections.id=resources.section_id;
But ofcourse it will not give you the array as you like.

if you are familiar with php then you can use this code i am writing only 2nd level you can write same way with third label
$final=array();
$c=-1;
$cid=false;
$cname=false;
$query = "SELECT c.*,s.*,r.* FROM courses AS c LEFT JOIN sections AS s ON c.id=s.course_id LEFT JOIN resources AS r ON r.section_id =s.id";
$result=mysql_query($query, $this->con) or die(mysql_error());
while($row= mysql_fetch_array($result)){
if($cid!=$row[2]){
$final['cources'][++$c]['id']=$cid=$row[0];
$final['cources'][$c]['name']=$cname=$row[1];
$s=-1;
}
$final['cources'][$c]['sections'][++$s]['id']=$row[2];
$final['cources'][$c]['sections'][$s]['course_id']=$row[3];
$final['cources'][$c]['sections'][$s]['name']=$row[4];
}
echo "<pre>";
print_r($final);
echo "</pre>";
//Outpur
Array
(
[cources] => Array
(
[0] => Array
(
[id] => 1
[name] => c1
[sections] => Array
(
[0] => Array
(
[id] => 1
[course_id] => 1
[name] => s1-1
)
[1] => Array
(
[id] => 1
[course_id] => 1
[name] => s1-1
)
)
)
[1] => Array
(
[id] => 2
[name] => c2
[sections] => Array
(
[0] => Array
(
[id] => 2
[course_id] => 2
[name] => s1-2
)
)
)
)
)

Related

How do I get MySQL query result as nested array using PHP?

Let's say I have this simple table where pos_level is the level of the position and pos_under is that the PID of the position on top of it.
In level 1, I have 'General Manager' and 'Supervisor'.
In level 2, under 'General Manager(PID: 1)' : 'Asst. Manager', under 'Supervisor(PID: 2)' : 'Marketing'.
In level 3, under 'Asst. manager(PID: 3)' : 'Sales' & 'Purchase', under 'Marketing(PID:2) : none.
+-----+-----------------+-----------+-----------+
| PID | pos_name | pos_level | pos_under |
+-----+-----------------+-----------+-----------+
| 1 | General Manager | 1 | 0 |
| 2 | Supervisor | 1 | 0 |
| 3 | Asst. Manager | 2 | 1 |
| 4 | Sales | 3 | 3 |
| 5 | Purchase | 3 | 3 |
| 6 | Marketing | 2 | 2 |
+-----+-----------------+-----------+-----------+
Now how do I make the query so I get a nested array as the result like this:
Array
(
[0] => Array
(
[pos_level] => 1
[pos_name] => General Manager
[pos_under] => Array
(
[0] => Array
(
[pos_level] => 2
[pos_name] => Asst. Manager
[pos_under] => Array
(
[0] => Array
(
[pos_level] => 3
[pos_name] => Sales
)
[1] => Array
(
[pos_level] => 3
[pos_name] => Purchase
)
)
)
)
)
[1] => Array
(
[pos_level] => 1
[pos_name] => Supervisor
[pos_under] => Array
(
[0] => Array
(
[pos_level] => 2
[pos_name] => Marketing
[pos_under] => Array
(
)
)
)
)
)
I have tried using multiple queries and using array_push using the results, but I have like around 100+ pos_name and I think it is messy, I have also tried using loop to keep running queries for each level and under, also tried using multiple tables for each level, but I am hoping I can use only 1 table and able to query the result as the nested array above for further use in my application.
All answers, comments, and suggestions are very welcomed. Thank you.
As mentioned above in the comments I doubt in general that the approach you are following is a good one. That is because it will not scale! You try to take all entries from your database and pack them into a single, hierarchical array. That means you create several copies of each entry in memory. What will happen with a growing number of entries? You face a constantly raising memory consumption of your script which will obviously make it fail ultimately.
Nevertheless I accepted the challenge and implemented a small demonstration of how to cleverly construct such nested structures with the help of references. Such references not only simplify the creation of nested structures, they also reduce the memory footprint by preventing that entries are copied by value again and again. This however will not solve the general scaling issue with this approach mentioned above, it can only help to reduce it.
I also took the liberty to make a small modification to your approach and introduce a new property "pos_over" to hold entries under an entry. Two reasons for that:
it usually is not a good idea to remove original available data and especially not to replace it with an altered meaning
the meaning of the term "pos_under" is to describe under what other entry an given entry is (to be) placed. But from the "upper" entries point of view the ones "below" it should not be referenced by "pos_under", right? That would contradict the original meaning of that term.
The following code does not depend on a database for demonstration purpose. Instead it reads the raw data from a CSV string and parses that. The actual building of the desired structure structure is marked further down by a comment. Also it should be mentioned that the code expects the original data to reference other entries only after they have been declared. So if processed from top to bottom each entry under another one can expect that other one to already exist.
Note, that the actual code to build that nested structure consists of only a mere 12 clever lines...
<?php
// the raw CSV data
$csvText = <<<CSV
PID|pos_name|pos_level|pos_under
1|General Manager|1|0
2|Supervisor|1|0
3|Asst. Manager|2|1
4|Sales|3|3
5|Purchase|3|3
6|Marketing|2|2
CSV;
// praparation of CSV data
foreach (explode("\n", $csvText) as $csvRow) {
$csvData[] = str_getcsv($csvRow, '|');
}
$csvTitle = array_shift($csvData);
$table = [];
foreach ($csvTitle as $titleKey=>$titleValue){
foreach ($csvData as $csvRow=>$csvColumn) {
foreach ($csvColumn as $csvKey=>$csvValue) {
$table[$csvRow][$csvTitle[$csvKey]] = $csvValue;
}
}
}
// creation of the structure
$catalog = [];
$structure = [];
foreach ($table as $key=>&$entry) {
$entry['pos_over'] = [];
if ($entry['pos_under'] == 0) {
$structure[] = &$entry;
$catalog[$entry['PID']] = &$structure[count($structure)-1];
} else {
$catalog[$entry['pos_under']]['pos_over'][] = &$entry;
$catalog[$entry['PID']] = &$catalog[$entry['pos_under']]['pos_over'][count($catalog[$entry['pos_under']]['pos_over'])-1];
}
}
// demonstration output
print_r($structure);
The output of above experiment is:
Array
(
[0] => Array
(
[PID] => 1
[pos_name] => General Manager
[pos_level] => 1
[pos_under] => 0
[pos_over] => Array
(
[0] => Array
(
[PID] => 3
[pos_name] => Asst. Manager
[pos_level] => 2
[pos_under] => 1
[pos_over] => Array
(
[0] => Array
(
[PID] => 4
[pos_name] => Sales
[pos_level] => 3
[pos_under] => 3
[pos_over] => Array
(
)
)
[1] => Array
(
[PID] => 5
[pos_name] => Purchase
[pos_level] => 3
[pos_under] => 3
[pos_over] => Array
(
)
)
)
)
)
)
[1] => Array
(
[PID] => 2
[pos_name] => Supervisor
[pos_level] => 1
[pos_under] => 0
[pos_over] => Array
(
[0] => Array
(
[PID] => 6
[pos_name] => Marketing
[pos_level] => 2
[pos_under] => 2
[pos_over] => Array
(
)
)
)
)
)

Instead of php while loop and multiple connections to mysql want to connect only once

Example of mysql is here http://sqlfiddle.com/#!9/68653/2
Mysql table (named topics)
TopicId | TopicName | ClosestUpperLevelId
--------------------------------------------
1 | Books | 0
2 | BooksAboutCss | 1
3 | BooksAboutHtml | 1
4 | BooksAboutCss1 | 2
5 | BooksAboutCss2 | 2
6 | BooksAboutHtml1 | 3
7 | BooksAboutHtml2 | 3
8 | E-Books | 0
9 | Magazines | 0
For top level topics ClosestUpperLevelId is 0. For subtopics ClosestUpperLevelId is TopicId of closest upper level topic
(TopicId - ClosestUpperLevelId)
Books (1-0)
BooksAboutCss (2-1)
BooksAboutCss1 (4-2)
BooksAboutCss2 (5-2)
BooksAboutHtml (3-1)
BooksAboutHtml1 (6-3)
BooksAboutHtml2 (7-3)
E-Books (8-0)
Magazines (9-0)
For example, i have created one page and location of the page is domain.com/Books/BooksAboutCss/BooksAboutCss2
Now i decided to edit the page. For example i want to edit location (topic or category) of the page and set it to domain.com/Magazines. So i need to fetch all topics, related with existing (saved) page. Then will create select boxes for each group (level) of topics.
At the moment tried to use php while and multiple times to connect to mysql and get data. Like below code. How can i get the same result without php while? How connect to mysql only once and fetch all necessary data? Do i need to use mysql while https://dev.mysql.com/doc/refman/5.1/en/while.html?
$topic_names_1[0]['UpperLevelNumberRenamed'] = 5;//just set some value to start to fetch
while ( trim($topic_names_1[0]['ClosestUpperLevelId']) != 0 ){
try {
$stmt_1 = $db->prepare('
SELECT `TopicId`, `TopicName`, `ClosestUpperLevelId`
FROM `topics`
WHERE `ClosestUpperLevelId` =
(
SELECT `ClosestUpperLevelId`
FROM `topics`
WHERE `TopicId` = ?
)
;');
$stmt_1->execute( array( trim($topic_names_1[0]['UpperLevelNumberRenamed']) ) );
$topic_names_1 = $stmt_1->fetchAll(PDO::FETCH_ASSOC);
echo '<pre>', print_r($topic_names_1), ' topic_names_1 __</pre>';
}//try {
catch (PDOException $e){
echo "<br> stmt_1 DataBase Error: " .htmlspecialchars( $e->getMessage() , ENT_QUOTES, "UTF-8").'<br>';
}
catch (Exception $e) {
echo " stmt_1 General Error: ".htmlspecialchars( $e->getMessage() ).'<br>';
}
}//while ( trim($topic_names[0]['UpperLevelNumberRenamed']) != 0 )
As result get arrays like this
Array
(
[0] => Array
(
[TopicId] => 4
[TopicName] => BooksAboutCss1
[ClosestUpperLevelId] => 2
)
[1] => Array
(
[TopicId] => 5
[TopicName] => BooksAboutCss2
[ClosestUpperLevelId] => 2
)
)
1 topic_names_1 __
Array
(
[0] => Array
(
[TopicId] => 2
[TopicName] => BooksAboutCss
[ClosestUpperLevelId] => 1
)
[1] => Array
(
[TopicId] => 3
[TopicName] => BooksAboutHtml
[ClosestUpperLevelId] => 1
)
)
1 topic_names_1 __
Array
(
[0] => Array
(
[TopicId] => 1
[TopicName] => Books
[ClosestUpperLevelId] => 0
)
[1] => Array
(
[TopicId] => 8
[TopicName] => E-Books
[ClosestUpperLevelId] => 0
)
[2] => Array
(
[TopicId] => 9
[TopicName] => Magazines
[ClosestUpperLevelId] => 0
)
)
1 topic_names_1 __
Update
Found one example with mysql while. Trying to create own code
Created this
SET `ClosestUpperLevelId` := 2;
WHILE `ClosestUpperLevelId` > 0
DO
SELECT `TopicId`, `TopicName`, `ClosestUpperLevelId`;
END WHILE;
and this (http://sqlfiddle.com/#!9/68653/7)
CREATE PROCEDURE dowhile()
BEGIN
DECLARE `ClosestUpperLevelId` INT DEFAULT 2;
WHILE `ClosestUpperLevelId` > 0 DO
SELECT `TopicId`, `TopicName`, `ClosestUpperLevelId`;
END WHILE;
END;
Got error...
You could just move your select statement before the while loop and take out the topic_id from the where clause. Then in your loop retrieve from the complete record set using an array search function or similar. By running same query once your dB will be able to cache results
Relational databases like MySQL aren't naturally good at storing hierarchical data, but there are ways of doing it. The two ways I know of are "adjacency lists" and "nested sets".
For an adjacency list, you'd simply store a "parent_id" field for each row. Root rows (that have no parent) can just have NULL for their parent_id field. Adjacency lists are easy to manage, but not very flexible and require recursive queries to find a path from root to leaf (unless you're only going 2 levels deep, then you can simply JOIN).
Here's an example:
id | parent_id | name
0 NULL grandfather
1 0 father
2 1 grandson
3 1 granddaughter
This query would help assemble the data:
SELECT * FROM
`people` AS p1
JOIN `people` p2
ON p1.id = p2.parent_id
Any more than those two levels and you need recursion. Alternatively, you could just query the entire table and assemble it in code.
Nested sets are a little more complicated, but allow you to easily query all the way up the tree for a given leaf node. It's much easier to understand nested sets at first by seeing a visual, check this out:
https://en.wikipedia.org/wiki/Nested_set_model
And here's what your schema would look like:
left | right | name
0 7 grandfather
1 6 father
2 3 grandson
4 5 granddaughter
And here's an example to fetch father and children:
SELECT *
FROM `people`
WHERE `left` >= 1 AND `right` <= 6
Nested sets have the downside that the entire table's left and right values need to be updated when the hierarchy changes.
Google "managing hierarchical data in mysql" for more information. I hope this helps.

php (PDO) simple way to turn rows in lookup table into simple array

given a very simple table structure thus:
mysql> describe songpart;
+----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+---------+------+-----+---------+----------------+
| id | int(11) | NO | MUL | NULL | auto_increment |
| partName | text | NO | | NULL | |
+----------+---------+------+-----+---------+----------------+
which results in an array like this in php (when queried)
Array ( [0] => Array ( [id] => 1 [0] => 1 [partName] => Lead Guitar [1] => Lead Guitar )
[1] => Array ( [id] => 2 [0] => 2 [partName] => Bass Guitar [1] => Bass Guitar )
[2] => Array ( [id] => 3 [0] => 3 [partName] => Drums [1] => Drums )
[3] => Array ( [id] => 4 [0] => 4 [partName] => Keyboard [1] => Keyboard ) )
Am I missing some simple trick to turn this into a simple array with id as the key like so:
Array ( [1] => Lead Guitar
[2] => Bass Guitar
[3] => Drums
[4] => Keyboard )
or is it possible to get PDO to deliver an array like this?
TiA
You can simply use the PDO::FETCH_KEY_PAIR is you have only 2 columns in your result.
You can also use the PDO::FETCH_GROUP and PDO::FETCH_ASSOC together if you have more. Example :
$result = $db->query('select * from channels')->fetchAll(PDO::FETCH_GROUP|PDO::FETCH_ASSOC);
This will yield an array indexed with the first column containing at each index an array of the result for this key. You can fix this by using array_map('reset', $result) to get your goal.
Example :
$result = array_map('reset', $db->query('select * from channels')->fetchAll(PDO::FETCH_GROUP|PDO::FETCH_ASSOC));
Try this:
$records = $pdo->query('SELECT id, partName FROM myTable');
$records->setFetchMode(PDO::FETCH_KEY_PAIR);
print_r($records->fetchAll());
For that you need to set only your desired fields in iteration.
$part_name = [];
$records = $pdo->query('SELECT * FROM your_table');
foreach ($records as $row) {
$part_name[$row['id']] = $row['partName'];
}

mysql select where in array (php, mysql)

Lets say i have an array in my mysql row:
a:3:{i:1;a:3:{i:0;s:1:"1";i:1;s:1:"3";i:2;s:1:"5";}i:4;a:3:{i:0;s:2:"21";i:1;s:2:"25";i:2;s:2:"29";}i:5;a:1:{i:0;s:2:"33";}}
It looks like this:
Array
(
[1] => Array
(
[0] => 1
[1] => 3
[2] => 5
)
[4] => Array
(
[0] => 21
[1] => 25
[2] => 29
)
[5] => Array
(
[0] => 33
)
)
Now, i am passing an array through _GET and i want to print out all rows that contain same values both in my mysql and passed array. For example, if i pass this array:
Array
(
[1] => Array
(
[0] => 5
)
)
A result should be shown, because my passed array contains option 5. I tried to do it like this:
$pecul = serialize($array);
$q=mysql_query("SELECT id from table WHERE options like '%$pecul%'")or die(mysql_error());
but it only prints out results with identical arrays.
You probably want to unserialize the data that's in your database first. Once it's in a PHP array you can perform a check for those options, so say:
if(in_array("5",$array)) {
$q=mysql_query("SELECT id from table WHERE options=5")or die(mysql_error());
}
If you need to do a query for all of the options, you can do a loop like so:
foreach($array as $option) {
$q=mysql_query("SELECT id from table WHERE options='$option'")or die(mysql_error());
}
But like mario said, you may want to think the options mechanism over and perhaps serialized data isn't what you need. This should hopefully work for you the way it is though.
the like operator match only match same it does not understand for example you have dem in like than it will also show the demolish so
i think for that you need first unserlize and than find by php by either loop or in_array() function
what %like% do suppose this is table Persons and you used the query
SELECT * FROM Persons WHERE City LIKE '%tav%'
+-------------+-------------+
| id | city |
+-------------+-------------+
| 1 | Sandnes |
+-------------+-------------+
| 2 | Stavanger |
+-------------+-------------+
so the result will be
+-------------+-------------+
| id | city |
+-------------+-------------+
| 2 | Stavanger | <----it has tav (s-tav-anger)
+-------------+-------------+
IN() Check whether a value is within a set of values
mysql> SELECT 2 IN (0,3,5,7);
-> 0
mysql> SELECT 'wefwf' IN ('wee','wefwf','weg');
-> 1
SELECT val1 FROM tbl1 WHERE val1 IN (1,2,'a');
View: IN MySql

How to arrange the obtained result's Array given Below, to get a desired result given below?

I Have a query
SELECT classid, COUNT(*) as cnt FROM tbl_name GROUP BY classid
on Table
| id | classid | contextid |
1 1 2
2 1 1
3 2 1
4 1 1
this will yields me the result by the use of an inbuilt library function moodle as
Array
(
[1] => stdClass Object
(
[classid] => 1
[classcnts] => 3
)
[2] => stdClass Object
(
[classid] => 2
[classcnts] => 1
)
)
I need the result in an array in the form of
Array(
[classid]=>[classcnts]
)
i.e
Array(
1=>3,
2=>1
)
So how can i arrange the available array to find the required array.
I am working in PHP
Thanks
try:
foreach($arr as $k){
$new[$k->classid] = $k->classcnts
}

Categories