There are lots of articles on getting distinct rows through a sql statement but I havent found the help I need after a SQL statement has already run. Im very new to PHP and I am using the DHTMLX library to render a grid. The unusual syntax used to render the grid is throwing me off big time. What I want is simply distinct rows in my grid only. Everything works great, I just need help getting a function together that will return distinct rows. In my code below I managed to get the two functions that are commented out to work great so I have been using them as a roadmap but I just can't pull the "distinct function" together due to a lack of familiarity with loops. Can someone help me with the syntax needed to have this function return distinct rows. I think array_unique will come into play but Im pretty lost at this point.
<?php
require("codebase/grid_connector.php");
$res = mysql_connect("localhost", "1newuser", "");
mysql_select_db("supplydb");
function distinct($result)
{
$grab = array_unique($rows->get_value("group"));
}
/*function formatting($row){
$data = $row->get_value("gpo_item");
if ($data == 1)
$row->set_value("gpo_item",Yes);
else
$row->set_value("gpo_item",No);
} */
/*function calck($action){
$data1 = $action->get_value("list_price");
$data2 = $action->get_value("sugg_price");
$sum = (($action->get_value("sugg_price") / $action->get_value("list_price")) - 1) * 100 ;
$sum2 = round($sum);
$sum3 = abs($sum2);
$action->set_value("discount",$sum3);
} */
$gridConn = new GridConnector($res, "MySQL");
//$gridConn->event->attach("beforeRender","formatting");
//$gridConn->event->attach("beforeRender","calck");
$gridConn->event->attach("beforeRender", "distinct");
$gridConn->render_sql("SELECT * FROM manufacturers JOIN submissions on manufacturers.manufacturer_id=submissions.manufacturer_id JOIN products on products.product_id=submissions.product_id JOIN product_group on submissions.category=product_group.id", "submission_id", "date,man_name,group,requesting_clinician, requesting_clinician_email, contract_number");
?>
All array_unique does is eradicates duplicate values within a given array.
Why can you not just specify a distinct lookup within your sql?
Related
I can't imagine I'm the first or the last to ask about this, but I was unable to find the answer by searching.
I have a large dataset of orders in SQL Server. I need to return every order with each of their line items and each of their payments. I'm looking for the most efficient way to query and iterate through this data. I am using PHP/MSSQL to pull the orders, looping through those with foreach and querying for a list of items and payments would be incredibly inefficient.
I've thought about creating a union to create one massive dataset that has a bunch of columns, one of which is a column indicating what type of record (payment,line_item,order_data). But that also seems pretty inefficient.
This is an outline of the process I have been working with:
<?php
// query for orders
$records = sql_function("select * from v_migration_orders");
$json_output = array();
foreach($record as $r){
$data['order']['org_code'] = $r['org_code'];
$data['order']['po_number'] = $r['po_number'];
...
// query for line items
$data['order']['line_items'] = array();
$items = sql_function("select * from v_migration_line_items where invoice = ".sanitize_function($r['invoice']));
foreach($items as $i){
$line['line_item_type'] = $i['line_item_type'];
$line['sku'] = $i['sku'];
$line['legacy_id'] = $i['legacy_id'];
...
array_push($data['order']['line_items'],$line);
}
// query for payments
$data['order']['payments'] = array();
$payments = sql_function("select * from v_migration_payments where invoice = ".sanitize_function($r['invoice']));
foreach($payments as $p){
$pyt['Amount'] = $p['Amount'];
$pyt['payment_type'] = $p['payment_type'];
$pyt['Check_Number'] = $p['Check_Number'];
...
array_push($data['order']['payments'],$pyt);
}
array_push($json_output,$data);
}
echo json_encode($json_output);
With a focus on efficiency, I'm hoping someone in the community can help point me in the right direction here. The number of calls from the web server to the database server are triple what I'd like them to be. The alternative I can think of is to find a way to place them all into a single trip to the database and have PHP parse the full response once it receives it.
Below is the structure of the needed data I've put into views. Note that I'm using the integer, Invoice as the key to join on.
v_migration_orders:
line_type (value is: "Order")
Org_Code
PO_Number
Invoice
Tax_Total
Order_Total
EMAIL
status
order_t_stamp
order_legacy_id
v_migration_orders_line_items:
line_type (value is "line_item")
line_item_type
Invoice
sku
legacy_id
TITLE
quantity
unit_price
v_migration_orders_payments:
line_type (value is "payment")
Invoice
Amount
payment_type
Check_Number
Card_Processor
cc_message
card_type
cardholder_name
card_expiry
payment_token
payment_t_stamp
Thanks in advance!
Hello stackoverflow, long time lurker first time asker. Anyways I am creating a project for my school and it is almost completely finished but I think I need help with the while loop. I am trying to sum the variables that are stored in the array and then output that into a useable variable that I can sum with another variable. The problem is the loop only does 1 iteration every time I refresh to page. So it will eventually get the full array but only one item at a time. Please let me know what I am going wrong, I'm sure its something dumb!
$bill= mysql_query("SELECT Transaction.date, Transaction.Price, Transaction.customer_customer_id, Service.cost, Service.user_id, Service.uname
FROM *.Transaction,*.Service
WHERE Transaction.customer_customer_id = Service.user_id AND Service.uname = '$uuname'");
$query_row=mysql_fetch_array($bill);
$userservice = ($query_row[cost]);
$userprice = ($query_row[Price]);
$row2=array();
while($row = mysql_fetch_array($bill))
{
$row2[] += $row['cost'];
}
$totalservice = array_sum($row2);
Thanks for any help you guys may have. This one is frying my brain.
Both your SQL and your PHP make no sense.
FROM *.Transaction,*.Service
...will throw an error in MySQL, but your code doesn't check for errors. I suspect it should be:
FROM Transaction, Service
While the PHP will parse and run.....
while($row = mysql_fetch_array($bill)) {
$row2[] += $row['cost'];
}
$totalservice = array_sum($row2);
Is a very strange way to populate an array. Why not just....
while($row = mysql_fetch_array($bill)) {
$totalservice+=$row['cost'];
}
Indeed, if you are just throwing away the rest of the data, then why are you fetching it from the database?
SELECT SUM(Service.cost)
FROM Transaction,Service
WHERE Transaction.customer_customer_id = Service.user_id
AND Service.uname = '$uuname'
In which case the join is also redundant:
SELECT SUM(Service.cost)
FROM Service
WHERE Service.uname = '$uuname'
Rewrite query as
SELECT Service.user_id, Service.uname, SUM(Service.cost) as cost
FROM djqrico_hotel.Transaction LEFT JOIN djqrico_hotel.Service ON Transaction.customer_customer_id = Service.user_id
WHERE Service.uname = '$uuname' GROUP BY Service.user_id
if you want to get sum of all user's transactions.
The problem is the loop only does 1 iteration every time I refresh to page.
Have you ran the query against the database and checked if it's returning more than one row?
Also, if all you want to do is sum the cost column you could do that in sql:
SELECT SUM(cost)[....]
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Is it possible to query a tree structure table in MySQL in a single query, to any depth?
I have an admin area I created that pulls data from the mysql database using php and display the results in a table. Basically it shows a parent category, then the first sub category below it, then the third level sub category/subject.
It works perfectly but as I am new to mysql and php I am sure that it the code needs to be improved in order to save db resources as while building the table I use 3 while loops and in each loop make a mysql query which I am sure is the wrong way to do it.
Can somebody offer me some assistance for the best way of doing this?
Here is the code:
$query = mysql_query("SELECT * FROM categories WHERE
parent_id is null
order by cat_id asc;", $hd)
or die ("Unable to run query");
while ($row = mysql_fetch_assoc($query)) {
echo '<tr style="font-weight:bold;color:green;"><td>'. $row ['cat_id'].'</td><td>'.$row['cat_name'].'</td><td>'.$row ['parent_id'].'</td><td>'.$row['active'].'</td><td>'.$row ['url'].'</td><td>'.$row['date_updated'].'</td></tr>' ;
$query2 = mysql_query("SELECT * FROM categories WHERE
(active = 'true' AND parent_id = ".$row ['cat_id'].")
order by cat_id asc;", $hd)
or die ("Unable to run query");
while ($row2 = mysql_fetch_assoc($query2)) {
echo '<tr style="font-weight:bold;"><td>'. $row2['cat_id'].'</td><td>'.$row2 ['cat_name'].'</td><td>'.$row2['parent_id'].'</td><td>'.$row2 ['active'].'</td><td>'.$row2['url'].'</td><td>'.$row2 ['date_updated'].'</td></tr>' ;
$query3 = mysql_query("SELECT * FROM categories WHERE
(active = 'true' AND parent_id = ".$row2 ['cat_id'].")
order by cat_id asc;", $hd)
or die ("Unable to run query");
while ($row3 = mysql_fetch_assoc($query3)) {
echo '<tr><td>'. $row3['cat_id'].'</td><td>'.$row3['cat_name'].'</td><td>'.$row3 ['parent_id'].'</td><td>'.$row3['active'].'</td><td>'.$row3 ['url'].'</td><td>'.$row3['date_updated'].'</td></tr>' ;
}
}
}
EDIT
Ok so I did a bit of research and this is where I am:
Probably for a small database my approach is fine.
For a bigger database using an array to store the data would probably mean I need to use a recursive approach which might use up too much memory. Would love to hear what people think, would it still be better than looping db queries in the nested while loops?
I found the following thread where there is an answer to do this without reccursion and with only one query. Not sure if I need to add a position column to my current design:
How to build unlimited level of menu through PHP and mysql
If I rebuild the design using the nested sets model instead of adjacency model then the mysql query would return the results in the required order however maintaining the nested sets design is above my head and I think would be overkill.
That's it. If anyone has any input on top of that please add to the conversation. There must be a winning approach as this kind of requirement must be needed for loads of web applications.
I would think you could do something like this:
SELECT * FROM categories
WHERE active = 'true'
ORDER BY parent_id, cat_id
This would give you all your categories ordered by parent_id, then by cat_id. You would then take the result set and build a multi-dimensional array from it. You could then loop through this array much as you currently do in order to output the categories.
While this is better from a DB access standpoint, it would also consume more memory as you need to keep this larger array in memory. So it really is a trade-off that you need to consider.
There is a lot to fix there, but I'll just address your question about reducing queries. I suggest getting rid of the WHERE clauses all together and use if statements within the while loop. Use external variables to hold all the results that match a particular condition, then echo them all at once after the loop. Something like this (I put a bunch of your stuff in variables for brevity)
//before loop
$firstInfoSet = '';
$secondInfoSet = '';
$thirdInfoSet = '';
//in while loop
if($parentID == NULL)
{
$firstInfoSet.= $yourFirstLineOfHtml;
}
if($active && $parentID == $catID) // good for query 2 and 3 as they are identical
{
$secondInfoSet.= $yourSecondLineOfHtml;
$thirdInfoSet.= $yourThirdLineOfHtml;
}
//after loop
echo $firstInfoSet . $secondInfoSet . $thirdInfoSet;
You can now make whatever kinds of groupings you want, easily modify them if need be, and put the results wherever you want.
--EDIT--
After better understanding the question...
$query = mysql_query("SELECT * FROM categories order by cat_id asc;", $hd);
$while ($row = mysql_fetch_assoc($query)){
if($row['parent_id'] == NULL){
//echo out your desired html from your first query
}
if($row['active'] && $row['parent_id']== $row['cat_id']){
//echo out your desired html from your 2nd and 3rd queries
}
}
I am using the sqlsrv driver for IIS so I can connect to a MS SQL server in PHP.
I've managed to convert a lot of my original mysql_ code and all going well, until I tried to SELECT some DateTime fields from the database. They were coming back as Date objects in PHP rather than strings, I found the fix which is adding this to the connection array:
'ReturnDatesAsStrings'=>1
Since doing that though my code is broken when trying to populate my recordset:
function row_read($recordset) {
if (!$recordset) {
die('<br><br>Invalid query :<br><br><bold>' . $this->sql . '</bold><br><br>' . sqlsrv_error());
}
$rs = sqlsrv_fetch_array($recordset);
return $rs;
}
The error is: sqlsrv_fetch_array(): 16 is not a valid ss_sqlsrv_stmt resource
There is such little amount of help on that error in Google so this is my only shot! I just don't get it.
row_read is called from within a While: while ($row = $db->row_read($rs)) {
Any ideas?
Just to add more code and logic - I do a simple SELECT of all my orders, then as it loops through them, I do another 2 SELECT's on the orders table then the customer table. It's falling down when I try these extra 2 'gets':
$this->db->sql = "SELECT * FROM TicketOrders";
$rs = $this->db->query($this->db->sql);
$this->htmlList->path("skin/search.bookings");
if ($this->db->row_count != 0) {
while ($row = $this->db->row_read($rs)) {
// Load the order row
$this->TicketOrders->get($this->db, $row['Id']);
// Load the customer row
$this->Customers->get($this->db, $row['CustomerId']);
Did you pass this resource variable by another function? If yes, you can try by executing the sqlsrv_query and executing sqlsrv_fetch_array in one function; don’t pass the ss_sqlsrv_stmt resource by another function. Hope that it will help.
Does your program involves a nested query function?
If so, the next question is: are you opening the same database in the inner query function?
Try these changes:
comment out the lines that open the database, including the { and } that enclose the function,
change the name of connection and array variables between the outer loop and the inner loop.
In other words, the outer loop may have:
$tring = sqlsrv_query($myConn, $dbx_str1);
while( $rs_row1 = sqlsrv_fetch_array($tring, SQLSRV_FETCH_ASSOC))
and the inner loop would have:
$tring2 = sqlsrv_query($myConn, $dbx_str2);
while( $rs_row2 = sqlsrv_fetch_array($tring2, SQLSRV_FETCH_ASSOC))
sqlsrv_fetch_array need a ss_sqlsrv_stmt resource. There must be something wrong with your SQL.
I am trying to figure out a script to take a MySQL query and turn it into individual queries, i.e. denormalizing the query dynamically.
As a test I have built a simple article system that has 4 tables:
articles
article_id
article_format_id
article_title
article_body
article_date
article_categories
article_id
category_id
categories
category_id
category_title
formats
format_id
format_title
An article can be in more than one category but only have one format. I feel this is a good example of a real-life situation.
On the category page which lists all of the articles (pulling in the format_title as well) this could be easily achieved with the following query:
SELECT articles.*, formats.format_title
FROM articles
INNER JOIN formats ON articles.article_format_id = formats.format_id
INNER JOIN article_categories ON articles.article_id = article_categories.article_id
WHERE article_categories.category_id = 2
ORDER BY articles.article_date DESC
However the script I am trying to build would receive this query, parse it and run the queries individually.
So in this category page example the script would effectively run this (worked out dynamically):
// Select article_categories
$sql = "SELECT * FROM article_categories WHERE category_id = 2";
$query = mysql_query($sql);
while ($row_article_categories = mysql_fetch_array($query, MYSQL_ASSOC)) {
// Select articles
$sql2 = "SELECT * FROM articles WHERE article_id = " . $row_article_categories['article_id'];
$query2 = mysql_query($sql2);
while ($row_articles = mysql_fetch_array($query2, MYSQL_ASSOC)) {
// Select formats
$sql3 = "SELECT * FROM formats WHERE format_id = " . $row_articles['article_format_id'];
$query3 = mysql_query($sql3);
$row_formats = mysql_fetch_array($query3, MYSQL_ASSOC);
// Merge articles and formats
$row_articles = array_merge($row_articles, $row_formats);
// Add to array
$out[] = $row_articles;
}
}
// Sort articles by date
foreach ($out as $key => $row) {
$arr[$key] = $row['article_date'];
}
array_multisort($arr, SORT_DESC, $out);
// Output articles - this would not be part of the script obviously it should just return the $out array
foreach ($out as $row) {
echo '<p>'.$row['article_title'].' <i>('.$row['format_title'].')</i><br />'.$row['article_body'].'<br /><span class="date">'.date("F jS Y", strtotime($row['article_date'])).'</span></p>';
}
The challenges of this are working out the correct queries in the right order, as you can put column names for SELECT and JOIN's in any order in the query (this is what MySQL and other SQL databases translate so well) and working out the information logic in PHP.
I am currently parsing the query using SQL_Parser which works well in splitting up the query into a multi-dimensional array, but working out the stuff mentioned above is the headache.
Any help or suggestions would be much appreciated.
From what I gather you're trying to put a layer between a 3rd-party forum application that you can't modify (obfuscated code perhaps?) and MySQL. This layer will intercept queries, re-write them to be executable individually, and generate PHP code to execute them against the database and return the aggregate result. This is a very bad idea.
It seems strange that you imply the impossibility of adding code and simultaneously suggest generating code to be added. Hopefully you're not planning on using something like funcall to inject code. This is a very bad idea.
The calls from others to avoid your initial approach and focus on the database is very sound advice. I'll add my voice to that hopefully growing chorus.
We'll assume some constraints:
You're running MySQL 5.0 or greater.
The queries cannot change.
The database tables cannot be changed.
You already have appropriate indexes in place for the tables the troublesome queries are referencing.
You have triple-checked the slow queries (and run EXPLAIN) hitting your DB and have attempted to setup indexes that would help them run faster.
The load the inner joins are placing on your MySQL install is unacceptable.
Three possible solutions:
You could deal with this problem easily by investing money into your current database by upgrading the hardware it runs on to something with more cores, more (as much as you can afford) RAM, and faster disks. If you've got the money Fusion-io's products come highly recommended for this sort of thing. This is probably the simpler of the three options I'll offer
Setup a second master MySQL database and pair it with the first. Make sure you have the ability to force AUTO_INCREMENT id alternation (one DB uses even id's, the other odd). This doesn't scale forever, but it does offer you some breathing room for the price of the hardware and rack space. Again, beef up the hardware. You may have already done this, but if not it's worth consideration.
Use something like dbShards. You still need to throw more hardware at this, but you have the added benefit of being able to scale beyond two machines and you can buy lower cost hardware over time.
To improve database performance you typically look for ways to:
Reduce the number of database calls
Making each database call as efficient as possible (via good design)
Reduce the amount of data to be transfered
...and you are doing the exact opposite? Deliberately?
On what grounds?
I'm sorry, you are doing this entirely wrong, and every single problem you encounter down this road will all be consequences of that first decision to implement a database engine outside of the database engine. You will be forced to work around work-arounds all the way to delivery date. (if you get there).
Also, we are talking about a forum? I mean, come on! Even on the most "web-scale-awesome-sauce" forums we're talking about less than what, 100 tps on average? You could do that on your laptop!
My advice is to forget about all this and implement things the most simple possible way. Then cache the aggregates (most recent, popular, statistics, whatever) in the application layer. Everything else in a forum is already primary key lookups.
I agree it sounds like a bad choice, but I can think of some situations where splitting a query could be useful.
I would try something similar to this, relying heavily on regular expressions for parsing the query. It would work in a very limited of cases, but it's support could be expanded progressively when needed.
<?php
/**
* That's a weird problem, but an interesting challenge!
* #link http://stackoverflow.com/questions/5019467/problem-writing-a-mysql-parser-to-split-joins-and-run-them-as-individual-query
*/
// Taken from the given example:
$sql = "SELECT articles.*, formats.format_title
FROM articles
INNER JOIN formats ON articles.article_format_id = formats.format_id
INNER JOIN article_categories ON articles.article_id = article_categories.article_id
WHERE article_categories.category_id = 2
ORDER BY articles.article_date DESC";
// Parse query
// (Limited to the clauses that are present in the example...)
// Edit: Made WHERE optional
if(!preg_match('/^\s*'.
'SELECT\s+(?P<select_rows>.*[^\s])'.
'\s+FROM\s+(?P<from>.*[^\s])'.
'(?:\s+WHERE\s+(?P<where>.*[^\s]))?'.
'(?:\s+ORDER\s+BY\s+(?P<order_by>.*[^\s]))?'.
'(?:\s+(?P<desc>DESC))?'.
'(.*)$/is',$sql,$query)
) {
trigger_error('Error parsing SQL!',E_USER_ERROR);
return false;
}
## Dump matches
#foreach($query as $key => $value) if(!is_int($key)) echo "\"$key\" => \"$value\"<br/>\n";
/* We get the following matches:
"select_rows" => "articles.*, formats.format_title"
"from" => "articles INNER JOIN formats ON articles.article_format_id = formats.format_id INNER JOIN article_categories ON articles.article_id = article_categories.article_id"
"where" => "article_categories.category_id = 2"
"order_by" => "articles.article_date"
"desc" => "DESC"
/**/
// Will only support WHERE conditions separated by AND that are to be
// tested on a single individual table.
if(#$query['where']) // Edit: Made WHERE optional
$where_conditions = preg_split('/\s+AND\s+/is',$query['where']);
// Retrieve individual table information & data
$tables = array();
$from_conditions = array();
$from_tables = preg_split('/\s+INNER\s+JOIN\s+/is',$query['from']);
foreach($from_tables as $from_table) {
if(!preg_match('/^(?P<table_name>[^\s]*)'.
'(?P<on_clause>\s+ON\s+(?P<table_a>.*)\.(?P<column_a>.*)\s*'.
'=\s*(?P<table_b>.*)\.(?P<column_b>.*))?$/im',$from_table,$matches)
) {
trigger_error("Error parsing SQL! Unexpected format in FROM clause: $from_table", E_USER_ERROR);
return false;
}
## Dump matches
#foreach($matches as $key => $value) if(!is_int($key)) echo "\"$key\" => \"$value\"<br/>\n";
// Remember on_clause for later jointure
// We do assume each INNER JOIN's ON clause compares left table to
// right table. Forget about parsing more complex conditions in the
// ON clause...
if(#$matches['on_clause'])
$from_conditions[$matches['table_name']] = array(
'column_a' => $matches['column_a'],
'column_b' => $matches['column_b']
);
// Match applicable WHERE conditions
$where = array();
if(#$query['where']) // Edit: Made WHERE optional
foreach($where_conditions as $where_condition)
if(preg_match("/^$matches[table_name]\.(.*)$/",$where_condition,$matched))
$where[] = $matched[1];
$where_clause = empty($where) ? null : implode(' AND ',$where);
// We simply ignore $query[select_rows] and use '*' everywhere...
$query = "SELECT * FROM $matches[table_name]".($where_clause? " WHERE $where_clause" : '');
echo "$query<br/>\n";
// Retrieve table's data
// Fetching the entire table data right away avoids multiplying MySQL
// queries exponentially...
$table = array();
if($results = mysql_query($table))
while($row = mysql_fetch_array($results, MYSQL_ASSOC))
$table[] = $row;
// Sort table if applicable
if(preg_match("/^$matches[table_name]\.(.*)$/",$query['order_by'],$matched)) {
$sort_key = $matched[1];
// #todo Do your bubble sort here!
if(#$query['desc']) array_reverse($table);
}
$tables[$matches['table_name']] = $table;
}
// From here, all data is fetched.
// All left to do is the actual jointure.
/**
* Equijoin/Theta-join.
* Joins relation $R and $S where $a from $R compares to $b from $S.
* #param array $R A relation (set of tuples).
* #param array $S A relation (set of tuples).
* #param string $a Attribute from $R to compare.
* #param string $b Attribute from $S to compare.
* #return array A relation resulting from the equijoin/theta-join.
*/
function equijoin($R,$S,$a,$b) {
$T = array();
if(empty($R) or empty($S)) return $T;
foreach($R as $tupleR) foreach($S as $tupleS)
if($tupleR[$a] == #$tupleS[$b])
$T[] = array_merge($tupleR,$tupleS);
return $T;
}
$jointure = array_shift($tables);
if(!empty($tables)) foreach($tables as $table_name => $table)
$jointure = equijoin($jointure, $table,
$from_conditions[$table_name]['column_a'],
$from_conditions[$table_name]['column_b']);
return $jointure;
?>
Good night, and Good luck!
In instead of the sql rewriting I think you should create a denormalized articles table and change it at each article insert/delete/update. It will be MUCH simpler and cheaper.
Do the create and populate it:
create table articles_denormalized
...
insert into articles_denormalized
SELECT articles.*, formats.format_title
FROM articles
INNER JOIN formats ON articles.article_format_id = formats.format_id
INNER JOIN article_categories ON articles.article_id = article_categories.article_id
Now issue the appropriate article insert/update/delete against it and you will have a denormalized table always ready to be queried.