How to combine multiple tables data and encode json as one array? - php

I have encoded json array from one month table data as below but how can I combine multiple data table with other month in one json array and GROUP BY dept after combining all data?
<?php
include("dbconfig.php");
$sql = "SELECT dept, SUM(ttlot) AS sum_ot, SUM(ttlnorm) AS sum_norm
FROM month_jan
GROUP BY dept
ORDER BY sum_ot DESC";
$result = mysqli_query($dbconfig, $sql) or die("Error in Selecting " . mysqli_error($dbconfig));
$category = array();
$category['name'] = 'Dept';
$series1 = array();
$series1['name'] = 'Normal';
$series2 = array();
$series2['name'] = 'OT';
$emparray = array();
while ($row = mysqli_fetch_assoc($result)) {
$category['data'][] = $row['dept'];
$series1['data'][] = $row['sum_norm'];
$series2['data'][] = $row['sum_ot'];
}
$result = array();
array_push($result,$category);
array_push($result,$series1);
array_push($result,$series2);
$json = json_encode($result,JSON_NUMERIC_CHECK);
echo $json;
mysqli_close($dbconfig);
?>
Output in January month:
[{"name":"Dept","data":["CNC","MACH","ANOD","BUFF","CAST","POLISH","SL","EPT","TUMB","TOOL","SHOT","QC","LOG","MAIN","LC","WWT","OG","NPD","E-COAT","SFT"]},{"name":"Normal","data":[47429.1,39975.7,34553.8,49075.9,28316.3,21237.1,13492.5,5848.2,7691.1,6963.9,5636.1,7555.8,5821.9,2161.2,1812,1191.7,1479.1,1299.6,11542.6,602]},{"name":"OT","data":[20041,17874,14431,13535.5,8800.5,5613.5,3569.5,3101,2327,2278,2237,2142,1810,942,690,456,297,110.5,66,50.5]}]
What result I want after combining four months:
[{"name":"Month","data":["Jan","Feb","Mac","Apr"]},{"name":"Normal","data":[504291,409757,295538,430759]},{"name":"OT","data":[89041,96874,81431,80535]}]
Does anyone can help me to solve the problem?

I did it manually by printing them, I think this is an alternative solution if you are having the some problem encoding to JSON of two tables.
Sometime rows with the same name produces an error on json_encode() function.
<?php
session_start();
require("../config.php");
$output = array();
$id = $_GET['id'];
if(isset($_SESSION['user'])){
$f_data ='';
$sql = "SELECT * FROM quiz WHERE subject_id=$id ORDER BY id DESC";
$query=$conn->query($sql);
while($row=$query->fetch_array()){
$sql2 = "SELECT * FROM q_details WHERE id=$row[2] ORDER BY id DESC LIMIT 1";
$query2=$conn->query($sql2);
while($row2=$query2->fetch_array()){
$f_data .= '{
"id":"'.$row[0].'",
"subject_id":"'.$row[1].'",
"questionaire_id":"'.$row[2].'",
"name":"'.$row[3].'",
"description":"'.$row[4].'",
"start":"'.$row[5].'",
"end":"'.$row[6].'",
"date":"'.$row[7].'",
"questionaire_name":"'.$row2[2].'",
"questionaire_description":"'.$row2[3].'"
},';
}
}
$f_data = substr($f_data, 0, -1);
echo '['.$f_data.']';
}
else{
echo"<script>window.open('http://localhost/lnhs_app/#/','_self')</script>";
}
?>
If you have rows with the same name you should rename the other like what I did on the example above (questionaire_name, questionaire_description).
It will produce this output:
[{ "id":"1", "subject_id":"2", "questionaire_id":"1", "name":"Quiz 1", "description":"Answer this quiz within 1 hour", "start":"7:30AM", "end":"8:30AM", "date":"08-01-18", "questionaire_name":"Right triangle", "questionaire_description":"Questionaire # 1" }]

With MySQL's JSON features and a combination of GROUP_CONCAT you can do some pretty awesome things with combining tables:
Your desired result:
[{"name":"Month","data":["Jan","Feb","Mac","Apr"]},{"name":"Normal","data":[504291,409757,295538,430759]},{"name":"OT","data":[89041,96874,81431,80535]}]
Utilizing GROUP_CONCAT you don't need joining tables but simply a group by field by which your fields are aggregated:
SELECT
CONCAT('{\"NAME\" : \"NORMAL\",',
'\"data\": [', GROUP_CONCAT( '"', SUM(ttlnorm), '"'), ']},',
'{\"NAME\" : \"OT\",',
'\"data\": [', GROUP_CONCAT( '"', SUM(ttlot), '"'), ']}',
)
from month_jan
GROUP BY dept;
If you are using a table structure that stores by month ("month_jan"), you will need to perform a union amongst your months or change your table structure so that all months needed are included within the same table. You can perform your concatenation on multiple levels based on your GROUP BY's. For example, if you have two group by fields, you will be able to nest your JSON:
ex: GROUP BY DEPT, MONTH
SELECT
CONCAT('{',
'\"DEPT\" :\"', dept,'\",',
'{\"NAME\" : \"Month\",',
'\"data\": [', GROUP_CONCAT( '"', month, '"'), ']},',
'{\"NAME\" : \"NORMAL\",',
'\"data\": [', GROUP_CONCAT( '"', SUM(ttlnorm), '"'), ']},',
'{\"NAME\" : \"OT\",',
'\"data\": [', GROUP_CONCAT( '"', SUM(ttlot), '"'), ']}',
)
from ttl_ot_norm
GROUP BY dept, month;
As a result, your data within your GROUP_CONCAT's will work out according to the form you expressed.
An alternate solution is to create a table that updates based on a cron job or table trigger and places this data into a JSON field type so it's continuously ready for retrieval with a cheap CPU cost. This way, you won't have the additional overhead of the concatenation and table joining on each query.
You can skip a lot of your PHP aggregation by combing GROUP_CONCAT and grouping your tables. I've saved hundreds of hours of programming over the years with this and a combination of MySQL's JSON features (link below)
Version 8 introduced these features where you don't need to create your own JSON like strings with JSON_ARRAYAGG and other great JSON features. Although, the above will work with version 5 and up.
https://dev.mysql.com/doc/refman/5.7/en/group-by-functions.html#function_group-concat
https://dev.mysql.com/doc/refman/5.7/en/json-creation-functions.html#function_json-array
VERSION 8:
https://dev.mysql.com/doc/refman/8.0/en/group-by-functions.html#function_group-concat

Related

How to display data from different tables when they have the same hour (datetime) using Google Charts and MySQL?

I'm trying to display multidimensional data from different tables, when there is a match in hour. So for example, value1 from table1 and value2 from table2 should be displayed in Y and the hour in X. I'm using Google Charts and MySQL. The code works when I select things from a single table, but when I try to mix results it gives this error: 'Data column(s) for axis #0 cannot be of type string'
I've tried to use WHERE and tried JOIN for linking the column Datetime from table1 with table2, but no success. I know I may still have to format Datetime to pick up the hour only, but I'm feeling completely stuck.
Here is what I have in table 1:
And in table 2:
I'm looking to have something like this:
Below is the code:
var data = google.visualization.arrayToDataTable([
['Datetime','Value1', 'Value2'],
<?php
$query = "SELECT table1.Datetime, table1.Value1, table2.Value2 ".
"FROM table1 ".
"WHERE table1.Datetime = table2.Datetime ".
"ORDER BY table1.Datetime ASC ".
"LIMIT 24";
$exec = mysqli_query($con,$query);
while($row = mysqli_fetch_array($exec)){
echo "[
'".$row['Datetime']."',
".$row['Value1']." ,
".$row['Value2']."
],";
}
?>
]);
Note: I'm limiting the chart to 24 rows, as when new data is inserted (every hour), only the last 24 hours of data are displayed.
I'm answering my own question, but this was thanks to the feedback provided in the comments by WhiteHat.
First problem: I was not including the second table in the FROM clause.
Second problem: My data example was not good as I had repetitions of the Datetime values.
Third problem: Matching the hour between tables was solved by adding the HOUR function in the code. See below:
var data = google.visualization.arrayToDataTable([
['Hour1','Value1', 'Value2'],
<?php
$query = "SELECT HOUR(table1.Datetime) AS Hour1, table1.Value1, table2.Value2 ".
"FROM table1, table2 ".
"WHERE HOUR(table1.Datetime) = HOUR(table2.Datetime) ".
"ORDER BY table1.Datetime ASC ".
"LIMIT 24";
$exec = mysqli_query($con,$query);
while($row = mysqli_fetch_array($exec)){
echo "[
'".$row['Hour1']."',
".$row['Value1']." ,
".$row['Value2']."
],";
}
?>
]);

How to remove duplicates from json in PHP

I have a MySQL, PHP code as follows.
$sql = "SELECT * FROM shipschedule WHERE ship_date BETWEEN '2016-08-01' AND '2016-8-31'";
$result = $mysqli->query($sql);
$e = array();
while($r = $result->fetch_array()) {
$rows = array();
$rows['title'] = $r['title'];
$rows['start'] = $r['ship_date'];
array_push($e, $rows);
}
echo json_encode($e);
The above php code echos
[{"title":"111","start":"2016-08-10"},
{"title":"111","start":"2016-08-10"},
{"title":"111","start":"2016-08-10"},
{"title":"222","start":"2016-08-17"},
{"title":"222","start":"2016-08-17"},
{"title":"222","start":"2016-08-16"}]
My question is how I can echo the above as follow instead. Please see that duplicate start dates will be removed by title.
[{"title":"111","start":"2016-08-10"},
{"title":"222","start":"2016-08-17"},
{"title":"222","start":"2016-08-16"}]
title 111 has the same 3 start dates, and I need to display it like
{"title":"111","start":"2016-08-10"},
title 222 has the same 2 start dates, and I need to display it like
{"title":"222","start":"2016-08-17"},
{"title":"222","start":"2016-08-16"}]
You could prevent receiving duplicates, and reduce requesting unnecessary data by adjusting your query.
SELECT DISTINCT title, start FROM ...
It would be much easier (and probably faster too) to just get the right (unique) data from MySQL. This can be achieved with the distinct modifier:
SELECT DISTINCT title, start
FROM shipschedule
WHERE ship_date BETWEEN '2016-08-01' AND '2016-8-31'

Generate html table based on 2x mysql db queries

I'm trying to show stuff queried from two tables, but on one html table. Data is shown for the last 30 days, based on which, an html table is being generated.
Currently I'm stuck using two queries and generating two html tables:
$query1 = mysqli_query( $con, "SELECT date, stuff* " );
while( $record = mysqli_fetch_array( $query1 ) ){
echo '<html table generated based on query>';
}
$query2 = mysqli_query( $con, "SELECT date, other stuff*" );
while( $record = mysqli_fetch_array( $query2 ) ){
echo '<another html table generated based on query2>';
}
Is there a possibility to show both queries on one html table instead?
Note that it gets tricky since we have dates on one table which are not necessarily found in the second table or vice-versa.
Thanks for the support guys. So far I'm stuck at this:
SELECT * FROM user_visit_logs
LEFT JOIN surfer_stats ON user_visit_logs.date = surfer_stats.date
UNION
SELECT * FROM user_visit_logs
RIGHT JOIN surfer_stats ON user_visit_logs.date = surfer_stats.date
The query completes, but the 2nd table fields are all null:
Furthermore, it breaks when I add additional clause like:
WHERE user_id = '{$_SESSION['user_id']}' ORDER BY date DESC LIMIT 30
I think you are after FULL OUTER JOIN concept:
The FULL OUTER JOIN keyword returns all rows from the left table (table1) and from the right table (table2)
In which you may use common dates as a shared row.
So the query will get to simple one:
$query = "
SELECT table1.date, stuff
FROM table1
LEFT OUTER JOIN table2 ON table1.date = table2.date
UNION
SELECT table2.date, other_stuff
FROM table1
RIGHT OUTER JOIN table2
ON table1.date = table2.date
";
$result = mysqli_query( $con, $query );
while( $record = mysqli_fetch_array( $result ) ){
echo '<html table generated based on query>';
}
Example
This is an schematic diagram of FULL OUTER JOIN concept:
After running into quite a few bumps with this one, I finally managed to merge 2 columns from each table and also to use where and sort clauses on them with the following query:
( SELECT user_visit_logs.user_id,user_visit_logs.date,unique_hits,non_unique_hits,earned,sites_surfed,earnings FROM user_visit_logs
LEFT OUTER JOIN surfer_stats ON user_visit_logs.user_id = surfer_stats.user_id AND user_visit_logs.date = surfer_stats.date where user_visit_logs.user_id = 23 ORDER BY date DESC LIMIT 30 )
UNION
( SELECT surfer_stats.user_id,surfer_stats.date,unique_hits,non_unique_hits,earned,sites_surfed,earnings FROM user_visit_logs
RIGHT OUTER JOIN surfer_stats ON user_visit_logs.user_id = surfer_stats.user_id AND user_visit_logs.date = surfer_stats.date where user_visit_logs.user_id = 23 LIMIT 30 )
Simplified, "user_visit_logs" and "surfer_stats" were the 2 tables needed to be joined.
Absolutely. Just pop them both into a variable:
$data = '';
$query = mysqli_query($con,"SELECT date, stuff* ");
while($record = mysqli_fetch_array($query)) {
$data.= '<tr><td>--Your Row Data Here--</td></tr>';
}
$query2 = mysqli_query($con,"SELECT date, other stuff*");
while($record = mysqli_fetch_array($query2)) {
$data .= '<tr><td>--Your Row Data Here--</td></tr>';
}
echo "<table>$data</table>";
Instead of using echo in your loop, you're just storing the results in $data. Then, you're echoing it out after all data has been added to it.
As for your second point, it's not a big deal if fields don't exist. If they're null, you'll just have a column that doesn't have data in it.
Here's an example with fake column names:
$data = '';
$query = mysqli_query($con,"SELECT date, stuff* ");
while($record = mysqli_fetch_array($query)) {
$data.= "<tr><td>{$record[id]}</td><td>{$record[first_name]}</td><td>{$record[last_name]}</td></tr>";
}
$query2 = mysqli_query($con,"SELECT date, other stuff*");
while($record = mysqli_fetch_array($query2)) {
$data .= "<tr><td>{$record[id]}</td><td>{$record[first_name]}</td><td>{$record[last_name]}</td></tr>";
}
echo "<table><tr><th>ID</th><th>First Name</th><th>Last Name</th></tr>$data</table>";
I have a feeling I may have misunderstood the need. If so, I apologize. If you can elaborate just a bit more I can change my answer :)

Organizing member join dates/member growth data into graphable chunks

I'm dabbling with pChart and would like to start with a simple line graph showing the growth in membership over time.
Y-axis would be # of membersX-axis would be time
For each time datapoint, I need a corresponding total members datapoint.
My user table is structured as:
[user_id] [join_date]
The approach I came up with on the bus to work this morning is:
$Q = " SELECT MONTH(join_date), DAY(join_date), COUNT(user_id)"
. " FROM user_basic_data GROUP BY join_date";
$R = mysql_query($Q);
$dateS = '';
$totalS = '';
$c = 0; // total members counter
while ($row = mysql_fetch_row($R)) {
$dateS .= $row[0].'-'.$row[1].','; // month-day,month-day,month-day
$c = $row[2] + $c; // new total for new date
$totalS .= $c.','; // total1,total2,total3
}
// trim trailing commas
$dateS = substr($dateS, 0, -1);
$totalS = substr($totalS, 0, -1);
echo "<p>$dateS</p>"; // Ex: 8-10,8-15,8-20
echo "<p>$totalS</p>"; // Ex: 12,17,23
Those string formats are how pChart likes the data, and I know the current query would need a year value as well for real use, so please don't get hung up on those points.
I'd like to know if there's a better way to go about getting the changing total members over time. I'm guessing handling it within MySQL would be faster, but I can't think of a way to do that.
Thank you for your time.
To get a running total, use:
SELECT DISTINCT
DATE(ubd.join_date) AS dt,
(SELECT COUNT(*)
FROM user_basic_data t
WHERE DATE(t.join_date) <= DATE(ubd.join_date)) AS num_users
FROM user_basic_data ubd
DATE returns dates as YYYY-MM-DD; If you still want Month-Day - use DATE_FORMAT by replacing the DATE(udb.join_date) with:
DATE_FORMAT(ubd.join_date), '%m-%d')
You don't need the logic to create the comma separated lists in PHP - just need to populate the two variables:
$Q = " SELECT GROUP_CONCAT(x.dt) AS dates,
GROUP_CONCAT(x.num_users) AS totals
FROM (SELECT DISTINCT
DATE(ubd.join_date) AS dt,
(SELECT COUNT(*)
FROM user_basic_data t
WHERE DATE(t.join_date) <= DATE(ubd.join_date)) AS num_users
FROM user_basic_data ubd ) x";
$R = mysql_query($Q);
while ($row = mysql_fetch_row($R)) {
echo "<p>$row[0]</p>"; // Ex: 8-10,8-15,8-20
echo "<p>$row[1]</p>"; // Ex: 12,17,23
}

Help with mysql sum and group query and managing results for jquery graph

I have a system I am trying to design that will retrieve information from a database, so that it can be plotted in a jquery graph. I need to retrieve the information and somehow put it in the necessary coordinates format (for example two coordinates var d = [[1269417600000, 10],[1269504000000, 15]];).
My table that I am selecting from is a table that stores user votes with fields:
points_id (1=vote up, 2=vote down),
user_id,
timestamp, (UNIX TIMESTAMP)
topic_id
What I need to do is select all the votes and somehow group them into respective days and then sum the difference between 1 votes and 2 votes for each day. I then need to somehow display the data in the appropriate plotting format shown earlier. For example April 1, 4 votes. The data needs to be separated by commas, except the last plot entry, so I am not sure how to approach that. I showed an example below of the kind of thing I need but it is not correct,
echo "var d=[";
$query=mysql_query(
"SELECT *, SUM(IF(points_id = \"1\", 1,0))-SUM(IF([points_id = \"2\", 1,0)) AS 'total'
FROM points LEFT JOIN topic ON topic.topic_id=points.topic_id
WHERE topic.creator='$user' GROUP by timestamp
HAVING certain time interval"
);
while ($row=mysql_fetch_assoc($query)){
$timestamp=$row['timestamp'];
$votes=$row['total'];
echo "[$timestamp,$vote],";
}
echo "];";
It would certainly be a lot saner to just use -1 as a downvote. Then you can simply run SUM(points_id)
To generate javascript-friendly notation, you can use good old mate json_encode.
$sql = "SELECT `timestamp`, SUM(points_id) FROM ..."; // yadda yadda
$result = mysql_query($sql);
$out = array();
while ($row = mysql_fetch_assoc($result)) {
$out[] = array((int) $row['timestamp'], (int) $row['total']);
}
echo "d = " . json_encode($out) . ";\n";

Categories