I'm calculating daily averages of $spent in a table using mysql. In php, I'd like to average all those daily averages into 1 average daily average. Eventually, I'd like to do this for multiple tables and graph the final averages in highcharts. Right now, I can't get php to average the results of my query. I am connecting to my database, but just not showing it in the code. See code below:
<?php
function array_average($arr){
$sum = array_sum($arr);
$num = sizeof($arr);
echo $sum/$num;
}
$sth = mysql_query("SELECT round((sum(d_power),2) as $perton FROM pheom.pheom_gb WHERE timestamp between subdate(curdate(), interval 3 month) and curdate() GROUP BY Day(timestamp) ORDER BY Timestamp");
$rows = array();
while($r = mysql_fetch_array($sth)) {
$rows['data'][] = $r['$perton'];
}
echo array_average($rows);
mysql_close($con);
?>
Why not do the calculation in SQL?
select avg($perton) as $perton
from (SELECT round(sum(d_power), 2) as $perton
FROM pheom.pheom_gb
WHERE timestamp between subdate(curdate(), interval 3 month) and curdate()
GROUP BY Day(timestamp)
) t;
Where is $perton coming from? Your query is in double quotes, meaning the variable is being expanded, but your array access is in single quotes so it is not.
If $perton is not defined, then I believe your query will be evaluated as:
SELECT round((sum(d_power),2) as FROM pheom.pheom_gb WHERE timestamp between subdate(curdate(), interval 3 month) and curdate() GROUP BY Day(timestamp) ORDER BY Timestamp
Which is a syntax error.
Additionally, I don't think array_sum works with multidimensional arrays. I would try:
echo array_average( $rows['data'] );
This has nothing to do with PHP or any other scripting language for that matter. This is exactly the kind of thing you should always try to solve within your DB query - you'll get cleaner solutions and much better performance than passing all data to your script just to make a calculation you could've gotten with a single query in the first place. The answer given by Gordon Linoff is the right one.
Related
I have run into a delayed processing time for a PHP program,
I have a MySQL record with over 1000 tables;
Each table is created once a new device is added, e.g assets_data_imeixx - to assets_data_imeixx1000th table
Each table contains about 45,000 rows of records inserted every 10 seconds,
Below is my PHP code to query the database and fetch all these records based on datetime.
Issue: The program executes without error but it takes about 1.3minutes to 4mins for very large records.
PHP Code:
$ms = mysqli connection string in config.php //$ms is OKAY
$user_id = '5';
$q = "SELECT * FROM `user_assets` WHERE `user`='".$user_id ."' ORDER BY `imei` ASC";
$r = mysqli_query($ms,$q);
$result = array(); //$result array to contain all data
while($row =mysqli_fetch_array($r)){
//fetch 7 days record
for ($i=1; $i < 7; $i++) {
$date = "-" . $i . " days";
$days_ago = date('Y-m-d', strtotime($date, strtotime('today')));
$sql1 = "SELECT * FROM assets_data_" . $row["imei"] . " WHERE dt_time LIKE '" . $days_ago . "%' LIMIT 1"; // its correct
//$result1 = $conn->query($sql1);
$result1 = mysqli_query($ms,$sql1);
$row2 = mysqli_fetch_array($result1);
echo $row['imei']." ".$row2['dt_server']."<br/>";
}
}
Above code fetches over 1000 devices from user_assets table, These IMEI each has its own table that contains over 45,000 records in each table of location data.
The for loop iterates over each IMEI table and records.
Above code runs without error but take so much time to complete, I want to find a solution to optimize and have code execute in a very short time max 5 seconds.
I need help and suggestions on optimizing and running this large scale of data and iteration.
(from Comment)
CREATE TABLE gs_object_data_863844052008346 (
dt_server datetime NOT NULL,
dt_tracker datetime NOT NULL,
lat double DEFAULT NULL,
lng double DEFAULT NULL,
altitude double DEFAULT NULL,
angle double DEFAULT NULL,
speed double...
(From Comment)
gs_object_data_072101424612
gs_object_data_072101425049
gs_object_data_072101425486
gs_object_data_072101445153
gs_object_data_111111111111111
gs_object_data_1234567894
gs_object_data_222222222222222
gs_object_data_2716325849
gs_object_data_2716345818
gs_object_data_30090515907
gs_object_data_3009072323
gs_object_data_3009073758
gs_object_data_352093088838221
gs_object_data_352093088839310
gs_object_data_352093088840045
gs_object_data_352121088128697
gs_object_data_352121088132681
gs_object_data_352621109438959
gs_object_data_352621109440203
gs_object_data_352625694095355
gs_object_data_352672102822186
gs_object_data_352672103490900
gs_object_data_352672103490975
gs_object_data_352672103490991
gs_object_data_352887074794052
gs_object_data_352887074794102
gs_object_data_352887074794193
gs_object_data_352887074794417
gs_object_data_352887074794425
gs_object_data_352887074794433
gs_object_data_352887074794441
gs_object_data_352887074794458
gs_object_data_352887074794474
gs_object_data_352887074813696
gs_object_data_352887074813712
gs_object_data_352887074813720
gs_object_data_352887074813753
gs_object_data_352887074813761
gs_object_data_352887074813803
900+ tables each having different location data.
Requirement: Loop through each table, fetch data for selected date range say:
"SELECT dt_server FROM gs_object_data_" . $row["imei"] . " WHERE dt_server BETWEEN '2022-02-05 00:00:00' AND '2022-02-12 00:00:00'";
Expected Result: Return result set containing data from each table containing information for the selected date range. That means having 1000 tables will have to be looped through each table and also fetch data in each table.
I agree with KIKO -- 1 table not 1000. But, if I understand the rest, there are really 2 or 3 main tables.
Looking at your PHP -- It is often inefficient to look up one list, then go into a loop to find more. The better way (perhaps 10 times as fast) is to have a single SELECT with a JOIN to do both selects at once.
Consider some variation of this MySQL syntax; it may avoid most of the PHP code relating to $days_ago:
CURDATE() - INTERVAL 3 DAY
After also merging the Selects, this gives you the rows for the last 7 days:
WHERE date >= CURDATE() - INTERVAL 7 DAY
(I did not understand the need for LIMIT 1; please explain.)
Yes, you can use DATETIME values as strings, but try not to. Usually DateTime functions are more efficient.
Consider "composite" indexes:
INDEX(imei, dt)
which will be very efficient for
WHERE imei = $imei
AND dt >= CURDATE() - INTERVAL 7 DAY
I would ponder ways to have less redundancy in the output; but that should mostly be done after fetching the raw data from the table(s).
Turn on the SlowLog with a low value of long_query_time; it will help you locate the worst query; then we can focus on it.
An IMEI is up to 17 characters, always digits? If you are not already using this, I suggest BIGINT since it will occupy only 8 bytes.
For further discussion, please provide SHOW CREATE TABLE for each of the main tables.
Since all those 1000 tables are the same it would make sense to put all that data into 1 table. Then partition that table on date, use proper indexes, and optimize the query.
See: Normalization of Database
Since you limit results to one user, and one row per device, it should be possible to execute a query in well below one second.
I have s MySQL Query where I want to pull data from my database but base it on the current month information
FROM lbs_trace_etrack WHERE (lbs_agent = '$slfirstname' AND DATE (lbs_date) = CURDATE()) ORDER BY lbs_date DESC LIMIT 0,50");
This string pulls out the information for the current day.
I have also tried the below string but get no results from it:
FROM lbs_trace_etrack WHERE (lbs_agent = '$slfirstname' AND MONTH(lbs_date) = (MONTH(NOW()) AND YEAR(lbs_date) = YEAR(NOW())
My table date format is as follow 2016-08-02
Or using PHP variables as so:
<?php
$start = date('Y-m-01'); // FIRST DAY OF CURRENT MONTH
$end = date("Y-m-t", strtotime(date("Y-m-d"))); // LAST DAY OF CURRENT MONTH
$sql = "SELECT * FROM lbs_trace_etrack WHERE lbs_agent = '".$slfirstname."' AND (lbs_date BETWEEN '".$start."' AND '".$end."')";
?>
I have done the following and it works
FROM lbs_trace_etrack WHERE lbs_agent = '$slfirstname' AND MONTH(lbs_date) = MONTH(CURDATE()) AND YEAR(lbs_date) = YEAR(CURDATE()) ORDER BY lbs_date ASC, lbs_time ASC
Thanks to all and Tijo for guidance
Assuming lbs_agent is a DATE field type as mentioned in comments, you could do this (note I am just showing the pertinent date part of your WHERE clause):
WHERE lbs_agent >= DATE_FORMAT(NOW() ,'%Y-%m-01')
It is important that you do not use a function call on the left (field definition) side of the WHERE comparison, as you will then not be able to leverage any index on that field. Doing this would require a full table scan with MySQL performing the function on this field for every row in the table such that the comparison can be made.
Feel free to use MySQL functions for the comparison value, as those would be calculated just once when the query is being planned. You would then be able to use an index on the field for quickly filtering the rows in the table that meet the criteria. From a query execution standpoint, this is basically that same as if your query has this WHERE clause:
WHERE lbs_agent >= '2016-08-01'
This is as compared to the examples in your question which would be executed as:
WHERE DATE(lbs_date) = '2016-08-03'
and
WHERE MONTH(lbs_date) = 8 AND YEAR(lbs_date) = 2016
Both of these would require full table scan since the values derived from the field are not able to be determined until the row is scanned.
You could try to extract the month, such as EXTRACT(MONTH from NOW())
you can use following code if it timestamp
MONTH(CURDATE())
Okay guys, this probably has an easy answer but has been stumping me for a few hours now.
I am using PHP/HTML to generate a table from a MySQL Table. In the MySQL table (TimeRecords) I have a StartTime and EndTime column. In my SELECT statement I am subtracting the EndTime from the StartTime and aliasing that as TotalHours. Here is my query thus far:
$query = "SELECT *,((EndTime - StartTime)/3600) AS TotalPeriodHours
FROM TimeRecords
WHERE Date
BETWEEN '{$CurrentYear}-{$CurrentMonth}-1'
AND '{$CurrentYear}-{$CurrentMonth}-31'
ORDER BY Date
";
I then loop that through an HTML table. So far so good. What I would like to do is to add up all of the TotalHours and put that into a separate DIV. Any ideas on 1) how to write the select statement and 2) where to call that code from the PHP/HTML?
Thanks in advance!
Try this
$query= "
SELECT ((EndTime - StartTime)/3600) AS Hours, otherFields, ...
FROM TimeRecords
WHERE
Date BETWEEN '{$CurrentYear} - {$CurrentMonth} - 1'
AND '{$CurrentYear}-{$CurrentMonth} - 31' ";
$records =mysql_query($query);
$sum= 0;
while($row=mysql_fetch_array($records))
{
echo"$row['otherFields']";
echo"$row['Hours']";
$sum+=$row['Hours'];
}
echo" Total Hours : $sum ";
Just use a single query with a Sum(). You could also manually calculate it if you're already displaying all rows. (If paginating or using LIMIT, you'll need a separate query like below.)
$query = "
SELECT Sum(((EndTime - StartTime)/3600)) AS SumTotalPeriodHours
FROM TimeRecords
WHERE
Date BETWEEN '{$CurrentYear} - {$CurrentMonth} - 1'
AND '{$CurrentYear}-{$CurrentMonth} - 31'
";
You can do this in the same query if you have a unique id using GROUP BY WITH ROLLUP
$query = "
SELECT unique_id,SUM((EndTime - StartTime)/3600) AS TotalPeriodHours
FROM TimeRecords
WHERE Date BETWEEN '{$CurrentYear}-{$CurrentMonth}-1'
AND '{$CurrentYear}-{$CurrentMonth}-31'
GROUP BY unique_id WITH ROLLUP
ORDER BY Date
";
In this instance the last result from your query with contain NULL and the overall total. If you don't have a unique ID you will need to do it in PHP as per Naveen's answer.
A few comments on your code:
Using SELECT * is not considered good practice. SELECT the columns you need.
Not all months have a day 31 so this may produce unexpected results. If you're using PHP5.3+, you can use
$date = new DateTime();
$endDate = $date->format( 'Y-m-t' );
The "t" flag here gets the last day of that month. See PHP docs for more on DateTime.
In my code, I am trying to find items in an activities table that are within the last day. This query is not returning any results, are there any problems with it? Is there a better query?
$curday = time() - (24*3600);
$query = "SELECT * FROM activities WHERE userid = '$userid' AND 'timestamp' > '$curday'";
There are two choices here, you can get and format the date through PHP or use SQL language to do it. I prefer to do it within the SQL, it also allows me to use the same query in a MySQL client.
This question is essentially the same thing: MySQL SELECT last few days?
This would be the new query:
$query = "SELECT * FROM activities WHERE userid = '$userid' AND 'timestamp' > DATE_ADD(CURDATE(), INTERVAL -1 DAY)";
you can try with unix function 'mktime' to get value of yesterday ..
as
$curday = mktime(0,0,0,date("m"),date("d")-1,date("Y"));
for reference
if your database will mysql only then you can extract yesterday in sql itself..
SELECT * FROM activities
WHERE userid = '$userid'
AND timestamp > DATE_SUB(CONCAT(CURDATE(), ' 00:00:00'), INTERVAL 1 DAY)
one more thing if timestamp is your column name don't put this column inside single quote ..
What you can use is DATE_SUB. This can be used as follows
SELECT * FROM activities
WHERE userid = '$userid'
AND timestamp > date_sub(current_date, interval 1 day)
This way you don't need to work with current date in PHP
in Informix it would be (TODAY - 1) if the column is type DATE
I am trying to get the time of the oldest timestamp in the MySQL table "comment" where the following conditions are met:
Loginid of timestamp is equal to $uid.
Timestamps were made within the last hour.
I have tried the code below and it echoes nothing for $minutes.
How can I get $minutes to echo the oldest timestamp that meets the two conditions above?
EDIT: datecommented is a timestamp.
EDIT 2: $minutes echoes out as a blank space. Could this be a PHP formatting issue?
$result = mysql_query('SELECT * FROM comment WHERE datecommented >= NOW() - INTERVAL 1 HOUR AND loginid = "$uid" ORDER BY datecommented ASC LIMIT 1');
$row = mysql_fetch_array($result);
$minutes = $row['datecommented'];
echo '<div>Test '.$uid.' test '.$minutes.' test.</div>';
please add error checking to your sql query, i suspect your sql is failing
or die(mysql_error());
like this
$result = mysql_query('SELECT * FROM comment WHERE datecommented >= NOW() - INTERVAL 1 HOUR AND loginid = "$uid" ORDER BY datecommented ASC LIMIT 1') or die(mysql_error());
at a guess i would expect the " double quotes around "$uid" should be single quotes,
which means you need to change the single quotes around the whole statement to double quotes
like this
$result = mysql_query("SELECT * FROM comment WHERE datecommented >= NOW() - INTERVAL 1 HOUR AND loginid = '$uid' ORDER BY datecommented ASC LIMIT 1") or die(mysql_error());
You should first try your query on MySQL‘s command line or a tool such as PHPMyAdmin, then you will know if the data being returned is how you expect it.
I also suggest to use a SELECT ... UNIX_TIMESTAMP(thefieldyouwantatimestampfrom) ... WHERE ... in case the column is not storing or returning timestamps as numbers.
Well, if the "datecommented" is in unixtimestamp, then of course it doesn't return the minutes, but seconds. As the unixtimestamp is in seconds.
Does your mysql query even work like that? Or is the actual query problematic too?
Is your $row outputting anything? Try print_r($row);.
However simply put, my answer would be, that you are not converting your raw unxitimestamp into minutes.
You can get your seconds to minutes like this: echo ($row['datecommented'] * 60);
And you can get the minute value of your unixtimestamp like this: echo date("i", $row['datecommented']);