I'm querying a database (SQL) to find systems sending requests to see which are various different versions of windows (and other OSs). I check for a few known flavours and then try to get a catch-all but the catch-all catches fewer than the individual terms and I can't work out why.
e.g. for this PHP/SQL:
$monthSelect = "SELECT DISTINCT(host) FROM $table WHERE date > ('$lastMonth') ";
$v61 = $db->query($monthSelect."AND sys LIKE '%win32_v6.1%'")->num_rows;
echo "v61: $v61<br>\n";
$v62 = $db->query($monthSelect."AND sys LIKE '%win32_v6.2%'")->num_rows;
echo "v62: $v62<br>\n";
$nWin = $db->query($monthSelect."AND sys LIKE '%win32_v%'")->num_rows;
echo "nWin: $nWin<br>\n";
I get the following output:
v61: 1907
v62: 2181
nWin: 4036
Where 4036 is obviously less than 2181+1907. But shouldn't $nWin be providing a superset of the $v62, $v61 and any other versions that are being found?
I'm at a loss as to how I could be getting these answers
You're doing a SELECT DISTINCT.
If there are duplicate hosts in both of the first 2 queries, they will only be counted once in the third query. So the total of the third query would be less than the sum of the first two.
Related
I've been researching this and I think I've found the best solution but I'm not 100% sure. I have a good handle on PDO but this is the first time I've encountered CASE in mysql. This code is working but I'm wondering if it's efficient? If I were to have multiple keys, I would have to write many arrays to be able to search and order. Is there a shorter way to write this code or is this the most efficient? Thanks!
$filters = "
AND (name LIKE :keys
OR note LIKE :keys
OR tagnum = :skeys)
";
$order = "
ORDER BY
CASE
WHEN tagnum = :skeys THEN 0
WHEN name = :skeys THEN 1
WHEN name LIKE :lkeys THEN 2
WHEN name LIKE :rkeys THEN 3
ELSE 4
END
ASC
";
$arr[':keys'] = "%$keys%"; // both wild cards
$arr[':skeys'] = $keys; // stripped keys, no wild cards
$arr[':lkeys'] = "$keys%"; // left key, right wild card
$arr[':rkeys'] = "%$keys"; // right key, left wild card
If you want to have complete control over how you order your resultset, then your solution is perfect. However, if you have multiple keys, then I would definitely consider using fulltext search because it will be faster and lot less comlicated to code. However, it has a different ranking algorithm than you have now.
im currently stuck with an issue. Ive built a basic "team-work platform" in which you can set tasks in a to-do list. I've implemented the functionality that can mark a task as complete by setting the value of done to 1
I need to be able to check if all of the tasks in the list are set to done, and if so echo something. My code checks for the value 1, but it settles with a single entry being set to 1. But i need it to check if all tasks have the value 1 and if they do it should echo something.
$res3 = mysql_query("SELECT * FROM tasks WHERE tasks.planet_id=1 AND team_id=$teamid AND done=1")
or die(mysql_error());
if ($res3 && mysql_num_rows($res3) > 0)
{
echo 'Complete!';
}
else
{
echo 'Not done yet!';
}
I'll try to give you an example of how i want it to work: Lets say i have 10 tasks in the table. I want the code to recognise when all 10 of these tasks are marked as done with the value 1 set. And then echo "all your tasks are complete". So it needs to somehow loop through all the entries in the table and check if they are all set to 1, and when they are all set to 1 it echoes something.
Please help! :)
Assuming that done is an integer can can be either 0 or 1, you could do something like:
SELECT COUNT(*) total, SUM(done) totalDone FROM tasks WHERE tasks.planet_id=1 AND team_id=$teamid;
And then test in your PHP code that total == totalDone.
Alternatively, if you really want to only get a row out of the database when total == totalDone (as your comments seem to suggest), you could write something like this:
SELECT * FROM (SELECT COUNT(*) total, SUM(done) totalDone FROM tasks WHERE tasks.planet_id=1 AND team_id=$teamid) _X WHERE _X.total = _X.totalDone;
But that just adds a lot of extra complexity for no real gain, and I wouldn't recommend doing it that way.
Note that you should not use mysql_* functions in new code, and should instead use either mysqli or PDO. mysql_* is not recommended for new code.
Also, you should be careful with using variables directly in query strings. That can easily lead to sql injection vulnerabilities. Instead, use parameterized queries with mysqli or PDO.
The answer to 'all tasks done' is best done with the question of how many tasks where done <> 1.
I.e.
SELECT count(*) as 'incomplete'
FROM tasks
WHERE tasks.planet_id=1
AND team_id=$teamid and done <> 1;
Therefore you're able to use the code:
if($res3) {
$incompleteQueryResult = mysql_fetch_assoc($res3);
if ($incompleteQueryResult['incomplete'] > 0) {
echo "Not done yet";
} else {
echo "Complete!";
}
} else {
echo "Could not retrieve completed tasks";
}
If you still need to retrieve both the number of completed tasks as well as the number of incomplete, you could modify the query similar to the following.
SELECT
IF(done = 1, 'complete', 'incomplete') as status,
COUNT(*) AS 'number_in_status'
FROM tasks
WHERE tasks.planet_id=1
AND team_id=$teamid
GROUP BY done
And you'll need to modify how you retrieve it in the PHP as well if so.
If you need to know all of the above, then either execute two queries (one as an aggregate/summary and one as the full data set) or keep track of it in a variable. e.g.
$numIncompleteTasks = 0;
while($row = mysql_fetch_assoc($res3)) {
$numIncompleteTasks += ! (bool) $row['done'];
}
// you now know how many tasks are incomplete.
You could modify this code to track both complete and incomplete.
Deprecation notice.
I'd recommend reviewing your use of mysql_* functions - PHP deprecated and removed these functions in recent versions of PHP.
The answer from jbafford will not work in certains conditions.
Let's imagine we have only three possible values: 0,1 and 2. In the case of 0+1+2, the algorithm will say that COUNT = SUM, when in reality, we have a {0,1,2} and not {1,1,1}.
Why do I ask? Because I'm looking for a way in MYSQL to check if it is all ones, and I ruled out COUNT=SUM or in my case, taking the average, as I have this exception in my data.
A way to handle this exception is to add a COUNT DISTINCT. If COUNT DISTINCT=1 and COUNT=SUM, then the dataset is only 1s.
I'm attempting to work with two sets of data from the same mySQL table in a PHP script. The idea is data is scraped from an API and into a database hourly. A second script then pulls the information out of the database and displays a rolling 6-hour delta.
I've run into a bit of a problem trying to create the delta from the two datasets. I need to run two mySQL queries to get the data I need (current and from 6 hours ago), but can't really think of a way to get the script to work without including the queries inside the loops that output each entry (These can run up to a couple of hundred times, and I don't think having that many mySQL queries running would be good?)
This is what I have so far:
//select the system table and pulls data acquired within the last hour.
$sql = "SELECT system, vp, vpthreshold, owner, time FROM SysData WHERE time > DATE_SUB(NOW(),INTERVAL 1 HOUR)";
$result = $conn->query($sql);
if ($result->num_rows > 0) {
// output data of each row
while($row = $result->fetch_assoc()) {
//Calculates contested percentage
$vpthreshold = $row["vpthreshold"];
$vp = $row["vp"];
$currentcontested = $vp/$vpthreshold*100;
//Catches potential divide by zeroes, echos system is stable.
if ($vp == 0.0){
echo $row["system"] . " " . "is Stable<br>";
}
//else output contested percentage with system name in readable format.
else{
echo $row["system"] . " " . "{$currentcontested}%" . "<br>";
}
}
}
There's a broadly identical statement that pulls and echos the second set of information underneath this. How can I get these two sets together so I can work with them? Very new to PHP and learning on the fly here.
You can look into nested queries. Something like the following:
SELECT (data_now.somevalue - data_prev.somevalue) as deltavalue FROM
(
(your first select statement) as data_now,
(your 6 hours ago select statement) as data_prev
);
This lets you select data from other select statements all in one go.
The 2 inner "select statements" you should replace with your respective queries. The results will be put temporarily into data_now and data_prev. You can then use these as normal tables in the outer select statement.
EDIT: To be more specific to what you want, here is an updated example:
SELECT (data_now.vp/data_now.vpthreshold - data_prev.vp/data_prev.vpthreshold) as deltavalue FROM
(
(SELECT system, vp, vpthreshold, owner, time FROM SysData WHERE time > DATE_SUB(NOW(),INTERVAL 1 HOUR)) as data_now,
(your 6 hours ago select statement) as data_prev
);
In your PHP code remember to reference the result as:
$row["deltavalue"]
or whatever you put after "as" in the outer SELECT.
first of all i have to tell you that it is my first step on php and JSON.
I decided to use JSON to get value from a customer SQL Table.
I get my results using this script :
mysql_connect($config['mysql_host'],$config['mysql_user'],$config['mysql_pass']);
//select database
#mysql_select_db($config['db_name']) or die( "Unable to select database");
mysql_query('SET CHARACTER SET utf8');
$fet=mysql_query('select * from vehicule');
$json = array();
while($r=mysql_fetch_array($fet)){
$json[] = $r;
}
header('Content-Type: application/json');
echo $json_data=json_encode($json);
Everything is ok, exept that my JSON results looks like :
0 = 462;
1 = "Hyundai ix20 crdi 115 panoramic sunsation";
10 = 1346450400;
11 = "462-Hyundai-ix20-crdi-115-panoramic-sunsation";
12 = 462;
...
id = 462;
kilometrage = 14400;
marque = 4;
modele = 137;
motorisation = 2;
ordre = 462;
prix = 17500;
puissance = 6;
titre = "Hyundai ix20 crdi 115 panoramic sunsation";
url = "462-Hyundai-ix20-crdi-115-panoramic-sunsation";
...
I have result of the table in 2 versions : one with 0:value, 1:value, 2... and the other one using the table key, how can i print only the second one ?
By the way can someone give me link so i can know by what i have to replace mysql which is think out of date ? (i'm a beginner few hours using PHP)
Thank you very much !
You have two different issues happening here. One is outright causing the issue you are seeing, and the other is a bad practice mistake that will leave you wide open for trouble in the long run.
The first issue is the one you're asking about. The mysql_fetch_array function (see the Docs here) expects a minimum of one input (the result input) that you are providing. It also has a second, optional input. That optional input defaults to MYSQL_BOTH, which returns an associative array with the results available both through keys (column names) and their indexes. Which is to say, that if you select the column 'id', you get it's value in both $array[0] and $array['id']. It's duplicated, and thus the JSON process carries over the duplication. You need to provide a second value to the function, either MYSQL_ASSOC to get $array['id'] or MYSQL_NUM to get $array[0].
Your second issue is the choice of functions. You're using the 'raw' mysql functions. These have been depreciated, which is a technical term that means 'these functions are no longer supported, but we've left them in to give you time to fix legacy code'. For legacy, read 'old'. Those functions will be going away soon, and you need to upgrade to a better option -- either the mysqli functions, or the PDO class. I strongly recommend the PDO class, as once you learn it it's easy to learn and has the advantage of being more portable. Whichever set you go with, you need to learn to use prepared statements as both a performance and security issue. Right at the moment, you're working with 'raw' statements which have a history of being very easy to interfere with via what's called an 'injection attack'. You can see a fictionalized example of such an attack here, and there are plenty of articles online about it. These attacks can be incredibly complex and difficult to fight, so using prepared statements (which handle it for you), is strongly recommended. In the specific example you're using here, you don't need to worry about it because you aren't including any user inputs, but it's an important habit to get into.
I have build a system using PHP-MySQL. This system is subject to a very heavy load, with thousands of selects,updates,inserts,deletes every minute.
I would like to optimize this system, to make it faster, and reduce load on the servers.
I have already introduced memcache, but mysql data is still needed.
So my question is, which method would be the best in this case.
Currently my queries would look like this:
$q = mysql_query($sql);
while(mysql_fetch_array($q)) {...
I have read that there is a little speed to gain by using mysql_fetch_assoc (?)
But perhaps there is an antirely different approach, when i start optimizing this system?
Thank you all - (Apologies for my limited english skills)
mysql_fetch_assoc vs mysql_fetch_array will duplicate less data thus use less memory. Since the data is presented associative and by index in the array, with that you will get some tiny optimization although will help if your dataset is big.
Try to use natural sort (AKA avoid SORT in query sentences) and LIMIT your result set if you can
Batch queries: instead run 100 inserts over the same table try to do a few of them small.
cache cache cache if you can: using redis or memcached.
if you generate pages that can be treated as static try to use HTTP headers to avoid browsers to request your site all the time
etc. etc.
I would recommend you to use the mysql keyword LIMIT to limit the result set.
Adding pagination to the mysql returning resultset will make your application lighter, the ui will load faster because of less rows to fetch and the mysql server will only receive the select queries when needed.
Basically this is the syntax of how to use limit.
SELECT * FROM Person LIMIT X,Y
Where X is the total row count to be retrieved and Y the offset.
Example:
SELECT * FROM Person LIMIT 10, 0
This query will return the first ten rows of the table Person, and:
SELECT * FROM Person LIMIT 10, 10
Will display the next 10
I've been doing some timing tests on various methods of getting information out of MySQL in PHP. The goal was to find the fastest way of transferring a column of data into a simple array. I've tested it against the enSEMBL database, which is usefully huge.
The following code was common for methods 1 to 8 (9 used GROUP_CONCAT & 10 used PDO):
$query = "SELECT DISTINCT `name` FROM species LIMIT 5000";
$result = $mysqli->query($query);
*Method code*
print_r(array_filter($species));
Method 1: Textbook method
while ($row = $result->fetch_row()) {
$species[] = $row[0];
}
Method 2: while and reset (NB some IDEs detect an error here)
while ($species[] = reset($result->fetch_row())) ;
Method 3: foreach and reset
foreach ($result->fetch_all() as $value) $species[] = reset($value);
Method 4: while, foreach and reset
while ($species[] = $result->fetch_row()) ;
foreach ($species as $key => $value) $species[$key] = reset($value);
Method 5: while and index
while ($row = $result->fetch_row()) $species[] = $row[0];
Method 6: foreach and index
foreach ($result->fetch_all() as $value) $species[] = $value[0];
Method 7: recurse the array
$species = call_user_func_array('array_merge', $result->fetch_all());
Method 8: array_column
$species = array_column($result->fetch_all(), 0);
Method 9: Using GROUP_CONCAT in query.
$species = explode(',', $result->fetch_row()[0]);
Method 10: PDO
$species = $sth->fetchAll(PDO::FETCH_COLUMN, 0);
Surprisingly Method 1 (Textbook) was consistently about 4 times longer than the practically identical Method 5, but took about the same time as Method 10 (PDO).
Method 2 was consistently the slowest method at 50x longer, presumably because the system is writing warnings somewhere.
Method 4 (two loops) was the second slowest, taking 10x longer.
As stated Methods 1(textbook) & 10 (PDO) were third.
Method 9 was fourth slowest (2x longer, and had the disadvantage of hitting the GROUP_CONCAT limit without any warning).
The fastest method, however, wasn't consistent. Take your pick from 3, 5, 6, 7 & 8.
Method 8 (array_column) was often the fastest way to do this, but not always. However I think it's the most elegant method and provides slightly more flexibility as it can return an associative array using any two columns selected by your query (but don't mess with the order in the query!)