I have a table A, with just two columns: 'id' and 'probably'. I need to go over a long list of ids, and determine for each one whether he is in A and has probability of '1'. What is the best way to do it?
I figured that it would be best to have 1 big query from A in the beginning of the script, and after that when I loop each id, I check the first query. but than i realized I don't know how to do that (efficiently). I mean, is there anyway to load all results from the first query to one array and than do in_array() check? I should mention that the first query should had few results, under 10 (while table A can be very large).
The other solution is doing a separate query in A for each id while I loop them. But this seems not very efficient...
Any ideas?
If you have the initial list of ids in array, you can use the php implode function like this:
$query = "select id
from A
where id in (".implode (',', $listOfIds).")
and probability = 1";
Now you pass the string as first parameter of mysql_query and receive the list of ids with probability = 1 that are within your initial list.
// $skip the amount of results you wish to skip over
// $limit the max amount of results you wish to return
function returnLimitedResultsFromTable($skip=0, $limit=10) {
// build the query
$query = "SELECT `id` FROM `A` WHERE `probability`=1 LIMIT $skip, $limit";
// store the result of the query
$result = mysql_query($query);
// if the query returned rows
if (mysql_num_rows($result) > 0) {
// while there are rows in $result
while ($row = mysql_fetch_assoc($result)) {
// populate results array with each row as an associative array
$results[] = $row;
}
return $results;
} else {
return false;
}
}
for each time you call this function you would need to increment skip by ten in order to retrieve the next ten results.
Then to use the values in the $results array (for example to print them):
foreach ($results as $key => $value) {
print "$key => $value <br/>";
}
Build a comma separated list with your ids and run a query like
SELECT id
FROM A
WHERE id IN (id1, id2, id3, ... idn)
AND probability = 1;
Your first solution proposal states that:
You will query the table A, probabyly using limit clause since A is a table with large data.
You will place the retrieved data in an array.
You will iterate through the array to look for the id's with probability of '1'.
You will repeat the first three steps several times until table A is iterated fully.
That is very inefficient!
Algorithm described above would require lots of database access and unneccessary memory (for the temporary array). Instead, just use a select statement with 'WHERE' clause and process with the data you want.
You need a query like the following I suppose:
SELECT id, probably FROM A WHERE A.probably = 1
If i understood you correctly, you should filter in the SQL query
SELECT * FROM A WHERE A.probably = 1
Related
$unique = array();
$sql = "SELECT ID, TitleName, ArtistDisplayName, Mix FROM values_to_insert as A
WHERE A.ID = ";
//Get a single row from our data that needs to be inserted...
while($result = $conn->query(($sql. $count)))
{
//Get the $data of the single row query for inserting.
$data = mysqli_fetch_row($result);
$count++;
//SQL to get a match of the single row of $data we just fetched...
$get_match = "SELECT TitleName_ti, Artist_ti, RemixName_ti from titles as B
Where B.TitleName_ti = '$data[1]'
and B.Artist_ti = '$data[2]'
and B.RemixName_ti = '$data[3]'
LIMIT 1";
//If this query returns a match, then push this data to our $unique value array.
if(!$result = $conn->query($get_match))
{
//If this data has been pushed already, (since our data includes repeats), then don't
//put a repeat of the data into our unique array. Else, push the data.
if(!in_array($unique, $data))
{
echo 'Pushed to array: ' . $data[0] . "---" . $data[1] . "</br>";
array_push($unique, $data);
}
else
echo'Nothing pushed... </br>';
}
}
This has taken 5+ minutes and nothing has even printed to screen. I'm curious as to what is eating up so much time and possibly an alternative method or function for whatever it is taking all this time up. I guess some pointers in the right direction would be great.
This code basically gets all rows, one at a time, of table 'A'. Checks if there is a match in table 'B', and if there is, then I don't want that $data, but if there isn't, I then check whether or not the data itself is a repeat because my table 'A' has some repeat values.
Table A has 60,000 rows
Table B has 200,000 rows
Queries within queries are rarely a good idea
But there appear to be multiple issues with your script. It might be easier to just do the whole lot in SQL and push the results to the array each time. SQL can remove the duplicates:-
<?php
$unique = array();
$sql = "SELECT DISTINCT A.ID,
A.TitleName,
A.ArtistDisplayName,
A.Mix
FROM values_to_insert as A
LEFT OUTER JOIN titles as B
ON B.TitleName_ti = A.ID
and B.Artist_ti = A.TitleName
and B.RemixName_ti = A.ArtistDisplayName
WHERE B.TitleName_ti IS NULL
ORDER BY a.ID";
if($result = $conn->query(($sql)))
{
//Get the $data of the single row query for inserting.
while($data = mysqli_fetch_row($result))
{
array_push($unique, $data);
}
}
As to your original query.
You have a count (I presume it is initialised to 0, but if a character then that will do odd things), and get the records with that value. If the first id was 1,000,000,000 then you have done 1b queries before you ever find a record to process. You can just get all the rows in ID order anyway by removing the WHERE clause and ordering by ID.
You then just get a single record from a 2nd query where the details match, but only process them if no record is found. You do not use any of the values that are returned. You can do this by doing a LEFT OUTER JOIN to get matches, and checking that there was no match in the WHERE clause.
EDIT - as you have pointed out, the fields you appear to be using to match the records do not appear to logically match. I have used them as you did but I expect you really want to match B.TitleName_ti to A.TitleName, B.Artist_ti to A.ArtistDisplayName and B.RemixName_ti to A.Mix
In my app, the user can type in an indefinite amount of categories to search by. Once the user hits submit, I am using AJAX to call my PHP script to query my DB and return the results that match what the user defined for the categories.
My category column is separated as so for each row: "blue,red,yellow,green" etc.
I have two questions:
How can I pass an array to MySQL (like so: [blue,yellow,green]) and then search for each term in the categories column? If at-least one category is found, it should return that row.
Can MySQL add weight to a row that has more of the categories that the user typed in, therefor putting it further to the top of the returned results? If MySQL cannot do this, what would be the best way to do this with PHP?
Thanks for taking the time and looking at my issue.
For the part 1 you can use the function below:
<?php
function createquery($dataarray){
$query="select * from table where ";
$loop=1;
foreach($dataarray as $data)
{
$query.="col='$data'";
if(count($dataarray)<$loop-1){
$query.=' or ';
}
$loop++;
}
return $query;
}
?>
This will return the long query.
use this some like this:
mysql_query("select * from table where category in (".implode($yourarray,',').")");
1)
Arrays are not passed to a MySQL database. What's past is a query which is a string that tells the database what action you want to preform. An example would be: SELECT * FROM myTable WHERE id = 1.
Since you are trying to use the values inside your array to search in the database, you could preform a foreach loop to create a valid SQL command with all those columns in PHP, and then send that command / query to the database. For example:
$array = array('blue', 'red', 'yellow', 'green');
$sql = "SELECT ";
foreach ($array as $value)
{
$sql .= $value.", ";
}
$sql .= " FROM myTable WHERE id = 1";
IMPORTANT! It is highly recommended to used prepared statements and binding your parameters in order not to get hacked with sql injection!
2)
You are able to order the results you obtained in whichever way you like. An example of ordering your results would be as follows:
SELECT * FROM myTable WHERE SALARY > 2000 ORDER BY column1, column2 DESC
I'm wondering why my MySQL COUNT(*) query always results in ->num_rows to be equal 1.
$result = $db->query("SELECT COUNT( * ) FROM u11_users");
print $result->num_rows; // prints 1
Whereas fetching "real data" from the database works fine.
$result = $db->query("SELECT * FROM u11_users");
print $result->num_rows; // prints the correct number of elements in the table
What could be the reason for this?
Because Count(*) returns just one line with the number of rows.
Example:
Using Count(*) the result's something like the following.
array('COUNT(*)' => 20);
echo $result['COUNT(*)']; // 20
Reference
It should return one row*. To get the count you need to:
$result = $db->query("SELECT COUNT(*) AS C FROM u11_users");
$row = $result->fetch_assoc();
print $row["C"];
* since you are using an aggregate function and not using GROUP BY
that's why COUNT exists, it always returns one row with number of selected rows
http://dev.mysql.com/doc/refman/5.1/en/counting-rows.html
Count() is an aggregate function which means it returns just one row that contains the actual answer. You'd see the same type of thing if you used a function like max(id); if the maximum value in a column was 142, then you wouldn't expect to see 142 records but rather a single record with the value 142. Likewise, if the number of rows is 400 and you ask for the count(*), you will not get 400 rows but rather a single row with the answer: 400.
So, to get the count, you'd run your first query, and just access the value in the first (and only) row.
By the way, you should go with this count(*) approach rather than querying for all the data and taking $result->num_rows; because querying for all rows will take far longer since you're pulling back a bunch of data you do not need.
I know this is very simple, but I haven't used PHP/MySQL in a while and I have been reading other threads/php website and can't seem to get it.
How can I query a single row from a MySQL Table and print out all of the fields that have data in them? I need to exclude the NULL fields, and only add those that have data to an html list.
To clarify, I would like to display the field data without specifying the field names, just for the reason that I have a lot of fields and will not know which ones will be NULL or not.
What you've outlined requires 4 basic steps:
Connect to the database.
Query for a specific row.
Remove the null values from the result.
Create the html.
Step 1 is quite environment specific, so that we can safely skip here.
Step 2 - SQL
SELECT * from <tablename> WHERE <condition isolating single row>
Step 3 - PHP (assuming that $query represents the executed db query)
//convert the result to an array
$result_array = mysql_fetch_array($query);
//remove null values from the result array
$result_array = array_filter($result_array, 'strlen');
Step 4 - PHP
foreach ($result_array as $key => $value)
{
echo $value \n;
}
Just SELECT * FROM table_name WHERE.... will do the trick.
To grab data from specific fields, it would be SELECT field_1,field_2,field_3....
you have to make a string which represent mysql query. Then there is function in php named mysql_query(). Call this function with above string as parameter. It will return you all results. Here are some examples
You need to do it like this...
First connect to your sql... Reference
Now make a query and assign it to a variable...
$query = mysqli_query($connect, "SELECT column_name1, column_name2 FROM tablename");
If you want to retrieve a single row use LIMIT 1
$query = mysqli_query($connect, "SELECT column_name1, column_name2 FROM tablename LIMIT 1");
If you want to fetch all the columns just use * instead of column names and if you want to leave some rows where specific column data is blank you can do it like this
$query = mysqli_query($connect, "SELECT * FROM tablename WHERE column_name4 !=''");
Now fetch the array out of it and loop through the array like this..
while($show_rows = mysqli_fetch_array($query)) {
echo $show_rows['column_name1'];
echo $show_rows['column_name2'];
}
If you don't want to include the column names in the while loop, you could do this:
while($show_rows = mysqli_fetch_array($query)) {
foreach( $show_rows as $key => $val )
{
echo $show_rows[$key];
}
}
I'm trying to fetch random no. of entries from a database by using
SELECT QNO FROM TABLE ORDER BY RAND() LIMIT 10
it returns a column of database.
If I want to save all the entries in a array, then which php function do I have to use to save the column.
Something along the lines of this?
$result = mysql_query("SELECT QNO FROM TABLE ORDER BY RAND() LIMIT 10");
$rows = array();
while ($row = mysql_fetch_row($result)) {
$rows[] = $row[0];
}
Updated to not use the $i variable as pointed out in the first post and the comment.
Look at some examples for how to run a query and get a result set.
http://www.php.net/mysqli
Once you have the result in a variable, do this:
$myarray = array();
while($row = mysqli_fetch_row($result))
$myarray[] = $row[0];
With PDO:
$qryStmt = $dbc->query('SELECT QNO FROM TABLE ORDER BY RAND() LIMIT 10');
$a = $qryStmt->fetchAll( PDO::FETCH_COLUMN );
BTW: If you just want to get one row by random, this is much faster esp. for large tables:
select * from table limit 12345,1;
where 12345 is just a random number calculated from the count() of rows.
see here, which is more for rails, but have a look at the comments too.
But be careful: in limit 12345,2 - the second row is not random but just the next row after the random row. And be careful: When I remember right (eg. SQLServer) rand() could be optimized by databases other than mysql resulting in the same random number for all rows which makes the result not random. This is important, when your code should be database agnostic.
a last one: do not mix up "random" with "hard to predict", which is not the same. So the order by example "select top 10 ... order by rand()" on SQLServer results in two different result sets when run twice, BUT: if you look at the 10 records, they lie close to each other in the db, which means, they are not random.