I have pulled in the data from a mysql database using select * with the intention of using the data several times without doing repeated sql enquiries using WHERE.
Using this data I am extracting rows that contain a search element using
while($row=mysql_fetch_array($query_result)){ <<<if match add to new array>>> }
As there are thousands of rows this is taking a longer time than I want.
I am trying to use:
$row=mysql_fetch_array($query_result);
$a = array_search($word_to_check, $row);
echo $a;
This extracts the correct sql headings but not the row number. What I want to achieve is
if $word is found in mysql_fetch_array($query_result) the add the row where it was found into the new array for processing.
Any thoughts? Thanks in advance.
Don't use mysql_* functions they are depracated. Use mysqli or pdo instead.
It's not wise to search in array of mysql results in php while it can be done in mysql. Let's say you have table and you want to find all numbers in number column that are greater than 5
SELECT FROM table_name WHERE number>5
to find text you can use simple clause
SELECT FROM table_name WHERE name = 'username'
You can also create more complex conditions.
From MYSQL manual:
WHERE clause, if given, indicates the condition or conditions that rows must satisfy to be selected. where_condition is an expression that evaluates to true for each row to be selected. The statement selects all rows if there is no WHERE clause
Check this link
If you want to limit the query to only once, fetch all the results into temporary array and do the search from it like below
<?php
$all_rows=array();
$match_rows=array();
$i=0;
$limit=100000;
while($row=mysql_fetch_array($query_result)){
$all_rows[]=$row;
if($i % $limit == 0){ // this part only functions every 100,000 cycles.
foreach($all_rows as $search_row){
if(array_search($word_to_check, $search_row)
$match_rows[]=$search_row;
}
$all_rows=array();//reset temporary array
}
$i++;
}
//This solution assumes the required word can be found in mulitple columns
Related
How do I use get_compiled_select or count_all_results before running the query without getting the table name added twice? When I use $this->db->get('tblName') after either of those, I get the error:
Not unique table/alias: 'tblProgram'
SELECT * FROM (`tblProgram`, `tblProgram`) JOIN `tblPlots` ON `tblPlots`.`programID`=`tblProgram`.`pkProgramID` JOIN `tblTrees` ON `tblTrees`.`treePlotID`=`tblPlots`.`id` ORDER BY `tblTrees`.`id` ASC LIMIT 2000
If I don't use a table name in count_all_results or $this->db->get(), then I get an error that no table is used. How can I get it to set the table name just once?
public function get_download_tree_data($options=array(), $rand=""){
//join tables and order by tree id
$this->db->reset_query();
$this->db->join('tblPlots','tblPlots.programID=tblProgram.pkProgramID');
$this->db->join('tblTrees','tblTrees.treePlotID=tblPlots.id');
$this->db->order_by('tblTrees.id', 'ASC');
//get number of results to return
$allResults=$this->db->count_all_results('tblProgram', false);
//chunk data and write to CSV to avoid reaching memory limit
$offset=0;
$chunk=2000;
$treePath=$this->config->item('temp_path')."$rand/trees.csv";
$tree_handle=fopen($treePath,'a');
while (($offset<$allResults)) {
$this->db->limit($chunk, $offset);
$result=$this->db->get('tblProgram')->result_array();
foreach ($result as $row) {
fputcsv($tree_handle, $row);
}
$offset=$offset+$chunk;
}
fclose($tree_handle);
return array('resultCount'=>$allResults);
}
To count how many rows would be returned by a query, essentially all the work must be performed. That is, it is impractical to get the count, then perform the query; you may as well just do the query.
If your goal is to "paginate" by getting some of the rows, plus the total count, that is essentially two separate actions (that may be combined to look like one.)
If the goal is to estimate the number of rows, then SHOW TABLE STATUS or SELECT Rows FROM information_schema.TABLES WHERE ... gives you an estimate.
If you want to see if there are, say "at least 100 rows", then this may be practical:
SELECT 1 FROM ... WHERE ... ORDER BY ... LIMIT 99,1
and see if you get a row back. However, this may or may not be efficient, depending on the indexes and the WHERE and the ORDER BY. (Show us the query and I can elaborate.)
Using OFFSET for chunking is grossly inefficient. If there is not a usable index, then it is performing essentially the entire query for each chunk. If there is a usable index, the chunks are slower and slower. Here is a discussion of why OFFSET is not good for "pagination", plus an efficient workaround: Pagination . It talks about how to "remember where you left off " as an efficient technique for chunking. Fetch between 100 and 1000 rows per chunk.
The flaw in your code is that it aims to select a subset of some records and their total count in the same query. This is impossible in MySQL, so you cannot generate such a query, hence, you get the error as mentioned. The problem is that if you do a
select ... from t where ... limit 0, 2000
then you get maximum 2000 records, so, if the total records matching the criteria have a count that is greater than the limit, then you will not get accurately the count from above, so, in that case you need a
select count(1) from t where ...
This means that you need to build your actual query (the code below your count_all_results call), see whether the number of results reaches the limit. If the number of results does not reach the limit, then you do not need to perform a separate query in order to get the count, because you can compute $offset * $chunk + $recordCount. However, if you get as many records as they can be, then you will need to build another query, without the order_by call, since the count is independent of your sort and get the counts.
$this->db->count_all_results()
Counting the number of returned results with count_all_results()
It's useful to count the number of results returned—often bugs can arise if a section of code which expects to have at least one row is passed zero rows. Without handling the eventuality of a zero result, an application may become unpredictably unstable and may give away hints to a malicious user about the architecture of the app. Ensuring correct handling of zero results is what we're going to focus on here.
Permits you to determine the number of rows in a particular Active Record query. Queries will accept Query Builder restrictors such as where(), or_where(), like(), or_like(), etc. Example:
echo $this->db->count_all_results('my_table'); // Produces an integer, like 25
$this->db->like('title', 'match');
$this->db->from('my_table');
echo $this->db->count_all_results(); // Produces an integer, like 17
However, this method also resets any field values that you may have passed to select(). If you need to keep them, you can pass FALSE as the second parameter:
echo $this->db->count_all_results('my_table', FALSE);
get_compiled_select()
The method $this->db->get_compiled_select(); is introduced in codeigniter v3.0 and compiles active records query without actually executing it. But this is not a completely new method. In older versions of CI it is like $this->db->_compile_select(); but the method has been made protected in later versions making it impossible to call back.
// Note that the second parameter of the get_compiled_select method is FALSE
$sql = $this->db->select(array('field1','field2'))
->where('field3',5)
->get_compiled_select('mytable', FALSE);
// ...
// Do something crazy with the SQL code... like add it to a cron script for
// later execution or something...
// ...
$data = $this->db->get()->result_array();
// Would execute and return an array of results of the following query:
// SELECT field1, field1 from mytable where field3 = 5;
NOTE:- Double calls to get_compiled_select() while you’re using the Query Builder Caching functionality and NOT resetting your queries will results in the cache being merged twice. That in turn will i.e. if you’re caching a select() - select the same field twice.
Rick James got me on the right track. I ended up having to chunk the results using pagination AND a nested query. Using LIMIT on even 1 chunk of 2000 records was timing out. This is the code I ended up with, which uses get_compiled_select('tblProgram') and then get('tblTrees O1'). Since I didn't use FALSE as the second argument to get_compiled_select, the query was cleared before the get() was run.
//grab the data in chunks, write it to CSV chunk by chunk
$offset=0;
$chunk=2000;
$i=10; //counter for the progress bar
$this->db->limit($chunk);
$this->db->select('tblTrees.id');
//nesting the limited query and then joining the other field later improved performance significantly
$query1=' ('.$this->db->get_compiled_select('tblProgram').') AS O2';
$this->db->join($query1, 'O1.id=O2.id');
$result=$this->db->get('tblTrees O1')->result_array();
$allResults=count($result);
$putHeaders=0;
$treePath=$this->config->item('temp_path')."$rand/trees.csv";
$tree_handle=fopen($treePath,'a');
//while select limit returns the limit
while (count($result)===$chunk) {
$highestID=max(array_column($result, 'id'));
//update progres bar with estimate
if ($i<90) {
$this->set_runStatus($qcRunId, $status = "processing", $progress = $i);
$i=$i+1;
}
//only get the fields the first time
foreach ($result as $row) {
if ($offset===0 && $putHeaders===0){
fputcsv($tree_handle, array_keys($row));
$putHeaders=1;
}
fputcsv($tree_handle, $row);
}
//get the next chunk
$offset=$offset+$chunk;
$this->db->reset_query();
$this->make_query($options);
$this->db->order_by('tblTrees.id', 'ASC');
$this->db->where('tblTrees.id >', $highestID);
$this->db->limit($chunk);
$this->db->select('tblTrees.id');
$query1=' ('.$this->db->get_compiled_select('tblProgram').') AS O2';
$this->db->join($query1, 'O1.id=O2.id');
$result=$this->db->get('tblTrees O1')->result_array();
$allResults=$allResults+count($result);
}
//write out last chunk
foreach ($result as $row) {
fputcsv($tree_handle, $row);
}
fclose($tree_handle);
return array('resultCount'=>$allResults);
So im developing a web page with the following sql query:
$sql=mysql_query("SELECT * FROM `fotf_images` WHERE `image_fotfnum` = '$Fivedigits'");
now, $fivedigits is a $_POST from a previous forms input data. So basically the form parses the mysql db for rows that contain ONLY $Fivedigits in a specific column. What i want to do, is output EVERY row that has these criteria. So far i used the following:
while ($row = mysql_fetch_array($sql, MYSQL_ASSOC)) {
print_r($row);
}
this only seems to output the first row, when i know as a fact there are exactly 2 rows that contain the criteria. Please help! Thanks!
try using
Select * from fotf_images
where Concat(image_fotfnum, '', field2, '', fieldn)
like concat('%','",$Fivedigits,"','%')
since its not only numeric you may need to use LOWER or UPPER case while comparing. This will help you search in every mentioned feilds of fotf_images table
In the MySQL Reference Manual, there's distinction between data definition statements and data manipulation statements.
Now I want to know if a query inserts a database record, updates one, deletes one or modifies the table structure and so on, or, more precisely, the exact number of affected rows, but only if it is applicable.
For example, the statement
SELECT *
FROM SomeTable
WHERE id=1 OR id=2
returns a number of affected rows (in this case 2), but with the SELECT statement, there's nothing modified in the database, so that number would be 0.
How to get the type of query?
I was looking for the same answer and stumbled across this article. It was last updated in August. In it, there is a section: "Determining the Type of a Statement" You basically can make the following assumptions: (copied from the article)
If columnCount() is zero, the statement did not produce a result set. Instead, it modified rows and you can invoke rowCount() to determine the number of affected rows.
If columnCount() is greater than zero, the statement produced a result set and you can fetch the rows. To determine how many rows there are, count them as you fetch them.
I'll save you the trouble and just paste the code sample here
$sth = $dbh->prepare ($stmt);
$sth->execute ();
if ($sth->columnCount () == 0)
{
# there is no result set, so the statement modifies rows
printf ("Number of rows affected: %d\n", $sth->rowCount ());
}
else
{
# there is a result set
printf ("Number of columns in result set: %d\n", $sth->columnCount ());
$count = 0;
while ($row = $sth->fetch (PDO::FETCH_NUM))
{
# display column values separated by commas
print (join (", ", $row) . "\n");
$count++;
}
}
I have been thinking of the same issue, and come to conclusion that I don't need no automation in this matter.
The only use for such an auto-detect is some magic function which will return number of affected rows. But such a magic, although adding a little sugar to the syntax, always makes code support a nightmare:
When you're calling a function, and it can return values of different types depends on the context, you cannot tell which one is returned at every particular moment. So, it makes debugging harder.
So, for sake of readability, just call appropriate function to get the result you need at the moment - affectedRows or numRows. It won't make your code bloated, but make it a lot readable.
I'm using this:
substr($statement->queryString, 0, strpos($statement->queryString, ' '));
where $statement is a PDOStatement object, a few things to note here are that you should verify before using this that $statement is a PDOStatement object, also we should probably take the strpos out of the substr statement in case strpos returns false, which would probably cause an error, finally, this only works with one word statement types, like SELECT, INSERT, etc and not multi-word statement types like ALTER TABLE
the script querys database and retrieves a single entry that has mulitple numbers
SELECT jnum from database where x = y
output = 11111,22222,33333,44444
So i explode that on , and get $variable[0] = 11111 and $variable[1]= 22222
What i want to do is perform a query on another table using each of those numbers (numbers will be different each time and there may be any number of numbers).
is there a way to structure a foreach for each entry in the array or a while loop that counts so that i can query the database for each of the values i get from output above.
i don't know if i am conveying what 'im trying to do here very clearly so i apologize in advance.
i get a single entry for the database table and it contains a string (11111,22222,33333)
i explode on , and get the array variable[]
there will not always be 3 entries sometime there could be 5 or 7 or 10 or 1 but each one will be unique.
but for each value i want to query a db table and retrieve all the rows that have that single number($variable[]) as an entry.
Not sure if a loop count or a foreach statement would work. any ideas?
Well assuming these are values in a single column there is no need to look you can use WHERE ... IN:
SELECT * FROM the_other_table WHERE some_col IN ('11111','22222','33333')
Check out foreach loops - http://php.net/manual/en/control-structures.foreach.php
foreach ($variable as $value) {
$myquery = "some query using $value";
// then execute your query
}
I have to retrieve the history of a user and I have 4 tables whose data depend on each other.I can retrieve the data using loops,but I instead used the "where IN ()" clause and I implode the output of the previous query.However,if the list I provide to "where IN()" is empty it return an error.Is it that IN() cannot be empty?
When imploding an array for the IN clause, i do one of two things
1: Check if you even need to run the query at all
if(!empty($some_array)) {
//run mysql query
}
else {
// if you need to do something if the array is empty, such as error or set some defaults, do it here
}
2: A value in the array initiliser which is not ever in the database (for example, if im selecting based on a auto incrememnt id, i use zero as a default array value to stop any issues with empty data sets, as zero will never be in my id column).
$some_array = array(0);
You can add an empty value to the start, such as IN (0,your values here)