I have more than 100,000 records in my books table, my database is MySQL. I am creating an autosuggestion using PHP. It is working but it is slow. I think it is because of bulk data. Is there any solution for searching data in effective manner and how can we improve speed of that searching process?
Here is my code:
<?php
$search=$_REQUET['searc'];
$qry="select book_name from books where book_name like '%$search'";
$result=mysql_query($qry);
echo "<ul>";
while($rows=mysql_fetch_array($result))
{
echo "<li>".$rows['book_name']."<li>;
}
echo "</ul>";
?>
apply indexing or you can use fulltext indexing and search
http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
$search = mysql_real_escape_string($_REQUEST['searc']);
$qry = "select book_name from books where book_name like '$search%' LIMIT 10";
and an ordinary index will solve the problem... for a while
Change your query as follows. It will fetch only first 30 items. Now the speed will increase. Also You can index your table correctly for speeding
$qry="select book_name from books where book_name like '%$search' LIMIT 0 , 30";
Try executing the following code in MYSQL:
ALTER TABLE `books` ADD INDEX `book_name` (`book_name`):
This will add an index to the table, this should speed up the search.
You are searching on the suffix of the column. Indexes work with the prefix so this query requires a full table scan. To fix this you can add a column that has the original in reverse, index it, and use it for searching:
alter table books add book_name_reverse tinytext;
update books set book_name_reverse = reverse(book_name);
alter table books add index(book_name_reverse);
After that you can search efficiently with:
$search_rev=strrev($_REQUET['searc']);
$qry="select book_name from books where book_name_reverse like '$search_rev%'";
But please use PDO or MySQLi prepared statements instead of doing this. Splicing user-submitted values into queries like this makes a classic SQL injection vulnerability.
Related
A self taught newbie here looking for advice...
I have very simple PHP query against a MySQL database that is suppose to generate a drop down menu populated with information from the MySQL database. I was wondering if based on the code included in this posting someone could steer me in the right direction or lend a suggestion as to what I should.
P.s. the MySQL database is products the table I am selecting from within is products and the column I am trying to select is description.
<?php
$adConn = mysqli_connect("localhost", "user", "password", "products");
$result = "SELECT * FROM products where Description order by descending";
$result = mysqli_query($adConn, $result);
echo "<select name='product'>";
while ($row = mysql_fetch_array($result))
{
echo "<option value='" . $row['Description'] . "'>" . $row['Description'] . "</option>";
}
echo "</select>";
?>
Your where condition is not correct. Also look for order by.
Try this :
$result = "SELECT * FROM products where Description = 'SOME VALUE' order by SOME FIELD desc;
Refer this : http://dev.mysql.com/doc/refman/5.7/en/select.html
Also you are mixing mysqli_* and mysql_*.
Please don't use mysql_* it is deprecated and removed from PHP 7.
For mysqli : http://php.net/manual/en/book.mysqli.php
If you're only looking for the descriptions from your products table, you can change your select statement to something like this:
SELECT description
FROM products
ORDER BY <column name> desc;
This will give you all the descriptions on your products table, in descending order based on a column name. Not sure what you are trying to order by on the products table, but you need to add a column name to your ORDER BY clause.
Note - You had description in your WHERE clause, so if you are looking for a specific description, you could do something like this instead:
SELECT description
FROM products
WHERE description = 'some specific description';
Returns one specific description only, thus no need for an ORDER BY clause.
You need to learn the basic concepts of how to make a SQL statement .
Here are some basic queries you can take a look at for example:
Simple straight forward select all:
SELECT * FROM products;
Select where column value equals to something:
SELECT * FROM products WHERE product_category=foo;
Select and order by column ascending(ASC) or descending(DESC) order :
SELECT * FROM products ORDER BY product_id DESC;
now lets combine them:
SELECT * FROM products
WHERE product_category = foo
ORDER BY product_id DESC;
It is good practice to keep all SQL functions in uppercase and column names and values in lowercase making it easy to read. Also, don't use mysql_* since it's deprecated and removed from PHP 7. Instead, use mysqli_*.
I want to thank everyone for your comments and suggestions. This web community turned out to be pretty good. I am glad that there are still people on the web willing to brainstorm and help a newbie!
Have a great day!
I develope a website for E-books, i have in database table for authors and table for publishers .. sometimes the author name is added also in publishers table as a publisher
Now i have his name as an author and a publisher .. when i search in the site for his name, it return twice because i search in authors table and in publishers table then merge two queries
this is my code :-
function generate_results($keyword, $row = 0) {
$result1 = $this->db->query("SELECT au_id,au_name,au_state,SUBSTR(au_info,1,190) AS au_info,au_img FROM d_author where (au_name LIKE '%$keyword%' or au_info LIKE '%$keyword%') and au_state = '1' limit $row,20");
$result2 = $this->db->query("SELECT pub_id,pub_name,pub_state,SUBSTR(pub_info,1,190) AS pub_info,pub_img FROM d_publishing where (pub_name LIKE '%$keyword%' or pub_info LIKE '%$keyword%') and and pub_state = '1' limit $row,20");
$results = array_merge($result1->result_array(), $result2->result_array());
return $results;
}
Now i want to modify the second query to something like that :
select all publishers from "publishers table" where the name of publisher is like $keyword and this $keyword doesn't exist in authors table ..
I mean if this name exist in authors don't select it in publishers
How can i translate that meaning to Mysql Query
First, check out sql-injections before continuing to develop in your ebook-application. Looks like your keyword is not checked to be a safe parameter. And just to be sure, do you know about csrf and xss? If not, check about that too. This is very important.
Secondaly, you should consider working on your database design to avoid having duplicated values. Check out "database normalization" for more information. Seems like you could do another table to extract your "contact information" like name, state, id etc. This would make it possible for your author-table and publisher-table to use a "contact_id" referencing the contact-information-table.
Last but not least, to answer your question, you can generelly solve such problems with an "anti join". Use a left join on the authors table in the second query and check for "IS NULL" on matches with the publisher table. More information here: http://explainextended.com/2009/09/18/not-in-vs-not-exists-vs-left-join-is-null-mysql/
I do not know if I understand your database design completly right, but it also seems like an UNION combined with a DISTINCT could help you - and you wouldn't even need this array_merge-stuff. I would suggest you to check out these two commands in the mysql docs.
I'm running a query to select an array of id's from one table, so that I can update another table with the data from the resulting dataset.
//db query result
$query = "SELECT image_id FROM jos_jxgallery_images ORDER BY jos_jxgallery_images.image_id DESC LIMIT 25";
$query_execute = mysql_query($query);
mysql_close($db_config);
while ($items = mysql_fetch_array($query_execute)) {
echo $items['image_id'];
echo '<br/>';
}
I think I need to do it in the while loop, I just have the echo there to see what's in items variable. That works ok. I think the thing to do is in the while loop..I'd like to replace the 'echoing' with an actual update SET query for my other table. Something like...
while ($items = mysql_fetch_array($query_execute)) {
$q = "UPDATE jx_gallery_images_ratings SET image_id ='".$items."";
mysql_query($q);
But the new table has no data. Is there just a better way to write this...maybe even as one query or something? Any help is appreciated.
EDIT: I should explain a little better. The table is empty, and I could go ahead and use an insert from one table to another just to get the id's there. However, after that...in a way it is somewhat a 'temp' table. But not really. Whatever ordering of image_id's I have created in my SELECT query from my first table (There are other rows to sort by other than image_id, like 'hits', for example)...so the second table needs to be updated with the same ordering of image_id's. Probably be running this several little snippet several times with a cron job. So, yeah, I'm trying to update the second table with the ordering of the SELECT query of the first table and just put the id's in my second table, again...according to the order of the first SELECT query.
If the table is empty, you should do an insert. You can do it in a single query like this:
INSERT INTO jx_gallery_images_ratings (image_id)
SELECT image_id FROM jos_jxgallery_images ORDER BY jos_jxgallery_images.image_id DESC LIMIT 25
Note that you probably wouldn't really need the ORDER BY, adn you could do it for all images at once by removing the LIMIT
Something like:
UPDATE tbl_updateme SET row_to_update = (SELECT row_you_need from tbl_target WHERE tbl_updateme.comparison_row = tbl_target.comparison_row)
INSERT INTO jx_gallery_images_ratings (image_id) (SELECT image_id FROM jos_jxgallery_images ORDER BY jos_jxgallery_images.image_id DESC LIMIT 25)
Or
Easy way is create trigger that updates table after selection with Dynamic SQL
I have been searching for a while on a way to select certain columns (fields) in SQL.
What I am trying to do is this... Lets say I have a table with 200 columns of data. I want to select an entire row but only the last 197 columns of data. Leaving out the first 3 columns that have dates and ID's.
It would be very time consuming to type out the 197 field names I wanted the data from. There has to be some easier way of doing it.
Any help or suggestions to point me in the correct direction?
If you are using MySQL, try something like this:
<?php
$query = "
SHOW COLUMNS FROM `Table`
WHERE `Field` NOT LIKE 'rowtoignore1'
AND `Field` NOT LIKE 'rowtoignore2'
AND `Field` NOT LIKE 'rowtoignore3'
";
$r = mysql_query($query) or die(mysql_error());
$queryfields = "";
while($f = mysql_fetch_assoc($r)) {
$queryfields .= "`{$f['Field']}`,";
}
$queryfields = substr($queryfields,0,strlen($queryfields)-1);
$query = "SELECT {$queryfields} FROM `Table` WHERE xyz";
?>
If you're using MySQL you can query the schema to get a column name by index, so you could use an iterative PHP routine to build your query, but it would require 197 select statements before you could run the one you really want. Messy and inefficient.
It seems simpler to do a SELECT * and then ignore the first three columns. If you use mysql_fetch_assoc() you can get the columns you want easily. Take a look at:
http://php.net/manual/en/function.mysql-fetch-array.php
If i am doing this task, i won't exclude the first 3 columns . If the 3 columns contains lengthy text / blob , then you could avoid, else better fetch those 200 columns .
Using this function mysql_list_fields() fetch fields corresponding to a table , and customize the select query to include required fields
Im trying to get this query to work but i get this error:
Unknown column 'zips.city' in 'having clause'
`$query = "SELECT
zips.*
FROM
zips
HAVING
zips.city LIKE '%$city%'
AND
zips.stateabbr LIKE '%$state%'
LIMIT 1";
$result = mysql_query($query) or die (mysql_error());`
my zips table has a city column, so im not sure what the problem is, i know im accessing the database because i can run this query with no errors:
$zip1query = "SELECT
zips.*
FROM
zips
WHERE
zips.zip = '$zip'
";
any advice would be much appreciated! thanks!
The having clause doesn't mean the same thing as the where clause : when running a simple query, you should use where -- which is what you did in your second query, that works.
having is used when the condition has to be applied on the result of a group by clause.
Which means that, here, your query should be build this way :
$query = "SELECT zips.*
FROM zips
where zips.city LIKE '%$city%'
AND zips.stateabbr LIKE '%$state%'
LIMIT 1";
With that, if you still have an error about a non-existing or not-found column (at least for city and/or stateabbr), it'll be because that column doesn't exist in your table.
In this case, there is not much we can do : you'll have to check the structure of your table, to determine which columns it contains.
You can check that structure using a web-based tool like phpMyAdmin, or using an SQL instruction such as :
desc zips;
For reference, quoting MySQL's manual page for select :
The SQL standard requires that HAVING
must reference only columns in the
GROUP BY clause or columns used in
aggregate functions. ...
Do not use HAVING for items that
should be in the WHERE clause.
For example, do not write the
following:
SELECT col_name FROM tbl_name HAVING col_name > 0;
Write this instead:
SELECT col_name FROM tbl_name WHERE col_name > 0;
...
The HAVING clause can refer to
aggregate functions, which the WHERE
clause cannot
Try using WHERE instead of HAVING.
The proper way to do it is by using a WHERE clause.
$query = "SELECT
zips.*
FROM
zips
WHERE
zips.city LIKE '%$city%'
AND
zips.stateabbr LIKE '%$state%'
LIMIT 1";
HAVING is to be used when you are GROUPing, see here for an explanation
o jeez sorry guys i figured out the problem, apparently i put a space before city when i named the columns in my table. so i renamed the column and it works thanks anyway chaps! but using the where function instead of having must speed things up alot, thanks guys!