I have a table with about 150 websites listed in it with the columns "site_name", "visible_name" (basically a formatted name), and "description." For a given page on my site, I want to pull site_name and visible_name for every site in the table, and I want to pull all three columns for the selected site, which comes from the $_GET array (a URL parameter).
Right now I'm using 2 queries to do this, one that says "Get site_name and visible_name for all sites" and another that says "Get all 3 fields for one specific site." I'm guess a better way to do it is:
SELECT * FROM site_list;
thus reducing to 1 query, and then doing the rest post-query, which brings up 2 questions:
The "description" field for each site is about 200-300 characters. Is it bad from a performance standpoint to pull this for all 150 sites if I'm only using it for 1 site?
How do I reference the specific row from the MySQL result set for the site specificed in the URL? For example, if the URL is "mysite.com/results?site_name=foo" how do I do the post-query equivalent of SELECT * FROM site_list where site_name=foo; ?
I don't know how to get the data for "site_name=foo" without looping through the entire result array and checking to see if site_name matches the URL parameter. Isn't there a more efficient way to do it?
Thanks,
Chris
PS: I noticed a similar question on stackoverflow and read through all the answers but it didn't help in my situation, which is why I'm posting this.
Thanks,
Chris
I believe what you do now, keeping sperated queries for getting a list of sites with just titles and one detailed view with description for a single given site, is good. You don't pull any unneeded data and both queries being very simple are fast.
It is possible to combine both your queries into one, using left join, something maybe like:
SELECT s1.site_name, s1.visible_name, s2.description
FROM site_list s1
LEFT JOIN
( SELECT site_name, description
FROM site_list
WHERE site_name = 'this site should go with description' ) s2
ON s2.site_name = s1.site_name
resulting in all sites without matching name having NULL as description, you could even sort it using
ORDER BY description DESC, site_name
to get the site with description as first fetched row, thus eliminating need to iterate through results to find it, but mysql would have to do a lot more work to give you this result, negating any possible gain you could hope for. So basically stick to what you have now, its good.
Generally, it's good practice to have an 'id' field in the table as an auto_increment value. Then, you would:
SELECT id,url,display_name FROM table;
and you'd have the 'id' value there to later:
SELECT * FROM table WHERE id=123;
That's probably your most efficient method if you had WAAAY more entries in the table.
However, with only 150 rows in the table, you're probably just fine doing
SELECT * FROM table;
and only accessing that last field for a matching row based on your criteria.
If you only need the description for the site named foo you could just query the database with SELECT * FROM site_list WHERE site_name = 'foo' LIMIT 1
Otherwise you would have to loop though the result array and do a string comparison on site_name to find the correct description.
Related
I have been trying to create a database for fun to get a better understanding of databases and using PHP to query them for a website I'm messing around with. Pretty much I have one database with 4 tables when a user enters a search term in a PHP search box my code searches the database for any entries containing the search term. Now I can easily get my code to search individual tables, but I cannot seem to get it to search all 4 tables and display the results on the same page.
info: making a database for skyrim
Table names: classes, powers, skills, shouts
column names: name, information
Here is a snippet of the code I have that works so far:
$raw_results = mysql_query("
SELECT *
FROM `xaviorin_skyrim`.`shouts` , `xaviorin_skyrim`.`classes`
WHERE (CONVERT(`UID` USING utf8) LIKE '%".$query."%' OR
CONVERT(`Name` USING utf8) LIKE '%".$query."%' OR
CONVERT(`Information` USING utf8) LIKE '%".$query."%')
") or die(mysql_error());`
Literally all I thought I would need to do is change the table name from "shouts" to say "classes" in a new raw_results line of code but that didn't work. I have attempted unions and joins and either keep screwing them up or just don't understand how to properly format them.
echo "<p><h3>".$results['Name']."</h3>".$results['Information']."</p>";
The code above this text is what displays the results on the page on my website... it works but I don't know how to combine the information from all 4 tables into one page. If I'm going about this in the wrong way and anyone can point me in the right direction I would GREATLY appreciate it... I've been trying to research the problem without finding a proper answer for near a month now.
The problem with your approach is that relational databases do a cross join when there are several query results from two different tables. So basically every match in one table will be combined with every match from the second table. When you have 3 entries in the first and 4 in the second table, you will get 3 * 4 = 12 entries in your query result. If you add more tables, you get even more results. You want to do a full text search in several tables that are totally unrelated, thus creating some kind of non-existing relation via cross joining them will not be useful.
What you actually want to do is a UNION ALL (UNION is slower because it prunes duplicates) of several queries:
SELECT name, information, 'shouts' AS tablename FROM shouts WHERE ...
UNION ALL
SELECT name, information, 'classes' AS tablename FROM classes WHERE ...
This will do search queries on every single table and then place the results in a single result. Also note that I added a third column to each query to ensure that the originating table is not lost after merging the results.
Unless you need to do some sorting afterwards, I would suggest that you do all statements separately. Combining them this way will most likely make the post-processing more complex. And several single queries will also be faster than one big query with UNION statements.
And as I mentioned in the comments: Don't use mysql_* functions!
I have one table called cf_posts
ID pk
user INT
subject VARCHAR
body TEXT
datetime TIMESTAMP
parent INT
category INT
mod INT
When a post is submitted to the forum the default parent is 0, when a post is submitted as a reply, then its parent is the ID of the original post.
How can I make it so that the default view of the forum main page is ordered that the most recently updated post (including the latest replies) would be at the "top" of the pile, and working down in date order? What would be the PHP/MySQL query?
The only workarounds I have seen for this are separate topics and reply tables, but I'd like to stay away from this approach if possible.
One workaround that I tried and failed was GROUP BY parent.
But this grouped all topics that had no replies together as one.
Another idea that I have yet to try is to make the parent id of the original post match the post ID, and not include matching ID and parent IDs in the output.
I look forward to hearing your thoughts.
SELECT mainPost.subject, lastPost.datetime
FROM cf_posts cfp,
(
SELECT *
FROM cf_posts subPost
WHERE subPost.parent = mainPost
ORDER BY subPost.datetime DESC
LIMIT 1
)lastPost
WHERE mainPost.parent IS NULL
This is done briefly, so there may be some syntax issues but I think this should help.
You can do the following: query each separate thing that you need, so maybe a query for each topic, Then you can use UNION to bunch all of them together to get one list. Now the trick to preserve an order is as followed. For each separate query append a column to the returned result called sort and set each instance of that to a higher int value, then you can guarantee that the final result is properly sorted. Go review UNION for select statements to get a better understanding of what I'm talking about.
My PHP application has the function product_fetch([parameters]) which returns 'Product' objects which information are stored in database.
In my admin area, there is a page called "Featured Products" which allows me to select 10 products to be displayed in the main page.
Now comes the problem: I made 10 select/combobox, each allows me to select one product, out of 400. So in order to make all the options, a query has to be made: SELECT * FROM products
Question: Is it correct to make such a query, even though there's hundreds of rows?
The solution you proposed is certainly do-able, and 400 rows is really meek compared to the upper limits of what MySQL is capable of handling. What is more concerning is the user experience here. Granted this will only affect you, but I would design myself something a little nicer than a bunch of <select>s. My idea is to start with just one textbox that autocompletes the names of your products. This can be accomplished if the product title has a fulltext index. Then your autocomplete script could use this query:
SELECT * FROM Products WHERE MATCH(title) AGAINST ('contents of textbox' IN BOOLEAN MODE);
There are plenty of jQuery plugins like Autocomplete that will handle the JS side (querying the server for autocomplete results). The one I just mentioned adds a term GET parameter which you could easily grab and throw into the query:
// You might want to only select the relevant columns
$stmt = $pdo->prepare('SELECT * FROM Products WHERE MATCH(title) AGAINST (:search IN BOOLEAN MODE)');
$stmt->execute(array(':search' => $_GET['term']);
// Output JSON results
echo json_encode($stmt->fetchall());
Once you type in (or click on an autocomplete result) in the one textbox, another should appear below it and the focus should go to it. From there you can type another product, and continue until you reach 10. Each textbox (unless there is only one) should have a delete link next to it that removes that product from the featured listing.
I'll look for a site that implements this kind of functionality so you can better understand what I'm talking about. Basically here, what you're searching for is achieved. You don't have to have 10 select boxes with 400 options and you don't need a SELECT * FROM Products.
You would be much better of specifying which produts you want in the query and only returning those if you have no intention of using any of the others at all.
You can do this using many methods. A simple one would be using an ID field in an in statement like this:
select col1, col2 from products where id in(1,4,12,5)
It might seem to make little difference, but what if your procuts table had a hundred thousand rows in it?
You could also have a flag in the table to say that the items are featured which would let you use something like this:
select col1, col2 from products where featured='Y'
Or you could even have a table that only has featured items (even just their ID) and join it to your main listing like this:
select
a.col1,
a.col2
from
products a
join featured b
on a.id=b.id
If you want to pick through your whole table, you can even use a simple limit clause that picks up a certain number of rows from the table and can be reused to get the next set:
select col1, col2 from products limit 10;
// Will pick the first 10 rows of data.
select col1, col2 from procuts limit 30,10;
// Will pick rows 31-40 (skipping first 30, then returning next 10)
The short version is though, no matter how you do it, pulling back a whole table of data to pick through it within PHP is a bad thing and to be avoided at all costs. It makes the database work much harder, uses more network between the database and PHP (even if they are on the same machine, it is transferring a lot more data between the two of them) and it will by default make PHP use a lot more resources to process the information.
SELECT * FROM tbl LIMIT $page,10;
The above query will select 10 entries from offset $page
Fetching all rows is a bad idea as you are using 10 only anyway. When your table expands to millions of rows, you will see a noticeable difference.
There is nothing fundamentally wrong with selecting all rows if that's what you mean to do. Hundreds of rows also shouldn't be a problem for the query in terms of performance. However, thousands or millions might if your database grows that big.
Think about the user - can they scroll through hundreds of products? Probably. If not, then maybe it's the UI design at fault, not the query.
I have a table in MySQL that I'm accessing from PHP. For example, let's have a table named THINGS:
things.ID - int primary key
things.name - varchar
things.owner_ID - int for joining with another table
My select statement to get what I need might look like:
SELECT * FROM things WHERE owner_ID = 99;
Pretty straightforward. Now, I'd like users to be able to specify a completely arbitrary order for the items returned from this query. The list will be displayed, they can then click an "up" or "down" button next to a row and have it moved up or down the list, or possibly a drag-and-drop operation to move it to anywhere else. I'd like this order to be saved in the database (same or other table). The custom order would be unique for the set of rows for each owner_ID.
I've searched for ways to provide this ordering without luck. I've thought of a few ways to implement this, but help me fill in the final option:
Add an INT column and set it's value to whatever I need to get rows
returned in my order. This presents the problem of scanning
row-by-row to find the insertion point, and possibly needing to
update the preceding/following rows sort column.
Having a "next" and "previous" column, implementing a linked list.
Once I find my place, I'll just have to update max 2 rows to insert
the row. But this requires scanning for the location from row #1.
Some SQL/relational DB trick I'm unaware of...
I'm looking for an answer to #3 because it may be out there, who knows. Plus, I'd like to offload as much as I can on the database.
From what I've read you need a new table containing the ordering of each user, say it's called *user_orderings*.
This table should contain the user ID, the position of the thing and the ID of the thing. The (user_id, thing_id) should be the PK. This way you need to update this table every time but you can get the things for a user in the order he/she wants using ORDER BY on the user_orderings table and joining it with the things table. It should work.
The simplest expression of an ordered list is: 3,1,2,4. We can store this as a string in the parent table; so if our table is photos with the foreign key profile_id, we'd place our photo order in profiles.photo_order. We can then consider this field in our order by clause by utilizing the find_in_set() function. This requires either two queries or a join. I use two queries but the join is more interesting, so here it is:
select photos.photo_id, photos.caption
from photos
join profiles on profiles.profile_id = photos.profile_id
where photos.profile_id = 1
order by find_in_set(photos.photo_id, profiles.photo_order);
Note that you would probably not want to use find_in_set() in a where clause due to performance implications, but in an order by clause, there are few enough results to make this fast.
I know i am writing query's wrong and when we get a lot of traffic, our database gets hit HARD and the page slows to a grind...
I think I need to write queries based on CREATE VIEW from the last 30 days from the CURDATE ?? But not sure where to begin or if this will be MORE efficient query for the database?
Anyways, here is a sample query I have written..
$query_Recordset6 = "SELECT `date`, title, category, url, comments
FROM cute_news
WHERE category LIKE '%45%'
ORDER BY `date` DESC";
Any help or suggestions would be great! I have about 11 queries like this, but I am confident if I could get help on one of these, then I can implement them to the rest!!
Putting a wildcard on the left side of a value comparison:
LIKE '%xyz'
...means that an index can not be used, even if one exists. Might want to consider using Full Text Searching (FTS), which means adding full text indexing.
Normalizing the data would be another step to consider - categories should likely be in a separate table.
SELECT `date`, title, category, url, comments
FROM cute_news
WHERE category LIKE '%45%'
ORDER BY `date` DESC
The LIKE '%45%' means a full table scan will need to be performed. Are you perhaps storing a list of categories in the column? If so creating a new table storing category and news_article_id will allow an index to be used to retrieve the matching records much more efficiently.
OK, time for psychic debugging.
In my mind's eye, I see that query performance would be improved considerably through database normalization, specifically by splitting the category multi-valued column into a a separate table that has two columns: the primary key for cute_news and the category ID.
This would also allow you to directly link said table to the categories table without having to parse it first.
Or, as Chris Date said: "Every row-and-column intersection contains exactly one value from the applicable domain (and nothing else)."
Anything with LIKE '%XXX%' is going to be slow. Its a slow operation.
For something like categories, you might want to separate categories out into another table and use a foreign key in the cute_news table. That way you can have category_id, and use that in the query which will be MUCH faster.
Also, I'm not quite sure why you're talking about using CREATE VIEW. Views will not really help you for speed. Not unless its a materialized view, which MySQL doesn't suppose natively.
If your database is getting hit hard, the solution isn't to make a view (the view is still basically the same amount of work for the database to do), the solution is to cache the results.
This is especially applicable since, from what it sounds like, your data only needs to be refreshed once every 30 days.
I'd guess that your category column is a list of category values like "12,34,45,78" ?
This is not good relational database design. One reason it's not good is as you've discovered: it's incredibly slow to search for a substring that might appear in the middle of that list.
Some people have suggested using fulltext search instead of the LIKE predicate with wildcards, but in this case it's simpler to create another table so you can list one category value per row, with a reference back to your cute_news table:
CREATE TABLE cute_news_category (
news_id INT NOT NULL,
category INT NOT NULL,
PRIMARY KEY (news_id, category),
FOREIGN KEY (news_id) REFERENCES cute_news(news_id)
) ENGINE=InnoDB;
Then you can query and it'll go a lot faster:
SELECT n.`date`, n.title, c.category, n.url, n.comments
FROM cute_news n
JOIN cute_news_category c ON (n.news_id = c.news_id)
WHERE c.category = 45
ORDER BY n.`date` DESC
Any answer is a guess, show:
- the relevant SHOW CREATE TABLE outputs
- the EXPLAIN output from your common queries.
And Bill Karwin's comment certainly applies.
After all this & optimizing, sampling the data into a table with only the last 30 days could still be desired, in which case you're better of running a daily cronjob to do just that.