I am building a dynamic news application for a site. The news will be split up into categories which may then contain subcategories(which may contain subcategories) and news articles. I have created a recursive function to return all of the category IDs for all subcategories of a certain category. I would like to run a single query to grab all of the news posts whose category is in the set that was returned by the recursive function.
When I try to use this (I'M USING PEAR DB)
//$myReturnedIDs is a comma delimited list of ids. 5,6,7,8,10,12
$oPrep = $oConn->prepare("SELECT NewsID FROM SiteNews WHERE NewsCategory IN (?)");
$oRes = $oConn->execute($oPrep, array($myReturnedIDs));
The resulting query looks like this:
SELECT NewsID FROM SiteNews WHERE NewsCategory IN('5,6,7,8,10,12')
while I need it to look like:
SELECT NewsID FROM SiteNews WHERE NewsCategory IN(5,6,7,8,10,12)
Now, I realize I could populate that portion of the query in the prepare statement, but I feel that leads to some possible hole for an attack. Am I being over cautious, since the data is not user generated? Is there some way to make this work? My other thought was to add a ? for each ID, amount would be determined before the prepare statement (after getting all the IDs), but that feels clunky to me. Any suggestions?
I'm going to guess that $myReturnedIDs is a comma-delimited string. So this:
array($myReturnedIDs)
yields a one-element array:
["5,6,7,8,10,12"]
If that's the case, then you want to use explode to make the array you want:
$oRes = $oConn->execute($oPrep, explode(",", $myReturnedIDs));
Hope that helps!
Related
I'm using implode to insert few values into one row in MySQL database.
implode(' ', $_POST['tag']);
Assuming that I have table named product with row named tags with 3 different values that inserted inside like this:
usb adapter charger
I have tried using this method using like operator (%), but that didn't worked.
$sql = "SELECT * FROM product WHERE tags='%usb%'";
How can I extract only one value from the imploded array using WHERE in mysql query?
I agree with the comments about re-designing the database. At first read it seems that using LIKE would definitely get the result you want but after reading #Patrick Q's pan - panther example, it makes a lot sense that LIKE is not really a good solution. There are ways to get exactly the tag string you're looking for but it may hurt the performance and the query will be longer and complex. Hence the following are to demonstrate how the query would look like with your current tags data value:
MySQL query:
SELECT tags,
SUBSTRING_INDEX(SUBSTRING_INDEX(tags,' ',FIND_IN_SET('usb',REPLACE(tags,' ',','))),' ',-1) v
FROM mytable
HAVING v = 'usb';
As you can see, there are a few functions being used just to get the exact string from the data cell. Since your example data was separating with spaces and FIND_IN_SET identify value separation by comma, REPLACE take place on the tags column first to replace spaces with comma. Then with SUBSTRING_INDEX twice to get the string using the location extracted in FIND_IN_SET. Finally at the end HAVING to get only the tag you're looking for.
Further demo here : https://www.db-fiddle.com/f/joDa7MNcQL2RakTgBa7qBM/3
I have a large database and even with indexes it's slowing down a lot when I use SELECT DISTINCT. I thought I'd change it to use an array instead but somehow it's not working. What am I doing wrong?
The original code is:
$product_countries = $this->query("SELECT DISTINCT country FROM db_products");
$listing_countries = array();
while ($listing_details = $this->fetch_array($product_countries)){
if (!empty($listing_details['country'])){
$listing_countries[] = $listing_details['country'];
}
}
Now, I tried changing it to:
// $product_countries = $this->query("SELECT DISTINCT country FROM db_products");
$product_countries = array("2084", "1993");
And the while part to:
while ($listing_details = $product_countries){
But that's not working. What am I doing wrong here? I'm pretty sure it's somewhere in that while line and the fact that the before it used a mySQL resource and then afterwards I changed it to an array, but I can't figure it out :(
Given:
$product_countries = array("2084", "1993");
There's no need for a while loop. Just get the contents of that array assigned to another array. Like this:
$product_countries = array("2084", "1993");
$listing_details = $product_countries;
And be done with it. (You already know that the values in the $product_countries array are already defined, and aren't going to evaluate to FALSE, so there's no need for any conditional test.
With the while loop as you show, how will that ever exit? Looks like a classic infinite loop to me.
If that's not the question you are asking...
As far as the MySQL query goes, the only appropriate index for the query would be a BTREE index (not a HASH index) with a leading column of country. Ideally (for this query) on just that single column:
... ON db_products(country)
The query should be equivalent to a GROUP BY query:
SELECT country FROM db_products GROUP BY country
I recommend you run an EXPLAIN on the query to verify that it's using an index and not doing a "Using filesort" operation. (This assumes that db_products is a table, and not some harebrained view definition.)
I'm not understanding why the conditional test on the return of the empty function is needed. If fetch_array returned a row, then variable is going to be defined, so that test really only checking if the value returned from the database evaluates to FALSE. If I wanted to exclude values of country that evaluate to FALSE, I would tend to include the appropriate predicate in the query definition.
First, this query:
SELECT DISTINCT country
FROM db_products
should be able to make good use of an index on db_products(country).
Second, you should have a separate table of countries, with db_products containing a countryId, referring to the other table. If you had that, then displaying the list of countries would be a no-brainer.
AS I stated in the comments, the better way to do this is have a list of countries in a separate table, and reference them in your product table.
This is called normalization, generally you should think of it like this. A product is a thing with certain attributes of which country is only one, it might have a price, or a SKU number for example..
Countries too are a thing, they may have abbreviations, shipping codes, laws that deal with this or that, many things. They are not as simple as say a SKU number or a price. But, could contain other useful data unrelated to a specific product, don't they deserve a table all their own?
It's hard to say the structure of this table or the relationship between a product and a country without knowing more. But, you may be able to eliminate a lot of redundant data. Ask yourself these two things:
Can a product have more then one country?
Can a country supply more then one product?
Those will tell you the relationship you need.
I'm making a car part system, to store all the parts inside mysql and then search for them.
Part adding goes like this:
you select up to 280 parts and add all the car info, then all the parts are serialized and put into mysql along with all the car info in a single row.
(for this example I'll say that my current database has 1000 cars and all of those cars have 280 parts selected)
The problem is that when I have 1000 cars with each of them having 280 parts, php and mysql starts getting slow and takes a lot of time to load the data, because the number of parts is 1000*280=280 000.
I use foreach on all of the cars and then put each part into another array.
The final array has 280 000 items and then I filter it by the selected parts in the search, so out of 28 000 parts it may have only have to print like 12 500 parts (if someone is searching for 50 different parts at the same time and 250 cars have that part).
Example database: http://pastebin.com/aXrpgeBP
$q=mysql_query("SELECT `id`,`brand`,`model`,`specification`,`year`,`fueltype`,`capacity`,`parts`,`parts_num` FROM `warehouse`");
while($r=mysql_fetch_assoc($q)){
$partai=unserialize($r['parts']);
unset($r['parts']); //unsetting unserialized parts so the whole car parts won't be passed into the final parts-only array
foreach($partai as $part){
$r['part']=$parttree[$part]; //$parttree is an array with all the part names and $part is the part id - so this returns the part name by it's id.
$r['part_id']=$part; // saves the part id for later filtering selected by the search
$final[]=$r;
}
}
$selectedparts=explode('|', substr($_GET['selected'], 0,strlen($_GET['selected'])-1)); //exploding selected part ids from data sent by jquery into an array
foreach($final as $f){
if(in_array($f['part_id'], $selectedparts)){
$show[]=$f; //filtering only the parts that need to be shown
}
}
echo json_encode($show);
This is the code I use to all the cars parts into arrays and the send it as json to the browser.
I'm not working on the pagination at the moment, but I'll be adding it later to show only 10 parts.
Could solution be to index all the parts into a different table once 24h(because new parts will be added daily) and then just stressing mysql more than php? Because php is doing all the hard work now.
Or using something like memcached to store the final unfiltered array once 24h and then just filter the parts that need to be shown with php?
These are the options I considered, but I know there must be a better way to solve this.
Yes, you should definitely put more emphasis on MySQL. Don't serialize the parts for each car into a single row of a single column. That's terribly inefficient.
Instead, make yourself a parts table, with columns for the various data items that describe each part.
part_id an autoincrement item.
car_id which car is this a part of
partnumber the part's external part number (barcode number?)
etc
Then, use JOIN operations.
Also, why don't you use a WHERE clause in your SELECT statement, to retrieve just the car you want?
Edit
If you're looking for a part, you definitely want a separate parts table. Then you can do a SQL search something like this.
SELECT w.id, w.model, w.specification, w.year, w.fueltype,
p.partnumber
FROM warehouse w
JOIN parts p ON (w.id = p.car_id)
WHERE p.partnumber = 'whatever-part-number-you-want'
This will take milliseconds, even if you have 100K cars in your system, if you index it right.
Your query should be something like:
<?php
$selectedparts=explode('|', substr($_GET['selected'], 0,strlen($_GET['selected'])-1)); //exploding selected part ids from data sent by jquery into an array
$where = ' id < 0 ';
foreach ($selectedparts AS $a){
$where .= " OR `parts` like '%".$a."%'";
}
$query = "SELECT * FROM `warehouse` WHERE ".$where." ORDER BY `id` ASC";//this is your query
//.... rest of your code
?>
Yes, look into has many relationships a car has many parts.
http://net.tutsplus.com/tutorials/databases/sql-for-beginners-part-3-database-relationships/
Then you can use an inner join to get the specified parts. You can do a where clause to match the specific partIds to filter out unwanted parts or cars.
I feel a little embarrassed as there is probably an easy solution, but I don't know enough MySQL to do it. How do I use one query to get data from each of these tables, and then return an array as illustrated below? Every attempt I make ends up returning either one tag, or returning multiple arrays of the same task, each with a different tag.
What should my query structure look like?
Thanks!
http://i.stack.imgur.com/ViqEs.png
The image's array actually shows how the data would look like after two queries, not one. To be able to do it in a single query, and this is because the data is not too complex, you could use a GROUP_CONCAT() to get all of the tags for a task and then use post-query logic to split the data into separate arrays.
The SQL query to get all of the requested data would be:
SELECT
tasks.*, GROUP_CONCAT(tag_name) AS tags
FROM
tasks LEFT JOIN tags ON tags.task_id=tags.id
WHERE
id=2
This query will return a single record; in that record, the column tags will hold a comma-separated list of all of the tags that belong to the task. You can split the data in that column into an array to build your desired structure.
An example, with PHP:
$result = mysql_query("SELECT tasks.*, GROUP_CONCAT(tag_name) AS tags FROM tasks LEFT JOIN tags ON tags.task_id=tags.id WHERE id=2");
// create the "$task" array that has a "task" and "tags" index
$task = array('task' => array(), 'tags' => array());
$task['task'] = mysql_fetch_assoc($result);
// split the comma-separated list of tags into an array
$task['tags'] = explode(',', $task['task']['tags']);
// delete the original "tags" entry that's returned by the sql query
unset($task['task']['tags']);
Please note that this example is void of any data validation, connection information, or other logic and should just be used as a rough idea as how you could split the data into your desired structure.
There seems to be no shortage of hierarchical data questions in MySQL on SO, however it seems they are mostly talking about managing such data in the database or actually retrieving recursively hierarchical data. My situation is neither. I have a grid of items I need to display. Each item can also have 0 or more comments associated with it. Right now, both the item, along with its data, are displayed in the grid as well as any comments belonging to that item. Usually there is some sort of drill down, dialog, or other user action required to see child data for a grid item but in this case we display both parent and child data in the same grid. Might not fit the de facto standards but it is what it is.
Right now the comments are retrieved by a separate MySQL query for every single parent item in the grid. I immediately cringe at this being aware of all the completely separate database queries that have to be run for a single page load. I haven't profiled but I wouldn't be too surprised if this is part of the slow page loads we sometimes see. I'd like to ideally bring this down to a single query or perhaps 2. However, I'm having difficulty coming up with a solution that sounds any better than what is currently being done.
My first thought was to flatten the comment children for each row with some sort of separator like '|' and then explode them back apart in PHP when rendering the page. The issue with this is it gets increasingly complicated with having to separate each field in a comment, and then each comment, and then account for the possibility of separator characters in the data. Just feels like a mess to maintain and debug.
My next thought was to left outer join the comments to the items and just account for the item duplicates in PHP. I'm working with Codeigniter's database library that returns a PHP array for database data. This sounds like potentially a lot of duplicated data in the resulting array which could possibly be system taxing for larger result sets. I'm thinking in most cases it wouldn't be too bad though so this option is currently at the top of my possibilities list. Ideally, if I understand MVC correctly, I should keep my database, business logic, and view/display as separate as possible. So again, ideally, there should not be any database "quirks" (for lack of a better word) apparent in the data returned by the model. That is, whatever calls for data from this model method, shouldn't be concerned with duplicate data like this. So I'd have to add on an additional loop to somehow eliminate the duplicate item array entries but only after I have retrieved all the child comments and placed them into their own array.
Two queries is another idea but then I have to pass numerous item IDs in the SQL statement for the comments and then go through and zip all the data together manually in PHP.
My goal isn't to get out of doing work here but I am hoping there is some more optimal (less resource intensive and less confusing to the coder) method I haven't thought of yet.
As you state in your question, using a join will bring back a lot of duplicate information. It should be simple enough to remove in PHP, but why bring it back in the first place?
Compiling a SQL statement with a list of IDs retrieved from the query for your list of items shouldn't be a problem (see cwallenpoole's answer). Alternatively, you could create a sub-query so that MySQL recreates the list of IDs for you - it depends on how intensive the sub-query is.
Select your items:
SELECT * FROM item WHERE description = 'Item 1';
Then select the comments for those items:
SELECT * FROM comment WHERE item_id IN (
SELECT id FROM item WHERE description = 'Item 1'
);
For the most part, I solve this type of problem using some sort of ORM Lazy-Loading system but it does not look like you've that as an option.
Have you considered:
Select all top-level items.
Select all second-level items by the ID's in the top-level set.
Associate the objects retrieved in 2 with the items found in 1 in PHP.
Basically (in pseudo-code)
$stmt = $pdo->query("SELECT ID /*columns*/ FROM ENTRIES");
$entries = array();
foreach( $row as $stmt->fetchAll(PDO::FETCH_ASSOC) )
{
$row['child-entities'] = array();
$entries[$row['id']] = $row;
}
$ids = implode(',',array_keys($entries));
$stmt = $pdo->query("SELECT PARENT_ID /*columns*/ FROM children WHERE PARENT_ID IN ($ids)");
foreach( $row as $stmt->fetchAll(PDO::FETCH_ASSOC) )
{
$entries[$row['parent_pid']]['child-entities'][] = $row;
}
$entries will now be an associative array with parent items directly associated with child items. Unless recursion is needed, that should be everything in two queries.