i dont know if i am doing right or wrong, please dont judge me...
what i am trying to do is that if a record belongs to parent then it will have parent id assosiated with it.. let me show you my table schema below.
i have two columns
ItemCategoryID &
ItemParentCategoryID
Let Suppose a record on ItemCategoryID =4 belongs to ItemCategoryID =2 then the column ItemParentCategoryID on ID 4 will have the ID of ItemCategoryID.
I mean a loop with in its own table..
but problem is how to run the select query :P
I mean show all the parents and childs respective to their parents..
This is often a lazy design choise. Ideally you want a table for these relations or/and a set number of depths. If a parent_id's parent can have it's own parent_id, this means a potential infinite depth.
MySQL isn't a big fan of infinite nesting depths. But php don't mind. Either run multiple queryies in a loop such as Nil'z's1, or consider fetching all rows and sorting them out in arrays in php. Last solution is nice if you pretty much always get all rows, thus making MySQL filtering obsolete.
Lastly, consider if you could have a more ideal approach to this in your database structure. Don't be afraid to use more than one table for this.
This can be a strong performance thief in the future. An uncontrollable amount of mysql queries each time the page loads can easily get out of hands.
Try this:
function all_categories(){
$data = array();
$first = $this->db->select('itemParentCategoryId')->group_by('itemParentCategoryId')->get('table')->result_array();
if( isset( $first ) && is_array( $first ) && count( $first ) > 0 ){
foreach( $first as $key => $each ){
$second = $this->db->select('itemCategoryId, categoryName')->where_in('itemParentCategoryId', $each['itemParentCategoryId'])->get('table')->result_array();
$data[$key]['itemParentCategoryId'] = $each['itemParentCategoryId'];
$data[$key]['subs'] = $second;
}
}
print_r( $data );
}
I don't think you want/can to do this in your query since you can nest a long way.
You should make a getChilds function that calls itself when you retrieve a category. This way you can nest more than 2 levels.
function getCategory()
{
// Retrieve the category
// Get childs
$childs = $this->getCategoryByParent($categoryId);
}
function getCategorysByParent($parentId)
{
// Get category
// Get childs again.
}
MySQL does not support recursive queries. It is possible to emulate recursive queries through recursive calls to a stored procedure, but this is hackish and sub-optimal.
There are other ways to organise your data, these structures allow very efficient querying.
This question comes up so often I can't even be bothered to complain about your inability to use Google or SO search, or to offer a wordy explanation.
Here - use this library I made: http://codebyjeff.com/blog/2012/10/nested-data-with-mahana-hierarchy-library so you don't bring down your database
Related
I can't figure out how to efficiently get SQL data for a Room/Rates/Dates=Amount...
First I load all the "RateData" for a date range with a PDO select. There are many rows, for many rooms, each with many rates... maybe, or maybe it is all empty except a couple of Amounts. It needs to display $0 for missing dates, so next...
I load the Rooms with PDO and loop through them, and for each room I load the Rates with PDO and loop through them (not a ton of rates per room, and not a ton of rooms, but possibly a very long date-range).
So then I loop through the date range and add $0 to the giant UI grid of Amounts by Rate/Date, nested under each Room. I have to do this anyway, as I also have a ton of logic on what to display in the parent Room row that averages the Rates and such.
So what I need to do is instead of using $0, I need to see if the Room/Rate/Date exists in RateData...
$RateAmount = 0;
$RateDataRow = $RateData.filter('Room=1 && Rate=1 && Date=2022-10-01');
If ($RateDataRow exists) {$RateAmount = $RateDataRow['Amount']}
How to I write the above sudocode in PHP?
The only alternative I can think of would be to do 1000's of SQL calls to populate the grid... which seems bad. Maybe it is not that bad though if PDO caches and doesn't actually query the DB for each grid cell. Please advise. Thanks.
I tried this:
$currcost = 0;
//if $ratedata exists for currdate + currrate + currroom
function ratematch($row)
{if (($row['RoomType_Code']=$currrate)
&& ($row['RatePlan_Code']=$currroom)
&& ($row[ 'Rate_Date']=$currdate->format("Y-m-d")))
{return 1;}
else {return 0;}
}
$match = array_filter($ratedata, 'ratematch');
if (!empty($match)) {$currcost = $match['Rate_Amount'];}
But got an error about redeclaring a function. I have to redeclare it because it is in a loop of currdate under a loop of currrate under a loop of currroom (about 1000 cells).
I made it work, I just have to manually loop through the RateData...
//Get Amount from DB
$currcost = 0;
foreach($ratedata as $datarow)
{if (($datarow['RoomType_Code']==$currroom)
&& ($datarow['RatePlan_Code']==$currrate)
&& ($datarow[ 'Rate_Date']==$currdate->format("Y-m-d")))
{$currcost = $datarow['Rate_Amount'];
break;
}
}
If anyone knows a more efficient way to "query" a previous query without a trip to the SQL server, please post about it. Looping through the fetchAll 1000's times seems bad, but not as bad as doing 1000's of WHERE queries on the SQL Server.
I suppose I could make a 3-dimentional array of $0, then just loop through the RateData once to update that array, and finally loop through the 3-dimentional array to do my other calculations (average rate per room per week sat-sun). Sounds hard though.
So, basically, i have to loop thought an array of 25000 items, then compare each item with another array's ID and if the ID's from the first array and the second match then create another array of the matched items. That looks something like this.
foreach ($all_games as $game) {
foreach ($user_games_ids as $user_game_id) {
if ($user_game_id == $game["appid"]) {
$game_details['title'][] = $game["title"];
$game_details['price'][] = $game["price"];
$game_details['image'][] = $game["image_url"];
$game_details['appid'][] = $game["appid"];
}
}
}
I tested this loop with only 2500 records from the first array ($all_games) and about 2000 records from the second array ($user_games_ids) and as far as i figured, it takes about 10 seconds for the execution of that chunk of code, only the loops execution. Is that normal? Should that take that long or I'm I approaching the issue from the wrong side? Is there a way to reduce that time? Because when i apply that code to 25000 records that time will significantly increase.
Any help is appreciated,
Thanks.
EDIT: So there is no confusion, I can't use the DB query to improve the performance, although, i have added all 25000 games to the database, i can't do the same for the user games ids. There is no way that i know to get all of the users through that API i'm accessing, and even there is, that would be really a lot of users. I get user games ids on the fly when a user enters it's ID in the form and based on that i use file_get_contents to obtain those ids and then cross reference them with the database that stores all games. Again, that might not be the best way, but only one i could think of at this point.
If you re-index the $game array by appid using array_column(), then you can reduce it to one loop and just check if the data isset...
$game = array_column($game,null,"appid");
foreach ($user_games_ids as $user_game_id) {
if (isset( $game[$user_game_id])) {
$game_details['title'][] = $game[$user_game_id]["title"];
$game_details['price'][] = $game[$user_game_id]["price"];
$game_details['image'][] = $game[$user_game_id]["image_url"];
$game_details['appid'][] = $game[$user_game_id]["appid"];
}
}
I am wondering if there is a way to do what I am doing more efficiently. Right now, I have a class that retrives statuses from the database. It's pretty simple and shouldn't really effect performance all that much.
public function get ($var1, $var2, $var3)
{
$feed = array(); //Initialize an empty array
//Query the database
$Statement = $this->Database->prepare("SELECT id, name, excerpt, post, timestamp, tags, title FROM posts WHERE col1 = ? AND col2 = ? AND col3 = ? ORDER BY id DESC LIMIT 15");
$Statement->execute(array($var1, $var2, $var3));
while($row = $Statement->fetch(PDO::FETCH_ASSOC))
{
$posts[] = array( "id" => $row["id"], /*etc...*/ );
}
return $posts;
} //end get
And then my page set up something like this which I know is not efficient at all:
<?php for ($count = 1; $count <= $total; $count++): //Display the calendar
echo $count;
$feed = $Feed->get($count, $total, $var3);
foreach ($feed as $post):
echo $post["id"];
endforeach;
endfor; ?>
I hope that makes sense. There's a lot more html thrown in there and everything. Right now there are only 18 rows in my database, and it takes 10 seconds to load the page. Which is really bad. I have to set it up this way because of the design of the site. So the foreach loop has to be within the for loop because the whole thing is set up as a calendar.
My question is whether it would be more efficient to select all of the rows, save them outside of the for loop and then work with that array, or whether it's better to run each query inside the foreach loop the way I'm doing it now (i've read a lot, and know that most people say this is a huge no no). And what kind of issues would I run into if I used the former option and there were say a million rows in the database.
I hope that makes sense. I'll update the question if it doesn't. Right now though about 30 queries are being made to only access 1 or 2 rows. But the only other option I could come up with is selecting all of the rows in the table, and then working with that array, but if there are pretend 1 million rows in the db, I feel like that would affect performance a lot more.
Am I right, and what are some solutions? Thanks
I just want to point out that I did resolve the issue. If anyone is wondering why the foreach loop was querying so sow it was because I accidentally deleted a line where I connected to the Facebook api within the foreach loop every time to gather the poster's information. So if anyone ever stumbles upon this question, just to be sure I want to clarify that making many facebook->api calls is a bad thing. save the info in your database and query that instead.
I would like to create an advanced search form much like a job site would have one that would include criteria such as keyword, job type, min pay, max pay, category,sub category etc...
My problem is deciding on how best to set this up so if I have to add categories to the parameters I'm not having to modify a whole bunch of queries and functions etc...
My best guess would be to create some sort of associative array out of all of the potential parameters and reuse this array but for some reason I feel like it's a lot more complex than this. I am using CodeIgniter as an MVC framework if that makes any difference.
Does anybody have a suggestion as how best to set this up?
Keep in mind I will need to be generating links such as index.php?keyword=designer&job_type=2&min_pay=20&max_pay=30
I hope my question is not to vague.
I don't know if it's what you need, but I usually create some search class.
<?php
$search = new Search('people');
$search->minPay(1000);
$search->maxPay(4000);
$search->jobType('IT');
$results = $search->execute();
foreach ($results as $result)
{
//whatever you want
}
?>
You can have all this methods, or have some mapping at __set() between method name and database field. The parameter passed to the constructor is the table where to do the main query. On the methods or mapping in the __set(), you have to take care of any needed join and the fields to join on.
There are much more 'enterprise-level' ways of doing this, but for a small site this should be OK. There are lots more ActiveRecord methods you can use as necessary. CI will chain them for you to make an efficient SQL request.
if($this->input->get('min_pay')) {
$this->db->where('min_pay <', $this->input->get('min_pay'));
}
if($this->input->get('keyword')) {
$this->db->like($this->input->get('keyword'));
}
$query = $this->db->get('table_name');
foreach ($query->result() as $row) {
echo $row->title;
}
To use Search criterias in a nice way you should use Classes and Interfaces.
Let's say for example you define a ICriteria interface. Then you have different subtypes (implementations) of Criteria, TimeCriteria, DateCriteria, listCriteria, TextSearch Criteria, IntRange Criteria, etc.
What your Criteria Interface should provide is some getter and setter for each criteria, you'll have to handle 3 usages for each criteria:
how to show them
how to fill the query with the results
how to save them (in session or database) for later usage
When showing a criteria you will need:
a label
a list of available operators (in, not in, =, >, >=, <, <=, contains, does not contains) -- and each subtypes can decide which part of this list is implemented
an entry zone (a list, a text input, a date input, etc)
Your main code will only handle ICriteria elements, ask them to build themselves, show them, give them user inputs, ask them to be saved or loop on them to add SQL criteria on a sql query based on their current values.
Some of the Criteria implementations will inherits others, some will only have to define the list of operators available, some will extends simple behaviors to add rich UI (let's say that some Date elements should provide a list like 'in the last day', 'in the last week', 'in the last year', 'custom range').
It can be a very good idea to handle the SQL query as an object and not only a string, the way Zend_Db_Select works for example. As each Criteria will add his part on the final query, and some of them could be adding leftJoins or complex query parts.
Search queries can be a pain sometimes, but not as big of a pain as pagination. Luckily, CodeIgniter helps you out a bit with this with their pagination library.
I think you're on the right track. The basic gist, I would say, is:
Grab your GET variables from the URL.
Create your database query (sanitize the GET values).
Generate the results set.
Do pagination.
Now, CodeIgniter destroys the GET variable by default, so make sure you enable http query strings in your config file.
Good luck!
I don't know anything about CodeIgniter, but for the search application I used to support, we had drop-down combo-boxes with category options stored in a database table and would rely on application and database cacheing to avoid round-trips each time the page was displayed (an opportunity for learning in itself ;-). When you update the table of job_type, location, etc. the new values will be displayed in your combo-box.
It depends on
how many categories you intend to have drop-down lists
how often you anticipate having to update the list
how dynamic you need it to be.
And the size of your web-site and overall activity are factors you will have to consider.
I hope this helps.
P.S. as you appear to be a new user, if you get an answer that helps you please remember to mark it as accepted, or give it a + (or -) as a useful answer
A pagination class is a good foundation. Begin by collecting query string variables.
<?php
// ...in Pagination class
$acceptableVars = array('page', 'delete', 'edit', 'sessionId', 'next', 'etc.');
foreach($_GET as $key => $value) {
if(in_array($key, $acceptableVar)) {
$queryStringVars[] = $key . '=' . $value;
}
}
$queryString = '?' . implode('&', $queryStringVars);
$this->nextLink = $_SEVER['filename'] . $queryString;
?>
Duplicate the searchable information into another table. Convert sets of data into columns having two values only like : a search for color=white OR red can become a search on 10 columns in a table each containing one color with value 1 or 0. The results can be grouped after so you get counters for each search filter.
Convert texts to full text searches and use MATCH and many indexes on this search table. Eventually combine text columns into one searchable column. The results of a seach will be IDs which you can then convert into the records with IN() condition in SQL
Agile Toolkit allows to add filters in the following way (just to do a side-by-side comparison with CodeIgniter, perhaps you can take some concepts over):
$g=$this->add('Grid');
$g->addColumn('text','name');
$g->addColumn('text','surname');
$g->setSource('user');
$conditions=array_intersect($_GET, array_flip(
array('keyword','job_type','min_pay'));
$g->dq->where($conditions);
$g->dq is a dynamic query, where() escapes values passed from the $_GET, so it's safe to use. The rest, pagination, column display, connectivity with MVC is up to the framework.
function maybeQuote($v){
return is_numeric($v) ?: "'$v'";
}
function makePair($kv){
+-- 7 lines: $a = explode('=', $kv);
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
function makeSql($get_string, $table){
+-- 10 lines: $data = explode('&', $get_string);
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
$test = 'lloyd=alive&age=40&weather=hot';
$table = 'foo';
print_r(makeSql($test, $table));
Is there a common way of dealing with tags in databases?
I'm thinking of using tinytext with pipes.
I think adding another table and using IDs might make it more complicated for little gain.
What's your preferred way of doing this?
and what is the right way of doing queries in a table to find results matching multiple or single tags?
Thanks
Implement a simple N:N relations.
Items:
-id
-name
-title
-foo
-bar
ItemTagRelations:
-Fkey itemId
-Fkey tagId
Tags:
-id
-name
-etc
I'll spread little heresy here.
Big boys, including this site are using denormalised schemas for tags for scalability reasons, storing comma, pipe or space delimited tags in text type field for each row and separate table for tags with counts. Upon inserting or updating an item just check what tags were added or dropped and update counts accordingly (explode to arrays old and new tag strings and do array_diff() ).
Now you have cheap way to display tag cloud with counts by simple SELECT * FROM tags, no fancy queries. To find items tagged with given name just do LIKE '%TAG%', this will work well for small traffic website (say less then 100k page views per day) and small data sets (again, say less than 100k of records). Above that you could use Fulltext Search to speed things up and ultimately proper search engine like Lucene or Sphinx.
Finding related tags, like here on SO, is easy too (Kohana specific code, LIKE based, MySQL specific):
$tags = array('foo', 'bar');
private function get_related_tags( $tags )
{
## Get db entries with specific tags and build array with counts
## is it cached already? ------------------------------------------------
$this->cache = Cache::instance();
$tags = array_filter( array_flip(array_flip($tags)) );
sort($tags);
$cache_name = implode('', $tags);
$cache = $this->cache->get( $cache_name );
if( $cache )
return $cache;
## not cached, fire up ---------------------------------------------------
$db = Database::instance();
## count tagged items ----------------------------------------------------
// build like string
$like = array();
foreach( $tags as $tag )
$like[] = "tags LIKE '%$tag%'";
$like = implode(' AND ', $like);
// get counts
$count = $db->query("SELECT count(id) AS count FROM `articles` WHERE $like")->current()->count;
## check what tags are related ------------------------------------------
$offset = 0;
$step = 300;
$related_tags = array();
while( $offset < $count )
{
$assets = $db->query("SELECT tags FROM `articles` WHERE $like ORDER BY id ASC LIMIT $step OFFSET $offset");
foreach($assets as $asset)
{
// tags
$input = explode( ' ', trim($asset->tags) );
foreach( $input as $k => $v )
{
if( $v == ''){
//do nothing, shouldnt be here anyway
}
elseif( array_key_exists($v, $related_tags) ){
$related_tags[$v]++;
}
else{
$related_tags[$v] = 1;
}
}
}
$offset += $step;
}
// remove already displayed from list
foreach( $tags as $tag )
unset( $related_tags[$tag] );
ksort($related_tags);
// set cache
$this->cache->set( $cache_name, array($related_tags, $count), 'related_tags_counts', 0);
return array($related_tags, $count);
}
This is not really cheap so I keep counts cached for given set of tags until I make changes to tags in articles table.
This setup is not perfect by any means, but certainly has some advantages. Schema is simple, getting tag cloud is straightforward, getting articles along with tags with one simple query (ie without subqueries). As main disadvantages I would see inability to rename or drop tag system-wide without amending every single row where it occurs, but hey, how often you do that anyway?
Currently I'm using this setup for few projects of mine and it works like a dream, but I must admit these are not high traffic websites (hence I get away with LIKE), next year I will be able to test it with busy site but I'm pretty sure it will do. Normalization nazis will vote me down perhaps, but I just love simplicity of it and I'm happy to trade off cpu cycles for that.
Actually I was going to post this tag system a while ago on SO and ask experts what they think of it so feel free to leave comments.
Traditionally, sorry for my English, I believe it's funny =)
EDIT
Since you've provided your requiremnents in comments, I think this setup is perfect for you. I've posted full Tag Model in pastie here, with methods to handle counts, Kohana specific but if you know Codeigniter you'll feel home. Just use it this way:
table TAGS: id, tag_name, tag_count
// insert new item/article
$tag_model->update_tags( $tags_str, null );
// update existing item
$tag_model->update_tags( $new_tags_str, $old_tags_str ); // $old_tags as stored in db
// delete item, you'll have to get item from db before deletion
$tag_model->update_tags( null, $old_tags_str );
I've amended the code as markdown have mangled it up, also queries are mySQL flavour, not SQLite.
Yes, don't scare away from normalization and having each tag in its own record. This will ultimately be the most flexible and with the correct indexing the fastest.