Related
Function render makes website 500% slow! Can anyone fix that please ?
Someone told me :
because it sends a database request on each iteration of the loop (it's not the only problem with this chunk of code but it's the most taxing one)
Yes I understand what that means. His way is:
you need to get all of the data before you start building the menu,
then you just insert the data instead of requesting more data on each
iteration
But i don't know how i must do it!
<?php
$menu_html='';
function render_menu($parent_id,$actmenuid)
{
$obj = new Database();
$con = $obj->dbconnectt();
global $menu_html;
$result=mysqli_query($con, "select * from tbl_menu where parent_id='$parent_id'");
if(mysqli_num_rows($result)==0) return;
if($parent_id==0){
$menu_html.='<ul class="topnav">';
}else{
$menu_html.='<ul>';
}
while($row=mysqli_fetch_array($result)) {
$childnum = $obj->recordcount("SELECT * FROM tbl_menu WHERE parent_id='".$row['id']."'");
if($childnum == 0){
$linkvalue='/category/'.$row['id'].'.html';
} else{
$linkvalue='#';
}
if($row['id']==$actmenuid && $actmenuid !=NULL){
$actv='class="active"';
}else{
$actv='';
}
$menu_html.='<li '.$actv.'>'.$row['title'].'';
render_menu($row['id'],$actmenuid);
$menu_html.='</li>';
}
$menu_html.='</ul>';return $menu_html;
}
if($isDsh==false){
echo render_menu(0,$actmenuid);
}
?>
Depending on how many records you have, try removing this query from inside the loop since it's running for every record on the first query.
$childnum = $obj->recordcount("SELECT * FROM tbl_menu WHERE parent_id='".$row['id']."'");
Change it a single query like this where it returns counts for each parent idea, and place it outside of the loop:
$parentcount = mysqli_query($con, ("SELECT parent_id, count(*) FROM tbl_menu GROUP BY parent_id");
There may be other issues, so please post the database structure and number of records that you're working with too.
Don't make recursive queries.
Having "more than 1000" rows is not too big. You can simply call everything from the table into php, then perform the recursive html build in php this will have a memory overhead, but far less processing overhead because you only ever make one trip to the db.
Alternatively (when your db table is prohibitively large), you should avoid gathering rows unnecessarily by adding a new column. The new column will store all "descendants" for the respective row when the row is INSERTed or update it when it is UPDATEd. Then you only need to reference this column when needing to call specific rows. In other words, do the recursive processing only once (when writing to the db) AND not when needing to display the data. This will, again, produce a finite result set in one query which can then be recursively traversed to build the desired output.
basically you need to do what #spudly has suggested.
But there is a small catch in his solution which depending on the number of the rows in yous tbl_menu table you may use a big chunk of memory to fetch all the records.
you can optimise it more with using his solution but changing the query to:
select
parent_tbl_menu.id,
count(child_tbl_menu.id) as cnt
from
tbl_menu as parent_tbl_menu
left join
tbl_menu as child_tbl_menu
on parent_tbl_menu.id = child_tbl_menu.parent_id
where
parent_tbl_menu.parent_id = ?
group by
parent_tbl_menu.id
This way you will only fetch the child records of a specific parent.
And please consider using prepared statements as your code has sql injection vulnerability.
Connect (from PHP to MySQL) only once for the entire web page.
Don't put a SELECT inside a loop if you can do all the work in a single SELECT, such as with a JOIN. (Exception: A "hierarchical" table needs the nested SELECT. Exception to the exception: MySQL 8.0 and MariaDB 10.2 can do it with a "recursive CTE".)
Don't fetch all the columns (SELECT *) when all you want it is a recordcount. Instead, SELECT COUNT(*) ... and use the number returned.
1000 of anything is probably excessive for a web page. Re-think the UI.
I am building a search engine for a school project, meant to allow users to look through a large database of schools, and I have set it up so that on one page, the user would be able to tick certain checkboxes, and that after hitting submit, the user would be returned with a list of schools which correspond to what was ticked.
The user is allowed to search schools nationwide along a number of parameters:
School type (high school, middle school, trade school, university, etc...)
School status (private or public)
City
State
etc.
For my question, I'll only focus on the part related to school types, as I can later apply the logic with the other form elements which function similarly.
For school types, along with a number of other form elements, I have decided to use checkboxes, so that for instance, if a user wants the search to return both middle schools and high schools, he would only need to tick both the "middle school" and "high school" checkboxes.
So, this implies using OR statements in my search, so that I would, for instance, have a SQL query like this:
SELECT schools.schoolId, schools.schoolName, villages.villageName FROM schools, villages
WHERE schools.villageId = villages.villageId
And here is where I would set up the condition that only the information pertaining to middle schools, high schools, and trade schools be returned by the query:
AND (schools.schoolType = "MiddleSchool" OR schools.schoolType = "HighSchool" OR schools.schoolType = "TradeSchool")
It is the construction of this statement that I am having trouble with.
As it is, because the school types can vary as the database evolves, I cannot hard code my query construction, nor do I wish to.
I use PHP to generate my query based on the form results.
To generate a list of all available school types, I read all the existing school types from the database (MySQL) using a DISTINCT query. I use this to build a list of all the available school types, which I use to generate the search form.
As such, I have to use a similar method to compare the form results with the database, meaning that, on my results page, to see which checkboxes have been ticked, I need to see what checkboxes could have existed at the moment the form was sent, so I once again read all of the available types from the database: (some code cut off, this is just the query I use which I know works, but this is just for context)
try
{
$pdo = new PDO("mysql:host=$servername;dbname=$dbname", $username, $password);
$req_sql_type_school = "SELECT DISTINCT schoolType AS types FROM schools";
$res_type_schools = $pdo->query($req_sql_type_school);
$res_type_schools->setFetchMode(PDO::FETCH_ASSOC);
} catch (PDOException $e)
{
die("Could not connect to the database $dbname :" . $e->getMessage());
}
After that, I iterate over the query results to create an array of the school types: (I know this works too, but again, it's just for context)
$arrayTypes = array();
while ($row = $res_type_schools->fetch())
{
$arrayTypes[]=$row['types'];
}
Then, this is where I struggle. It is at this point that I must create the "AND..." statement to the start of the query which I mentioned above, which was:
SELECT schools.schoolId, schools.schoolName, villages.villageName FROM schools, villages
WHERE schools.villageId = villages.villageId
To see whether or not something was ticked by the user, I iterate over the array to check whether or not the corresponding checkbox was ticked. To do this, here is the code I wrote. Note: basictext() just formats the string to the format I used for my checkbox values, which is lowercase-only strings with no spaces and no accents):
foreach($arrayTypes as $arr)
{
if(isSet($_GET[basictext($arr)]))
{
echo "[x] Type ".$arr." <br/>";
}
else
{
echo "[ ] Type ".$arr." <br/>";
}
}
This allows me to check which types were checked by the user on the search form.
However, I don't know how to translate this to the AND statement I wanted, which was this one:
AND (schools.schoolType = "MiddleSchool" OR schools.schoolType = "HighSchool" OR schools.schoolType = "TradeSchool")
I had thought of placing each type inside parentheses and separating each type with an OR, and only writing inside the parentheses if the type was indeed checked.
This resulted in something like this:
AND ((schools.schoolType = 'MiddleSchool') OR () OR () OR (schools.schoolType = 'HighSchool') OR () OR () OR ())
Needless to say, it was very ugly, and it also didn't work, so I canned it. The problem is that I don't know how many "checked types" there can be, so, it's a little difficult to build a proper query.
One solution I've thought of while writing this is to place each "valid" type inside an array, and then use that array, whose length I would then know. Then I would be able to create the expected AND statement, because I would then have a definite amount of checked checkboxes, and it would also allow me to completely ignore the non-ticked values. Do you think that would work?
If your schools.schoolType column is an arbitrary VARCHAR data type, there needs to be some relationship between the data the user submits and the data used in the query, unless you predefine a list of possible values beforehand.
It looks like you have some relationship between the arrayTypes array of yours and the $_GET value. So assuming the query wants HighSchool, this can be detected if the user submits highschool using your basictext() function.
From there I would do something like this:
$selectedTypes = array();
foreach($arrayTypes as $arr)
{
if(isSet($_GET[basictext($arr)]))
{
echo "[x] Type ".$arr." <br/>";
$selectedTypes[] = $arr;
}
else
{
echo "[ ] Type ".$arr." <br/>";
}
}
This would populate the selectedTypes array with the matching types of what the user submits.
Using this it sounds like you want to add an IN clause to your final query.
$placeholders = implode(',', array_fill(0, count($selectedTypes), '?'));
$template = "SELECT schools.schoolId,
schools.schoolName,
villages.villageName
FROM schools,
villages
WHERE schools.villageId = villages.villageId
AND schools.schoolType IN ($placeholders)";
$stmt = $pdo->prepare($template);
$stmt->execute($selectedTypes);
$result = $stmt->fetchAll();
$stmt = null;
If selectedTypes contains ["HighSchool", "MiddleSchool", "TradeSchool"] then $placeholders will contain [?, ?, ?]. This is used to construct an SQL template for a prepared statement query using the selectedTypes array as the input data.
$result ought to have the data you're after.
So if I'm understanding you right, you simply need to write something to concatenate fragments of an OR statement based on a bit array. Instead of adding an OR for every entry regardless of if it is checked or not, do something like
first = true
string orClause = ""
array schoolTypes
array checkedSchoolTypes
for(i = 0; i < checkedSchoolTypes.length; i++)
if(first && checkedSchoolTypes[i])
orClause += "schools.schoolType = " + schoolTypes[i]
first = false
else if(checkedSchoolTypes[i])
orClause += "OR schools.schoolType = " + schoolTypes[i]
query = "This is your query" + orClause + "rest of your query"
*Note this is a solution for a school project not a production environment. If you were do this in a production environment in that case you would want to pass information through a paramaterized stored procedure and generate your statement in the sql based off of the information passed in from the form.
Instead of doing the or as you are suggesting you could do and IN statement as others are suggesting the code would basically be the same with a different format. Let me know if you need any clarifications.
I'm writing a filter/sorting feature for an application right now that will have text fields above each column. As the user types in each field, requests will be sent to the back-end for sorting. Since there are going to be around 6 text fields, I was wondering if there's a better way to sort instead of using if statements to check for each variable, and writing specific queries if say all fields were entered, just one, or just two fields, etc.
Seems like there would be a lot of if statements. Is there a more intuitive way of accomplishing this?
Thanks!
Any initial data manipulation, such as sorting, is usually done by the database engine.
Put an ORDER BY clause in there, unless you have a specific reason the sorting needs done in the application itself.
Edit: You now say that you want to filter the data instead. I would still do this at the database level. There is no sense in sending a huge dataset to PHP, just for PHP to have to wade through it and filter out data there. In most cases, doing this within MySQL will be far more efficient than what you can build in PHP.
Since there are going to be around 6 text fields, I was wondering if there's a better way to sort instead of using if statements to check for each variable
Definitely NO.
First, nothing wrong in using several if's in order.
Trust me - I myself being a huge fan of reducing repetitions of code, but consider these manually written blocks being the best solution.
Next, although there can be a way to wrap these condition ns some loop, most of time different conditions require different treatment.
however, in your next statements you are wrong:
and writing specific queries
you need only one query
Seems like there would be a lot of if statements.
why? no more than number of fields you have.
here goes a complete example of custom search query building code:
$w = array();
$where = '';
if (!empty($_GET['rooms'])) $w[]="rooms='".mesc($_GET['rooms'])."'";
if (!empty($_GET['space'])) $w[]="space='".mesc($_GET['space'])."'";
if (!empty($_GET['max_price'])) $w[]="price < '".mesc($_GET['max_price'])."'";
if (count($w)) $where="WHERE ".implode(' AND ',$w);
$query="select * from table $where";
the only fields filled by the user going to the query.
the ordering is going to be pretty the same way.
mesc is an abbreviation for the mysql_real_escape_string or any other applicable database-specific string escaping function
select * from Users
order by Creadted desc, Name asc, LastName desc, Status asc
And your records will be sorted by order from query.
First by Created desc, then by Name asc and so on.
But from your question I can see that you are searching for filtering results.
So to filter by multiple fileds just append your where, or if you are using any ORM you can do it through object methods.
But if its simple you can do it this way
$query = "";
foreach($_POST['grid_fields'] as $key => $value)
{
if(strlen($query) > 0)
$query .= ' and '
$query .= sprintf(" %s LIKE '%s' ", mysql_real_escape_string($key), '%' .mysql_real_escape_string($value) .'%');
}
if(strlen($query) > 0)
$original_query .= ' where ' . $query;
this could help you to achieve your result.
No. You cannot avoid the testing operations when sorting the set, as you have to compare the elements in the set in same way. The vehicle for this is an if statement.
Could you take a look at this?
WHERE (ifnull(#filter1, 1) = 1 or columnFilter1 = #filter1)
and (ifnull(#filter2, 1) = 1 or columnFilter2 = #filter2)
and (ifnull(#filter3, 1) = 1 or columnFilter3 = #filter3)
and (ifnull(#filter4, 1) = 1 or columnFilter4 = #filter4)
and (ifnull(#filter5, 1) = 1 or columnFilter5 = #filter5)
and (ifnull(#filter6, 1) = 1 or columnFilter6 = #filter6)
Please let me know if I'm misunderstanding your question.. It's not like an IF statement batch, and is pretty lengthy, but what do you think?
I need to grab data from two tables, but I know theres a better, more tidier way to do this. Is it some kind of JOIN i need?
I'll show you my code and you'll see what I mean:
if ($rs[firearm] != "") {
$sql_result2 = mysql_query("SELECT * FROM db_firearms WHERE name='$rs[firearm]'", $db);
$rs2 = mysql_fetch_array($sql_result2);
$sql_result3 = mysql_query("SELECT * FROM items_firearms WHERE player='$id'", $db);
$rs3 = mysql_fetch_array($sql_result3);
if ($rs3[$rs2[shortname]] < 1) {
mysql_query("UPDATE mobsters SET firearm = '' WHERE id ='$id'");
}
}
This question is clear, but your code example has alot of formatting issues and I cannot give you direct answer, based on your example code.
The reason, why your example is unclear, is because.. with what are you going to join the tables? From one table you are selecting by name='$rs[firearm]' and from another by player='$id'. You have to provide the hidden data, like $rs and also $id.
You should definitely read these about mysql join and mysql left join. But I will try to give you an example based on your code, with fixed syntax. (Keep in mind, that I'm no mysql join expert, I did not test this code and also I do not know the joining conditions.) And also, the system structure is unclear.
As I understood, this what your tables do, correct?
mobsters - Users table
items_firearms - Links from users table to items table
db_firearms - Items table
So basically, my example does this: It will have preloaded $rs value, from the users table. It will check, if there is a entry inside the links table and hook the result with them items table. However, if the links table or even the items table can return multiple entries, then this doesn't work and you need to loop your results in much more smarter way.
// I can only assume, that $id is the ID of the player
$id = 2;
// Since I dont know the $rs value, then Im going to make some up
$rs = array(
'id' => 33,
'firearm' => 'famas'
);
if ($rs['firearm']) {
$result = mysql_fetch_array(mysql_query("SELECT ifa.*, dbfa.* FROM `items_firearms` AS `ifa` LEFT JOIN `db_firearms` AS `dbfa` ON `ifa.shortname` = `dbfa.shortname` WHERE `ifa.player` = '$id'"));
if ($result['id']) {
mysql_query("UPDATE `mobsters` SET `firearm` = '' WHERE `id` = '$id'", $db);
}
}
It is pretty clear, that you are new to PHP and mysql.. So I think you should probably edit your question and talk about your higher goal. Briefly mention, what your application are you building..? What are you trying to do with the mysql queries..? Maybe provide the table structure of your mysql tables..? I'm sure, that you will get your questions votes back to normal and also we can help you much better.
NOTES
You have to quote these types of variables: $rs[firearm] -> $rs['firearm']
If you want to check if your $rs['firearm'] equals something, then there is a better way then $rs[firearm] != "". The most simple is if ($rs['firearm']) {echo 'foo';}, but will produce a notice message, when all errors reporting mode. You can use isset() and empty(), but keep in mind, that isset() checks whether the variable has been set.. Meaning, even if its false, then it has been set. empty() reacts to undefined and empty variable the same, without any messages.
Also, "" means NULL, so if you even need to use "", then use NULL instead...much neater way..
I strongly recommend to use mysql class. You can understand the basics behind that idea from this answer. This is gonna make things much more easier for you. Also, mysql class is a must-have when dealing with dynamic applications.
if ($rs3[$rs2[shortname]] < 1) { .. makes no sense.. Do you want to check if the value is empty? Then (simple): if (!$rs3[$rs2[shortname]]) { .. and a very strict standard: if (empty($rs3[$rs2[shortname]])) { ..
Also you have to quote your sql queries, see my examples above.
Is the last mysql query missing $db?
I would like to create an advanced search form much like a job site would have one that would include criteria such as keyword, job type, min pay, max pay, category,sub category etc...
My problem is deciding on how best to set this up so if I have to add categories to the parameters I'm not having to modify a whole bunch of queries and functions etc...
My best guess would be to create some sort of associative array out of all of the potential parameters and reuse this array but for some reason I feel like it's a lot more complex than this. I am using CodeIgniter as an MVC framework if that makes any difference.
Does anybody have a suggestion as how best to set this up?
Keep in mind I will need to be generating links such as index.php?keyword=designer&job_type=2&min_pay=20&max_pay=30
I hope my question is not to vague.
I don't know if it's what you need, but I usually create some search class.
<?php
$search = new Search('people');
$search->minPay(1000);
$search->maxPay(4000);
$search->jobType('IT');
$results = $search->execute();
foreach ($results as $result)
{
//whatever you want
}
?>
You can have all this methods, or have some mapping at __set() between method name and database field. The parameter passed to the constructor is the table where to do the main query. On the methods or mapping in the __set(), you have to take care of any needed join and the fields to join on.
There are much more 'enterprise-level' ways of doing this, but for a small site this should be OK. There are lots more ActiveRecord methods you can use as necessary. CI will chain them for you to make an efficient SQL request.
if($this->input->get('min_pay')) {
$this->db->where('min_pay <', $this->input->get('min_pay'));
}
if($this->input->get('keyword')) {
$this->db->like($this->input->get('keyword'));
}
$query = $this->db->get('table_name');
foreach ($query->result() as $row) {
echo $row->title;
}
To use Search criterias in a nice way you should use Classes and Interfaces.
Let's say for example you define a ICriteria interface. Then you have different subtypes (implementations) of Criteria, TimeCriteria, DateCriteria, listCriteria, TextSearch Criteria, IntRange Criteria, etc.
What your Criteria Interface should provide is some getter and setter for each criteria, you'll have to handle 3 usages for each criteria:
how to show them
how to fill the query with the results
how to save them (in session or database) for later usage
When showing a criteria you will need:
a label
a list of available operators (in, not in, =, >, >=, <, <=, contains, does not contains) -- and each subtypes can decide which part of this list is implemented
an entry zone (a list, a text input, a date input, etc)
Your main code will only handle ICriteria elements, ask them to build themselves, show them, give them user inputs, ask them to be saved or loop on them to add SQL criteria on a sql query based on their current values.
Some of the Criteria implementations will inherits others, some will only have to define the list of operators available, some will extends simple behaviors to add rich UI (let's say that some Date elements should provide a list like 'in the last day', 'in the last week', 'in the last year', 'custom range').
It can be a very good idea to handle the SQL query as an object and not only a string, the way Zend_Db_Select works for example. As each Criteria will add his part on the final query, and some of them could be adding leftJoins or complex query parts.
Search queries can be a pain sometimes, but not as big of a pain as pagination. Luckily, CodeIgniter helps you out a bit with this with their pagination library.
I think you're on the right track. The basic gist, I would say, is:
Grab your GET variables from the URL.
Create your database query (sanitize the GET values).
Generate the results set.
Do pagination.
Now, CodeIgniter destroys the GET variable by default, so make sure you enable http query strings in your config file.
Good luck!
I don't know anything about CodeIgniter, but for the search application I used to support, we had drop-down combo-boxes with category options stored in a database table and would rely on application and database cacheing to avoid round-trips each time the page was displayed (an opportunity for learning in itself ;-). When you update the table of job_type, location, etc. the new values will be displayed in your combo-box.
It depends on
how many categories you intend to have drop-down lists
how often you anticipate having to update the list
how dynamic you need it to be.
And the size of your web-site and overall activity are factors you will have to consider.
I hope this helps.
P.S. as you appear to be a new user, if you get an answer that helps you please remember to mark it as accepted, or give it a + (or -) as a useful answer
A pagination class is a good foundation. Begin by collecting query string variables.
<?php
// ...in Pagination class
$acceptableVars = array('page', 'delete', 'edit', 'sessionId', 'next', 'etc.');
foreach($_GET as $key => $value) {
if(in_array($key, $acceptableVar)) {
$queryStringVars[] = $key . '=' . $value;
}
}
$queryString = '?' . implode('&', $queryStringVars);
$this->nextLink = $_SEVER['filename'] . $queryString;
?>
Duplicate the searchable information into another table. Convert sets of data into columns having two values only like : a search for color=white OR red can become a search on 10 columns in a table each containing one color with value 1 or 0. The results can be grouped after so you get counters for each search filter.
Convert texts to full text searches and use MATCH and many indexes on this search table. Eventually combine text columns into one searchable column. The results of a seach will be IDs which you can then convert into the records with IN() condition in SQL
Agile Toolkit allows to add filters in the following way (just to do a side-by-side comparison with CodeIgniter, perhaps you can take some concepts over):
$g=$this->add('Grid');
$g->addColumn('text','name');
$g->addColumn('text','surname');
$g->setSource('user');
$conditions=array_intersect($_GET, array_flip(
array('keyword','job_type','min_pay'));
$g->dq->where($conditions);
$g->dq is a dynamic query, where() escapes values passed from the $_GET, so it's safe to use. The rest, pagination, column display, connectivity with MVC is up to the framework.
function maybeQuote($v){
return is_numeric($v) ?: "'$v'";
}
function makePair($kv){
+-- 7 lines: $a = explode('=', $kv);
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
function makeSql($get_string, $table){
+-- 10 lines: $data = explode('&', $get_string);
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
$test = 'lloyd=alive&age=40&weather=hot';
$table = 'foo';
print_r(makeSql($test, $table));