Related
I am trying to figure out how to delete all ids in the database that do not exist in an array. I have been trying to use NOT IN in my query but I am not sure why it wont work when running it in a script the same way it works when I manually enter it into mysql. Here is an example.
mysqli_query($con, "DELETE FROM table WHERE id NOT IN ($array)");
$array is a list of ids from a json api. I use CURL to fetch the ids and I am trying to delete all ids in the database that do not match the ids in $array.
First I use another simple CURL script to scrape the apis and insert the ids found into the database and what I am trying to do here is basically make a link/data checker.
If the ids in the database are not found in the array when rechecking them then I want them deleted.
I thought that the query above would work perfect but for some reason it doesn't. When the query is ran from a script the mysql log shows the queries being ran as this.
Example:
DELETE FROM table WHERE id NOT IN ('166')
or this when I am testing multiple values.
DELETE FROM table WHERE id NOT IN ('166', '253', '3324')
And what happens is it deletes every row in the table every time. I don't really understand because if I copy/paste the same query from the log and run it manually myself it works perfect.
I have been trying various ways of capturing the array data such as array_column, array_map, array_search and various functions I have found but the end result is always the same.
For right now, just for testing I am using these 2 bits of code for testing 2 different apis which gives me the same sql query log output as above. The functions used are just a couple random ones that I found.
//$result is the result from CURL using json_decode
function implode_r($g, $p) {
return is_array($p) ?
implode($g, array_map(__FUNCTION__, array_fill(0, count($p), $g), $p)) :
$p;
}
foreach ($result['data'] as $info){
$ids = implode_r(',', $info['id']);
mysqli_query($con, "DELETE FROM table WHERE id NOT IN ($ids)");
}
And
$arrayLength = count($result);
for($i = 0; $i < $arrayLength; $i++) {
mysqli_query($con, "DELETE FROM table WHERE id NOT IN ('".$result[$i]['id']."')");
}
If anyone knows what is going on i'd appretiate the help or any suggestions on how to achieve the same result. I am using php 7 and mysql 5.7 with innodb tables if that helps.
It probably doesn't work because your IN value is something like this
IN('1,2,3,4');
When what you want is this
IN('1','2','3','4')
OR
IN( 1,2,3,4)
To get this with implode include the quotes like this
$in = "'".implode("','", $array)."'";
NOTE whenever directly inputting variables into SQL there is security Implications to consider such as SQLInjection. if the ID's are from a canned source you're probably ok, but I like to sanitize them anyway.
You can't mix array and string.
This works:
mysqli_query($con, "DELETE FROM table WHERE id NOT IN (".implode(',', $ids).")");
I have table of dogs in my DB and I want to retrieve N latest added dogs.
Only way that I found is something like this:
Dogs:all()->where(time, <=, another_time);
Is there another way how to do it? For example something like this Dogs:latest(5);
Thank you very much for any help :)
You may try something like this:
$dogs = Dogs::orderBy('id', 'desc')->take(5)->get();
Use orderBy with Descending order and take the first n numbers of records.
Update (Since the latest method has been added):
$dogs = Dogs::latest()->take(5)->get();
My solution for cleanliness is:
Dogs::latest()->take(5)->get();
It's the same as other answers, just with using built-in methods to handle common practices.
Dogs::orderBy('created_at','desc')->take(5)->get();
You can pass a negative integer n to take the last n elements.
Dogs::all()->take(-5)
This is good because you don't use orderBy which is bad when you have a big table.
You may also try like this:
$recentPost = Article::orderBy('id', 'desc')->limit(5)->get();
It's working fine for me in Laravel 5.6
I use it this way, as I find it cleaner:
$covidUpdate = COVIDUpdate::latest()->take(25)->get();
Ive come up with a solution that helps me achieve the same result using the array_slice() method. In my code I did array_slice( PickupResults::where('playerID', $this->getPlayerID())->get()->toArray(), -5 ); with -5 I wanted the last 5 results of the query.
The Alpha's solution is very elegant, however sometimes you need to re-sort (ascending order) the results in the database using SQL (to avoid in-memory sorting at the collection level), and an SQL subquery is a good way to achieve this.
It would be nice if Laravel was smart enough to recognise we want to create a subquery if we use the following ideal code...
$dogs = Dogs::orderByDesc('id')->take(5)->orderBy('id')->get();
...but this gets compiled to a single SQL query with conflicting ORDER BY clauses instead of the subquery that is required in this situation.
Creating a subquery in Laravel is unfortunately not simply as easy as the following pseudo-code that would be really nice to use...
$dogs = DB::subQuery(
Dogs::orderByDesc('id')->take(5)
)->orderBy('id');
...but the same result can be achieved using the following code:
$dogs = DB::table('id')->select('*')->fromSub(
Dogs::orderByDesc('id')->take(5)->toBase(),
'sq'
)->orderBy('id');
This generates the required SELECT * FROM (...) AS sq ... sql subquery construct, and the code is reasonably clean in terms of readability.)
Take particular note of the use of the ->toBase() function - which is required because fromSub() doesn't like to work with Eloquent model Eloquent\Builder instances, but seems to require a Query\Builder instance). (See: https://github.com/laravel/framework/issues/35631)
I hope this helps someone else, since I just spent a couple of hours researching how to achieve this myself. (I had a complex SQL query builder expression that needed to be limited to the last few rows in certain situations).
For getting last entry from DB
$variable= Model::orderBy('id', 'DESC')->limit(1)->get();
Imagine a situation where you want to get the latest record of data from the request header that was just inserted into the database:
$noOfFilesUploaded = count( $request->pic );// e.g 4
$model = new Model;
$model->latest()->take($noOfFilesUploaded);
This way your take() helper function gets the number of array data that was just sent via the request.
You can get only ids like so:
$model->latest()->take($noOfFilesUploaded)->puck('id')
use DB;
$dogs = DB::select(DB::raw("SELECT * FROM (SELECT * FROM dogs ORDER BY id DESC LIMIT 10) Var1 ORDER BY id ASC"));
Dogs::latest()->take(1)->first();
this code return the latest record in the collection
Can use this latest():
$dogs = Dogs::latest()->take(5)->get();
When I execute a PDO statement, internally a result set is stored, and I can use ->fetch() to get a row from the result.
If I wanted to convert the entire result to an array, I could do ->fetchAll().
With Laravel, in the Query Builder docs, I only see a way to get an array result from executing the query.
// example query, ~30,000 rows
$scores = DB::table("highscores")
->select("player_id", "score")
->orderBy("score", "desc")
->get();
var_dump($scores);
// array of 30,000 items...
// unbelievable ...
Is there any way to get a result set from Query Builder like PDO would return? Or am I forced to wait for Query Builder to build an entire array before it returns a value ?
Perhaps some sort of ->lazyGet(), or ->getCursor() ?
If this is the case, I can't help but see Query Builder is an extremely short-sighted tool. Imagine a query that selects 30,000 rows. With PDO I can step through row by row, one ->fetch() at a time, and handle the data with very little additional memory consumption.
Laravel Query Builder on the other hand? "Memory management, huh? It's fine, just load 30,000 rows into one big array!"
PS yes, I know I can use ->skip() and ->take() to offset and limit the result set. In most cases, this would work fine, as presenting a user with 30,000 rows is not even usable. If I want to generate large reports, I can see PHP running out of memory easily.
After #deczo pointed out an undocumented function ->chunk(), I dug around in the source code a bit. What I found is that ->chunk() is a convenience wrapper around multiplying my query into several queries queries but automatically populating the ->step($m)->take($n) parameters. If I wanted to build my own iterator, using ->chunk with my data set, I'd end up with 30,000 queries on my DB instead of 1.
This doesn't really help, too, because ->chunk() takes a callback which forces me to couple my looping logic at the time I'm building the query. Even if the function was defined somewhere else, the query is going to happen in the controller, which should have little interest in the intricacies of my View or Presenter.
Digging a little further, I found that all Query Builder queries inevitably pass through \Illuminate\Database\Connection#run.
// https://github.com/laravel/framework/blob/3d1b38557afe0d09326d0b5a9ff6b5705bc67d29/src/Illuminate/Database/Connection.php#L262-L284
/**
* Run a select statement against the database.
*
* #param string $query
* #param array $bindings
* #return array
*/
public function select($query, $bindings = array())
{
return $this->run($query, $bindings, function($me, $query, $bindings)
{
if ($me->pretending()) return array();
// For select statements, we'll simply execute the query and return an array
// of the database result set. Each element in the array will be a single
// row from the database table, and will either be an array or objects.
$statement = $me->getReadPdo()->prepare($query);
$statement->execute($me->prepareBindings($bindings));
return $statement->fetchAll($me->getFetchMode());
});
}
See that nasty $statement->fetchAll near the bottom ?
That means arrays for everyone, always and forever; your wishes and dreams abstracted away into an unusable tool Laravel Query Builder.
I can't express the valley of my depression right now.
One thing I will say though is that the Laravel source code was at least organized and formatted nicely. Now let's get some good code in there!
Use chunk:
DB::table('highscores')
->select(...)
->orderBy(...)
->chunk($rowsNumber, function ($portion) {
foreach ($portion as $row) { // do whatever you like }
});
Obviously returned result will be just the same as calling get, so:
$portion; // array of stdObjects
// and for Eloquent models:
Model::chunk(100, function ($portion) {
$portion; // Collection of Models
});
Here is a way to use the laravel query builder for making the query, but to then use the underlying pdo fetch to loop over the record set which I believe will solve your problem - running one query and looping the record set so you don't run out of memory on 30k records.
This approach will use all the config stuff you setup in laravel so you don't have to config pdo separately.
You could also abstract out a method to make this easy to use that takes in the query builder object, and returns the record set (executed pdo statement), which you would then while loop over as below.
$qb = DB::table("highscores")
->select("player_id", "score")
->orderBy("score", "desc");
$connection = $qb->getConnection();
$pdo = $connection->getPdo();
$query = $qb->toSql();
$bindings = $qb->getBindings();
$statement = $pdo->prepare($query);
$statement->execute($bindings);
while ($row = $statement->fetch($connection->getFetchMode()))
{
// do stuff with $row
}
i dont know if i am doing right or wrong, please dont judge me...
what i am trying to do is that if a record belongs to parent then it will have parent id assosiated with it.. let me show you my table schema below.
i have two columns
ItemCategoryID &
ItemParentCategoryID
Let Suppose a record on ItemCategoryID =4 belongs to ItemCategoryID =2 then the column ItemParentCategoryID on ID 4 will have the ID of ItemCategoryID.
I mean a loop with in its own table..
but problem is how to run the select query :P
I mean show all the parents and childs respective to their parents..
This is often a lazy design choise. Ideally you want a table for these relations or/and a set number of depths. If a parent_id's parent can have it's own parent_id, this means a potential infinite depth.
MySQL isn't a big fan of infinite nesting depths. But php don't mind. Either run multiple queryies in a loop such as Nil'z's1, or consider fetching all rows and sorting them out in arrays in php. Last solution is nice if you pretty much always get all rows, thus making MySQL filtering obsolete.
Lastly, consider if you could have a more ideal approach to this in your database structure. Don't be afraid to use more than one table for this.
This can be a strong performance thief in the future. An uncontrollable amount of mysql queries each time the page loads can easily get out of hands.
Try this:
function all_categories(){
$data = array();
$first = $this->db->select('itemParentCategoryId')->group_by('itemParentCategoryId')->get('table')->result_array();
if( isset( $first ) && is_array( $first ) && count( $first ) > 0 ){
foreach( $first as $key => $each ){
$second = $this->db->select('itemCategoryId, categoryName')->where_in('itemParentCategoryId', $each['itemParentCategoryId'])->get('table')->result_array();
$data[$key]['itemParentCategoryId'] = $each['itemParentCategoryId'];
$data[$key]['subs'] = $second;
}
}
print_r( $data );
}
I don't think you want/can to do this in your query since you can nest a long way.
You should make a getChilds function that calls itself when you retrieve a category. This way you can nest more than 2 levels.
function getCategory()
{
// Retrieve the category
// Get childs
$childs = $this->getCategoryByParent($categoryId);
}
function getCategorysByParent($parentId)
{
// Get category
// Get childs again.
}
MySQL does not support recursive queries. It is possible to emulate recursive queries through recursive calls to a stored procedure, but this is hackish and sub-optimal.
There are other ways to organise your data, these structures allow very efficient querying.
This question comes up so often I can't even be bothered to complain about your inability to use Google or SO search, or to offer a wordy explanation.
Here - use this library I made: http://codebyjeff.com/blog/2012/10/nested-data-with-mahana-hierarchy-library so you don't bring down your database
I would like to create an advanced search form much like a job site would have one that would include criteria such as keyword, job type, min pay, max pay, category,sub category etc...
My problem is deciding on how best to set this up so if I have to add categories to the parameters I'm not having to modify a whole bunch of queries and functions etc...
My best guess would be to create some sort of associative array out of all of the potential parameters and reuse this array but for some reason I feel like it's a lot more complex than this. I am using CodeIgniter as an MVC framework if that makes any difference.
Does anybody have a suggestion as how best to set this up?
Keep in mind I will need to be generating links such as index.php?keyword=designer&job_type=2&min_pay=20&max_pay=30
I hope my question is not to vague.
I don't know if it's what you need, but I usually create some search class.
<?php
$search = new Search('people');
$search->minPay(1000);
$search->maxPay(4000);
$search->jobType('IT');
$results = $search->execute();
foreach ($results as $result)
{
//whatever you want
}
?>
You can have all this methods, or have some mapping at __set() between method name and database field. The parameter passed to the constructor is the table where to do the main query. On the methods or mapping in the __set(), you have to take care of any needed join and the fields to join on.
There are much more 'enterprise-level' ways of doing this, but for a small site this should be OK. There are lots more ActiveRecord methods you can use as necessary. CI will chain them for you to make an efficient SQL request.
if($this->input->get('min_pay')) {
$this->db->where('min_pay <', $this->input->get('min_pay'));
}
if($this->input->get('keyword')) {
$this->db->like($this->input->get('keyword'));
}
$query = $this->db->get('table_name');
foreach ($query->result() as $row) {
echo $row->title;
}
To use Search criterias in a nice way you should use Classes and Interfaces.
Let's say for example you define a ICriteria interface. Then you have different subtypes (implementations) of Criteria, TimeCriteria, DateCriteria, listCriteria, TextSearch Criteria, IntRange Criteria, etc.
What your Criteria Interface should provide is some getter and setter for each criteria, you'll have to handle 3 usages for each criteria:
how to show them
how to fill the query with the results
how to save them (in session or database) for later usage
When showing a criteria you will need:
a label
a list of available operators (in, not in, =, >, >=, <, <=, contains, does not contains) -- and each subtypes can decide which part of this list is implemented
an entry zone (a list, a text input, a date input, etc)
Your main code will only handle ICriteria elements, ask them to build themselves, show them, give them user inputs, ask them to be saved or loop on them to add SQL criteria on a sql query based on their current values.
Some of the Criteria implementations will inherits others, some will only have to define the list of operators available, some will extends simple behaviors to add rich UI (let's say that some Date elements should provide a list like 'in the last day', 'in the last week', 'in the last year', 'custom range').
It can be a very good idea to handle the SQL query as an object and not only a string, the way Zend_Db_Select works for example. As each Criteria will add his part on the final query, and some of them could be adding leftJoins or complex query parts.
Search queries can be a pain sometimes, but not as big of a pain as pagination. Luckily, CodeIgniter helps you out a bit with this with their pagination library.
I think you're on the right track. The basic gist, I would say, is:
Grab your GET variables from the URL.
Create your database query (sanitize the GET values).
Generate the results set.
Do pagination.
Now, CodeIgniter destroys the GET variable by default, so make sure you enable http query strings in your config file.
Good luck!
I don't know anything about CodeIgniter, but for the search application I used to support, we had drop-down combo-boxes with category options stored in a database table and would rely on application and database cacheing to avoid round-trips each time the page was displayed (an opportunity for learning in itself ;-). When you update the table of job_type, location, etc. the new values will be displayed in your combo-box.
It depends on
how many categories you intend to have drop-down lists
how often you anticipate having to update the list
how dynamic you need it to be.
And the size of your web-site and overall activity are factors you will have to consider.
I hope this helps.
P.S. as you appear to be a new user, if you get an answer that helps you please remember to mark it as accepted, or give it a + (or -) as a useful answer
A pagination class is a good foundation. Begin by collecting query string variables.
<?php
// ...in Pagination class
$acceptableVars = array('page', 'delete', 'edit', 'sessionId', 'next', 'etc.');
foreach($_GET as $key => $value) {
if(in_array($key, $acceptableVar)) {
$queryStringVars[] = $key . '=' . $value;
}
}
$queryString = '?' . implode('&', $queryStringVars);
$this->nextLink = $_SEVER['filename'] . $queryString;
?>
Duplicate the searchable information into another table. Convert sets of data into columns having two values only like : a search for color=white OR red can become a search on 10 columns in a table each containing one color with value 1 or 0. The results can be grouped after so you get counters for each search filter.
Convert texts to full text searches and use MATCH and many indexes on this search table. Eventually combine text columns into one searchable column. The results of a seach will be IDs which you can then convert into the records with IN() condition in SQL
Agile Toolkit allows to add filters in the following way (just to do a side-by-side comparison with CodeIgniter, perhaps you can take some concepts over):
$g=$this->add('Grid');
$g->addColumn('text','name');
$g->addColumn('text','surname');
$g->setSource('user');
$conditions=array_intersect($_GET, array_flip(
array('keyword','job_type','min_pay'));
$g->dq->where($conditions);
$g->dq is a dynamic query, where() escapes values passed from the $_GET, so it's safe to use. The rest, pagination, column display, connectivity with MVC is up to the framework.
function maybeQuote($v){
return is_numeric($v) ?: "'$v'";
}
function makePair($kv){
+-- 7 lines: $a = explode('=', $kv);
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
function makeSql($get_string, $table){
+-- 10 lines: $data = explode('&', $get_string);
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
$test = 'lloyd=alive&age=40&weather=hot';
$table = 'foo';
print_r(makeSql($test, $table));