Atm I am building a very specific solution for an existing application written in Laravel. The solution executes queries in c++ modifies data, does sorting and returns the results. This c++ program is loaded in via a PHP extension and serves a single method to handle this logic.
The method provided by the extension should be implemented in Laravel using Eloquent, I've been looking at the source code for ages to find the specific method(s) that execute the queries build with Eloquensts Builder.
Where can I find the methods that actually perform the queries?
Why c++? I hear you think. The queries should be executed on multiple schemas (and/or databases) over multiple threads for improved performance. Atm 100+ schemas are being used with each containing thousands of records per table.
After a lot of troubleshooting and testing I have found a solution to my problem. In the class Illuminate\Database\Query\Builder you can find a method called runSelect(). This method runs a select statement against the given connection and returns the selected rows as an array.
/**
* Run the query as a "select" statement against the connection.
*
* #return array
*/
protected function runSelect()
{
return $this->connection->select(
$this->toSql(), $this->getBindings(), ! $this->useWritePdo
);
}
What I did to test my implementation in c++ to run the selects, I mapped the return values of $this->getBindings() to a new array to do some necessary modifications to some strings and did a simple str_replace_array on the prepared statement to get the full query. Eventually the c++ program will execute the prepared statemend and not the parsed query.
The modified method to suit my case looks like this. This has been done quick and dirty for now to test if it is possible, but you get the idea. Works as a charm except for the count() method in eloquent.
/**
* Run the query as a "select" statement against the connection.
*
* #return array
*/
protected function runSelect()
{
$bindings = [];
foreach ($this->getBindings() as $key => $value) {
$bindings[] = $value; // Some other logic to manipulate strings will be added.
}
$query = str_replace_array('?', $bindings, $this->toSql());
$schemas = ['schema1', 'schema2', 'schema3', 'schema4']; // Will be fetched from master DB.
return runSelectCpp($schemas, $query);
}
Related
I have problem here with query result from Eloquent, I tried to query from DB and put in variable $contractList in my mount() method and the result as expected. But when I tried to retrieve specific data from $contractList with $contractList->find($id), the result not same as in mount() method.
Here is query from mount():
public function mount(){
$contractList = Kontrak::select(['id', 'mou_id'])->with(['order:id,kontrak_id', 'order.worklist:id', 'order.worklist.khs:id,mou_id,worklist_id,khs', 'amdNilai:id,kontrak_id,tgl_surat'])->withCount('amdNilai')->get()
}
Here the result:
But when I tried to find specific data from $contractList, properties that shown not same as in mount:
public function itemSelected($id)
{
//amd_nilai_count not showing
$kontrak = $this->contractList->find($id);
if ($kontrak->amd_nilai_count == 1) {
$this->nilai_amd = $this->calculateNilai($id);
}
}
Here is the result called from itemSelected():
I have tried use get() but the result still problem, how to get same properties same as in mount().By the way im use laravel & livewire.
As i read your comments you seem to mix up ActiveRecords ORM with an Data Mapper ORM. Laravel uses active records.
Laravel will always fetch models on every data operation.
Kontrak::select('name')->find(1); // select name from kontraks where id = 1;
Kontrak::find(1); // select * from kontraks where id = 1;
This will always execute two SQL calls no matter what and the objects on the heap will not be the same. If you worked with Doctrine or similar, this would be different.
To combat this you would often put your logic in services or similar.
class KontrakService
{
public function find(int $id) {
return Kontrak::select(['id', 'mou_id'])->find($id);
}
}
Whenever you want the same logic, use that service.
resolve(KontrakService::class)->find(1);
However, many relationship operations is hard to do with this and then it is fine to just fetch the model with all the attributes.
I'm wondering if anything like the following already exists for Laravel? It's a trait I wrote called CarefulCount.
What it does: It returns a count of related models (using any defined relation), but only hits the DB if absolutely necessary. First, it tries two options to avoid hitting the DB, if the information is already available:
Was the count retrieved using withCount('relation') when the model was retrieved - i.e. does $model->relation_count exist? If so, just return that.
Has the relation been eager-loaded? If so, count the models in the Eloquent collection without hitting the DB, with $model->relation->count().
Only then resort to calling $model->relation()->count() to retrieve the count from the DB.
To enable it for any model class, you simply need to include the trait with use CarefulCount. You can then call $model->carefulCount('relation') for any defined relation.
For example, in my application there is a suburbs table with a has-many relation to both the users table and churches tables (i.e. there can be many users and many churches in a single suburb). Simply by adding use CarefulCount to the Suburb model, I can then call both $suburb->carefulCount('users') and $suburb->carefulCount('churches').
My use case: I've come across this a number of times - where I need a count of related models, but it's in a lower-level part of my application that may be called from several places. So I can't know how the model was retrieved and whether the count information is already there.
In those situations, the default would be to call $model->relation()->count(). But this can lead to the N+1 query problem.
In fact, the specific trigger came from adding Marcel Pociot's excellent Laravel N+1 Query Detector package to my project. It turned up a number of N+1 query problems that I hadn't picked up, and most were cases when I had already eager-loaded the related models. But in my Blade templates, I use Policies to enable or disable deleting of records; and the delete($user, $suburb) method of my SuburbPolicy class included this:
return $suburb->users()->count() == 0 && $suburb->churches()->count() == 0;
This introduced the N+1 problem - and obviously I can't assume, in my Policy class (or my Model class itself), that the users and churches are eager-loaded. But with the CarefulCount trait added, that became:
return $suburb->carefulCount('users') == 0 && $suburb->carefulCount('churches') == 0;
Voila! Tinkering with this and checking the query log, it works. For example, with the users count:
If $suburb was retrieved using Suburb::withCount('users'), no extra query is executed.
Similarly, if it was retrieved using Suburb::with('users'), no extra query is executed.
If neither of the above were done, then there is a select count(*) query executed to retrieve the count.
As I said, I'd love to know whether something like this already exists and I haven't found it (either in the core or in a package) - or whether I've plain missed something obvious.
Here's the code for my trait:
use Illuminate\Support\Str;
trait CarefulCount
{
/**
* Implements a careful and efficient count algorithm for the given
* relation, only hitting the DB if necessary.
*
* #param string $relation
*
* #return integer
*/
public function carefulCount(string $relation): int
{
/*
* If the count has already been loaded using withCount('relation'),
* use the 'relation_count' property.
*/
$prop = Str::snake($relation) . "_count";
if (isset($this->$prop)) {
return $this->$prop;
}
/*
* If the related models have already been eager-loaded using
* with('relation'), count the loaded collection.
*/
if ($this->relationLoaded($relation)) {
return $this->$relation->count();
}
/*
* Neither loaded, so hit the database.
*/
return $this->$relation()->count();
}
}
I have a Foos entity associated to foos table.
I'm trying to delete many rows from there using createQueryBuilder().
I am failing with that task, and nothing is logged, no exception, and no DELETE queries sent to mysql (I tried to log all queries in mysql).
Am I missing something?
/** #var Doctrine\ORM\EntityManagerInterface $this->entityManager */
$qb = $this->entityManager->createQueryBuilder()
->delete('Foos', 'foo')
->where('foo.some_column = :someAttr and another_column != :anotherAttr')
->setParameter('someAttr', $someAttr)
->setParameter('anotherAttr', $anotherAttr);
I inspected the getDQL() and the parts and query seems to be correct. But the test rows are not being deleted as expected.
How can I debug or make it right?
You are not executing the query.
$qb, in your example, is just a QueryBuilder, as evidenced by the call to createQueryBuilder().
After you are done building the query, you need to get the built object and execute it:
$qb->getQuery()->execute();
I feel like I'm having a moment where I'm missing something small here; I've been having issues using the insert() method on the QueryBuilder component on Dotrine DBAL 2.2.x / 2.3.x.
I did some investigation and here's the snippet from the QueryBuilder page from the DBAL Documantation
The \Doctrine\DBAL\Query\QueryBuilder supports building SELECT, INSERT, UPDATE and DELETE queries. Which sort of query you are building depends on the methods you are using.
It goes on further to explain code examples, such that I can simply do:
$builder = $connection->createQueryBuilder();
$result = $builder
->insert('table_name')
// ...
To use the query builder in Insert Mode. Except when I do I get a complaint here from PHP:
Fatal error: Call to undefined method Doctrine\DBAL\Query\QueryBuilder::insert()
On further inspection of The QueryBuilder.php Source Code
I see no reference to any method insert(...), no class to inherit this from, no traits added to the QueryBuilder that could expose the insert mechanism. In addition I see this right at the top:
/* The query types. */
const SELECT = 0;
const DELETE = 1;
const UPDATE = 2;
There's no insert query type; there is however this interesting method comment for execute():
/**
* Execute this query using the bound parameters and their types.
*
* Uses {#see Connection::executeQuery} for select statements and {#see Connection::executeUpdate}
* for insert, update and delete statements.
*
* #return mixed
*/
Bottom Line:
This is a massive project with 100's of maintainers, I'm more likely to find my interpretation suspect here than a screwup on something so fundamental over numerous versions, but I cannot for the life of me figure out what I'm missing. Please help me see the obvious.
It depends on your version. Insert has been added since v2.5.0-BETA3.
Viz https://github.com/doctrine/dbal/blob/master/lib/Doctrine/DBAL/Query/QueryBuilder.php#L563
and commit
You can decide to update package version or check this alternative solution
If I have an associated entity which is a collection, what options do you have when fetching?
e.g. Lets say I have a $view entity with this definition inside it:
/**
* #ORM\OneToMany(targetEntity="\Gutensite\CmsBundle\Entity\View\ViewVersion", mappedBy="entity")
* #ORM\OrderBy({"timeMod" = "DESC"})
*/
protected $versions;
public function getVersions() {
return $this->versions;
}
And I want to get the all the versions associated with the entity like this:
$view->getVersions();
This will return a collection. Great. But is it possible to take that collection and filter it by criteria, e.g. newer than a certain date? Or order it by some (other) criteria?
Or at this point are you just expected to do a query on the repository:
$versions = $em->getRepository("GutensiteCmsBundle:View\ViewVersion")->findBy(array(
array(
'viewId', $view->getId(),
'timeMod', time()-3600
)
// order by
array('timeMod', 'DESC')
));
There is a surprisingly unknown feature in recent versions of Doctrine, which makes these sort of queries much easier.
It doesn't seem to have a name, but you can read about it in the Doctrine docs at 9.8 Filtering Collections.
Collections have a filtering API that allows to slice parts of data from a collection. If the collection has not been loaded from the database yet, the filtering API can work on the SQL level to make optimized access to large collections.
In your case you could write a method like this on your View entity.
use Doctrine\Common\Collections\Criteria;
class View {
// ...
public function getVersionsNewerThan(\DateTime $time) {
$newerCriteria = Criteria::create()
->where(Criteria::expr()->gt("timeMod", $time));
return $this->getVersions()->matching($newerCriteria);
}
}
This will do one of two things:
If the collection is hydrated, it will use PHP to filter the existing collection.
If the collection is not hydrated, it will fetch a partial collection from the database using SQL constraints.
Which is really great, because hooking up repository methods to your views is usually messy and prone to break.
I also like #igor-pantovic's answer, although I've seen the method cause some funny bugs.
I would personally avoid using order by on annotation directly. Yes, you are supposed to do a query, just as you would if you were using raw SQL without Doctrine at all.
However, I wouldn't do it at that point but even before. In your specific case I would create an ViewRepository class:
class ViewRepository extends EntityRepository
{
public function findWithVersionsNewerThan($id, \DateTime $time)
{
return $this->createQueryBuilder('view')
->addSelect('version')
->join('view.versions', 'version')
->where('view.id = :id')
->andWhere('version.timeMod > :time')
->setParameter('time', $time)
->setParameter('id', $id)
->getQuery()
->getOneOrNullResult();
}
}
Now you can do:
$yourDateTime = // Calculate it here ... ;
$view = $em->getRepository("GutensiteCmsBundle:View\ViewVersion")->findWithVersionsNewerThan($yourDateTime);
$versions = $view->getVersions(); // Will only contain versions newer than datetime provided
I'm writing code from the top of my head here directly so sorry if some syntax or method naming error sneaked in.