We're running CakePHP v3.x with a Postgres database. I need to select some records whose latitude and longitude are within X distance from another point. PostGIS has a function that does just this but it seems that I have to right raw SQL queries in order to use it.
I'm not asking for help writing the raw query, but I am looking for confirmation as to whether the raw query approach is the correct way to use this extension while making best use of the framework. I searched and haven't found any libraries to extend the CakePHP ORM to include this. Perhaps there's a third option I haven't thought of.
[NOTE: This does not work...]
public function fetch()
{
$maxMetersAway = 10 * 1000;
$lat = $this->request->query['lat'];
$lng = $this->request->query['lng'];
$stops = $this->Stops->find('all')
->where("ST_DWithin( POINT($lng,$lat), POINT(Stops.lng,Stops.lat), $maxMetersAway")
->toArray();
$this->set(['stops'=>$stops, '_serialize' => true]);
}
I got the CakePHP query working with this:
$stops = $this->Stops->find('all')
->where(['ST_DWithin( ST_SetSRID(ST_MakePoint(' . $longitude . ', ' . $latitude . '),4326)::geography, ST_SetSRID(ST_MakePoint(Stops.longitude, Stops.latitude),4326)::geography, ' . $maxMetersAway . ')'])
->toArray();
Updated: Original didn't actually work with the $maxMetersAway properly.
Related
I have a Laravel app being served on Azure. I am using an AJAX request to poll data for a javascript chart.
The AJAX requests a URL defined in my routes (web.php), thus:
Route::get('/rfp_chart_data', 'DataController#chart_data')->name('chart_data');
That controller method runs a postgresql query and returns a JSON file. This all works fine.
However, after experiencing some performance issues, I decided to monitor the postgres queries and discovered that the same query was running 3 times for each request to this URL.
This happens regardless of whether I:
access the URL via an AJAX request
go directly to the URL in a browser
access the URL via cURL
This (AFAIK) eliminates the possibility that this is some sort of missing img src issue (e.g. What can cause a double page request?)
Thanks very much for any help...
EDIT:
Image of the duplicate queries in postgres pg_stat_activity -- this is from 1 web request:
EDIT:
Full controller code:
<?php
namespace App\Http\Controllers;
use App\AllRfpEntry;
use DB;
use Illuminate\Http\Request;
use Yajra\Datatables\Facades\Datatables;
class DataController extends Controller {
/**
* Displays datatables front end view
*
* #return \Illuminate\View\View
*/
//|| result_url || '\">' || result_title || '</a>'
public function chart_data(Request $request) {
$binding_array = array();
$chart_data_sql = "SELECT relevant_dates.date::date,
CASE WHEN award_totals.sum IS NULL
THEN 0
ELSE award_totals.sum
END
as sum
,
CASE WHEN award_totals.transaction_count IS NULL
THEN 0
ELSE award_totals.transaction_count
END
as transaction_count FROM
(
SELECT * FROM generate_series('" . date('Y-m-01', strtotime('-15 month')) . "'::date, '" . date('Y-m-01') . "'::date, '1 month') AS date
) relevant_dates
LEFT JOIN
(
SELECT extract(year from awarded_date)::text || '-' || RIGHT('0' || extract(month from awarded_date)::text, 2) || '-01' as date, sum(award_amount)::numeric as sum, COUNT(award_amount) as transaction_count FROM all_rfp_entries
WHERE awarded_date >= '" . date('Y-m-01', strtotime('-15 month')) . "'
AND awarded_date <= '" . date("Y-m-d") . "' AND award_status = 'AWARDED'
AND award_amount::numeric < 10000000000";
if ($request->get('rfp_company_filter')) {
$binding_array['rfp_company_filter'] = $request->get('rfp_company_filter');
$chart_data_sql .= " AND company = :rfp_company_filter";
};
if ($request->get('rfp_source_filter')) {
$binding_array['rfp_source_filter'] = $request->get('rfp_source_filter');
$chart_data_sql .= " AND rfp_source = :rfp_source_filter";
}
if ($request->get('exclude_fed_rev')) {
$chart_data_sql .= " AND rfp_source != 'US FED REV' ";
}
if ($request->get('rfp_year_filter')) {
$binding_array['rfp_year_filter'] = $request->get('rfp_year_filter');
$chart_data_sql .= " AND year = :rfp_year_filter";
}
if ($request->get('rfp_priority_level_filter')) {
$binding_array['rfp_priority_level_filter'] = $request->get('rfp_priority_level_filter');
$chart_data_sql .= " AND priority_level = :rfp_priority_level_filter";
}
if ($request->get('rfp_search_input_chart')) {
$binding_array['rfp_search_input_chart'] = $request->get('rfp_search_input_chart');
$chart_data_sql .= " AND search_document::tsvector ## plainto_tsquery('simple', :rfp_search_input_chart)";
}
$chart_data_sql .= " GROUP BY extract(year from awarded_date), extract(month from awarded_date)
) award_totals
on award_totals.date::date = relevant_dates.date::date
ORDER BY extract(year from relevant_dates.date::date), extract(month from relevant_dates.date::date)
";
return json_encode(DB::select($chart_data_sql, $binding_array));
}
public function data(Request $request) {
$query = AllRfpEntry::select('id', 'year', 'company', 'result_title', 'award_amount', 'edit_column', 'doc_type', 'rfp_source', 'posted_date', 'awarded_date', 'award_status', 'priority_level', 'word_score', 'summary', 'contract_age', 'search_document', 'link');
if ($request->get('exclude_na')) {
$query->where('all_rfp_entries.company', '!=', 'NA');
}
if ($request->get('clicked_date')) {
$query->where('all_rfp_entries.awarded_date', '>', $request->get('clicked_date'));
$query->where('all_rfp_entries.awarded_date', '<=', $request->get('clicked_date_plus_one_month'));
}
if ($request->get('filter_input')) {
$query->whereRaw("search_document::tsvector ## plainto_tsquery('simple', '" . $request->get('filter_input') . "')");
}
$datatables_json = datatables()->of($query)
->rawColumns(['result_title', 'edit_column', 'link'])
->orderColumn('award_amount', 'award_amount $1 NULLS LAST')
->orderColumn('priority_level', 'priority_level $1 NULLS LAST');
if (!$request->get('filter_input')) {
$datatables_json = $datatables_json->orderByNullsLast();
}
if (!$request->get('filter_input') and !$request->get('clicked_date')) {
$count_table = 'all_rfp_entries';
$count = DB::select(DB::raw("SELECT n_live_tup FROM pg_stat_all_tables WHERE relname = :count_table "), array('count_table' => $count_table))[0]->n_live_tup;
$datatables_json = $datatables_json->setTotalRecords($count);
}
$datatables_json = $datatables_json->make(true);
return $datatables_json;
}
}
EDIT:
An even crazier wrinkle...
I have a method in Laravel on this server pointed at a postgres database on another server to ingest new data. I just found out that even THAT method (the one pointing at the external server) is generating multiple queries on the external postsgres server!
Unless I'm missing something, this obviates an nginx issue or an issue with my provider (Azure), or a problem with any one specific method. Even a straight database connection over port 5432 (I'm assuming that's how Laravel accesses external databases) is generating the multiplier effect, so it must be something screwy with my Laravel installation... but no closer to figuring out what.
The best way to debug this is to start a debug session with xdebug and stepping through the code while you keep an eye on the stdout/logging output in a separate window.
Set a breakpoint on the first line in your controller, when it breaks there should be 0 queries done. If not you know something weird is going on in routing/the request. Then step through the function calls used in building the query.
It might be that one of the functions you use triggers executing the query (as suggested by Aaron Saray) or methods are missing (as suggested by Diogo Gomes) but this is hard to tell without knowing the code or stepping through an execution context step-by-step.
If you do not have a debugger you can always use dd($data); at any line to stop processing too and dump the data given. It will just take a little longer because you'll be doing a new request for each step in the code.
For what it's worth, I tracked this down to multiple postgres worker processes (which you can set in your postgres .conf file). So, not a bug, nor an issue with Laravel -- just parallel workers spawned by postgres.
So I have been working around with neo4j and php-client graph aware, until now i have make it work kind of ok. Until now that i try to run a query returning a count() and can't find how to catch the information, the query that i run is the next function:
function net_in_common($user, $other){
global $client;
$searchquery = "MATCH (a:user)-[r:IN_NET]->(b:user)<-[s:IN_NET]-(c:user) WHERE a.username = '" . $user . "' AND c.username = '" . $other . "' return count(DISTINCT b) as incommon";
$result = $client->run($searchquery);
return $result;
}
but when i try to echo it by
$common = net_in_common($user1, $user2);
echo $common->value('incommon');
i get absolute and completely nothing, it even dispatch an error that break the php code but i can't find the mistake itself.
it's a different way of fetching the value of a count() or something that i should do different??
The $result variable in your function returns you a Result object which itself contains a collection of ResultRecord objects (all is explained in the README of the client https://github.com/graphaware/neo4j-php-client#working-with-result-sets).
So, for reading the incommon value you would have to do :
$common = net_in_common($user1, $user2);
echo $common->firstRecord()->get('incommon');
Also, using php functions like this doesn't really reflect how we use php in (almost) 2017, maybe you can share a complete example of your project so we can investigate what's wrong, normally calling the value on a Result object should trigger an exception.
I'm using SQL in Yii framework.
I need to show the person's latest active week (it's number and date).So I wrote following code:
public function latestWeek()
{
$datalogin=//the login is working fine
$sql ="SELECT w.number,MAX(w.start_date)
FROM tbl_person_week t, tbl_week w
WHERE t.person_id=$this->id AND t.week_id=w.id";
$query = mysqli_query($datalogin, $sql);
return $query;
}
Now , I checked this query on the server and it works fine (almost) but first thing: I need to convert it into string , because yii's CgridView can't read it , and I couldn't find a working solution for this.
Second: on the server , it gave me the max date indeed , but not it's correct number , but the first number available. How can I fix this as well?
Queries like that should never be used in objective framework. If yu want to execute your own query, you should do it this way:
$sql = "your sql code";
$array = Yii::app()->db->createCommand($sql)->queryAll();
As result you will get multidimensional array with selected columns and rows
If you want to use it in grid view, you should do it this way:
$count = Yii::app()->db->createCommand($sql)->queryScalar();
$dataProvider = new CSqlDataProvider($sql, array('totalItemCount'=>$count));
$this->widget('zii.widgets.grid.CGridView', array(
'id'=>'grid-id',
'dataProvider'=> $dataProvider,
));
You can also use connection other than Yii::app()->db. Check CDbConnection class in docs.
edit: if you wanna use queries like mysql_fetch_assoc, check out also queryRow() method instead of queryAll()
Use Mysql_fetch _array
public function latestWeek()
{
$datalogin=//the login is working fine
$sql ="SELECT w.number,MAX(w.start_date)
FROM tbl_person_week t, tbl_week w
WHERE t.person_id=$this->id AND t.week_id=w.id";
$query = mysqli_query($datalogin, $sql);
while($row = mysqli_fetch_array($query)){
echo $row;
}
}
Assuming from your qu. that you want the week number and start date as one string, you have to concatenate the two columns in the sql.
You also need to specify that the week number is from the row with the maximum start date, which isn't as simple as you might first think.
I don't like injecting the person_id straight into SQL, it isn't awful in this case but is a bad habit to get into security-wise. There are binding methods available in the framework and I agree with Arek, that you should lean on the yii framework as much as possible.
To get the scalar string value, if you are insisting on using your own SQL.. I suggest the following:
$sql='
SELECT CONCAT('Week ',tw.number,' starting ',tw.start_date)
FROM tbl_week tw
JOIN (
SELECT MAX(twi.start_date) max_start_date
FROM tbl_week twi
JOIN tbl_person_week tpwi
ON tpwi.week_id = twi.id
AND tpwi.person_id = :person_id
) i
ON tw.start_date = i.max_start_date;
';
$command=Yii::app()->db->createCommand($sql);
$command->bindParam(":person_id", $this->id);
return $command->queryScalar();
I just wanted some advice really.
I am building a project where by I want to allow the users to search either by location or by a reference say restaurant name from the data that may exist in my db already.
Currently I have part implemented the location search where by the user can type in location say "nw5" and I then go get the approx lat/long and query my db with them coords. All good and works well.
I want to now add in a feature where the user can say "I know the restaurant name" - whacks it in and brings back the results accordingly.
I am sort of lost on how best to play this tbh.
Any insight would be appreciated.
Thanks
Raj
You're going to need some client-side magic for something like an autocomplete - jqueryui.com has a good one. Then you'll need a server-side ajax handler to run a query. Here's one I use, this is Doctrine but you could whip together a similar sql query:
function getLocByLabelChunk($chunkFromUser) {
$lower = strtolower($chunkFromUser);
$ls = str_split($lower);
$first = strtoupper($ls[0]);
$firstUp = $first;
for ($i=1; $i < count($ls); $i++) {
$firstUp .= $ls[$i];
}
$dql = "
SELECT l.locID as lID, l.label, l.label as value
FROM Entity\Location l
WHERE
l.label LIKE ?2
OR l.label LIKE ?3
OR l.label LIKE ?4
)
";
$q = $em->createQuery($dql);
$q->setParameter(2,"%" . $chunkFromUser. "%");
$q->setParameter(3,"%" . $lower . "%");
$q->setParameter(4,"%" . $firstUp . "%");
$locations = $q->getArrayResult();
return $locations;
}
I use apache solr 3.6, and php solr client.
Now i do simple solr queries like
$query = 'url:"'.$searchSubject.'"';
where $searchSubject is one word.
But I want to make some queries with two or more words.
I don't understand how i must add some other parameters into code of PHP SOLR CLIENT.
Now i can add only query, offset , limit .
$response = $this->solr->search($query, $offset = 0, $limit = 10000);
You can just insert multiple words into the parameter $query using the format specified in lucene query parser syntax
$query = 'url:"' . $searchSubject . '"';
if you would like to search for two urls, the query can be
$query = 'url:("' . $searchSubject1 . '" OR "' . $searchSubject2 . '")';
In solr you can simply create query for multiple words.
$query = url:"($searchSubject1 $searchSubject2 $searchSubject3)";
space between $searchSubject1 and $searchSubject2 will take default OR condition.