Ok I am building a takeaway finder that will find takeaways within a set distance of a uk postal code. What will happen is the user puts his/her postcode in an input box and clicks submit, the site then searches for takeaways near the user. But the catch is that this search is based on the individual takeaways delivery distance. So if a takeaway has a delivery distance of say 12 miles and the persons postcode is within 12 miles of the takeaway it will show in the results.
So far I have uk postcode database with lang and lat coordinates and also the takeaway database table holds the takeaways own postcode and its delivery distance but not the long and lat values of the takeaways postcode.
What I am asking for is not so much the code but help with the logic in how to do this.
I have the following query that will find all postcodes within a set radius of a given long and lat but Im not sure if its in miles and if it is the fastest it could be:
SELECT * , 6371 * ACos( Cos( RADIANS( latitude ) ) * Cos( RADIANS( 56.0062 ) ) * Cos( RADIANS( - 3.78189 ) - RADIANS( longitude ) ) + Sin( RADIANS( latitude ) ) * Sin( RADIANS( 56.0062 ) ) ) AS Distance
FROM postcodes
HAVING Distance <= '10'
ORDER BY Distance
LIMIT 3720 , 30
For performance, consider eliminating fields you don't need. The problem is you're sorting on a calculated value, so each row needs to be examined.
Ideally, you would perform additional filtering to reduce the number of rows you need. Perhaps matching prefixes of postal codes could help. You may make the observation that if the first X characters of the postal code don't match, then it must be more than 12 miles away.
If you have a lot of fields to retrieve, you can also see a big performance boost from late row lookup. In your case, this is particularly helpful, because you can provide a much smaller dataset for MySQL to sort.
The idea would be pull only the ID and distances for each record, sort them, and then pull the top N records (however many you need). You can then use the IDs you fetched to join back to the original table and retrieve the rest of the data. This helps because it allows MySQL to use less memory when performing the sort and if your dataset isn't in memory, you can potentially avoid some disk seeks as well depending on how big your rows are.
Another completely separate option. If you're focusing solely on the UK, you could consider using some type of projection into a Cartesian coordinate system. I believe OSGB is probably suitable for the UK, and should give minimal error.
This opens up the possibility of using MySQL's spatial extensions to add an R-tree index on a series of point columns. This wouldn't give you accurate enough distances on it's own, but it would enable you to narrow down the data set to a significantly smaller portion where true distances could be efficiently calculated.
Related
I'm doing some compare of prices in same location based on GPS coordinates.
So e.g. I've got an item with coordinates:
lat: 45.815005
lng: 15.978501
I want to search in my MySQL database for each items which are e.g. 500 meters around that place. (it doesn't need to be circle, just 500 meters on X and 500 meters on Y both ways). Lat and lng are stored as a separate columns type float(10,6) in my DB
I am aware of the fact it's not easy to calculate exact lng and lat but I'm fine if I miss few meters each site.
This is pretty complex question but I would be thankful for any advise which will kickoff my start.
Calculating the distance between two coordinates isn't actually that difficult given the haversine formula.
SELECT
-- stuff here
, ( 6371000 * acos( cos( radians(45.815005) ) * cos( radians( stuff.lat ) ) * cos( radians( stuff.lng ) - radians(15.978501) ) + sin( radians(45.815005) ) * sin(radians(stuff.lat)) ) ) AS distance
FROM
stuff
HAVING
distance < 500
Referenced Answer
Necessary changes from the original answer:
The constant offered in the original answer supplied the values for miles or kilometers. I've changed the constant here to work with meters.
The constants have changed to use your coordinates. You might want to adapt the query a little further to make those parameters instead of constants.
The having expression changed a little to reflect your desire for 500 meters. Again, this might be something you want to parameterize.
This question already has answers here:
MySQL Great Circle Distance (Haversine formula)
(9 answers)
Closed 8 years ago.
I'm using the following query to get the next local events using their location. My database is growing and I've got events from all over Europe. So basically, the query calculates distances from events of two different countries (like Spain and Germany) which is really not useful since I'm looking for the next events within 20 KM.
# latitude & longitude are the fields in DB
# 43.57 & 3.85 are given values representing the local test point
SELECT ( 6366 * ACOS( COS( RADIANS( 43.57 ) ) * COS( RADIANS( latitude ) ) * COS( RADIANS( longitude ) - RADIANS( 3.85 ) ) + SIN( RADIANS( 43.57 ) ) * SIN( RADIANS( latitude ) ) ) ) dist
FROM events
HAVING dist >0 AND dist <=20
ORDER BY event_time ASC
LIMIT 0 , 5
So basically, this query is going to get all the distances of all the events before being able to use the HAVING.
Is there a better way?
Calculate the longitude and latitude of the four points of a box that encloses the radius surrounding the test point. Over distances of a few kilometres you can ignore errors due to the curvature of the Earth.
We can use a rough approximation to limit an initial search, then refine things later.
At 50 degrees North (roughly Paris) a degree of latitude (North/South) is approximately 111.229km. A degree of longitude (East/West) is approximately 71.695km (I got these numbers from this page From this 20Km is approximately 10.8 seconds of latitude, and 16.74 seconds of longitude. Calculate a bounding box by adding and subtracting these numbers from the latitude and longitude of your test point.
Query for locations in that box using longitude and latitude.
If you're worried that your bounding box may be too large, or that an event in a corner of the box is outside the radius, calculate an accurate distance for those points you've already identified and filter out the few you don't want.
Note
These figures approximate for 50deg North. As you move north of that your value for km of longitude will become too long, and south of that it will be too short. You could use a small table of lookup values, or calculate the actual distance from the latitude, or simply increase the initial bounding box in the longitude direction. The box is just a first approximation, so as long as it's bigger than the target area it's exact size doesn't matter too much.
does anyone know how to do a query which will allow a user to specify their lat / lng and a radius, and then search what could be a very large amount of locations which also have a lat lng + a radius and see if any overlap, and then allow for sorting by distance?
We've been asked to allow for companies to place markers on a map of suburbs that they operate in, with the ability to do a radius, but I can't find any queries on Google that do this.
I'm currently using the following query which does the job,
(SELECT ( 6371.01 * acos( cos( radians( $lat ) ) * cos( radians( location.lat ) )
cos( radians( location.lng ) - radians( $lng ) )
sin( radians( $lat ) ) * sin( radians( location.lat ) ) ) ) AS distance
FROM location
HAVING distance <= $radius
ORDER BY distance ASC) as distance
But obviously this doesn't allow for the locations to have a radius.
The second part of this question, this search query is already part of a very large query.. are there any recommendations on optimizing this at all? Every time I google'd for examples on queries like the above there was always people warning about how expensive this query is. At the moment the only solution I can think of is to have a separate search server.
Very simple answer: use the spacial geometry functions in mySQL (http://dev.mysql.com/doc/refman/5.5/en/spatial-extensions.html)
These have been around for a while, but got a nice upgrade in 5.6 that will make it easier to do your select. http://www.percona.com/blog/2013/10/21/using-the-new-spatial-functions-in-mysql-5-6-for-geo-enabled-applications/ has some details on it.
If you don't have 5.6 then use a "grid" system. Basically, when you store the point (x,y), you also store a grid identifier (X,Y) that covers a larger square. When searching other points nearby you know it's within grid (X-1 to X+1) and (Y-1 to Y+1); this reduces the number of points you need to search. Then check both the actual difference for x and y are within a square in mySQL, then, once you've reduced it massively, then do your cos/sin maths.
I have been searching quite a bit for an answer, but maybe I'm just not using the correct terminology. I am creating an app that will access a database to return a list of other users that are within a certain distance of the users location. I've never worked with this type of data, and I don't really know what the values mean. I'd like to do all the calculations on the backend with either MySQL or PHP. Currently, I am storing the latitude and longitude as doubles within the database. I can access them and store them, but I have no idea how I might be able to sort them based on distance. Perhaps I should be using a different type or some technique that is common in this area. TIA.
It sounds like you need to use the haversine formula which gets the distance between two sets of long/lat coordindates (adjusting for curvature of the earth).
If you run a query with that as an output, you can easily sort them based on minimum distance from the user.
Here is a link to implementing the haversine in 9 commonly used languages and here is a SO question which implements it inside a SQL query.
Here is the query that you could adapt (gets anything within 25 miles ordered from closest to furthest):
SELECT
id,
( 3959 * acos( cos( radians(37) ) * cos( radians( lat ) ) * cos( radians( lng ) - radians(-122) ) + sin( radians(37) ) * sin( radians( lat ) ) ) ) AS distance
FROM
markers
HAVING
distance < 25
ORDER BY
distance
LIMIT
0 , 20;
I would suggest using Vicenty's Inverse Formula (http://en.wikipedia.org/wiki/Vincenty's_formulae) instead of the Haversine Great Circle distance, since Vincenty's been shown to be more accurate (Vincenty assumes the earth is an oblate spheroid instead of a perfect sphere, which is more accurate).
Here's the original Vincenty paper for the formula:
http://www.ngs.noaa.gov/PUBS_LIB/inverse.pdf - Section 4
Here's the actual code from the Android platform that is used to calculate distance for distanceTo(Location), which uses Vincenty's Inverse Formula: https://github.com/android/platform_frameworks_base/blob/master/location/java/android/location/Location.java#L272
As to sorting distances based on a database query, for optimum performance you'll want to use a spatial database that allows spatial queries. MySQL has a spatial database plugin:
http://dev.mysql.com/doc/refman/5.0/en/spatial-extensions.html
Check out this post, which should give you the details to go from there, including notes on precision using Vicenty:
Geo-Search (Distance) in PHP/MySQL (Performance)
I have a database of activities , each activities could be held on 3 days , each day contains a postal code .
So the database looks like that (+ alot of other fields)
In another database i have a Geo Location info (postal code , lat , long)
Now users can enter there postal code and a radius and activities in that radius will appear.
Question :
1 - What is the best way to accomplish that ?
Solution in mind
Make a view of all possible postal codes from the activities and join it on the Geo table to get their Lat/Lng
then when a user search for a postal code , get the Lat/Lng and do the mathematical equation to get all postal codes near that point .
But i don't think in term of performance this is a good way since i will have to apply the query on 3000+ activities
Codes found for distance
Finding locations nearby with MySQL (Haversine Formula)
SELECT id, ( 3959 * acos( cos( radians(37) ) * cos( radians( lat ) ) * cos( radians( lng ) - radians(-122) ) + sin( radians(37) ) * sin( radians( lat ) ) ) ) AS distance
FROM markers
HAVING distance < 25
ORDER BY distance LIMIT 0 , 20;
What do you guys think ?
You would need a GIS to do that (and use spatial index), but mysql is not capable of it - mysql GIS functionality can handle just rectangles. PostreSQL is capable of GIS.
Easiest would really be the math expression. It would be best if you could use some Projected coordinates (lat & lon are sphere coordinates). Convert whole database in this projected coordination system and than just use simple expression (without need to use trigonometric functions):
(activity_x - postal_code_x)^2 + (activity_y -
postal_code_y)^2 < distance^2
note that the Earth is a sphere, which means this will only work exactly for smaller distances (say < 1000 km). But anyway I think you don't need exact circle...
I was wrong in term of performance , the query took less than 0,5 second to calculate the distance on the 3000+ activities .