I am in the very early stages of creating a simple web-based tool for a client for internal use that can search their existing data of housing options based on certain criteria, one of which being distance from a given point. I will be using PHP and MySQL.
It's basically like an apartment locator.
The searching within a radius is the one thing I see that could pose a problem. I know this can be done easily if I had the lat/long coordinates already, but as it stands right now, I would have to convert all the addresses, and there are over 100.
The goal is to type in an address and return a list of properties X miles from that address.
Is there a way to do the lat/long conversion on the fly instead of converting all their existing property addresses to lat/long?
If I do need to do a batch conversion of addresses, so be it, but moving forward, is there a way to automate that when adding a new record?
Is there some kind of customizable apartment locator API I could use for this?
I'm not looking for any specific code, just need to be pointed in the right direction to figure out what I need to do to accomplish this.
At the moment I have a web app that uses Google Maps APIs to get distance between one location (store) and a list of other locations (techs), as its all cached in a DB, it only has to run once. But its very slow that first time....
What im needing to do now is make a front end for customers, where they can enter in their address and find the closest store. As there is a tone of stores, trying to get addresses of each store and comparing it with the customers address using Google Maps API is going to take forever.
So what techniques are available to make this faster? Is it possible to filter out most of the addresses with the difference of lat and long? Or is their an API method that I can give Google Maps with a long list of addresses and have it return the closest address?
What you're supposed to do is when adding the stores to the database also include their lat/lngs. This way when the user enters their address, you just need to geocode their address(get the lat/lng), and use the haversine formula (which is just math so it runs fast) to get the closest locations.
https://developers.google.com/maps/articles/phpsqlsearch_v3
You can use the google geocoding api to get/insert the lat/lng of the stores already in your database.
I currently have a locator that searches tens of thousands of stores and it executes in under a second using this method.
Unfortunately there isn't such a function in Google maps API and you have to implement it yourself. A good start is the mysql point data type to store the locations geo location and a spatial index. A spatial index reduces the dimension complexity and makes nearest-neighbor lookup simple. A spatial index is a r-tree and a space filling curve, for example a hilbert curve. Here is a nice article about hilbert curves: http://blog.notdot.net/2009/11/Damn-Cool-Algorithms-Spatial-indexing-with-Quadtrees-and-Hilbert-Curves. Here is an example of a spatial index with mysql: Selecting nearest geo coordinates using spatial index on mysql table not working. Here is an example of mysql distance function Correct way of finding distance between two coordinates using spatial function in MySql. Here is an example of MySql 5.1 http://www.elevatedcode.com/articles/2009/03/06/speeding-up-location-based-searches/.
I have a mongo collection of store locations complete with coordinates. I am creating an app that will get the users coordinates and find the closest store to their location. What is the most effective way to sort through the stores to get the result. How do I then get how many miles away the store is?
I am using the standard PHP mongo driver.
You can use the MongoDB GeoSpatial query to perform this query. This feature is documented here: http://www.mongodb.org/display/DOCS/Geospatial+Indexing
I can answer the very last part of your question "How do I then get how many miles away the store is?", you can use the Google Maps API to find out the driving distance between the current location and the nearest store. As for the first bit, I need more information, which I've alluded to in the comments.
I have a whitelist of cities. Let's say, Seattle, Portland, Salem. Using GeoIP, I'd detect user city. Let's call it $user_city. Based on $user_city, I want to display classified-listings from nearest city from my whitelist (Seattle || Portland || Salem) with in 140 miles. If city is not listed in 140 miles, I'd just show a drop-down and ask user to manually select a city.
There are a few ways of doing this:
calculate this on the fly (I found an algorithm in one of SO answers)
with help of DB (let me explain):
create a table called regions
regions will have
city 1 | city 2 | distance (upto 140 miles)
city 1= cities from whitelist
city 2= any city within 140 miles from city 1
This would create a reasonable sized table. If my whitelist has 200 cities, and there are 40 cities (or towns) within 140 miles of each city. This would create 8000 rows.
Now, when a user comes to my site:
1) I check if user is from whitelist city already (city 1 column). If so, display that city
2). If not, check if $user_city is in "city 2" column
2a) if it is, get whitelist city with lowest distance
2b) if it is not, display drop-down for manual input
Final constraint: whichever method we select, it has to work from within iFrame. I mean, can I create this page on my mysite1.com and embed this page inside someothersite2.com inside an iframe? Will it still be able to get user_city and find nearest whitelisted city? I know there are some cross-domain scripting rules so I am not sure if iFrame would be able to get user-ip address, pass it to GeoIP, and resolve it to $user_city
So, my question:
How best to do this? If a lot of people embed my page in their page (using iframe) then my server would get pounded 10000s of times per second (wishful thinking, but let's assume that's the case). I don't know if a DB would be able to handle so much pounding. I don't want to have to pay for more DB servers or web-servers. I want to minimize resource-requirement at my end. So, I don't mind offloading a bit of work to user's browser via JavaScript.
EDIT:
Some answers have recommended storing lat, long and then doing the Math. The reason I suggested creating a 'regions' table is that this way all math is precomputed. If I have a "whitelist" of cities, and if I precompute all possible nearby city for each whitelisted city. Then I don't have to compute distance (using Haversine algorithm for eg) everytime.
Is it possible to offload all of this to user's browser via some crafty use of Java Script? I don't want to overload my server for a free service. It might make money but I am very close to broke and I am afraid my server would go down before I make enough money to pay for the upgrades.
So, the three constraints of this problem are 1) should work from inside iframe (I am hoping this will go viral and every blogger would want to embed my site into their page's iframe. 2) should be very fast 3) should minimize load on my server
Use one table City and do a mysql math-calculation for every query, with the addition of a cache layer eg memcache. Fair performance and very flexible!
Use two tables City (id,lat,lng,name) and Distance (city_id1,city_id2,dist), get your result by a traditional JOIN. (Could use a cache layer too.) Not very flexible.
Custom data structure: CityObj (id,lat,lng,data[blob]) just serialize and compress a php-array of the cities and store it. This might rise your eyebrows but as we know the bottleneck is never CPU or memory, it's disc IO. This is one read from an index of an INT as apposed to the JOIN which uses a tmp-table. This is not very flexible but will be fast and scalable. Easy to shard and cluster.
Is it possible to offload all of this to user's browser via some crafty use of Java Script? I don't want to overload my server for a free service. It might make money but I am very close to broke and I am afraid my server would go down before I make enough money to pay for the upgrades.
Yes, it is possible...using Google Maps API and the geometry library. The function you are looking for is google.maps.geometry.spherical.computeDistanceBetween. Here is an example that I made a while ago that might help get you started. I use jQuery here. Take a look at the source to see what's happening and modify as needed. Briefly:
supplierZips is an Array of zip codes comparable to your city whitelist.
The first thing I do on page load is geocode the whitelist locations. You can actually do this ahead of time and cache the results, if your city whitelist is constant. This'll speed up your app.
When the user enters a zip code, I first check if it's a valid zip from a json dataset of all valid zip codes in the U.S.( http://ampersand.no.de/maps/validUSpostalCodes.json, 352 kb, data generated from zip code data at http://www.geonames.org).
If the zip is valid, I compute the location between that zip and each location in the whitelist, using the aforementioned computeDistanceBetween in the Google Maps API.
Hope this helps get you started.
You just have to get the lat and the long of each city and add it to the database.
So every city only has 1 record. No distances are stored on the position on the globe.
Once you have that you can easily do a query with using haversine formula ( http://en.wikipedia.org/wiki/Haversine_formula ) to get the nearest cities within a range.
know there are some cross-domain scripting rules so I am not sure if iFrame would be able to get user-ip address
It will be possible to get the user ip or whatever if you just get the info from the embedded page.
I don't know if a DB would be able to handle so much pounding
If you have that many requests you should have by then found a way to make a buck with it :-) which you can use for upgrades :D
Your algorithm seems generally correct. What I would do is use PostGIS (a postgresql plugin, and easier to set up than it looks :-D). I believe the additional learning curve is totally worth it, it is THE standard for geodata.
If you put the whitelist cities in as POINTs, with latitudes and longitudes, you can actually ask PostGIS to sort by distance to a given lat/lon. It should be much more efficient than doing it yourself (PostGIS is very optimized).
You could get lats and longs of your user cities (and the whitelist cities) by using a geocoding API like Yahoo Placefinder or Google Maps. What I would do would be to have a table (either the same as the whitelist cities or not) that stores city name, lat, and lon, and do lookups on that. If the city name isn't found though, hit the API you are using, and cache the result in the table. This way you'll quickly not need to hit the API except for obscure places. The API is fast too.
If you're really going to be seeing that kind of server load, you may want to look into using something besides PHP though (such as node.js). Incidentally you shouldn't have any trouble geocoding from an iframe, from the Point of View of the server, its just like the browser is going to that page "normally".
I have a MySQL database of addresses (my custom points of interest - not Googles) and their associated Lat/Long. What would be the simplest way for a user to my site (their lat/long is also stored in their user profile) to be shown a:
google map with their own lat/long as a starting (center) point
default radius of 10 Miles
all Points of Interest within that 10mile radius
If it is simpler to use zipcodes for the POI's and the user I have those as well.
Thanks in advance for any advice given, or information leading to tutorials that could get this done.
You can see how to get the lat/lon radius on another question: php mysql compare long and lat, return ones under 10 miles
You will need to implement access to the google maps api and dump the variables into the points of interest. You may be able to glean some useful information from a cakephp helper:
http://github.com/cdburgess/helpers/blob/master/google_map.php
You should be able to strip out the function and use it properly in PHP.