How do I get the latitude/longitude from inputing a city/state name using the Maxmind geolite2 database?
(I installed the geolite2 php API already for ip lookups but now in addition I want to use it in reverse, so to speak)
If there is another practical way to do this in php by all means let me know but I DO NOT want to use the google geo-encoder API or any other external API for this.
THANK YOU!
Simply put: you can't. Not only does the API not provide the means to do this (MaxMind is an IP-based geocoding provider,) the database doesn't come remotely close to containing every city you might want to look up.
To get at the few cities that are in there you can grab the city database in CSV format and import the tables (there are two: "locations" has city and state names while "blocks" has latitude and longitude information) into your own SQL database to do the lookups from there.
If you change your mind about using another API, you might want to take a look at geocoder-php and choose a provider that supports address-based geocoding which is what you're after.
Related
I want to put together a PHP script to resolve city name (nothing else is needed) with a good resolution just for a single country (IRAN). As I have to query the DB for multiple times, its better to go through a downloadable local version.
I have read most of posts on stackoverflow and since now I have tested these:
GeoIP City from maxmind sounds good, but is not free.
GeoIP from maxmind, has a low level of accuracy (about 50-60%)
ip2country.net has an IP-2-City Database but not free and does not resolve city names for Iran.
I also tried the DB#.Lite from ipinfodb.com which has an API here without any success. The problem is that, it does not detect many city names.
I also tried hostip.info API, but it seems to be too slow.
There is a free php class with local DB which resolves only the country name.
I dont know if there is chance, using Piwik with this GeoIP plugin. It would be appreciated to have ideas if someone knows about it.
ipinfo.io is another service which does not resolve city names with accuracy.
I dont know if there is a way to use Google analytics to resolve city names, as I think google would be better than any other service regarding countries like Iran.
Any good idea would be really appreciated.
This is a tough one and hard to do reliably. I have given it a go in the past and it went something like this
Obtain a database of IP addresses, plus cities and countries (http://lite.ip2location.com/database-ip-country-region-city OR http://ipinfodb.com/ip_database.php)
Get the IP address and query for it against those tables to find the city and country
Finally check if its in Iran using the country column
There are paid for services that can do this really quickly for you. It might take you ages to get something working that is still unreliable because you simply do not have the data. I would seriously consider http://www.maxmind.com/en/city_per - unless of course this is a completely none commercial project and $ is a no no.
If you can get the lat and long from an IP table, even without the city data then you may want to then use something like this to check for the nearest city of Javascript is an optio n - Finding nearest listed (array?) city from known location.
What about the browsers Share Location feature?
If a browser-based solution works for your use case, you might want to look at MaxMind's GeoIP2 JavaScript API. It first attempts to locate the user using HTML5 geolocation and if that fails or is inaccurate, it reverts to MaxMind's GeoIP2 City data (not GeoLite). MaxMind provides a free attribution version.
Sometimes we need to use local Geo-IP database instead of web services for particular purposes. This is my experience:
I downloaded database form https://db-ip.com. I was searching a lot and finally I found this one more reliable. but still two more problem:
1-The "IP address to city" database is too huge to upload on MYSQL database on hosting as it pass time out limit.
2-The Database is base on IP address compare of http://lite.ip2location.com Database is base of IP Numbers.
So I developed a simple .NET app to solve those problems.
The solution can be downloaded from here: https://github.com/atoosi/IP2Location-Database-Luncher
At the moment I have a web app that uses Google Maps APIs to get distance between one location (store) and a list of other locations (techs), as its all cached in a DB, it only has to run once. But its very slow that first time....
What im needing to do now is make a front end for customers, where they can enter in their address and find the closest store. As there is a tone of stores, trying to get addresses of each store and comparing it with the customers address using Google Maps API is going to take forever.
So what techniques are available to make this faster? Is it possible to filter out most of the addresses with the difference of lat and long? Or is their an API method that I can give Google Maps with a long list of addresses and have it return the closest address?
What you're supposed to do is when adding the stores to the database also include their lat/lngs. This way when the user enters their address, you just need to geocode their address(get the lat/lng), and use the haversine formula (which is just math so it runs fast) to get the closest locations.
https://developers.google.com/maps/articles/phpsqlsearch_v3
You can use the google geocoding api to get/insert the lat/lng of the stores already in your database.
I currently have a locator that searches tens of thousands of stores and it executes in under a second using this method.
Unfortunately there isn't such a function in Google maps API and you have to implement it yourself. A good start is the mysql point data type to store the locations geo location and a spatial index. A spatial index reduces the dimension complexity and makes nearest-neighbor lookup simple. A spatial index is a r-tree and a space filling curve, for example a hilbert curve. Here is a nice article about hilbert curves: http://blog.notdot.net/2009/11/Damn-Cool-Algorithms-Spatial-indexing-with-Quadtrees-and-Hilbert-Curves. Here is an example of a spatial index with mysql: Selecting nearest geo coordinates using spatial index on mysql table not working. Here is an example of mysql distance function Correct way of finding distance between two coordinates using spatial function in MySql. Here is an example of MySql 5.1 http://www.elevatedcode.com/articles/2009/03/06/speeding-up-location-based-searches/.
I have a whitelist of cities. Let's say, Seattle, Portland, Salem. Using GeoIP, I'd detect user city. Let's call it $user_city. Based on $user_city, I want to display classified-listings from nearest city from my whitelist (Seattle || Portland || Salem) with in 140 miles. If city is not listed in 140 miles, I'd just show a drop-down and ask user to manually select a city.
There are a few ways of doing this:
calculate this on the fly (I found an algorithm in one of SO answers)
with help of DB (let me explain):
create a table called regions
regions will have
city 1 | city 2 | distance (upto 140 miles)
city 1= cities from whitelist
city 2= any city within 140 miles from city 1
This would create a reasonable sized table. If my whitelist has 200 cities, and there are 40 cities (or towns) within 140 miles of each city. This would create 8000 rows.
Now, when a user comes to my site:
1) I check if user is from whitelist city already (city 1 column). If so, display that city
2). If not, check if $user_city is in "city 2" column
2a) if it is, get whitelist city with lowest distance
2b) if it is not, display drop-down for manual input
Final constraint: whichever method we select, it has to work from within iFrame. I mean, can I create this page on my mysite1.com and embed this page inside someothersite2.com inside an iframe? Will it still be able to get user_city and find nearest whitelisted city? I know there are some cross-domain scripting rules so I am not sure if iFrame would be able to get user-ip address, pass it to GeoIP, and resolve it to $user_city
So, my question:
How best to do this? If a lot of people embed my page in their page (using iframe) then my server would get pounded 10000s of times per second (wishful thinking, but let's assume that's the case). I don't know if a DB would be able to handle so much pounding. I don't want to have to pay for more DB servers or web-servers. I want to minimize resource-requirement at my end. So, I don't mind offloading a bit of work to user's browser via JavaScript.
EDIT:
Some answers have recommended storing lat, long and then doing the Math. The reason I suggested creating a 'regions' table is that this way all math is precomputed. If I have a "whitelist" of cities, and if I precompute all possible nearby city for each whitelisted city. Then I don't have to compute distance (using Haversine algorithm for eg) everytime.
Is it possible to offload all of this to user's browser via some crafty use of Java Script? I don't want to overload my server for a free service. It might make money but I am very close to broke and I am afraid my server would go down before I make enough money to pay for the upgrades.
So, the three constraints of this problem are 1) should work from inside iframe (I am hoping this will go viral and every blogger would want to embed my site into their page's iframe. 2) should be very fast 3) should minimize load on my server
Use one table City and do a mysql math-calculation for every query, with the addition of a cache layer eg memcache. Fair performance and very flexible!
Use two tables City (id,lat,lng,name) and Distance (city_id1,city_id2,dist), get your result by a traditional JOIN. (Could use a cache layer too.) Not very flexible.
Custom data structure: CityObj (id,lat,lng,data[blob]) just serialize and compress a php-array of the cities and store it. This might rise your eyebrows but as we know the bottleneck is never CPU or memory, it's disc IO. This is one read from an index of an INT as apposed to the JOIN which uses a tmp-table. This is not very flexible but will be fast and scalable. Easy to shard and cluster.
Is it possible to offload all of this to user's browser via some crafty use of Java Script? I don't want to overload my server for a free service. It might make money but I am very close to broke and I am afraid my server would go down before I make enough money to pay for the upgrades.
Yes, it is possible...using Google Maps API and the geometry library. The function you are looking for is google.maps.geometry.spherical.computeDistanceBetween. Here is an example that I made a while ago that might help get you started. I use jQuery here. Take a look at the source to see what's happening and modify as needed. Briefly:
supplierZips is an Array of zip codes comparable to your city whitelist.
The first thing I do on page load is geocode the whitelist locations. You can actually do this ahead of time and cache the results, if your city whitelist is constant. This'll speed up your app.
When the user enters a zip code, I first check if it's a valid zip from a json dataset of all valid zip codes in the U.S.( http://ampersand.no.de/maps/validUSpostalCodes.json, 352 kb, data generated from zip code data at http://www.geonames.org).
If the zip is valid, I compute the location between that zip and each location in the whitelist, using the aforementioned computeDistanceBetween in the Google Maps API.
Hope this helps get you started.
You just have to get the lat and the long of each city and add it to the database.
So every city only has 1 record. No distances are stored on the position on the globe.
Once you have that you can easily do a query with using haversine formula ( http://en.wikipedia.org/wiki/Haversine_formula ) to get the nearest cities within a range.
know there are some cross-domain scripting rules so I am not sure if iFrame would be able to get user-ip address
It will be possible to get the user ip or whatever if you just get the info from the embedded page.
I don't know if a DB would be able to handle so much pounding
If you have that many requests you should have by then found a way to make a buck with it :-) which you can use for upgrades :D
Your algorithm seems generally correct. What I would do is use PostGIS (a postgresql plugin, and easier to set up than it looks :-D). I believe the additional learning curve is totally worth it, it is THE standard for geodata.
If you put the whitelist cities in as POINTs, with latitudes and longitudes, you can actually ask PostGIS to sort by distance to a given lat/lon. It should be much more efficient than doing it yourself (PostGIS is very optimized).
You could get lats and longs of your user cities (and the whitelist cities) by using a geocoding API like Yahoo Placefinder or Google Maps. What I would do would be to have a table (either the same as the whitelist cities or not) that stores city name, lat, and lon, and do lookups on that. If the city name isn't found though, hit the API you are using, and cache the result in the table. This way you'll quickly not need to hit the API except for obscure places. The API is fast too.
If you're really going to be seeing that kind of server load, you may want to look into using something besides PHP though (such as node.js). Incidentally you shouldn't have any trouble geocoding from an iframe, from the Point of View of the server, its just like the browser is going to that page "normally".
So, I am trying to develop an application that will display user listings. The site should detect user location (I am using maxmind api for that) and then show listsings from user's location + cities within a user-specified radius.
How do I do this? MaxMind API lets me detect user's city by IP address but how do I find nearby cities?
Reference site: www.oodle.com (you can also manually change city+radius).
Sanguine
Rather than store and compare cities, store and compare latitudes and longitudes, which are concrete locations rather than ambiguous names. All of MaxMind's GeoIP databases provide them. A quick Google search should provide you the math to calculate distances between points on the earth.
If you actually want to find nearby cities, not nearby users as you've said, then you need a database mapping cities to locations. Again, MaxMind provides this with all their databases. Go to their website, go to the page about the database you purchased or downloaded, and look at the instructions for inserting the CSV format into a SQL database. That'll get you the latitude and longitude of each city. Then, again, a Google search will provide you the math to calculate the distance between two points on the earth (lat/long pairs) in a SQL query. Order by that calculation to get the nearest cities.
Sorry to give you only advice rather than code, but there's a lot of little things you've just gotta do yourself to build this site.
I'm developing a store location application.
Looking up a store, it currently shows the location in googlemaps based on address and zip code.
Now I want to build a function which also shows other shops within 500 meter radius.
To do this, I have to do a proximity search / calculation.
My biggest question, is how I should approach this.
I did find this link, which has some example code. But I'm unsure if I can use the code (and which of the codes I should use). Does anyone have better examples?
Also I'm thinking of adding a new table to the database, which stores the geo code for each store. Do I need more fields than 'id', 'latitude' and 'longitude' ?
UPDATE
I just found this link at phpro.org. It looks like it's just what I need! Has anyone used their examples and can comment upon it?
You cant radius search directly with the google maps API, however, you should know (or can figure them out via geocoding their addresses) the latitude and longitude of each point of interest (POI) you want to include in the search.
After this you can use the Great Circle equation to search for proximity, and it turns out to be very fast. We have implemented this as a stored procedure for the locator service at my work and use it to search through >3500 locations with response times under 0.1 seconds.
Some SQL implementations contain geospatial extensions. In those implementations you can directly write a WHERE clause that filters the results by distance from a specified point.
Check the documentation of your SQL implementation. If it has geospatial POINT type, then it makes sense to enter the coordinates as POINTs rather than a lat/lng pair, and consider using that field as a SPATIAL KEY if you're going to be accessing the table mainly by location.