I'm trying to find something, preferably F/OSS, that can generate a Google Maps overlay from KML and/or KMZ data.
We've got an event site we're working on that needed to accommodate ~16,000 place markers last year and will likely have at least that many again this year. Last year, the company that had done the site just fed the KML data directly to the gMaps API and let it place all of the markers client side. Obviously, that became a performance nightmare and tended to make older browsers "freeze" (or at least appear frozen for several minutes at a time).
Ideally this server side script would take the KML, the map's lat/lon center, and the map zoom level and appropriately merge all of the visible place markers into a single GIF or PNG overlay.
Any guidance or recommendations on this would be greatly appreciated.
UPDATE 10/8/2008 - Most of the information I've come across here and other places would seem to indicate that lessening the number of points on the map is the way to go (i.e. using one marker to represent several when viewing from a higher altitude/zoom level). While that's probably a good approach in some cases, it won't work here. We're looking for the visual impact of a US map with many thousand markers on it. One option I've explored is a service called PushPin, which when fed (presumably) KML will create, server side, an overlay that has all of the visible points (based on center lat/lon and zoom level) rendered onto a single image, so instead of performing several thousand DOM manipulations client side, we merge all of those markers into a single image server side and do a single DOM manipulation on the client end. The PushPin service is really slick and would definitely work if not for the associated costs. We're really looking for something F/OSS that we could run server side to generate that overlay ourselves.
You may want to look into something like Geoserver or Mapserver. They are Google map clones, and a lot more.
You could generate an overlay that you like, and Geoserver(I think mapserver does as well) can give you KML, PDF, png, and other output to mix your maps, or you could generate the whole map by yourself, but that takes time.
Not sure why you want to go to a GIF/PNG overlay, you can do this directly in KML. I'm assuming that most of your performance problem was being caused by points outside the user's current view, i.e. the user is looking at New York but you have points in Los Angeles that are wasting memory because they aren't visible. If you really have 16,000 points that are all visible at once for a typical then yes you'll need to pursue a different strategy.
If the above applies, the procedure would be as follows:
Determine the center & extent of the map
Given that you should be able to calculate the lat/long of the upper left and lower right corners of the map.
Iterate through your database of points and check each location against the two corners. Longitude needs to be greater (signed!) than the upper left longitude and less than the lower right longitude. Latitude needs to be less than the upper left latitude (signed!) and greater than the lower right latitude. Just simple comparisons, no fancy calculations required here.
Output the matching points to a temporary KML for the user.
You can feed KML directly into Google Maps and let it map it, or you can use the Javascript maps API to load the points via KML.
It might not solve your exact problem here, but for related issues you might also look into the Google Static Maps API. This allows you to create a static image file with placemarkers on it that will load very quickly, but won't have the interactivity of a regular Google map. Because of the way the API is designed, however, it can't handle anywhere near 16,000 points either so you'd still have to filter down to the view.
I don't know how fare you are with your project but maybe you can take a look at GeoDjango? This modified Django release includes all kinds of tools to store locations; convert coordinates and display maps, the easy way. Offcourse you need some Python experience and a server to run it on, but once you've got the hang of Django it works fast and good.
If you just want a solution for your problem try grouping your results at lower zoom levels, a good example of this implementation can be found here.
This is a tough one. You can use custom tilesets with Google Maps, but you still need some way to generate the tiles (other than manually).
I'm afraid that's all I've got =/
OpenLayers is a great javascript frontend to multiple mapping services or your own map servers. Version 2.7 was just released, which adds some pretty amazing features and controls.
Related
I've desperately searched for the right solution that could handle up to 50 000 markers ( and why not even more if server can handle it ) while being easy on my server and I've been doing this for days.
Concept is something like this:
What would be the best solution / language for clustering server
side? Im not looking for a huge working code but I need to be pushed to right direction.
How to "cache" or "mark" already loaded clusters / markers so that you
wouldn't reload markers / clusters that has already been loaded when map is beeing zoomed in/out or panned?
Theory:
Right way to go seems similar to this:
If map is zoomed out and there are a lot of markers, handle clustering in server
If map is zoomed in to a point that all markers don't fit to viewport anymore, cluster only markers that are in viewport - also server
If zoom gets so big that number of markers in viewport gets small (< 500 for example), cluster in browser via JS
Data I could use in server side and which I could also send to browser via JSON:
Map bounds sent via ajax to server when request is made
Total count of markers from database query
Coordinates of markers from database query
Probably something else important / useful that Im currently not realizing
I'm working with google maps applications with 50,000 100,000 polygon It has proved very useful the possibility of organizing the polygons into a series of regions, (sheets logic, areas or similar). Using an array of regions already displayed that is sent to the server via ajax along with the coordinates you bound allowed me to select only regions not yet displayed and thus reduce both the bandwidth allocation is the time for rendering polygons,
For the rest of the strategy is right. for a more thorough evaluation should get in-depth technical little practicable in a social as SO.
For the server side language they are all okay, I personally right now I'm using PHP and framework Yii2 with PDO driver for MySQL. The important thing is to use an environment makes it very easy and efficient access to the database.
Under these conditions, the use of a good organization of the cluster and the display of polygons to regions when the zoom level allows it makes applications google maps very performing
Please take note that the appearance of markers is faster than that of the polygons as it is less the number of coordinates involved and theirs graphic characteristics .
For a general question like your is difficult find a good answer online. For the regions depends of the dimension of the area you manage and the average density of the markers. For area not near the pole you can use a fraction of the coord.. or if your geo point have some attribute related to area you can use them. You can find formulas online for link the zoom with an approximate dimension of the area. Depend also how the collection of markers is done. In my case during the collection i eval some attribute for this needs.
For the check is a markers or a group of a markers are already in maps i use simply vector and use this (via ajax) for dinamic exchange with the server. Firts time the vector is empty and progressively the vectore contain element for "know" what are already displayed.
I'm trying to figure out how to build a statistical map for my web app. Here's what I've got:
I have a MySQL database of zip codes, and each zip code has latitude & longitude.
I have users who have declared what zip code they live in.
I even have a haversine query which will show how many users exist within, for example, 25 miles of a given latitude/longitude, based on their zip code.
My question is this: Using this information, how could I approach building a statistical map for a web application using PHP?
I would be fine with using just a US map or even a North American map for now, but I'm just not sure how to build that map. Some options I've considered:
Show a colored dot on the map, larger or smaller depending on the number of users near that location. I'm not sure how to do this, though, especially if those dots were to overlap!
Show individual "pushpins" where the users are. Seems like this could get out of hand if my user base grows significantly
So back to my question. If I had 300 users in Dallas, 4,000 in NYC, 45 users in Detroit, 403 in Chicago... how would I be able to represent that on a map -- and also how would I draw that map in a web application built on PHP?
You are trying to build a three-dimensional (probably even more dimensions) data display.
Your dimensions are:
X-Location
Y-Location
The value at every location
This really does not define anything about the visual appearance, though.
A simple approach might be to calculate the absolute number of users per state and then color the state on the map according to some scale. You also might calculate the percentage of users living in a state compared to the absolute number and color that instead.
A different approach would be to put a dot for every user on the map, and if this dot was printed before, to change it's color instead, e.g. make it brighter.
In the end, it really depends on what your actual data is and if your approach on visualizing it displays some significant information - but this can only be confirmend after you see it.
As you are looking for a web application have you considered Google Maps. Factor 1. can be implemented using the MarkerClusterer library. A DEMO showing this. The data from your database can be loaded using AJAX.
What mean-stat-equation should I use when I have an image with N-number sample-size of selections?
I have a unique problem for which i was hoping to get some advice, so that i don't miss out on anything.
The Problem: To find the most favored/liked/important area on an image based on user selection of areas in different selection ratios.
Scenario: Consider an Image of a dog, and hundreds of users selecting area over this image in various resolutions, the obvious area of focus in most selections will be the area containing the dog. I can record the x1,x2,y1,y2 co-ordinates and put them into a db, now if i want to automatically generate versions of this image in a set of resolutions i should be able to recognize the area with the max attraction of the users.
The methods i think could work are:
Find the average center point of all selections and base the selection in that. - Very simple but would not be as accurate.
Use some algorithm like K Means or EM Clustering but i don't know which one would be best suited.
Looking forward to some brilliant solution to my problem
More info on the problem:
The Actual image will be most probably be a 1024x768 image, and the selections made on it will be of the most common mobile phone resolutions. The objective is to automatically generate mobile phone wallpapers by intelligent learning based on user selections.
I believe that you have two distinct problems identified above:
ONE: Identification of Points
For this, you will need to develop some sort of heuristic for identifying whether a point should be considered or not.
I believe you mentioned that hundreds of users will be selection locations over this image? Hundreds may be a lot of points to cluster. Consider excluding outliers (by removing points which do not have a certain number of neighbors within a particular distance)
Anything you can do to reduce your dataset will be helpful.
TWO: Clustering of Points
I believe that K Means Clustering would be best suited for this particular problem.
LINK
Your particular problem seems to closely mirror the standard Cartesian coordinate clustering examples used in explaining this algorithm.
What you're trying to do appears to be NP-Hard, but should be satisfied by the classical approximations.
Once clustered, you can take an average of the points within that cluster for a rather accurate approximation.
In Addition:
You dataset sounds like it will already be tightly clustered. (i.e. Most people will pick the dog's face, not the side of it's torso.) You need to be aware of local minima. LINK These can really throw a wrench into your algorithm. Especially with a small number of clusters. Be aware that you may need a bit of dynamic programming to combat this. You can usually introduce some variance into your algorithm, allowing the average points to "pop out" of these local minima. Local Minima/Maxima
Hope this helps!
I think you might be able to approach your problem in a different way. If you have not heard of Seam Carving then I suggest you check it out, because the data you have available to use is perfectly suited to it. The idea is that instead of cropping an image to resize it, you can instead remove paths of pixels that are not necessarily in a straight line. This allows you to resize an image while retaining more of the 'interesting' information.
Ordinarily you choose paths of least energy, where energy here is some measurement of how much the hue/intensity changes along the path. This will fail when you have regions of an image that are very important (like a dog's face), but where the energy of those regions is not necessarily very high. Since you have user data indicating what parts of the image are very important you can make sure to carve around those regions of the image by explicitly adding a little energy to a pixel every time someone selects a region with that pixel.
This video shows seam carving in action, it's cool to watch even if you don't think you'll use this. I think it's worth trying, though, I've used it before for some interesting resizing applications, and it's actually pretty easy to implement.
I have a country map in .png format, I need to highlight specific states in this map dynamically using PHP. What's the best way to do this?
Thanks
You would need to create a list of all the points defining the polygon encasing each state, then use imagepolygon to fill in the appropriate polygons on the map image with some color.
Realistically you probably want to find a 3rd party library/component (i.e. Flash movie, JavaScript map and library, or PHP class designed for this) rather than writing it on your own from scratch. Highlighting a map isn't exactly a new problem and solving it again from scratch is tedious.
If you're talking about US States, I would recommend trying Google Maps, and the following links. I used them for the exact purpose you're talking about (votes by state) recently, and it worked great:
http://econym.org.uk/gmap/example_states2.htm
http://econym.org.uk/gmap/epoly.htm
http://econym.org.uk/gmap/states.xml
I'm looking for a quick way to drop in a map for something like Minecraft into a pan/zoomable thing, with support for custom coordinates (like defining where is 0,0 or something)
Can't seem to figure this out with google maps, any ideas? I have tile pngs for multiple zoomlevels, I have coordinates, just no clue how to implement it.
There are several ways to get the look and feel of a map.
One huge graphic
In case you've got only one huge picture of the map that is maybe even over 100 MB in size, use the service Zoom.it. Although it's supported by Microsoft and therefore uses Silverlight (JavaScript support available as well, though), it's a handy service. Besides, the homepage also stores the automatically generated tiles and hence serves as a webhoster as well.
Several tiles at different zoom levels
In this case it's probably best to use the Google Maps JavaScript API V3. It helps you realise all your map plans. In your case you'd have to overlay your map on the real world map. To be honest, that does sound weird as all coordinates on your specific map are also related to a region on the map of our beloved planet Earth. However, it works like a charm. Should you need webspace to upload your huge map in advance, use a service like Wuala for this purpose.
All-in-one Map Generator
As you want to create a map of your Minecraft world, it's maybe best to directly use Tectonicus, a deep map renderer that also creates all required HTML pages to render your map using the Google Maps API. You can find an example map here.