I am making a PHP-based RPG, city-building game. My idea is to create a 9X9 grid for a castle layout with the center 3X3 being the inner castle and upon clicking on that section they will see an 6X6 grid of the inner castle. The players will also be able to acquire tiles of the outer grid turning them into inner castle. Every tile is able to be built upon. What would the best way to represent this in the database, taking scalability into account?
The only approach I have come up so far is a 3-column table with (idcastle, Y, X), X being a string of 18 numbers. I would use substr to see if there is a building on that tile.
However, I think that I will run into scalability issues if there are a lot of castles (since each castle requires 18 rows).
It depends how you query the database. I would suggest dynamically making your id so it was
castleid_x_y_z that way you are only querying against the primary key making it very quick.
Plus use something like redis to handle it as it generally is limited only by network speed. If you have too many castles you would just push people to a second server. You don't even need to worry about scaling as you wouldn't split 1 castle over two servers
Related
I'm sitting in a situation where i have to build a statistics module which can store user related statistical informations.
Basically, all thats stored is a event identifier, a datetime object and the amount of times this event has been fired and the id of the object which is being interacted with.
Ive made similar systems before, but never anything that has to store the amount of informations as this one.
My suggestion would be a simple tabel in the database.
etc. "statistics" containing the following rows
id (Primary, auto-increment)
amount (integer)
event (enum -(list,click,view,contact)
datetime (datetime)
object_id (integer)
Usually, this method works fine, enabling me to store statistics about the object in a given timeframe ( inserting a new datetime every hour or 15 minutes, so the statistics will update every 15 minute )
Now, my questions are:
is theres better methods or more optimized methods of achieving
and building a custom statistics module.
since this new site will receive massive traffic, how do i go about the paradox that index on object id will cause slower update response time
How do you even achieve live statistics like etc. analytics? Is this solely about the server size and processing power? Or is there a best practice.
I hope my questions are understandable, and i'm looking forward to get wiser on this topic.
best regards.
Jonas
I believe one of the issues you are going to run into is you wanting two worlds of transactional and analytical. Which is fine in small cases, but when you start to scale, especially into realm of 500M+ records.
I would suggest separating the two, you generate events and keep track of just the event itself. You would then run analytical queries to get things such as count of events per object interaction. You could have these counts or other metric calculations aggregated into a report table periodically.
As for tracking events, you could either do that with keeping them in a table of occurrences of events, or have something before the database that is doing this tracking and it is then providing the periodic aggregations to the database. Think of the world of monitoring systems which use collect agents to generate events which go to an aggregation layer which then writes a periodic metric snapshot to an analytical area (e.g. CollectD to StatsD / Graphite to Whisper)
Disclaimer, I am an architect for InfiniDB
Not sure what kind of datasource you are using, but as you grow and determine amount of history etc... you will probably face sizing problems as most people typically do when they are collecting event data or monitoring data. If you are in MySQL / MariaDB / PostegreSQL , I would suggest you check out InfiniDB (open source columnar MPP database for analytics); It is fully open source (GPLv2) and will provide the performance you need to do queries upon billions and TBs of data for answering those analytical questions.
I have a dilemma that I need to figure out.
So I am building a website, where people can go watch a competitive game (Such as Counter Strike: Global Offensive), perhaps using either a Twitch TV stream, or actually through the matchmaking streaming services that the game may offer (In the case of this example, CS: GO TV). While playing, members can place "bets" on which teams will win, using some form of credits with no real value. Of course, the issue here, is that the site will need to be able to pull the score from the game, and update in real time. So sticking with the example of CS:GO, is there a portion of the Steamworks API, that would allow for real-time pulling of a game's score, through some kind of PHP or JavaScript method?
I'm sorry to tell you that you can't, for now.
In the API description of the CS:GO Competitive Match Information says:
It would be interesting to be able to find out competitive match information -- exactly like what DOTA 2 has. It could contain all the players in the map, with their steamids and competitive ranks, the score at half time/full time. There are probably a few more bits of info that could also be included. Pigophone2 16:54, 14 September 2013 (PDT)
To answer your question, there is no Steam developed API that does this.
However many websites still do exactly what you are looking for.
My guess is that they use a regularly updated script which parses websites like ESEA and ESL and pull data about those matches. After all, they are the ones who host almost all big games that people care about.
You'll need to keep up-to-date with private leagues though, as they don't typically publish live stats in an easily parse-able format. GOSU Gamers can help you track any new players that come to the big-league table.
I'm planning on integrating a reasonable ranking/voting system into an existing application.
I'm familiar with how the traditional 5 star rating systems work and know the common pitfalls/problems associated with them therefore was wondering if there is other ways (I've heard of Wilsons, Bayesian etc. but not really sure on how to implement this with the below structure):
I'm planning on allowing users to vote on content between 1 to 10 via the contents page.
The score and total votes for that content will be displayed on the contents page.
I will also be displaying/listing the Top 10 Content so I'd need the method to be fair/realistic and not make a vote of 10 with total votes of 1 to go straight to number 1.
I'm using PHP and MySQL, I have a table for the content (which has a content_id which I guess I can JOIN on).
I'm wondering if you can suggest a way/method which achieves the above, I'd appreciate if you can attach some example PHP code and example MySQL schema so I can better understand it, as I've google'd and may have found potential solutions such as Wilsons and Bayesian...yet they provide a lengthy article with confusing mathematical equations - and mention no way which achieves the above (ie. the score....and implenting the method in PHP/MySQL) or atleast due to there not being any example PHP/MySQL code me misunderstanding this.
Perhaps this is easier then I think - I don't know as I've never had the need to implement this sort of "more complex" ranking/voting functionality before - so I'd appreciate your responses.
You should start by watching this video on youtube : Building Web Reputation Systems.
To emphasize the point, let me direct you to XKCD.
As for DB structure, you need following parts:
list of items ( with total_votes column )
list of user, which have voted
intersection table for the items-users ( with rating column, if you go with 5star thing )
I am currently developing a kind of google maps overview widget that displays locations as markers on the map. The amount of markers varies from several hundreds up to thousands of markers (10000 up). Right now I am using MarkerClusterer for google maps v3 1.0 and the google maps javascript api v3 (premier) and it works pretty decent for lets say a hundred markers. Due to the fact that the number of markers will increase I need a new way of clustering the markers. From what I read the only way to keep the performance up is moving the clustering from the client-side to the server-side. Does anyone know a good PHP5 library which is able to get this done for me?
Atm I am digging deeper into the layer mechanisms of google maps. Maybe there are also a few leading PHP librarys I could start to check out? I also ran across FusionTables but since I need clustering I think this might not be the right solution.
Thanks in advance!
I don't know of a server-side library that'll do the job for you. I can however give you some pointers on how to implement one yourself.
The basic approach to clustering is simply to calculate the distance between your markers and when two of them are close enough you replace them with a single marker located at the mid-point between the two.
Instead of just having a limitation on how close to each other markers may be, you may also (or instead) choose to limit the number of clusters/markers you want as a result.
To accomplish this you could calculate the distance between all pairs of markers, sort them, and then merge from the top until you only have as many markers/clusters as you wish.
To refine the mid-point positioning when forming a cluster you may take into account the number of actual markers represented by each of the two to be merged. Think of that number as a weight and the line between the two markers as a scale. Then instead of always choosing the mid-point, choose the point that would balance the scale.
I'd guess that this simple form of clustering is good enough if you have a limited number of markers. If your data set (# of markers and their position) is roughly static you can calculate clustering on the server once in a while, cache it, and server clients directly from the cache.
However, if you need to support large scale scenarios potentially with markers all over the world you'll need a more sophisticated approach.
The mentioned cluster algorithm does not scale. In fact its computation cost would typically grow exponentially with the number of markers.
To remedy this you could split the world into partitions and calculate clustering and serve clients from each partition. This would indeed support scaling since the workload can be split and performed by several (roughly) independent servers.
The question then is how to find a good partitioning scheme. You may also want to consider providing different clustering of markers at different zoom levels, and your partitioning scheme should incorporate this as well to allow scaling.
Google divide the map into tiles with x, y and z-coordinates, where x and y are the horizontal and vertical position of the tile starting from the north-west corner of the map, and where z is the zoom level.
At the minimum zoom level (zero) the entire map consist of a single tile. (all tiles are 256x256 pixels). At the next zoom level that tile is divided into four sub tiles. This continues, so that in zoom level 2 each of those four tiles has been divided into four sub tiles, which gives us a total of 16 tiles. Zoom level 3 has 64 tiles, level 4 has 256 tiles, and so on. (The number of tiles on any zoom level can be expressed as 4^z.)
Using this partitioning scheme you could calculate clustering per tile starting at the lowest zoom level (highest z-coordinate), bubbling up until you reach the top.
The set of markers to be clustered for a single tile is the union of all markers (some of which may represent clusters) of its four sub tiles.
This gives you a limited computational cost and also gives you a nice way of chunking up the data to be sent to the client. Instead of requesting all markers for a given zoom level (which would not scale) clients can request markers on a tile-by-tile basis as they are loaded into the map.
There is however a flaw in this approach: Consider two adjacent tiles, one to the left and one to the right. If the left tile contains a marker/cluster at its far right side and the right tile contains a marker/cluster at its far left side, then those two markers/clusters should be merged but won't be since we're performing the clustering mechanism for each tile individually.
To remedy this you could post-process tiles after they have been clustered so that you merge markers/clusters that lay on the each of the four edges, taking into account each of the eight adjacent tiles for a given tile. This post-merging mechanism will only work if we can assume that no single cluster is large enough to affect the surrounding markers which are not in the same sub tile. This is, however, a reasonable assumption.
As a final note: With the scaled out approach you'll have clients making several small requests. These requests will have locality (i.e. tiles are not randomly requested, but instead tiles that are geographically close to each other are also typically accessed together).
To improve lookup/query performance you would benefit from using search keys (representing the tiles) that also have this locality property (since this would store data for adjacent tiles in adjacent data blocks on disk - improving read time and cache utilization).
You can form such a key using the tile/sub tile partitioning scheme. Let the top tile (the single one spanning the entire map) have the empty string as key. Next, let each of its sub tiles have the keys A, B, C and D. The next level would have keys AA, AB, AC, AD, BA, BC, ..., DC, DD.
Apply this recursively and you'll end up with a partitioning key that identifies your tiles, allows quick transformation to x,y,z-coordinates and has the locality property. This key naming scheme is sometimes called a Quad Key stemming from the fact that the partitioning scheme forms a Quad Tree. The locality property is the same as you get when using a Z-order curve to map a 2D-value into a 1D-value.
Please let me know if you need more details.
This article has some PHP examples for marker clustering:
http://www.appelsiini.net/2008/11/introduction-to-marker-clustering-with-google-maps
You could try my free clustering app. It is capable of more pins than the clientside google maps api. It offers kmeans an grid based clustering.
https://github.com/biodiv/anycluster
I am building a social networking site. i hope for some high traffic in it. i am using php and mysql in it. i already started with RDBMS kind of database. I read that many high traffic sites use key value database model. In my situation which one should i go for ? and i guess it would be better to decide it at this early stage itself
For now, stick with MySQL in a traditional RDBMS format if that is what you are most familiar with. Getting your site up and running as fast as possible is WAY more important than worrying about scale issues at the 1st stages of building a site.
That being said, it doesn't hurt to keep scale concerns in mind as you design parts of the system. MySQL is already very good at some basic scaleability pieces, such as sharding, so you will probably be just fine for quite a while. Having a good DB design, with plenty of indexes, will also keep you running if you do hit sufficiently high traffic levels.
Since you expect high traffic volume (don't we all?), I would highly suggest logging / tracking the load on your server so that you can measure the actual traffic and determine if you truly do need to scale (up or out are both good options depending on the load characteristics)
I think it would be good if you had the key value tables for relationships between friends.
For example person a could have 500 friends and person b could have 100. That means that to prevent duplicated data being copied over and over again in one table, you would get the id of the person one and id of the his friends and put them in a table. This will result in faster searches and inserts and updates because your are working with integers.
E.g
table friends_relationship
id - friend
1 - 2
1 - 3
2 - 4
3 - 4
you need to make sure that the relationships are unique