i cam across some older topics about this but since there is a lot of new techniques for creating web content i have to ask again.
My background:
I have good knowledge of xhtml/css/javascript/php/mysql. I have no trouble making a webshop from scratch including a comprehensive database and a cms to easily add/modify products. I am very comfortable using PHP and mysql to store data. A attempt at a tribal wars clone does not scare me at all. I also made a couple of games with c#/xna but i am still considering myself novice. I dabbled with flash and feel this could probably be used to get the result i want.
My project:
A 2D mmo tactical shooter. Players should be able to move across a fairly large maps and get position updates of other users very quickly. The game will hold a lot of maps and most maps should be able to handle just a couple of players. Hub maps and maps of interest however should be able to handle at least 100 players without lag.
It would be awesome if combat could be resolved real time on the map but i already thought of other ways to resolve this like opening an instance map where players could enter to join combat. I'm also considering a more tactical turn based approach on combat but it depends on the pro's and con's of real time combat.
To sum it up:
I am sure i can manage everything except for the graphical implementation. Mysql should not have much trouble querying the local player positions in a couple of milliseconds.
I cam across a game named Deeppolis that pretty much resembles graphically what i want. I'm still considering making this as a downloadable client but i am very interested what my options are if i'm going to do this browser based as browser based games tend to get more people attracted.
Thanks for reading this wall of text :D.
you would not want to "query mysql" to get player positions if this is real time. You will want to create a socket server that all clients connect to. the socket server would keep a subscriber list of all connected clients in memory. this way when one client sends a message to the socket server, the socket server can immediately push needed data to whichever clients need it. you would also want to keep a model of the game state in memory in your socket server to prevent cheating etc. you would only use a database for long term storage, everything else should be in memory. also keep in mind shared servers will likely not allow you access to sockets to create a socket server, so you will need a private server.
client side: javascript & socket.io (socket client)
server side: javascript & node.js (socket server)
Related
I'm currently building a facebook alike chatbox, and I have encounter several considerations and problems along the way.
I had been googling useful resources all the time,like simple chatbox example or tutorial online.
My goal is to build one just like facebook/gmail chatbox and CometChat, I know it's hard and too much thing to scale behind the scene, but all I want to do is building it as simple as possible, and figuring out how facebook/gmail chatbox implement their chat functionality.
Progress:
I have finished facebook-like chatbox structure where I have sidebar at the right displaying online friends i can chat with, and popup chatbox at the bottom, and it is able to expand and minimize it.
I also have finished simple chatting based on MySQL database.
There's a table with 4 columns 'sender', 'receiver', 'message', 'time' for storing conversation.
My chatbox works this way:
1.The user send a message, and my front-end javascript will fetch the message the user type in and send the message to php file on the server via Ajax.
2. backend php file will store this message to MySQL.
3. The front-end will call the update function every 3 seconds to update the chatbox content if receiver send message to the sender, and show it out in frontend's chat.
I'm not sure this is a good way and long way to do, and I'm really concerned about it.
If users grow and grow, I have to think of ways to scale it well or my database and server will explode and frontend users might feel high latency in updating conversation.
Is BigTable a right way to do this if you have millions of users online?
How does facebook store their customer's text message or chat history in the backend well??
How does chat app like Whatapp store their text message?
Is it able to let the users chat directly to another user without storing state in server?
If I want to implement the chat history functionality in my chatbox, what is a good way to do ??
I am thinking server can create .txt file for each conversation in their file system, and it has a database table column to store the file path. Is this a good way and right way to do with chat history, I know its possible to do it this way, but im not sure if its a right way or good way.
I know this could be a huge, detailed application.
I'm asking not a detailed implementation but a big picture, concept of building it!
thank you!.
That's a good question and here's an attempt at answering it.
I believe you are thinking about scalability a bit too early. Your IM app might not reach the projected number of users for it to stop performing well. Consider enhancing your small product and scale as you go as much as is needed.
Disk I/O is one of the issues that you will face scaling your web application. Storing communication directly onto the disk with txt file might not be a reliable solution.
Push your technology stack to its limits before considering changing it or switching to something else. I assume you are using a relational database for your storage (since you mentioned columns and rows, which is not an ultimate indicator but still), there are other options out there that have good benchmarking results at the expense of multiple other compromises. (NoSQL: which you referred to as BigTable) is one option. Relational databases are great, they have been for quite a long time the industry standard but currently there are alternative solutions that are quite promising.
Look into NoSQL document based datastorage solutions such as MongoDB, CoucheDB or even Casandra and there are many others. There is a considerable amount of information about the performance of each, under specific circumstances and situations. Choose what is best for the problem at hand and not what is most fashionable or hipped.
Another option would be to outsource your scalability problems to a 3rd Party provider such as Firebase. In this situation all you have to worry about is your product and not what's happening under the hood.
Store only the data that you need and archive or dismiss what you don't.
With scalability there are generally 2 broad categories: Horizontal and Vertical scaling.
Horizontal: means adding more nodes to your system i.e. adding more server instances to handle the extra load. There are many cloud solution providers out there that make this genre of scaling very cheap and instantaneous.
Vertical: means adding more resources to the node you are currently running your app from in addition to use specific technologies that allow you to take full advantages of your resources. This optimization happens on the level of the instance resources (i.e. CPU, RAM, Disk Space etc...) and your data storage, programming language of choice, algorithms you are using etc... You might realize that php and mysql aren't the tools for this job, but that's arguable.
Read More about it here
Distributed Systems architects / programmers also take advantage of other (faster) programming languages at runtime (such as C, C++ or even Java) to speed up certain tasks. Look into how you can dissect your application into smaller decoupled modules / components that can run independently. (But i'm not sure if you will ever reach this stage with an IM client unless it becomes as popular as Whatsapp or Facebook chat).
I advise you to grab and read a couple of books about scaling web applications and leveraging cloud computing. Study scalable architectures and design your application depending on your business logic based on them.
This is a very broad and complex topic, I'm sure others might have additional interesting insight on the matter.
I'm working on a chat application which I would love to use a SQL db for.
My problem is, after a few google searches, i have people telling me from one site, that using a DB would be much slower then using a normal file (e.g Text or JSON file), but then on some other sites, people are saying the complete opposite. And I don't know about you guys, but when it comes to creating web apps for users, the users always come first.
So as much as I'd love to use a SQL DB as 1.) I have good experience with it and 2.) it allows me to make the application much more cooler (more features). but if it would slow things down on the users end (a noticeable lag), then its a no-no.
Either way, I will be "polling" the server continuously with AJAX and PHP to check the file/DB (for new messages, contact requests, ect ect).
Also, incase your wondering, the application wont be like a 1-to-1 chat, it will have "rooms" where multiple users can join and talk with all users joining in. The users will also be able to request a "private chat" with another user, where a 1-to-1 connection opens up.
So, MySQL Database OR a boring TEXT/JSON/OTHER file, in regards to performance?
Oh, one more thing, I don't want to use any third party libraries or APIs. Hate relying on other peoples work (been let down to many times).
If you're looking to implement an IRC clone, I think you've chosen all the wrong tools.
The best way to do this would be to write a custom HTTP server that handles everything in memory. No databases, no constant polling of files. When a message arrives, you simply loop through the correct in-memory list and dispatch the message to other users. For the browser to server connection, I suggest "Comet" (with web sockets for browsers that support them, if you're feeling up to it).
PHP likely isn't the language of choice for this, because pretty much all work done with PHP is based on traditional short, isolated requests. For a long-running process which serves multiple clients in real time, I'd suggest something like Python or Node.js.
You don't really want to be storing chats in files, that can create a management nightmare, I would recommend you go with MySQL and to make sure it works probably go with Sockets instead of AJAX polling, Sockets will scale really well.
However there isn't much around about how you can integrate socket based chats with MySQL.
I have done a few tests and have a basic example working here: https://github.com/andrefigueira/PHP-MySQL-Sockets-Chat
It makes use of Ratchet (http://socketo.me/) for the creation of the chat server in PHP.
And you can send chat messages to the DB by sending the server JSON with the information of who is chatting, (if of course you have user sessions)
I would like to create a chess multiplayer game,1player vs 1player .
I will use flash for animations and client and a server-side language for checking if the moves are correct, record each player action to a database, decide winner and give some credits to the winner.
My main programming language is PHP and I will have someone do the flash work depending on my directions.
I would like to know if anybody tried to do this before , or if you have any tutorial, guide, steps to follow when developing p2p games ? I want to do this efficiently and to be able to support up to at least 100 players on a VDS with 512 RAM .
I am planning to have the game work like this :
1) Initialise variables like credits, player names inside the flash client , after retrieving them from php
2) Start the game
3) If I am player2 and player1 is thinking of a decision, the flash client will send requests to the PHP file to check the database if player1 made any decision. If so, it will be my turn to do a decision and move a piece
4) When my turn comes , no requests are sent to the PHP anymore, until I want to move a piece . When I try to do that ,it sends a request and the PHP will check if my move is valid, and return some variables that flash will use to show error or move pointer back to player1
The game will also have a timer that will sync with PHP. Each player can think 20 seconds to do a move.
1)Any suggestions for my implementation ?What im looking for are suggestions to improve efficiency on this and reduce server load and if someone ever had any experience with a similar project, how did they do it, or how would they do it ?
2)My implementation of periodical HTTP requests isnt such a good idea, because I know that the server can become overloaded, I'd like to know if someone can suggest any alternative but secure option . 100 users at the same time would mean about 100 HTTP requests/second. Is that a number that a VDS can handle for days ?
Later Edit :
After the replies that I got on this post , I came to the conclusion that using sockets instead of HTTP requests every second would be the best idea to use when waiting for a move on the opposite player.
My game would have 100 tables available for players, with 1 socket for communication and 1 port for each table . So 1 socket with 100 ports. PHP will communicate to flash the socket and port for the table , when the game starts and the initialisation variables are sent.
I plan to use flash to listen to the socket server and see what happens when the opposite player must move .
When I move , flash will check if the move is valid from it's side, then send data to PHP . PHP will return OK or NOT and will also update the data on the socket port.
About the socket , Im not sure javascript would be a good option for this, so Im thinking about PHP sockets or JAVA sockets.
PHP sockets seem to be my first option , due to familiarity with the language.
My 2nd would be to rent a smartfoxserver and my 3rd would be to use the services of Player.IO.
Im planing to create a very nice game and sell it. The graphics are pretty much done and they look great. The hardest part is doing this efficiently and easy to install after.
Thank you for your time ! Regards
The server load for a game like this should be really minimal. If you're using flash, I would take advantage of the socket object and just open a socket to your server, that way you don't need to pester it with questions about what has changed... it will notify the clients when/if something noteworthy has.
Otherwise your plan is a pretty good one, so, you'll probably have to ask some specific questions about things that you're stumped by.
If you REALLY want to reduce server load, write the chess logic in the flash client and let IT decide if an attempted move (by the player) is illegal, or if the player has lost or whatnot... use the server strictly for message relay and data storage.
It's sometimes a mistake to design a project around what you have available, such as PHP knowledge and access to a server with 512 MB of RAM. There are a lot of projects where you should consider your resources first, but an independent, first time game project isn't one of them.
Even though your primary language is PHP, you shouldn't restrict yourself from working on the Flash portion yourself. Also, using a web server and database as a communication method for a multiplayer game will result in some very clunky behavior.
You aren't limited by your VDS either. You can use a micro instance on Amazon EC2 for free for a year. If setting up an EC2 server from scratch seems daunting, there are tutorials all over the internet that take you from start to finish.
The front end of your game can be written in Javascript or Flash AS3. There are a lot of game development libraries available for both. Flixel, Flashpunk, and PushButton are free AS3 game libraries. MelonJS is a Javascript/HTML5 game library. All of them have extensive documentation and a good community.
You can write the server backend using node.js and use socket.io to handle your game communications, even if you don't know anything about packets -- or you could use something like Player.IO if you want to pay a little money and have an outside service worry about that for you.
If you write the backend properly, one hundred users won't be a problem for your server. You'll need to start thinking about new hardware when you get in the thousands.
And since you're looking for general game making advice, start reading articles on blogs like #AltDevBlogADay. You'll pick up a lot of insight on design from other people passionate about games.
I am building a "multiplayer world" with jQuery and PHP. Here is a bit how it works:
User's character's positions are taken from a database, user is plotted accordingly (the position values are CSS values - left and top)
User is able to move about using the arrow keys on the keyboard, making their character move using jQuery animations. While this is happening (on each arrow press) the user's position values are inserted into a database and updated.
In order to make this "global" (so users see each other) as you could say, the values need to be updated all at once for each user using AJAX
The problem I am having is I need to continuously call a JavaScript function I wrote which connects to the MySQL server and grabs values from a database table. And this function needs to be called constantly via setInterval(thisFunction, 1000); however my host just suspended me for overloading the server's resources and I think this was because of all my MySQL queries. And even after grabbing values from my database repeatedly, I had to insert values every few seconds as well so I can imagine that would cause a crash over time if enough clients were to login. How can I reduce the amount of queries I am using? Is there another way to do what I need to do? Thank you.
This is essentially the same thing as a chat system with regards to resource usage. Try a search and you'll find many different solution, including concepts like long polling and memcached. For example, check this thread: Scaling a chat app - short polling vs. long polling (AJAX, PHP)
You should look into long polling - http://en.wikipedia.org/wiki/Push_technology. This method allows you to establish a connection with your server, and then update it only when you need to. However by the sounds of it, you have a pretty intensive thing going on if you want to update every time, you may want to look into another way of storing this data, but if your wondering how big companies do it, they tend to have mass amounts of servers to handle it for them, but they will also use a technique similar to long polling.
You could store all the positions in memory using memcached See http://php.net/manual/fr/book.memcached.php and save them all at once every few seconds into the database (if needed).
You could use web sockets to overcome this problem. Check out this nettuts tutorial.
There is another way, it's to emulate or use actual sockets. Instead of constantly pulling the data (refreshing to check if there are new records), you can push the data over WebSockets which works in Chrome at the moment (at least to my knowledge, didn't try it in FF4) or you can use Node.js for leaner long pooling. That way, the communication between players will be bi-directional without the need of MySQL for storing positions.
Checkout Tornado
From their site:
Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed. The FriendFeed application is written using a web framework that looks a bit like web.py or Google's webapp, but with additional tools and optimizations to take advantage of the underlying non-blocking infrastructure.
The framework is distinct from most mainstream web server frameworks (and certainly most Python frameworks) because it is non-blocking and reasonably fast. Because it is non-blocking and uses epoll, it can handle thousands of simultaneous standing connections, which means it is ideal for real-time web services. We built the web server specifically to handle FriendFeed's real-time features — every active user of FriendFeed maintains an open connection to the FriendFeed servers. (For more information on scaling servers to support thousands of clients, see The C10K problem.)
I developed a board game app (for android/iphone) and added a multiplayer component to it. Was thinking of using app engine (java) for the backend, so implemented that way, but turns out the cpu costs are probably going to be too high (app engine is probably not the best choice for this sort of app).
I don't have much experience with php's performance, am wondering if anyone can give me an idea if I'm heading down a bad path translating it to php. Basic steps would be:
Http request is made to my server, player is trying to perform a move (it's a turn-based game).
Fetch game state from database. This will probably be stored as a flat json string (there's not much point to creating a schema for the gamestate).
Deserialize json string into workable gamestate objects (like houses, cars, etc).
Perform gamestate manipulations finally (like moving the player's car to a new house etc).
Serialize modified gamestate, persist back to database.
Send json response back to player, which will contain updated gamestate.
I don't know if that's a lot of work to try doing in the context of a single http operation. Some numbers:
Expecting about 500 players using the service at any given time.
Each game consists of only 2 to 8 players, each game is completely independent.
The json gamestate object is about 25k in size when serialized.
The json gamestate object will have to deserialize about 300 objects to restore the complete state of the game (houses, cars, etc).
Is this unrealistic? The best solution I think would be to get a dedicated server, and just leave all the gamestates in memory, but I'm trying to get by with some of the shared hosting platforms for now. This means I have to keep going back and forth to the database to get gamestate, deserializing and serializing.
Thanks for any thoughts
As already mentioned, any serious hosting company will easily be able to cope with the traffic. I just wanted to mention a couple points :
Are you sure you want to send the entire game state back and forth each time? You should be able to get away with sending only what changed... This of course depends a lot on how your game works.
The only other issue you might have to consider is lag : a server request can easily take a second or more for the whole encode/send/wait/receive/decode process. Now if your game is turn-based this probably isn't a major hindrance, but it's always something to keep in mind (also think of people who are on a slow connection, ie no 3G coverage).
In any case, you're looking pretty good right now. If I were you, I'd go and write a few tests to see how it goes. Other than that, good luck! ;-)
This sounds ok. You may want to consider using a NoSQL database such as MongoDB if you are just going to be storing json, but this may not be available on a shared host.
i am pretty certain a shared host would cope with that traffic, in my experience most of the websites on shared hosting get hardly any traffic anyway
you can use memcache or something like that as you said, there are plenty of cheap vps options out there if you want to go down that track.
i would get it running first and get 500 people playing, then when it becomes a problem then deal with it. no point fighting a fire that hasn't started yet