Good time of the day, dear developers!
I am not any kind of network programming pro, but it happened that I have faced necessity to develop socket-server on php (no way for using Java) for flash multiplayer browser-game (standard features like locations, team battles, etc).
The main problem is that TCP is point-to-point protocol and it completely occupies given port.
Of course it is possible to create some kind of queue, which will manage connections to the socket, but this solution doesn't seems to be the fittest one.
It seems to me that using interval of "fair" ports (from 2000 to 2200, for example) is more fitting solution, because one request may take a lot of time to execute and players won't be happy to wait in queue.
But how can I implement this "port inteval" strategy?
The solutions that I see are:
launcing php-script per every port (he-he, 2 hundreds of launched scripts!);
somehow forking the initial process to new processes (2 hundreds of processed? Not nice too), one for every port;
additional while-loop, which listens all the ports (looks very bad);
using threading or something like that (the problem is that php is single-threaded, as far as I know; pcntl?).
But somehow I don't like any of them, or at least don't know how to implement them in the best possible way.
What is the best existing strategy to handle multiple requests from multiple users per time unit without delays, and how to implement this strategy in php? We have our own Debian-server, so it is possible to use any required php extensions.
Any advice about development, planning and implementation of such kind of systems is highlhy appreciated.
Thank You!
Edit 1:
By the way, I've forgotten to mention some extra details.
For example, if we are trying to develop chat application, we need some sort of fixed (I mean persistent) connections for each user. For example, we have 80 users in chat, and then one of them posted a message, which server tries to handle and send to all other connected users, also putting an entry to the history file or something like that.
In this situation polling server for new messages every 10 seconds from each of 80 users is craziness, so the need in persistent connection gets obvious.
But I don't know what is the best way to implement something like this, considering that not all requests are handled instanlty.
have you looked at http://www.smartfoxserver.com/ or http://www.electro-server.com/ ?
Best way for multi user communication for Flash is RTMP. Look for FMS or Red5. It's not in php, but I think this is correct way.
Related
I'm working on a chat application which I would love to use a SQL db for.
My problem is, after a few google searches, i have people telling me from one site, that using a DB would be much slower then using a normal file (e.g Text or JSON file), but then on some other sites, people are saying the complete opposite. And I don't know about you guys, but when it comes to creating web apps for users, the users always come first.
So as much as I'd love to use a SQL DB as 1.) I have good experience with it and 2.) it allows me to make the application much more cooler (more features). but if it would slow things down on the users end (a noticeable lag), then its a no-no.
Either way, I will be "polling" the server continuously with AJAX and PHP to check the file/DB (for new messages, contact requests, ect ect).
Also, incase your wondering, the application wont be like a 1-to-1 chat, it will have "rooms" where multiple users can join and talk with all users joining in. The users will also be able to request a "private chat" with another user, where a 1-to-1 connection opens up.
So, MySQL Database OR a boring TEXT/JSON/OTHER file, in regards to performance?
Oh, one more thing, I don't want to use any third party libraries or APIs. Hate relying on other peoples work (been let down to many times).
If you're looking to implement an IRC clone, I think you've chosen all the wrong tools.
The best way to do this would be to write a custom HTTP server that handles everything in memory. No databases, no constant polling of files. When a message arrives, you simply loop through the correct in-memory list and dispatch the message to other users. For the browser to server connection, I suggest "Comet" (with web sockets for browsers that support them, if you're feeling up to it).
PHP likely isn't the language of choice for this, because pretty much all work done with PHP is based on traditional short, isolated requests. For a long-running process which serves multiple clients in real time, I'd suggest something like Python or Node.js.
You don't really want to be storing chats in files, that can create a management nightmare, I would recommend you go with MySQL and to make sure it works probably go with Sockets instead of AJAX polling, Sockets will scale really well.
However there isn't much around about how you can integrate socket based chats with MySQL.
I have done a few tests and have a basic example working here: https://github.com/andrefigueira/PHP-MySQL-Sockets-Chat
It makes use of Ratchet (http://socketo.me/) for the creation of the chat server in PHP.
And you can send chat messages to the DB by sending the server JSON with the information of who is chatting, (if of course you have user sessions)
I would like to create a chess multiplayer game,1player vs 1player .
I will use flash for animations and client and a server-side language for checking if the moves are correct, record each player action to a database, decide winner and give some credits to the winner.
My main programming language is PHP and I will have someone do the flash work depending on my directions.
I would like to know if anybody tried to do this before , or if you have any tutorial, guide, steps to follow when developing p2p games ? I want to do this efficiently and to be able to support up to at least 100 players on a VDS with 512 RAM .
I am planning to have the game work like this :
1) Initialise variables like credits, player names inside the flash client , after retrieving them from php
2) Start the game
3) If I am player2 and player1 is thinking of a decision, the flash client will send requests to the PHP file to check the database if player1 made any decision. If so, it will be my turn to do a decision and move a piece
4) When my turn comes , no requests are sent to the PHP anymore, until I want to move a piece . When I try to do that ,it sends a request and the PHP will check if my move is valid, and return some variables that flash will use to show error or move pointer back to player1
The game will also have a timer that will sync with PHP. Each player can think 20 seconds to do a move.
1)Any suggestions for my implementation ?What im looking for are suggestions to improve efficiency on this and reduce server load and if someone ever had any experience with a similar project, how did they do it, or how would they do it ?
2)My implementation of periodical HTTP requests isnt such a good idea, because I know that the server can become overloaded, I'd like to know if someone can suggest any alternative but secure option . 100 users at the same time would mean about 100 HTTP requests/second. Is that a number that a VDS can handle for days ?
Later Edit :
After the replies that I got on this post , I came to the conclusion that using sockets instead of HTTP requests every second would be the best idea to use when waiting for a move on the opposite player.
My game would have 100 tables available for players, with 1 socket for communication and 1 port for each table . So 1 socket with 100 ports. PHP will communicate to flash the socket and port for the table , when the game starts and the initialisation variables are sent.
I plan to use flash to listen to the socket server and see what happens when the opposite player must move .
When I move , flash will check if the move is valid from it's side, then send data to PHP . PHP will return OK or NOT and will also update the data on the socket port.
About the socket , Im not sure javascript would be a good option for this, so Im thinking about PHP sockets or JAVA sockets.
PHP sockets seem to be my first option , due to familiarity with the language.
My 2nd would be to rent a smartfoxserver and my 3rd would be to use the services of Player.IO.
Im planing to create a very nice game and sell it. The graphics are pretty much done and they look great. The hardest part is doing this efficiently and easy to install after.
Thank you for your time ! Regards
The server load for a game like this should be really minimal. If you're using flash, I would take advantage of the socket object and just open a socket to your server, that way you don't need to pester it with questions about what has changed... it will notify the clients when/if something noteworthy has.
Otherwise your plan is a pretty good one, so, you'll probably have to ask some specific questions about things that you're stumped by.
If you REALLY want to reduce server load, write the chess logic in the flash client and let IT decide if an attempted move (by the player) is illegal, or if the player has lost or whatnot... use the server strictly for message relay and data storage.
It's sometimes a mistake to design a project around what you have available, such as PHP knowledge and access to a server with 512 MB of RAM. There are a lot of projects where you should consider your resources first, but an independent, first time game project isn't one of them.
Even though your primary language is PHP, you shouldn't restrict yourself from working on the Flash portion yourself. Also, using a web server and database as a communication method for a multiplayer game will result in some very clunky behavior.
You aren't limited by your VDS either. You can use a micro instance on Amazon EC2 for free for a year. If setting up an EC2 server from scratch seems daunting, there are tutorials all over the internet that take you from start to finish.
The front end of your game can be written in Javascript or Flash AS3. There are a lot of game development libraries available for both. Flixel, Flashpunk, and PushButton are free AS3 game libraries. MelonJS is a Javascript/HTML5 game library. All of them have extensive documentation and a good community.
You can write the server backend using node.js and use socket.io to handle your game communications, even if you don't know anything about packets -- or you could use something like Player.IO if you want to pay a little money and have an outside service worry about that for you.
If you write the backend properly, one hundred users won't be a problem for your server. You'll need to start thinking about new hardware when you get in the thousands.
And since you're looking for general game making advice, start reading articles on blogs like #AltDevBlogADay. You'll pick up a lot of insight on design from other people passionate about games.
Assume that I have a simple VPS setup with LAMP (so with PHP and MySQL on the same server and no other strings attached). And assume that I want to make a self-written ajax chat client on my website.
Obviously, each participant in the conversation would have to listen constantly for new things being said. Since it is very well possible that two or more participants say something in the very same second (and refreshing more than once per second would likely cause insane system load), it seems to me that I would need to store for each participant a list of things that happened since the last refresh.
Which would be the "best" way to do this (in terms of system load)? In the following, an "event" just 'any participant saying anything in the chat'. Clearly, this could be used for a more general as well.
(A) Use MySQL, connecting to the db every second and asking for events WHERE participant_id = $participant_id? (and then deleting all of these so they're only fetched once)
(B) Create a file $participant_id.php and append the events to it (in PHP format so that it can be included, and then empty or delete the file at the next refresh?
(C) Does anyone know any other useful alternatives?
An alternative would be to use a socket connection. Each person connected to the socket server daemon would be able to send a message to the daemon, the daemon would then send the message out to all or a partial list of subscribers which makes chat instantaneous with no need to save the data at all.
A good way to create socket connections from a client is socket IO. See below.
http://socket.io/
A good technology to use for creating a socket server daemon is node.js. This is a server side event driven javascript based library. Very efficient for things like this. See below.
http://nodejs.org/
On both A and B you are still effectively polling. You will either poll MySQL which really isn't too bad, or you can get notified on select() of a file change BUT you will still need to parse to see if the new data is the right stuff on the file-side.
For conceptual and support ease-of-use, it is really hard to beat a database as you won't have to worry about locking semantics. Debugging and message tracking are clean in this structure.
I however recommend you investigate the msg_send() and msg_receive() (of PHP) functions to put this data into an underlying message queue. Your problem seems to be a message queueing problem that should be solved by that mechanism.
Does anyone know any other useful alternatives?
If you search simple solutions on PHP, I can offer 2 ways:
Cache
It mean that you keep MySQL for store data, but install APC (this solution is simplest and fastest for small servers and applications) or Memcached (better for using width several servers). For each read-request you check APC/Memcached for you data and ask MySQL only if your cache is removed or updated. And on each write-request you inserting data in MySQL and update cache.
Other DB
In this case you change MySQL for one of memory-base DB (for example MongoDB). And you may not afraid hard disk usage.
Why is HTTP-Polling so laggy?
What I have is a button, and whenever a user clicks it a MySQL database field gets updated and the value is displayed to the user. I'm polling every 800 milliseconds and it's very laggy/glitchy. Sometimes when clicking the button it doesn't register it. And I actually need to be polling quite a bit more frequent than every 800 milliseconds.
This is also with just 1 user on the website at a time... When in the end there is going to be many at once.
HTTP-streaming/Long-polling/Websockets instead of polling
When you need real-time information you should avoid polling(frequently). Below I would try to explain why this is wrong. You could compare it to a child in the back of your car screaming every second "are we there yet" while you are replying "we are not there yet" all the time.
Instead you would like to have something like long-polling/HTTP-streaming or websockets. You could compare this to a child in the back of your car telling you to let him know when "we are there" instead of asking us every second. You could imagine this is way more efficient then the previous example.
To be honest I don't think PHP is the right tool for this kind of applications(yet). Some options you have available are:
hosted solutions:
http://pusherapp.com:
Pusher is a hosted API for quickly,
easily and securely adding scalable
realtime functionality via WebSockets
to web and mobile apps.
Our free Sandbox plan includes up to
20 connections and 100,000 messages
per day. Simply upgrade to a paid plan
when you're ready.
http://beaconpush.com/
Beaconpush is a push service for
creating real-time web apps using
HTML5 WebSockets and Comet.
host yourself:
http://socket.io:
Socket.IO aims to make realtime apps
possible in every browser and mobile
device, blurring the differences
between the different transport
mechanisms
When becoming very big the "host yourself" solution is going to less expensive, but on the other hand using something like pusherapp will get you started easier(friendly API) and also is not that expensive. For example pusherapp's "Bootstrap" can have 100 concurrent connections and 200,000 messages per day for $19 per month(but when small beaconpush is cheaper => do the math :)). As a side-note this plan does not include SSL so can not be used for sensitive data. I guess having a dedicated machine(VPS) will cost you about the same amount of money(for a simple website) and you will also have to manage the streaming solution yourself, but when getting bigger this is probably way more attractive.
Memory instead of Disc
whenever a user clicks it a MySQL
database field gets updated and the
value is displayed to the user
When comparing disc I/O(MySQL in standard mode) to memory it is extremely slow. You should be using an in-memory database like for example redis(also has persistent snapshots) or memcached(completely in memory) to speed up the process. I myself really like redis for it's insane speed, simplicity and persistent snapshots. http://redistogo.com/ offers a free plan with 5MB of memory which will probably cover your needs. If not the mini plan of $5 a month will probably cover you, but when getting even bigger a VPS will be cheaper and in my opinion the prefered solution.
Best solution
The best solution(especially if you are getting big) is to host socket.io/redis yourself using a VPS(cost money). If really small I would use redistogo, if not I would host it myself. I would also start using something like beaconpush/pusherapp because of it's simplicity(getting started immediately). Hosting socket.io(advice to play with it on your own machine for when getting big) is pretty simple, but in my opinion more difficult than beaconpush/pusherapp.
Laggy/glitchy? Sounds like a client-side problem. As does the button thing. I'd get your JavaScript in order first.
As for polling, 0.8 sounds a bit time-critical. I don't know about most countries, but here in the third world simple network packets may get delayed for as long a few seconds. (Not to mention connection drops, packet losses and the speed of light.) Is your application ready to cope with all that?
As for an alternate approach, I agree with #Vern in that an interrupt-driven one would be much better. In HTTP terms, it translates to a long-standing HTTP request that does not receive a response until the server has some actual data to send, minimizing delay and bandwidth. (AFAIK) it's an older technique than AJAX, though has been named more recently. Search for "COMET" and you'll end up with both client- and server-side libraries.
there are many things that might cause the lag that you are experiencing. Your server might be able to process the requests fast enough, but if the connection between your client and the server is slow, then you'll see the obvious lag.
The first thing you should try is to ping the server and see what response time you're getting.
Secondly, rather than poll, you might want to consider an interrupt driven approach. This means that only when your server replies, will you send out your next request. This makes sense, so that many clients won't be flooding the server with requests till the point the server cannot cope. This is especially true, then the RTT (Round-Trip-Time) of your request is pretty long.
Hope it helps. Cheers!
A good place to start would be to use a tool like Firebug in Mozilla Firefox that will allow you to watch the requests being sent to the server and look for bottlenecks.
Firebug will break down each part of the request, so you can see if you are having trouble talking to the server or if it is simply taking a long time to come up with a response.
Along with #Vern's answer I would also say that if at all possible I would have the server cache the data ahead of time and then all of the clients will pull from that same cache and not need separate MySQL calls to reach the same data for every update. Then you just have your PHP update the cache whenever the actual DB data changes.
By cache I mean having php write to a file on the sever side, and then clients will simply look at the contents of that one file to see the most updated info. There might be better ways of caching, but being that I have never done this personally before, this is the first solution that popped into my mind.
I wrote a PHP chatscript with JQUERY on the userend pullstyle. It reloaded a pull.php page once a second and retrieved only new chat records from my sql database since the last check.
However I got an email from my host saying I was using too much bandwidth. Bleh. My chat was working just as I wanted it and this happened. I'm scared to do a COMET php chat because I was told it uses a separate process or thread foreach user.
I suppose I just need a strategy that's good, and more efficient than what I had.
Well you've got the premise down. 1 second recalls are far too frequent - can anyone bang out a response in a second and expect it to go through? It takes me at least 2 seconds to get my SO messages together / relatively typo free.
So part 1 - increase the recall time. You kick it up to 10 seconds, the app is still going to feel pretty speedy, but you're decreasing processing by a factor of 10 - that's huge. Remember every message includes extra data (in the HTML and in all the network layers that support it), and every received message by the server requires processing to decide what to do with it.
Part 2 - You might want to introduce a sliding scale of interactivity speed. When the user isn't typing and hasn't received a message in a while the recheck rate could be decreased to perhaps 30 seconds. If the user is receiving more than one message in a "pull" or has received a message for the last 3 consecutive pulls then the speed should increase (perhaps even more frequent than 10 seconds). This will feel speedy to users during conversations, and not slow/laggy when there's nothing going on.
Part 3 - Sounds like you're already doing it, but your pull should be providing the minimum possible reply. That is just the HTML fragment to render the new messages, or perhaps even just a JSON structure that can be rendered by client-side JavaScript. If you're getting all the messages again (bad) or the whole page again (worse) you can seriously decrease your bandwidth this way.
Part 4 - You may want to look at alternate server software. Apache has some serious overhead... and that's for good reason, you've got all your plugs running, apache config, .htaccess config, rewrites, the kitchen sink - you might be able to switch to a lighter weight http server - nginx, lighttp - and realise some serious performance increase (strictly speaking it won't help much with bandwidth - but one wonders if that's what the host is really complaining about, or that your service consumes serious server resources).
Part 5 - You could look at switching host, or package - a horrible thought I'm sure, but there may be packages with more bandwidth/processor (the later being more key I suspect) available... you could even move to your own server which allows you to do what you want without the service provider getting involved.
You don't need to check for a new message every second, kick it down to maybe 3 or 4. Make sure you are only using 1 query and that the file loaded is as small as possible. And I would always add a timeout... it would stop reloading after a minute or 2 of no activity.
Make these changes and see where that puts you.
Post some code if you want to see if we can help.
I believe that comet is the only solution.
Threads are much lighter than processes, so you would want to use a COMET type technology with a light threaded web server like lighthttpd.
Take a look at Simple Chat. It's very tiny and extensible i think.
All you need to do is "move ahead with one of the standard solutions available"....
Comet, Web Socket, XMPP are a few words you will hear every now n then, just pick one of them and it should be more efficient than classic periodic ajax call based php chat.
If you want to go ahead with PHP and jQuery based solution, try these two working examples of browser based chat system using XMPP: http://bit.ly/jaxlBoshChat (for one-to-one chat example) and http://bit.ly/jaxlBoshMUC (for multi-user chat example). Let me know if you face any problem setting them up.