I am in the middle of making a social network, and i want it to be as smooth as facebook.
Like if you look in a console and look at logging, it doesn't update all the time with ajax calls.
on my site i have to load: notifications(the number of new notifs and the notifs themselves), friend requests(same as notifications), online friends(if there are any online it will load the pictures of the online users.) thats 6 ajax calls that is loaded every 10 second. and this causes a huge bandwidth waste and server requests.
Therefore i thought, what if the SERVER told the CLIENT when there was a new update instead of the CLIENT asking the SERVER every 10 seconds.
i have googled this problem and read about ajax push, and a framework called comet.
i just can't seem to find any info on how to implement this on jQuery.
I looked briefly into Comet. It appears to be ambitious, experimental and won't run on just any old server.
As I understand it, Comet doesn't really push as such but does something called "long polling", which I won't try to describe here. The web already has several good texts on the subject.
Personally, I would stick with the current plan (conventional AJAX) but make one general purpose call with all the necessary data bundled into an object and JSON encoded. This will reduce 6 requests down to one (every 10 seconds).
You can box-clever by returning nulls within the returned object for information that hasn't changed thereby minimising the length of each response.
As far as I know, you must make significant modifications on your webserver to get this thing to work. Also, server side php is not really a good option.
Somebody had already asked something similar here: Using comet with PHP?
You can try socket.io on node.js too. It works great for real time communication
http://socket.io/
Related
I am a programmer at an internet marketing company that primaraly makes tools. These tools have certian requirements:
They run in a browser and must work in all of them.
The user either uploads something (.csv) to process or they provide a URL and API calls are made to retrieve information about it.
They are moving around THOUSANDS of lines of data (think large databases). These tools literally run for hours, usually over night.
The user must be able to watch live as their information is processed and is presented to them.
Currently we are writing in PHP, MySQL and Ajax.
My question is how do I process LARGE quantities of data and provide a user experience as the tool is running. Currently I use a custom queue system that sends ajax calls and inserts rows into tables or data into divs.
This method is a huge pain in the ass and couldnt possibly be the correct method. Should I be using a templating system or is there a better way to refresh chunks of the page with A LOT of data. And I really mean a lot of data because we come close to maxing out PHP memory and is something we are always on the look for.
Also I would love to make it so these tools could run on the server by themselves. I mean upload a .csv and close the browser window and then have an email sent to the user when the tool is done.
Does anyone have any methods (programming standards) for me that are better than using .ajax calls? Thank you.
I wanted to update with some notes incase anyone has the same question. I am looking into the following to see which is the best solution:
SlickGrid / DataTables
GearMan
Web Socket
Ratchet
Node.js
These are in no particular order and the one I choose will be based on what works for my issue and what can be used by the rest of my department. I will update when I pick the golden framework.
First of all, you cannot handle big data via Ajax. To make users able to watch the processes live you can do this using web sockets. As you are experienced in PHP, I can suggest you Ratchet which is quite new.
On the other hand, to make calculations and store big data I would use NoSQL instead of MySQL
Since you're kind of pinched for time already, migrating to Node.js may not be time sensitive. It'll also help with the question of notifying users of when the results are ready as it can do browser notification push without polling. As it makes use of Javascript you might find some of your client-side code is reusable.
I think you can run what you need in the background with some kind of Queue manager. I use something similar with CakePHP and it lets me run time intensive processes in the background asynchronously, so the browser does not need to be open.
Another plus side for this is that it's scalable, as it's easy to increase the number of queue workers running.
Basically with PHP, you just need a cron job that runs every once in a while that starts a worker that checks a Queue database for pending tasks. If none are found it keeps running in a loop until one shows up.
I'm trying to figure out a way for users of a website (say a student and teacher) to share a secure connection where real time updates on one page are viewed by both of them.
From research I've concluded that some of the real time updates could be performed using ajax and javascript.
But I'm stumped as to how users could share a connection where only the two users would be viewing the updates that take place on the website (such as flash animations of a drawing board.) I'm also confused how you would even begin to set up a connection like this.
I've looked intp php sessions and cookies but I'm not sure I'm doing the right research.
Any pointers as to how two specific users could share a secure connection where real time updates are viewed by the both of them only. I don't want a terse response please. I'm looking for specific details like functions and syntax specific to php. I appreciate the help and will rate you up if you give me good answers!
You cannot share a secure connection (e.g. HTTPS) its one client to one server.
If both clients are logged in and have a background AJAX task running in the browser, is it acceptable to have each client "pull" every few seconds the same data to display for both users?
This would require the "drawing board" updates to also be sent continuously back to the server to share the updated data with the other client. I'm sure there will be an event you can use to trigger the post of data (e.g. on mouse up).
If performance is an issue, you'd want to use a better server technology like Java that is able to keep session state between requests without having to persist to a database.
You can look at ajax push techniques. I used comet once where an administrator posted messages and everybody who was logged in saw that message appear on their screen. I don't know if comet supports PHP. I only used it with JSP. Just search for "ajax push" in Google.
Flash allows for connections between users, I think they refer to them as sockets.
If you want to use Ajax, et al, you need a server side technology that supports push.
Node is the standard in this regard, and you can set up a Heroku instance for free.
There are others, and you need to learn tools before even beginning to learn application.
Among the many overviews, this might interest you:
http://arstechnica.com/business/2012/05/say-hello-to-the-real-real-time-web/?1
A few good examples where this is happening:
Google Docs
Etherpad
HTML5 Games: Multi player
Techniques you can use (with varying browser support)
HTML5 WebSockets (Wikipedia; MDN; HTML5 Demo)
Comet (Wikipedia)
Really pushing data to a web browser client from a server (which would do that when it receives something from another client) is only possible with WebSockets as far as I know. Other mechanism would either require browser plugins or a stand-alone application.
However with Comet (through AJAX) you can get really close to pushing data by polling the server periodically for data. However contrary to traditional polling (e.g. where a clients asks for data every 5 seconds), with the Comet principle the server will hold that periodic request hostage for, say, up to 30 seconds. The server will not send back data until either it has data or the time out is reached. That way, during those 30 seconds, any data that the server receives can be instantly pushed back to the other clients. And right after that the client starts a new 30 second session, and so forth.
Although both Comet and WebSockets should work with a PHP backend served by Apache. I'd recommend looking into NodeJS (as server technology) for this.
There is a lot of information regarding Comet on the internet, I suggest you google it, maybe start at Wikipedia.
The great thing about Comet is that it is more a principle than a technology. It uses what we already have (simple HTTP requests with AJAX), so browser support is very wide.
You could also do a combination, where you use sockets if supported and fallback to Comet.
I'm sure you have looked into this. The opinion that this can be done via ajax is misleading to believe that two users of a website can communicate via javascript.
As you are aware, javascript happens on the client and ajax is essentially 'talking to the server without a page change or refresh'.
The communication between two users of the website has to happen via the server - php and some chosen datastore.
Hope that wasn't terse.
cheers, Rob
I'm here to ask you if what I think is the right way to go around coding this.
I have a site that receives private messages and I wish a flag to show up the moment the person receives a message. Should I check for new messages every 3 seconds and show the flag if there is a new message or is there a better way?
If I did it in ajax, I was thinking check every 3-5 seconds for new messages, and once there's a flag, stop checking for more.
My only concern is, if it checks every 3-5 second, will it cause any lag or glitchyness for the person when they're typing? Lets say they're typing out a paragraph somewhere, I don't want their writing to glitch while it checks those 3-5 second intervals.
One of my coder friends mentioned there is a method with Ping(?) or something like that. Where the person is always connected to the server and when there's a change it notifies the user. I'm totally unsure of how this works.
Anyone know how facebook does it? haha.
Thank you!
If you have done the AJAX well, it should not lag/glitch while typing. Something like 3-5s is good as its fast enough but won't slow down server/browser.
Did he mean "push"? In push the messages are pushed to client in realtime, client is not asking if there is new messages. This is most likely the method Facebook is using.
One of my coder friends mentioned there is a method with Ping(?) or
something like that
To be honest I really don't like periodic refresh(polling at intervals), because tt has scaling problems(I got notice from hosting provider when using periodic refresh). You should use more efficient transports like for example:
WebSocket
Adobe® Flash® Socket
AJAX long polling
AJAX multipart streaming
Forever Iframe
JSONP Polling(cross domain)
To use this you could for example use:
hosted solution pusher with API and generous free plan. This gives you max 20 concurrent and 100,000 messages per day, but no SSL, so do not transmit sensitive information over the wire. They also provide third-party PHP client available at github implementing REST API.
socket.io (I like this a lot)
tornado
netty
Anyone know how facebook does it? haha.
For chat they use Erlang. They also have open-sourced tornado(see link above) which they required from friendfeed which they acquired in the past. Facebook is a PHP-shop, but they decided to not use PHP for this, because PHP can not yet do this efficiently. Anyway they are using one of the efficients transports above.
What way will be best to write online chat with js? If i would use AJAX and update information about users and messages every 5sec - HTTP requests and answers will make big traffic and requests will make high server load.
But how another? Sockets? But how..
You seem to have a problem with the server load, so I'll compare the relevant technologies.
Ajax polling:
This is the most straightforward. You do setTimeout loop every 5 seconds or so often to check for new chat messages or you set an iframe to reload. When you post a message, you also return new messages, and things shouldn't get out of order. The biggest drawback with this method is that you're unlikely to poll with a frequency corresponding to how often messages are posted. Either you'll poll too quickly, and you'll make a lot of extra requests, or you'll poll too slowly and you'll get chunks of messages at a time instead of getting them in a real-time-ish way. This is by far the easiest method though.
HTTP Push
This is the idea that the server should tell the client when there are new messages, rather than the client continually bothering the server asking if there are any new ones yet. Imagine the parent driving and kid asking "are we there yet?", you can just have the parent tell the kid when they get there.
There are a couple ways to both fake this and do it for real. WebSockets, which you mentioned, are actually creating a stream between the client and the server and sending data in real time. This is awesome, and for the 4 out of 10 users that have a browser that can do it, they'll be pretty psyched. Everyone else will have a broken page. Sorry. Maybe in a couple years.
You can also fake push tech with things like long-polling. The idea is that you ask the server if there are any new messages, and the server doesn't answer until a new message has appeared or some preset limit (30 seconds or so) has been reached. This keeps the number of requests to a minimum, while using known web technologies, so most browsers will work with it. You'll have a high connection concurrency, but they really aren't doing anything, so it should have too high a server cost.
I've used all of these before, but ended up going with long polling myself. You can find out more about how to actually do it here: How do I implement basic "Long Polling"?
You should chose sockets rather than AJAX Polling, However there isn't much around about how you can integrate socket based chats with MySQL.
I have done a few tests and have a basic example working here: https://github.com/andrefigueira/PHP-MySQL-Sockets-Chat
It makes use of Ratchet (http://socketo.me/) for the creation of the chat server in PHP.
And you can send chat messages to the DB by sending the server JSON with the information of who is chatting, (if of course you have user sessions)
There are a few ways that give the messages to the client immediately:
HTML5 Websockets
good because you can use them like real sockets
bad because only a few browsers support it
Endlessly-loading frame
good because every browser supports it
not so cool because you have to do AJAX requests for sending stuff
you can send commands to the client by embedding <script> tags in the content
the script gets executed immediately, too!
...
So, in conclusion, I would choose the second way.
The typical approach is to use long polling. Though better not do this in PHP (PHP will need one process for each connection thus drastically limiting the number of possible visitors to your site). Instead use node.js. It's perfect for chats.
I am currently working on a light php framework to use with some high request ajax for my site, and have run into an interesting problem that has me completely stumped. The ajax is for a series of notifications, so the javascript sends off an ajax request for new information every 30 seconds. This ajax is active on every page of the entire site, so I realize its a lot of requests as several hundred users are browsing the site at any given moment, many with several windows open.
Syntax wise everything is perfect. The problem is, when I activated the ajax for my community, there is a build up of 30 - 40 SLEEP commands in the MySQL database. All of which seem to ignore the set timeout of 10 seconds. It effects the entire site's performance as a result.
My understanding is that left over sleep commands are the result of a connection that hasn't been closed. I am use the MySQLi object oriented class and have extended it with a few functions of my own. Everything is by the text book and double checked against the documentation, and I've made sure that every little piece of the MySQLi class is closed and released.
If anyone has any wisdom or experience with this type of effect, I would be most grateful for any advice.
Have you tried a push approach versus your current pull/polling approach? Take a look at Comet:
Comet a new approach to AJAX applications
Comet (wikipedia)
Comet programming: Using AJAX to simulate server push
Ajax Push Engine
However, there are some downsides, the main one being you have to maintain an open connection, and some browsers limit the number of connections you can maintain. The following article talks about the pros and cons of using Comet and suggests a hybrid-polling approach:
The allure of Comet
Here is a stackoverflow question that talks about comet also:
Compatibility of Comet with current technology