Im starting a new project very soon, and im considering long polling to notify the users that they have a new private message/Notification that they have to check, really similar to how Facebook uses to notify you that someone has posted something about you/Liked a photo of you..
From what I have read, cometd seemed like a really good option to start with.
Then, other ways started emerging, like: Socket.io, and node.js ..
Now, my question is; Which one do you think is the best option for this case and why?
What I need to do exactly is the following;
User 1 logs into their account
User 2 sends User 1 a message which gets stored into the database and a flag is generated, (If possible?!)
The PHP script responsible for User 1 should pick up the flag, and push a notification to User 1
I know how to take care of the javascript side, but I have never done anything similar to long polling.
Im using jQuery as javascript library and PHP for the server side.
So recommendations and any good resources to do this?
It would be beneficial to use a combination of php and also node.js. node.js is made for use with persistant connections, and push communication instead of poll.
http://nodejs.org/
Here is a quick video i found: http://vimeo.com/29099827
Related
Currently I'm making a chat application where only admin and users chat, no user-to-user chat . The design is: every chat is stored in database and each 2 seconds user and admin make an AJAX request (to a php file) to see if there is a new chat dialogue, and if there is, pull the data into the textbox. It all seems normal and working good.
Problem is as more user is talking to admin at the same time the AJAX request is becoming a lot, and by testing, the web performance already decreased with only 5 users chatting at the same time. And the input is slow too, every time user press enter they got to enter the data into database first before the admin can read it (and vice versa).
I have been told that using JSON is a recommended way, but I have no idea how to do it, can someone please at least tell me how's the design or flow is going to be if use JSON? Or is there a better way to make it? (by the way, using node.js is currently impossible for my current hosting, so don't put it in suggestion lists, sucks I know).
You should change the AJAX responder phps output to JSON. (you can use the json_encode php function for example.) And you should parse(eval) this in javascript.
I am a bit sceptic. It think It could reduce the network usage by more than 50%.
Maybe you can try a message queue, like 0mq or rabbitmq.
There are a lot of chat examples around.
I am a programmer at an internet marketing company that primaraly makes tools. These tools have certian requirements:
They run in a browser and must work in all of them.
The user either uploads something (.csv) to process or they provide a URL and API calls are made to retrieve information about it.
They are moving around THOUSANDS of lines of data (think large databases). These tools literally run for hours, usually over night.
The user must be able to watch live as their information is processed and is presented to them.
Currently we are writing in PHP, MySQL and Ajax.
My question is how do I process LARGE quantities of data and provide a user experience as the tool is running. Currently I use a custom queue system that sends ajax calls and inserts rows into tables or data into divs.
This method is a huge pain in the ass and couldnt possibly be the correct method. Should I be using a templating system or is there a better way to refresh chunks of the page with A LOT of data. And I really mean a lot of data because we come close to maxing out PHP memory and is something we are always on the look for.
Also I would love to make it so these tools could run on the server by themselves. I mean upload a .csv and close the browser window and then have an email sent to the user when the tool is done.
Does anyone have any methods (programming standards) for me that are better than using .ajax calls? Thank you.
I wanted to update with some notes incase anyone has the same question. I am looking into the following to see which is the best solution:
SlickGrid / DataTables
GearMan
Web Socket
Ratchet
Node.js
These are in no particular order and the one I choose will be based on what works for my issue and what can be used by the rest of my department. I will update when I pick the golden framework.
First of all, you cannot handle big data via Ajax. To make users able to watch the processes live you can do this using web sockets. As you are experienced in PHP, I can suggest you Ratchet which is quite new.
On the other hand, to make calculations and store big data I would use NoSQL instead of MySQL
Since you're kind of pinched for time already, migrating to Node.js may not be time sensitive. It'll also help with the question of notifying users of when the results are ready as it can do browser notification push without polling. As it makes use of Javascript you might find some of your client-side code is reusable.
I think you can run what you need in the background with some kind of Queue manager. I use something similar with CakePHP and it lets me run time intensive processes in the background asynchronously, so the browser does not need to be open.
Another plus side for this is that it's scalable, as it's easy to increase the number of queue workers running.
Basically with PHP, you just need a cron job that runs every once in a while that starts a worker that checks a Queue database for pending tasks. If none are found it keeps running in a loop until one shows up.
I am a PHP developer and the title basically says it all. However I was hoping on some more in-depth information as I am starting to get confused about how the flow for the project I work on should go.
For an (web) application I need to implement a feature like Facebook does it with notifying users about replies/comments and instantly showing these.
I figured I could use long-polling with ajax requests but this does not seem to be a nice solution as the notifications never really are instant and it is resource heavy.
So I should use some form of sockets if I understand correctly, and Node.Js would be a good choice. So based on the last assumption I now get confused about the work flow.
I thought about two possible solutions:
1) It seems to me, that if I would use Node.Js I could skip using PHP at all and base the application on Node.js only.
2) Or I could use PHP as a base and only use Node.js for notifying users and instantly showing messages but saving the data using PHP and Mysql.
These two possibilities confuse me and I can't make up my mind about what would be the "best" and cleanest way.
I do not have much experience in Node.js, played with it for a while. But managing and saving data seems to be hard in Node.js so that is why I came up with option 2.
I know Facebook is build on PHP so I am assuming that they save the data via PHP and notify / instantly show replies and comments via Node.
Could someone help me out on this?
Thanks in advance!
EDIT:
I just noticed, Stackoverflow does something similar. I get a notification in the upper left, and below my question a box with "new answer to this question". I am really interested in the technologie(s) used.
Well you could use node.js for the notifications and PHP for your app.
By googling I found this about real-time-notifications.
You could also just use node.js with socket.io, but this means that you have to learn new technologies as you mention that you have no experience with node.
I haven't used it but you could check this project, for websockets in PHP.
When you have an update that you want to notify users you can use the publish subscriber pattern to notify the intrested in this update.
Take a look in Gearman too.
Personally, I've built a notification system using the pubsub mechanism of redis, with node.js+socket.io. Everytime that there is an update on a record then there is a publish on the appropriate channel. If the channel has listeners then they will be notified. I also store the last 20 notifications in a Redis list.
The appplication is built in PHP. The notification system is built in node.js. They are different applications that see the same data. The communication occurs via redis. For example in the Facebook context:
1) A user updates his status.
2) PHP stores this to the database and Redis
3) Redis knows that this update must publish to the status channel of the specific user and it does.
4) All the friends of the specific user are listening to his status channel (here comes node.js)
5) Node.js pushes the notification in the browser with socket.io
As for facebook, I have read in an article that is using long polling for supporting older browsers. Not sure for this though, needs citation...
AFAIK It would be via two simple methods :
First one that could be very simple is adding a Boolean column to each record that determines if it has been notified or not.
The second method is creating a table to insert all notifications.
However, I'm not sure if there are alternative methods for better performance, But first method is what I do commonly myself. But I think Facebook is using 2nd method, because it has to notify each one to a lot of users.
Your question maybe dublicate of:
Facebook like notifications tracking (DB Design)
Database design to store notifications to users
You could use Server Side Events it involves a bit of JavaScript but nothing overly complicated I think.
The main bulk of this method is PHP though, so you would just use the PHP to query your DB for notifications and SSE will push them to the user.
It does have some limitations though, most notably it's not supported by IE (huge surprise) thought i'd mention it anyway to let you know of other possibilities.
Hope this helps
I am looking to create a Web Chat system using PHP, MySQL and JavaScript.
Currently, I am storing messages in a MySQL database with an incremental ID (Yes, it is indexed), a timestamp, the sender, and the message itself. I am then using AJAX to query the database every 500ms, seeing if there are any more recent messages than the last one received. However, I have a feeling that this is probably horribly inefficient as it will result in a large load on the MySQL server when multiple users are online. Having looked around a bit on Google and on here, everything seems to point to this way of doing it.
My question: is there a better way to do this? Any tips on how to reduce the load on the server would also be welcome.
I'm using PHP 5.3, on an Apache webserver, so libraries or plugins compatible with those would be fine.
EDIT:
Forgot to mention in the original post, but I'm not worried about supporting IE or other outdated browsers.
Potentially viable basic approach:
Cache your 50 most recent messages in memcache. Reset this whenever a new entry is added to the database. When new users connect, serve them these 50 messages to populate their chatroom.
Use a third party service like http://www.pubnub.com/ to send messages to your clients. Whenever a new message is sent to your chatroom, send it out on pubnub. Your server code will do this after writing to your database successfully.
notes: I'm not affiliated with pubnub. You don't need to use 50 messages above either. You don't even have to give them any messages when they connect depending on how you want to set it up. The point is that you want to avoid your users reading from your database in this case - that model isn't likely to scale for this type of application.
Ideally, an evented environment would be ideal for this kind of app. The LAMP stack is not particularly well suited.
I would recommend using this library, Pubnub. Pubnub is an easy way to send radio signals via javascript, or any TCP language (such as PHP) - and javascript instantly recieves the sent messages.
In PHP, you could simply have it save to your database - then use Pubnub's PHP API's to send the message to everyone else on the page.
If your familiar with Html, Javascript, and PHP - it can be fairly easy to learn. I would recommend it.
You are asking about a web chat system specifically built in PHP, MySQL and HTML with JavaScript. There are many options including Pre-built solutions: http://www.cometchat.com/ and http://www.arrowchat.com/ which all have chat comet services powered by a cloud offering like http://www.pubnub.com/ with options to host it yourself. See more about CometServices http://www.cometchat.com/cometservice/third-party-alternatives where you compare the service providers. There are several more options, however I recommend starting there. If you needs something more simple, like HTML and JavaScript only solution, you can check out http://www.pubnub.com/blog/build-real-time-web-apps-easy which is a blog about building real-time web apps easy with an example chat app in 10 lines of JavaScript Code. The solution Cuts Development Time by providing full Cross Platform for all browsers and mobile devices.
You should look into ajax long polling, in a nutshell this a simple ajax call but will not return a result from the server if there is no new data. You just do a simple loop on the server side until new data will be available then return it. Of course you have to stop this eventually if there's no result to send to client after a while (eg. 1 minute) then restart the call.
I suppose, that chat is too intensive for storage engines MySQL. Maybe, MEMORY table type will be ok, never used it. I spoken to several developers and everybody agree, that best option for chat is Memcache or even writing your own custom daemon (with memory-only storage as weel).
For client part you may read about short-polling, long-poling and web-sockets / sockets via flash/Java object.
using AJAX to query the database every 500ms
Is short-polling.
Sockets are a better solution than AJAX polling, however isn't much around about how you can integrate socket based chats with MySQL.
I have done a few tests and have a basic example working here: https://github.com/andrefigueira/PHP-MySQL-Sockets-Chat
It makes use of Ratchet (http://socketo.me/) for the creation of the chat server in PHP.
And you can send chat messages to the DB by sending the server JSON with the information of who is chatting, (if of course you have user sessions)
I'm here to ask you if what I think is the right way to go around coding this.
I have a site that receives private messages and I wish a flag to show up the moment the person receives a message. Should I check for new messages every 3 seconds and show the flag if there is a new message or is there a better way?
If I did it in ajax, I was thinking check every 3-5 seconds for new messages, and once there's a flag, stop checking for more.
My only concern is, if it checks every 3-5 second, will it cause any lag or glitchyness for the person when they're typing? Lets say they're typing out a paragraph somewhere, I don't want their writing to glitch while it checks those 3-5 second intervals.
One of my coder friends mentioned there is a method with Ping(?) or something like that. Where the person is always connected to the server and when there's a change it notifies the user. I'm totally unsure of how this works.
Anyone know how facebook does it? haha.
Thank you!
If you have done the AJAX well, it should not lag/glitch while typing. Something like 3-5s is good as its fast enough but won't slow down server/browser.
Did he mean "push"? In push the messages are pushed to client in realtime, client is not asking if there is new messages. This is most likely the method Facebook is using.
One of my coder friends mentioned there is a method with Ping(?) or
something like that
To be honest I really don't like periodic refresh(polling at intervals), because tt has scaling problems(I got notice from hosting provider when using periodic refresh). You should use more efficient transports like for example:
WebSocket
Adobe® Flash® Socket
AJAX long polling
AJAX multipart streaming
Forever Iframe
JSONP Polling(cross domain)
To use this you could for example use:
hosted solution pusher with API and generous free plan. This gives you max 20 concurrent and 100,000 messages per day, but no SSL, so do not transmit sensitive information over the wire. They also provide third-party PHP client available at github implementing REST API.
socket.io (I like this a lot)
tornado
netty
Anyone know how facebook does it? haha.
For chat they use Erlang. They also have open-sourced tornado(see link above) which they required from friendfeed which they acquired in the past. Facebook is a PHP-shop, but they decided to not use PHP for this, because PHP can not yet do this efficiently. Anyway they are using one of the efficients transports above.