Hey there stackoverflow!
I am developing a community with Laravel3 right now I am trying to achieve some Facebook style private messaging.
I did the coding part, but me and my friends are so agree about adding real-time message notifications. I did my research, I cannot say I find a good article about this, some of them starts with oh you know everything about matrix so lets socketsocketsocketsocketsocket I am so confused how to start, where to start, what is this anyway, many of this say go with MongoDB never use MySQL again. Dude what the heck? I am using MySQL I created a nice private message system in Laravel, I want to add real-time notify!
All I want to do is
UserA sends a message to UserB
Message inserted into privmsg table.
Sockets or whatever you suggest, tells UserB's browser there is +1 new message from UserA
UserB sees there is one unread message without refresh his page and click to read it .
How can I achieve this? Socket.io best choice for that? if so how can I use it? any snippets would be so great! or tutorial about my situation :)
I would really grateful
You suffer from the problem of thinking "X is always better than Y so just always use X". There's probably a name for it... maybe even a book or two written. Who knows. Let's start with your first question:
what is this shit anyway, many of this say "go with MongoDB never use MySQL again." Dude what the heck?
You should stop spending time with whomever said that. MySQL and MongoDB are database systems for two very different types of databases. They are often referred to as table-based and document-based. With MySQL (any many other databases that utilize SQL... and probably some that don't), your data is stored in a set of relational tables outlined by a very specific schema. Each record in this table conforms to a specific set of fields with a specific set of types. This type of database is perfect for many kinds of data.
MongoDB is the document-based variety of databases, commonly referred to as "NoSQL" (meaning not-only-SQL). Each "document" can have a whole structure, complete with nodes that have children and grand children. Each document can have its own distinct set of data. Documents are stored in "collections". This type of database has some advantages... it can be quite fast for certain types of operations. This being said, it is terrible for other things, such as when you have a bunch of data that is all identical in type. Data aggregation is very slow on databases like these (but it is getting better all the time!).
The point I'm trying to make is that MySQL and MongoDB are just different tools, designed for different jobs. Stop pounding a nail with your screwdriver just because your friend tells you that screws are better than nails.
All I want to do is: UserA sends a message to UserB; Message inserted into privmsg table.; Sockets or whatever you suggest, tells UserB's browser there is +1 new message from UserA; UserB sees there is one unread message without refresh his page and click to read it .
Again, pick the tool for the job. Knowing what your tools are is a good start. Socket.IO is meant to set up a communications channel between a server and a client. It provides web-socket-like functionality, commonly used between Node.js servers and web browsers (but can be used in other contexts as well!). Its two main features are that it provides fallback transports when Web Sockets are not available (making it compatible with old browsers), and it wraps up an event messaging system in some nice and simple calls. You don't have to worry about the underlying communication. Just emit and event on one end, and it is fired on the other. Simple.
For the actual communication between your servers and the browser, Socket.IO is an excellent choice. It provides near-real-time communication. However, Socket.IO isn't just some magic that will solve all of your problems for you. It would be useless to almost everyone if it were.
Since your messages need to persist, storing them in a database is a good idea. What I would do:
On message send, insert a copy into the database
On insert, fire a notice over your pub/sub to the other servers in your cluster
Any server that has a connection to the user getting the message will see this notice from the other server.
That server will load the user's message data out of the database and emit it over Socket.IO
You want a tutorial? The example on the Socket.IO home page is quite good: http://socket.io/
Related
I am a PHP developer and the title basically says it all. However I was hoping on some more in-depth information as I am starting to get confused about how the flow for the project I work on should go.
For an (web) application I need to implement a feature like Facebook does it with notifying users about replies/comments and instantly showing these.
I figured I could use long-polling with ajax requests but this does not seem to be a nice solution as the notifications never really are instant and it is resource heavy.
So I should use some form of sockets if I understand correctly, and Node.Js would be a good choice. So based on the last assumption I now get confused about the work flow.
I thought about two possible solutions:
1) It seems to me, that if I would use Node.Js I could skip using PHP at all and base the application on Node.js only.
2) Or I could use PHP as a base and only use Node.js for notifying users and instantly showing messages but saving the data using PHP and Mysql.
These two possibilities confuse me and I can't make up my mind about what would be the "best" and cleanest way.
I do not have much experience in Node.js, played with it for a while. But managing and saving data seems to be hard in Node.js so that is why I came up with option 2.
I know Facebook is build on PHP so I am assuming that they save the data via PHP and notify / instantly show replies and comments via Node.
Could someone help me out on this?
Thanks in advance!
EDIT:
I just noticed, Stackoverflow does something similar. I get a notification in the upper left, and below my question a box with "new answer to this question". I am really interested in the technologie(s) used.
Well you could use node.js for the notifications and PHP for your app.
By googling I found this about real-time-notifications.
You could also just use node.js with socket.io, but this means that you have to learn new technologies as you mention that you have no experience with node.
I haven't used it but you could check this project, for websockets in PHP.
When you have an update that you want to notify users you can use the publish subscriber pattern to notify the intrested in this update.
Take a look in Gearman too.
Personally, I've built a notification system using the pubsub mechanism of redis, with node.js+socket.io. Everytime that there is an update on a record then there is a publish on the appropriate channel. If the channel has listeners then they will be notified. I also store the last 20 notifications in a Redis list.
The appplication is built in PHP. The notification system is built in node.js. They are different applications that see the same data. The communication occurs via redis. For example in the Facebook context:
1) A user updates his status.
2) PHP stores this to the database and Redis
3) Redis knows that this update must publish to the status channel of the specific user and it does.
4) All the friends of the specific user are listening to his status channel (here comes node.js)
5) Node.js pushes the notification in the browser with socket.io
As for facebook, I have read in an article that is using long polling for supporting older browsers. Not sure for this though, needs citation...
AFAIK It would be via two simple methods :
First one that could be very simple is adding a Boolean column to each record that determines if it has been notified or not.
The second method is creating a table to insert all notifications.
However, I'm not sure if there are alternative methods for better performance, But first method is what I do commonly myself. But I think Facebook is using 2nd method, because it has to notify each one to a lot of users.
Your question maybe dublicate of:
Facebook like notifications tracking (DB Design)
Database design to store notifications to users
You could use Server Side Events it involves a bit of JavaScript but nothing overly complicated I think.
The main bulk of this method is PHP though, so you would just use the PHP to query your DB for notifications and SSE will push them to the user.
It does have some limitations though, most notably it's not supported by IE (huge surprise) thought i'd mention it anyway to let you know of other possibilities.
Hope this helps
I'm building a chat widget using pusher service (http://pusher.com) and I need to save all the messages sent by users into database, so that it should be accessible after a period of time. I'm using mysql database and the only way that comes to my mind is that make a new insert each time the chat message event is triggered, but I'm afraid it will not be as fast as it should be.
What databases and techniques would you prefer in this case for saving chat messages?
One approach you can take is to use MySQL's INSERT DELAYED. When you do this, the client does not have to wait for the insert to complete. Rather, the server will queue the inserts and execute them as it can. So the process/thread thats handling the chat will not have to wait on them. But you'll still have a chat history stored in the database that can be retrieved. Assuming you are using pusher to share 'live' messages as they come in, and thus dont need immediate access to the chat history, this might do the trick for you.
Unless you're planning on creating a chat system including thousands of people I'm fairly sure that a properly structured mysql solution will handle this no problem. Where I work we have a intranet chat solution based on php and mysql and it is running flawlessly (for our medium sized team of ~100 some people).
One suggestion I do have is to make sure you understand mysql indexing. Make sure your chat system is truly utilizing the indexes to their fullest potential. You could greatly increase performance this way.
If concurrency really is becoming an issue for you I hear a lot of people are having good experiences with node.js. But if you're comfortable in php and mysql I'd say go that way first.
Happy coding!
(I know this is old but,) There is an interesting demo here (albeit in Rails) that does what you're talking about. It includes a possible schema.
https://github.com/dipth/pusher_demo
I'm working on a chat application which I would love to use a SQL db for.
My problem is, after a few google searches, i have people telling me from one site, that using a DB would be much slower then using a normal file (e.g Text or JSON file), but then on some other sites, people are saying the complete opposite. And I don't know about you guys, but when it comes to creating web apps for users, the users always come first.
So as much as I'd love to use a SQL DB as 1.) I have good experience with it and 2.) it allows me to make the application much more cooler (more features). but if it would slow things down on the users end (a noticeable lag), then its a no-no.
Either way, I will be "polling" the server continuously with AJAX and PHP to check the file/DB (for new messages, contact requests, ect ect).
Also, incase your wondering, the application wont be like a 1-to-1 chat, it will have "rooms" where multiple users can join and talk with all users joining in. The users will also be able to request a "private chat" with another user, where a 1-to-1 connection opens up.
So, MySQL Database OR a boring TEXT/JSON/OTHER file, in regards to performance?
Oh, one more thing, I don't want to use any third party libraries or APIs. Hate relying on other peoples work (been let down to many times).
If you're looking to implement an IRC clone, I think you've chosen all the wrong tools.
The best way to do this would be to write a custom HTTP server that handles everything in memory. No databases, no constant polling of files. When a message arrives, you simply loop through the correct in-memory list and dispatch the message to other users. For the browser to server connection, I suggest "Comet" (with web sockets for browsers that support them, if you're feeling up to it).
PHP likely isn't the language of choice for this, because pretty much all work done with PHP is based on traditional short, isolated requests. For a long-running process which serves multiple clients in real time, I'd suggest something like Python or Node.js.
You don't really want to be storing chats in files, that can create a management nightmare, I would recommend you go with MySQL and to make sure it works probably go with Sockets instead of AJAX polling, Sockets will scale really well.
However there isn't much around about how you can integrate socket based chats with MySQL.
I have done a few tests and have a basic example working here: https://github.com/andrefigueira/PHP-MySQL-Sockets-Chat
It makes use of Ratchet (http://socketo.me/) for the creation of the chat server in PHP.
And you can send chat messages to the DB by sending the server JSON with the information of who is chatting, (if of course you have user sessions)
Assume that I have a simple VPS setup with LAMP (so with PHP and MySQL on the same server and no other strings attached). And assume that I want to make a self-written ajax chat client on my website.
Obviously, each participant in the conversation would have to listen constantly for new things being said. Since it is very well possible that two or more participants say something in the very same second (and refreshing more than once per second would likely cause insane system load), it seems to me that I would need to store for each participant a list of things that happened since the last refresh.
Which would be the "best" way to do this (in terms of system load)? In the following, an "event" just 'any participant saying anything in the chat'. Clearly, this could be used for a more general as well.
(A) Use MySQL, connecting to the db every second and asking for events WHERE participant_id = $participant_id? (and then deleting all of these so they're only fetched once)
(B) Create a file $participant_id.php and append the events to it (in PHP format so that it can be included, and then empty or delete the file at the next refresh?
(C) Does anyone know any other useful alternatives?
An alternative would be to use a socket connection. Each person connected to the socket server daemon would be able to send a message to the daemon, the daemon would then send the message out to all or a partial list of subscribers which makes chat instantaneous with no need to save the data at all.
A good way to create socket connections from a client is socket IO. See below.
http://socket.io/
A good technology to use for creating a socket server daemon is node.js. This is a server side event driven javascript based library. Very efficient for things like this. See below.
http://nodejs.org/
On both A and B you are still effectively polling. You will either poll MySQL which really isn't too bad, or you can get notified on select() of a file change BUT you will still need to parse to see if the new data is the right stuff on the file-side.
For conceptual and support ease-of-use, it is really hard to beat a database as you won't have to worry about locking semantics. Debugging and message tracking are clean in this structure.
I however recommend you investigate the msg_send() and msg_receive() (of PHP) functions to put this data into an underlying message queue. Your problem seems to be a message queueing problem that should be solved by that mechanism.
Does anyone know any other useful alternatives?
If you search simple solutions on PHP, I can offer 2 ways:
Cache
It mean that you keep MySQL for store data, but install APC (this solution is simplest and fastest for small servers and applications) or Memcached (better for using width several servers). For each read-request you check APC/Memcached for you data and ask MySQL only if your cache is removed or updated. And on each write-request you inserting data in MySQL and update cache.
Other DB
In this case you change MySQL for one of memory-base DB (for example MongoDB). And you may not afraid hard disk usage.
After searching the web for a good Comet and also and asking you guys what my best option is, I've chose to go with Orbited. The problem is if you need a good documentation about Comet you won't find. I've installed Orbited and It seems It works just fine.
Basically, I want to constantly check a database and see if there is a new data. If there is, I want to push it to my clients and update their home page but I can't find any good and clear doc explaining how constantly check the database and push the new info to Orbited and then to the clients. Have you guys implemented that?
Also, how many users can Orbited handle?
Any ideas?
You could add a database trigger that sends messages to your message queue when the database got changed. This is also suggested here. Or, if it is only your app talking to the database, you could handle this from within the app via a Subject/Observer pattern, notifying the queue whenever someone called an action changing something in the DB.
I don't know how good or bad Orbited scales.
Have a reference table that keeps track of the last updated time of the source table. Create a update/delete/insert trigger for the source table that updates the time in the reference table.
Your comet script should keep checking the reference table for any change in the time. If the change is noticed, you can read the updated source table and push the data to your client's home page. Checking the reference table in a loop is faster because the MySQL will serve the results from its cache if nothing has changed.
And sorry, I don't know much about Orbited.
I would use the STOMP protocol with Orbited to communicate and push data to clients. Just find a good STOMP client with PHP and get started.
Here is an example of a use case with STOMP, although the server side is written in Ruby:
http://fuglyatblogging.wordpress.com/2008/10/
I don't know if PHP with Apache (if that's what you are using) is the best suite for monitoring database changes. Read this article, under the section title "Orbited Server", for an explanation: http://thingsilearned.com/2009/06/09/starting-out-with-comet-orbited-part-1/
EDIT: If you want to go the route with PHP through a web server, you need to make one, and one only, request to a script that starts the monitoring and pushes out changes. And if that script times out or fails, you need to start a new one. A bit fugly :) A nicer, cleaner way would be, for example, to use twisted with python to start a monitoring process, completely separated from the web-server.