I've written a php file that changes a MySql table entry when it receives an http post. I would also like the php file to send out a notification to the table entry's owner. This idea is similar to a chat room or instant messengering program. I've looked at php chat scripts but I really need something that has a very simple interface that is customizable. Can anyone point me in the right direction?
So you want to synchronize a set of clients, do you?
If so, look at the Long Polling technique. It's quite simple: The client opens a connection but the server does not respond until data is updated.
On the downside this won't work well with PHP. You will need to sleep() several connections, therefore blocking PHP processes.
If you have the possibility I would recommend using node.js to do stuff like that. Long Polling Chats are quire simple to implement using node ;)
I would use a named look for event triggerning and a jabber bot (extensions exist for several languages).
http://www.xaprb.com/blog/2007/08/29/how-to-notify-event-listeners-in-mysql/
Related
I am a programmer at an internet marketing company that primaraly makes tools. These tools have certian requirements:
They run in a browser and must work in all of them.
The user either uploads something (.csv) to process or they provide a URL and API calls are made to retrieve information about it.
They are moving around THOUSANDS of lines of data (think large databases). These tools literally run for hours, usually over night.
The user must be able to watch live as their information is processed and is presented to them.
Currently we are writing in PHP, MySQL and Ajax.
My question is how do I process LARGE quantities of data and provide a user experience as the tool is running. Currently I use a custom queue system that sends ajax calls and inserts rows into tables or data into divs.
This method is a huge pain in the ass and couldnt possibly be the correct method. Should I be using a templating system or is there a better way to refresh chunks of the page with A LOT of data. And I really mean a lot of data because we come close to maxing out PHP memory and is something we are always on the look for.
Also I would love to make it so these tools could run on the server by themselves. I mean upload a .csv and close the browser window and then have an email sent to the user when the tool is done.
Does anyone have any methods (programming standards) for me that are better than using .ajax calls? Thank you.
I wanted to update with some notes incase anyone has the same question. I am looking into the following to see which is the best solution:
SlickGrid / DataTables
GearMan
Web Socket
Ratchet
Node.js
These are in no particular order and the one I choose will be based on what works for my issue and what can be used by the rest of my department. I will update when I pick the golden framework.
First of all, you cannot handle big data via Ajax. To make users able to watch the processes live you can do this using web sockets. As you are experienced in PHP, I can suggest you Ratchet which is quite new.
On the other hand, to make calculations and store big data I would use NoSQL instead of MySQL
Since you're kind of pinched for time already, migrating to Node.js may not be time sensitive. It'll also help with the question of notifying users of when the results are ready as it can do browser notification push without polling. As it makes use of Javascript you might find some of your client-side code is reusable.
I think you can run what you need in the background with some kind of Queue manager. I use something similar with CakePHP and it lets me run time intensive processes in the background asynchronously, so the browser does not need to be open.
Another plus side for this is that it's scalable, as it's easy to increase the number of queue workers running.
Basically with PHP, you just need a cron job that runs every once in a while that starts a worker that checks a Queue database for pending tasks. If none are found it keeps running in a loop until one shows up.
I am looking to create a Web Chat system using PHP, MySQL and JavaScript.
Currently, I am storing messages in a MySQL database with an incremental ID (Yes, it is indexed), a timestamp, the sender, and the message itself. I am then using AJAX to query the database every 500ms, seeing if there are any more recent messages than the last one received. However, I have a feeling that this is probably horribly inefficient as it will result in a large load on the MySQL server when multiple users are online. Having looked around a bit on Google and on here, everything seems to point to this way of doing it.
My question: is there a better way to do this? Any tips on how to reduce the load on the server would also be welcome.
I'm using PHP 5.3, on an Apache webserver, so libraries or plugins compatible with those would be fine.
EDIT:
Forgot to mention in the original post, but I'm not worried about supporting IE or other outdated browsers.
Potentially viable basic approach:
Cache your 50 most recent messages in memcache. Reset this whenever a new entry is added to the database. When new users connect, serve them these 50 messages to populate their chatroom.
Use a third party service like http://www.pubnub.com/ to send messages to your clients. Whenever a new message is sent to your chatroom, send it out on pubnub. Your server code will do this after writing to your database successfully.
notes: I'm not affiliated with pubnub. You don't need to use 50 messages above either. You don't even have to give them any messages when they connect depending on how you want to set it up. The point is that you want to avoid your users reading from your database in this case - that model isn't likely to scale for this type of application.
Ideally, an evented environment would be ideal for this kind of app. The LAMP stack is not particularly well suited.
I would recommend using this library, Pubnub. Pubnub is an easy way to send radio signals via javascript, or any TCP language (such as PHP) - and javascript instantly recieves the sent messages.
In PHP, you could simply have it save to your database - then use Pubnub's PHP API's to send the message to everyone else on the page.
If your familiar with Html, Javascript, and PHP - it can be fairly easy to learn. I would recommend it.
You are asking about a web chat system specifically built in PHP, MySQL and HTML with JavaScript. There are many options including Pre-built solutions: http://www.cometchat.com/ and http://www.arrowchat.com/ which all have chat comet services powered by a cloud offering like http://www.pubnub.com/ with options to host it yourself. See more about CometServices http://www.cometchat.com/cometservice/third-party-alternatives where you compare the service providers. There are several more options, however I recommend starting there. If you needs something more simple, like HTML and JavaScript only solution, you can check out http://www.pubnub.com/blog/build-real-time-web-apps-easy which is a blog about building real-time web apps easy with an example chat app in 10 lines of JavaScript Code. The solution Cuts Development Time by providing full Cross Platform for all browsers and mobile devices.
You should look into ajax long polling, in a nutshell this a simple ajax call but will not return a result from the server if there is no new data. You just do a simple loop on the server side until new data will be available then return it. Of course you have to stop this eventually if there's no result to send to client after a while (eg. 1 minute) then restart the call.
I suppose, that chat is too intensive for storage engines MySQL. Maybe, MEMORY table type will be ok, never used it. I spoken to several developers and everybody agree, that best option for chat is Memcache or even writing your own custom daemon (with memory-only storage as weel).
For client part you may read about short-polling, long-poling and web-sockets / sockets via flash/Java object.
using AJAX to query the database every 500ms
Is short-polling.
Sockets are a better solution than AJAX polling, however isn't much around about how you can integrate socket based chats with MySQL.
I have done a few tests and have a basic example working here: https://github.com/andrefigueira/PHP-MySQL-Sockets-Chat
It makes use of Ratchet (http://socketo.me/) for the creation of the chat server in PHP.
And you can send chat messages to the DB by sending the server JSON with the information of who is chatting, (if of course you have user sessions)
Assume that I have a simple VPS setup with LAMP (so with PHP and MySQL on the same server and no other strings attached). And assume that I want to make a self-written ajax chat client on my website.
Obviously, each participant in the conversation would have to listen constantly for new things being said. Since it is very well possible that two or more participants say something in the very same second (and refreshing more than once per second would likely cause insane system load), it seems to me that I would need to store for each participant a list of things that happened since the last refresh.
Which would be the "best" way to do this (in terms of system load)? In the following, an "event" just 'any participant saying anything in the chat'. Clearly, this could be used for a more general as well.
(A) Use MySQL, connecting to the db every second and asking for events WHERE participant_id = $participant_id? (and then deleting all of these so they're only fetched once)
(B) Create a file $participant_id.php and append the events to it (in PHP format so that it can be included, and then empty or delete the file at the next refresh?
(C) Does anyone know any other useful alternatives?
An alternative would be to use a socket connection. Each person connected to the socket server daemon would be able to send a message to the daemon, the daemon would then send the message out to all or a partial list of subscribers which makes chat instantaneous with no need to save the data at all.
A good way to create socket connections from a client is socket IO. See below.
http://socket.io/
A good technology to use for creating a socket server daemon is node.js. This is a server side event driven javascript based library. Very efficient for things like this. See below.
http://nodejs.org/
On both A and B you are still effectively polling. You will either poll MySQL which really isn't too bad, or you can get notified on select() of a file change BUT you will still need to parse to see if the new data is the right stuff on the file-side.
For conceptual and support ease-of-use, it is really hard to beat a database as you won't have to worry about locking semantics. Debugging and message tracking are clean in this structure.
I however recommend you investigate the msg_send() and msg_receive() (of PHP) functions to put this data into an underlying message queue. Your problem seems to be a message queueing problem that should be solved by that mechanism.
Does anyone know any other useful alternatives?
If you search simple solutions on PHP, I can offer 2 ways:
Cache
It mean that you keep MySQL for store data, but install APC (this solution is simplest and fastest for small servers and applications) or Memcached (better for using width several servers). For each read-request you check APC/Memcached for you data and ask MySQL only if your cache is removed or updated. And on each write-request you inserting data in MySQL and update cache.
Other DB
In this case you change MySQL for one of memory-base DB (for example MongoDB). And you may not afraid hard disk usage.
I'm using PHP, AJAX, a MySQL database, and a lot of jQuery to prototype web-based chat system (similar to Facebook Chat). I'm stuck on how to "listen" for incoming chats... when to know someone is trying to chat me... and to know that it is a new chat, not an existing chat.
Right now, I'm polling to see if there has been new insertions in the database tables but it seems very inefficient... a lot of overhead for the server.
Is there a way to receive a notification when, for example, a row has been inserted in a table in a MySQL database so that instead of having constantly poll, I can just be notified and then go look at what as inserted?
If there is a better and more efficient way to create this one-on-one chat relationship, please give me some suggestions.
Thanks, Hristo
You have to use polling, but you can use a technique called Comet which involves long-polling, i.e. sending out an ajax request that will be held by the server until a chat request comes in.
http://en.wikipedia.org/wiki/Comet_(programming))
I think polling is the only way for JavaScript to be pushed server side changes.
After searching the web for a good Comet and also and asking you guys what my best option is, I've chose to go with Orbited. The problem is if you need a good documentation about Comet you won't find. I've installed Orbited and It seems It works just fine.
Basically, I want to constantly check a database and see if there is a new data. If there is, I want to push it to my clients and update their home page but I can't find any good and clear doc explaining how constantly check the database and push the new info to Orbited and then to the clients. Have you guys implemented that?
Also, how many users can Orbited handle?
Any ideas?
You could add a database trigger that sends messages to your message queue when the database got changed. This is also suggested here. Or, if it is only your app talking to the database, you could handle this from within the app via a Subject/Observer pattern, notifying the queue whenever someone called an action changing something in the DB.
I don't know how good or bad Orbited scales.
Have a reference table that keeps track of the last updated time of the source table. Create a update/delete/insert trigger for the source table that updates the time in the reference table.
Your comet script should keep checking the reference table for any change in the time. If the change is noticed, you can read the updated source table and push the data to your client's home page. Checking the reference table in a loop is faster because the MySQL will serve the results from its cache if nothing has changed.
And sorry, I don't know much about Orbited.
I would use the STOMP protocol with Orbited to communicate and push data to clients. Just find a good STOMP client with PHP and get started.
Here is an example of a use case with STOMP, although the server side is written in Ruby:
http://fuglyatblogging.wordpress.com/2008/10/
I don't know if PHP with Apache (if that's what you are using) is the best suite for monitoring database changes. Read this article, under the section title "Orbited Server", for an explanation: http://thingsilearned.com/2009/06/09/starting-out-with-comet-orbited-part-1/
EDIT: If you want to go the route with PHP through a web server, you need to make one, and one only, request to a script that starts the monitoring and pushes out changes. And if that script times out or fails, you need to start a new one. A bit fugly :) A nicer, cleaner way would be, for example, to use twisted with python to start a monitoring process, completely separated from the web-server.