I'm trying to find a efficient way to watch the server log on a webpage, i don't mind building an app i just can't work out the best way to do it.
Is there a way to keep a stream open to a file with php and to the browser? or will it have to be done by polling the file every x seconds?
Thanks in advance,
Shadi
The best solution is definitely AJAX in some capacity. The only way to have the server "push" to you the way you describe (maintain an open stream) would require the HTTP connection to remain open which would ultimately trigger timeouts and consume a lot of resources. I would look into the Cometd library. The downside to this is that I believe it depends on Java although the site does mention perl, python and "other languages." In the worst case, you could use a specific jetty implementation just for log monitoring on a specific port. Regardless, that framework would most likely be your best bet.
Any web-based chat mechanism essentially uses a push architecture and would be good to look at for some inspiration. In this case, instead of users creating messages that are fired to other users, the server creates the events (when a log message is generated). Check out this article on Facebook chat for some insight into how they do it. Google chat might be worth looking into if you can find some stuff on the architecture.
For the actual logging, I'm not sure if you are in need of help for that, but log4php which is currently under incubation might be a good place to start as it provides you with a configuration that can simultaneously log to an arbitrary number of "loggers" like database, file, socket, etc. You could likely find one that would allow you to tie it into whatever push framework you elect to use.
Good luck!
Remember that the web model is essentially stateless (disconnected). Having that in mind when a client submits a request, the server processes the request and then send a response accordingly. You can have track of the clients action using cookies and/or sessions, but the resources reserved for a request are released after the response is submitted back.
I think that the best way to meet your goal, is to develop a web services that checks for the status of the log and fetch the diff (if any). Your app may consist of a web page with a div that will display the diff from the web service.
A script with a timer will trigger the call to the web service.
I will try to do something like this in a few weeks, and I will post the entire solution on moropo blog (spanish). You can ask for a post translation using the comments.
The best way to do it is to use AJAX to pull the file content every x seconds, giving the illusion of real time.
If you do want real time, you can use an XMPP server, but from what I can see, the first solution is far sufficient and does't require a lot of work.
Try wonlog.
https://www.npmjs.com/package/wonlog
You can stream multiple log files to a web browser.
Related
I am writing a JavaScript for an in-browser IM client for the sake of practicing and learning JavaScript and AJAX.
I need to be able to check for a change in the file size of a text file that is being used as a temporary storage for 40-80 SQL entries that contain messages so that it can update the display.
At the moment I am using a setInterval function to periodically check for a change in file size using short PHP script, but this can cause issues, if the interval is to long, messages are delayed, if it is shorter, it means a lot of php scripts running very quickly, which takes up server resources.
What is the best way to do this if the main concern is to reduce server resource usage?
(I am running my server off of a rather low tech PC I've scraped together(2gb ram, 2.8ghz AMD seperon processor))
Preferably, I would want to do this using an AJAX event triggered by someone sending a message, I.E. When user B triggers the event that edits the file by pressing enter, that triggers a function on user A's side that updates the HTML file
Any ideas? I am open to any solution to this particular problem. I gave specific examples of what I want to happen in the specific languages in order to give a better idea of what it is I am attempting to do.
If there is a way to do this that isn't JavaScript/PHP, I'd also be open to exploring that as an option.
Doing this with PHP can be a bit cumbersome. You could try doing something like long polling where you keep the HTTP request open until the server has new data to send to the user. If messages are sent frequently, this might not be ideal. You might want to consider using event-driven web technologies like node.js with something like Socket.IO.
In any case, you'll likely want to maintain a connection with the server if you want to get the message in near real-time. There are ways to use WebSockets with PHP as well, but PHP isn't really the best for this because it's not designed to keep scripts running for long periods (also see What exactly entails setting up a PHP Websocket Server?).
Browsers & HTTP/ AJAX generally work by a "pull" model. The browser/ or AJAX sends the server a request, then the server answers a response.
There isn't generally much provision for the server to contact the browser, to "push" an event. This can however be simulated by a long-running request, to which the server writes data when the event/ or events occur.
For example, this could be a request that answers "empty" after a timeout of 10-30 seconds.. or the server returns & answers immediately, if there are event(s) in its queue.
With a Java server this is easy to do, and I've used this successfully for event notification in a major integration project a few years back.
However I'm not sure in PHP how much ability there is (probably very near zero) to maintain an overall server state, coordinate or communicate between threads/requests, or maintain event queues.
You could look into something like a Java webapp running on Tomcat. All you need is a basic web.xml and one Servlet class, and you can build just about anything from there.
I'm working on feeding the client realtime events using event-streams and HTML5 SSEs client-side.
But some of my events will actually come from form submissions by other clients.
What's the best method for detecting these form submissions (so as to append them to the event-stream script) ASAP (after they occur)?
So essentially, I need realtime cross-script messaging between multiple instances of different scripts instantiated by different clients, analagous to X-doc messaging in JS, but for PHP.
The best I can come up with is to repeatedly poll a subdir of /tmp for notification files, which is a terrible solution.
Often you can use MYSQL to play the role of the tmp dir you were talking about. This is more portable because they don't have to be on the same server to do this and the data is separate. However the scripts will have to manually check the mysql location to see if the other one has taken care of this. The other option is to open sockets and write back and forth in real time or to use some prebuilt tool for just this purpose which I'm pretty sure might exist.
If you want the events to be triggered near to realtime, then you need to handle them synchronously - which means running a daemon. And the simplest way to implement a daemon which can synchronize data across client connections is to use an event based server. There's a nice implementation of the latter using php here - there are plenty of examples of how to daemonize a PHP process on the interent. Then just open a blocking connection to the server from your PHP code / access this via comet.
I have been in search of making live websites by using PHP. (COMET) I have been searching for a very long time already. (around a month) I have even checked some PHP chat scripts and used on my webserver, but I had some problems on most of them. (will be explained)
So far, most of the people tells the same thing: PHP isn't the best language could be used with COMET. The chat/comet examples are hacky at best.
I am asking this because, I want to have some features on my websites, like allow logged in people to chat with each other. That is where I need an alive connection to PHP pages. I am also planning on making a browser based game, and an alive connection will still be a must!
AJAX was a rare thing 2 years ago, shined with Facebook. Now pretty much everyone uses it, and it became a standard on web development. Now, the COMET based websites are growing. Youtube, Google+, Facebook, EA's Battlelog and such. I believe I should learn how to use it.
Okay, here are my questions. (Some of the information at below are the ones I gathered while searching on Google. Not sure if they're correct or not.)
Some languages like Python have special web servers designed for this job. I believe one of them is called Tornado Web Server. Developed and configured to simulate thousands of alive connections. I believe there is no such option in Appserv, but some people told NGINX can handle it with a decent configuration. Is it true? What configurations should be made? Is there any PHP web servers for this job?
Some of the most suggested things are:
a. Using APE.
b. Using Socket.IO
c. Node.js
Is there any way we can use them with PHP? Which one is the most promising? Could you give some guidance on them? Is there anything else than these?
I have used a comet chat script. Basically, you kept querying database and output the result with flush() to browser by sleeping it with usleep() in a loop. (This one became hard to understand so I will provide a quick code)
while(true)
{
// query database
// output the result
// flush the browser
// sleep for few seconds to lower cpu load
}
usleep() function basically destroyed my web server on Windows based operating systems. Is it normal to rely on usleep() on comet applications which runs on windows based OS? I mean, is there even a way to "sleep" PHP scripts? No matter what I do, CPU load goes to %100 on both WIN and UNIX servers.
Is PHP "really" that weak on this area? Should I give up with PHP and focus on other languages? If so, which language would you suggest? (That language should be promising. For example, there is no much use of AS3 after HTML5 addition, and AS3 is more likely to die soon while JS shines.)
What is WebSync? Can it be used with PHP?
Please bear in mind that I need to use COMET to make following applications:
A chat script, where logged in players will be able to chat eachother.
A browser based game. I already use JSON/AJAX and things like that when coding, but to receive opponents steps, I should pull the data, so an alive connection is needed. (Don't imagine advanced games, I am speaking about games like chess at best.)
I would appreciate if you can give me a short guidance. After all, I have been getting confused day by day since there are SO MANY random articles on internet. People even call setInterval() as COMET, so it is a complete mess.
There needs to be some clarification here. What you're looking for is not language specific per se. If you wanted to achieve the comet functionality with PHP you'd be relying on the Web Server (e.g Apache) to do the HTTP streaming. Also you need to understand that this is not supported in HTTP versions prior to HTTP/1.1. With that said, if you want to have a non-blocking event based web server, you're better off looking at Tornado and Node as suggested.
Comet is a programming technique that enables web servers to send data to the client without having any need for the client to request it This technique will produce more responsive applications than classic AJAX The user must create a request (for example by clicking on a link) or a periodic AJAX request must happen in order to get new data fro the server.
but it's create lots of traffic on your web server. If you want to build chat application in PHP use pusher which is a third party service and easy to use.
here is a link for pusher https://pusher.com/tutorials/realtime_chat_widget
the second suggestion is use ratchet for creating a chat application.
here is link for ratchet http://socketo.me/docs/hello-world
i hope it will help you
I want to create a live, checkers-like app, which will work like this: There will be multiple icons/avatars displayed on this checkerboard like surface. I want to have a command prompt beneath this board, or some other sort of interface, that will allow them to control a certain avatar, and get it to preform actions. Multiple users will be using it at one time, and I will all be able to view the other user's changes/actions to the checkerboard.
What I'm wondering is: what's the best way to do this? I've got my HTML, CSS, and JS approach down, but not my data storage method. I know that, using PHP, I've got the choices to use either: file-based storage, MYSQL, or some other method. I need to know which is better, because I don't want to have server-lag, poor-response time, or some other issue, especially in this case since actions will be preformed every other second 2 or so, by these multiple users.
I've done similar stuff before, but I'm wanting to hear how others would handle it (advice, etc.) from more experienced programmers.
Sounds like a great project for node.js!
To clarify, node.js is a server-side implementation of javascript. What you'll want is a comet based application (a web-based client application that receives server side pushes instead of the client constantly polling the server), which is exactly what node.js is good at.
Traditional ajax calls for your clients to poll the server for data. This creates enormous overhead for both the client and the server. Allowing the server to push requests directly to the client without the client repeatedly asking solves the overhead issue and creates a more responsive interface. This is accomplished by holding asynchronous client connections on the server and only returning when the server has something to respond with. Once the server responds with data, another connection is immediately created and held by the server again until data is ready to be sent.
You may be able to accomplish the same thing with PHP, but I'm not that familiar with PHP and Comet type applications.
Number of users and hosting costs will play into your file vs DB options. If you're planning on more than a couple of users, I'd stick to the database. There are some NoSQL options available out there, but in my experience MySQL is much faster and more reliable than those options.
Good luck with your project!
http://en.wikipedia.org/wiki/Comet_%28programming%29
http://www.nodejs.org/
http://zenmachine.wordpress.com/2010/01/31/node-js-and-comet/
http://socket.io/ - abstracts away the communication layer with your clients based on their capability (LongPolling, WebSockets, etc.)
MySQL and XCache !!!!
Make sure you use predefined statements so MySQL does not need to compile the SQL again. Also memtables could be used to use memory storage
Of course make use of indexes appropriately.
If the 'gamestate' is not that important you can even store everything in XCache.
Remember that XCache does not store data persistently (after Apache restart)
I am developing a non-real time browser RPG game (think Kingdom of Loathing) which would be played from within a Flash app. At first I just wanted to make the communication with server using simply URLLoader to tell PHP what I am doing, and using $_SESSION to store data needed in-between request.
I wonder if it wouldn't be better to base it on a socket connection, an app residing on a server written in Java or Python. The problem is I have never ever written such an app so I have no idea how much I'd have to "shift" my thoughts from simple responding do request (like PHP) to continuously working application. I won't hide I am also concerned about the memory and CPU usage of such Server app, when for example there would be hundreds of users connected. I've done some research.
I have tried to do some research, but thanks to my nil knowledge on the sockets subject I haven't found anything helpful. So, considering the fact I don't need real time data exchange, will it be wise to develop the server side part as socket server, not in plain ol' PHP?
Since your game isn't something that's working in realtime you probably don't need to go down the socket route, though it's certainly a viable option. The nice thing about sockets is that updates would be instant without requiring page refresh (or server poll), so you're right to at least consider it.
If you do want to do a more real-time server setup, you might consider using something like Electroserver - this abstracts out much of the setup for you so you don't have to write your own server from scratch, plus it's free up to a certain number of concurrent users if I recall correctly.
Finally, a third option you have is a modified POST approach using AMF. Look into AMFPHP, it lets you call methods on a PHP back-end directly from your flash application. A little bit faster and easier than simply using POST stuff, but not quite as seamless as a socket connection or a specifically built gaming server.
Lots of options out there, it sounds like you are aware of this and kudos for trying to come up with the best approach rather than just rolling with what you know! I hope this helps, let me know if you have any questions.
Here's a link to Electroserver - http://www.electro-server.com/