How can New Relic tap into my app with a simple install? How does it know all the methods, requests, etc?
It works for RoR, PHP, etc.
Can anyone explain the technology behind it? I'm interested in tapping into my Rails app, but I want to do so smoothly like New Relic.
Thanks
First up, you will not manage to duplicate the functionality of NewRelic on your own. Ignoring the server-side, the rpm Gem is a pretty complex piece of software, doing a lot of stuff. Have a look at the source if you want to see how it hooks into the Rails system. The source is a worth a read, as it does some cool stuff in terms of threading and marshaling of the data before sending it back to their servers.
If you want a replacement because Newrelic is expensive (and rightly so, it's awesome at what it does), then have a look at the FreeRelic project on Github.
They are using ASPECT ORIENTED PROGRAMMING CONCEPTS AND Reflection heavily for Intercepting original method call and adding instrumentation around that.
In a general way, New Relic's gem inserts kinda middleware in your web framework, and collects data from your endpoint (think as a rails route) until it's response. After every "harvesting time" (defaults to 60 seconds), it sends a post request to NR services with this data.
You can also tailor data you need with Custom Metrics, Custom Events.
Is also possible to do queries with NRQL and build graphs with that (like you would do in Graphana).
They have a customize service for Wordpress too, but is a bit messy in the start.
Some options if you want to save some money is configure cloudwatch + datadog, but I would give a shot to their service if uptime is crucial for your app.
For a rails solution you could simply implement a more verbose logging level (development/debug level) and interrogate the production.log file for specific events, timings etc
For Java they are attaching a Java agent to JVM which intercepts method calls and monitor them. You can use AspectJ to replicate the same behaviour and log every method call to wherever you want, let's say create custom Cloudwatch metrics.
In case of Java it's bytecode ingestion. They "hacking" the key methods of your application server and add their code in it. Then they send relevant transaction info to their server, aggregating it and you can see the summary. It's really complicated process so I don't think one dev can implement it.
If you’re already familiar with New Relic’s application monitoring
then you probably know about New Relic’s agents that run in-process
on web apps collecting and reporting all sorts of details about whats
happening in the app. RUM leverages the agents to dynamically inject
JavaScript into pages as they are built. The injected JavaScript
collects timing information in the browser and contains details that
identify the specific app and web transaction processed on the
backend as well as how time was spent in the app for each request.
When a page completes loading in an end user’s browser, the
information is sent back to New Relic asynchronously – so it doesn’t
effect page load time.
You can turn RUM on/off via your application settings in New Relic.
As well, you can turn RUM on/off via the agent’s configuration file
(newrelic.yml – a ‘browser_monitoring auto_instrument’ flag has been
introduced).
The agents have been enhanced to automatically inject JavaScript into
the HTML pages so using RUM is as simple as checking the checkbox on
the New Relic control panel. However, if you’d prefer more control,
you can use New Relic’s Agent API to generate the JavaScript and thus
control exactly when and where the header and footer scripts are
included.
Related
I have several PHP libraries (scripts and classes and function files) that I want to make available as a service that is web accessible. I am trying to be as deliberate with the wording as possible since it seems that 'Web Service' is rather nuanced. From what I can tell there are 2 main flavors of Web Service, REST and WSDL/SOAP, with the later seeming to be more applicable to what I want to do, but it seems like a lot of overhead and possibly overkill. Could I simply make a PHP page that accepts a parameter of "function" to indicate what action to take, then echo out the response like normal? Requiring the construction of a SOAP message as part of an AJAX call seems horrible.
What is the difference between a requesting a PHP page and a Web Service response (aside from the SOAP protocol)?
Would you ever return a JSON string in SOAP?
Are the implementations separate, exclusive or in parallel?
Could you, or even want, to use Apache rewrites to accomplish nearly the same effect as REST or WSDL? Directing the request to a page appending a parameter for the requested action.
OR am I over thinking all this and should not worry about SOAP and just got with the standard PHP function parameter and return text or json?
I am also looking ahead a bit, since I work with a lot of legacy code bases, Ruby, Perl, Python, and Java, and would eventually want to make a Service from them as well. Or at least incorporate the libraries somehow.
I am going to recommend this book to you, which is an amazing reference for advanced PHP topics, and is very current. It has a chapter that focuses on networking with PHP, and a specific section on creating your own PHP-based web services. It also contains loads upon loads of other up-to-date kung fu for PHP developers.
http://www.amazon.com/PHP-Advanced-Object-Oriented-Programming-QuickPro/dp/0321832183/
I can tell you what worked for me.
I had to create a small web service in which an outside application needed to get a list of products. I echo'ed a JSON encoded array, while using .htpasswd to protect the data from prying eyes :). The data was accessible very easily with a small CURL script, and it took about 2-3 hours.
If you need the web service users to manage information, if you need an ACL, you will have to look into SOAP and/or REST more. For what I needed - it was more than enough.
I've developed an application that I would like to use meteor.js for real time updates (I want to enhance but not change my program, for example when a user adds a comments make it update in real-time ) . Problem is meteor.js uses node.js (so javascript as server-side code). I use LAMP stack, Is it possible to get PHP to feed data into meteor.js from mysql.
Meteor is more than just an 'interactive webapplication'-builder or javascript framework. The idea is to have only one programming language (besides HTML/CSS for markup) to do all the work. Basically it creates a 'remote server' (in the clients browser) it can push data to and at the same time it publishes various API's to the users system. The data passed through these API's / connections has a specific structure which has to be adhered at all time.
Meteor is built around NodeJS, which makes it hard (if not impossible) to run it without this backend. Sure you can try to mimic the backend using PHP, but it would be a waste of time. Reading your question you'll be better of using a javascript framework like jQuery or Prototype. Unlike Meteor you will need to do the AJAX calls (POST & CallBack) yourself, but you can actually decide which backend you want to use yourself (including PHP / MySQL).
If you want to do this anyway you need to check the Meteor & NodeJS source code to see what the minimum requirements are to make Meteor run under PHP. The PHP stack has to interpret the commands Meteor sends and receivers, but this won't be an easy task.
You can use comet (or reverse ajax) for realtime updates.
Trying to marry node.js with PHP doesn't sound like a worthwhile path to go down. If someone insisted on using a system like Meteor.js, yet with a PHP back-end, it would make more sense to look at AngularJS which is mainly the client side.
Of course, that is different technology stack. If someone really insisted on the blending, one could consider using server side sockets to interact with PHP Web services; and/or use mongodb and/or mysql-node to interact with the same databases.
I released a meteorite package that interacts with a Wordpress site that has the Wordpress JSON API. A quick fix. For now.
Comes with a backend call that will return the raw data, or a publication that stores the posts using their id's instead of a randomly generated mongoid. And some basic templates to get you started including a Session variable that keeps track of the currently selected post.
I'm working on it a lot more and will eventually have a version that directly makes mysql calls from node so you won't need php or Wordpress; just the ability to access the mysql database (which can be remote, with the appropriate configuration, or on the same machine).
I am developing a non-real time browser RPG game (think Kingdom of Loathing) which would be played from within a Flash app. At first I just wanted to make the communication with server using simply URLLoader to tell PHP what I am doing, and using $_SESSION to store data needed in-between request.
I wonder if it wouldn't be better to base it on a socket connection, an app residing on a server written in Java or Python. The problem is I have never ever written such an app so I have no idea how much I'd have to "shift" my thoughts from simple responding do request (like PHP) to continuously working application. I won't hide I am also concerned about the memory and CPU usage of such Server app, when for example there would be hundreds of users connected. I've done some research.
I have tried to do some research, but thanks to my nil knowledge on the sockets subject I haven't found anything helpful. So, considering the fact I don't need real time data exchange, will it be wise to develop the server side part as socket server, not in plain ol' PHP?
Since your game isn't something that's working in realtime you probably don't need to go down the socket route, though it's certainly a viable option. The nice thing about sockets is that updates would be instant without requiring page refresh (or server poll), so you're right to at least consider it.
If you do want to do a more real-time server setup, you might consider using something like Electroserver - this abstracts out much of the setup for you so you don't have to write your own server from scratch, plus it's free up to a certain number of concurrent users if I recall correctly.
Finally, a third option you have is a modified POST approach using AMF. Look into AMFPHP, it lets you call methods on a PHP back-end directly from your flash application. A little bit faster and easier than simply using POST stuff, but not quite as seamless as a socket connection or a specifically built gaming server.
Lots of options out there, it sounds like you are aware of this and kudos for trying to come up with the best approach rather than just rolling with what you know! I hope this helps, let me know if you have any questions.
Here's a link to Electroserver - http://www.electro-server.com/
I'm trying to find a efficient way to watch the server log on a webpage, i don't mind building an app i just can't work out the best way to do it.
Is there a way to keep a stream open to a file with php and to the browser? or will it have to be done by polling the file every x seconds?
Thanks in advance,
Shadi
The best solution is definitely AJAX in some capacity. The only way to have the server "push" to you the way you describe (maintain an open stream) would require the HTTP connection to remain open which would ultimately trigger timeouts and consume a lot of resources. I would look into the Cometd library. The downside to this is that I believe it depends on Java although the site does mention perl, python and "other languages." In the worst case, you could use a specific jetty implementation just for log monitoring on a specific port. Regardless, that framework would most likely be your best bet.
Any web-based chat mechanism essentially uses a push architecture and would be good to look at for some inspiration. In this case, instead of users creating messages that are fired to other users, the server creates the events (when a log message is generated). Check out this article on Facebook chat for some insight into how they do it. Google chat might be worth looking into if you can find some stuff on the architecture.
For the actual logging, I'm not sure if you are in need of help for that, but log4php which is currently under incubation might be a good place to start as it provides you with a configuration that can simultaneously log to an arbitrary number of "loggers" like database, file, socket, etc. You could likely find one that would allow you to tie it into whatever push framework you elect to use.
Good luck!
Remember that the web model is essentially stateless (disconnected). Having that in mind when a client submits a request, the server processes the request and then send a response accordingly. You can have track of the clients action using cookies and/or sessions, but the resources reserved for a request are released after the response is submitted back.
I think that the best way to meet your goal, is to develop a web services that checks for the status of the log and fetch the diff (if any). Your app may consist of a web page with a div that will display the diff from the web service.
A script with a timer will trigger the call to the web service.
I will try to do something like this in a few weeks, and I will post the entire solution on moropo blog (spanish). You can ask for a post translation using the comments.
The best way to do it is to use AJAX to pull the file content every x seconds, giving the illusion of real time.
If you do want real time, you can use an XMPP server, but from what I can see, the first solution is far sufficient and does't require a lot of work.
Try wonlog.
https://www.npmjs.com/package/wonlog
You can stream multiple log files to a web browser.
I need to start a PHP script via Apache - which will basically continue to run - it will be used for a small instant messaging widget - so it must continually run to update other users on new messages. What timeout features or other barriers should I be weary of in Apache? Will some feature of Apache not allow me to run a PHP process continually?
Based on your context (instant messaging widget), you probably do not want to keep the PHP script running because:
anything on the page after the widget will not be loaded;
you will have to flush the output, otherwise user will not be able to see anything.
There are many other ways to implement instant messaging, like AJAX. I suggest you do a search on this topic.
Update: I have recently come to the realization that there is a particular name for this type of web application model, which may be a better reference than simply stating "AJAX". It's called Comet (wikipedia link).
set_time_limit will help you.