I have been working on a site that makes some pretty big use of AJAX and dynamic JavaScript on the front end and it's time to start stress testing. But how do you properly stress test something that requires clicking several links on the front-end? One way I was able to easily hit every page of the site quickly and repeatedly was to point a Google Mini at it. But that's not going to click links and then navigate Modal windows and things like that.
Edit - I should point out that the site is done in PHP5 and the JavaScript library used is jQuery. Not sure if this would make any difference but felt it might be useful to know.
JMeter is great at this. You may record your sessions and tweak them to your liking.
So-called 'ajax load testing' is a recurring subject on this site, and is often confused. So let's get it straight: There is really no difference between load testing a normal web page and load testing with ajax. It all boils down to discrete requests; they just happen to not be full page refreshes.
One thing to keep in mind is there is a distinct difference between load testing the server processing the requests (a load test) and the performance on screen of the UI components being updated (how well your javascript performs.)
Simple load test example:
initial page load
login
navigate?
5-10 'ajax' requests (or whatever may fit your application usage pattern)
logout
There are load testing tools that can support AJAX. For example, WebLoad
http://www.radview.com/solutions/ajax-load-testing.aspx
What you really want is to stress test is the server's ability to handle the ajax requests. Use a load tool that looks at the requests while "recording" the test, and then tune as appropriate. I have only used the vs test edition one, so I can't point you to a low cost one.
I disagree with Nathan and Freddy to some degree. They are correct that "AJAX testing" is really no different in that HTTP requests are made. But it's not that simple. See my article on Ajaxian.com on Why Load Testing Ajax is Hard.
JMeter, Pylot, and The Grinder are all great tools for generating HTTP requests (I personally recommend Pylot). But at their core, they don't act as a browser and process JavaScript, meaning all they do is replay the traffic they saw at record time. If those AJAX requests were unique to that session, they may not be suitable/correct to replay in large volumes.
The fact is that as more logic is pushed down in to the browser, it becomes much more difficult (if not impossible) to properly simulate the traffic using traditional load testing tools.
In my article, I give a simple example of how difficult it becomes to test something like Google's home page when you want to query 1000's of different search terms (an important goal during load testing). To do it with JMeter/Pylot/Grinder you effectively end up re-writing parts of the AJAX code (in your case w/ jQuery) over again in the native language of the tool.
It gets even more complex if your goal is to measure the response time as perceived by the user (which is arguably the most important thing at the end of the day). For really complex applications that use Comet/"Reverse Ajax" (a technique that keeps open sockets for long periods of time), traditional load tools don't work at all.
My company, BrowserMob, provides a load testing service that uses Firefox browsers, powered by Selenium, to drive hundreds or thousands of real browsers, allowing you to measure and time the performance of visual elements as seen in the browser. We also support traditional virtual users (blind HTTP traffic) and a simulated browser (via HtmlUnit).
All that said, usually a mix of a service like BrowserMob plus traditional load testing is the right approach. That is, real browsers are great for a full-fidelity load test, but they will never be as economical as "virtual users", since they require 10-100X more RAM and CPU. See my recent blog post on whether to simulate or not to simulate virtual users.
Hope that helps!
You could use something like openSTA.
This allows a session with a web site to be recorded and then played back via a relatively simple script language.
You can also easily test web services and write your own scripts.
It allows you to put scripts together in a test in any way you want and configure the number of iterations, the number of users in each iteration, the ramp up time to introduce each new user and the delay between each iteration. Tests can also be scheduled in the future.
It's open source and free.
It produces a number of reports which can be saved to a spreadsheet. We then use a pivot table to easily analyse and graph the results.
Related
I am playing around with data scraping scripts. For now I am starting with PHP/cURL. Reason one I am interested in learning this is to learn how these are designed to help protect my own websites against those sneaky malicious ones. The second reason is to design these in a way that acts like a human for the purpose of avoiding undue burden on a website owners server.
If I use this in real life, it would be simply to use it to automate what I currently already do manually but I don't want to abuse this process however, I am a bit lazy so rather not do it manually.
To perform like a human:
1) Send header that look like a browser.
2) Send a referrer that represents the source of the link (sequence of pages).
3) Create delays randomized similar to how a human would search per page fetch.
4) Clear cookies when done. (Have to learn more about this, not sure how cookies function in a web scraper environment)
If the tools used above are done correctly, is IP Proxy switching necessary? Are there any other considerations I should be aware? Still learning about this so just curious at this point.
Any idea how to implement this (http://fluin.com/63) using MySQL+PHP+Javascript(mootools)?
In a nutshell, it's a realtime threaded conversational web app.
Update:
This uses http://www.ape-project.org/home.html
Any idea how to implement realtime stuff without AJAX push (ape)?
Install Firefox.
Install Web Development toolbar
Install Firebug
Install HttpFox
Read docs of above tools re how to use, what they can do.
Go to http://fluin.com/63. Use above tools to inspect.
Read up on Databases and data models, and MySQL.
Build your own.
Well, this depends on your definition of realtime, which, in its technical meaning, is simply impossible with public ip networks and traditional tcp stack, for you have no control over timing.
Closer to the topic though, to get any web page updated without direct user intervention, you'd have to use javascript to poll server for changes since the last successful poll, and do this over certain intervals of time. In calculating these intervals you'll have to consider both network/server load, and the delay that is comfortable for the user.
The server, of course, will have to store the new data and its timely status (creation timestamps are one way of doing it), to be able to distinguish between content already delivered to various clients.
As soon as the server reports new content, it is inserted into a dom page via javascript and the user sees the response.
This is a bit general, of course, but you should get the idea.
Isn't it like a shoutbox ? here an example of one
Doing this properly using PHP only is very hard. When you have 5 users you could use long-polling, but it will definitely not scale when you have let's say 1000 users.
Using comet with PHP?
The screencast(link) in my post shows how you could implement it, but it has a couple of flaws:
It touches the disc(disc is very slow compared to memory).
To make matters worse it also polls the disc frequently(filemtime()).
Maybe phet(PHP) is able to scale. You should try that out.
To make it scale I think you need at least:
a good implementation of long-polling(at least long-polling. You have better transports) that can handle load.
keep data in memory(much faster than dics) using something like redis or memcached.
I would use:
node.js with socket.io(video) module.
to keep data in memory I would use node_redis(video).
So, I'm new to dynamic web design (my sites have been mostly static with some PHP), and I'm trying to learn the latest technologies in web development (which seems to be AJAX), and I was wondering, if you're transferring a lot of data, is it better to construct the page on the server and "push" it to the user, or is it better to "pull" the data needed and create the HTML around it on the clientside using JavaScript?
More specifically, I'm using CodeIgniter as my PHP framework, and jQuery for JavaScript, and if I wanted to display a table of data to the user (dynamically), would it be better to format the HTML using CodeIgniter (create the tables, add CSS classes to elements, etc..), or would it be better to just serve the raw data using JSON and then build it into a table with jQuery? My intuition says to do it clientside, as it would save bandwidth and the page would probably load quicker with the new JavaScript optimizations all these browsers have now, however, then the site would break for someone not using JavaScript...
Thanks for the help
Congratulations for moving to dynamic sites! I would say the following conditions have to be met for you to do client-side layout (it goes without saying that you should always be doing things like filtering DB queries and controlling access rights server side):
Client browser and connection capabilities are up to snuff for the vast majority of use cases
SEO and mobile/legacy browser degradation are not a big concern (much easier when you synthesize HTML server side)
Even then, doing client-side layout makes testing a lot harder. It also produces rather troublesome synchronization issues. With an AJAX site that loads partials, if part of the page screws up, you might never know, but with regular server-side composition, the entire page is reloaded on every request. It also adds additional challenges to error/timeout handling, session/cookie handling, caching, and navigation (browser back/forward).
Finally, it's a bit harder to produce perma-URLs in case someone wants to share a link with their friends or bookmark a link for themselves. I go over a workaround in my blog post here, or you can have a prominent "permalink" button that displays a dynamically rendered permalink.
Overall, especially when starting out, I would say go with the more kosher, better supported, more tutorialed, traditional approach of putting together the HTML server side. Then dip in some AJAX here and there (maybe start out with form validation or auto-completion), and then move on up.
Good luck!
It is much better to do the heavy lifting on the server side.
In CodeIgniter you create a view, looping through all the rows in the table adding in the classes or whatever else you would need. There is no reason at all to do this in Javascript.
Javascript is a sickly abused language with unfortunate syntax. Why on earth would you want to load a page, and then issue a AJAX call to load up some JSON objects to push into a table is beyond me. There is little reason to do that.
Javascript (and jQuery) is for end user enhancement. Make things slide, flash, disappear! It is not for data processing in even the mildest of loads. The end user experience would be crap because you're relying on their machine to process all the data when you have a server that is infinitely more capable and even designed for this specifically.
It depends on your target market and the goal of your site.
I strongly believe in using the client side where ever you can to offload work from the server. Obviously its important you do it correctly so it remains fast for the end user.
On sites where no-js support is important (public websites, etc), you can have fallbacks to the server. You end up doubling code in these situations but the gains are very beneficial.
For advanced web applications, you can decided if making JS a requirement is worth the trade of losing a (very) few users. For me, if I have some control over the target market, I make it a requirement and move on. It almost never makes sense to spend a ton of time to support a small percentage of potential audience. (Unless the time is spent on accessibility which is different, and VERY important regardless of how many people fit into this group on your site.)
The important thing to remember, is touch the DOM as little as possible to get the job done. This often means building up an HTML string and using a single append action to add it to the page vs looping through a large table and adding one row at a time.
It's better to do as much as possible on the server-side because 1) you don't know if the client will even have JavaScript enabled and 2) you don't know how fast the client-side processing will be. If they have a slow computer and you make them process the entire site, they're going to get pretty ticked off. JavaScript/jQuery is only supposed to be used to enhance your site, not process it.
You got the trade-off correctly. However, keep in mind that you can activate compression in the server side, which will probably make adding repetitive markup to format the table a small bandwidth cost.
Keep also in mind that writing Javascript that works in all browsers (including hand-helds) is more complicated than doing the same server side in PHP. And don't forget that the "new JavaScript optimizations" do not apply to the same extent to browsers of handheld devices.
I do a great deal of AJAX app development and I can tell you this from my experience. a good balance between the two is key.
do the raw data server-side but use javascript to make any modifications that you would need to it. such as paging, column sorting, row striping, etc.
I absolutely love doing everything in AJAX heh.. but there are some short falls to doing it using AJAX, and that's SEO. search engines do not read javascript, so for the sake of your website's page rank, I would say have all data served up server side and then formatted and made look cool client-side.
The reason I love AJAX so much is because it drastically speeds up your APP usage by the user as it only loads the data you need to load where you need to load it, versus load the entire page every time you do something... you can do a whole bunch of stuff, such as hide/show rows/columns (we are talking about table manipulation here because you mentioned a table) and even with these show/hide actions add delete actions where when you click a delete row or button it deletes that row not only visually but in the database all done via AJAX calls to server-side code.
in short.
raw data: server-side sending to the client the raw data in html layout (tables for table structured data, however I do everything else in divs and other flexible html tags, only do tables for column/row style data)
data formatting: client-side which also includes any means of interacting with the data. adding to it, deleting from it, sorting it differently etc. This achieves two things. SEO, and User Experience (UX).
I know this seems like a really subjective question, but from what I'm reading, I'm generally confused. We are developing a web-based application that has tabbed windows, etc-very much a "desktop-like" application graphically. About half the page never or rarely changes, and the other half is content that could be handled dynamically within a main window.
So from the ajax proponents, this seems like a perfect example of something that would benefit from ajax. The only thing stopping me (other than lack of ability :-D) is that I'm concerned we would degrade the speed of the app (which isn't really all that great anyway...)
Can anyone either shed some light on this subject or point me to more reading material on performance differences when changing lots of data at once via ajax?
About the differences that can often be seen :
If you load more data via AJax requests, and less via "full-page" requests, it means :
less data will go through the network (only content of pages, not menus, sidebars, ...)
less thins to generate on the PHP/SQL side (same : no menu, ...)
But more Ajax requests can also mean more requests, depending on how your application is developped
Which can mean more load on the server...
You also might want to think about :
Indexation by search engines,
Accessibility,
Graceful degration,
and all the possible stuff like that...
Which, in the end, can mean you'll maybe spend lots of additionnal time developping the "more Ajax" version ; and time is money -- and a few days of developper-time can cost more than renting/buying a second server.
In the end, I would say what really matter is not "performance" : you'll probably have greater gains spending a couple of days optimizing what you already have (profiling on the backend side, minifying/compressing JS/CSS, merging JS/CSS files to have less HTTP requests, ...) than by transforming everything to "full-Ajax".
And, of course, what really matters is user-experience !
As someone who just finished developing a similiar application i can say that using ajax to create a "desktop - like" application.... well... looks cool....
But there are certain points that you should be considering before you start...
Dealing with forms when you a load a page with ajax will require the additional knowledge of a javascript framework like Jquery.... Because loading a page portion of a page with ajax does not affect the DOM tree of the page and hence the newly added elements will not be identified by javascript functions.... That's where you will be needing the livequery plugin for jquery.
If you are looking do develop a site optimized for search engine indexing.... ajax - refreshed content may not be a good idea. I developed only the backend part. i.e the administrator module like a desktop application. The frontend where the user actually interact was your normal, day-to-day, website...!!
For the following pretty straightforward task: query a list of products from a DB and present it on a webpage,
consider 2 setups:
Setup1: PHP script queries. All content is built on the server and the entire page is served back to the client.
Setup2: Static HTML "page skeleton" requesting content using AJAX. Received content is parsed on the client side using Javascript and rendered using innerHTML or similar.
Of course the second setup only makes sense when you have pages, categories and tags for the client user to choose from.
I need to compare these two, at least by means of:
time it will take content to be served
user experience (setup1 is delivered as a whole, setup2 is delivered in "two parts")
scalability - how do the setups compare when I have 100,000 queries daily
Any thoughts on the issue will be much appreciated.
You may find the following question helpful: Smarty Vs. Javascript/AJAX
I brought up a few points in my answer to that question:
You should use server-side scripts to show any data that is known at the moment the page is loaded. In this case, you know the list of products should be displayed. The fact that a question's answers should be shown is known at page load.
You should only use AJAX calls to load dynamic data that is not known at the moment the page is loaded. For example, when you click the "comments" link under a question or answer on Stack Overflow. The fact that you want to view a particular question's comments is not known at page load.
Javascript should not be required to access core functionality of your site.
You should gracefully degrade functionality when Javascript is disabled. For example, Stack Overflow works just fine with Javascript disabled. You don't have access to real-time Markdown previews or dynamic badge notices, but the core functionality is still intact.
A single HTTP request to a server-generated page will load significantly faster than a request to load a page that makes five or six additional AJAX calls, especially on high latency connections (like cellular networks). See Yahoo's Best Practices for Speeding Up Your Website.
You should think of Javascript as a bonus feature that might not be enabled, not as something that should be used to construct critical pieces of your website. There are exceptions to this rule. If you want to do some sort of pagination in which you click a "next page" button and only the product list changes, AJAX might be the right choice. You should, however, make sure users without Javascript are not excluded from viewing the entire list.
There's nothing more frustrating than when a page can't be accessed because the web developer didn't obey the KISS principle. As an example, take Friendly's Restaurants. I wanted to check out their menu while I was at the mall, so I loaded their website on my iPhone, only to find out that you literally can't get any meaningful information about the restaurant without Flash. It's nice to have fancy menus with swooshing desserts flying everywhere, but in the end, I just wanted to see the items on their menu. I couldn't do that because they required Flash. Graceful degradation in service would have been helpful in that case.
Some things on the web can't be done effectively without Javascript. Displaying a list of products is not one of them. If you are still unsure, look at how other popular websites do things. I think you'll find most of the successful, well-engineered websites follow the guidelines listed above.
AJAX is probably better choice when only a small part of the page changes.
I would recommend starting with the server side version and then building AJAX on top of that. This way you will get also a version of your site that works without javascript, which you probably need anyway if you care about being indexed in search engines.
But first concentrate on creating a page that just works - you can always optimize it later.
Performance on the client has many factors. What is running at the time, what browser, what the content is, what the CSS of the page is, how full is the browser's cache, what plug-ins are installed, what is happening on the network, etc. Just remember that when you are playing with the numbers.
Unless the implementation sucks, AJAX should win hands down. Among the benefits are:
parallelism due to parallel requests on the client side (i.e. you can use multiple server CPU cores to serve parts of one served web page, that can't be done easily using PHP)
refreshing only small parts of the page is faster (less data to transfer, generate ...)
it scales much better since the server has less work to do (esp. if you can offload some of the processing needed for generating html to the client instead of just delivering it)
Dynamic pages like http://www.startpagina.nl/ have been doing this successfully since way before the recent AJAX fad (1 static file delivered, all customization done on the client side - last time I checked anyway).
Of course you can screw things up with either method so that it becomes slower than the other.