I am creating an application that will use Web Sockets for a notification system. Is it better to have the application in an iframe with the Web Sockets in the parent so there isn't a new connection every time a page is loaded? Or maybe it should re-connect?
What are your thoughts?
If anyone has any other way in PHP to get push like notifications without sending a AJAX request every 10 seconds then let me know.
Thanks.
This is one of the options you have described.
The problem with that option is that there wont be direct control over the content of that inner iframe, and you will need to implement push message window communication between parent of iframe, in order to be able to change iframe src attribute, in case someone will refresh parent page, and iframe should refresh to actual state, not initial page.
Second problem, there will be no SEO at all. So your page wont be crawlerable by search engine robots. If SEO is important for your application - then this is not an option.
In WebSockets, if you work with sessions, it is important to make session available for normal PHP script and WebSockets logic, in order to keep consistent access to data it self. PHP will make it not an easy task at all.
You might consider Long Pull technique as well, as it allows to open one AJAX request and then get responses back, and this request can last some time but will eventually close and have to be reopened on client-side.
Another option is to review actual application architecture, and think of single-page application. It have as well cons and pros.
Good thing about it, is UX will be much higher. Response times as well as you will load less content and data.
Pros are that it requires lots of development on front-end side in javascript. As well there is two major routes you can do single-page applications. Consistent and inconsistent. In first case you need to make sure that your back-end will server static html on refresh or just navigating to specific link, the same way as your single-page application would generate using java-script. Then it solves issues with SEO. While inconsistent approach, will just be purely on javascript (front-end), and will have issues with SEO.
WebSockets usually used with single-page applications, for example Facebook Chat is great example of such. Or Google Talk while you are in Gmail account.
They are not meant to be used with often refreshed pages, as Handshake process is a bit more heavy than normal HTTP request.
Related
We have a page with multiple layouts. The difference between these layouts, and the way they're generated, is significant enough that they need to be done server-side, in PHP.
To A/B test these layouts, we created a page redirect experiment via Google Optimize, which would add a custom variable in the URL https://website.test/page?layout_id=123. This worked fine, but the problem with this is the delay between the page load and when the DOM is ready, forcing the client to see the original layout before they're redirected.
We then decided to move the redirect to the server-side script in PHP.
One way is using https://github.com/theiconic/php-ga-measurement-protocol/ (which is based on Google's Measurement Protocol) and according to https://developers.google.com/optimize/devguides/experiments#implement-experiment, we need to decide on our own which variant to serve, however, it still requires a clientId or userId to identify the client and this is only delivered via a cookie called _ga, after the first load.
The problem with this approach is that users visiting the page for the first time cannot be identified, and therefore it messes the results.
What is the best approach for such a situation, where the server needs to know in advance what layout to deliver?
Not sure this question is still relevant but we implemented our own mechanics (a simple A/B testing engine in PHP) which determines what page/code to show. The Optimize is only used to setup the variants and track the results.
Here is a document from google that explains it:
https://developers.google.com/optimize/devguides/experiments
I've done web scraping before but it was never this complex. I want to grab course information from a school website. However all the course information is displayed in a web scraper's nightmare.
First off, when you click the "Schedule of Classes" url, it directs you through several other pages first (I believe to set cookies and check other crap).
Then it finally loads a page with an iframe that apparently only likes to load when it's loaded from within the institution's webpage (ie arizona.edu).
From there the form submissions have to be made via buttons that don't actually reload the page but merely submit a AJAX query and I think it just manipulates the iframe.
This query is particularly hard for me to replicate. I've been using PHP and curl to simulate a browser visiting the initial page, gather's the proper cookies and such. But I think I have a problem with the headers that my curl function is sending because it never lets me execute any sort of query after the initial "search form" loads.
Any help would be awesome...
http://www.arizona.edu/students/registering-classes -> "Schedule of Classes"
Or just here:
http://schedule.arizona.edu/
If you need to scrape a site with heavy JS / AJAX usage - you need something more powerful than php ;)
First - it must be full browser with capability to execute JS, and second - there must be some api for auto-browsing.
Assuming that you are a kid (who else would need to parse a school) - try Firefox with iMacros. If you are more seasoned veteran - look towards Selenium.
I used to scrap a lot of pages with JS, iframes and all kinds of that stuff. I used PhantomJS as a headless browser, that later I wrapped with PhantomCurl wrapper. The wrapper is a python script that can be run from command line or imported as a module
Are you sure you are allowed to scrape the site?
If yes, then they could just give you a simple REST api?
In rare case when they would allow you to get to the data, but would not provide API, my advice would be to install some software to record your HTTP interaction with web site, maybe wireshark, or some HTTP proxy, but it is important that you get all details of http requests recorded. After you have that, analyze it, and try to replay it up to the latest bit.
Among possible chores, it might be that at some point in time server sends you generated javascript, that needs to be executed by the client browser in order to get to the next step. In this case you would need to figure how to parse received javascript, and figure out how to move next.
An also good idea would be not to fire all your http requests in burst mode, put put some random delays so that it appears to the server more "human" like.
But in the end you need to figure out if all this is worth the trouble? Since almost any road block to scraping can be worked around, but it can get quite involved and time consuming.
I'm developing a web site, which would make use of PHP, Javascript (JQuery) and use AJAX to connect the two. My question is, how should the coding process go.
I know that Javascript is supposed to be used as an extra kick, and should not be relied upon because it can be turned off. So, should I code the entire site in PHP, and then after all of that is done, add the JQuery code, or should I do both side by side?
If you decide to use AJAX as a core part of the site then you are basically excluding people without javascript which depending on your application can be a legitimate design decision. If you choose to do that then you should check if the user has JavaScript and warn them if they do not.
If you are requiring JavaScript you can develop with it simultaneously with the development of your server side code PHP code. If not, and JavaScript is just a UI enhancement it should be added in later.
Either way validating user input should always also be done on the server side in addition to the client side. All security related code should be only on the server side.
If you are creating a Rich Internet Application than Javascript/Flash/Silverlight will become a requirement of the user in order to use your website. In that case you should perform a check to ensure the user has the correct plugin or javascript enabled. Otherwise display a page which states your site requires it, etc.
If you're just trying to use JavaScript to enhance your site but without it adding a ton of development than the backend would be developed first or if you need to support a large client base then different versions of your site could be made to support JavaScript and non-JavaScript users.
If you want your site to work even if Javascript is disabled, I would develop the site with PHP first and then add your Javascript enhancements. For example, on one site I developed, there is a calendar that lists upcoming events. When the user clicks on that event, a colorbox will appear and the details of that event will be loaded into that colorbox using AJAX. If Javascript is disabled, clicking on the event will just take the user to the event's page.
I think it depends on the approach you take as a programmer.
If you take a top-down approach and start from the user interface and its features, then start from the Javascripts and HTML markup. In the process you can find out how your server API should respond.
If you take a "server capabilities" approach and implement what you can do in the server, then obviously you start implementing that part first. Then you'll continue with the markup and client javascript code, and adapt it to the available APIs that you built. (And probably, in the process, adapt them too).
In both cases, a bit of a design on paper wouldn't hurt.
Of course, as other people have answered, it also greatly depends on how extensive your javascript interface is, how much burden it takes away from PHP, and if you intend to provide an HTML-only interface where the PHP would need to do much of the work.
For instance, let's say you have a table in your code and you want the user to allow to sort it by different columns. This can be done in Javascript, and can be done in PHP, and it can be done in both. It's up to you and your decisions.
If you're planning with jQuery and Ajax in mind then JavaScript off would be almost a completely separate project. However if that is the case, I'd recommend taking the following steps.
Develop the data access and business logic layers.
Build the PHP UI layer w/ full page reloads, etc.
Build an API over the DAL and BL that can be called from JS.
Build the jQuery / Ajax UI from the ground up.
I'm skeptical about the pure server-side UI implementation though, and I'd go about it only if your user base will likely prefer that over the now mainstream JQuery based one.
I have been working on a site that makes some pretty big use of AJAX and dynamic JavaScript on the front end and it's time to start stress testing. But how do you properly stress test something that requires clicking several links on the front-end? One way I was able to easily hit every page of the site quickly and repeatedly was to point a Google Mini at it. But that's not going to click links and then navigate Modal windows and things like that.
Edit - I should point out that the site is done in PHP5 and the JavaScript library used is jQuery. Not sure if this would make any difference but felt it might be useful to know.
JMeter is great at this. You may record your sessions and tweak them to your liking.
So-called 'ajax load testing' is a recurring subject on this site, and is often confused. So let's get it straight: There is really no difference between load testing a normal web page and load testing with ajax. It all boils down to discrete requests; they just happen to not be full page refreshes.
One thing to keep in mind is there is a distinct difference between load testing the server processing the requests (a load test) and the performance on screen of the UI components being updated (how well your javascript performs.)
Simple load test example:
initial page load
login
navigate?
5-10 'ajax' requests (or whatever may fit your application usage pattern)
logout
There are load testing tools that can support AJAX. For example, WebLoad
http://www.radview.com/solutions/ajax-load-testing.aspx
What you really want is to stress test is the server's ability to handle the ajax requests. Use a load tool that looks at the requests while "recording" the test, and then tune as appropriate. I have only used the vs test edition one, so I can't point you to a low cost one.
I disagree with Nathan and Freddy to some degree. They are correct that "AJAX testing" is really no different in that HTTP requests are made. But it's not that simple. See my article on Ajaxian.com on Why Load Testing Ajax is Hard.
JMeter, Pylot, and The Grinder are all great tools for generating HTTP requests (I personally recommend Pylot). But at their core, they don't act as a browser and process JavaScript, meaning all they do is replay the traffic they saw at record time. If those AJAX requests were unique to that session, they may not be suitable/correct to replay in large volumes.
The fact is that as more logic is pushed down in to the browser, it becomes much more difficult (if not impossible) to properly simulate the traffic using traditional load testing tools.
In my article, I give a simple example of how difficult it becomes to test something like Google's home page when you want to query 1000's of different search terms (an important goal during load testing). To do it with JMeter/Pylot/Grinder you effectively end up re-writing parts of the AJAX code (in your case w/ jQuery) over again in the native language of the tool.
It gets even more complex if your goal is to measure the response time as perceived by the user (which is arguably the most important thing at the end of the day). For really complex applications that use Comet/"Reverse Ajax" (a technique that keeps open sockets for long periods of time), traditional load tools don't work at all.
My company, BrowserMob, provides a load testing service that uses Firefox browsers, powered by Selenium, to drive hundreds or thousands of real browsers, allowing you to measure and time the performance of visual elements as seen in the browser. We also support traditional virtual users (blind HTTP traffic) and a simulated browser (via HtmlUnit).
All that said, usually a mix of a service like BrowserMob plus traditional load testing is the right approach. That is, real browsers are great for a full-fidelity load test, but they will never be as economical as "virtual users", since they require 10-100X more RAM and CPU. See my recent blog post on whether to simulate or not to simulate virtual users.
Hope that helps!
You could use something like openSTA.
This allows a session with a web site to be recorded and then played back via a relatively simple script language.
You can also easily test web services and write your own scripts.
It allows you to put scripts together in a test in any way you want and configure the number of iterations, the number of users in each iteration, the ramp up time to introduce each new user and the delay between each iteration. Tests can also be scheduled in the future.
It's open source and free.
It produces a number of reports which can be saved to a spreadsheet. We then use a pivot table to easily analyse and graph the results.
When is it appropriate to use AJAX?
what are the pros and cons of using AJAX?
In response to my last question: some people seemed very adamant that I should only use AJAX if the situation was appropriate:
Should I add AJAX logic to my PHP classes/scripts?
In response to Chad Birch's answer:
Yes, I'm referring to when developing a "standard" site that would employ AJAX for its benefits, and wouldn't be crippled by its application. Using AJAX in a way that would kill search rankings would not be acceptable. So if "keeping the site intact" requires more work, than that would be a "con".
It's a pretty large subject, but you should be using AJAX to enhance the user experience, without making the site totally dependent on it. Remember that search engines and some other visitors won't be able to execute the AJAX, so if you rely on it to load your content, that will not work in your favor.
For example, you might think that it would be nice to have users visit your blog, and then have the page dynamically load the newest article(s) with AJAX once they're already there. However, when Google tries to index your blog, it's just going to get the blank site.
A good search term to find resources related to this subject is "progressive enhancement". There's plenty of good stuff out there, spend some time following the links around. Here's one to start you off:
http://www.alistapart.com/articles/progressiveenhancementwithjavascript/
When you are only updating part of a page or perhaps performing an action that doesn't update the page at all AJAX can be a very good tool. It's much more lightweight than an entire page refresh for something like this. Conversely, if your entire page reloads or you change to a different view, you really should just link (or post) to the new page rather than download it via AJAX and replace the entire contents.
One downside to using AJAX is that it requires javascript to be working OR you to construct your view in such a way that the UI still works without it. This is more complicated than doing it just via normal links/posts.
AJAX is usually used to perform an HTTP request while the page is already loaded (without loading another page).
The most common use is to update part of the view. Note that this does not include refreshing the whole view since you could just navigate to a new page.
Another common use is to submit forms. In all cases, but especially for forms, it is important to have good ways of handling browsers that do not have javascript or where it is disabled.
I think the advantage of using ajax technologies isn't only for creating better user-experiences, the ability to make server calls for only specific data is a huge performance benefit.
Imagine having a huge bandwidth eater site (like stackoverflow), most of the navigation done by users is done through page reloads, and data that is continuously sent over HTTP.
Of course caching and other techniques help this bandwidth over-head problem, but personally I think that sending huge chunks of HTML everytime is really a waste.
Cons are SEO (which doesn't work with highly based ajax sites) and people that have JavaScript disabled.
When your application (or your users) demand a richer user experience than a traditional webpage is able to provide.
Ajax gives you two big things:
Responsiveness - you can update only parts of a web page at a time if need be (saving the time to re-load a page). It also makes it easier to page data that is presented in a table for instance.
User Experience - This goes along with responsiveness. With AJAX you can add animations, cooler popups and special effects to give your web pages a newer, cleaner and cooler look and feel. If no one thinks this is important then look to the iPhone. User Experience draws people into an application and make them want to use it, one of the key steps in ensuring an application's success.
For a good case study, look at this site. AJAX effects like animating your new Answer when posted, popups to tell you you can't do certain things and hints that new answers have been posted since you started your own answer are all part of drawing people into this site and making it successful.
Javascript should always just be an addition to the functionality of your website. You should be able to use and navigate the site without any Javascript involved. You can use Javascript as an addition to existing functionality, for example to avoid full-page reloads. This is an important factor for accessibility. Javascript should never be used as the only possibility to reach or complete a request on your site.
As AJAX makes use of Javascript, the same applies here.
Ajax is primarily used when you want to reload part of a page without reposting all the information to the server.
Cons:
More complicated than doing a normal post (working with different browsers, writing server side code to hadle partial postbacks)
Introduces potential security vulnerabilities (
You are introducing additional code that interacts with the server. This can be a problem on both the client and server.
On the client, you need ways of sending and receiving responses. It's another way of interacting with the browser which means there is another point of entry that has to be guarded. Executing arbritary code, posting data to a non-intended source etc. There are several exploits for Ajax apps that have been plugged over time, but there will always be more.
)
Pros:
It looks flashier to end users
Allows a lot of information to be displayed on the page without having to load all at the same time
Page is more interactive.