A quick Google search of "jquery rss parser" returns many cool plugins.
That being said, what are the pros and cons of adding rss content to my website using server-side versus client-side technology?
It could be cool to implement it client side, as then you could build really cool user interface, where new RSS items are fetched every few secs / mins (consider Masonry plugin). Client side would allow you to load RSS feeds and display them as soon as content loaded from a source, rather than waiting until all feeds are loaded.
Server side - you would have to consider some sort of caching as parsing feeds may be very time consuming and no one likes waiting....
Here a few pros and cons:
PROs
Content Dynamically Updated
Can be flexible to drive the most relevant content to the visitor
Allows you to style and manipulate the data to keep the site looking consistant
CONs
Client side requires javascript to be enabled on the browser/device.
Server side while more versatile may require additional installation of modules on the server to work properly
Server side will not auto update without a page reload
RSS Feed structure could change requiring a rewrite of your code.
RSS Feed not outputting properly causing your site not to show the right information (improper tag use or unescaped characters)
May not be updated right away, I know a few feeds that update on a weekly basis
If you are looking for a way to add content to your site. Look into creating or using a Content Management System. There are available plugins that do the same and will allow you some control over how the data is displayed or interpreted, not to mention can also cache an older feed. A list of most common Content Management Systems can be found here: http://en.wikipedia.org/wiki/List_of_content_management_systems
I was looking for a list like the following:
Benefits of using client-side technology for RSS feed:
Content can be updated without page reloads
Content can be sorted/manipulated without page reloads
Benefits of server-side technology for RSS feed:
Improves SSO
Doesn't require the client to have JavaScript enabled.
Related
As you can read in the title I have question about php and caching an xml feed obtained from a localhost API.
The xml feed is already cached inside of the API, so that the data don't needs to be read every time from the database. (.NET Service) -> there are multiple feeds running on the server.
The XML feed contains job advertisements (Meta data, keywords and also the HTML), the size of the feeds are different between 100kb and 5mb. Also the time periods how often new data is added is pretty different. (This is often daily or in shorter intervall). As mentioned above this is already handled in the holy .NET Service.
Now my actual question is:
Is it useful to cache the data again for a microsite which have the XML feed as a datasource for rendering a job overview.
If i wouldn't cache anything, for every request the data would be read from the API, filtered and rendered in to an html.
Now I'm not sure if an additional cache would enhance the performance. And where should I probably add a cache. (After the XML is sent back from the API, or after the data is deserialized).
My actual question for the above scenario:
Is it useful to add an additional cache? (performance reasons)
If yes, is there a simple php solution or a php library to use?
Thanks,
Hans Rudolf
The question is how to get ajax calls source code? this is not crawled, for example how to crawl the pictures on a link like this? http://www.tiendeo.nl/Catalogi/amsterdam/16558&subori=web_sliders&buscar=Boni&sw=1366
If you do inspect element, then it will show you the right code in the middle where the pictures are. But how to crawl this? If you click the next page, then it will have other images in the source. How to get the source for all images?
If I understand your question correctly (How do I crawl information loaded into the page by ajax calls?), the answer is that you either need some sort of javascript-aware crawler, or you need to inspect the javascript to figure out what resources are being polled to load the content you're interested in. From PHP you should be able to send curl get requests to these URLs and receive the same responses the site's javascript is using to render the entries.
The latter option has some rewards--namely that you will most likely be able to get simple, easy-to-use JSON responses to your requests.
As with most web scraping efforts, it tends to be the case that some content providers won't appreciate your interest in their data (especially if you're collecting it in ways that put undue strain on their systems or resources.) Keep in mind that they make take steps (technical or legal) to stop you if they notice/mind.
Addendum:
If you're hoping to crawl a variety of similar sites without needing to look through the source to find the resources they're using, (let's say for the sake of argument that you're just trying to naively scrape all images over a certain size from several sites selling the same sort of items), you'd need the former option--some sort of javascript-aware scraper. I don't know if such a thing exists, but it wouldn't surprise me.
I am just stuck a little in making a choice between PHP chart Lib and JavaScript Chart Lib. I do understand that PHP if for the server side and Javascript for the client side. My problem is what difference does it make when using their charting libraries. Is it performance issue or what?
I want to understand the difference in using PHP chart Libs and JavaScript Chart Libs. Please am not looking for examples of their chart libraries. I am looking for why i should choose one over the other.
I tried to google 'php chart vs javascript chart' but didn't get any links that can give me
the difference.
EDIT 1
1)
If this question has been answered before, then point me there.
2)
Am developing the application for internet
EDIT 2
1)
I have found out about PHPChart PHPChart which has both PHP source code and JavaScript source code. If anyone has experience in that library, does it may be solve the problem of server side load (bandwidth issues) etc.. I am thinking since it has both the PHP and JavaScript source then it may be the best to use. Am just assuming. :-)
Thank you very much
Both ways of creating graphs have their own pros and cons.
If you decide to do it using PHP, first you need to make sure that you have all the required graphical libraries installed (e.g. GD, which might not always available on shared hosts).
Assuming you have them, the first negative thing in my opinion is that you will end up with static images. Of course, it's not always a bad thing, as that ensures compatibility with all the clients, be those with or without javascript support, however, it takes away the dynamics of graphs generated on the client side using javascript. Your users won't be able to zoom, move, slide, full screen or do anything that they could with the likes of Highcharts or Flot.
Another con is that images take up more bandwidth than, say, JSON. The bigger you want to have your graph, the more colors it contains, the longer your clients will have to wait till your page loads. And just because those loads are not asynchronous, they will have to wait for the images to load before they will see the rest of the page.
With javscript libraries everything is different though. You only request the data required for your graph and you only request it when your page loads. The amount of data is usually smaller than an image would be plus you can compress your output with GZ to make it even smaller. Users will see nice spinners informing them that the graph is loading instead of some incomplete webpage.
Another thing to take into account is - what if you decide to show a nice table with data in them below each graph? If you chose to render images on the server, you would end up having to add new functionality just to get the data. With JSON, however, you just make one call, render the graph and display the table. Maybe calculate totals or do whatever you want with it. Hand it out to people as an API if you wish, after all :)
If you ask me, I would definitely go with client-side graphs as most of the devices have nice HTML5 support nowadays and being able to display a graph on an Android phone, or an iPhone or an iPad shouldn't pose a problem. If you only need images and you don't wish to expose the original data, go with PHP.
My opinion is that having a server side solution (i.e. php) takes away any browser compatibility issues you may have with a client side solution (i.e. javascript) and hence support issues.
A benfit of using JS is that it does offload resources from your server to the client because you may only have to generate some light weight data (e.g. JSON , XML) and the rendering occurs on the client. You will have to investigate how many hits your server is likely to get, etc to determine if resource is an isuse with PHP or JS.
However, using Php to create images of charts you can always get around the performance/resource issue by using a cache of the image files and serving from the cache (it's a just a folder of images) instead of generating a new one. Whether you cna use a cache will depend on your usage. If clients require up to the second data and its always changing, obviously a cache may not be of use.
Here's what I see :
Using PHP
Increase load on the server for the request
Will work everywhere
Also, like someone said here and made me think of it, you can cache the image that PHP give you, reducing bandwith (no lib to download) and reducing load (cache)
Using Javascript
Decrease load but increase the bandwitch and addition http request (to load the JS lib)
Will work where JS is available
But remember, PHP take more load then an HTTP request.
Also, always remember, Javascript is made for effects and specials stuffs you need to display.
There is one PHP render advantage that no one told about. Since sometime you need to include chart as image into PDF, DOC, XLS etc. file or email it – you have no other way except to render chart on server and store it as image to be inserted.
For data manipulation you use PHP.
For visual and behavioral effects you use JavaScript.
For that reason, you should use Javascript as its designed for visual behavior. Plus it will put less load on your server as all processing will be client side. As more people use your application simultaneously, it will start to slow down as your server will be doing a lot more then it has to.
Hope that helps :)
I am Creating a web application that will get homework from schools website.
I have been creating rss feeds for the website using dapper Creating an rss feed converting that into html and then putting that into an Iframe on my website.
I think there has to be a better way to do this.
I don't care what language this would be done in but i would prefer to do it client-side with php or javascript.
I'm intermidiete level php and not so good at java script.
If you could provide detailed explanation with some code that would be great.
also I have legal permission from my school which will be the first for this to go live to get the homework.
I'm assuming you are about create an php application to retrieve the rss from external site (which already have a specific url). Well, i think its better to build a public API first at the source app/external site, to get more multimedia support.
But if you're have no problem with parsing an xml format into html directly, then you can use cURL instead iFrame(or if you didnt have cURL installed in your php, you can modify lil bit and use my Proxy Library,a PHP class which enables you to make call to internal and external site with general HTTP request by opening a socket connection to the remote host, originally writen as Codeigniter Library.)
That way, it will be easier to presenting a html view/page rather than proccess it via iFrame.
I think this is a real challenging one!
I write a website for my local football league, www.rdyfl.co.uk , and include javascript code snippets from the F.A's Full-Time system where we generate our fixtures, linking in tables fixtures recent results etc.
For another feature I want to add to the site I need to scrape the 'Upcoming Fixtures' for each agegroup and division but when I examine the source I have two problems.
The fixtures content is generated by javascript and therefore I need to see the generated source and not just the source.
When I view the generated source using Firefox the team names are actually further javascript links and not the name itself.
I basically want to somehow download the fixtures on a regular basis and write then to a mysql database ?
I have asked the F.A. and they have no more options available to access the data ?
Having never coded for scraping before can anyone point me to a simple solution or does anyone fancy the challange?
This question was asked a long time ago, but I noticed it was active today 🤷.
You should be able to scrape the website using a headless browser such as Puppeteer. Using Puppeteer you are able to access a URL and execute JavaScript or interact with the website as you would with an ordinary browser. Parsing the output DOM and storing it should then be relatively straightforward.
There are plenty of articles on this topic using Puppeteer.
The latest version of OutWit Hub is doing a pretty good job on dynamic content. The source scraped by outwit to extract links, images, documents and tables and text is the updated DOM. You can certainly make a job to grab what you need using these.
Custom scrapers are still applied to the static source in version 1.0.3 but version 1.1.x (still in beta) will offers the choice between the static source and the dynamically modified DOM.
Scrapping content produced by Javascript is challenging. AFAIK you will need to do this with AJAX. Hopefully the content has some css that you can grab with jQuery or at least some id's. Do you have id's or classes that you can grab?