I'm terrible at keeping track of my bills, so I wanted to create something automated. I also wanted the challenge of making it myself.
My questions:
Is it possible to have a webpage connect to another domain (any utility website i.e. timewarnercable.com) with the proper login credentials and retrieve the dollar amount I owe, then send me an email or even just display it on the webpage?
I've already got a webpage setup that has all my account info stored in it (don't worry it's only a local site!) and I can click a button and the info I have stored sends a POST request to the utility login site. This logs me in to my account page and then I can view the bill. But don't want it to open another page..I'd rather load the content of that page in the background, scan for the code where its says my $ owed, then capture that somehow, then return the dollar amount onto the webpage.
If so, is this possible to design this with Ruby (Rails) or php, with Javascript / AJAX.
Thanks!
What you're basically asking about is "page scraping", but your scenario is more complicated. You would have to fake the login post, capture and store any cookie/session info returned to you in the response and use that in subsequent requests to the site. You may also have to deal with redirects, depending on the site.
I have found nodejs actually quite useful for scraping pages since it has plugins that provide dom selectors (there is a jquery plugin) - you're using javascript for server-side programming.
Check if the site has API and if the site provides that, will make your life a ton easier.
Some banks like BankOfAmerica have applications that already do this - they aggregate your accounts and bills from other sites, see if your bank can do this.
Related
There several threads on SO regarding this, but I just need to know how to READ a cookie from siteb.com on sitea.com that opens siteb.com on a iframe, IF this is really the recommended way to go.
Based on this post the author says:
Cookies can be read in an iframe if they were set outside of the
iframe
But I have no idea how to achieve this. Let me explain a bit more about what im trying to design so maybe you can point me in the right direction.
siteb.com is my website, where users login and signup, each time they do, a cookie is set like many normal authentication systems.
sitea.com is a generic site, where I can insert html and javascript code, from sitea I need -if exists- to read the login cookie of siteb. I think an iframe on sitea loading siteb will do the trick, but again, i have no idea how to access that cookies inside the iframe. Is there an easy way to do this?
Another approach i was thinking is to use cross domain iframe communication techniques, but they are not elegant, way complex and some of them fails in certain browsers, the most robust ones uses jquery but I don't want to insert jquery on sitea.
Here's what you need: http://easyxdm.net/ - load this library on both sitea.com and the siteb.com iframe. It makes cross-domain parent-iframe communication "just work" in every browser, using the fastest method avaliable in each browser. (Also, the author, https://stackoverflow.com/users/128035/sean-kinsey does a fantastic job of helping anyone who has trouble with the library - just check the mailing list archives)
Then add a tiny bit of JavaScript to your siteb.com iframe to read cookies and pass them to easyxdm and then add a bit of JavaScript to sitea.com to set up easyxdm (including creating the iframe, I think) and receive the cookie value from it. There's lots of examples on the website to help you get started.
I am trying to develop a back-end Ad Checking application in PHP. We have lots of places where ads can be shown and almost all of them has its unique requirements (they are shown inside games, that is why everyone is different in size, weight, format, etc.). Since this can cause lots of confusions on campaigns targeting different games (with the agencies sending us ads with the wrong formats), we need to check every ad to make sure it works as expected.
The application works fine if our clients send us the Ad File to check. But most of the time they send Adserver tags, so they can keep track of the results, and those tags generally are javascript and iframe tags.
The problem is how to securelly allow the insertion of javascript and iframe code, and render that to the page on the postback page (for visual checking), on all browsers, without risking XSS problems?
At the moment, I've managed to post the code and print to the postback page on all browsers, but Chrome, because of the security measures. But since I'm gonna have to tackle that issue anyway, how can I make it work on all browsers and still be sure the application is safe?
Thanks.
Decio
If you include 3rd party javascript code, you are always risking XSS. Let's say, you're testing the code from ad provider:
<script src="http://some3rdPartySite.com/script.js"></script>
Script can be altered to be displayed and to cause different results on target domain and on all other pages. That is, you can see how it wants to be showed to you, but you can't be sure that it will be the same in some of your customer's games.
Ad provider can change the script on it's server at any moment, possibly after your tests.
If you do not trust your ad vendors, I suggest to let them choose image + URL or iframe URL + dimensions. You can do "bad stuff" with Iframe too (like frame-busting thus redirecting from your customer page), but you still must obey cross-domain policy. This is not the case if you let javascript code to be executed.
You could also provide your own analytics for ad vendors to fill their needs.
Anyway, for testing javascript results you could use Selenium drivers with any browser: http://seleniumhq.org/. It let's you to load any page, execute javascript code and get results. In this way you could load your javascript code, search DOM for nodes and check their dimensions etc.
Also, you can look at sahi: http://sahi.co.in/w/
I'm looking at a domain registration site that looks like it uses jquery to process users data inputed and to register domains.
What I was wondering is if it's possible for users to be able to fill in data on a form on my website and then when the user is ready to complete payment, be taken to the actually domain registration site where all the data they typed in on my site will be posted to the domain reg site.
So basically, the users fills in a load of info on my site, AND attempts to check for domain availability on my site. Once the users has found the domain they want, they will be redirected over to the actual domain reg site where all their info will be posted.
Now I know if the domain reg site used PHP to process all the stuff, it wouldn't be a problem. But they don't use PHP.
Do you guys reckon this could be possible?
I'm not sure this would be possible in any amount of time that would make it worth it to you. Without knowing any of their back-end code, it's going to be extraordinarily difficult. Edit: I should add that I did look through some of their jQuery code and it looks as though they're using ajax .post() to submit data. Where this data goes and what responses are expected is anyone's guess, though...
That said... there are quite a few domain registrars that offer real APIs to let you do what you want... or even let you go one step further and offer the ability to register domains directly through your website. Sometimes you can set your own price, as well.
Here are links to some of these APIs:
Namecheap: http://developer.namecheap.com/docs/
GoDaddy: http://www.godaddy.com/reseller/domain-reseller-api.aspx
eNom: http://www.enom.com/resellers/Interfaceinfo.asp
I'd personally recommend NameCheap, but for the purposes of your question, any of these should do.
I can't make any promises but say you used jquery ajax to pull in the form the would have to fill out. Said form would then be on your client side so in theory I think you could use their input ids to fill out the form using javascript/jquery. All this would technically be client side. To bad that other site does not have an api for purchases.
Do you have control over the domain registration site? There are many ways you can send the user's input over to that site, but of course it has to be looking for this posted data and know how to handle it. PHP is not necessary to handle the data that is passed in. For example, if you send your info to the domain registration site via a form GET method, the info will become part of the URL, which can be accessed and parsed via javascripts window.location property.
Is it possible to get remote username when I get a referral link without involving any server side code from the referral link?
Do you mean like if I clicked a link to your site on Stack Overflow, you would want to be able to see that my username is "Agent Conundrum"? No, you can't do that without the help of the referring site. The only information you should be able to get is the (permanently misspelled) HTTP_REFERER in the $_SERVER superglobal array, which tells you the page the user came from. Even then, there are ways to block or change this so you shouldn't count on it being set (especially since it wouldn't be set if the user navigated directly to your page via the address bar).
Frankly, I wouldn't want to use a site that leaked personal information (and for some sites, even the username qualifies as personal information), and I wouldn't want to use a site that tries to harvest such leaked information without my knowledge.
Generally, any site where you have a legitimate reason to broadcast this information would have some sort of API built in, like FacebookConnect. Even then, it should be strictly opt-in for the user.
As a general thing: no. The HTTP protocol does not involve the transmission of a remote user name.
Hey, it could help to answer if you would be a little more specific on which kind of service are you trying to fetch the data from.
Large/Public services tend to have somekind of an accessible API that you can fork on your referrer, but other than that its mostly that you need to regexp the site and know the structure of the HTML pretty much.
I am developing an online learning system (PHP, MySQL and Javascript). I would like to track what pages and how long each users spent on each page. Ideally, I would like to record this in a MySQL database. My question is 2 fold:
1. What kind of fields would I include in my db table to record multiple pages accessed?
2. Is this problem best approached by server side only or by using javascript ? e.g. server side: hidden form fields with a page id attached, page id is passed to db and recorded?or Javascript: record all actions in Javascript variables and somehow pass to db at end of session?
Really I am just looking for some high level guidance on an approach as opposed to code snippets.
GF
PHP isn't my normal language, but I would think about creating a module of code that can be called from the top of each of your scripts, that basically logs away "I served this page, with these form variables, at xxx ... ". To be more precise, I would record that in a table.
If you need to know when the user left your page, for a page on another site or perhaps shut down their browser for instance, then a purely server side solution isn't going to cut the mustard. In that event, you are going to have to start thinking about JavaScript, and intercepting events - such as the onUnload event...
Have a read here...
While I know of no solution that can track individual users out of the box (I'm sure there are some), I am pretty sure you could customize Piwik to do this. Piwik aims to be a self-hostable alternative to Google Analytics. It is open source and build on Zend Framework and MySql.
Piwik collects usage statistics through a JavaScript tracking code and a Webbug image for fallback. Basically, what you would need to do is pass the logged in user's user id to the tracking script and then write a plugin that knows how to handle this information.