I have to download a page to parse some value in it. I would like to use PHP, it download the page, parse the data and return html with results. But i have to login before on the site to get the target page. How can i do it with php?
Best option is to follow the logon process when logging in manually using a browser like Google Chrome. You need to enable the network monitor. Press F12 to enable the developer tools, navigate to tab 'Network' and enable the checkmark at 'Preserve Log'. You can optionally select the disable cache checkmark.
Then clear all history and cookies so you're sure the site doesn't logon automatically.
No you're set to logon manually through the website. Type the site's URL for the login page and watch the items in the developer tools roll by. When your login is complete, head over to the top of the list of items in the network and look for a POST entry in the second column. This usually indicates the browser posting the logon information to the website.
Most sites respond using a 30x response and place a cookie. Now you now how the site operates.
Have a look at my answer to a similar question: PHP curl login couldn't pass login page
and use the CURL library to first logon, receive the cookie and while keeping the connection open get the page after the login you need.
Related
This is a really interesting problem related to cookies, I believe.
My index.php has a form, and it posts to post.php. post.php then manipulates the data and uses curl to then post it to webinarjam.net, a 3rd-party service not controlled by me.
webinarjam.net then displays a short success message that basically just contains a unique URL. The unique URL is to a thank-you page (hosted by webinarjam.net).
This all works beautifully. But here is the problem:
Clicking the URL only works in Incognito mode (?!) (i.e. clearing browser cookies first). Otherwise, webinarjam.net simply displays "Internal Server Error".
I have no idea why the presence of cookies would PREVENT a page from displaying.
How could I change my post.php such that the unique thank-you page URL will display correctly even without using Incognito mode of the browser?
Figuring this out would enable me to simply automatically redirect the user to that thank-you page URL upon form submission. (Currently, redirecting would just bring her to a page that says "Internal Server Error".)
P.S. In case this helps, I've also used the Advanced Rest Client extension within Chrome to try to post the same query data to webinarjam.net; the resulting unique thank-you page URL is then able to be displayed without using Incognito mode. So... what I need help figuring out is: what difference exists between the way my post.php and the Advanced Rest Client extension are posting to webinarjam.net?
I finally noticed that it wasn't just Incognito mode that worked; it was any browser OTHER than my main one.
And I eventually figured out that it was because my main browser was logged into the EverWebinar site.
I guess EverWebinar barfs (fails) whenever a logged-in admin of a webinar submits a form to sign up for that webinar (even using a different email address).
It seems like upon form submission, the system notices the session/cookies of the logged-in admin, and it says "no matter what, someone who owns this webinar shouldn't be signing up for it."
I have an application that has a button which opens a blank page for linkedin oauthentication.
My question is, when the user completes authentication and processing for linkedin, how do I tell the original page that this process is complete?
I was thinking about creating an ajax method that tells the database that the user is in oauth and when they are complete we tell the same database that the process is over and the original page will find out.
Any ideas?
See: How can I do OAuth request by open new window, instead of redirect user from current page?
The trick is the window.opener property, available from the popup. Using that you could do something simple like a reload window.opener.location.reload() or possibly something more complicated using postMessage (in either case the code would live in the page that OAuth redirects to on completion).
After having completed an online registration process, I want to check if the user is using an iPhone, and in that case give the option of opening App Store to download the app. Here's what I've coded so far:
In PHP, check $_SERVER['HTTP_USER_AGENT'] for the presence of the substring "iPhone".
If so, output JavaScript code that, before redirecting to the welcome page, offers the possibility of going to App Store using a confirm box.
Redirect to itms-apps://itunes.apple.com/url-to-my-app using window.location = ... in JavaScript.
This works. However, when the user once again opens Safari, the page which I redirected from is still open. This doesn't make any sense in my case. I want to redirect to the welcome page regardless of whether the user chooses to open the App Store. If I try to write another window.location line below the first one to perform a second redirect, Safari simply skips the link to the App Store.
I've considered redirecting from a hidden iframe, placing some kind of timer on the second redirect, experimenting with different combinations of JavaScript and HTTP header redirects and so on. None of the solutions I've thought of so far seems really solid, though. How do I do this if I want it to work gracefully across browsers and versions?
The only way to do this is to use the welcome page itself to do the iTunes redirect.
I know how to load a page from another website and analyze it but the website I'm trying to load some pages from, doesn't let unregistered users to visit those pages. I do have a username and a password to load those pages normally in my browser, but I'm wondering if I could do it in PHP or not? :/
I'm not sure what information I should give you about the website but if what I already told is incomplete just ask what information I should give.
Thanks.
Most websites use cookies to store information related to your authentication status.
To get past this programatically, you'll have to send this information everytime you make a request. Here is how you can get it -
Using a tool like firebug, inspect the cookies the site sends when you login manually.
Write code to capture this information and send this cookie with subsequent requests
Note: Do check the ToS of the site you are trying to scrape. Some sites do not permit you to scrape or use their content without prior permission
Research php SOAP. It includes the tools to log in to another site, giving your php script access to the HTML which would normally be presented to the browser. BTW, this is called scraping
I would like to enable an "auto login" button for my users. By pressing the button the users will be logged in to a different site with the username and password that I have added inside the code.
My site uses php and this site is written on asp.
Is this possible?
Thanks in advance
You'd have to use CURL in PHP in order to send the POST data to your ASP script in the remote site.
Nevertheless, the ASP site might have some inner-validations which can lead to refuse your request, it's worth a try, though!
To set CURL to user POST, check out the setopt CURL function options, you have to set the CURLOPT_POST option to TRUE, but you might find that (depending on the ASP site), you need to activate/deactivate other options.
Cheers!
This depends on the website you're trying to login to.
In a website not using a key-based system for each visitor this can be achieved pretty easily.
First navigate to the page yourself that contains the login form and show it's page source.
Jot down every <input>-tag's name and value and determine which one is the username & password. Also note the form's action to see where the data is going towards.
Now you can use curl to send a user to the website, just inject the post data and apply your own username & password to it.
A. If you don't own the remote site:
Have you tried to post your authentication parameters directly to the ASP.net page? Then you could also try using CURL but either way, be rest assured that the site owner might lock you out anytime by implementing simple CSRF protection.
B. If you own the remote site:
You can share sessions with a unique key using a common database.