I am using cURL to display the contents of a page behind a login system. I am able to successfully login and display the first page behind the login system, but any subsequent pages are unable to be displayed.
My understanding of the problem is that cURL follows the headers that are provided after logging in. So, if the order is login.php -> home.php, and I want to go to account.php, I would need another header pointing to that page.
Is that correct? Can I use cURL to display the contents of other pages after logging in?
You probably need to save and transmit cookies. This can be done in PHP cURL like this:
$ch = curl_init();
// set your regular options
curl_setopt($ch, CURLOPT_URL, 'http://somedomain.com/login.php');
curl_setopt($ch, CURLOPT_POSTFIELDS, array('user'=>'foo', 'pass'=>'bar'));
// set where cookies are saved
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
// set where cookies are retrieved from when sent to the server
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
// execute login
curl_exec($ch);
// do another request
curl_setopt($ch, CURLOPT_URL, 'http://somedomain.com/restricted_page.php');
curl_setopt($ch, CURLOPT_HTTPGET, 1);
curl_exec($ch);
You need to handle cookies. They're needed for the session authentication.
Get the Cookie header from the first call and send it with all subsequent calls.
You must know how login pages (usually) work. When you log into a site the site will send you a session cookie that you will send again to the website everytime you request a new page. So if you want to emulate the login you must store cookies returned by the login and then include them in every other request to the site.
Related
If I go to a website, example.com/page1.php, it is a log-in page. When I log-in, it takes me to example.com/page2.php. If I close my browser and come back to page1 later, I’m still logged in and it automatically takes me to page2. That means there’s a cookie set and it knows I already logged in.
I want to use file_get_contents to get page2.php. When I try it, I get the contents of the log-in page instead. I assume that’s because file_get_contents doesn’t know a cookie is set and page2 is saying, you shouldn’t be here, you’re not logged in, so it bumps me back to page 1.
I realize I can use cURL to do the log-in, create a cookie and get the contents, like this….
$url = 'https://www.example.com/page1.php'; // the url of the login page
$post_data = "urerid=myusername&password=mypassword "; // The login data to post
$ch = curl_init(); // Create a curl object
curl_setopt($ch, CURLOPT_URL, $url ); // Set the URL
curl_setopt($ch, CURLOPT_POST, 1 ); // This is a POST query
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data); //Set the post data
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // Get the content
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // Follow Location redirects
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt'); // Set cookie storing files
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
$output = curl_exec($ch); // Execute the action to login
My problem is , I don’t want to log-in again (for reasons I don’t want to get into). Is there a way to let file_get_contents, cURL, or some other function, know I’m previously logged in and get the contents of page2. Since example.com is setting a cookie, can I access that cookie somehow and use it to avoid logging in again?
Why it wont work:
If the website is creating security cookie against xss , you cant simply take one user cookies and send request from diffrent IP while using them.
Even if the website is not using security hash , you cant access cookies belongs to a diffrent domain due security resones (you dont want that gmail.com could access your microsoft.com cookies)
to cut it short - the only way that could work is by :
Use SSO (with partner to the destination domain).
Use Cross-Domain support (with partner to the destination domain).
Get Access Tokens (like facebook is doing) if supported by the destination domain.
Request your users to login from your domain (by trusting you - which is bad) in order to let your site be able to access the other domain data.
I am trying to use a button on my php web-application to launch a logged-in session on another website. In other words I want my application to:
open a new tab/window (achieved)
go to another website + login or
(alternatively) collect the session data needed for the target site to consider the current browser logged in.
This is achieved (in an incomplete manner with the following code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$postdata);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);
curl_setopt($ch, CURLOPT_REFERER, $url);
$result = curl_exec ($ch);
curl_close ($ch);
print $result;
This successfully visualise the "logged-in" page of the remote site but whenever I click on any of the functionalities of such remote site I get and obvious 404. This is because I am just printing the output of the successful login via cURL and my browser is not dealing with the remote application on the target website. E.g. my address bar says I am in local.dev/loggedin.php instead of being at secure.targetsite.com/loggein.php.
This maybe helpful: Once logged-in via the browser, the target website sets a session cookie that allows the session to survive for a certain amount of time so that may also be useful. Can my web-application just fetch and store the session data from the auth procedure carried out by curl and use it to login?
This might not be possible to be done via cURL..
I was thinking of just parsing the response header for the cookie and use php setcookie() but it does not work: I get bounced by the remote app as if I was never logged in.
Please be patient, I am not an expert in the use of curl.
I have done that for a few of my own applications, but it should work for almost anything that can be logged in via an html form submission. You can't use curl for this because it is running on your web server (whether that is on your local machine or in the cloud somewhere is irrelevant) and not actually being run by your browser. Your PHP application needs to open a new tab/window with a page that includes an HTML that includes all necessary fields, method="get" or "post" as appropriate, and action="the destination login URL". Then just add an automatic form submission - e.g., with jQuery $('#form_id').submit() on page load.
I have created a simple classic ASP script that will take a username and password from a post and create session variables. This script works fine if i use a standard html form and redirect to this page. I have a php site and I want to log users into both websites when they log into the php site. To do this i wanted to add a curl request to the login script in php. This would send the password and username over to the script and create the session variables. The response i get from the curl request would suggest that it worked, but it doesnt seem to be saving the session.
Here is the curl request.
$postinfo = "username=".$username."&password=".$password;
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);
I dont want to paste the full asp script, but this is roughly how it works. The session persists when i login using a html form so i know its working correctly. When the curl request is finished executing it seems that the session variables are populated, but when i visit another page the session does not exist.
'do some stuff with the db to check if the credentials work.
if success = true then
Session("userid") = userid
Session("login") = "good"
Response.Write("Login successful - " & Session("userid"))
else
Response.Write("Login Failed")
end if
When i run the curl request the response is "Login successful - 123". This means not only is the login working, but its also setting the session value. The problem is that when i try to visit the asp site it does not detect any session data.
I have verified that the all links are pointing to https://www.website.com. Both websites are under the same domain name, just 2 different subdirectories/languages. They are both running on the same server.
I have a situation whereby when a page loads, I send some authentication data (in this case the associative array $data) which is verified by a script on another domain. Code below:
$cookie_path = 'cookies.txt';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.mysite.com/verify');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_path);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
$result = curl_exec($ch);
the site then sets a session (in this case I am using the codeigniter framework and sessions are set like: $this->session->set_userdata('logged_in', true); )
however when I load the external site in an iframe it does not seem to be able to detect that the session is set and redirects to the login page.
How do I ensure that my session cookie is being sent properly and can be accessed by an iframe?
Your curl script is running server side and storing the cookie for the second site there, but your browser is loading the second site in the client. You can share cookies across domains.
If you control the site you are attempting to create the session on, you may be able to pass the session ID to the PHP script, then generate the iframe URL dynamically, including the session ID as a query string, eg:
http://www.brainbell.com/tutors/php/php_mysql/Encoding_the_session_ID_as_a_GET_variable.html
Edit
To clarify, if you control the script on the second site, you can modify it to provide the SESSIONID of the authenticated session to your CURL script, which your PHP script making the cURL request can then incorporate into the dynamically generated iFrame src URL.
You can set cookies via:
http://php.net/manual/en/function.setcookie.php
However, you can't set cookies for domains outside of your script's domain.
I would like to make a php script that can capture a page from a website. Think file_get_contents($url).
However, this website requires that you fill in a username/password log-in form before you can access any page. I imagine that once logged-in, the website sends your browser an authentication cookie and with every consequent browser request, the session info is passed back to the website to authenticate access.
I want to know how i can simulate this behavior of the browser with a php script in order to gain access and capture a page from this website.
More specifically, my questions are:
How do I send a request that
contains my log-in details so that
the website replies with the session
information/cookie
How do i read the session
information/cookie
How do i pass back this session
information with every consequent
request (file_get_contents, curl) to
the website.
Thanks.
Curl is pretty well suited to do it. You don't need to do anything special other than set the CURLOPT_COOKIEJAR and CURLOPT_COOKIEFILE options. Once you've logged in by passing the form fields from the site the cookie will be saved and Curl will use that same cookie for subsequent requests automatically as the example below illustrates.
Note that the function below saves the cookies to cookies/cookie.txt so make sure that directory/file exists and can be written to.
$loginUrl = 'http://example.com/login'; //action from the login form
$loginFields = array('username'=>'user', 'password'=>'pass'); //login form field names and values
$remotePageUrl = 'http://example.com/remotepage.html'; //url of the page you want to save
$login = getUrl($loginUrl, 'post', $loginFields); //login to the site
$remotePage = getUrl($remotePageUrl); //get the remote page
function getUrl($url, $method='', $vars='') {
$ch = curl_init();
if ($method == 'post') {
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $vars);
}
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies/cookies.txt');
$buffer = curl_exec($ch);
curl_close($ch);
return $buffer;
}
The http pecl extension can do it, also PEAR::HTTP_Client, Snoopy and many many other libraries/classes.
If you (for whatever reason) you want to achieved this with file_get_contents, you can use the stream context options for the http wrapper to set POST and cookie parameters and stream_get_meta_data to read the response headers including the cookies.