I've got NTLM (Active Directory) based service, and I need to write a PHP application. Normally, users are logging in to website with Activre Directory credentials, and it's ok.
But what I want to do, is to let them type in their credentials to PHP-written site, which in next step will use cURL to authenticate users to that Active Directory based site where they normally log in.
And this part is hard. I need then to keep session of users that through PHP cURL script authenticated to Active Directory based site in order to use them again later
(CRON querying site to determine that it has changed and automatically do some operations when this happens, which normally user has do manually).
In order to NOT store their credentials to authenticate again when this change happens, I somehow need to store NTLM session in PHP cURL site to every user that authenticated to
that site through this PHP cURL site.
My question is: Is that even possible?
Thanks in advance.
#Willem Mulder
The code you've posted actually does cookie-storing, but that is not my point becouse I've already done that (sorry for not writing it before). What I got so far is:
$cookie_file_path = dirname(__FILE__) . '/cookies.txt';
$ch = curl_init();
//==========================================================================
curl_setopt($ch, CURLOPT_USERPWD, $username. ':' . $password);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_FAILONERROR, 0);
curl_setopt($ch, CURLOPT_MAXREDIRS, 100);
//==========================================================================
$ret = curl_exec($ch);
By using options CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR, cURL does the cookie storing in local file "cookies.txt". The problem is, that when I comment CURLOPT_USERPWD option (after authenticating and storing cookie, so theoretically I have session), I cannot authorize to website. Perhaps it reinitializes NTLM Handshake authorisation and is expecting username and password, which I don't want to store.
I want to store session info only, to provide service this session info and omit second authentication, but cURL seems to not take this data from cookie file, and REWRITES it with not relevant data send to me from service as response to NOT AUTHRORISED access request.
Well, yes you could
$ch = curl_init('http://www.google.com/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// Get headers too with this line
curl_setopt($ch, CURLOPT_HEADER, 1);
$result = curl_exec($ch);
// Get cookie
preg_match('/^Set-Cookie:\s*([^;]*)/mi', $result, $m);
var_dump(parse_url($m[1]));
// And then of course store it somewhere :-)
As seen here how to get the cookies from a php curl into a variable
Related
I am trying to create a portal such that users use SSO to access all services in the company I work for.
My goal is for the users to log in to my portal, then they will be automatically logged in to all the websites of our company and they will only need to click the icon of the website to access it without the need to enter their credentials every time, we are using Active Directory for authentication in all these websites.
I inspected the login form for each website and got the URLs of the login scripts along with the variable names of the username and password fields, then I tried to send POST requests to those scripts using cURL in PHP with the credentials in the "CURLOPT_POSTFIELDS" option.
$cookie_file_path = getcwd() . '/cookie.txt';
$agent = $_SERVER['HTTP_USER_AGENT'];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,'http://example.com/login.php');
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
$loginpage_html = curl_exec ($ch);
curl_close ($ch);
$CREDENTIALS = "username=test&password=test";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,'http://example.com/login.php');
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$CREDENTIALS);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_REFERER, 'http://example.com/login.php');
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
$logon_result = curl_exec ($ch);
echo $logon_result;
Current output:
Users are automatically logged in to all the websites when they login to my portal, but when trying to navigate it looks like they haven't logged in in the first place as if the session is destroyed even though it first shows that you're logged in.
Expected output:
Users are automatically logged in to all the websites when they login to my portal and can access those websites as if they are normally logged in.
And as a beginner, I want to ask is this actually a good way to achieve SSO in PHP and Active Directory or is there a better way to do this.
Thank you.
I think your code is failing to log in users because the session cookies aren't being saved to the clients web browser. For security reasons your site won't be able to create a cookie for another website. Since your users don't have the session cookie for the target site they will be treated like a new user.
A more common way to achieve a single sign experience would be to use SAML. Since you mentioned your company used Active Directory you can use ADFS as the Identity Provider. Some of your enterprise applications may already support SAML authentication.
I'm trying to record data from Philippine Stock Exchange website. I have found that they have an endpoint which is http://www.pse.com.ph/stockMarket/companyInfo.html?method=fetchHeaderData&company=29&security=146
I can clearly access it using any browsers except when I go into incognito mode where I'm being shown with a content saying Access Denied and it never stops loading. When I try to access it using PHP I'm quite sure that what is happening is the same as the later.
I'm trying to access it using PHP to no avail, here are the attempts I tried:
file_get_contents
cURL with user agent
cURL with temporary cookies
Tried all in localhost and in live server.
Code:
$c = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.pse.com.ph/stockMarket/companyInfo.html");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_COOKIESESSION, true );
curl_setopt($ch, CURLOPT_COOKIEJAR, $c);
curl_setopt($ch, CURLOPT_COOKIEFILE, $c);
curl_setopt($ch, CURLOPT_POSTFIELDS, "method=fetchHeaderData&ajax=true&company=29&security=146");
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER["HTTP_USER_AGENT"]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
var_dump(curl_exec($ch));
curl_close ($ch);
I don't have any clear idea on why and how does this happen. Can someone explain to me why it happens and what are the possible solutions (PHP only if possible)
I have reviewed other developer's approach on this API (They all implemented it using Java) and it is just a simple POST request and it is done. I have not verified though if their code is still working. I can't post links to their repository (limited).
SOLUTIONS:
Problem 1. Can't access API
$posts = array(
"method"=>"fetchHeaderData",
"ajax"=>"true",
"company"=>29,
"security"=>146
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.pse.com.ph/stockMarket/companyInfo.html");
curl_setopt($ch, CURLOPT_POSTFIELDS,$posts);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
var_dump(curl_exec($ch));
curl_close ($ch);
It seems I have two different problems. I can now access and use the API using the code above. No need for other options. Turning the post data into array fixed the problem.
Problem 2. Access Denied
On the problem about the Access Denied, it is cookie related. Answered below by #Wayne.
Unfortunately, I can't accept two answers.
Try this solution. convert your post data in array then pass this array in CURLOPT_POSTFIELDS
$posts = array(
"method"=>"fetchHeaderData",
"ajax"=>"true",
"company"=>29,
"security"=>146
);
$c = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.pse.com.ph/stockMarket/companyInfo.html");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_COOKIESESSION, true );
curl_setopt($ch, CURLOPT_COOKIEJAR, $c);
curl_setopt($ch, CURLOPT_COOKIEFILE, $c);
curl_setopt($ch, CURLOPT_POSTFIELDS,$posts);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER["HTTP_USER_AGENT"]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
var_dump(curl_exec($ch));
curl_close ($ch);
It is because they have their server setup to stop you from doing that. They are securing the data with a cookie.
Cookie details
When you visit the site http://www.pse.com.ph/stockMarket/companyInfo.html it gives you a cookie as it knows you are a human visitor.
In your browser tools enter
document.cookie
to see your cookie. It will provide you an individual the data because you have the cookie.
Remove the cookie
document.cookie = "JSESSIONID=; expires=Thu, 01 Jan 1970 00:00:00 UTC; path=/;";
and visit
http://www.pse.com.ph/stockMarket/companyInfo.html?method=fetchHeaderData&company=29&security=146
without going to get a cookie http://www.pse.com.ph/stockMarket/companyInfo.html first you will get the 403 (Forbidden)
Also they do not have jsonp with a callback so an ajax request will violate the cross domain security. Requests for the JSON must be from pages that originate from their domain or an approved domain.
Why would they do that.
Likely their licence to the information does not allow them to give it to other websites, or they need/want to get paid to provide the information to other websites. Or they have terms of use for the information.
Where can you get the data ... data wants to be free
I don't see anyplace on their site http://www.pse.com.ph where they have API information and how to request permission to access it.
Programable web has been the number one source for finding APIs, they have 96 stock APIs listed ... Obviously I can not just copy their data and past it here, but one of these API may work for you?
I'm trying to use cURL to login to a website, but it doesn't seem to be working. I started by testing it on my own site which works, but when I try on the site that I actually need it wont login, and just gets the login page.
$loginUrl = 'https://www.ekomi.co.uk/login.php';
$postData = 'login_email=-----&login_passwort=-----';
$ch = curl_init();
$agent = $_SERVER['HTTP_USER_AGENT'];
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_URL, $loginUrl);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postData);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies/cookies.txt');
$postResult = curl_exec($ch);
print_r($postResult);
It does give an error that $_SERVER index is undefined, but it still allowed me to login to my site, also I am assuming that the $postData should be the names of the user and password fields.
Is there anything wrong with the script that could be the problem, or could it maybe be the way which the site logs in?
Edit
The problem seems to be because of the # symbol in the login email, I've tried putting %40 as suggested in other questions but this doesn't work.
First thing you must navigate to the page and get the csrf (Cross-site request forgery-basically it checks that the login form is seen by the user, sets a random hash in the session and checks if you give it back) and then make the post.
So a little trivia first..
There is written in ASP.NET website, which uses NTLM protocol to authenticate users that want to log in. It's perfectly ok when they normally use it, they type in website URL, they provide their credentials, authenticate and maintain session in web browser.
What I want to do, is create PHP website that will act as bot. It is my companys internal website and I am approved to do so. The problem I run into, is managing session. Users will be able to type in their credentials in my PHP website, and my PHP website will authenticate them to target site, using cURL.
The code I got so far is:
$cookie_file_path = dirname(__FILE__) . '/cookies.txt';
$ch = curl_init();
//==============================================================
curl_setopt($ch, CURLOPT_USERPWD, $username. ':' . $password);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_FAILONERROR, 0);
curl_setopt($ch, CURLOPT_MAXREDIRS, 100);
//=============================================================
$ret = curl_exec($ch);
Above code logs in to target website by cURL (which manages NTLM handshake, as it seems), and fetches websites content. It also stores Session ID that is sent back in cookie file.
What I'm trying to do next, is comment the CURLOPT_USERPWD option, in hope that this script will use session ID stored in cookie file to authenticate previously logged in user in second execution of this script. It could get rid of user credentials and do not store it anywhere that way, becouse it is not safe to store it in manually created session, database, or anywhere else.
I need this becouse bot will be using CRON to periodically check if website status has changed and perform some user actions as reaction to this. But to do this, user first must be authenticated, and his username and password must not be stored anywhere, so I have to use session information estabilished when he initially logged in.
CURL seems to NOT DO THIS. When I execute script second time with commented CURLOPT_USERPWD option, it does not use stored cookie to keep beeing authenticated. Instead, it REWRITES cookie file with not relevant data send to me from service as response to NOT AUTHRORISED access request.
My questions are:
Why cURL doesnt use stored session information to keep beeing authenticated?
Is there any way to maintain this session with cURL and NTLM protocol based website?
Thanks in advance.
A few Month ago I had a similar problem then you. I tried to get a connection to a navision soap api. Navision use the ntlm authentication. The problem is that curl doesn't native support ntlm so you have to do it yourself.
A blog post that helped me a lot in this situation was the following:
http://rabaix.net/en/articles/2008/03/13/using-soap-php-with-ntlm-authentication
** Edit
Sorry i misread you question.
You problem is simple.
Just receive the header from a request with this line
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
You can then get from the result of curl_exec function, the Set-Cookie header.
preg_match('/^Set-Cookie:\s*([^;]*)/mi', $ret, $match);
$cookie = parse_url($match[0]);
Now you can store it somewhere, and use it on the 2ten request.
I have the same problem and i solved it using curl_setopt($ch, CURLOPT_COOKIEFILE, ""); line of code. The string should be exactly empty.
I want to login via curl and maintain the cookies and session information for further calls.i have created cookie text file in the same directory and used the CURLOPT_COOKIEJAR ,CURLOPT_COOKIEFILE to maintain the cookie in CUL. whenever i had try to call login api it take the old cookie and show the previous user information. i need to maintain different user cookies and maintain session like normal browser handle. how to do that. any one give idea to do it.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HEADER,0); // TRUE to include the header in the output.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // it will follow with server redirects
curl_setopt($ch,CURLOPT_AUTOREFERER,1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);//ssl certificate verifyer
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2); //ssl certificate host
// Set the location of and send the cookies
curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) . "/cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, dirname(__FILE__) . "/cookies.txt");
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
$result = curl_exec($ch); //execute curl and store data in result
You can modify
dirname(__FILE__) . "/cookies.txt"
Into something like
dirname(__FILE__) . '/user_cookies/' . $username . '.txt'
You will need to sanitize username for that line so that it will not contain any invalid characters.
Also, set /user_cookies/ permissions to something like 777.
This way you won't need to check if user has cookies or not. If not, the file will be created. If user has them, existing file content will be used.
You could also store cookies in database, but that's way more complicated.