Log in and collect data from extranet page using CURL and cookies - php

I am writing a script which will log in to a SEP extranet page using cookie and grab some cookies using CURL (i need to collect data about prices of products on this page). My code looks like this:
//username and password of accounts
//trim-remove whitespaces from strings
$username = trim("John");
$password = trim("123");
//set the directory for the cookie using defined document root var
$dir = DOC_ROOT."/ctemp";
//build a unique path with every request to store
//the info per user with custom func.
$path = build_unique_path($dir);
//login form action url
$url="http://destination.com";
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
I get an error:
"NetworkError: 404 Not Found –
https://api.skype.com/detection/detection_as3.swf" . It connetcts to
the server but doesn't collect cookies properly/entirely, I receive:
active, sap-usercontext, sap-appcontext, sap-hostid and MYSAMSSO2,
which is not what I need.
Does anyone have an idea why this code doesn't work?
I tried to change curl_setopt parameters but it didn't worked. I also looked for similar questions or examples but I haven't found anything helplful

Related

Login to Airbnb via CURL

I am trying to do API calls to Airbnb, but you must be logged in to make these calls, and with no public API, I am left to try and come up with a solution.
I am first trying to login via CURL, I am trying to use this piece of code but I'm not having much luck.
$url = "https://www.airbnb.co.uk/login";
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//for certain features, set the cookie the site has - this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, $url);
$html_data = curl_exec($ch);
curl_close($ch);
echo $html_data;
The response I get is:
{
success: false,
redirect: ""
}
Where am I going wrong ( I have declared the vars, username, and password but have obviously left that out).
You can't login to Airbnb by curl because Airbnb uses google captcha which can't be resolved. This is why it returns success: false.
If you want to do something with their application you should try to go here and apply as partner in order to use their API.

PHP script to read html after logging in

I am trying to create a php script which logs into a website and then navigates to (potentially multiple) web pages which require a user to be logged in to see them. I think I have successfully logged in using cURL but I don't know how to "stay logged in" to read file contents after having logged in. I suspect my issue has something to do with using the same cookies file but I am unsure. This is what I currently have:
<?
//login form action url
$url="https://randomwebsite.com/users/sign_in";
$postinfo = "utf8=%E2%9C%93&authenticity_token=ncgU2234iEMObg3334d3Iag%2BPBrmEerybFZ3X2fvVsUTmpjkPDQMgteeGlVDxFRfHjmHbvIYaTchvM2psKFzI%2BAHIpCw%3D%3D&user%5Botp_attempt%5D=step_1&user%5Blocale%5D=en&user%5Blogin%5D=randomuser&user%5Bpassword%5D=123456";
$cookie_file_path = "cookies1.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
$curlexecresponse = curl_exec($ch);
//echo("\ncurlexecresponse: " . $curlexecresponse);
// Now that we're logged in we can finally read the web pages we want to...
$url2 = "url to members only page";
curl_setopt($ch, CURLOPT_URL, $url2);
curl_setopt($ch, CURLOPT_POST, 0);
$data = curl_exec($ch);
echo("\n\n data is: " . $data); // this is empty
curl_close($ch);
?>
How can I read the HTML in pages after having logged in?
You need to send the cookie with every request after login in.
$url2 = "url to members only page";
//Add this line
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_URL, $url2);
curl_setopt($ch, CURLOPT_POST, 0);

How would I log into a site with curl?

I've been trying to log into a site (www.namemc.com/login) with a curl script for about 5 hours now and I think I'm going crazy trying to do so.
I was wondering if you would be able to check what's wrong with my code. I think the site may be blocking it somehow, but I'm not sure.
Code:
<?php
$username = 'example#gmail.com';
$password = 'example';
$accountUrl = 'https://namemc.com/login';
$postdata = "email=".$username."&password=".$password;
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_NOBODY, 0);
curl_setopt($ch, CURLOPT_URL, $accountUrl);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$result = curl_exec($ch);
echo $result;
?>
curl is a tool to transfer data from or to a server. The command is designed to work without user interaction.
You should be using either a cookie or a Header to maintain a session in your code like:
cookie="cookie.txt";
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie);
Also have you tried arguements in double quotes instead of single .
and ensure send all form data which is needed in $postdata.

PHP Curl Login Captcha

I'm trying to do a curl to use a private webpage after the login, the login form call an action to website/entrar/sair/ with the post vars - login, senha(password), lembrar(remember - cookies), g-recaptcha-response (captcha system) and logar. After a sucessful login the page is redirected to website/private (the page I want to use). The problem is that I dont know what to do with captcha, after I get a sucessful login I get a cookie that is valid for next 24 hours. So what I need to do is, when I access localhost/index.php, it will show the page website/private, if the cookie has expired, it will show the anti captcha from website/entrar/sair/, so I can do a manual verification and get another cookie valid for 24 hours.
The code I have so far:
$path = $_SERVER["DOCUMENT_ROOT"]."/teste/ctemp";
$url="http://website.com/entrar/sair/";
$postinfo = "login=xxx&senha=xxx&g-recaptcha-response=xxxx&lembrar=on&logar=";
$website="http://website.com/private";
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
//page with the content I want to grab
curl_setopt($ch, CURLOPT_URL, $website);
$html = curl_exec($ch);
curl_close($ch);
echo $html;
The post form: Image

What am I doing wrong? CURL PHP Cookies

I'm using curl to login to a website, It manages to log me in and display the page back saying "Thanks for logging in Jack Brown" and I can see the member area.
However a cookie file is not being created on my server "/tmp/cookie.txt"
Since I have logged in, I then want to use curl again to retrieve data from the members area but when running this part of the page I just get the "Please login to continue".
The first bit of code is for logging in (this logs me in okay but don't create a cookie file):
<?php
$email = 'email#here.com';
$password = 'passwordhere';
$rememberMe = '';
$redirect = '';
// initial login page which redirects to correct sign in page, sets some cookies
$URL = 'https://www.website.co.uk/home';
$coookie = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$page = curl_exec($ch);
$URL2 = "login url is normally here"; // this is our post url
$postdata = "_58_login=".$email."&_58_password=".$password."&_58_rememberMe=".$rememberMe."&_58_redirect=".$redirect;
$post = substr($post, 0, -1);
// set additional curl options using our previous options
curl_setopt($ch, CURLOPT_URL, $URL2);
curl_setopt($ch, CURLOPT_REFERER, $URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$page = curl_exec($ch); // make request
// try to find the actual login form
if (!preg_match('/Thanks for loggin in/is', $page, $form)) {
die('Erorr: Could not login!');
}
var_dump($page); // should be logged in
// END OF LOGIN
This next bit of code is in the same document, and is used to open another page in the members area which I can pull content from however this page just returns saying that i'm not logged in:
$URLOPEN = 'http://www.website.co.uk/membersareacontent';
$URL5 = 'http://www.website.co.uk/thanksforlogginin';
curl_setopt($ch, CURLOPT_URL, $URLOPEN);
curl_setopt($ch, CURLOPT_REFERER, $URL5);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$page2 = curl_exec($ch);
var_dump($page2); // should be show members content
Perhaps because the variable name has an extra o when declared at the top then the one you're using in your cURL later on:
$coookie = tempnam ("/tmp", "CURLCOOKIE"); // Notice coookie with the extra o
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie); // Spelled with only two o's here
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie); // Spelled with only two o's here
Edit: sbeliv01 has point that cookie is misspelled
If not try to add this at top of your code to display if there is any error:
error_reporting(E_ALL);
ini_set("display_errors",1);
And this code to check if file is available to write, and make sure cookie file is writeable:
// change $cookie_file_path to yours
$fp = fopen($cookie_file_path,'wb') or die("can't open cookie file");
fclose($fp);
Also i noticed you have not set your cookie options here:
// set additional curl options using our previous options
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $URL2);
curl_setopt($ch, CURLOPT_REFERER, $URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

Categories