PHP script to read html after logging in - php

I am trying to create a php script which logs into a website and then navigates to (potentially multiple) web pages which require a user to be logged in to see them. I think I have successfully logged in using cURL but I don't know how to "stay logged in" to read file contents after having logged in. I suspect my issue has something to do with using the same cookies file but I am unsure. This is what I currently have:
<?
//login form action url
$url="https://randomwebsite.com/users/sign_in";
$postinfo = "utf8=%E2%9C%93&authenticity_token=ncgU2234iEMObg3334d3Iag%2BPBrmEerybFZ3X2fvVsUTmpjkPDQMgteeGlVDxFRfHjmHbvIYaTchvM2psKFzI%2BAHIpCw%3D%3D&user%5Botp_attempt%5D=step_1&user%5Blocale%5D=en&user%5Blogin%5D=randomuser&user%5Bpassword%5D=123456";
$cookie_file_path = "cookies1.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
$curlexecresponse = curl_exec($ch);
//echo("\ncurlexecresponse: " . $curlexecresponse);
// Now that we're logged in we can finally read the web pages we want to...
$url2 = "url to members only page";
curl_setopt($ch, CURLOPT_URL, $url2);
curl_setopt($ch, CURLOPT_POST, 0);
$data = curl_exec($ch);
echo("\n\n data is: " . $data); // this is empty
curl_close($ch);
?>
How can I read the HTML in pages after having logged in?

You need to send the cookie with every request after login in.
$url2 = "url to members only page";
//Add this line
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_URL, $url2);
curl_setopt($ch, CURLOPT_POST, 0);

Related

Login to Airbnb via CURL

I am trying to do API calls to Airbnb, but you must be logged in to make these calls, and with no public API, I am left to try and come up with a solution.
I am first trying to login via CURL, I am trying to use this piece of code but I'm not having much luck.
$url = "https://www.airbnb.co.uk/login";
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//for certain features, set the cookie the site has - this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, $url);
$html_data = curl_exec($ch);
curl_close($ch);
echo $html_data;
The response I get is:
{
success: false,
redirect: ""
}
Where am I going wrong ( I have declared the vars, username, and password but have obviously left that out).
You can't login to Airbnb by curl because Airbnb uses google captcha which can't be resolved. This is why it returns success: false.
If you want to do something with their application you should try to go here and apply as partner in order to use their API.

PHP Curl Login Captcha

I'm trying to do a curl to use a private webpage after the login, the login form call an action to website/entrar/sair/ with the post vars - login, senha(password), lembrar(remember - cookies), g-recaptcha-response (captcha system) and logar. After a sucessful login the page is redirected to website/private (the page I want to use). The problem is that I dont know what to do with captcha, after I get a sucessful login I get a cookie that is valid for next 24 hours. So what I need to do is, when I access localhost/index.php, it will show the page website/private, if the cookie has expired, it will show the anti captcha from website/entrar/sair/, so I can do a manual verification and get another cookie valid for 24 hours.
The code I have so far:
$path = $_SERVER["DOCUMENT_ROOT"]."/teste/ctemp";
$url="http://website.com/entrar/sair/";
$postinfo = "login=xxx&senha=xxx&g-recaptcha-response=xxxx&lembrar=on&logar=";
$website="http://website.com/private";
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
//page with the content I want to grab
curl_setopt($ch, CURLOPT_URL, $website);
$html = curl_exec($ch);
curl_close($ch);
echo $html;
The post form: Image

PHP cURL URL Not Found

I am new to using cURL....whenever I run this I get a page that says "The requested URL /files/ was not found on this server." But, if I go to the link on a browser then it shows up. Please assist. I have looked at similar questions but could not find a solution to my problem.
<?php
$username="user";
$password="pass";
$url="http://website.com/login/";
$cookie="cookie.txt";
$postdata = "login=$username&pass=$password";
//set the directory for the cookie using defined document root var
$dir = DOC_ROOT;
//the info per user with custom func.
$path = $dir;
//login form action url
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_exec($ch);
//page with the content I want to grab
curl_setopt($ch, CURLOPT_URL, "http://website.com/files/");
//do stuff with the info with DomDocument() etc
$html = curl_exec($ch);
echo $html;
curl_close($ch);
?>
$postdata = array(
'login' => $username,
'pass' => $password
);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
Change
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
to
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
Make sure you've write permission on the current dir so curl can create the cookie file cookie.txt
you should end-up with something like :
<?php
$username="user";
$password="pass";
$url_login="http://website.com/login/";
$url_files="http://website.com/files/";
$cookie="cookie.txt";
$postdata = "login=$username&pass=$password";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url_login);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, $url_files);
$html = curl_exec($ch);
echo $html;
curl_close($ch);
?>
NOTE:
Sometimes, less is more.
The URL you provided http://website.com/files/ has a redirection, so CURL has to have option enabled to follow redirects.
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
While in your code you have
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
0 means false, so you are disabling following on HTTP 30x

Login via curl and parsed data in php

I try to login to a forum via curl. The forum generates a unique id everytime the login page is displayed. That kinda bugs me :( I am able to parse the id and use it. But somehow the id is changed when I try the actual login.
My code so far:
$ch = curl_init();
$url ="http://theforum.dk/index.php/Login/";
$cookie="cookie.txt";
//Fetching the id
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch); (1)
$pos = strpos($output, '"t"')+11;
$pos2 = strpos($output, " />",$pos)-1;
$unique_id = substr($output, $pos,$pos2-$pos);
echo $unique_id; //Shows correct id
//Filling the POST array and submitting it
$arrSubmit="username=TESTUSER&action=login&password=TESTPASS&useCookies=1&url=&t=".$unique_id;
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $arrSubmit);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookies);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookies);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, false);
//Exectute and display result
echo curl_exec($ch); (2)
The output is something like "your id isn't valid anymore". The id at (1) differs from the id in the page source at (2) :(
The login form just a basic HTML form- Submitting the data isn't the problem, just using the fetched id from the start.
Do you guys got any idea?
Any help is appreciated :)
Hi my curl login example.
Good luck
$login_email = 'mehmet';
$login_pass = '112233';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.mysites.com/giris');
curl_setopt($ch, CURLOPT_POSTFIELDS,'username='.urlencode($login_email).'&password='.urlencode($login_pass));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "http://www.mysites.com/package-details/237");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curl_handle = curl_init ("http://www.mysites.com/package-details/237");
curl_setopt ($curl_handle, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, true);
$page = curl_exec($ch);
echo $page;

What am I doing wrong? CURL PHP Cookies

I'm using curl to login to a website, It manages to log me in and display the page back saying "Thanks for logging in Jack Brown" and I can see the member area.
However a cookie file is not being created on my server "/tmp/cookie.txt"
Since I have logged in, I then want to use curl again to retrieve data from the members area but when running this part of the page I just get the "Please login to continue".
The first bit of code is for logging in (this logs me in okay but don't create a cookie file):
<?php
$email = 'email#here.com';
$password = 'passwordhere';
$rememberMe = '';
$redirect = '';
// initial login page which redirects to correct sign in page, sets some cookies
$URL = 'https://www.website.co.uk/home';
$coookie = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $URL);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$page = curl_exec($ch);
$URL2 = "login url is normally here"; // this is our post url
$postdata = "_58_login=".$email."&_58_password=".$password."&_58_rememberMe=".$rememberMe."&_58_redirect=".$redirect;
$post = substr($post, 0, -1);
// set additional curl options using our previous options
curl_setopt($ch, CURLOPT_URL, $URL2);
curl_setopt($ch, CURLOPT_REFERER, $URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$page = curl_exec($ch); // make request
// try to find the actual login form
if (!preg_match('/Thanks for loggin in/is', $page, $form)) {
die('Erorr: Could not login!');
}
var_dump($page); // should be logged in
// END OF LOGIN
This next bit of code is in the same document, and is used to open another page in the members area which I can pull content from however this page just returns saying that i'm not logged in:
$URLOPEN = 'http://www.website.co.uk/membersareacontent';
$URL5 = 'http://www.website.co.uk/thanksforlogginin';
curl_setopt($ch, CURLOPT_URL, $URLOPEN);
curl_setopt($ch, CURLOPT_REFERER, $URL5);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$page2 = curl_exec($ch);
var_dump($page2); // should be show members content
Perhaps because the variable name has an extra o when declared at the top then the one you're using in your cURL later on:
$coookie = tempnam ("/tmp", "CURLCOOKIE"); // Notice coookie with the extra o
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie); // Spelled with only two o's here
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie); // Spelled with only two o's here
Edit: sbeliv01 has point that cookie is misspelled
If not try to add this at top of your code to display if there is any error:
error_reporting(E_ALL);
ini_set("display_errors",1);
And this code to check if file is available to write, and make sure cookie file is writeable:
// change $cookie_file_path to yours
$fp = fopen($cookie_file_path,'wb') or die("can't open cookie file");
fclose($fp);
Also i noticed you have not set your cookie options here:
// set additional curl options using our previous options
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $URL2);
curl_setopt($ch, CURLOPT_REFERER, $URL);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);

Categories