I would like to get the html of the page. After some google-ing, i found following code, which isn't working.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$passwd");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
echo $output;
edit: something like this then:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, '');
curl_setopt($ch, CURLOPT_POSTFIELDS,'username='.urlencode($username).'&passwd='.urlencode($passwd));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "");
$output = curl_exec($ch);
echo $output
This just shows the target page, it doesn't log in, nor does it return the html code.
edit 2: now tried with $data = array('username' => $username, 'passwd' => $passwd); and curl_setopt($ch, CURLOPT_POSTFIELDS,$data);
This shows the same as above + an error: Request Entity Too Large
That page does not use HTTP Basic Authentication.
You need to make an HTTP request to match what submitting the form would sent, then you need to process the response (which will probably involve storing cookies).
Related
With this code:
<?php
function request($url, $post, $cook)
{
$ch = curl_init();
$cookie_file_path = "vskcookies.txt";
$urll = 'http://auto.vsk.ru/login.aspx';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_USERAGENT,'"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Trident/7.0; Touch; .NET4.0C; .NET4.0E; Tablet PC 2.0)"');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
$result = curl_exec($ch);
echo "FIELDS:\n\n".$post;
echo "\n\nHEADERS:\n\n";
curl_close($ch);
return $result;
}
$result = request($_POST['url'], $_POST['data'], $_POST['cook']);
if ($result === FALSE)
echo('error');
else
echo($result);
?>
I am getting two cookies as I need but with http 411 error in body:
The same request, the same code, but in the end, right before curl_exec I add
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: application/x-www-form- urlencoded;charset=utf-8"));
As the result, I am getting correct body, but now only one cookie (I need both):
Another variants:
This code
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;charset=utf-8;Content-Length: '.strlen($post)));
Gives 411 error and correct cookies for reason.
This
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length:'.strlen($post)));
Causes nothing: just nothing happens.
Also after both (first)variants, vskcookies.txt contains only one (pool) cookie.
Why that?
It looks like, when I add header it erases request’s body(post fields).
UPDATE
For this code
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;'));
I just sitting and clicking on button that makes ajax request of the script above on my page, getting every time 411 error response (also cookies/time in header updates every time), but after about 10 clicks I got http 200 and the page I need. Then again many times 411 and then again a single 200. By the way cookies file still has only one cookie.
Wtf?
As I mentioned in question’s update, sometimes I get 200 response with two cookies and body I need.
So I wrote this is bad unstable solution, often it can remain about 20 seconds:
<?php
function request($url, $post, $cook)
{
$sta=0;
while($sta != '200')
{
$ch = curl_init();
$cookie_file_path = "vskcookies.txt";
$urll = 'http://auto.vsk.ru/login.aspx';
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_USERAGENT,'"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 10.0; Trident/7.0; Touch; .NET4.0C; .NET4.0E; Tablet PC 2.0)"');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $urll);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
//curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Cache-Control: no-cache;Content-Type: application/x-www-form- urlencoded;'));
//curl_setopt($ch, CURLOPT_POSTREDIR, 2);
$result = curl_exec($ch);
$sta = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
}
echo "FIELDS:\n\n".$post;
echo "\n\nHEADERS:\n\n";
return $result;
}
$result = request($_POST['url'], $_POST['data'], $_POST['cook']);
if ($result === FALSE)
echo('error');
else
echo($result);
?>
Bet there is a much better correct answer.
I want to use PHP to get this html content, the final aim I want to get m3u8 link
It requires log-in
http://tv24.vn/livetv/vtv1.html
This is information to test:
user: h2132704#trbvm.com
pass: 12345678
I tried to use curl POST but unsuccessful!
<?php
$username = 'h2132704#trbvm.com';
$password = '12345678';
$loginUrl = 'http://tv24.vn/livetv/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $loginUrl);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'user='.$username.'&pass='.$password);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$store = curl_exec($ch);
$return=file_get_contents("http://tv24.vn/livetv/vtv1.html"); preg_match('/file: "(.*?)"/', $return, $m3u8);
echo $m3u8;
?>
Any solution?
Thanks much !
Something like this should do it
$username = 'h2132704%40trbvm.com';
$password = '12345678';
$url = 'http://tv24.vn/account/login';
//login form action url
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = "cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
//page with the content I want to grab
curl_setopt($ch, CURLOPT_URL, "http://tv24.vn/livetv/vtv1.html");
//do stuff with the info with DomDocument() etc
$html = curl_exec($ch);
preg_match('/file: "(.*?)"/', $html, $m3u8);
print_r($m3u8[1]);
curl_close($ch);
I am new to using cURL....whenever I run this I get a page that says "The requested URL /files/ was not found on this server." But, if I go to the link on a browser then it shows up. Please assist. I have looked at similar questions but could not find a solution to my problem.
<?php
$username="user";
$password="pass";
$url="http://website.com/login/";
$cookie="cookie.txt";
$postdata = "login=$username&pass=$password";
//set the directory for the cookie using defined document root var
$dir = DOC_ROOT;
//the info per user with custom func.
$path = $dir;
//login form action url
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_exec($ch);
//page with the content I want to grab
curl_setopt($ch, CURLOPT_URL, "http://website.com/files/");
//do stuff with the info with DomDocument() etc
$html = curl_exec($ch);
echo $html;
curl_close($ch);
?>
$postdata = array(
'login' => $username,
'pass' => $password
);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
Change
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
to
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
Make sure you've write permission on the current dir so curl can create the cookie file cookie.txt
you should end-up with something like :
<?php
$username="user";
$password="pass";
$url_login="http://website.com/login/";
$url_files="http://website.com/files/";
$cookie="cookie.txt";
$postdata = "login=$username&pass=$password";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url_login);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, $url_files);
$html = curl_exec($ch);
echo $html;
curl_close($ch);
?>
NOTE:
Sometimes, less is more.
The URL you provided http://website.com/files/ has a redirection, so CURL has to have option enabled to follow redirects.
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
While in your code you have
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
0 means false, so you are disabling following on HTTP 30x
I'm trying to log into IPS Light and exporting a XML file. The url of the export page is unique and cannot be directly referenced in the script; Meaning that I will be required to click the necessary links on the homepage that will redirect me to the XML Export page.
require_once( dirname(__FILE__) . "/configuration.php" );
$username = "username";
$password = "password";
$dir = site_path."/ctemp";
$path = site_path."/ctemp/test";
$url="https://ipslight.upu.org/";
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, "https://ipslight.upu.org/IPSLight.dll");
$html = curl_exec($ch);
curl_close($ch);
I try to login to a forum via curl. The forum generates a unique id everytime the login page is displayed. That kinda bugs me :( I am able to parse the id and use it. But somehow the id is changed when I try the actual login.
My code so far:
$ch = curl_init();
$url ="http://theforum.dk/index.php/Login/";
$cookie="cookie.txt";
//Fetching the id
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch); (1)
$pos = strpos($output, '"t"')+11;
$pos2 = strpos($output, " />",$pos)-1;
$unique_id = substr($output, $pos,$pos2-$pos);
echo $unique_id; //Shows correct id
//Filling the POST array and submitting it
$arrSubmit="username=TESTUSER&action=login&password=TESTPASS&useCookies=1&url=&t=".$unique_id;
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $arrSubmit);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookies);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookies);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, false);
//Exectute and display result
echo curl_exec($ch); (2)
The output is something like "your id isn't valid anymore". The id at (1) differs from the id in the page source at (2) :(
The login form just a basic HTML form- Submitting the data isn't the problem, just using the fetched id from the start.
Do you guys got any idea?
Any help is appreciated :)
Hi my curl login example.
Good luck
$login_email = 'mehmet';
$login_pass = '112233';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.mysites.com/giris');
curl_setopt($ch, CURLOPT_POSTFIELDS,'username='.urlencode($login_email).'&password='.urlencode($login_pass));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "http://www.mysites.com/package-details/237");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curl_handle = curl_init ("http://www.mysites.com/package-details/237");
curl_setopt ($curl_handle, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, true);
$page = curl_exec($ch);
echo $page;