I am new to using cURL....whenever I run this I get a page that says "The requested URL /files/ was not found on this server." But, if I go to the link on a browser then it shows up. Please assist. I have looked at similar questions but could not find a solution to my problem.
<?php
$username="user";
$password="pass";
$url="http://website.com/login/";
$cookie="cookie.txt";
$postdata = "login=$username&pass=$password";
//set the directory for the cookie using defined document root var
$dir = DOC_ROOT;
//the info per user with custom func.
$path = $dir;
//login form action url
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_exec($ch);
//page with the content I want to grab
curl_setopt($ch, CURLOPT_URL, "http://website.com/files/");
//do stuff with the info with DomDocument() etc
$html = curl_exec($ch);
echo $html;
curl_close($ch);
?>
$postdata = array(
'login' => $username,
'pass' => $password
);
curl_setopt($ch, CURLOPT_POST, TRUE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
Change
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
to
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
Make sure you've write permission on the current dir so curl can create the cookie file cookie.txt
you should end-up with something like :
<?php
$username="user";
$password="pass";
$url_login="http://website.com/login/";
$url_files="http://website.com/files/";
$cookie="cookie.txt";
$postdata = "login=$username&pass=$password";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url_login);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, $url_files);
$html = curl_exec($ch);
echo $html;
curl_close($ch);
?>
NOTE:
Sometimes, less is more.
The URL you provided http://website.com/files/ has a redirection, so CURL has to have option enabled to follow redirects.
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
While in your code you have
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
0 means false, so you are disabling following on HTTP 30x
Related
I am trying to create a php script which logs into a website and then navigates to (potentially multiple) web pages which require a user to be logged in to see them. I think I have successfully logged in using cURL but I don't know how to "stay logged in" to read file contents after having logged in. I suspect my issue has something to do with using the same cookies file but I am unsure. This is what I currently have:
<?
//login form action url
$url="https://randomwebsite.com/users/sign_in";
$postinfo = "utf8=%E2%9C%93&authenticity_token=ncgU2234iEMObg3334d3Iag%2BPBrmEerybFZ3X2fvVsUTmpjkPDQMgteeGlVDxFRfHjmHbvIYaTchvM2psKFzI%2BAHIpCw%3D%3D&user%5Botp_attempt%5D=step_1&user%5Blocale%5D=en&user%5Blogin%5D=randomuser&user%5Bpassword%5D=123456";
$cookie_file_path = "cookies1.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
$curlexecresponse = curl_exec($ch);
//echo("\ncurlexecresponse: " . $curlexecresponse);
// Now that we're logged in we can finally read the web pages we want to...
$url2 = "url to members only page";
curl_setopt($ch, CURLOPT_URL, $url2);
curl_setopt($ch, CURLOPT_POST, 0);
$data = curl_exec($ch);
echo("\n\n data is: " . $data); // this is empty
curl_close($ch);
?>
How can I read the HTML in pages after having logged in?
You need to send the cookie with every request after login in.
$url2 = "url to members only page";
//Add this line
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_URL, $url2);
curl_setopt($ch, CURLOPT_POST, 0);
I want to use PHP to get this html content, the final aim I want to get m3u8 link
It requires log-in
http://tv24.vn/livetv/vtv1.html
This is information to test:
user: h2132704#trbvm.com
pass: 12345678
I tried to use curl POST but unsuccessful!
<?php
$username = 'h2132704#trbvm.com';
$password = '12345678';
$loginUrl = 'http://tv24.vn/livetv/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $loginUrl);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'user='.$username.'&pass='.$password);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$store = curl_exec($ch);
$return=file_get_contents("http://tv24.vn/livetv/vtv1.html"); preg_match('/file: "(.*?)"/', $return, $m3u8);
echo $m3u8;
?>
Any solution?
Thanks much !
Something like this should do it
$username = 'h2132704%40trbvm.com';
$password = '12345678';
$url = 'http://tv24.vn/account/login';
//login form action url
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = "cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
//page with the content I want to grab
curl_setopt($ch, CURLOPT_URL, "http://tv24.vn/livetv/vtv1.html");
//do stuff with the info with DomDocument() etc
$html = curl_exec($ch);
preg_match('/file: "(.*?)"/', $html, $m3u8);
print_r($m3u8[1]);
curl_close($ch);
I'm trying to log into IPS Light and exporting a XML file. The url of the export page is unique and cannot be directly referenced in the script; Meaning that I will be required to click the necessary links on the homepage that will redirect me to the XML Export page.
require_once( dirname(__FILE__) . "/configuration.php" );
$username = "username";
$password = "password";
$dir = site_path."/ctemp";
$path = site_path."/ctemp/test";
$url="https://ipslight.upu.org/";
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, "https://ipslight.upu.org/IPSLight.dll");
$html = curl_exec($ch);
curl_close($ch);
I try to login to a forum via curl. The forum generates a unique id everytime the login page is displayed. That kinda bugs me :( I am able to parse the id and use it. But somehow the id is changed when I try the actual login.
My code so far:
$ch = curl_init();
$url ="http://theforum.dk/index.php/Login/";
$cookie="cookie.txt";
//Fetching the id
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch); (1)
$pos = strpos($output, '"t"')+11;
$pos2 = strpos($output, " />",$pos)-1;
$unique_id = substr($output, $pos,$pos2-$pos);
echo $unique_id; //Shows correct id
//Filling the POST array and submitting it
$arrSubmit="username=TESTUSER&action=login&password=TESTPASS&useCookies=1&url=&t=".$unique_id;
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $arrSubmit);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookies);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookies);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, false);
//Exectute and display result
echo curl_exec($ch); (2)
The output is something like "your id isn't valid anymore". The id at (1) differs from the id in the page source at (2) :(
The login form just a basic HTML form- Submitting the data isn't the problem, just using the fetched id from the start.
Do you guys got any idea?
Any help is appreciated :)
Hi my curl login example.
Good luck
$login_email = 'mehmet';
$login_pass = '112233';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.mysites.com/giris');
curl_setopt($ch, CURLOPT_POSTFIELDS,'username='.urlencode($login_email).'&password='.urlencode($login_pass));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "http://www.mysites.com/package-details/237");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curl_handle = curl_init ("http://www.mysites.com/package-details/237");
curl_setopt ($curl_handle, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, true);
$page = curl_exec($ch);
echo $page;
I would like to get the html of the page. After some google-ing, i found following code, which isn't working.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$passwd");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
echo $output;
edit: something like this then:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, '');
curl_setopt($ch, CURLOPT_POSTFIELDS,'username='.urlencode($username).'&passwd='.urlencode($passwd));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "");
$output = curl_exec($ch);
echo $output
This just shows the target page, it doesn't log in, nor does it return the html code.
edit 2: now tried with $data = array('username' => $username, 'passwd' => $passwd); and curl_setopt($ch, CURLOPT_POSTFIELDS,$data);
This shows the same as above + an error: Request Entity Too Large
That page does not use HTTP Basic Authentication.
You need to make an HTTP request to match what submitting the form would sent, then you need to process the response (which will probably involve storing cookies).