I'm trying to parse a website with curl.
However the first screen is a empty form.
So far I'm getting an error (301-302)
Here is my code:
<?php
$url = "https://www.interiorhealth.ca/HedgehogPortal/?returnUrl=/HedgehogPortal/facility/Table";
$url_redirect = "https://www.interiorhealth.ca/HedgehogPortal/facility/Table";
define('USER_AGENT', 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.2309.372 Safari/537.36');
define('COOKIE_FILE', 'cookie.txt');
define('FORM_URL', $url);
define('REDIRECT_URL', $url_redirect);
$postValues = array();
//Initiate cURL.
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, REDIRECT_URL);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($postValues));
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($curl, CURLOPT_USERAGENT, USER_AGENT);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_REFERER, FORM_URL);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, false);
$server_output = curl_exec($curl);
//Check for errors!
if(curl_errno($curl)){
throw new Exception(curl_error($curl));
} else {
echo $server_output;
}
?>
Any help or pointers would be gladly appreciated.
Thank you!
Related
i have a question from json_decode url instagram https://www.instagram.com/username/?__a=1 return Null with php
$newwul = "http://www.instagram.com/username/?__a=1";
$xo = connect($newwul);
$xx = json_decode($xo,true);
$endcount = $xx['logging_page_id'];
$userid = $xx['graphql']['user']['id'];
var_dump($xo);//return NULL
var_dump($xx);//return NULL
they is my function connect
function connect($urlx){
$cookie_file_path = "cookies/c.txt";
$agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36";
$urlex = ($urlx);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $urlex);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'cookie:ig_cb=1; rur=PRN; mid=XV_v7gAEAAFbvgXMUd0PClkTvnla; csrftoken=fs1r3L5SxkJvrN6g2w239nhMgppQGVLc; urlgen="{\"185.56.80.156\": 43350}:1i2zL0:8TaH7yECjymjQH80LduQiFUJZE0"',
));
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
curl_setopt($ch, CURLOPT_REFERER, $urlex);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT ,100);
curl_setopt($ch, CURLOPT_TIMEOUT, 100);
return $req = curl_exec($ch);
curl_close($ch);
}
any solution please?
Please view example for curl instagram:
<?php
$newwul = "http://www.instagram.com/username/?__a=1";
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $newwul);
curl_setopt($curl, CURLOPT_HEADER, 0);
curl_setopt($curl, CURLOPT_USERAGENT, " Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1216.0 Safari/537.2");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_TIMEOUT, 60);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
$data_curl = curl_exec($curl);
curl_close($curl);
var_dump($data_curl);
?>
Im trying to automate a login system that i created.
I want to get the cookie after logged to use in another code.
How i get this cookies now: I manually log in to the site, get first cookie, go to the second page, to generate the second cookie.
After searching some articles here, put some codes together, with no sucess. Used fiddle to get the login parameters, that are just like this: perfil=1&txtLogin=*****&txtSenha=*****&envia=1
<?php
$loginUrl =
'https://www.comprasnet.gov.br/seguro/loginPortalFornecedor.asp';
define('perfil', '1');
define('txtLogin', '*****');
define('txtSenha', '*****');
define('envia', '1');
define('USER_AGENT', 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/35.0.2309.372 Safari/537.36');
define('COOKIE_FILE', 'cookie.txt');
define('LOGIN_FORM_URL',
'https://www.comprasnet.gov.br/seguro/loginPortalFornecedor.asp');
define('LOGIN_ACTION_URL',
'https://www.comprasnet.gov.br/seguro/loginPortalFornecedor.asp');
$postValues = array(
'perfil' => perfil,
'txtLogin' => txtLogin,
'txtSenha' => txtSenha,
'envia' => envia
);
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, LOGIN_ACTION_URL);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, http_build_query($postValues));
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($curl, CURLOPT_COOKIEJAR, realpath(COOKIE_FILE) );
curl_setopt($curl, CURLOPT_USERAGENT, USER_AGENT);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_REFERER, LOGIN_FORM_URL);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, false);
curl_exec($curl);
if(curl_errno($curl)){
throw new Exception(curl_error($curl));
}
curl_setopt($curl, CURLOPT_URL,
'https://www.comprasnet.gov.br/pregao/fornec/Acompanhar.asp');
curl_setopt($curl, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt($curl, CURLOPT_USERAGENT, USER_AGENT);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
echo curl_exec($curl);
?>
'
What i want is that this code save the cookie from this website, if possible in this format: ASPSESSIONIDSEDTDDRT=KGJKDBOCKJEIPMNCMOGFJCBD; ASPSESSIONIDSGCQADQT=GAKNEBOCNLIHEDGKNIKHECPH.
I dont know php, just read some articles here, to try to make this piece of code.
I changed the cod and now i can login and save cookie to a txt file. Below the code that is working.
<?php
$url = "https://www.comprasnet.gov.br/seguro/loginPortalFornecedor.asp";
$referer = "...";
$user_agent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.118 Safari/537.36";
$url_notas = "https://www.comprasnet.gov.br/pregao/fornec/Acompanhar.asp";
$referer_notas = "...";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_REFERER, $referer);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'perfil=1&txtLogin=xxxxx&txtSenha=xxxxx&envia=1');
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) . '/cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, dirname(__FILE__). '/cookie.txt');
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
print_r(curl_exec($ch));
print_r(curl_errno($ch));
print_r(curl_getinfo($ch));
echo "<hr>";
curl_setopt($ch, CURLOPT_URL, $url_notas);
curl_setopt($ch, CURLOPT_REFERER, $referer_notas);
curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_COOKIEJAR, dirname(__FILE__) . '/cookie2.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, dirname(__FILE__). '/cookie.txt');
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
print_r(curl_exec($ch));
print_r(curl_errno($ch));
print_r(curl_getinfo($ch));
?>
Now i have other problem. When it log in it automatically shows the redirection page and stop the code. After sucessful log to https://www.https://www.comprasnet.gov.br/seguro/loginPortalFornecedor.asp, it goes to https://www.comprasnet.gov.br/intro.htm, but when i run mu code it logs in and goes to https://127.0.0.1/intro.htm. How can i stop it to go to this page, so i can implement some code below it.
I am trying log in a website which has security code for 2 days like you see below.
(By the way, I can get security number from html codes of web page.)
First, I have to connect to website and get the code(254), than post username, password and security number. But there is a big problem. I am getting security number and the page is reloading and the code is changing everytime when I try post to webpage this variables.
Here is my PHP code:
<?php
include('simple_html_dom.php');
$url = "http://example.com/index.php";
$useragent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36";
$tmpfname = dirname(__FILE__).'/cookies.txt';
$ch = curl_init(); // GET REQUEST
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_COOKIEJAR, $tmpfname);
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
$str = curl_exec($ch);
echo $str;
curl_close($ch);
preg_match('~<td><img src="number/(.*?).jpg"></td>~', $str, $securityNo); // I am gettin security number
echo $securityNo[1]; // It print true
$username = "USER";
$password = "PASS";
$postfields = array();
$postfields['UserName'] =$username; // Username
$postfields['Password'] = $password; // Password
$postfields['code'] = $securityNo[1]; // Code
$postfields['LOGIN'] = "LOGIN"; // BUTTON
$curl =curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($curl, CURLOPT_REFERER, $url);
curl_setopt($curl, CURLOPT_USERAGENT, $useragent);
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_COOKIEFILE, $tmpfname);
curl_setopt($curl, CURLOPT_POSTFIELDS, $postfields);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 0);
$ret = curl_exec($curl);
curl_close($curl);
echo $ret;
?>
I printed screen both 'echo $str' (get request) and 'echo $ret' (post request) but there is different 2 codes in screen, not same code printed.
I researched on internet too much, but I got nothing. I can't solve this.
Thanks in advance!
i'm trying to make a curl request to a https based url. But seems nothing returned back..
I tested with many proxy that worked for non https.
$att = "cookie.txt";
$ip = "121.14.138.56:81";
$curl = curl_init("https://www.att.com");
curl_setopt($curl, CURLOPT_PROXY, $ip);
curl_setopt($curl, CURLOPT_COOKIEJAR, $att);
curl_setopt($curl, CURLOPT_COOKIEFILE, $att);
curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/27.0.1453.94 Safari/537.36");
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 50);
curl_setopt($curl, CURLOPT_TIMEOUT, 50);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
$reply = curl_exec($curl);
curl_close($curl);
echo $reply;
Seems this code is working. Note: only some proxy support https..
$proxy = "205.221.221.111:8080";
$curl = curl_init("https://www.att.com");
if (isset($proxy)) {curl_setopt($curl, CURLOPT_PROXY, $proxy);}
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_COOKIEJAR, $att);
curl_setopt($curl, CURLOPT_COOKIEFILE, $att);
curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt($curl, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/27.0.1453.94 Safari/537.36");
curl_setopt($curl, CURLOPT_COOKIE, 'cookie.txt');
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_COOKIESESSION, true);
curl_setopt($curl, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt($curl, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($curl, CURLOPT_TIMEOUT, 30);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
$reply = curl_exec($curl);
curl_close($curl);
var_dump(curl_getinfo($curl));
echo $reply;
i have problem that the "PHP" cURL function don't save the cookie !
curl function :
$file['cookie'] = "cooki";
function cURL_Page($url='',$var=''){
global $file;
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT,20);
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31');
curl_setopt($curl, CURLOPT_REFERER, "http://www.libertyreserve.com/en/login");
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
if($var) {
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $var);
}
curl_setopt($curl, CURLOPT_COOKIE,$file['cookie']);
curl_setopt($curl, CURLOPT_COOKIEFILE,$file['cookie']);
curl_setopt($curl, CURLOPT_COOKIEJAR,$file['cookie']);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 3);
curl_setopt($curl, CURLOPT_HEADER, 0);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, FALSE);
$result = curl_exec($curl);
curl_close($curl);
return $result;
}
in the file "cooki" the are no results
p.s : i curl a https page
thanks all