I want to read the xml from this url:
https://username:password#wetter.ws.siag.it/Weather_V1.svc/web/getLastProvBulletin?lang=de
but even when I try it with simplexml_load_file($url) I get this error:
parser error : Document is empty
And when I want to test it with file_get_contents($url) i get an empty string
""
Is it possible that this website blocks scripts? When yes, why it works with the wordpressplugin WP All import
I also tried it with curl:
private function curl($url) {
$headers[] = "User-Agent:Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13";
$headers[] = "Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$headers[] = "Accept-Language:en-us,en;q=0.5";
$headers[] = "Accept-Encoding:gzip,deflate";
$headers[] = "Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$headers[] = "Keep-Alive:115";
$headers[] = "Connection:keep-alive";
$headers[] = "Cache-Control:max-age=0";
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_ENCODING, "gzip");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
$data = curl_exec($curl);
curl_close($curl);
return $data;
}
... same issue
Related
I need to track a tracking number in this tracking website. However, the form is in iFrame div, how do I POST request using CURL in PHP? Below is my code.
iFrame src:
https://home.abxexpress.com.my/track_multiple.html
Original src:
https://abxexpress.com.my/Home/Tracking
Controller:
public function track($trackingID)
{
$data = array(
'tairbillno' => $trackingID
);
print_r($data);
$url = file_get_contents('https://home.abxexpress.com.my/track_multiple.html');
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$result = curl_exec($curl);
curl_close($curl);
echo $result;
}
Sample of tracking number:
99951513362
you can use this php-curl Code to send a simple post request to callback-page to get tracking info in html format,
after you can print content direct in your website(its work correctly,with the original style ), also you can use this content to extract all info you want , by any DOM extract librery like : php DOMDocument
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://home.abxexpress.com.my/track_multipleResult.asp?vsearch=True');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "tairbillno=99951513362");//replace number your traking number
curl_setopt($ch, CURLOPT_ENCODING, 'gzip, deflate');
$headers = array();
$headers[] = 'Connection: keep-alive';
$headers[] = 'Pragma: no-cache';
$headers[] = 'Cache-Control: no-cache';
$headers[] = 'Upgrade-Insecure-Requests: 1';
$headers[] = 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36';
$headers[] = 'Origin: https://home.abxexpress.com.my';
$headers[] = 'Content-Type: application/x-www-form-urlencoded';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9';
$headers[] = 'Sec-Fetch-Site: same-origin';
$headers[] = 'Sec-Fetch-Mode: navigate';
$headers[] = 'Sec-Fetch-User: ?1';
$headers[] = 'Sec-Fetch-Dest: document';
$headers[] = 'Referer: https://home.abxexpress.com.my/track_multiple.html';
$headers[] = 'Accept-Language: it,it-IT;q=0.9,en-US;q=0.8,en;q=0.7,ar;q=0.6';
$headers[] = 'Cookie: ASPSESSIONIDAWDDSADQ=LLOJGMICLOOKHGEHJHPBMBKE';
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result = curl_exec($ch);
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close($ch);
Our college system is using TCS (Total Campus Solution), Which is an online platform to track attendance and score of students by logging in with their username and passwords.
I want to create a PHP script that logging to their account if username+password is known and extract contents from HTML out.
This is because The vendor of the TCS is not providing any API's to access their database.
Here is the website: http://117.232.108.164/OnlineTCS/Login.aspx
Username: b14080
Password: b14080
(*I'm giving it here because the website hosts nothing sensitive and is read-only)
Here is my sample PHP cURL script:
<?php
$login_url = 'http://117.232.108.164/OnlineTCS/Login.aspx';
//These are the post data username and password
$post_data = 'ddlInstitution_ddl=KMEA Engineering College&ddlAcademicYear_ddl=2017-2018&txtUserName_txt=b14080&txtPassword_txt=b14080';
//Create a curl object
$ch = curl_init();
//Set the useragent
$agent = 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31';
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
//Set the URL
curl_setopt($ch, CURLOPT_URL, $login_url );
//This is a POST query
curl_setopt($ch, CURLOPT_POST, 1 );
//Set the post data
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
//We want the content after the query
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Follow Location redirects
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
/*
Set the cookie storing files
Cookie files are necessary since we are logging and session data needs to be saved
*/
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
//Execute the action to login
$postResult = curl_exec($ch);
echo $postResult;
?>
And $postResult is displaying only the HTML of the login page. Why and How to make this up
You have the keys to the post data incorrect.
You are missing some post fields
I broke the request into two separate request rather than use CURLOPT_FOLLOWLOCATION,true
I get the cookies from the first request (with 302 redirect)
Then pass the cookies to the redirect URL.
I return the HTML as plain text because the page is generated with JavaScript. For HTML change the header from text/plain to header('Content-Type: text/html; charset=utf-8');
This code is tested and works (debug code is commented out):
<?php
header('Content-Type: text/plain; charset=utf-8');
$post = array('txtUserName$txt'=>'b14080',
'txtPassword$txt'=>'b14080',
'btnLogin.x'=>"47",
'btnLogin.y'=>"16",
'ddlInstitution$ddl'=>"1",
'__VIEWSTATE'=>"/wEPDwUKMTM4NDc5NjkwOA9kFgQCAQ9kFggCAw8WAh4JaW5uZXJodG1sBTE8dWwgc3R5bGU9J2xpc3Qtc3R5bGUtdHlwZTogbm9uZTsnPjxsaT48L2xpPjwvdWw+ZAIHD2QWCgIBD2QWBAIBDxAPFgIeC18hRGF0YUJvdW5kZ2QQFQEYS01FQSBFbmdpbmVlcmluZyBDb2xsZWdlFQEBMRQrAwFnFgFmZAIDDw8WDh4RQ29udHJvbFRvVmFsaWRhdGUFA2RkbB4MRXJyb3JNZXNzYWdlBRdJbnN0aXR1dGlvbiBpcyBSZXF1aXJlZB4SRW5hYmxlQ2xpZW50U2NyaXB0Zx4PU2V0Rm9jdXNPbkVycm9yZx4MSW5pdGlhbFZhbHVlBQItMR4PVmFsaWRhdGlvbkdyb3VwBQRTYXZlHgdWaXNpYmxlZ2RkAgMPZBYEAgEPEA8WAh8BZ2QQFQgJMjAxNy0yMDE4CTIwMTYtMjAxNwkyMDE1LTIwMTYJMjAxNC0yMDE1CTIwMTMtMjAxNAkyMDEyLTIwMTMJMjAxMS0yMDEyCTIwMTAtMjAxMRUIAjExAjEwATkBOAE0ATMBNQE2FCsDCGdnZ2dnZ2dnFgFmZAIDDw8WDh8CBQNkZGwfAwUQWWVhciBpcyBSZXF1aXJlZB8EZx8FZx8GBQItMR8HBQRTYXZlHwhnZGQCBQ9kFgZmDw9kFgIeBWNsYXNzBQ1SZXF1aXJlZEZpZWxkZAICDw8WDh8CBQN0eHQfAwUVVXNlciBOYW1lIGlzIFJlcXVpcmVkHwRnHwVnHgRUZXh0ZR8HBQRTYXZlHwhnZGQCBA8PFgIfCmVkZAIHD2QWBmYPD2QWAh8JBQ1SZXF1aXJlZEZpZWxkZAICDw8WDh8CBQN0eHQfAwUUUGFzc3dvcmQgaXMgUmVxdWlyZWQfBGcfBWcfCmUfBwUEU2F2ZR8IZ2RkAgQPDxYCHwplZGQCCw8PFgIfCGgWAh4HT25DbGljawVEamF2YXNjcmlwdDpsb2NhdGlvbi5ocmVmPSdUZWFjaGVyRXZhbHVhdGlvbkxvZ2luLmFzcHgnO3JldHVybiBmYWxzZTtkAgkPFCsAAhQrAAMPFgIeF0VuYWJsZUFqYXhTa2luUmVuZGVyaW5naGRkZGRkAgsPFgIfCGhkAgIPFgIfCGhkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYCBQhidG5Mb2dpbgURUmFkV2luZG93TWFuYWdlcjGdhT6X6If1jy9RfnfHobAg2K4rSIeelkBCJSy3rq2wpw==",
'__EVENTVALIDATION'=>"/wEdABQgdITQYl0qSK+FUmoppd6yyBxeNODIYGb6hAb21KeJ7kXTFXzD8TPDckb0BpfOFEp8Q9GPFDMhnrO3EjMK7WDTfhXHcTNPmygrTq8U9nAZtWQdVtgFq+rE5JnIhzW6znHLgPIn5luRNqETic6Gh3lCzr/wuij3wzWkzglRmmW3l5eBKc1N3yXvWoV9t2A0ySUKrC+DRHeW4g3m+f2DrbA3AvnW9Ev6X0f6wNxHrCudiSSDXC6OOeSFHGK58wOzO+HMoHg3/YyHTBQ6cquFZ2oJblFY4hVirLH9o7SsXfoOK5utqpARm6HgYh9wA9bYTxNCuiJCR6m/KNe29Xd5W0RD9Pi9q20fDF3kt6+0eIryN4mI/M6bhYv0cQCm2IXbuc7UW0vN9k39fUY9+DzXmYv3op4oRunf14dz2Zt2+QKDELhHwE4yeHGuxfNtQ50+95KNfuiPvtdexWTG2c3kPRh1");
$request = array();
$request[] = 'Host: 117.232.108.164';
$request[] = 'Connection: keep-alive';
$request[] = 'Pragma: no-cache';
$request[] = 'Cache-Control: no-cache';
$request[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8';
$request[] = 'User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36';
$request[] = 'DNT: 1';
$request[] = 'Referer: http://117.232.108.164/OnlineTCS/Login.aspx';
$request[] = 'Accept-Encoding: gzip, deflate';
$request[] = 'Accept-Language: en-US,en;q=0.8';
$url = 'http://117.232.108.164/OnlineTCS/Login.aspx';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FAILONERROR,true);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_HEADER, true);
$data = curl_exec($ch);
if (curl_errno($ch)){
$data .= 'Retreive Base Page Error: ' . curl_error($ch);
}
else {
$info = rawurldecode(var_export(curl_getinfo($ch),true));
$skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE));
$responseHeader= substr($data,0,$skip);
// The folowing line is for debug
// echo "$info\n$responseHeader\n$data\n\n";
// Get the cookies:
$e = 0;
while(true){
$s = strpos($responseHeader,'Set-Cookie: ',$e);
if (!$s){break;}
$s += 12;
$e = strpos($responseHeader,';',$s);
$cookie = substr($responseHeader,$s,$e-$s) ;
$s = strpos($cookie,'=');
$key = substr($cookie,0,$s);
$value = substr($cookie,$s);
$cookies[$key] = $value;
}
// Create cookie for subsequent Requests:
$cookie = '';
$show = '';
$head = '';
$delim = '';
foreach ($cookies as $k => $v){
$cookie .= "$delim$k$v";
$delim = '; ';
}
}
$request = array();
$request[] = 'Host: 117.232.108.164';
$request[] = 'Connection: keep-alive';
$request[] = 'Pragma: no-cache';
$request[] = 'Cache-Control: no-cache';
$request[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8';
$request[] = 'User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36';
$request[] = 'DNT: 1';
$request[] = 'Referer: http://117.232.108.164/OnlineTCS/Login.aspx';
$request[] = 'Accept-Encoding: gzip, deflate';
$request[] = 'Accept-Language: en-US,en;q=0.8';
$url = 'http://117.232.108.164/OnlineTCS/StudentLogin/StudentLoginForm.aspx';
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_POST, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FAILONERROR,true);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, false);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_COOKIESESSION , true );
curl_setopt($ch, CURLOPT_COOKIE, $cookie );
$data = curl_exec($ch);
$skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE));
echo substr($data,$skip); //do not transmit header
/* Debug Code
if (curl_errno($ch)){
$data .= 'Retreive Base Page Error: ' . curl_error($ch);
}
else {
$info = rawurldecode(var_export(curl_getinfo($ch),true));
// Get the cookies:
$skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE));
$responseHeader= substr($data,0,$skip);
echo "$info\n$responseHeader\n$data\n\n";
$e = 0;
while(true){
$s = strpos($responseHeader,'Set-Cookie: ',$e);
if (!$s){break;}
$s += 12;
$e = strpos($responseHeader,';',$s);
$cookie = substr($responseHeader,$s,$e-$s) ;
$s = strpos($cookie,'=');
$key = substr($cookie,0,$s);
$value = substr($cookie,$s);
$cookies[$key] = $value;
}
*/
}
?>
I'm trying to download the list data from goodreads with curl in php but it always returns the same result for different pages. I mean, the result of ?page=1 and ?page=2 is the same. How can i solve this problem? Sorry for my bad english, thanks.
$data1 = getGoodreadsList("http://www.goodreads.com/list/show/43005.En_yi_T_rk_e_eviriler?page=");
var_dump($data1);
public function getGoodreadsList($link){
$result = [];
for($page=1;$page<29;$page++){
$html = str_get_html($this->downloadUrl($link . strval($page)));
foreach($html->find(".bookTitle") as $book){
array_push($result, "http://www.goodreads.com" . $book->href);
}
$html->clear();
unset($html);
if($page == 2){
break;
}
}
return $result;
}
private function downloadUrl($url){
$headers[] = "User-Agent:'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36'";
$headers[] = "Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$headers[] = "Accept-Language:en-us,en;q=0.5";
$headers[] = "Accept-Encoding:gzip,deflate";
$headers[] = "Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$headers[] = "Keep-Alive:115";
$headers[] = "Connection:keep-alive";
$headers[] = "Cache-Control:max-age=0";
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_VERBOSE, false);
curl_setopt($curl, CURLOPT_ENCODING, "gzip");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
$data = curl_exec($curl);
curl_close($curl);
return $data;
}
I got a problem with downloading video from videoslasher.com. I wrote a script to get source link and it is working fine.
Example link:
http://storage2.videoslasher.com/free/K/K1/K1Z6TBW88QCM.flv?h=XeahiwUFe_yHiduYx5EzBg&e=1377200810
But when i am downloading with script i get 403 (forbidden).
Code downloading video:
$headers[] = 'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0';
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
$headers[] = 'Accept-Language: pl,en-us;q=0.7,en;q=0.3';
$headers[] = 'Accept-Encoding: gzip, deflate';
$headers[] = 'Connection: keep-alive';
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.videoslasher.com/static/player/flowplayer.commercial-3.2.7.swf');
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt( $ch, CURLOPT_HTTPHEADER, $headers );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE,'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_BUFFERSIZE, 4096000);
echo curl_exec($ch);
curl_close($ch);
Does anyone know what could be the problem?
The below code gives different output on localhost and server.
I tried changing useragents and browsers. Am I missing any header?
<?php
$headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
$headers[] = 'Connection: Keep-Alive';
$headers[] = 'Content-type: application/x-www-form-urlencoded;charset=UTF-8';
$headers[] = 'Accept-Language: en-us,en;q=0.5';
$headers[] = 'Accept-Encoding gzip,deflate';
$headers[] = 'Keep-Alive: 300';
$headers[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';
//$headers[] = 'Content-Length: 0';
$curl = curl_init();
curl_setopt($curl, CURLOPT_HEADERFUNCTION, 'dbg_curl_data');
curl_setopt($curl, CURLOPT_WRITEFUNCTION, 'dbg_curl_data');
curl_setopt($curl, CURLINFO_HEADER_OUT, true);
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7');
//curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.95 Safari/537.11');
curl_setopt($curl, CURLOPT_URL, "http://www.facebook.com/login.php");
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_ENCODING, 'gzip');
//curl_setopt($curl, CURLOPT_HEADER, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_VERBOSE, 1);
//curl_setopt($curl, CURLOPT_POST, true);
$html = curl_exec ($curl);
//curl_close ($curl);
echo '<fieldset><legend>request headers</legend>
<pre>', htmlspecialchars(curl_getinfo($curl, CURLINFO_HEADER_OUT)), '</pre>
</fieldset>';
echo '<fieldset><legend>response</legend>
<pre>', htmlspecialchars(dbg_curl_data(null)), '</pre>
</fieldset>';
echo '<fieldset><legend>$html</legend>
<pre>', htmlspecialchars($html), '</pre>
</fieldset>';
function dbg_curl_data($curl, $data=null) {
static $buffer = '';
if ( is_null($curl) ) {
$r = $buffer;
$buffer = '';
return $r;
}
else {
$buffer .= $data;
return strlen($data);
}
}
?>
Output on Localhost : http://pastebin.com/UbpTVc4f
Output on Server : http://pastebin.com/NURDksJy
I want output as its on localhost.
Why don't you start with something simple before you complicate your request as much as it is now and is hard to debug. For example, this code (run it in a separate file):
$curl = curl_init();
curl_setopt($curl, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7');
curl_setopt($curl, CURLOPT_URL, "http://www.facebook.com/login.php");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_HEADER, 1);
$html = curl_exec ($curl);
print_r($html);
Should give you HTTP/1.1 302 Found in return.