Including XML file, and setting cookies - php

Alright, for a project, there's an XML page, which uses the user's cookies to generate output.
I don't have access to this page, but wish to extract information from it using PHP.
My thoughts so far were some sort of PHP include that sets cookies to the externally included file.
Any pointers or suggestions would be great. Thanks.
EDIT: I have no control over the domain that the XML page is hosted on.

You could use cURL and include a cookie jar:
From http://curl.haxx.se/libcurl/php/examples/cookiejar.html:
<?php
/*
This script is an example of using curl in php to log into on one page and
then get another page passing all cookies from the first page along with you.
If this script was a bit more advanced it might trick the server into
thinking its netscape and even pass a fake referer, yo look like it surfed
from a local page.
*/
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/checkpwd.asp");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "UserID=username&password=passwd");
ob_start(); // prevent any output
curl_exec ($ch); // execute the curl command
ob_end_clean(); // stop preventing output
curl_close ($ch);
unset($ch);
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,"http://www.myterminal.com/list.asp");
$buf2 = curl_exec ($ch);
curl_close ($ch);
echo "<PRE>".htmlentities($buf2);
?>

Related

PHP CURL script runs but it does not set the cookie

Im trying to set a cookie through PHP CURL for more than twenty four hour for no avail.
Before i have been setting cookies in my browser by adding them as parameters in a url as shown below
http://localhost/setc.php?userid=123&panelid=1
but now i need to set the cookie when i run a script(setcookie.php)
below is the latest of various types of code that i tried.
setcookie.php
$c = curl_init('http://localhost/atst.php?userid=628929&panelid=1');
curl_setopt($c, CURLOPT_VERBOSE, 1);
curl_setopt($c, CURLOPT_COOKIE, 'userid=123; panelid=1');
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
$page = curl_exec($c);
curl_close($c);
it still does not create the cookie, can anybody help out
P.S : if you guys too cant figure this out at least give me a hint/guide on how to set a simple cookie without any complications
The cookiejar is only saved when you close the curl handle using curl_close($ch).
From the manual:
CURLOPT_COOKIEFILE The name of the file containing the cookie data. The cookie file can be in Netscape format, or just plain HTTP-style headers dumped into a file. If the name is an empty string, no cookies are loaded, but cookie handling is still enabled.
CURLOPT_COOKIEJAR The name of a file to save all internal cookies to when the handle is closed, e.g. after a call to curl_close.
http://www.php.net/manual/en/function.curl-setopt.php
$ckfile = tempnam ("/tmp/", "CURLCOOKIE");
$BASEURL='http://localhost/openx/www/api/json/index.php/main/authenticate/';
$POSTFIELDS='username='.$username.'&password='.$password.'';
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/cookieFileName");
curl_setopt($ch, CURLOPT_URL,'http://localhost/openx/www/api/json/index.php/main/authenticate/');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $POSTFIELDS);
ob_start(); // prevent any output
$result=curl_exec ($ch); // execute the curl command
ob_end_clean(); // stop preventing output
$result = curl_exec($ch);
curl_close($ch);

Curl Php Jsessionid cannot get or set

okay so I am trying to curl to a website using jsessionid this is new to me.
I have a curl php script shown below how can I get the correct jsession id cookies and set them correctly .
<?php
$url = 'http://www.example.com/i/sec/stats.do';
$ch = curl_init();
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); // Cookie management.
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result= curl_exec ($ch);
curl_close ($ch);
echo $result;
?>
I then get this output how can I get and set the jsessionid.
This document you requested has moved temporarily.
It's now at https://www.example.com/i/sec/stats.do;jsessionid=c7dnSdlXc18Zpmqj1Tv1Rxq5TZDwD7dCpt5dpbg7LXmp1gnZs9V9!212735760!1390241207640.
Possibly you are getting 301 or 302 redirect response from your curl request. Use this option to handle redirection.
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
But, if the site doesn't return the 301, 302 redirection response, then parse the url from the response, and call another curl request.

jibberish returned when using CURLOPT_URL

Im trying to grab a pages data using CURLOPT_URL, to do so ive used the below code, which is working fine for other pages (with the exception of where the page uses relative paths to its css / js in which case those dont load) .
function grab_page($site){
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_TIMEOUT, 40000000);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_URL, $site);
ob_start();
return curl_exec ($ch);
ob_end_clean();
curl_close ($ch);
}
echo grab_page("$page_to_get");
But when i load the page i get returned a screen of jibberish like this, but a whole page, same when i view the source.
Looking at the source of the page, through my browser, they seem to be using charset=utf-8", im not sure if that has anything to do with it though ? Any ideas ?
Calling:
curl_setopt($ch,CURLOPT_ENCODING , "gzip");
will fix it if the encoding is know to be gzipped or as you stated
curl_setopt($ch,CURLOPT_ENCODING , "");
should tickle curl into negotiating the encoding itself (why this is not the default is beyond me)

curl problem, can't download full web page

With this code I'm trying to download this web page: http://www.kayak.com/s/...
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,'http://www.kayak.com/s/search/air?ai=kayaksample&do=y&ft=ow&ns=n&cb=e&pa=1&l1=ZAG&t1=a&df=dmy&d1=4/10/2010&depart_flex=exact&r1=y&l2=LON&t2=a&d2=11/10/2010&return_flex&r2=y');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_REFERER,"http://wwww.google.com");
$content = curl_exec ($ch);
echo $content;
You can see the demo at: http://www.pointout.org/test.php
As you can see the part with prices is missing.
What could be wrong?
This is not going to work the way you think it will. The reason is the prices are not in the initial HTML response that you get. Rather, there is some Javascript magic occurring which is using AJAX to load the prices when the page is loaded.

Download file attached to header with Curl and Php

I'm connecting to a website daily to collect some statistics, the website runs .net to make things extra difficult. What i would like to do is to mechanize this process.
I go to http://www.thesite.com:8080/statistics/Login.aspx?ReturnUrl=%2Fstatistics%2Fdataexport.ashx%3FReport%3D99, (the return url is /statistics/dataexport.ashx?Report=99 decoded).
The Login.aspx displays a form, in which I enter my user/pass and when the form is submitted the dataexport.ashx starts to download the file directly. The filename delivered is always statistics.csv.
I have experimented with this a few days now. Are there any resources or does anyone have some kind of hint of what I should try next?
Here is some of my code.
<?php
// INIT CURL
$ch = curl_init();
// SET URL FOR THE POST FORM LOGIN
curl_setopt($ch, CURLOPT_URL, $url);
// ENABLE HTTP POST
curl_setopt ($ch, CURLOPT_POST, 1);
// SET POST PARAMETERS : FORM VALUES FOR EACH FIELD
$viewstate = urlencode('/wEPDwUKM123123daE2MGQYAQUeX19Db250cm9sc1JlcXVpcmVQb3N0QmFja0tleV9fFgEFGG1fTG9naW4kTG9naW5JbWFnZUJ1dHASdasdRvbij2MVoasdasdYibEXm/eSdad4hS');
$eventval = urlencode('/wEWBAKMasd123LKJJKfdAvD8gd8KAoCt878OED00uk0pShTQHkXmZszVXtBJtVc=');
curl_setopt ($ch, CURLOPT_POSTFIELDS, "__VIEWSTATE=$viewstate"."__EVENTVALIDATION=$eventval&UserName=myuser&Password=mypassword");
// IMITATE CLASSIC BROWSER'S BEHAVIOUR : HANDLE COOKIES
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
// FOLLOW REDIRECTS AND READ THE HEADER
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, true);
// EXECUTE REQUEST (FORM LOGIN)
$store = curl_exec ($ch);
// print the result
print_r($store);
// CLOSE CURL
curl_close ($ch);
?>
Thanks
Trikks
You also need to use CURLOPT_COOKIEFILE to send the cookies along with the next request. Another thing if i remember correctly is that ASPX would set unique value each time for variables like __VIEWSTATE. See if these 2 pointers help.

Categories