Unable to exchange Session Data in CURL - php

I call invoke.php using CURL from curl.php from my localhost. In invoke.php i store some data in session. But when i try to access those session data from curl.php, don't get those session data. How i get those values?
Content of curl.php
`include_once ('session.php');
$handles = array();
$urlArray = array('http://localhost/invoke.php' );
foreach($urlArray as $url){
// create a new single curl handle
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
// add this handle to the multi handle
curl_multi_add_handle($mh,$ch);
// put the handles in an array to loop this later on
$handles[] = $ch;
}
// execute the multi handle
$running=null;
do {
curl_multi_exec($mh,$running);
// added a usleep for 0.25 seconds to reduce load
usleep (250000);
} while ($running > 0);
// get the content of the urls (if there is any)
for($i=0;$i
{
// get the content of the handle
// $output.= curl_multi_getcontent($handles[$i]);
// remove the handle from the multi handle
curl_multi_remove_handle($mh,$handles[$i]);
}
echo SessionHandler :: getData('DATA');
`
Content of invoke.php
include_once ('session.php');
echo SessionHandler :: setData('DATA', 'HELLO WORLD');

There are two possible issues:
You can't be sure that both scripts will access same session (have same session id), actually I can bet they have separate sessions. Of course, you could enforce that by sending the session id from invoke.php to curl.php by adding an extra parameter tot the URL and then use that parameter to force the session id in curl.php
Second possible issue is that the session variables are read on session_start (that's when the $_SESSION is populated), but you modify the session content after you start the session (in session.php, I assume), that's why any changes (even if the conditions on 1. will be met), will not reflect into already opened session in invoke.php. I think you should force session to close and restart it.

I had the same issue. This is how I resolved it:
function getContent($url){
$cookiesStr = '';
foreach($_COOKIE as $k => $v){
$cookiesStr.= $k . '=' . $v . '; ';
}#end COOKIE
$cookiesStr = rtrim($cookiesStr, ' ');
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
curl_setopt ($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_COOKIE, $cookiesStr);
ob_start();
curl_exec ($ch);
curl_close ($ch);
$string = ob_get_contents();
ob_end_clean();
return $string;
}#end getContent
You have to have a tmp cookie jar or it will produce an infinite loop. You have to send cookies or you will not get a result.

I able to solve this problem. I set curl_
setopt($ch, CURLOPT_COOKIE,session_name().'='.session_id());
to use same session id and use php's session_write_close() function to write session data.

Related

how do php scripts for data mining from web pages work?

[Edited for better explanation and code included]
Hi! I have a php script on my web server that logs in to my heat pump web interface nibeuplink.com and gets all my temperature readings and so forth and returns them in a json-format.
freeboard.io is a free service for visualizing data, so I'm making a freeboard.io for my heat pump values. in freeboard.io I can add any json data as a data source, so I have added the link to my php-script. It fetches the data once but it seems there is some kind of cached values that it uses after that so they are not updated with new values from the script. freeboard.io uses a get-function to get the url. If i use a normal web browser to run the php script and refresh it, the values are updated - and also immediately updated in freeboard.io. Freeboard.io has a setting to automatically update the data source every 5 seconds.
It seems that there is something that triggers the script correctly when it is fetched from my web browser, but not when it is fetched from freeboard.io that uses a get function every 5 seconds to get new data.
in freeboard I can add headers to the get request, is there some header that would help me here to discard any cached data?
I hope that explains my problem better.
Is there anything i can add to my code in the beginning to always force an override of any cached data?
<?php
/*
* read nibe heatpump values from nibeuplink status web page and return them in json format.
* based on: https://www.symcon.de/forum/threads/25663-Heizung-Nibe-F750-Nibe-Uplink-auslesen-auswerten
* to get the code which is required as parameter, log into nibe uplink, open status page of your heatpump, and check url:
* https://www.nibeuplink.com/System/<code>/Status/Overview
*
* usage: nibe.php?email=<email>&password=<password>&code=<code>
*/
// to add additional debug output to the resulting page:
$debug = false;
date_default_timezone_set('Europe/Helsinki');
$date = time();
// Create temp file to store cookies
$ckfile = tempnam ("/tmp", "CURLCOOKIE");
// URL to login page
$url = "https://www.nibeuplink.com/LogIn";
// Get Login page and its cookies and save cookies in the temp file
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Accepts all CAs
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile); // Stores cookies in the temp file
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec($ch);
// Now you have the cookie, you can POST login values
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 2);
curl_setopt($ch, CURLOPT_POSTFIELDS, "Email=".$_GET['email']."&Password=".$_GET['password']);
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile); // Uses cookies from the temp file
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // Tells cURL to follow redirects
$output = curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, "https://www.nibeuplink.com/System/".$_GET['code']."/Status/ServiceInfo");
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt($ch, CURLOPT_POST, 0);
$result = curl_exec($ch);
$pattern = '/<h3>(.*?)<\/h3>\s*<table[^>]*>.+?<tbody>(.+?)<\/tbody>\s*<\/table>/s';
if ($debug) echo "pattern: <xmp>".$pattern."</xmp><br>";
$pattern2 = '/<tr>\s*<td>(.+?)<span[^>]*>[^<]*<\/span>\s*<\/td>\s*<td>\s*<span[^>]*>([^<]*)<\/span>\s*<\/td>\s*<\/tr>/s';
if ($debug) echo "pattern2: <xmp>".$pattern2."</xmp><br>";
preg_match_all($pattern, $result, $matches);
// build json format from matches
echo '{';
$first = true;
foreach ($matches[1] as $i => $title) {
echo ($first ? '"' : ',"').trim($title).'":{';
$content = $matches[2][$i];
preg_match_all($pattern2, $content, $values);
$nestedFirst = true;
foreach ($values[1] as $j => $field) {
echo ($nestedFirst ? '"' : ',"').trim($field).'":"'.$values[2][$j].'"';
$nestedFirst = false;
}
echo "}";
$first = false;
}
echo ",\"time\":{\"Last fetch\":\"$date\"}";
echo "}";
if ($debug) {
echo "<pre><xmp>";
echo print_r($matches);
echo "<br><br>";
echo $result;
echo "</xmp></pre>";
}
?>
You can make an ajax call to php script to refresh the part of webpage. I don't understand what do you mean by io i.e. are you talking about fetching the data from database and if any changes occurred in database then only newly added records must be fetched. If you mean it in that sense then you can use cookie to track any new records added into database and only if it finds new records it can make ajax call to php script to run your algorithm on fetched total dataset.

how do I get my browser session from a get request

How do I use my current browser session and create a GET request in PHP based on that?
I tried the following:
$ch = curl_init();
// set url
curl_setopt($ch, CURLOPT_URL, "http://localhost/x/");
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//enable headers
curl_setopt($ch, CURLOPT_HEADER, 1);
//get only headers
curl_setopt($ch, CURLOPT_NOBODY, 1);
// $output contains the output string
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
$headers=array();
$data=explode("\n",$output);
$headers['status']=$data[0];
array_shift($data);
foreach($data as $part){
$middle=explode(":",$part);
$headers[trim($middle[0])] = trim($middle[1]);
}
//print all headers as array
echo "<pre>";
print_r($headers);
echo "</pre>";
after checking, it seems that it is not really using my session to create the GET request, rather, it is creating an entire new GET request.
I also tried to pull the headers only of a get_headers() but that seemed to have the same issue for some reason. Is there a way to pull up my current session from a browser and use that in a GET request rather than having a new one every time? (I will be in the same machine so the session is mine). In short, I want to initiate a GET request using my current session.

CRON Job using Curl.

I have a PHP Code, which reloads itself with another GET value. Like: example.com?number=453 and it keeps doing this for days. I was doing this in the browser. But i found cron job is way better.
So, I need to use CURL to reload the page with a new GET value like ?number=550. So this is the code, which i use (found it on stackoverflow)
function curl_download($Url){
// is cURL installed yet?
if (!function_exists('curl_init')){
die('Sorry cURL is not installed!');
}
// OK cool - then let's create a new cURL resource handle
$ch = curl_init();
// Now set some options (most are optional)
// Set URL to download
curl_setopt($ch, CURLOPT_URL, $Url);
// Set a referer
curl_setopt($ch, CURLOPT_REFERER, "http://www.example.org/yay.htm");
// User agent
curl_setopt($ch, CURLOPT_USERAGENT, "MozillaXYZ/1.0");
// Include header in result? (0 = yes, 1 = no)
curl_setopt($ch, CURLOPT_HEADER, 0);
// Should cURL return or print out the data? (true = return, false = print)
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Timeout in seconds
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
// Download the given URL, and return output
$output = curl_exec($ch);
// Close the cURL resource, and free system resources
curl_close($ch);
return $output;
}
$output = curl_download("http://www.example.com/yourscript.php?number='$requestsDone'");
And at the end of the page, i use this, to call the function
curl_download($Url);
But i am getting this error:
Notice: Undefined variable: Url
in the last line, i.e curl_download($Url);
This is how we call the function right? What's wrong? Also is there any mistake or improvement, I can make in the code?
You not assign values ​​to variables $Url.
Before you can assign values ​​to variables $Url. That like $Url = "http://domaintest.com/?number=550";
curl_download($Url);

PHP file_get_contents doesn't retrieve cookies

I've read some topics about this problem..and some people says to use cURL (althought I don't know how..)
I have two files, index.php and response.php.
index.php
/* After the body tag, in the middle of screen */
<?php echo file_get_contents('http://localhost/football/classes/response.php?type=clients'); ?>
This code was working perfectly until I realize that I need to retrieve the same info, but with cookies, when the page loads.
My old response.php file was this:
switch($_REQUEST['type']){
case 'clients':
$content = $load->clients();
echo $content;
break;
}
But now I need to do the same code but with a parameter, inside the function clients(). This parameter is a cookie.
switch($_REQUEST['type']){
case 'clients':
$display = 0;
if(isset($_COOKIE['display'])){
$display = 1;
}
$content = $load->clients($display);
echo $content;
break;
}
I do always receive $display = 0; because PHP doesn't detect the cookie. Although, this cookie is initialized in Chrome Cookies. Even if I do var_dump($_COOKIE); I still don't receive nothing.
I know that this problem is because of file_get_contents() How can I solve this problem?
Thanks.
Edit: Tried the #Martin solution, without success. file_get_contents receive cookies
<?php
$ckfile = tempnam ("/tmp", "CURLCOOKIE");
$ch = curl_init ("http://localhost/football/index.php");
curl_setopt ($ch, CURLOPT_COOKIEJAR, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
$ch = curl_init ("http://localhost/football/classes/response.php?type=clients");
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
echo $output;
?>
The page keeps on looping and doesn't stop. It shows nothing.
The super global $_COOKIE only contains the cookies that your visitor sent to your Web server when it requested your page, it will never contain the cookies that another server may send to your application when it loads another page.
But you can still retrieves all the response headers using $http_response_header.
More info in the PHP doc : $http_response_header

How to let Curl use same cookie as the browser from PHP

I have a PHP script that does an HTTP request on behalf of the browser and the outputs the response to the browser. Problem is when I click the links from the browser on this page it complains about cookie variables. I'm assuming it needs the browsers cookie(s) for the site.
how can I intercept and forward it to the remote site?
This is how I forward all browser cookies to curl and also return all cookies for the curl request back to the browser. For this I needed to solve some problems like getting cookies from curl, parsing http header, sending multiple cookies and session locking:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// get http header for cookies
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
// forward current cookies to curl
$cookies = array();
foreach ($_COOKIE as $key => $value)
{
if ($key != 'Array')
{
$cookies[] = $key . '=' . $value;
}
}
curl_setopt( $ch, CURLOPT_COOKIE, implode(';', $cookies) );
// Stop session so curl can use the same session without conflicts
session_write_close();
$response = curl_exec($ch);
curl_close($ch);
// Session restart
session_start();
// Seperate header and body
list($header, $body) = explode("\r\n\r\n", $response, 2);
// extract cookies form curl and forward them to browser
preg_match_all('/^(Set-Cookie:\s*[^\n]*)$/mi', $header, $cookies);
foreach($cookies[0] AS $cookie)
{
header($cookie, false);
}
echo $body;
In fact, it is possible. You just have to take the cookie ofthe browser and pass it as a parameter to curl to mimik the browser.
It's like a session jacking...
Here is a sample code:
// Init curl connection
$curl = curl_init('http://otherserver.com/');
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
// You can add your GET or POST param
// Retrieving session ID
$strCookie = 'PHPSESSID=' . $_COOKIE['PHPSESSID'] . '; path=/';
// We pass the sessionid of the browser within the curl request
curl_setopt( $curl, CURLOPT_COOKIE, $strCookie );
// We receive the answer as if we were the browser
$curl_response = curl_exec($curl);
It works very well if your purpose is to call another website, but this will fail if you call your web server (the same that is launching the curl command). It's because your session file is still open/locked by this script so the URL you are calling can't access it.
If you want to bypass that restriction (call a page on the same server), you have to close the session file with this code before you execute the curl :
$curl = curl_init('http://sameserver.com/');
//...
session_write_close();
$curl_response = curl_exec($curl);
Hope this will help someone :)
From curl_setopt:
By default, libcurl always stores and loads all cookies, independent if they are session cookies or not.
However you may need to set cookies directly, which can be done using:
curl_setopt($ch, CURLOPT_COOKIE, 'foo=bar');
Which is the same as the Set-Cookie HTTP header. Check you're not using curl_setopt($ch, CURLOPT_COOKIESESSION, true) as this will make libcurl ignore some cookies.
You can't.
If you curl the request, you will need to parse the output, and replace all links so they go thru your server.
www.yourdomain.com/f?=www.someotherdomain.com/realpage
The only way this would work is if you use persistent cookies in your curl request. CURL can keep cookies itself. Assign a session ID to the cookie file (in curl) so subsequent requests get the same cookies. When a user clicks a link, you will need to curl the request again.
It is a security issue to allow site1 to set cookies for site2. Imagine if you could set cookies in the browser for paypal and trick the user into thinking they had logged int or some other malicious action.
The Cookie is usually sent with the HTTP request header like
Host stackoverflow.com
User-Agent ...
Accept-Language en-us,en;q=0.5
Referer http://stackoverflow.com/unanswered
Cookie bla=blabla;blubb=blu
So I guess that just have to modify the cookie part in your header.
PiTheNumber's answer was great but I ran into some issues with it that caused it to still print the headers to the page. So I adjusted it to use the more reliable curl_getinfo function. This version also follows redirects.
public function get_page_content( $url ) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_HEADER, 1);
// Forward current cookies to curl
$cookies = array();
foreach ($_COOKIE as $key => $value) {
if ($key != 'Array') {
$cookies[] = $key . '=' . $value;
}
}
curl_setopt( $ch, CURLOPT_COOKIE, implode(';', $cookies) );
$destination = $url;
while ($destination) {
session_write_close();
curl_setopt($ch, CURLOPT_URL, $destination);
$response = curl_exec($ch);
$curl_info = curl_getinfo($ch);
$destination = $curl_info["redirect_url"];
session_start();
}
curl_close($ch);
$headers = substr($response, 0, $curl_info["header_size"]);
$body = substr($response, $curl_info["header_size"]);
// Extract cookies from curl and forward them to browser
preg_match_all('/^(Set-Cookie:\s*[^\n]*)$/mi', $headers, $cookies);
foreach($cookies[0] AS $cookie) {
header($cookie, false);
}
return $body;
}

Categories