Get remote file size from HTTPS url - php

Till now i used the following function to get filesize of files from url.It works perfectly fine if the url is http but fails when its https.Can anyone update it to work with https.
<?php
/**
* Returns the size of a file without downloading it, or -1 if the file
* size could not be determined.
*
* #param $url - The location of the remote file to download. Cannot
* be null or empty.
*
* #return The size of the file referenced by $url, or -1 if the size
* could not be determined.
*/
function curl_get_file_size( $url ) {
// Assume failure.
$result = -1;
$curl = curl_init( $url );
// Issue a HEAD request and follow any redirects.
curl_setopt( $curl, CURLOPT_NOBODY, true );
curl_setopt( $curl, CURLOPT_HEADER, true );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $curl, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT'] );
$data = curl_exec( $curl );
curl_close( $curl );
if( $data ) {
$content_length = "unknown";
$status = "unknown";
if( preg_match( "/^HTTP\/1\.[01] (\d\d\d)/", $data, $matches ) ) {
$status = (int)$matches[1];
}
if( preg_match( "/Content-Length: (\d+)/", $data, $matches ) ) {
$content_length = (int)$matches[1];
}
// http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
if( $status == 200 || ($status > 300 && $status <= 308) ) {
$result = $content_length;
}
}
return $result;
}
?>
This is not my code btw and the full credit goes to NebuSoft
Source:
PHP: Remote file size without downloading file
Regards

You need to add the below cURL parameter to your existing set.
curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, 0);
Adding this will stop cURL from verifying the peer's certificate.

In this, you have not used curl host verify'r, you have to set this for the communication verify purpose.
Just add this to your parameters
curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, false);

Try strlen(file_get_contents($url)); this will get the length of content through this

use this one:
<?php
$remoteFile = $url;//remote url
$ch = curl_init($remoteFile);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); //not necessary unless the file redirects (like the PHP example we're using here)
$data = curl_exec($ch);
curl_close($ch);
if ($data === false) {
echo 'cURL failed';
exit;
}
$contentLength = 'unknown';
$status = 'unknown';
if (preg_match('/^HTTP\/1\.[01] (\d\d\d)/', $data, $matches)) {
$status = (int)$matches[1];
}
if (preg_match('/Content-Length: (\d+)/', $data, $matches)) {
$contentLength = (int)$matches[1];
}
echo 'HTTP Status: ' . $status . "\n";
echo 'Content-Length: ' . $contentLength;
?>
Refer: http://in3.php.net/filesize

function remotefileSize($url) {
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_NOBODY, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_MAXREDIRS, 3);
curl_exec($ch);
$filesize = curl_getinfo($ch, CURLINFO_CONTENT_LENGTH_DOWNLOAD);
curl_close($ch);
if ($filesize) return $filesize;
}
echo remotefileSize('http://www.google.com/images/srpr/logo4w.png');
//Output
//19978 bytes

Related

fetch X-WP-Total and X-WP-TotalPages response headers using php curl request

I'm trying to read the X-WP-* headers using a curl response requested on the WordPress API for fetching posts (See reference:https://developer.wordpress.org/rest-api/using-the-rest-api/pagination/) e.g https://news.harvard.edu/wp-json/wp/v2/posts.
however I'm unable to read the http response headers returned.
$url = "https://news.harvard.edu/wp-json/wp/v2/posts";
$headers = [];
$data=[];
curl_setopt($curl, CURLOPT_CONNECTTIMEOUT ,0);
curl_setopt($curl, CURLOPT_TIMEOUT, 0);
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HEADER, true);
curl_setopt($curl, CURLOPT_HEADERFUNCTION,
function ($curl, $header) use (&$headers) {
$len = strlen($header);
$header = explode(':', $header, 2);
if (count($header) < 2) // ignore invalid headers
return $len;
$headers[strtolower(trim($header[0]))][] = trim($header[1]);
return $len;
}
);
$response = curl_exec($curl);
if ($response == false) {
if(curl_errno($curl)>=1){
$error = curl_errno($curl);
curl_close($curl);
$data = ["curl_error"=>$error];
}
}else{
$data = [
"headers"=>$headers
];
curl_close($curl);
}
var_dump($data);
// $data doesn't contain X-WP-Total and X-WP-TotalPages
You were nearly there with the above but it lacks any SSL support which is why I believe it was failing. If you set the CURLOPT_HEADER=true option (as above) this code will still work but you'll not be able to decode the JSON response - you do not need to actually print the headers so set this as false or omit completely.
$url = "https://news.harvard.edu/wp-json/wp/v2/posts";
$cacert='c:/wwwroot/cacert.pem';
/*
Download a copy of CACERT.pem from
https://curl.haxx.se/docs/caextract.html
save to webserver and modify the $cacert variable
to suit - ensuring that the path you choose is
readable.
*/
$headers = [];
$data=[];
# slightly modified version of original function
# only appends matching headers to output variable.
$callback=function( $ch, $header ){
global $headers;
$len = strlen( $header );
$header = explode(':', $header, 2 );
if( count( $header ) < 2 ) return $len;
$param = strtolower( trim( $header[0] ) );
$value = trim( $header[1] );
if( strstr( $param, 'x-wp' ) ) $headers[ $param ]=$value;
return $len;
};
$curl=curl_init();
# The request will fail without including options for dealing with SSL.
if( parse_url( $url, PHP_URL_SCHEME )=='https' ){
curl_setopt( $curl, CURLOPT_SSL_VERIFYPEER, true );
curl_setopt( $curl, CURLOPT_SSL_VERIFYHOST, 2 );
curl_setopt( $curl, CURLOPT_CAINFO, $cacert );
}
curl_setopt( $curl, CURLOPT_URL, trim( $url ) );
curl_setopt( $curl, CURLOPT_CONNECTTIMEOUT ,0 );
curl_setopt( $curl, CURLOPT_TIMEOUT, 0 );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
# we do not need the header in the output
curl_setopt( $curl, CURLOPT_HEADER, false );
# but we do need to process request headers - find any X-WP* headers
curl_setopt( $curl, CURLOPT_HEADERFUNCTION, $callback );
$res = curl_exec($curl);
curl_close( $curl );
# process the required headers...
printf( '<pre>%s</pre>', print_r( $headers, true ) );
$json=json_decode( $res );
# do stuff with JSON data
This outputs the following:
Array
(
[x-wp-total] => 25118
[x-wp-totalpages] => 2512
)

Setting cookies for user/browser after a CURL post request [duplicate]

I have some problem with PHP Curl and cookies authentication.
I have a file Connector.php which authenticates users on another server and returns the cookie of the current user.
The Problem is that I want to authenticate thousands of users with curl but it authenticates and saves COOKIES only for one user at a time.
The code for connector.php is this:
<?php
if(!count($_REQUEST)) {
die("No Access!");
}
//Core Url For Services
define ('ServiceCore', 'http://example.com/core/');
//Which Internal Service Should Be Called
$path = $_GET['service'];
//Service To Be Queried
$url = ServiceCore.$path;
//Open the Curl session
$session = curl_init($url);
// If it's a GET, put the GET data in the body
if ($_GET['service']) {
//Iterate Over GET Vars
$postvars = '';
foreach($_GET as $key=>$val) {
if($key!='service') {
$postvars.="$key=$val&";
}
}
curl_setopt ($session, CURLOPT_POST, true);
curl_setopt ($session, CURLOPT_POSTFIELDS, $postvars);
}
//Create And Save Cookies
$tmpfname = dirname(__FILE__).'/cookie.txt';
curl_setopt($session, CURLOPT_COOKIEJAR, $tmpfname);
curl_setopt($session, CURLOPT_COOKIEFILE, $tmpfname);
curl_setopt($session, CURLOPT_HEADER, false);
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
curl_setopt($session, CURLOPT_FOLLOWLOCATION, true);
// EXECUTE
$json = curl_exec($session);
echo $json;
curl_close($session);
?>
Here is the process of authentication:
User enters username and password: Connector.php?service=logon&user_name=user32&user_pass=123
Connector.php?service=logosessionInfo returns info about the user based on the cookies saved earlier with logon service.
The problem is that this code saves the cookie in one file for each user and can't handle multiple user authentications.
You can specify the cookie file with a curl opt. You could use a unique file for each user.
curl_setopt( $curl_handle, CURLOPT_COOKIESESSION, true );
curl_setopt( $curl_handle, CURLOPT_COOKIEJAR, uniquefilename );
curl_setopt( $curl_handle, CURLOPT_COOKIEFILE, uniquefilename );
The best way to handle it would be to stick your request logic into a curl function and just pass the unique file name in as a parameter.
function fetch( $url, $z=null ) {
$ch = curl_init();
$useragent = isset($z['useragent']) ? $z['useragent'] : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2';
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_POST, isset($z['post']) );
if( isset($z['post']) ) curl_setopt( $ch, CURLOPT_POSTFIELDS, $z['post'] );
if( isset($z['refer']) ) curl_setopt( $ch, CURLOPT_REFERER, $z['refer'] );
curl_setopt( $ch, CURLOPT_USERAGENT, $useragent );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, ( isset($z['timeout']) ? $z['timeout'] : 5 ) );
curl_setopt( $ch, CURLOPT_COOKIEJAR, $z['cookiefile'] );
curl_setopt( $ch, CURLOPT_COOKIEFILE, $z['cookiefile'] );
$result = curl_exec( $ch );
curl_close( $ch );
return $result;
}
I use this for quick grabs. It takes the url and an array of options.
In working with a similar problem I created the following function after combining a lot of resources I ran into on the web, and adding my own cookie handling. Hopefully this is useful to someone else.
function get_web_page( $url, $cookiesIn = '' ){
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true, //return headers in addition to content
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
CURLINFO_HEADER_OUT => true,
CURLOPT_SSL_VERIFYPEER => true, // Validate SSL Certificates
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_COOKIE => $cookiesIn
);
$ch = curl_init( $url );
curl_setopt_array( $ch, $options );
$rough_content = curl_exec( $ch );
$err = curl_errno( $ch );
$errmsg = curl_error( $ch );
$header = curl_getinfo( $ch );
curl_close( $ch );
$header_content = substr($rough_content, 0, $header['header_size']);
$body_content = trim(str_replace($header_content, '', $rough_content));
$pattern = "#Set-Cookie:\\s+(?<cookie>[^=]+=[^;]+)#m";
preg_match_all($pattern, $header_content, $matches);
$cookiesOut = implode("; ", $matches['cookie']);
$header['errno'] = $err;
$header['errmsg'] = $errmsg;
$header['headers'] = $header_content;
$header['content'] = $body_content;
$header['cookies'] = $cookiesOut;
return $header;
}
First create temporary cookie using tempnam() function:
$ckfile = tempnam ("/tmp", "CURLCOOKIE");
Then execute curl init witch saves the cookie as a temporary file:
$ch = curl_init ("http://uri.com/");
curl_setopt ($ch, CURLOPT_COOKIEJAR, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
Or visit a page using the cookie stored in the temporary file:
$ch = curl_init ("http://somedomain.com/cookiepage.php");
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
This will initialize the cookie for the page:
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
Here you can find some useful info about cURL & cookies http://docstore.mik.ua/orelly/webprog/pcook/ch11_04.htm .
You can also use this well done method https://github.com/alixaxel/phunction/blob/master/phunction/Net.php#L89 like a function:
function CURL($url, $data = null, $method = 'GET', $cookie = null, $options = null, $retries = 3)
{
$result = false;
if ((extension_loaded('curl') === true) && (is_resource($curl = curl_init()) === true))
{
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_FAILONERROR, true);
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
if (preg_match('~^(?:DELETE|GET|HEAD|OPTIONS|POST|PUT)$~i', $method) > 0)
{
if (preg_match('~^(?:HEAD|OPTIONS)$~i', $method) > 0)
{
curl_setopt_array($curl, array(CURLOPT_HEADER => true, CURLOPT_NOBODY => true));
}
else if (preg_match('~^(?:POST|PUT)$~i', $method) > 0)
{
if (is_array($data) === true)
{
foreach (preg_grep('~^#~', $data) as $key => $value)
{
$data[$key] = sprintf('#%s', rtrim(str_replace('\\', '/', realpath(ltrim($value, '#'))), '/') . (is_dir(ltrim($value, '#')) ? '/' : ''));
}
if (count($data) != count($data, COUNT_RECURSIVE))
{
$data = http_build_query($data, '', '&');
}
}
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
}
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, strtoupper($method));
if (isset($cookie) === true)
{
curl_setopt_array($curl, array_fill_keys(array(CURLOPT_COOKIEJAR, CURLOPT_COOKIEFILE), strval($cookie)));
}
if ((intval(ini_get('safe_mode')) == 0) && (ini_set('open_basedir', null) !== false))
{
curl_setopt_array($curl, array(CURLOPT_MAXREDIRS => 5, CURLOPT_FOLLOWLOCATION => true));
}
if (is_array($options) === true)
{
curl_setopt_array($curl, $options);
}
for ($i = 1; $i <= $retries; ++$i)
{
$result = curl_exec($curl);
if (($i == $retries) || ($result !== false))
{
break;
}
usleep(pow(2, $i - 2) * 1000000);
}
}
curl_close($curl);
}
return $result;
}
And pass this as $cookie parameter:
$cookie_jar = tempnam('/tmp','cookie');
You can define different cookies for every user with CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR. Make different file for every user so each one would have it's own cookie-based session on remote server.
Solutions which are described above, even with unique CookieFile names, can cause a lot of problems on scale.
We had to serve a lot of authentications with this solution and our server went down because of high file read write actions.
The solution for this was to use Apache Reverse Proxy and omit CURL requests at all.
Details how to use Proxy on Apache can be found here:
https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html

Login to site using php curl [duplicate]

I have some problem with PHP Curl and cookies authentication.
I have a file Connector.php which authenticates users on another server and returns the cookie of the current user.
The Problem is that I want to authenticate thousands of users with curl but it authenticates and saves COOKIES only for one user at a time.
The code for connector.php is this:
<?php
if(!count($_REQUEST)) {
die("No Access!");
}
//Core Url For Services
define ('ServiceCore', 'http://example.com/core/');
//Which Internal Service Should Be Called
$path = $_GET['service'];
//Service To Be Queried
$url = ServiceCore.$path;
//Open the Curl session
$session = curl_init($url);
// If it's a GET, put the GET data in the body
if ($_GET['service']) {
//Iterate Over GET Vars
$postvars = '';
foreach($_GET as $key=>$val) {
if($key!='service') {
$postvars.="$key=$val&";
}
}
curl_setopt ($session, CURLOPT_POST, true);
curl_setopt ($session, CURLOPT_POSTFIELDS, $postvars);
}
//Create And Save Cookies
$tmpfname = dirname(__FILE__).'/cookie.txt';
curl_setopt($session, CURLOPT_COOKIEJAR, $tmpfname);
curl_setopt($session, CURLOPT_COOKIEFILE, $tmpfname);
curl_setopt($session, CURLOPT_HEADER, false);
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
curl_setopt($session, CURLOPT_FOLLOWLOCATION, true);
// EXECUTE
$json = curl_exec($session);
echo $json;
curl_close($session);
?>
Here is the process of authentication:
User enters username and password: Connector.php?service=logon&user_name=user32&user_pass=123
Connector.php?service=logosessionInfo returns info about the user based on the cookies saved earlier with logon service.
The problem is that this code saves the cookie in one file for each user and can't handle multiple user authentications.
You can specify the cookie file with a curl opt. You could use a unique file for each user.
curl_setopt( $curl_handle, CURLOPT_COOKIESESSION, true );
curl_setopt( $curl_handle, CURLOPT_COOKIEJAR, uniquefilename );
curl_setopt( $curl_handle, CURLOPT_COOKIEFILE, uniquefilename );
The best way to handle it would be to stick your request logic into a curl function and just pass the unique file name in as a parameter.
function fetch( $url, $z=null ) {
$ch = curl_init();
$useragent = isset($z['useragent']) ? $z['useragent'] : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2';
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_POST, isset($z['post']) );
if( isset($z['post']) ) curl_setopt( $ch, CURLOPT_POSTFIELDS, $z['post'] );
if( isset($z['refer']) ) curl_setopt( $ch, CURLOPT_REFERER, $z['refer'] );
curl_setopt( $ch, CURLOPT_USERAGENT, $useragent );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, ( isset($z['timeout']) ? $z['timeout'] : 5 ) );
curl_setopt( $ch, CURLOPT_COOKIEJAR, $z['cookiefile'] );
curl_setopt( $ch, CURLOPT_COOKIEFILE, $z['cookiefile'] );
$result = curl_exec( $ch );
curl_close( $ch );
return $result;
}
I use this for quick grabs. It takes the url and an array of options.
In working with a similar problem I created the following function after combining a lot of resources I ran into on the web, and adding my own cookie handling. Hopefully this is useful to someone else.
function get_web_page( $url, $cookiesIn = '' ){
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true, //return headers in addition to content
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
CURLINFO_HEADER_OUT => true,
CURLOPT_SSL_VERIFYPEER => true, // Validate SSL Certificates
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_COOKIE => $cookiesIn
);
$ch = curl_init( $url );
curl_setopt_array( $ch, $options );
$rough_content = curl_exec( $ch );
$err = curl_errno( $ch );
$errmsg = curl_error( $ch );
$header = curl_getinfo( $ch );
curl_close( $ch );
$header_content = substr($rough_content, 0, $header['header_size']);
$body_content = trim(str_replace($header_content, '', $rough_content));
$pattern = "#Set-Cookie:\\s+(?<cookie>[^=]+=[^;]+)#m";
preg_match_all($pattern, $header_content, $matches);
$cookiesOut = implode("; ", $matches['cookie']);
$header['errno'] = $err;
$header['errmsg'] = $errmsg;
$header['headers'] = $header_content;
$header['content'] = $body_content;
$header['cookies'] = $cookiesOut;
return $header;
}
First create temporary cookie using tempnam() function:
$ckfile = tempnam ("/tmp", "CURLCOOKIE");
Then execute curl init witch saves the cookie as a temporary file:
$ch = curl_init ("http://uri.com/");
curl_setopt ($ch, CURLOPT_COOKIEJAR, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
Or visit a page using the cookie stored in the temporary file:
$ch = curl_init ("http://somedomain.com/cookiepage.php");
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
This will initialize the cookie for the page:
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
Here you can find some useful info about cURL & cookies http://docstore.mik.ua/orelly/webprog/pcook/ch11_04.htm .
You can also use this well done method https://github.com/alixaxel/phunction/blob/master/phunction/Net.php#L89 like a function:
function CURL($url, $data = null, $method = 'GET', $cookie = null, $options = null, $retries = 3)
{
$result = false;
if ((extension_loaded('curl') === true) && (is_resource($curl = curl_init()) === true))
{
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_FAILONERROR, true);
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
if (preg_match('~^(?:DELETE|GET|HEAD|OPTIONS|POST|PUT)$~i', $method) > 0)
{
if (preg_match('~^(?:HEAD|OPTIONS)$~i', $method) > 0)
{
curl_setopt_array($curl, array(CURLOPT_HEADER => true, CURLOPT_NOBODY => true));
}
else if (preg_match('~^(?:POST|PUT)$~i', $method) > 0)
{
if (is_array($data) === true)
{
foreach (preg_grep('~^#~', $data) as $key => $value)
{
$data[$key] = sprintf('#%s', rtrim(str_replace('\\', '/', realpath(ltrim($value, '#'))), '/') . (is_dir(ltrim($value, '#')) ? '/' : ''));
}
if (count($data) != count($data, COUNT_RECURSIVE))
{
$data = http_build_query($data, '', '&');
}
}
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
}
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, strtoupper($method));
if (isset($cookie) === true)
{
curl_setopt_array($curl, array_fill_keys(array(CURLOPT_COOKIEJAR, CURLOPT_COOKIEFILE), strval($cookie)));
}
if ((intval(ini_get('safe_mode')) == 0) && (ini_set('open_basedir', null) !== false))
{
curl_setopt_array($curl, array(CURLOPT_MAXREDIRS => 5, CURLOPT_FOLLOWLOCATION => true));
}
if (is_array($options) === true)
{
curl_setopt_array($curl, $options);
}
for ($i = 1; $i <= $retries; ++$i)
{
$result = curl_exec($curl);
if (($i == $retries) || ($result !== false))
{
break;
}
usleep(pow(2, $i - 2) * 1000000);
}
}
curl_close($curl);
}
return $result;
}
And pass this as $cookie parameter:
$cookie_jar = tempnam('/tmp','cookie');
You can define different cookies for every user with CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR. Make different file for every user so each one would have it's own cookie-based session on remote server.
Solutions which are described above, even with unique CookieFile names, can cause a lot of problems on scale.
We had to serve a lot of authentications with this solution and our server went down because of high file read write actions.
The solution for this was to use Apache Reverse Proxy and omit CURL requests at all.
Details how to use Proxy on Apache can be found here:
https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html

How do I get youtube video direct link file size info using php curl?

I want to display the size of youtube video size on the direct link. I have a link to an mp4 format file that is 98 MB in size. I want to display the size when you browse this link your will get file.
Direct url link:
$url ='https://r4---sn-a8au-p5qs.googlevideo.com/videoplayback?signature=9216B94F5EA16F023DE3D34C6881F8AFA20E1EBA.4B56920C4B2BAB848AC847E0EB4C1E01FB01685C&itag=22&ratebypass=yes&expire=1424904527&id=o-AA8OyXA6gxAGHzkwqf94rIO0LTrhD8iHuUc9lMI9ED76&pl=46&fexp=905657%2C907263%2C916942%2C923382%2C927622%2C934601%2C934954%2C9406984%2C943917%2C947225%2C947240%2C947601%2C948124%2C951703%2C952302%2C952605%2C952612%2C952901%2C955301%2C957201%2C959701&mm=31&ipbits=0&dur=1366.192&mt=1424882845&ms=au&key=yt5&upn=SIiUMnjm5o0&source=youtube&sparams=dur%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Cmm%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&mv=m&initcwndbps=1567500&ip=2001%3A4802%3A7805%3A101%3Abe76%3A4eff%3Afe20%3A31e4&sver=3&requiressl=yes&title=PHP%3A+Create+Your+Own+MVC+%28Part+1%29';
php curl code I am using to get code
function retrieve_remote_file_size($url){
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$data = curl_exec($ch);
$size = curl_getinfo($ch, CURLINFO_CONTENT_LENGTH_DOWNLOAD);
curl_close($ch);
return $size;
}
It is giving me 1419. How do I get the correct file size?
getID3 supports video formats. See: http://getid3.sourceforge.net/
$getID3 = new getID3;
$file = $getID3->analyze($filename);
echo("Filesize: ".$file['filesize']." bytes<br />");
Note: You must include the getID3 classes before this will work! See the above link.
If you have the ability to modify the PHP installation on your server, a PHP extension for this purpose is ffmpeg-php. See: http://ffmpeg-php.sourceforge.net/
edit:
Found something about this here:
Here's the best way (that I've found) to get the size of a remote file. Note that HEAD requests don't get the actual body of the request, they just retrieve the headers. So making a HEAD request to a resource that is 100MB will take the same amount of time as a HEAD request to a resource that is 1KB.
$curl = curl_init( $url );
// Issue a HEAD request and follow any redirects.
curl_setopt( $curl, CURLOPT_NOBODY, true );
curl_setopt( $curl, CURLOPT_HEADER, true );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $curl, CURLOPT_USERAGENT, get_user_agent_string() );
$data = curl_exec( $curl );
curl_close( $curl );
if( $data ) {
$content_length = "unknown";
$status = "unknown";
if( preg_match( "/^HTTP\/1\.[01] (\d\d\d)/", $data, $matches ) ) {
$status = (int)$matches[1];
}
if( preg_match( "/Content-Length: (\d+)/", $data, $matches ) ) {
$content_length = (int)$matches[1];
}
// http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
if( $status == 200 || ($status > 300 && $status <= 308) ) {
$result = $content_length;
}
}
return $result;
}
?>
Usage:
$file_size = curl_get_file_size( "http://stackoverflow.com/questions/2602612/php-remote-file-size-without-downloading-file" );
The 1491 bytes you get is the length of the redirect. Add the following curl option:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, True);
According to the documentation it should be true by default, but it wasn't for me.

PHP Curl And Cookies

I have some problem with PHP Curl and cookies authentication.
I have a file Connector.php which authenticates users on another server and returns the cookie of the current user.
The Problem is that I want to authenticate thousands of users with curl but it authenticates and saves COOKIES only for one user at a time.
The code for connector.php is this:
<?php
if(!count($_REQUEST)) {
die("No Access!");
}
//Core Url For Services
define ('ServiceCore', 'http://example.com/core/');
//Which Internal Service Should Be Called
$path = $_GET['service'];
//Service To Be Queried
$url = ServiceCore.$path;
//Open the Curl session
$session = curl_init($url);
// If it's a GET, put the GET data in the body
if ($_GET['service']) {
//Iterate Over GET Vars
$postvars = '';
foreach($_GET as $key=>$val) {
if($key!='service') {
$postvars.="$key=$val&";
}
}
curl_setopt ($session, CURLOPT_POST, true);
curl_setopt ($session, CURLOPT_POSTFIELDS, $postvars);
}
//Create And Save Cookies
$tmpfname = dirname(__FILE__).'/cookie.txt';
curl_setopt($session, CURLOPT_COOKIEJAR, $tmpfname);
curl_setopt($session, CURLOPT_COOKIEFILE, $tmpfname);
curl_setopt($session, CURLOPT_HEADER, false);
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
curl_setopt($session, CURLOPT_FOLLOWLOCATION, true);
// EXECUTE
$json = curl_exec($session);
echo $json;
curl_close($session);
?>
Here is the process of authentication:
User enters username and password: Connector.php?service=logon&user_name=user32&user_pass=123
Connector.php?service=logosessionInfo returns info about the user based on the cookies saved earlier with logon service.
The problem is that this code saves the cookie in one file for each user and can't handle multiple user authentications.
You can specify the cookie file with a curl opt. You could use a unique file for each user.
curl_setopt( $curl_handle, CURLOPT_COOKIESESSION, true );
curl_setopt( $curl_handle, CURLOPT_COOKIEJAR, uniquefilename );
curl_setopt( $curl_handle, CURLOPT_COOKIEFILE, uniquefilename );
The best way to handle it would be to stick your request logic into a curl function and just pass the unique file name in as a parameter.
function fetch( $url, $z=null ) {
$ch = curl_init();
$useragent = isset($z['useragent']) ? $z['useragent'] : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2';
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt( $ch, CURLOPT_POST, isset($z['post']) );
if( isset($z['post']) ) curl_setopt( $ch, CURLOPT_POSTFIELDS, $z['post'] );
if( isset($z['refer']) ) curl_setopt( $ch, CURLOPT_REFERER, $z['refer'] );
curl_setopt( $ch, CURLOPT_USERAGENT, $useragent );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, ( isset($z['timeout']) ? $z['timeout'] : 5 ) );
curl_setopt( $ch, CURLOPT_COOKIEJAR, $z['cookiefile'] );
curl_setopt( $ch, CURLOPT_COOKIEFILE, $z['cookiefile'] );
$result = curl_exec( $ch );
curl_close( $ch );
return $result;
}
I use this for quick grabs. It takes the url and an array of options.
In working with a similar problem I created the following function after combining a lot of resources I ran into on the web, and adding my own cookie handling. Hopefully this is useful to someone else.
function get_web_page( $url, $cookiesIn = '' ){
$options = array(
CURLOPT_RETURNTRANSFER => true, // return web page
CURLOPT_HEADER => true, //return headers in addition to content
CURLOPT_FOLLOWLOCATION => true, // follow redirects
CURLOPT_ENCODING => "", // handle all encodings
CURLOPT_AUTOREFERER => true, // set referer on redirect
CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect
CURLOPT_TIMEOUT => 120, // timeout on response
CURLOPT_MAXREDIRS => 10, // stop after 10 redirects
CURLINFO_HEADER_OUT => true,
CURLOPT_SSL_VERIFYPEER => true, // Validate SSL Certificates
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_COOKIE => $cookiesIn
);
$ch = curl_init( $url );
curl_setopt_array( $ch, $options );
$rough_content = curl_exec( $ch );
$err = curl_errno( $ch );
$errmsg = curl_error( $ch );
$header = curl_getinfo( $ch );
curl_close( $ch );
$header_content = substr($rough_content, 0, $header['header_size']);
$body_content = trim(str_replace($header_content, '', $rough_content));
$pattern = "#Set-Cookie:\\s+(?<cookie>[^=]+=[^;]+)#m";
preg_match_all($pattern, $header_content, $matches);
$cookiesOut = implode("; ", $matches['cookie']);
$header['errno'] = $err;
$header['errmsg'] = $errmsg;
$header['headers'] = $header_content;
$header['content'] = $body_content;
$header['cookies'] = $cookiesOut;
return $header;
}
First create temporary cookie using tempnam() function:
$ckfile = tempnam ("/tmp", "CURLCOOKIE");
Then execute curl init witch saves the cookie as a temporary file:
$ch = curl_init ("http://uri.com/");
curl_setopt ($ch, CURLOPT_COOKIEJAR, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
Or visit a page using the cookie stored in the temporary file:
$ch = curl_init ("http://somedomain.com/cookiepage.php");
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);
This will initialize the cookie for the page:
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
Here you can find some useful info about cURL & cookies http://docstore.mik.ua/orelly/webprog/pcook/ch11_04.htm .
You can also use this well done method https://github.com/alixaxel/phunction/blob/master/phunction/Net.php#L89 like a function:
function CURL($url, $data = null, $method = 'GET', $cookie = null, $options = null, $retries = 3)
{
$result = false;
if ((extension_loaded('curl') === true) && (is_resource($curl = curl_init()) === true))
{
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_FAILONERROR, true);
curl_setopt($curl, CURLOPT_AUTOREFERER, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
if (preg_match('~^(?:DELETE|GET|HEAD|OPTIONS|POST|PUT)$~i', $method) > 0)
{
if (preg_match('~^(?:HEAD|OPTIONS)$~i', $method) > 0)
{
curl_setopt_array($curl, array(CURLOPT_HEADER => true, CURLOPT_NOBODY => true));
}
else if (preg_match('~^(?:POST|PUT)$~i', $method) > 0)
{
if (is_array($data) === true)
{
foreach (preg_grep('~^#~', $data) as $key => $value)
{
$data[$key] = sprintf('#%s', rtrim(str_replace('\\', '/', realpath(ltrim($value, '#'))), '/') . (is_dir(ltrim($value, '#')) ? '/' : ''));
}
if (count($data) != count($data, COUNT_RECURSIVE))
{
$data = http_build_query($data, '', '&');
}
}
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
}
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, strtoupper($method));
if (isset($cookie) === true)
{
curl_setopt_array($curl, array_fill_keys(array(CURLOPT_COOKIEJAR, CURLOPT_COOKIEFILE), strval($cookie)));
}
if ((intval(ini_get('safe_mode')) == 0) && (ini_set('open_basedir', null) !== false))
{
curl_setopt_array($curl, array(CURLOPT_MAXREDIRS => 5, CURLOPT_FOLLOWLOCATION => true));
}
if (is_array($options) === true)
{
curl_setopt_array($curl, $options);
}
for ($i = 1; $i <= $retries; ++$i)
{
$result = curl_exec($curl);
if (($i == $retries) || ($result !== false))
{
break;
}
usleep(pow(2, $i - 2) * 1000000);
}
}
curl_close($curl);
}
return $result;
}
And pass this as $cookie parameter:
$cookie_jar = tempnam('/tmp','cookie');
You can define different cookies for every user with CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR. Make different file for every user so each one would have it's own cookie-based session on remote server.
Solutions which are described above, even with unique CookieFile names, can cause a lot of problems on scale.
We had to serve a lot of authentications with this solution and our server went down because of high file read write actions.
The solution for this was to use Apache Reverse Proxy and omit CURL requests at all.
Details how to use Proxy on Apache can be found here:
https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html

Categories