Php curl get amount of bytes written - php
How can i know how much data is written in php curl .
Here is my code which downloads i.e writes the data to my local server from a remote url . But i want to know how much data has been written till now .
<?php
$url = 'https://speed.hetzner.de/1GB.bin';
$path = $_SERVER['DOCUMENT_ROOT'] . '/1gb.bin';
$fp = fopen($path, 'w');
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FILE, $fp);
$data = curl_exec($ch);
curl_close($ch);
fclose($fp);
I use this for the downloaded size in bytes (including the size of the file as it is the body of the response) (after you call curl_exec($ch);)
// $ch is the curl handle
$info = curl_getinfo($ch);
echo $info['size_download'];
CURLINFO_SIZE_DOWNLOAD - Total number of bytes downloaded
This is quoted from libcurl documenation
The amount is only for the latest transfer and will be reset again
for each new transfer. This counts actual payload data, what's also
commonly called body. All meta and header data are excluded and will
not be counted in this number.
And this for the size of the request you made with curl in bytes
$info = curl_getinfo($ch);
echo $info['request_size'];
CURLINFO_REQUEST_SIZE - Total size of issued requests, currently only for HTTP requests
you can also use the function with the opt parameter set to one of the function constants, like
echo curl_getinfo($ch, CURLINFO_REQUEST_SIZE);
echo curl_getinfo($ch, CURLINFO_SIZE_DOWNLOAD);
getinfo() function
As you told in comments by Dharman, don't switch off CURLOPT_SSL_VERIFYPEER. If you want to use https requests check this php-curl-https
Related
php://input empty when sending long JSON via raw POST
I am sending raw JSON POST data with cURL via PHP, and the server receives it correctly when the JSON is not too long. However, it seems to receive an empty body when it's bigger. This is how I get the body from the server: $dataReceived = file_get_contents("php://input"); When I log $dataReceived on the server side, the expected content is in the log when not big, e.g. this: {"products":[],"allProductIds":["898","920","937","947","979","1007","1044","1064","1124","1146","1162","1178","1194","1215","1239","1291","1346","1395","1444","1629","1673","1695","1757","1868","1929","1992","2018","2050","2096","2112","2136","2178","2225","2280","2303","2318","2340","2394","2447","2459","2476","2486","2510","2520","2558","2591","2626","2655","2689","2727","2975","3175","3224","3283","3311","3333","3342","3353","3376","3389","3400","3419","3469","3506","3535","3610","3722","3765","3782","3802","4002","4020","4039","4060","4094","4130","4171","4213","4246","4281","4317","4348","4426","4473","4502","4545","4558","4563","4663","4688","4713","4733","4757","4875","4924","4984","5028","5057","5088","5102","5117","5145","5174","5197","5216","5236","5255","5275","5310","5336","5369","5422","5433","5464","5477","5505","5516","5541","5563","5587","5616","5627","5673","5703","5744","5773","5822","5839","5903","5999","6075","6196","6326","6483","6794","8360","8429","8599","8724","8798","8953","9150","9198","9247","9295","9347","9415","9463","9636","9991","10105","10219","10337","10451","10566","10681","10796","10912","11134","11244","11354","11465","11576","11687","11798","11874","11924","11973","12025","12074","12125","12175","12640","12699","12838","12844","12848","12852","12862","12866","12870","12874","12880","12884","12888","12894","12935","12943","12947","12952","12958","12962","12966","12970","12974","12979","12988","12992","12997","13001","13010","13012","13016","13020","13026","13030","13034","13039","13049","13059","13093","13128","13144","13148","13154","13158","13162","13172","13182","13217","13223","13368","2851","3828","4804"],"brandlyProductIds":["229082","226046","223246","216800","216750","215456","213613","213157","143436","143406","143362","143312","134066","134022","221020","242298","144021","143921","242348","333387","333424","333443","333517","333624","333681","334021","334043","334086","334135","334159","334184","334220","334261","334318","334351","334376","334398","334448","334505","334530","334552","334581","334606","334639","334672","334701","334734","334763","334792","334825","334962","335099","335325","335382","335439","335464","335482","335491","335500","335521","335532","335553","335575","335680","335770","335795","335877","335991","336032","336049","336066","336088","336276","336293","336310","336329","336370","336411","336452","336493","336530","336561","336598","336631","336705","336751","336782","336819","336850","336859","336977","337002","337033","337058","337083","337133","337207","337256","337366","337403","337428","337456","337473","337490","337515","337540","337565","337582","337599","337616","337637","337682","337710","337738","337791","337800","337831","337881","337906","337928","337953","337981","338006","338031","338056","338099","338135","338175","338201","338244","338269","338340","338422","187730","338815","338968","339121","339427","340235","340308","340461","340614","340687","340840","340987","341060","341133","341206","341279","341352","341425","341675","341981","342072","342254","342350","342441","342532","342714","342805","342896","343042","343115","343188","343261","343407","343480","343553","343626","343690","343754","343818","343882","343946","344010","225836","225770","344210","344214","344216","344218","303935","303610","303601","344223","344234","344236","344238","344241","204360","344254","344256","344262","344264","344268","344270","344272","344274","344277","344279","344281","344283","344285","344287","344288","344290","344292","344295","344299","344301","344303","344308","344313","344330","344333","344335","344337","344340","344342","344344","344349","344354","344371","150836","344434"]} But i get an empty $dataReceived for example when the sent JSON is the one in this pastebin https://pastebin.com/p5ZLrxig This is the code i have to send the request, with verbose log protected static function _apiCall(string $url, array $additionalHeaders = [], string $method="GET", string $data=''){ if($method=="POST"){ $curlLog = __DIR__ . '/curl-log.txt'; $f = fopen($curlLog, 'w'); $ch = curl_init($url); curl_setopt($ch, CURLOPT_VERBOSE, true); curl_setopt($ch, CURLOPT_STDERR, $f); curl_setopt($ch, CURLOPT_STDOUT, $f); curl_setopt($ch, CURLOPT_POST, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $hdr = [ 'Content-Type: application/json', 'Content-Length: ' . strlen($data) ]; foreach($additionalHeaders as $h) $hdr[] = $h; curl_setopt($ch, CURLOPT_HTTPHEADER, $hdr); curl_setopt($ch, CURLOPT_POSTFIELDS, $data ); $info = curl_getinfo($ch); $result = curl_exec($ch); curl_close($ch); fclose($f); mail("myemail#email.com","$url curl info",print_r($info,true) . ", result:$result, curl log: " . file_get_contents("file://$curlLog") . "\r\n, data:$data"); } else { // ... } } This is the log i get in the email for the big JSON: https://pastebin.com/0pYHTk4h Now, can it be an issue with POST size limit, either in PHP or Apache settings? But i doubt so, since the data is just 33Kb Content-Length: 34581 Also, if it was the case, shouldn't we expect the server (Apache + PHP 7.4.x) to return an HTTP error code like HTTP 500? Instead, we have HTTP 100 and HTTP 200. Seems not related to JSON formatting too, again because we'd expect the server to return an error code. One aspect that may be relevant: the called URL (from which i am logging php://input on server side) actually is a custom Wordpress REST API endpoint (that i have created by calling register_rest_route inside a rest_api_init action handler). I am not currently aware of POST size limits or related issues with WP API, i have no idea if the issue can be with WP. Any hints are appreciated!
How to download client-site a video with cURL / PHP?
If the following PHP is runnig, I would like to download the file with `cURL on the client-site. It means, if one of my website visitor run's an action, which starts this PHP file, it should download the file on this PC. I tried with different locations, but without any success. If I run my code, it does always download it on my WebSever, which is not this, what I want. <?php //The resource that we want to download. $fileUrl = 'https://www.example.com/this-is-a-example-video'; //The path & filename to save to. $saveTo = 'test.mp4'; //Open file handler. $fp = fopen($saveTo, 'w+'); //If $fp is FALSE, something went wrong. if($fp === false){ throw new Exception('Could not open: ' . $saveTo); } //Create a cURL handle. $ch = curl_init($fileUrl); //Pass our file handle to cURL. curl_setopt($ch, CURLOPT_FILE, $fp); //Timeout if the file doesn't download after 20 seconds. curl_setopt($ch, CURLOPT_TIMEOUT, 20); //Execute the request. curl_exec($ch); //If there was an error, throw an Exception if(curl_errno($ch)){ throw new Exception(curl_error($ch)); } //Get the HTTP status code. $statusCode = curl_getinfo($ch, CURLINFO_HTTP_CODE); //Close the cURL handler. curl_close($ch); if($statusCode == 200){ echo 'Downloaded!'; } else{ echo "Status Code: " . $statusCode; } ?> How can I change the cURL downloading process to the client-site?
PHP cannot run client-side. You could use cURL to download data to the server (without saving it to a file) and then output that data to the client. Don't do this: //Open file handler. $fp = fopen($saveTo, 'w+'); or this: //Pass our file handle to cURL. curl_setopt($ch, CURLOPT_FILE, $fp); Then capture the output: //Execute the request. curl_exec($ch); Should be: //Execute the request. $output = curl_exec($ch); Then you can: echo $output; … but make sure you set the Content-Type and consider setting the Content-Length response headers. You might also want Content-Disposition. Under most circumstances, it would probably be better to simply send the browser to fetch the file directly instead of proxying it through the server. $fileUrl = 'https://www.example.com/this-is-a-example-video'; header("Location: $fileUrl");
readfile vulnerability how to avoid
I heard that the function readfile is vulnerable as a php function, so I disabled it, and I have a script to download files, which uses this function: $url = strip_tags($_GET['path']); $fileName = strip_tags($_GET['file']); header ("Content-type: octet/stream"); header ("Content-disposition: attachment; filename=".$fileName.";"); header ("Content-Length: ".filesize($url.$fileName)); readfile($url.$fileName); exit; If it is vulnerable, is there a way to avoid it or replace it with another function, I tried cURL, but the download failed, I got from this link Using cURL: <?php set_time_limit(0); $fp = fopen (dirname(__FILE__) . '/localfile.tmp', 'w+');//This is the file where we save the information $ch = curl_init(str_replace(" ","%20",$url));//Here is the file we are downloading, replace spaces with %20 curl_setopt($ch, CURLOPT_TIMEOUT, 50); curl_setopt($ch, CURLOPT_FILE, $fp); // write curl response to file curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); curl_exec($ch); // get curl response curl_close($ch); fclose($fp); ?> Thanks in advance
readfile isn't vulnerable per se. A possible attack vector is only given, when you pass un-sanitized data as parameter to it. I mean $url. Your example doesn't show where URL comes from. Let's use the worst case and assume $url comes from $_POST, e.g. $url = $_POST['url']; and combine it with a nice Path Traversal attack (https://www.owasp.org/index.php/Path_Traversal). You could limit the file access to the current dir (__DIR__). Or use realpath() and exclude/restrict what readfile can do. Referencing: How to avoid path traversal attacks Ok, i've tested you cURL example. It works! <?php $url = 'http://www.thetimes.co.uk/tto/news/rss'; set_time_limit(0); $fp = fopen (dirname(__FILE__) . '/localfile.tmp', 'w+');//This is the file where we save the information $ch = curl_init(str_replace(" ","%20",$url));//Here is the file we are downloading, replace spaces with %20 curl_setopt($ch, CURLOPT_TIMEOUT, 50); curl_setopt($ch, CURLOPT_FILE, $fp); // write curl response to file curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); curl_exec($ch); // get curl response curl_close($ch); fclose($fp); This downloads the content of the news/rss feed URL and saves it to localfile.tmp.
How to call "CURL" to call external URL
How to use "CURL" to call external URL since the url_fopen is disabled for security reasons. I want to open a pdf file. For security reason url_fopen function is disabled. So anyone can help me? function Header() { $this->SetY(20); $this->Image("images/logo-s.jpg", 120,0,80,20, "JPG", "www.example.com"); $this->Ln(4); }
There are lots of answers for connecting, downloading or saving something to a URL by CURL function, but even if the CURL be disabled in your server you can still use stream_context_create, check this question To do it with CURL try $url = 'http://www.example.com/images/logo-s.jpg'; $path = '/path/to/images/logo-s.jpg'; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); $data = curl_exec($ch); curl_close($ch); file_put_contents($path, $data); // to save it somewhere
First initialize a curl handler with: $curl_handler = curl_init("http://www.linktomypdf.com/"); If you need to read it into a file: $fp = fopen("mypdf.pdf", "w"); Now use curl_setopt() to set the option for curl_handler: curl_setopt($curl_handler, CURLOPT_FILE, $fp); First parameter is always curl_handler, second is which option you want to set , third is the value you want to set. So this call set CURLOPT_FILE to be $fp. There are list of options you can find it here: https://php.net/manual/en/function.curl-setopt.php To make it easy to read the code: $curl_handler = curl_init("http://www.linktomypdf.com/"); $fp = fopen("mypdf.pdf", "w"); curl_setopt($curl_handler, CURLOPT_FILE, $fp); curl_exec($ch); //execute curl session curl_close($ch); // close curl_handler fclose($fp); // close file
Send cURL request from URL?
Greetings, I'm looking for a way to send a curl request given a full url. All of the examples and documentation I can find look something like this: $fullFilePath = 'C:\temp\test.jpg'; $upload_url = 'http://www.example.com/uploadtarget.php'; $params = array( 'photo'=>"#$fullFilePath", 'title'=>$title ); $ch = curl_init(); curl_setopt($ch, CURLOPT_VERBOSE, 1); curl_setopt($ch, CURLOPT_URL, $upload_url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $params); $response = curl_exec($ch); curl_close($ch); The problem is that the file, "test.jpg" is actually generated dynamically by a script on the server (so it doesn't exist on the file system). How can I send the request, instead using $file = "http://www.mysite.com/generate/new_image.jpg" One solution that came to mind was loading "new_image.jpg" into memory with fopen or file_get_contents() but once I get to that point I'm not sure how to send it as POST to another site
By far the easiest solution is going to be to write the file to a temporary location, then delete it once the cURL request is complete: // assume $img contains the image file $filepath = 'C:\temp\tmp_image_' . rand() . '.jpg' file_put_contents($filepath, $img); $params = array( 'photo'=>"#$filepath", 'title'=>$title ); // do cURL request using $params... unlink($filepath); Note that I am inserting a random number to avoid race conditions. If your image is not particularly big, it would be better to use md5($img) in your filename instead of rand(), which could still result in collisions.