I need to download a large file and also need to get the headers in order to get the redirect location (status code 302).
I use this:
curl_setopt($ch, CURLOPT_FILE, $file);
curl_setopt($ch, CURLOPT_HEADER, true);
Now the problem is the following:
It is writing the header data as well to the output file.
Is there a way to only write the content?
Edit:
Just to make it clear. All i want is to get the headers, but i want that only the body get's written to the file. Also please keep in mind that i can't store the body data into a variable since its a video data and therefore to big (memory_limit).
With the help of #Bishop i was able to get the working solution:
$url = 'example.com';
$file = fopen('body.txt', 'w');
$file_header = fopen('headers.txt', 'w')
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_WRITEHEADER, $file_header);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch, CURLOPT_FILE, $file);
$data = curl_exec($ch);
Now i have the header in a seperate file and can get it from there by using:
file_get_contents('headers.txt');
Related
I am trying to upload a file (input type and named "upload") through the echonest api but am struggling to get the curl upload working.
http://developer.echonest.com/docs/v4/track.html#upload
Can anybody point me into the right direction to get the data-binary and header: application/octet-stream POST method working.
This A HTTP POST request with Content-Type "application/octet-stream", with the local file as the body of the request, and the parameters in the URL.
the curl php:
$localfile = $_FILES['upload']['name'];
$fp = fopen($localfile, 'r');
$post = array('$localfile'=>'#$localfile','api_key'=>'xx','filetype'=>'mp3');
$ch = curl_init();
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true); // --data-binary
curl_setopt($ch, CURLOPT_URL, 'http://developer.echonest.com/api/v4/track/upload');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, 'Content-type: application/octet-stream');
$response = curl_exec($ch);
$result = json_decode($response,true);
$id = $result['response']['track']['id'];
curl_close($ch);
fclose($fp);
echo $id;
The problem might have something to do with your use of variables inside a single-quoted string. Also, improper use of the # symbol. Try this...
array("$localfile"=>"$localfile",'api_key'=>'xx','filetype'=>'mp3');
Though that doesn't look right either. You probably want this..
array("localfile"=>"$localfile",'api_key'=>'xx','filetype'=>'mp3');
I have images that are hosted on a CDN, and I want to upload that image to some other system via their API. Normally I would just use curl and put an # character in front of the file to upload. Since this is a URL, that won't work since it expects the file to be local. I don't want to copy it local first since the files could be videos and that could be time consuming.
It seems there should be a way to do this with PHP curl. I'm open to using sockets instead, but I'm not sure how to simulate how curl does the submission.
<?php
$save_file_url = file_get_contents($your_files_CDN_file_URL);
$fp = fopen('test','w');
fwrite($fp,$save_file_url);
$save_file = realpath('test') ;
$url = "http://URL_TO_SEND_FILE/get_file.php";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible;)");
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
$post = array(
"file_is_this"=> "#".$save_file,
"app"=>'test'
);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$response = curl_exec($ch);
?>
And you will recieve file at get_file.php.
I would like to download a file with Curl.
The problem is that the download link is not direct, for example:
http://localhost/download.php?id=13456
When I try to download the file with curl, it download the file download.php!
Here is my curl code:
###
function DownloadTorrent($a) {
$save_to = $this->torrentfolder; // Set torrent folder for download
$filename = str_replace('.torrent', '.stf', basename($a));
$fp = fopen ($this->torrentfolder.strtolower($filename), 'w+');//This is the file where we save the information
$ch = curl_init($a);//Here is the file we are downloading
curl_setopt($ch, CURLOPT_ENCODING, "gzip"); // Important
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_URL, $fp);
curl_setopt($ch, CURLOPT_HEADER,0); // None header
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1); // Binary trasfer 1
curl_exec($ch);
curl_close($ch);
fclose($fp);
}
Is there a way to download the file without knowing the path?
You may try CURLOPT_FOLLOWLOCATION
TRUE to follow any "Location: " header that the server sends as part
of the HTTP header (note this is recursive, PHP will follow as many
"Location: " headers that it is sent, unless CURLOPT_MAXREDIRS is
set).
So it will result into:
function DownloadTorrent($a) {
$save_to = $this->torrentfolder; // Set torrent folder for download
$filename = str_replace('.torrent', '.stf', basename($a));
$fp = fopen ($this->torrentfolder.strtolower($filename), 'w+');//This is the file where we save the information
$ch = curl_init($a);//Here is the file we are downloading
curl_setopt($ch, CURLOPT_ENCODING, "gzip"); // Important
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER,0); // None header
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1); // Binary transfer 1
curl_exec($ch);
curl_close($ch);
fclose($fp);
}
Set the FOLLOWLOCATION option to true, e.g.:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
Options are documented here: http://www.php.net/manual/en/function.curl-setopt.php
Oooh !
CURLOPT_FOLLOWLOCATION work perfect...
The problem is that I use CURLOPT_URL for fopen(), I simply change CURLOPT_URL whit CURLOPT_FILE
and it works very well!
thank you for your help =)
I am trying to parse a page which contains some links. These links, if followed, will redirect to some files to download.
For example, Download which redirects to <a href="http://example.com/1.pdf".
I don't want to download the file, I just want to get the file link (int this case http://example.com/1.pdf).
I am trying this:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE); // Return in string
curl_setopt($ch, CURLOPT_URL, $url);
curl_exec($ch);
var_dump(curl_getinfo($ch));
But, it gives me the file contents.
Does anyone have any idea how to this?
==EDIT==
Thank you guys. I solved it like this:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_exec($ch);
$info = curl_getinfo($ch);
Now, $info contains the header and I can the link from it.
The reason the output is being sent to the screen is because you're telling cURL to do so. If you want to store the response in a variable the following line:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, FALSE);
should read:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
Then, actually retrieve the returned output from curl_exec like so:
$output = curl_exec($ch);
Once you have the returned HTML content from the remote page in the $output variable you can use DOMdocs or regex (but preferably DOM) to parse out any information you want.
UPDATE
I can't tell because the question is vaguely worded: is there actually a Location header redirect happening? If so, you'll want to do as #heiko suggests to prevent cURL from following the redirect and retrieve the headers. Then you can easily parse the contents of the location header:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
curl_setopt($ch, CURLINFO_HEADER, TRUE); // add header output
# make sure to not follow Location: Header
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, FALSE);
# add Response Header to Output, so that you can find the Location-Header in there!
curl_setopt($ch, CURLINFO_HEADER_OUT, TRUE);
Use RETURN TRANSFER as 1, also use htmlentities() if you want to display HTML source on your page , else just echo the variable ( to display the page [redirects to google] ).
<?php
$url = "http://www.google.co.in";
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // Return in string
curl_setopt($ch, CURLOPT_URL, $url);
$varx = curl_exec($ch);
echo htmlentities($varx);
?>
With the $varx variable , use Regular Expressions to match which data you want.
I get files by their urls by this code
file_get_contents($_POST['url'];
Then I do something with them.
But I don't want to operate with big files, how do I limit size of received file?
It should throw an error if file is bigger than 500kb.
See my answer to this question. You need to have the cURL extension, with which you can make a HEAD HTTP request to the remote server. The response will let you know how big the file is, and you can then decide accordingly.
You are interested specifically in this line:
$size = curl_getinfo($ch, CURLINFO_CONTENT_LENGTH_DOWNLOAD);
Agree with #Jon
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url); //specify the url
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$head = curl_exec($ch);
$size = curl_getinfo($ch,CURLINFO_CONTENT_LENGTH_DOWNLOAD);
if(<limit the $size>){
file_get_contents($url);
}