Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
Well, here's my cURL script inside bash that works without any issues!
#!/bin/bash
fileid="1yvklOFopnep8twiqAQecmMUoAbQVzU0r"
filename="MyFile.mp4"
curl -c ./cookie -s -L "https://drive.google.com/uc?export=download&id=${fileid}" > /dev/null
curl -Lb ./cookie "https://drive.google.com/uc?export=download&confirm=`awk '/download/ {print $NF}' ./cookie`&id=${fileid}" -o ${filename}
However I'm trying to rewrite this into a simple PHP script, although it appears not to be working correctly; here's the code:
<?php
define('FILENAME', 'MyFile.mp4');
define('FILE_ID', '1yvklOFopnep8twiqAQecmMUoAbQVzU0r');
$GlobalFileHandle = null;
function get_confirm($id)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://drive.google.com/uc?export=download&id=".$id);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
$result = curl_exec($ch);
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close($ch);
preg_match_all("/confirm=([0-9A-Za-z]+)&/", $result, $output_array);
return $output_array[1][0];
}
function get_file($id, $confirm)
{
global $GlobalFileHandle;
$GlobalFileHandle = fopen(FILENAME, 'w+');
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://drive.google.com/uc?export=download&confirm='.$confirm.'&id='.$id);
curl_setopt($ch, CURLOPT_FILE, $GlobalFileHandle);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_WRITEFUNCTION, 'curlWriteFile');
curl_exec($ch);
if (curl_errno($ch)) {
echo 'Error:' . curl_error($ch);
}
curl_close($ch);
fclose($GlobalFileHandle);
}
function curlWriteFile($cp, $data)
{
global $GlobalFileHandle;
return fwrite($GlobalFileHandle, $data);
}
$confirm_code = get_confirm(FILE_ID);
echo "We got our confirm code! ".$confirm_code;
get_file(FILE_ID, $confirm_code);
However it appears the file is not being downloaded & the MyFile.mp4 remains empty?
You seem to mix several options in an invalid way here.
Since you set CURLOPT_RETURNTRANSFER the data is returned by curl_exec().
So
$data = curl_exec($ch);
fwrite($GlobalFileHandle, $data);
should do the trick. If you want to use the callbacks, do not set CURLOPT_RETURNTRANSFER at all.
Another option is, to set CURLOPT_FILE to write the data directly to a file handle (don't set CURLOPT_RETURNTRANSFER either then):
curl_setopt(CURL_FILE, $GlobalFileHandle);
Furthermore, you need to set the CURLOPT_COOKIEFILE to your cookiejar to have the cookies read correctly. The CURLOPT_COOKIEJAR option only sets the file where to store cookies to. You need both, so add:
curl_setopt(CURL_COOKIEFILE, 'cookies.txt');
For more details, refer to the curl_setopt PHP manpage
Seems like a curl_exec is missing in function get_file.
Related
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 12 months ago.
Improve this question
I need the same result as when calling curl from terminal/consol, even if the answer is 302. Unfortunately I didn't get any data during my attempts.
$ curl bretten.work
Moved Permanently.
Your server seems to block requests if no user agent is set. Try this:
<?php
$ch = curl_init('http://bretten.work/');
curl_setopt($ch, CURLOPT_USERAGENT, 'curl/7.68.0');
curl_exec($ch);
To diagnose further problems try to compare curl -vvv https://example.com/ with this:
<?php
$ch = curl_init('http://example.com/');
#curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
$cv = curl_version();
$useragent = 'curl';
if (isset($cv['version'])) {
$useragent .= '/' . $cv['version'];
}
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_VERBOSE, true);
$streamVerboseHandle = fopen('php://temp', 'w+');
curl_setopt($ch, CURLOPT_STDERR, $streamVerboseHandle);
$result = curl_exec($ch);
if ($result === false) {
printf("cUrl error (#%d): %s<br>\n",
curl_errno($ch),
htmlspecialchars(curl_error($ch)))
;
}
rewind($streamVerboseHandle);
$verboseLog = stream_get_contents($streamVerboseHandle);
echo "cUrl verbose information:\n",
"<pre>", htmlspecialchars($verboseLog), "</pre>\n";
To make php curl follow redirects, the FOLLOWLOCATION setting is needed, as such:
<?php
$curl = curl_init('http://bretten.work/');
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_exec($curl);
See https://evertpot.com/curl-redirect-requestbody/ for more options.
I'm making a web and one of its functions need to get content from itself. Using curl with the following function:
function get_page_code($url)
{
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$data = curl_exec($ch);
if (curl_errno($ch)) {
echo curl_error($ch);
echo "\n<br />";
$contents = '';
} else {
curl_close($ch);
}
if (!is_string($data) || !strlen($data)) {
$data = '';
}
return $data;
}
The issue appears when I noticed I need to have a cookie so the content I want to get actually appears, so my question is how can I send cookies with this curl function? Of course I've those cookies on my computer but once sent the request to get the html code of the introduced url those cookies are not there.
Cookies I want to send with curl:
session: ab1b298aslc
gender: m
I'd grateful with any help :-)
EDIT:
Tried adding this lines to the function and nothing happened:
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Cookie: gender=m"));
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Cookie: session=ab1b298aslc"));
I'm trying to do the bare minimum, just to get it working.
Here is my Google Script:
function doPost(e) {
return ContentService.createTextOutput(JSON.stringify(e.parameter));
}
Here is my PHP code:
$url = 'https://script.google.com/a/somedomain.com/macros/s/### script id ###/exec';
$data['name'] = "Joe";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-type: multipart/form-data"));
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data));
// curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$result = curl_exec($ch);
$error = curl_error($ch);
Executing this, $result is true.
If I uncomment the CURLOPT_RETURNTRANSFER line, $result =
<HTML>
<HEAD>
<TITLE>Bad Request</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Bad Request</H1>
<H2>Error 400</H2>
</BODY>
</HTML>
$error is always empty.
I would use doGet() but I need to send some rather large POSTs that will exceed what GET can handle.
How can I post to a Google script and return data?
------ UPDATE ------
I've just learned my lead developer tried this some time ago and concluded doPost() errors when returning so apparently it's not just me. My take is that Google is simply not reliable enough to use. I would love for someone to prove me wrong.
------ UPDATE 2 - THE FIX ---------
Apparently this was the problem:
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data));
needs to be:
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
No idea why http_build_query() caused it to error.
Try reading the documentation for CURLOPT_POSTFIELDS and you'll see that is says To post a file, prepend a filename with # and use the full path. That looks what you are trying to do. Note that in php 5.5, the CURLFile class was introduced to let you post files.
If you are using php 5.5 or later, you might try this:
$url = 'https://script.google.com/a/somedomain.com/macros/s/### script id ###/exec';
// create a CURLFile object:
$cfile = new CURLFile('file.pdf','application/pdf'); // you can also optionally use a third parameter
// your POST data...you may need to add other data here like api keys and stuff
$data = array("fileName" => $cfile);
$ch = curl_init();
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-type: multipart/form-data"));
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
// FROM THE DOCS:
// If value is an array, the Content-Type header will be set to multipart/form-data (so you might skip the line above)
// As of PHP 5.2.0, value must be an array if files are passed to this option with the # prefix
// As of PHP 5.5.0, the # prefix is deprecated and files can be sent using CURLFile
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
// set this to TRUE if you want curl_exec to retrieve the result
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$result = curl_exec($ch);
if ($result === FALSE) {
echo "The result is FALSE. There was a problem\n";
$error = curl_error($ch);
var_dump($error);
die();
} else {
echo "success!\n";
var_dump($result);
}
// this can give you more information about your request
$info = curl_getinfo($ch);
if ($info === FALSE) {
echo "curlinfo is FALSE! Something weird happened";
}
var_dump($info); // examine this output for clues
EDIT: If you are not getting any error, and $result comes back with something like "Bad Request" then you will need to inspect the result more closely to find out what the problem is. A well-behaved API should have informative information to help you fix the problem. If the API doesn't tell you what you did wrong, you can examine the curlinfo you get from these commands:
$info = curl_getinfo($ch);
var_dump($info); // examine this output for clues
if $result and $info don't tell you what you've done wrong, try reading the API documentation more closely. You might find a clue in there somewhere.
If you can't figure out what the problem is using these tactics, there's not much else you can do with your code. You'll need more information from the maintainers of the API.
You need to look at your HTTP Request header to see what is actually being posted.
When trouble shooting I add these options:
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FAILONERROR,true);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
CURLINFO_HEADER_OUT will add "request_header" to curl_getinfo()
You also want to look at these curl_getinfo() elements.
request_size
size_upload
upload_content_length
request_header
I am trying to open homepages of websites and extract title and description from it's html markup using curl with php, I am successful in doing this to an extent, but many websites are there I am unable to open. My code is here:
function curl_download($Url){
if (!function_exists('curl_init')){
die('Sorry cURL is not installed!');
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $Url);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
// $url is any url
$source=curl_download($url);
$d=new DOMDocument();
$d->loadHTML($source);
$title=$d->getElementsByTagName("title")->item(0)->textContent)
$domx = new DOMXPath($d);
$desc=$domx->query("//meta[#name='description']")->item(0);
$description=$desc->getAttribute('content');
?>
This code is working fine for most websites but there are many whome it doesn't even able to open. What can be the reason?
When I tried getting headers of those websites using get_headers function, its working fine, but these are not being opened using curl. Two of these websites are blogger.com and live.com.
Replace:
$output = curl_exec($ch);
with
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
$output = curl_exec($ch);
if (!$output) {
echo curl_error($ch);
}
and see why Curl is failing.
It's a good idea to always check the result of function calls to see if they succeeded or not, and to report when they fail. While a function may work 99.999% of the time, you need to report the times it fails, and why, so the underlying cause can be identified and fixed, if possible.
I'm having dificulties to query a webform using CURL with a PHP script. I suspect, that I'm sending something that the webserver does not like. In order to see what CURL realy sends I'd like to see the whole message that goes to the webserver.
How can I set-up CURL to give me the full output?
I did
curl_setopt($ch, CURLOPT_VERBOSE, TRUE);
but that onyl gives me a part of the header. The message content is not shown.
Thanks for all the answers! After all, they tell that It's not possible. I went down the road and got familiar with Wireshark. Not an easy task but definitely worth the effort.
Have you tried CURLINFO_HEADER_OUT?
Quoting the PHP manual for curl_getinfo:
CURLINFO_HEADER_OUT - The request string sent. For this to work, add
the CURLINFO_HEADER_OUT option to the handle by calling curl_setopt()
If you are wanting the content can't you just log it? I am doing something similar for my API calls
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, self::$apiURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POST, count($dataArray));
curl_setopt($ch, CURLOPT_POSTFIELDS, $dataString);
$logger->info("Sending " . $dataString);
self::$results = curl_exec($ch);
curl_close($ch);
$decoded = json_decode(self::$results);
$logger->debug("Received " . serialize($decoded));
Or try
curl_setopt($ch, CURLOPT_STDERR, $fp);
I would recommend using curl_getinfo.
<?php
curl_exec($ch);
$info = curl_getinfo($ch);
if ( !empty($info) && is_array($info) {
print_r( $info );
} else {
throw new Exception('Curl Info is empty or not an array');
};
?>