I want to generate google drive direct download link without opening web page.
I have found 1 site which do same thing but I did not find how he is doing.
below is the site which generates direct download link without opening google drive page.
https://links-safety.com/download.php?id=0B475ByfcR9n4a1JMVEZxQno2Tmc
0B475ByfcR9n4a1JMVEZxQno2Tmc is google file and replace with any file.
can anyone tell me how can I do that?I want to make same page like above site.
I tried this url but its not working. instead of starting download it opens page.
https://drive.google.com/uc?id=0BwSYfbOPSw89Rno4LTZpSGF6RUE
A quick look at the source of the site, it generates the following javascript.
<script>
setTimeout(function() {
window.location.href = "https://doc-0o-a0-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/nemv3kbhggb02fn2933p80ec5vsi521t/1512345600000/10410701494873540224/*/0B475ByfcR9n4a1JMVEZxQno2Tmc?e=download";
}, 300);
</script>
But where does that url come from?
With a little fiddling and watching the console on when it does ask to confirm on google, you can see it does a POST request and returns some json.
)]}'
{"disposition":"SCAN_CLEAN","downloadUrl":"https://doc-0c-9k-docs.googleusercontent.com/docs/securesc/aonp7ed1gjmns5tl5di4fl0psa1cppk8/16panrhf9etu7sqgddkaduij5anokf8t/1512345600000/17294955007197410767/17294955007197410767/0ByzJffaEk18uN2ZLeGlIRGVOaDJmWS1WU1RUN3dXUGdtUUx3?e\u003ddownload","fileName":"install.sh","scanResult":"OK","sizeBytes":4936}
So just mock that with PHP
Make a json POST request to that url, google will respond with the json, then just strip out )]}' json decode it, then use a header to redirect to the file.
<?php
$id = '0B475ByfcR9n4a1JMVEZxQno2Tmc';
$ch = curl_init('https://drive.google.com/uc?id='.$id.'&export=download');
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POSTFIELDS, []);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'Content-Type: application/json',
... see notice below
));
$result = curl_exec($ch);
$object = json_decode(str_replace(')]}\'', '', $result));
exit(header('Location: '. $object->downloadUrl));
Edit (08-04-18)
Looks like some additional headers have been added, if missing it will throw a 400 Bad Request. No biggie, it's still easy to mock it by looking at the request headers when downloading a file from your own drive. I'm unwilling to share a copy&paste solution, as the above still works you just need to add some headers and StackOverflow is not a free coding service nor am I required to maintain every answer I've ever written. Good luck.
Related
From everything I've read, it seems that this is an impossible. But here is my scenario:
I need to scrape a table's content containing for sale housing information. The page is not password protected or anything, but you first have to click an "I Agree" link on the previous page so that a cookie gets set saying you agree that the content may not be 100% accurate. You are only then shown the data. Is there any way at all to accomplish this using php/jquery/javascript? I know you cannot create an iframe because of the fact that it is cross-domain. I also do not have access to this other website.
Thanks for any answers, as I'm not really expecting anything positive. :) And many thanks if you can tell me how to do this. :D
Use server side script (PHP using cURL) to crawl the website and return the information you need. Make sure you set the appropriate HTTP header with your request that represents the "I agree" cookie.
Sample:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://www.example.com/');
curl_setopt($ch, CURLOPT_COOKIE, 'I_Agree=1');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$responseBody = curl_exec($ch);
curl_close($ch);
// Read the information you need from $responseBody and return it as response body
?>
Now you can access the information from your website by calling your server side script above. For details about how to use cURL take a look at the documentation.
CURL can store or recall cookies from a file depending on the options you set. Here is the "cookiejar" example:
http://curl.haxx.se/libcurl/php/examples/cookiejar.html
Check out the CURLOPT_COOKIEJAR and CURLOPT_COOKIEFILE options
I am searching 3 days for an answer and I cannot find one because I always find some obstacles.
I need to load a web page (the reason for this is to accept a cookie) and then at the same time read the source code of the new page without hitting it again. The reason for this is that the page is dynamic so the content will change.
I have tried to do this using iFrame(document.body.innerHTML) but the fact that these pages run on different servers I hit cross-site scripting issues.
I have also tried writing a php script using get_contents but this doesn't allow the cookie to be stored in my local.
This is driving me crazy.... Any suggestion will be helful! Need to use PHP or Javascript for this and any other suggestion will be useful as well.
When you are on the page document.body.innerHTML will give you the page source.
Edit: I didn't realize you were loading it like that. See this SO question.
It can be done using cURL in PHP.
A rough implementation:
$ch = curl_init('http://www.google.com/');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$data = curl_exec($ch);
preg_match('/^Set-Cookie: (.*?);/m', $data, $cookies);
var_dump($cookies);
var_dump($data);
$data will contain the entire response, so we need to parse out the cookie headers ourselves.
If available on your system, HttpRequest would make this easier.
I need to make a POST in JSON format to an HTTPS web page in a remote server and receive an answer in JSON format.
The data to be send it to the remote server is take it from the URL (bar)<---Done in PHP
My problem is to send this data and receive an answer.
I tried making it in PHP, and HTML using cURL(php) and submit(html).
The results: In PHP I can't send anything.
In HTML I can submit the data, get an answer but I can't catch in my code.
I see the answer using Wireshark, and as I see the POST is make it after a negotiation protocol, and as I said I receive an answer(encoded due to HTTPS, I think).
Now I need receive that answer in my code to generate an URL link so I'm considering to use Java Script.
I never do something similar before.
Any suggestion will be appreciated, thanks.
I'm using the following code with not result but a 20 seconds of delay until a blank page.
<?php
$url = 'https://www.google.com/loc/json';
$body = '{"version":"1.1.0","cell_towers":[{"cell_id":"48","location_area_code":1158,"mobile_country_code":752,"mobile_network_code.":7,"age":0,"signal_strength":-71,"timing_advance":2255}]}';
$c = curl_init();
curl_setopt($c, CURLOPT_URL, $url);
curl_setopt($c, CURLOPT_POST, true);
curl_setopt($c, CURLOPT_POSTFIELDS, $body);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
//curl_setopt($c, CURLOPT_HTTPHEADERS,'Content-Type: application/json');
$page = curl_exec($c);
echo($page);
//print_r($page);
curl_close($c);
?>
New info
I Just get new very important info
"The Gears Terms of Service prohibits direct use of the Google location server (http://www.google.com/loc/json) via HTTP requests. This service may only be accessed through the Geolocation API."
So, I was going trough the wrong way, and from now I will start to learn about Gears in order to apply the Gears API.
Cheers!
There's no real reason PHP couldn't do the PHP for you, if you set things up properly.
For instance, it may require a cookie that it had set on the client browser at some point, which your PHP/curl request doesn't have.
To do proper debugging, use HTTPFox or Firebug in Firefox, which monitor the requests from within the browser itself, and can show the actual data, not the encrypted garbage that wireshark would capture.
Of course, you could use the client browser as a sort of proxy for your server. Browser posts to the HTTPS server, gets a response, then sends that response to your server. But if that data is "important" and shouldn't be exposed, then the client-side solution is a bad one.
I know that using cURL i can see the destination URL, pointing cURL to URL having CURLOPT_FOLLOWLOCATION = true.
Example :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "www.example1.com");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$result = curl_exec($ch);
$info = curl_getinfo($ch); //Some information on the fetch
curl_close($ch);
$info will have the url of the final destination which can be www.example2.com.
I hope my above understanding is correct. Please let me know if not!.
My main question is, what all type of redirection cURL will be able to know?
Apache redirect, javascript redirects, form submition redirects, meta-refresh redirects!?
update
Thanks for your answeres #ceejayoz and #Josso. So is there a way by which I can follow all the redirect programatically through php?
cURL will not follow JS or meta tag redirects.
I know this answer is a little late, but I ran into a similar issue and needed more than just following the HTTP 301/302 status redirects. So I wrote a small library that will also follow rel=canonical and og:url meta tags.
https://github.com/mattwright/URLResolver.php
I found meta refresh tags to not provide much benefit, but they are used if no head or body html tag is returned.
As far as I know, it only follows HTTP Header redirects. (301 and 302).
curl is a multi-protocol library, which provides just a little HTTP support but not much more that will help in your case. You could manually scan for the meta refresh tag as workaround.
But a better idea was to check out PEAR HTTP_Request or the Zend_Http class, which more likely already provide something like this. Also phpQuery might be relevant, as it comes with its own http functions, but could easily ->find("meta[refresh]") if there's a need. Or look for a Mechanize-like browser class: Is there a PHP equivalent of Perl's WWW::Mechanize?
I just found this on the php site. It parses the response to find redirects and follows them. I don't think it gets every type of redirect, but it's pretty close
http://www.php.net/manual/en/ref.curl.php#93163
I'd copy it here but I don't want to plagiarize
I'm trying to perform a redirect using cURL. I can load the page fine, that's not a problem, but if I load say google.com non of the images load and the site does not work (obviously because its just printing the HTML and not actually doing a redirect).
Is there any way to perform a redirect using cURL? Sort of similar to how ...
header("Location: http://google.com");
... works?
Any help would be much appreciated.
Well, from my understading, it seems like OP want's to redirect the user to the search results URL.
Using the GoogleAPI would be a first choice and to achieve something like that, I would do this:
<?php
$query = "firefox";
$apiUrl = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=".urlencode($query);
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $apiUrl);
$content = curl_exec($ch);
$content = json_decode($content);
$luckyUrl = $content->responseData->results[0]->unescapedUrl;
header("Location: ".$luckyUrl);
?>
The code above works like 'I feel lucky'....
Use curl with -L
-L/--location
(HTTP/HTTPS) If the server reports that the requested page has
moved to a different location (indicated with a Location: header
and a 3XX response code), this option will make curl redo the
request on the new place. If used together with -i/--include or
-I/--head, headers from all requested pages will be shown. When
authentication is used, curl only sends its credentials to the
initial host. If a redirect takes curl to a different host, it
won't be able to intercept the user+password. See also --loca‐
tion-trusted on how to change this. You can limit the amount of
redirects to follow by using the --max-redirs option.
When curl follows a redirect and the request is not a plain GET
(for example POST or PUT), it will do the following request with
a GET if the HTTP response was 301, 302, or 303. If the response
code was any other 3xx code, curl will re-send the following
request using the same unmodified method.
So when using cURL
add
curl_setopt($process, CURLOPT_FOLLOWLOCATION, 1);
I'm afraid it is impossible to force the client's browser to send certain POST values and refers, you can only force it to go somewhere, hence header().
Does this answer your question?
It's should to work.pls try this: header( 'Location: http://www.google.com' ).Use the (')single cote instead of "(double)