Currently I have page say page1.php. Now in a certain event, I just want to run another link say http://example.com without actually refreshing this page. The link is a kind of script which updates my database. I tried using shell_exec('php '.$url); where $url='http://example.com' however it showed me an error that could not open file so I suppose shell_exec works only for internal files present on the server. Is there a way to do this directly or I have to go with AJAX? Thanks in advance
Try using curl to send the request to the server with php.
$url = 'http://example.com';
$ch = curl_init();
curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_exec($ch);
curl_close($ch);
Alternatively you could try file_get_contents
file_get_contents('http://example.com');
I would do this front-end and I would use JSONP: much clean and safer IMHO.
Related
How does one download a file from a web page without a direct path to the file. For example a URL with GET information instead of the path. The code below seems to be downloading the actual page html instead of the file...
Not sure what I'm doing wrong. I also would like to augment this to also perform on sites that require logins but I think I would just have to add
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password")
to the code?
$output_filename = "advanced.exe";
$host = "http://download.cnet.com/Advanced-SystemCare-Free/3001-2086_4-10407614.html?hlndr=1";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $host);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, false);
curl_setopt($ch, CURLOPT_REFERER, "http://download.cnet.com");
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HEADER, 0);
$result = curl_exec($ch);
curl_close($ch);
$fp = fopen($output_filename, 'w');
fwrite($fp, $result);
fclose($fp);
The link you have there isn't the actual link to the file, only the page that initiates the download. By the looks of it, the page uses JavaScript to trigger the download, so you would want to dig through their code to find out exactly how they do it. Then you can find the real URL to the file.
A simple way, if you are doing this only for one file, would be to download the file in your browser, and then access the URL it used from the browser's download manager. (In Firefox, for example, right click the file and choose "Copy Download Link")
I also would like to augment this to also perform on sites that require logins but I think I would just have to add ...
That would work only for HTTP based authentication. If the site uses a traditional login form, this will not work. You'd have to submit several, sequential HTTP requests via CURL, using cookies to store the session state.
I'm trying to parse a shopping site webpage using CURL in PHP.
the url is: http://computers.pricegrabber.com/printers/HP-Officejet-Pro-8600-Plus-All-One-Wireless-Inkjet-Printer/m916995235.html/zip_code=97045/sort_type=bottomline
Here's the code I use.
function getWebsiteCURL($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
echo getWebsiteCURL("http://computers.pricegrabber.com/printers/HP-Officejet-Pro-8600-Plus-All-One-Wireless-Inkjet-Printer/m916995235.html/zip_code=97045/sort_type=bottomline");
It works, but I couldn't get the full HTML code.
Anybody have any idea why?
TIA.
This is often caused by connection timeout.
Try setting the opt:
CURLOPT_TIMEOUT => 120
curl cannot interpret Javascript, you can see what curl will see if you disable javascript in a browser and navigate to the page. If you need to interpret Javascript then I would use a headless browser like phantomjs. In PHP you could use PHP PhantomJS.
With permission from the other site, I am supposed to periodically download an image hosted on another site and include it in a collection of webcam photos on our site with an external link to the contributing site.
This hasn't been any issue for any of the other sites, but with this particular one, I can't open the image to resize it and save it to our server. It's not a hotlinking issue because I can create a plain ole' <img src="http://THEIRIMAGE /> on a page on our site and it works fine.
I've tried using $img = new Imagick($sourceFilePath) directly as with all the others, as well as trying to use PHP's copy and also trying to copy the image using cURL but when doing so, the page just times out with no results at all.
Here's the image in question: http://island-alpaca.selfip.com:10202/SnapshotJPEG?Resolution=640x480&Quality=Standard
Like I've said, I'm able to do this sort of thing with several other webcams, but it isn't working with this one, and I am stuck as to why it isn't. Any help would be greatly appreciated.
Thank you.
In order to reduce bandwidth and server load some sites block certain bots from accessing their content. Your cURL request needs to more closely mimic an actual browser, which would include a referrer (usually), user agent, etc. It could also be that there is a redirect and you haven't told cURL to follow redirects.
Try setting more opts like this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch,CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_USERAGENT, 'PHP');
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
If that doesn't work, get the user agent from an actual browser and put it in there and see if that makes a difference.
Try using file_get_contents().
for example:
$url = 'http://island-alpaca.selfip.com:10202/SnapshotJPEG?Resolution=640x480&Quality=Standard';
$outputfile = "tmp/image" . date("Y-m-d_H.i.s");
$cmd = "wget -q \"$url\" -O $outputfile";
exec($cmd);
$temp_img = file_get_contents($outputfile);
$img = new Imagick($temp_img);
Can you try this and get back to me?
I try to program a webboot using PHP/CURL, but I face a problem in handling a specific page that it's loading some contents dynamically !! .. to explain more :
when I try to download the page using PHP/CURL, I do not get some contents ! then I discovered that this contents are loaded after page is loaded. and this is why CURL does not handle these missed contents.
can any one help me !
my sample code is :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $reffer);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, $redirect);
curl_setopt($ch, CURLOPT_COOKIEFILE, ABSOLUTE_PATH."Cookies/cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, ABSOLUTE_PATH."Cookies/cookies.txt");
$result = curl_exec($ch);
What URL are you trying to load? It could be that the page you're requesting has one or more AJAX requests that load content in after the fact. I don't think that cURL can accomodate runtime-loaded information via AJAX or other XHR request.
You might want to look at something like PhantomJS, which is a headless WebKit browser which will execute the page fully and return the dynamically assembled DOM.
Because the page uses javascript to load the content, you are not going to be able to do this via cURL. Check out this page for more information on the problem: http://googlewebmastercentral.blogspot.com/2007/11/spiders-view-of-web-20.html
Alright I am practicing using cURL to login to different webservices. For this pariticular try, I am doing YouTube. This was a pretty big challenge, but I finally got it...almost.
After posting the HUGE amount of post data tags to the login page, you get sent to a checkCookie kind of thing. The checkcookie page verifies that you have the right cookies and then redirects you to youtube.com (logged into your account) This is whats messing me up.
When I have this:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
I get the source of the checkcookie page. It simply says "Document Moved". This isn't what I want, I want to get the source of me being logged in stored into a variable. So I tried something else...
When I use this setup:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
I get sent to the youtube page and I am logged in! It seems to work! Except...I don't want to be redirected off my script. My goal is to get the source of youtube.com with me logged in.
In other words, the cURL is logging in just fine, the problem is I literally get redirected to YouTube. Which I don't want.
Any suggestions? It's like I need to follow the redirects...but not be redirected.
Thanks for any help!
try this:
<?php
function getURL($url) {
$curlHandle = curl_init(); // init curl
curl_setopt($curlHandle, CURLOPT_URL, $url); // set the url to fetch
curl_setopt($curlHandle, CURLOPT_HEADER, 0);
curl_setopt($curlHandle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curlHandle, CURLOPT_TIMEOUT,30);
curl_setopt($curlHandle, CURLOPT_POST, 0);
$content = curl_exec($curlHandle);
curl_close($curlHandle);
return $content;
}
?>
Looks like you are getting redirected because you echo the curl_exec, in which there is a javascript code for redirection. Since you are likely requesting that from your browser, it runs the code and redirects you to YouTube. If that's the case, obvious solution would be to turn off JS or filter what you echo to yourself