I am trying to use curl through a proxy, when I dont use a proxy it works fine but if I use a proxy it returns null, no error messages or anything.
I have already tried other opened questions but their answers are inconclusive.
Here is my code:
<?php
$url = 'http://dynupdate.no-ip.com/ip.php';
$proxy = '122.117.225.161:8080';
//$proxyauth = 'user:pass';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
//curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_HEADER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
var_dump($curl_scraped_page);
?>
The proxy there is a free working proxy (I manually checked), I have also tried it with a private proxy (I used the user:pass authentication method) but still returns null.
Any information will be helpful.
Thanks!
i am using with curl libary to scrape web site.
when i am using proxy nothing now shown.
i took the ip proxy from hide my ass list.
here is my code. someone can tell me what is wrong ?
$url = 'http://www.sherut24.co.il';
$proxy = '189.218.145.44:10000';
$ch = curl_init(); // Initialise a cURL handle
// Setting proxy option for cURL
// Set any other cURL options that are required
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
$results = curl_exec($ch); // Execute a cURL request
curl_close($ch); // Closing the cURL handle
echo $results;
Modern browsers support HTTPS proxies that can be connected to via a PAC file (see https://www.igvita.com/2012/06/25/spdy-and-secure-proxy-support-in-google-chrome/ if you're not familiar).
I'm trying to replicate the same thing and connect to such a proxy via php CURL, but I'm simply getting a blank response, no headers or content.
My code is as follows:
$url = "http://checkip.dyndns.com";
$proxy = "proxy.domain.com:443";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL , 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'test:test');
curl_setopt($ch, CURLOPT_PROXYTYPE, "CURLPROXY_HTTP");
$response = curl_exec($ch);
curl_close($ch);
echo $response;
Any ideas?
If anyone is interested, the solution to this is to use spdycat instead of curl: https://github.com/tatsuhiro-t/spdylay
There's no support for connecting to a proxy with HTTPS with curl yet. There is a work-in-progress branch in git though: https://github.com/bagder/curl/tree/HTTPS-proxy
We will appreciate help to get that in shape to merge.
I am using curl to extract some data of a website.First time it works perfectly , but now it not parse any data from the website.I think they are blocked my server ip.So I can use public proxy ip.But still it not working.My code is ,
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $urls);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_PROXYPORT, '3127');
curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP');
curl_setopt($ch, CURLOPT_PROXY, '204.93.54.15');
$original_file = curl_exec($ch);
curl_close($ch);
I can't get user name and password of public proxies.Is it important to set public proxy's username and password?It don't show any errors.Any one please help me.
Here is a good answer to use proxy in curl
How to use CURL via a proxy?
Please check this. If you have any queries feel free to ask.
Thanks
Close your connection when you are done with it. That should do the trick.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $urls);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXYPORT, '3127');
curl_setopt($ch, CURLOPT_PROXYTYPE, 'HTTP');
curl_setopt($ch, CURLOPT_PROXY, '204.93.54.15');
$original_file = curl_exec($ch);
print_r($original_file);
curl_close($ch);
I am looking to set curl to use a proxy server. The url is provided by an html form, which has not been a problem. Without the proxy it works fine. I have found code on this and other sites, but they do not work. Any help in finding the correct solution would be much appreciated. I feel that the bellow are close, but that I am missing something. Thank You.
The bellow code I adapted from here http://www.webmasterworld.com/forum88/10572.htm but it returns an error message about a missing T_VARIABLE on line 12.
<?
$url = '$_POST[1]';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_PROXY, '66.96.200.39:80');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST,'GET');
curl_setopt ($ch, CURLOPT_HEADER, 1)
curl_exec ($ch);
$curl_info = curl_getinfo($ch);
curl_close($ch);
echo '<br />';
print_r($curl_info);
?>
The bellow is from curl through proxy returns no content
<?
$proxy = "66.96.200.39:80";
$proxy = explode(':', $proxy);
$url = "$_POST[1]";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy[0]);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy[1]);
curl_setopt($ch, CURLOPT_HEADER, 1);
$exec = curl_exec($ch);
echo curl_error($ch);
print_r(curl_getinfo($ch));
echo $exec;
?>
is currently live on pelican-cement.com but also does not work.
UPDATE:
Thank you for all your help, I made the above changes. Now it only returns a blank screen.
<?
$url = $_POST['1'];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_PROXY, '66.96.200.39:80');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST,'GET');
curl_setopt ($ch, CURLOPT_HEADER, 1);
curl_exec ($ch);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
?>
Here is a working version with your bugs removed.
$url = 'http://dynupdate.no-ip.com/ip.php';
$proxy = '127.0.0.1:8888';
//$proxyauth = 'user:password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
//curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
I have added CURLOPT_PROXYUSERPWD in case any of your proxies require a user name and password.
I set CURLOPT_RETURNTRANSFER to 1, so that the data will be returned to $curl_scraped_page variable.
I removed a second extra curl_exec($ch); which would stop the variable being returned.
I consolidated your proxy IP and port into one setting.
I also removed CURLOPT_HTTPPROXYTUNNEL and CURLOPT_CUSTOMREQUEST as it was the default.
If you don't want the headers returned, comment out CURLOPT_HEADER.
To disable the proxy simply set it to null.
curl_setopt($ch, CURLOPT_PROXY, null);
Any questions feel free to ask, I work with cURL every day.
I have explained use of various CURL options required for CURL PROXY.
$url = 'http://dynupdate.no-ip.com/ip.php';
$proxy = '127.0.0.1:8888';
$proxyauth = 'user:password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url); // URL for CURL call
curl_setopt($ch, CURLOPT_PROXY, $proxy); // PROXY details with port
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth); // Use if proxy have username and password
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); // If expected to call with specific PROXY type
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // If url has redirects then go to the final redirected URL.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); // Do not outputting it out directly on screen.
curl_setopt($ch, CURLOPT_HEADER, 1); // If you want Header information of response else make 0
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
root#APPLICATIOSERVER:/var/www/html# php connectiontest.php
61e23468-949e-4103-8e08-9db09249e8s1 OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 10.172.123.1:80 root#APPLICATIOSERVER:/var/www/html#
Post declaring the proxy settings in the php script file issue has been fixed.
$proxy = '10.172.123.1:80';
curl_setopt($cSession, CURLOPT_PROXY, $proxy); // PROXY details with port
Here is a well tested function which i used for my projects with detailed self explanatory comments
There are many times when the ports other than 80 are blocked by server firewall so the code appears to be working fine on localhost but not on the server
function get_page($url){
global $proxy;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
//curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_HEADER, 0); // return headers 0 no 1 yes
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // return page 1:yes
curl_setopt($ch, CURLOPT_TIMEOUT, 200); // http request timeout 20 seconds
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // Follow redirects, need this if the url changes
curl_setopt($ch, CURLOPT_MAXREDIRS, 2); //if http server gives redirection responce
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); // cookies storage / here the changes have been made
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // false for https
curl_setopt($ch, CURLOPT_ENCODING, "gzip"); // the page encoding
$data = curl_exec($ch); // execute the http request
curl_close($ch); // close the connection
return $data;
}