PHP Curl Keep Proxy connection live - php

CURLOPT_VERBOSE explains, the system takes time for connecting to PROXY before it actually makes a request to a web link.
$ch = curl_init("$url");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_ENCODING, "gzip");
curl_setopt($ch, CURLOPT_PROXY, "$ip");
curl_setopt($ch, CURLOPT_PROXYPORT, "$po");
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_close($ch);
I want to execute multiple links without closing curl to reduce latency. Is there any way to keep the PROXY live and do multiple requests to links?

When curl uses a proxy, the connection keeps open until the call of curl_close.
Two important points,
First/initial proxy connection always gonna take time (latency)
When the connection is established, don't close the connection, simply adjust parameters according to the requirements.
For an example:
//$ch = curl_init("$url");
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_ENCODING, "gzip");
curl_setopt($ch, CURLOPT_PROXY, "$ip");
curl_setopt($ch, CURLOPT_PROXYPORT, "$po");
curl_setopt($context, CURLOPT_URL, $url); // Request One
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_setopt($context, CURLOPT_URL, $url); // Request Two
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_setopt($context, CURLOPT_URL, $url); // Request Three
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_close($ch); //Closing after all requests
The first request will take some time in connection establishment to the proxy which depends on multiple factors. After the first request, all requests will be faster and will happen based on actual proxy speed(provided server script >= proxy speed).
I think we can't do the persistent connection. If you know how to do that, please share :-)

Related

Stripe API - PHP Curl request behind a proxy

I'm trying to make a request behind a proxy to stripe api using php and curl. The following works if I'm not behind a proxy:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://api.stripe.com/v1/balance");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERPWD, "$a_pass");
$out = curl_exec($ch); # returns my balance
curl_close($ch);
A request to any other domain using the proxy works:
$username = 'user';
$password = 'pass';
$proxy = 'proxy:port';
$ch = curl_init('https://api64.ipify.org?format=json');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, "$username:$password");
$result = curl_exec($ch); # returns the proxy ip
curl_close($ch);
But if I mix both, It doesn't work and a 403 error is returned by stripe api:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://api.stripe.com/v1/balance");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERPWD, "$a_pass");
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, "$username:$password");
$out = curl_exec($ch);
if (curl_errno($ch)) {
print_r($curl_error($ch)); # 403
}
curl_close($ch);
It seems the CURLOPT_USERPWD is lost if the request is made using a proxy.
Any idea how to use a CURLOPT_PROXYUSERPWD, CURLOPT_PROXY and CURLOPT_USERPWD sucessfully on the same request?
Notes:
I cannot install any additional software on this machine.

PHP libcurl socks5 not working

I try to connect to socks5 proxy using libcurl, however I receive the answer SOCKS: authentication failed. I also tried to connect to the proxy server through a Powershell script with cURL – it works, so the proxy server is available and my login and password are correct. I tried a lot of solutions... But connection via PHP-script still does not work.
PHP version – 5.4, cURL version – 7.19. What can I do?
[UPDATE] Here is my current code. I want to send message with my Telegram bot :)
$proxy = '<hostname>:<port>';
$proxy_auth = '<username>:<password>';
$url = 'https://api.telegram.org/bot<botID>/sendMessage?'.'chat_id='.$chat_id.'&parse_mode='.$parse_mode.'&text='.'test';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxy_auth);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HEADER, true);
$curl_scraped_page = curl_exec($ch);
$error = curl_error($ch);
echo $error;
echo $curl_scraped_page;
curl_close($ch);
Here is the complete answer for anyone else having the same issue.
//Curl setup
$url = "https://google.com";
$proxy = "10.23.2.2:8080";
$password = "username:secret_passowrd";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);
curl_setopt($ch, CURLOPT_HEADER, true);
//proxy's address
curl_setopt($ch, CURLOPT_PROXY, $proxy);
//proxy's password
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $password);
//in order to follow redirects.
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
//in order to use the result of the query
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
//run query
$curl_scraped_page = curl_exec($ch);
$error = curl_error($ch);
echo $error;
echo $curl_scraped_page;
curl_close($ch);
Hope that helps.

Make Request With PHP And Curl Using Proxy And Allow Cookies

My Code:
function skype_resolver($username) {
$url = "http://skyperesolver.net/api/?isapi=true&get=skype&user=" . $username;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIE, true);
curl_setopt($ch, CURLOPT_PROXY, '177.12.236.216:80');
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
return $output;
}
And The Response Is:
Please Enable Cookies
How do i do?
In my browser the response of "http://skyperesolver.net/api/?isapi=true&get=skype&user=xxuser" is:
190.255.121.56 Country: Colombia State: Bogota D.C. City: Bogotá ISP: COLOMBIA TELECOMUNICACIONES S.A. ESP VPN Dectected: No
Thanks.
You will need to specify timeout as the proxy is slow, and the request take time, and no need for any cookie.
function skype_resolver($username) {
$url = "http://skyperesolver.net/api/?isapi=true&get=skype&user=" . $username;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // If url has redirects then go to the final redirected URL.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); //to return the transfer as a string of the return value of curl_exec() instead of outputting it out directly.
curl_setopt($ch, CURLOPT_PROXY, '177.12.236.216:80');
curl_setopt($ch, CURLOPT_TIMEOUT, 30); //The maximum number of seconds to allow cURL functions to execute.
curl_setopt($ch, CURLOPT_COOKIEFILE, '/tmp/cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, '/tmp/cookies.txt');
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
}
You need to specify the proxy port:
curl_setopt($ch, CURLOPT_PROXY, "YOUR PROXY HOST");
curl_setopt($ch, CURLOPT_PROXYPORT, "YOUR PROXY PORT");

proxy using curl and php

i am using with curl libary to scrape web site.
when i am using proxy nothing now shown.
i took the ip proxy from hide my ass list.
here is my code. someone can tell me what is wrong ?
$url = 'http://www.sherut24.co.il';
$proxy = '189.218.145.44:10000';
$ch = curl_init(); // Initialise a cURL handle
// Setting proxy option for cURL
// Set any other cURL options that are required
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_COOKIESESSION, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
$results = curl_exec($ch); // Execute a cURL request
curl_close($ch); // Closing the cURL handle
echo $results;

How to use CURL via a proxy?

I am looking to set curl to use a proxy server. The url is provided by an html form, which has not been a problem. Without the proxy it works fine. I have found code on this and other sites, but they do not work. Any help in finding the correct solution would be much appreciated. I feel that the bellow are close, but that I am missing something. Thank You.
The bellow code I adapted from here http://www.webmasterworld.com/forum88/10572.htm but it returns an error message about a missing T_VARIABLE on line 12.
<?
$url = '$_POST[1]';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_PROXY, '66.96.200.39:80');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST,'GET');
curl_setopt ($ch, CURLOPT_HEADER, 1)
curl_exec ($ch);
$curl_info = curl_getinfo($ch);
curl_close($ch);
echo '<br />';
print_r($curl_info);
?>
The bellow is from curl through proxy returns no content
<?
$proxy = "66.96.200.39:80";
$proxy = explode(':', $proxy);
$url = "$_POST[1]";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy[0]);
curl_setopt($ch, CURLOPT_PROXYPORT, $proxy[1]);
curl_setopt($ch, CURLOPT_HEADER, 1);
$exec = curl_exec($ch);
echo curl_error($ch);
print_r(curl_getinfo($ch));
echo $exec;
?>
is currently live on pelican-cement.com but also does not work.
UPDATE:
Thank you for all your help, I made the above changes. Now it only returns a blank screen.
<?
$url = $_POST['1'];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_PROXY, '66.96.200.39:80');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST,'GET');
curl_setopt ($ch, CURLOPT_HEADER, 1);
curl_exec ($ch);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
?>
Here is a working version with your bugs removed.
$url = 'http://dynupdate.no-ip.com/ip.php';
$proxy = '127.0.0.1:8888';
//$proxyauth = 'user:password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
//curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
I have added CURLOPT_PROXYUSERPWD in case any of your proxies require a user name and password.
I set CURLOPT_RETURNTRANSFER to 1, so that the data will be returned to $curl_scraped_page variable.
I removed a second extra curl_exec($ch); which would stop the variable being returned.
I consolidated your proxy IP and port into one setting.
I also removed CURLOPT_HTTPPROXYTUNNEL and CURLOPT_CUSTOMREQUEST as it was the default.
If you don't want the headers returned, comment out CURLOPT_HEADER.
To disable the proxy simply set it to null.
curl_setopt($ch, CURLOPT_PROXY, null);
Any questions feel free to ask, I work with cURL every day.
I have explained use of various CURL options required for CURL PROXY.
$url = 'http://dynupdate.no-ip.com/ip.php';
$proxy = '127.0.0.1:8888';
$proxyauth = 'user:password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url); // URL for CURL call
curl_setopt($ch, CURLOPT_PROXY, $proxy); // PROXY details with port
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth); // Use if proxy have username and password
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); // If expected to call with specific PROXY type
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // If url has redirects then go to the final redirected URL.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0); // Do not outputting it out directly on screen.
curl_setopt($ch, CURLOPT_HEADER, 1); // If you want Header information of response else make 0
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
root#APPLICATIOSERVER:/var/www/html# php connectiontest.php
61e23468-949e-4103-8e08-9db09249e8s1 OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 10.172.123.1:80 root#APPLICATIOSERVER:/var/www/html#
Post declaring the proxy settings in the php script file issue has been fixed.
$proxy = '10.172.123.1:80';
curl_setopt($cSession, CURLOPT_PROXY, $proxy); // PROXY details with port
Here is a well tested function which i used for my projects with detailed self explanatory comments
There are many times when the ports other than 80 are blocked by server firewall so the code appears to be working fine on localhost but not on the server
function get_page($url){
global $proxy;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
//curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_HEADER, 0); // return headers 0 no 1 yes
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); // return page 1:yes
curl_setopt($ch, CURLOPT_TIMEOUT, 200); // http request timeout 20 seconds
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // Follow redirects, need this if the url changes
curl_setopt($ch, CURLOPT_MAXREDIRS, 2); //if http server gives redirection responce
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt"); // cookies storage / here the changes have been made
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // false for https
curl_setopt($ch, CURLOPT_ENCODING, "gzip"); // the page encoding
$data = curl_exec($ch); // execute the http request
curl_close($ch); // close the connection
return $data;
}

Categories