I'm trying to get the content of a stream over HTTPS, but I have to go over an HTTP proxy.
I'd like not to use cURL but rather use fopen with a context argument.
The thing is, I can't make it work over HTTPS (HTTP is working fine though).
This DOES NOT work :
$stream = stream_context_create(Array("http" => Array("method" => "GET",
"timeout" => 20,
"proxy" => "tcp://my-proxy:3128",
'request_fulluri' => True
)));
echo file_get_contents('https://my-stream', false, $context);
This DOES work (cURL) :
$url = 'https://my-stream';
$proxy = 'my-proxy:3128';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$curl_scraped_page = curl_exec($ch);
curl_close($ch);
echo $curl_scraped_page;
Does somebody know what is wrong with the first piece of code ? if it works with cURL there has to be a way to make it work with a context.
I tried to change the context options to a bunch of different values woth no luck.
Any help would be greatly appreciated !
Thanks.
You did not specifiy the exact error message, try adding ignore_errors => true. But if you are getting a 400 Bad Request from Apache, the problem you are probably hitting, is a Server Name Indication & host header mismatch. There is also a PHP bug related to this: https://bugs.php.net/bug.php?id=63519
Try the following fix until this bug is resolved:
$stream = stream_context_create(array(
'http' => array(
'timeout' => 20,
'proxy' => 'tcp://my-proxy:3128',
'request_fulluri' => true
),
'ssl' => array(
'SNI_enabled' => false // Disable SNI for https over http proxies
)
));
echo file_get_contents('https://my-stream', false, $context);
Related
I'm trying to download a file that needs to be authenticated through a client digital certificate, I already have the certificate but I do not know how to configure it in curl.
$useragent = '...';
$post = array( ... );
$certPass = '123456';
$certPath = _DIR_PATH.'cert/';
$certPfx = $certPath.'certificate.pfx';
$cert = $certPath.'certificate.pem';
$url = 'https://www.url.com/path/to/access';
$ch = curl_init( $url );
$options = array(
CURLOPT_FAILONERROR => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_AUTOREFERER => true,
CURLOPT_HEADER => true,
CURLOPT_NOBODY => true,
CURLOPT_CAINFO => $cert,
CURLOPT_CAPATH => $certPath,
CURLOPT_SSH_PRIVATE_KEYFILE => $certPfx,
CURLOPT_SSLCERT => $cert,
CURLOPT_SSLCERTPASSWD => $certPass,
CURLOPT_SSL_VERIFYHOST => 2,
CURLOPT_SSL_VERIFYPEER => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $post,
CURLOPT_USERAGENT => $useragent,
CURLOPT_COOKIE => 'ASP.NET_SessionId='.$cookie
);
curl_setopt_array( $ch, $options );
$resp = curl_exec($ch);
$ch_errno = curl_errno($ch);
$ch_erro = curl_error($ch);
curl_close($ch);
I am always getting the message: SSL certificate problem: unable to get local issuer certificate.
Can someone help me?
PHP cURL: Fixing the “SSL certificate problem: unable to get local issuer certificate” error.
If you are using PHP’s cURL functions to connect to a HTTPS URL, you might come across the following error:
SSL certificate problem: unable to get local issuer certificate. (cURL error code 60)
This is a common error that occurs whenever you attempt to use PHP’s cURL functions to connect to a HTTPS website. Essentially, your cURL client has not been configured to connect to SSL-enabled websites.
The Quick Fix.
CURLOPT_SSL_VERIFYHOST: This option tells cURL that it must verify the host name in the server cert.
CURLOPT_SSL_VERIFYPEER: This option tells cURL to verify the authenticity of the SSL cert on the server.
For example.
$url = 'https://google.com';
//Initiate cURL.
$ch = curl_init($url);
//Disable CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER by
//setting them to false.
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
//Execute the request.
curl_exec($ch);
//Check for errors.
if(curl_errno($ch)){
throw new Exception(curl_error($ch));
}
You need to configure your php.ini porperly with current (valid) certificates.
curl.cainfo = "/etc/php7.2/cacert.pem"
openssl.cafile = "/etc/php7.2/cacert.pem"
Look at https://curl.haxx.se/docs/caextract.html to download the current one. After that restart your webserver.
Do not something like
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
which will work, but disables any verification and security.
I'm writing a multithreaded php client that makes a https requests to an apache reversed proxy and measures some statistics. I'm writing a bachelor thesis about improving the performance with TLS Session Resumption. Now I need to do a proof of concept that proves/disproves this. At the moment I have this code:
$this->synchronized(function($this){
$this->before = microtime(true);
}, $this);
$url = 'https://192.168.0.171/';
# Some dummy data
$data = array('name' => 'Nicolas', 'bank account' => '123462343');
// use key 'http' even if you send the request to https://...
$options = array(
'http' => array(
'header' => "Content-type: application/x-www-form-urlencoded\r\n",
'method' => 'POST',
'content' => http_build_query($data)
),
"ssl" => array(
"verify_peer" => false,
"verify_peer_name" => false,
"ciphers" => "HIGH:!SSLv2:!SSLv3"
)
);
$context = stream_context_create($options);
$result = file_get_contents($url, false, $context);
$this->synchronized(function($this){
$this->after = microtime(true);
}, $this);
$this->counter_group->write($this->before, $this->after, $result);
This code works to do a full handshake, but I can't seem to figure out how to do an resumed handshake in php?
Any help would be greatly appreciated!
You can try PHP curl and use CURL_LOCK_DATA_SSL_SESSION
from PHP documentation http://php.net/manual/en/function.curl-share-setopt.php
CURL_LOCK_DATA_SSL_SESSION Shares SSL session IDs, reducing the time
spent on the SSL handshake when reconnecting to the same server. Note
that SSL session IDs are reused within the same handle by default
As you can read from the description above, the session id is reused by the same handle. But if you want to share between handles you can use curl_share_init for example
$sh = curl_share_init();
curl_share_setopt($sh, CURLSHOPT_SHARE, CURL_LOCK_DATA_SSL_SESSION);
curl_share_setopt($sh, CURLSHOPT_SHARE, CURL_LOCK_DATA_DNS);
then you can reuse $sh between different requests
$ch1 = curl_init('https://192.168.0.171');
curl_setopt($ch1, CURLOPT_SHARE, $sh);
curl_setopt($ch1, CURLOPT_SSLVERSION, 6); // TLSV1.2
curl_setopt($ch1, CURLOPT_SSL_CIPHER_LIST, 'TLSv1');
curl_setopt($ch1, CURLOPT_POST, 1);
curl_setopt($ch1, CURLOPT_POSTFIELDS,
http_build_query( array('name' => 'Nicolas', 'bank account' => '123462343') ));
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec($ch1);
and then reuse ( resumed handshake )
$ch2 = curl_init('https://192.168.0.171');
curl_setopt($ch2, CURLOPT_SHARE, $sh);
curl_setopt($ch2, CURLOPT_SSLVERSION, 6); // TLSV1.2
curl_setopt($ch2, CURLOPT_SSL_CIPHER_LIST, 'TLSv1');
curl_setopt($ch2, CURLOPT_RETURNTRANSFER, true);
// ( ... )
curl_exec($ch2);
and close connections
curl_close($ch1);
curl_close($ch2);
But you also need to play with CURLOPT_SSLVERSION and CURLOPT_SSL_CIPHER_LIST . Also, I think you should switch to a different language as PHP has its own quirks, and if you prove or disproves thesis, it's better to use something closer to bare metal so you are sure the extra layer (PHP) doesn't break your benchmarks. I did measure the performance of both requests and it's a bit counter-intuitive but the second one is almost twice slower.
I'm trying to get xml data from the https. When I execute or parse this request using curl it's taking so much time.
$url = 'https://www.iformbuilder.com/exzact/dataXML.php?';
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_REFERER, $url);
$result = curl_exec($ch);
curl_close($ch);
$File = "data.xml";
$Handle = fopen($File, 'w');
fwrite($Handle, $result);
fclose($Handle);
If I write http instead of https it works. I don't know what's the reason behind this.
Try using the following options:
CURLOPT_USERAGENT => 'Mozilla/5.0',
CURLOPT_COOKIEFILE => 'fb.tmp',
CURLOPT_COOKIEJAR => 'fb.tmp',
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_SSL_VERIFYHOST => 0,
CURLOPT_SSL_VERIFYPEER => 0,
CURLOPT_TIMEOUT => 60
You are only using one of the SSl ops.
I solve this by writing http instead of https in URL than its working fine .whats reasone behind this that i dont know
PHP's curl doesn't seem to include the Host header when doing a request:
<?php
$handle = curl_init('http://example.com/');
curl_setopt_array($handle, array(
CURLOPT_USERAGENT => 'test ua',
CURLOPT_REFERER => 'http://example.org/',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_PROXY => '127.0.0.1:8080',
CURLOPT_PROXYTYPE => CURLPROXY_HTTP,
//Doesn't work if you specify it manually or leave it out:
CURLOPT_HTTPHEADER => array(
'Host: example.com'
)
));
print_r(curl_getinfo($handle));
echo curl_exec($handle);
?>
Host header is defined by the argument of curl_init()
The Host header is already automatically by PHP when you specify the request URL.
The following example oughta be sufficient confirmation:
<?php
$handle = curl_init('http://pgl.yoyo.org/http/browser-headers.php');
curl_setopt($handle, CURLOPT_RETURNTRANSFER, true);
echo curl_exec($handle);
?>
The output HTML lists the request headers, among which we see Host: pgl.yoyo.org, as expected.
This was a totally unrelated bug, which happened by having a linebreak inside the user-agent, eg.:
curl_setopt_array($handle, array(
CURLOPT_USERAGENT => "user-agent\n"
));
I want to use a third party's web service. To use the web service I need to connect with HTTPS. My problem is that for the development process I have a test api with an invalid certificate. I would like to set SoapClient no to verify the server's certificate. Here is the way I tried:
$opts = array(
'ssl' => array(
'verify_peer' => false
),
'https' => array(
'curl_verify_ssl_peer' => false,
'curl_verify_ssl_host' => false
)
);
$streamContext = stream_context_create($opts);
$client = new SoapClient("https://urlToSoapWs",
array(
'login' => 'user',
'password' => 'pwd',
'authentication' => SOAP_AUTHENTICATION_BASIC,
'local_cert' => file_get_contents('C:/somelocation/1.pem'),
'passphrase' => 'passphrase',
'stream_context' => $streamContext
));
I also tried with CURL and worked! But I want to use SoapClient. You can find the code with CURL below:
// create a new cURL resource
$ch = curl_init("https://urlToSoapWs");
// setting the request type to POST:
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: text/xml"));
// setting the authorization method to BASIC:
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
// supplying your credentials:
curl_setopt($ch, CURLOPT_USERPWD, "user:pwd");
$body = "<SOAP-ENV:Envelope>somexmlhere</SOAP-ENV:Envelope>";
// filling the request body with your SOAP message:
curl_setopt($ch, CURLOPT_POSTFIELDS, $body);
// configuring cURL not to verify the server certificate:
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSLCERT, "pathToTheCertificatePemFile");
curl_setopt($ch, CURLOPT_SSLCERTPASSWD, "pwd");
//curl_setopt($ch, CURLOPT_SSLCERTTYPE, "PEM");
curl_setopt($ch, CURLOPT_SSLKEY, "pathTotheKeyFile");
curl_setopt($ch, CURLOPT_SSLKEYPASSWD, "pwd");
// telling cURL to return the HTTP response body as operation result
// value when calling curl_exec:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// calling cURL and saving the SOAP response message in a variable which
// contains a string like "<SOAP-ENV:Envelope ...>...</SOAP-ENV:Envelope>":
$result = curl_exec($ch);
// closing cURL:
curl_close($ch);
If you have found the bug in the code that I provided using the SoapClient please post it.
Thanks.
Maybe not the invalid Certificate is a Problem, more the SSLv2/3 Handshake; can you try manually specifing a Cipher like this:
$stream_opts = array(
// 'ssl'=>array('ciphers'=>"3DES" // also working
// further ciphers on http://www.openssl.org/docs/apps/ciphers.html
'ssl'=>array('ciphers'=>"SHA1"
)
);
$myStreamContext = stream_context_create($stream_opts);
$soapOptions['stream_context'] = $stream_opts;
$soapClient = new SoapAuthClient("https://...", $soapOptions);
Good luck!
It looks like you have hit this authentication plus SSL bug in SoapClient. You can either recompile php with the patch included in that link, or wait until they integrate it in the official build.