I'm trying to use PHPCrawl (http://sourceforge.net/projects/phpcrawl/) to trawl a website delivered over HTTPS.
I can see that there is support for SSL in the PHPCrawlerHTTPRequest class (openSocket method):
// If ssl -> perform Server name indication
if ($this->url_parts["protocol"] == "https://")
{
$context = stream_context_create(array('ssl' => array('SNI_server_name' => $this->url_parts["host"])));
$this->socket = #stream_socket_client($protocol_prefix.$ip_address.":".$this->url_parts["port"], $error_code, $error_str,
$this->socketConnectTimeout, STREAM_CLIENT_CONNECT, $context);
}
The problem lies in the call to stream_socket_client - although it returns a zero error_code, and no error_str, this->socket is still false.
The documentation for the method states the following:
If the value returned in errno is 0 and the function returned FALSE, it is an indication that the error occurred before the connect() call.
(See http://php.net/manual/en/function.stream-socket-client.php)
So I've tried to use an example provided in the comments section to modify the stream context using 'stream_context_set_option' to set verify_host and verify_peer to false - neither of which seems to have any effect.
I'm not very proficient in PHP or the intricacies of web - does anyone know either:
What condition (specifically) can cause this call to fail?
OR
A workaround for the issue?
I should note - I am using Facebook (HTTPS) as the test server.
I've found the issue -
PHP versions 5.6.x turn peer verification on by default, and apparently the necesarry cert isn't found sometimes (see this bug report)
The workaround is to drop back to a PHP version prior to 5.6
Old topic, but I had the same problem using the PHPCrawler. What worked for me, is what an user wrote on sourceforge (Source: https://sourceforge.net/p/phpcrawl/bugs/86/#5993).
What you have to do, is to rewrite the stream_context_create call on line 547 in PHPCrawlerHTTPReqeust.class.php into the following:
$context = stream_context_create(array(
'ssl' => array(
'SNI_server_name' => $this->url_parts["host"],
'verify_peer' => false,
'verify_peer_name' => false,
)
));
Hope this helps someone in the future.
Related
Other similar questions have not helped me resolve this.
I have to retrieve a WSDL file using a client certificate + private key combination from my webserver calling another external SOAP API.
$wsdl = 'https://www.example.com?wsdl';
$endpoint = 'https://www.example.com';
$sslContext = stream_context_create($contextOptions);
$options = [
'local_cert' => '/var/www/combo.pem',
'passphrase' => 'Pass1',
'cache_wsdl' => WSDL_CACHE_MEMORY,
'trace' => 1,
'stream_context' => stream_context_create([
'ssl' => [
'ciphers' => 'RC4-SHA',
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true
]
])
];
try{
$soapClient = new SoapClient($wsdl, $options);
}
catch(Exception $e)
{
var_dump($e);
}
The error I'm getting is:
SOAP-ERROR: Parsing WSDL: Couldn't load from '..domain..' : failed to load external entity "..domain..?wsdl"
I tried numerous settings and none of them made any difference to this response. I tried no settings, empty array.
What has worked:
Establishing a raw connection via CURL, so the certificate file is fine
Retrieving WSDL information from SoapUI, so the WSDL destination is correct and the certificate file was okay
phpinfo() returns SoapClient is enabled, OpenSSL is enabled. What else could I try or check?
SOAP error could come from invalid character encoding or maybe some HTTP header missing like 'User-Agent' when you query the remote server.
Try to adding User-Agent to options like the sample below.
$options = array(
'http' => array(
'user_agent' => 'PHP_Embedded_Soap_Client'
)
);
PS: I would not recommend to strict ciphers to: RC4-SHA
I wanted to comment this but based on others' suggestions of using file_get_contents and your reply mentioning that it returned false, I'm now kind of sure it's more of a connection problem. Please check followings:
It's obvious that your actual $wsdl value is different than what is posted but please make sure that .com?wsdl is not happening in your code. it should be .com/?wsdl.
Check your DNS settings. A lookup through dnslookup can help identify problem.
Make sure date & time setting of server is correct. It can lead to SSL errors.
If none of above helped. You might consider downloading WSDL content with other tools (such as cURL) and on different machines to identify the cause of problem.
I am connecting to dynamics 365. It used to work perfectly, i curl to get the token then i use it as an authorization header along with php soapclient and it works, i connect i create a client and i can call my methods.
All of a sudden it decided not to work, and where it used to connect as SOAP 1.1 now it enforced SOAP 1.2
After changing from SOAP 1.1 to SOAP 1.2 ( because I got the error of binding mismatch where it said expecting application/soap+xml and text/xml was found ) So I changed versions and that error disappeared and got replaced with ERROR Fetching HTTP Headers.
That error got stuck for the longest time, people suggested to increase timeout but i put it as high as 500 800 5000 all the same.
Then all of a sudden, it started giving me SOAP ERROR Parsing schema element already defined. I did not change my code, i played for awhile with the headers but to no avail, I even removed the authorization header just to see what is going on and that did nothing i kept getting the same error.
SOAP-ERROR: Parsing Schema: element 'http://schemas.datacontract.org/2004/07/Microsoft.Dynamics.Ax.Xpp:XppObjectBase' already defined [string:Exception:private]
everytime I try to connect I get different kind of parsing schema error even though i am not changing anything in my code:
SOAP-ERROR: Parsing Schema: element 'http://schemas.microsoft.com/2003/10/Serialization/:anyType' already defined [string:Exception:private]
and another
SOAP-ERROR: Parsing Schema: element 'http://schemas.datacontract.org/2004/07/Microsoft.Dynamics.AX.KernelInterop:ProxyBase' already defined [string:Exception:private]
and then sometimes it does get through for a second but with fetching http header error again..
so i can not create a client instance anymore now..
where before i was able to create a client instance but i get an error when I call the method of "Error Fetching HTTP Headers"
something is definitely not stable because my errors are not one.
now some stated the wsdl could be faulty, but this is microsoft and the person i am in contact keeps saying he can not doing anything about it.
Help is this a PHP problem or a dynamics problem or wsdl custom made problem .
And how to solve this.
Thank you.
UPDATE
I'm sorry I mentioned earlier it is Dynamics AX , it turns out it is Dynamics 365 D365. I will keep dynamics ax tag in case it helps someone who needs the solutions provided.
UPDATE
Following is the connection code I am using:
function getAuthenticationHeader()
{
//Each variable has the values for our server
//resource
$appResource = urlencode($appADResource);
//clientID
$appClientID = urlencode($appADClientId);
//appSecret
$appSecret = urlencode($appADSecret);
//username
$appUserID = urlencode($appUserID);
// Password
$appUserPassword = urlencode($password);
// Construct the body for the STS request
$authenticationRequestBody = 'resource='.$appResource.'&client_id='.$appClientID.'&client_secret='.$appSecret.'&grant_type=password&username='.$appUserID.'&password='.$appUserPassword.'&scope=openid';
//Using curl to post the information to STS and get back the authentication response
$ch = curl_init();
// set url
$stsUrl = 'https://login.microsoftonline.com/'.$appTenantId.'/oauth2/token';
curl_setopt($ch, CURLOPT_URL, $stsUrl);
// Get the response back as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// Set the parameters for the request
curl_setopt($ch, CURLOPT_POSTFIELDS, $authenticationRequestBody);
// By default, HTTPS does not work with curl.
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// read the output from the post request
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
// decode the response from sts using json decoder
$tokenOutput = json_decode($output);
return $tokenOutput->{'token_type'}.' '.$tokenOutput->{'access_token'};
}
try
{
//WSDL Link
$url = "https://urlToOurServer/services/webservice?wsdl";
$authorizationToken = getAuthenticationHeader();
$context = stream_context_create(array(
'ssl' => array(
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true
),
'https' => array(
'curl_verify_ssl_peer' => false,
'curl_verify_ssl_host' => false
),
'http' => array(
'header' =>'Authorization: '.$authorizationToken
)
));
//Create array of Soap Options
$arrOpt = array(
"soap_version" => SOAP_1_2,
"cache_wsdl" => WSDL_CACHE_NONE,
"exceptions" => true,
'trace' => true,
'encoding' => 'UTF-8',
'stream_context' => $context
);
}catch(Exception $e)
{
print_r($e);
}
I also found this in my wsdl
<sp:IssuedToken sp:IncludeToken="http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702/IncludeToken/AlwaysToRecipient">
<sp:RequestSecurityTokenTemplate>
<trust:TokenType xmlns:trust="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0
</trust:TokenType>
<trust:KeyType xmlns:trust="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
http://docs.oasis-open.org/ws-sx/ws-trust/200512/Bearer
</trust:KeyType>
</sp:RequestSecurityTokenTemplate>
<wsp:Policy>
<sp:RequireInternalReference/>
How can I connect to SAML for Token ?
If everything is pretty much the same, but it's not working, the first thing to do is rule out the most basic AX issues. These may not solve your issue but will be a good first step.
now some stated the wsdl could be faulty, but this is microsoft and the person i am in contact keeps saying he can not doing anything about it.
Whomever that person is, you need to confirm they've done the following:
Confirm the environment and specifically the CIL is fully compiled. Do a full AXBuild and a full CIL to be sure during non-business hours and ensure the output is good. It's basically saying "recompile everything".
Refresh the WCF configuration in the client configuration you are using to connect to AX. This client configuration may be a *.axc file or it may just be the active one. Also refresh the business connector WCF. This is separate and may be what you are using to connect to AX. This is what most people are talking about.
Here's a little article that talks about creating a configuration, but I'll discuss below.
An AX client configuration ultimately is a bunch of text. It's either stored in an .axc file or stored in the registry in a few locations. The Business Connector client config may be the one that is getting missed in your scenario.
If you follow the link above and create a new .axc configuration file and ensure you've clicked "Refresh Configuration" before exporting, when you open the file up in Notepad, you'll see wcfconfig and a bunch of XML following it. That XML is what you're trying to get updated. Creating a new AXC here is just an exercise to help you understand what it is. You can delete the file after you're done looking.
Now, you've basically created a specific configuration file, but that doesn't mean anything is using it. If you call AX32.exe it will default to the one that is loaded in that config screen. Using a file is a way to very specifically choose one. Your code is probably using a specific AXC somewhere that needs either replaced or refreshed OR it's using one that's saved in this window:
It is very likely it is using one of the two that are saved in that configuration window. When you refresh in that window, it ultimately saves the WCF XML in the windows registry on the machine that is hosting the client and/or the AOS in subfolders in HKLM\SOFTWARE\Microsoft\Dynamics\6.0\Configuration. The key(s) is wcfconfig paired with wcfconfigversionid, which just stores a GUID to see if it's up-to-date.
When I say two, I mean most people don't even bother to look at the Business Connector AXC. It's what is highlighted in yellow in my image, and you need to specifically choose and refresh it. This could be important for you. In my image, I do not have it chosen. You need to drop the menu down and choose it.
On a dev machine, you can just clear both of those keys and refresh and you should see whatever configuration you're working on update.
This is a long post, but it's important to rule this part out first. If you have someone who's reasonably experienced administering AX they should know how to ensure these are refreshed.
Since you're saying this is not Dynamics AX, but one of the Dynamics 365 versions. The AX version used to be called Dynamics 365 for Finance and Operations Enterprise Edition but they've changed the licensing/naming again, so I don't even know what it's technically called. Most people call it Dynamics 365 for Operations or some variant.
Either way, you should test the service following the below method. We would need to see more information about the service details and call, so following the below is most likely best.
https://learn.microsoft.com/en-us/dynamics365/fin-ops-core/dev-itpro/data-entities/third-party-service-test
I am attempting to use #abraham's TwitterOAuth 0.5.3 library for PHP, but when I make a request to request a token for the 3-legged authorization, I receive an HTTP 500 as a response.
Here is how I have the code set up in PHP:
<?php
/* Start session and load library. */
session_start();
require_once('config.php');
require_once('twitteroauth/autoload.php');
use Abraham\TwitterOAuth\TwitterOAuth;
/* Build TwitterOAuth object with client credentials. */
$connection = new TwitterOAuth(CONSUMER_KEY, CONSUMER_SECRET);
/* Get temporary credentials. */
// Error occurs on the following line, unable to dump $request_token
$request_token = $connection->oauth('oauth/request_token', array('oauth_callback' => OAUTH_CALLBACK));
//print_r($request_token); // <-- Never reached!!
I know that this problem is not within the Twitter API, as I have verified that I can access my Twitter account via the Dev console.
In addition, I have verified to some degree that the TwiterOAuth library is working, by following the Authorization flow example provided with the library. The example can also access my Twitter account.
I just can't figure out what is going on as I am unable to properly authorize my PHP application to have access to my Twitter account.
What am I doing wrong?
It turns out that a response was never obtained. As a result, attempting to process a response, when there was none resulted in errors on the server side.
One of the PHP functions that Twitter OAuth relies upon is curl. I had tested to see if curl_init existed:
print function_exists('curl_init') ? 'curl_init is enabled' : 'curl_init is disabled';
and I erroneously assumed that curl_exec was also enabled. (Why would you leave curl_init enabled, but only disable curl_exec?)
That assumption was incorrect as my web hosting provider has disabled curl_exec "due to security concerns" and I was unaware of this. In addition, my call to use the Twitter API has worked in the past, so this was new behavior.
It took me a while to come back to testing curl_exec. I verified that I was receiving a valid TwitterOauth object and eventually wound my way into the TwitterOauth class and into the request function.
I was receiving no curl error, but was the response from curl_exec was null (not TRUE or FALSE as expected). I thought that this was unusual and at first thought that curl was missing a configuration option.
However, it was not.
So, if you run into problems with this library (which has worked great for me in the past), it may be that your hosting provider disabled curl_exec.
You can test this scenario via the following PHP code:
print function_exists('curl_exec') ? 'curl_exec is enabled' : 'curl_exec is disabled';
My problem was fixed in another way. After checking (as per jhenderson2099 answer) that my hosting had curl_exec enabled (which it did). I found out that my problem was caused by two lines in src/TwitterOauth.php (TwitterOauth class):
$bundlePath = CaBundle::getSystemCaRootBundlePath(); <-- Comment this line
$options = [
// CURLOPT_VERBOSE => true,
CURLOPT_CONNECTTIMEOUT => $this->connectionTimeout,
CURLOPT_HEADER => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYHOST => 2,
CURLOPT_SSL_VERIFYPEER => true,
CURLOPT_TIMEOUT => $this->timeout,
CURLOPT_USERAGENT => $this->userAgent,
$this->curlCaOpt($bundlePath) => $bundlePath,<-- Comment this line
];
so that your code will look like this:
//$bundlePath = CaBundle::getSystemCaRootBundlePath();
$options = [
// CURLOPT_VERBOSE => true,
CURLOPT_CONNECTTIMEOUT => $this->connectionTimeout,
CURLOPT_HEADER => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYHOST => 2,
CURLOPT_SSL_VERIFYPEER => true,
CURLOPT_TIMEOUT => $this->timeout,
CURLOPT_USERAGENT => $this->userAgent,
//$this->curlCaOpt($bundlePath) => $bundlePath,
];
I am trying to perform a Bing Search by using the Windows Azure Marketplace API, I have downloaded their guide and sample code. The code prepares a HTTPS request with basic authentication, however I am constantly getting the following error:
Warning: file_get_contents(https://api.datamarket.azure.com/Data.ashx/Bing/SearchWeb/Web?Query=%27washburn%27&Adult=%27Off%27&$top=50&$format=Atom): failed to open stream: Connection refused
The php code (from Microsoft's document):
$context = stream_context_create(array(
'http' => array(
'proxy' => 'tcp://127.0.0.1:8888',
'request_fulluri' => true,
'header' => "Authorization: Basic " . base64_encode($accountKey.":".$accountKey)
)
));
Does anyone know what is causing the error please? I have correctly set the $accountKey and I tested it in a browser. What puzzles me a little is 127.0.0.1:8888 and also base64_encode($accountKey.":".$accountKey) , how come you need the $accountKey both before and after the : while when using a browser you are supposed to leave the username blank and just input the account key into the password field?
I have solved it and here is what I have found for future people that would be doing the same thing:
I commented out the line where it says:
'proxy' => 'tcp://127.0.0.1:8888',
'request_fulluri' => true,
and also set base64_encode("ignored:".$accountKey) instead
Base on what I read on MSDN, the username part is said to be ignored, so it shouldn't matter what value it is. I was thinking perhaps the length or the special characters in the key screwed things up so I replaces it with ignored (or anything really).
That did the trick and I can parse the returned JSON data. Good luck!
I've tried using Matt Harris' Twitter OAUTH library (https://github.com/themattharris/tmhOAuth) replacing default data with my keys and tokens, but for some reason I can't get a valid response code.
The url I'm testing with ends with a port (8888), but I'm not sure if that is to do with it. I'm tailing the PHP log and there are no errors.
$tweet_text = 'Hello world!';
print "Posting...\n";
$result = post_tweet($tweet_text);
print "Response code: " . $result . "\n";
function post_tweet($tweet_text) {
require_once('tmhOAuth.php');
$connection = new tmhOAuth(array(
'consumer_key' => '(hidden)',
'consumer_secret' => '(hidden)',
'user_token' => '(hidden)',
'user_secret' => '(hidden)',
));
$connection->request('POST',
$connection->url('1/statuses/update'),
array('status' => $tweet_text));
return $connection->response['code'];
}
Does anybody have any ideas?
Many thanks.
I had this problem and it's due to this recent change in December 2011:
"defaulted CURLOPT_SSL_VERIFYPEER to true"
If you dump $connection->response as Darren advises, you may see an error such as this one which I had:
string(165) "error setting certificate verify locations:"
Either make sure that the certificate file cacert.pem is in place, or disable SSL_VERIFYPEER.
I'm willing to bet that this will fix your issue (although it may not be the most secure solution):
$connection = new tmhOAuth(array(
'consumer_key' => '(hidden)',
'consumer_secret' => '(hidden)',
'user_token' => '(hidden)',
'user_secret' => '(hidden)',
'curl_ssl_verifypeer' => false
));
Update February 2015
Looking back on this answer today, I realise that advising people to set curl_ssl_verifypeer to false is not a very good answer (at this point you can no longer be sure you're talking to Twitter, so in fact it's a terrible answer). Instead, take the other advice I gave and ensure the appropriate Root CA Certificate file (cacert.pem) is in place.
If you compare with https://github.com/themattharris/tmhOAuth/blob/master/examples/tweet.php your code might need to change to look like:
$code=$connection->request('POST', ...);
return $code;
However, looking at the source code, reveals two things. First your code should be just as good as that, because $this->response['code'] is set to the value that gets returned. Second that that function (actually curlit()) can also return void. When it does that response['code'] is undefined. (This was looking like a promising twitter library until I saw that design mistake.)
Probing even further, it would only return void when $this->config['prevent_request'] exists and is true. You're not doing that, and we've gone full circle to not being able to explain the behaviour you see.
So, my next troubleshooting step would be to put error_reporting(E_ALL|E_NOTICE) at the top, and then check the error logs for more clues. Also do a print_r($connection->response) after your call to request() to see what else you have in there.
Looks like you need to make sure you have a current cert.
According to TMH's git repo:
Version 0.60 hardened the security of the library and defaulted curl_ssl_verifypeer to true. As some hosting providers do not provide the most current certificate root file it is now included in TMH's repository. If the version is out of date OR you prefer to download the certificate roots yourself, you can get them from: http://curl.haxx.se/ca/cacert.pem
Before upgrading the version of tmhOAuth that you use, be sure to verify the SSL handling works on your server by running the examples/verify_ssl.php script.