I've written a simple spider to test various things with Fiddler. The script makes a few requests with Zend_Http_Client->request() using the same instance of the Zend_Http_Client class ($client in the example below).
When using Fiddler and Zend_Http_Client, only the first Zend_Http_Client->request() works; subsequent requests fail with "Unable to read response, or response is empty". Here's my Zend_Http_Client configuration with Fiddler:
$config = array(
'adapter' => 'Zend_Http_Client_Adapter_Proxy',
'proxy_host' => '127.0.0.1',
'proxy_port' => 8888,
'timeout' => 60,
'useragent' => 'Local Site Spider Test',
'keepalive' => true,
'sslusecontext' => true
);
$client = new Zend_Http_Client('http://www.site.com/', $config);
Here's a simplified example of what would fail, using $client from above:
$response = $client->request();
echo $response->getHeadersAsString();
$client->setUri('http://www.site.com/file.html');
$response = $client->request();
echo $response->getHeadersAsString();
The spider itself works 100% as intended when not using a proxy, so the code itself is fine. Fiddler is also working, capturing all requests from all processes (tested with WinInet as well as various browsers).
Related
I'm using PHP + Laravel to send some requests.
I want to capture every thing from sending request to the end. So I did this:
$verbose_log_stream = fopen('./curl.log', 'a+');
$client = new GuzzleHttp\Client([
'curl' => [
CURLOPT_VERBOSE => true,
CURLOPT_STDERR => $verbose_log_stream,
],
]);
I realized CURL also have some more options for better tracing called --trace-ascii and --trace-time
Anyone knows how can I set these two trace options to the Guzzle CURL options?
This is the response image
When I try to connect to website to scrap with proxy it gives me this error.
the object is an instance of Symfony\Component\HttpClient\HttpClient
What can be issue. below is my code to connect to the proxy
$client = new Client(HttpClient::create(['headers' => ['User-Agent' => $agent], 'proxy' => "http://5.183.253.91:8085"])
$client->request('GET', $url])
I have tried adding proxy with like this too but it didnt work out.
$client->request('GET', $url, ['proxy' => "http://5.183.253.91:8085"])
I've been trying to use the European Union's website to validate TIN numbers (Europa TIN validation website - there is a WSDL available at the bottom of the page)
The problem I'm having is that when I try to make a new SoapClient the function fails immediately when building the client. At first I was having a "Failed to load external entity" and I assumed it was because the WSDL has a secure connection. After searching around I found some answers that said that the problem could have to do with the certificate being outdated and the most recent versions of PHP throw errors in that case, so I disabled certificate validation with:
// Stream context due to certificate problems
$streamContext = stream_context_create(array(
'ssl' => array(
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true
)
));
But now I'm getting another error: "failed to open stream: HTTP Request failed! HTTP/1.1 502 Bad Gateway". Any thoughts about how to solve this issue? If I remove "https://" from the link I get the same result as before with the "failed to load external entity" message.
Now here's the real brain picker. If I try to use chrome's extension "Boomerang" to test SOAP calls on the WSDL it works absolutely perfectly, so I have no idea what's wrong here... Anyone can easily try this by attempting to make a soap call in a PHP file.
Here's the full code:
public static function validateTIN($tin) {
// Stream context due to certificate problems
$streamContext = stream_context_create(array(
'ssl' => array(
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true
)
));
// Create our soap client
$client = new SoapClient('https://ec.europa.eu/taxation_customs/tin/checkTinService.wsdl', array(
'exceptions' => 0,
'trace' => 1,
'connection_timeout' => 1800,
'stream_context' => $streamContext
));
dd($client->__getFunctions());
return true;
}
After spending hours researching and trying to figure out the problem I found a 2 years old post that somehow served my purpose.
Unfortunately I can't mark this as duplicate so I'll just link it here: SOAP error parsing wsl couldn't load from but works on wamp
I couldn't find this question because Google didn't tag it as using the Europa services.
Either way, my problem was that I needed to specify my user agent explicitly because Europa web services are too outdated and can't resolve IPv6 requests, only IPv4. Like so:
$opts = array(
'http'=>array(
'user_agent' => 'PHPSoapClient'
)
);
$context = stream_context_create($opts);
$client = new SoapClient('http://ec.europa.eu/taxation_customs/vies/checkVatService.wsdl',
array('stream_context' => $context,
'cache_wsdl' => WSDL_CACHE_NONE));
$result = $client->checkVat(array(
'countryCode' => 'DK',
'vatNumber' => '47458714'
));
The example uses the CheckVAT SOAP function but it works just as well for the CheckTIN function.
I have a PHP script trying to connect to a WSDL.
I need to allow self signed AND give basic auth details.
Using SOAP UI, when I connect to the WSDL I am prompted for username / password.
I got this working.
I also found out that each request also requires basic auth (so on the request screen, I have to select Auth, then basic, enter same credentials as I used on the prompt).
How to I do this auth in PHP
As I said, I can connect, not a problem, I seem to kill the service or timeout if I try to make a request
<?php
$context = stream_context_create(array(
'ssl' => array(
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true
)
));
$data = array(
'columnA' => 'dataA',
'columnB' => 'dataB',
'columnC' => 'dataC');
$url = 'https://111.111.111.111:1234/dir/file';
$login = 'username';
$pwd = 'password';
$client = new soapClient(null, array(
'location' => $url,
'uri' => '',
'login' => $login,
'password' => $pwd,
'stream_context' => $context
));
echo "\n\r---connected---\n\r";
$result = $client ->requestName($data);
print_r($result);
?>
My output is
---connected---
Then it seems to hang.
I have tried wrapping it round a try catch and I had the same result.
Any suggestions??
From the Manual soapclient support the http basic auth.
For HTTP authentication, the login and password options can be used to
supply credentials. For making an HTTP connection through a proxy
server, the options proxy_host, proxy_port, proxy_login and
proxy_password are also available. For HTTPS client certificate
authentication use local_cert and passphrase options. An
authentication may be supplied in the authentication option. The
authentication method may be either SOAP_AUTHENTICATION_BASIC
(default) or SOAP_AUTHENTICATION_DIGEST.
$wsdl = "http://example/services/Service?wsdl";
$option = array(
"trace"=>1,
"login"=>"admin",
"password"=>"admin",
);
$client = new SoapClient($wsdl,$option);
But when I initiate the soapclient, it will throw this error
Exception: Unauthorized
I also have tried to put the auth in the url, like
$wsdl = "http://admin:admin#example/services/Service?wsdl";
But it also doesn't works.
Finally I solved it by add authentication to the option. The manual says the authentication default value is the basic auth, but only when I explicitly set it, it can work.
$option = array(
"trace"=>1,
"login"=>"admin",
"password"=>"admin",
"authentication"=>SOAP_AUTHENTICATION_BASIC
);
Try url encoding your username and password inside the url that you are using:
$url = 'http://'.urlencode('yourLogin').':'.urlencode('yourPassword').'#111.111.111.111:1234/dir/file';
Also I don't see you make use of the wsdl in your code example. You can always download a copy of the wsdl locally and then reference that local copy. You can download the wsdl anyway you want (with php, curl, manually).
my first time doing a SOAP Request fails and fails again.
I have to send some Data via SOAP but i dont get a stable connection.
I use the SOAP Extension of PHP. My Code looks like the following.
$certificate = file_get_contents(DATA_PATH.'/modules/va/misc/CKTC.cer');
$options = array(
'uri' => 'https://data2.kroschke.net/service/MeinAutoUeberfuehrungen',
'allow_self_signed' => true,
'verify_peer' => true,
'local_cert' => $certificate,
'trace' => 1,
'exceptions' => true,
);
$wsdl = 'https://data2.kroschke.net/service/MeinAutoUeberfuehrungen?WSDL';
$SOAPClient = new SoapClient($wsdl, $options);
fb($SOAPClient->__getFunctions());
Now my Problem is, that the SOAP Request always runs into a timeout.
I checked my php settings and SOAP and OpenSSL are activated.
I also tried it with a .pem instead of a .cer File. Same Problem.
This is what i get all the time:
Warning: SoapClient::SoapClient(https://data2.kroschke.net/service/MeinAutoUeberfuehrungen?WSDL): failed to open stream: Connection timed out in
Anyone who can help?
PS: Dont mind the fb() Function. Just a function to print everything into FireBug.
just try to open https://data2.kroschke.net or https://data2.kroschke.net/service/MeinAutoUeberfuehrungen?WSDL in you browser: you'll get a timeout, too.
the best solution would be to get in contact with the provider of that soap-server (kroschke.com/kroschke.de ?) and ask them why their server is down or seems to hang.
EDIT:
this doesn't seem to be the problem, so your only option is to set the timeout to a higher value. for this, just add connection_timeout (value in seconds) to your options:
$options = array(
'uri' => 'https://data2.kroschke.net/service/MeinAutoUeberfuehrungen',
'allow_self_signed' => true,
'verify_peer' => true,
'local_cert' => $certificate,
'trace' => 1,
'exceptions' => true,
'connection_timeout'=> 30
);
if it still times out, set the value even higher - if it still times out then, try to test the soap-communication with a program like soapUI before implementing it with PHP - if you get problems with soapUI, too, contact the provider of that soap-server and ask why their methods take such a long time.
for more information, take a look at the soapclient and it's options.