I'm new here to write, but read lots of answers in the past few years, so first of all: thanks for the lot of help you gave me so far! So, this is what makes me crazy today, and couldn't find the answer anywhere:
I have a web service (an Openfire plugin) and a LAMP-based backoffice (on a separate box), which manages the Openfire remotely. After migrating the Openfire to a new (3rd) box, the backoffice code stopped working. Here is a snippet:
<?php
header("Content-Type: text/plain");
$url = "http://gbbackup.dyndns.org:9090/plugins/goldsteinAdmin/goldsteinadmin"; echo $url."\n";
$ch = curl_init($url); echo "handle: ".$ch."\n";
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_STDERR, fopen('php://stdout', 'w'));
$result = curl_exec($ch); echo "Result: ".$result."\n";
if ($result === false) echo 'cURL error '.curl_errno($ch).': '.curl_error($ch)."\n";
print_r(curl_getinfo($ch));
curl_close($ch);
?>
Curl's verbose output is:
* About to connect() to gbbackup.dyndns.org port 9090 (#0)
* Trying 81.183.210.206... * Connection refused
* couldn't connect to host
* Closing connection #0
Now the strange thing is if I try via command line, it works perfectly:
user#login01:~/public_html/gb$ curl -v "http://gbbackup.dyndns.org:9090/plugins/goldsteinAdmin/goldsteinadmin"
* About to connect() to gbbackup.dyndns.org port 9090 (#0)
* Trying 81.183.210.206... connected
* Connected to gbbackup.dyndns.org (81.183.210.206) port 9090 (#0)
> GET /plugins/goldsteinAdmin/goldsteinadmin HTTP/1.1
> User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
> Host: gbbackup.dyndns.org:9090
> Accept: */*
>
< HTTP/1.1 200 OK
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Set-Cookie: JSESSIONID=12e1urcewodgr;Path=/
< Content-Type: application/json;charset=ISO-8859-1
< Transfer-Encoding: chunked
<
{"type":"error","msg":"RequestNotAuthorised"}
* Connection #0 to host gbbackup.dyndns.org left intact
* Closing connection #0
What could be the difference between the PHP and the command line curl? I have removed all the fancy extras just to test the connection itself, but some difference must be remained.
I made some more tests, here are the results:
PHP curl to other websites (e.g. google.com): works
inserted the link into Chrome on my own machine: works
tested command line curl via php system(): does not work
I also wanted to tcpdump the differences, but on the other box, where I have root privileges, both PHP and command line version works
It seems just these two boxes do not like eachother :-)
Thanks in advance for your help!
With PHP you must provide the port separately (not in the URL) like:
$url = "http://gbbackup.dyndns.org/plugins/goldsteinAdmin/goldsteinadmin";
...
curl_setopt($ch, CURLOPT_PORT, 9090);
...
Related
(I edited the title, see EDIT below: GET path in the response contains spaces, requested path does not.)
I am working for a company that uses a "template system" to enable their markets (different countries) to host small web apps and making them "feel" like part of the main market website by pulling header, navigation, footer of the main website via curl:
So a php script uses curl to pull a special empty CMS page that contains only the relevant elements and placeholders for the web app to replace. This way, the web apps always have the current legal footer, header and navigation links to the main sections on the CMS and feel like being part of the cms albeit being hosted on their own somewhere else.
In this case, depending on an i18n parameter in the url, the script is curling a different template for each of ~50 markets.
Now the curious case: This is working for all templates but one, which is returning error 400 when being curled via https.
But when curled via http, 301 is returned, then another request to the previously faulty https URL is made, only this time it returns 200.
Does not work:
* Trying [server_ip]...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x562a5065a130)
* Connected to www.[company_name].pt ([server_ip]) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.[company_name].pt
* start date: Aug 23 19:02:44 2022 GMT
* expire date: Nov 21 19:02:43 2022 GMT
* subjectAltName: host "www.[company_name].pt" matched cert's "*.[company_name].pt"
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x562a5065a130)
> GET [path] HTTP/2
Host: www.[company_name].pt
User-Agent:Mozilla/5.0 (compatible; Integrator https://[app_name].[company_name].com)
Accept:text/html
Accept-Encoding:gzip,deflate
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive:115
Connection:keep-alive
Cache-Control:max-age=0
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* The requested URL returned error: 400
* stopped the pause stream!
* Connection #0 to host www.[company_name].pt left intact
Does work:
* Trying [server_ip]...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x562a5066c0f0)
* Connected to www.[company_name].pt ([server_ip]) port 80 (#0)
> GET [path] HTTP/1.1
Host: www.[company_name].pt
User-Agent:Mozilla/5.0 (compatible; Integrator https://[app_name].[company_name].com)
Accept:text/html
Accept-Encoding:gzip,deflate
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive:115
Connection:keep-alive
Cache-Control:max-age=0
< HTTP/1.1 301 Moved Permanently
< Content-Type: text/html
< Content-Length: 185
< Connection: keep-alive
< Location: https://www.[company_name].pt/[path]
<
* Ignoring the response-body
* Connection #0 to host www.[company_name].pt left intact
* Issue another request to this URL: 'https://www.[company_name].pt/[path]'
* Trying [server_ip]...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x562a5066c0f0)
* Connected to www.[company_name].pt ([server_ip]) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=*.[company_name].pt
* start date: Aug 23 19:02:44 2022 GMT
* expire date: Nov 21 19:02:43 2022 GMT
* subjectAltName: host "www.[company_name].pt" matched cert's "*.[company_name].pt"
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x562a5066c0f0)
> GET [path]
Host: www.[company_name].pt
User-Agent:Mozilla/5.0 (compatible; Integrator https://[app_name].[company_name].com)
Accept:text/html
Accept-Encoding:gzip,deflate
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive:115
Connection:keep-alive
Cache-Control:max-age=0
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< content-type: text/html; charset=utf-8
< content-length: 15305
< date: Thu, 22 Sep 2022 09:25:35 GMT
< server: Apache
< x-robots-tag: noindex
< strict-transport-security: max-age=31536000
< x-content-type-options: nosniff
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< referrer-policy: strict-origin-when-cross-origin
< cache-control: public, max-age=3600
< expires: Thu, 22 Sep 2022 10:25:35 GMT
< pragma: public
< accept-ranges: bytes
< content-encoding: gzip
<
* Connection #1 to host www.[company_name].pt left intact
Calling the template in the browser works, with http and https.
curl from CLI works also for both http and https and returns 200, the error only occurs with PHP.
The script (a bit blown up for error search):
function curl($url){
$headers[] = "User-Agent:Mozilla/5.0 (compatible; Integrator https://[app_name].[company_name].com)";
$headers[] = "Accept:text/html";
$headers[] = "Accept-Encoding:gzip,deflate";
$headers[] = "Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$headers[] = "Keep-Alive:115";
$headers[] = "Connection:keep-alive";
$headers[] = "Cache-Control:max-age=0";
/**/debug_to_console("init curl");
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_ENCODING, "gzip");
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($curl, CURLOPT_SSLVERSION, 6);
curl_setopt($curl, CURLOPT_FAILONERROR, 1);
curl_setopt($curl, CURLOPT_VERBOSE, 1);
curl_setopt($curl, CURLOPT_STDERR, $verbose = fopen('php://temp', 'rw+'));
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$data = curl_exec($curl);
if(curl_errno($curl)){
debug_to_console("curl url: " . $url);
debug_to_console("curl error: " . curl_error($curl));
//echo 'Request Error:' . curl_error($curl);
}
echo "Verbose information:\n<pre>", !rewind($verbose), htmlspecialchars(stream_get_contents($verbose)), "</pre>\n";
curl_close($curl);
return $data;
}
As mentioned, on other templates from other markets (on the same server) this is working fine. And I was told, that the instances are duplicated, so they all should have the same settings. Again, they are all on the same server. Unfortunately, I can't get them to check further until I'm pointing them into the right direction what to look at.
Tests with ssllabs.com showed no differences between the instances.
Being a Front End Dev, I'm really not getting any further at this point, after trying multiple suggestions for similar errors. I'm suspecting a wrong setting on that instance, but what instructions/hints can I give the server team?
Sorry I had to remove IPs and company/app names.
Thank you.
Edit: Not sure if this is relevant, for some reason, the line > GET /[path] HTTP/2 is different in the faulty curl attempt:
Every successful curl has one space between the path and HTTP/2, the faulty one has three spaces between the path and HTTP/2.
When using http instead of https, the line also has three spaces > GET /[path] HTTP/1.1), gets the 301 response and after redirect, the new attempt has only one space > GET /[path] HTTP/2.
The URL is not the problem, it is absolutely identical to other instances where there is no problem. Does the server add a space character itself on the initial curl attempt? What could cause added spaces? This has to be server side, the other instances don't mangle the initial GET path.
I am trying to make a curl request to Paypal sandbox to get access token but every time when I make a request to this url
https://api.sandbox.paypal.com/v1
It sends back a 400 in response.I have tried sending curl request to any test url too and I get a 200 response.Moreover, if I send a request via terminal the reponse is 200 issue is only occuring with php-curl.Another noticable thing is this request works fine on localhost it only gets 400 on live server.
This is how i am making the curl request
$username = $params['paypal_client_id'];
$password = $params['paypal_secret'];
$headers = array(
"Accept-Language: en_US",
"Accept: application/json"
);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_USERPWD, trim($username) . ':' . trim($password));
curl_setopt($ch, CURLOPT_POST, 1);
$data = curl_exec($ch);
This is how i am making request via terminal
curl https://api.sandbox.paypal.com/v1/oauth2/token -H "Accept: application/json" -H "Accept-Language: en_US" -u "{Client_id}:{Client_secret}" -d "grant_type=client_credentials"
The response i am getting is
Verbose info :* About to connect() to api.sandbox.paypal.com port 443 (#1)
* Trying 173.0.82.78...
* Connected to api.sandbox.paypal.com (173.0.82.78) port 443 (#1)
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA256
* Server certificate:
* subject: CN=api.sandbox.paypal.com,OU=PayPal Production,O="PayPal, Inc.",L=San Jose,ST=California,C=US
* start date: Aug 21 00:00:00 2018 GMT
* expire date: Aug 20 12:00:00 2020 GMT
* common name: api.sandbox.paypal.com
* issuer: CN=DigiCert Global CA G2,O=DigiCert Inc,C=US
* Server auth using Basic with user 'client_id'
> POST /v1/oauth2/token?grant_type=client_credentials HTTP/1.1
Authorization: Basic 'client_id'
Host: api.sandbox.paypal.com
Accept-Language: en_US
Accept: application/json
Content-Length: -1
Content-Type: application/x-www-form-urlencoded
Expect: 100-continue
< HTTP/1.1 400 Bad Request
< Date: Wed, 23 Jan 2019 12:23:30 GMT
< Server: Apache
< Content-Length: 338
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
<
* Closing connection 1
Any help regarding this issue will be really appreciated Thanks.
As the comments already point out, it's hard to debug this remotely. I lack reputation to add a comment, so will suggest this answer instead. This is a complex problem, so I'm afraid this answer will, at best, point you in the right direction rather than give you a specific walk-through.
Firstly, you would be much better off using the paypal PHP SDK which at the time of writing can be found on this page:
https://github.com/paypal/PayPal-PHP-SDK/
If that link is broken, I'm sure Google will assist. This will almost certainly solve your problems.
However, to answer your question (or that of anyone else who, for some reason, can't use the canonical library) - as can be seen from the link above, Paypal are quite strict about their security. I strongly suspect that your command line curl is using a different SSL connection protocols from your php-curl. I would be surprised if this page didn't answer solve your problem:
https://github.com/paypal/tls-update#php
I have reproduced it here in case of link breakage:
PHP requirements
PHP uses the system-supplied cURL library, which requires OpenSSL 1.0.1c or later. You might need to update your SSL/TLS libraries.
Guidelines
Find OpenSSL in these locations:
OpenSSL installed in your operating system's openssl version.
OpenSSL extension installed in your PHP. Find this in your php.ini.
OpenSSL used by PHP_CURL - curl_version().
These OpenSSL extensions can be different, and you update each one separately.
PayPal and other PHP SDKs use the same OpenSSL extension that PHP_CURL uses to make HTTP connections. The PHP_CURL OpenSSL extension must support TLSv1.2.
The php_curl library uses its own version of the OpenSSL library, which is not the same version that PHP uses, which is the openssl.so file in php.ini.
To verify your PHP and TLS versions
To find the openssl_version information for cURL, run:
php -r 'echo json_encode(curl_version(), JSON_PRETTY_PRINT);'
The returned php_curl version might be different from the openssl version because they are different components.
When you update your OpenSSL libraries, you must update the php_curl OpenSSL version and not the OS OpenSSL version.
Download cacert.pem and TlsCheck.php.
In a shell on your production system, run:
php -f TlsCheck.php
On success:
PayPal_Connection_OK
On failure:
curl_error information
Notes:
Make sure that your command line test uses the same versions of PHP and SSL/TLS libraries that your web server uses.
If you use MAMP or XAMPP as your development set up, the PHP that is packaged with them uses an earlier version of OpenSSL, which you cannot easily update. For more information about this issue and a temporary workaround, see Unknown SSL protocol error.
One final thing - if this is the cause of your error, I'm going to assume your live server is at least a couple of years old. It's totally unrelated and none of my business, but if it's that out of date you probably have a whole bunch of other vulnerabilities. I would strongly suggest updating or even spinning up a new server (which may well be easier said than done...)
Today i am having a weird problem with elasticsearch and curl. Elasticsearch is running fine but when i try to connect it with curl, it waits forever and no data returns.
I tried to get verbose output from curl, it can connect to elasticsearch, but after connect, it receives no data.
curl -v http://10.0.0.1:9200/_cat/indices
* Trying 10.0.0.1...
* TCP_NODELAY set
* Connected to 10.0.0.1 (10.0.0.1) port 9200 (#0)
> GET /_cat/indices HTTP/1.1
> Host: 10.0.0.1:9200
> User-Agent: curl/7.52.1-DEV
> Accept: */*
>
It waits forever. Also elasticsearch php is not working as same.
When i tried to open same url from same machine with lynx, it can get data successfully. But when i tried to open same url with wget, it waits forever too. Curl can connect other external elasticsearch hosts without problem. Also we can connect to this elasticsearch host from other hosts without problem.
The other weird thing is, when i tried to get /_cat/health with curl or wget, they can get the data.
curl -v http://10.0.0.1:9200/_cat/health
* Trying 10.0.0.1...
* TCP_NODELAY set
* Connected to 10.0.0.1 (10.0.0.1) port 9200 (#0)
> GET /_cat/health HTTP/1.1
> Host: 10.0.0.1:9200
> User-Agent: curl/7.52.1-DEV
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: text/plain; charset=UTF-8
< content-length: 63
<
1545812391 11:19:51 mycluster green 2 2 382 381 0 0 0 0 - 100.0%
* Curl_http_done: called premature == 0
* Connection #0 to host 10.0.0.1 left intact
There is no firewall on both machines. I tried to update curl, nothing changed. I also coudln't find any usefull information on syslog, elasticsearch logs etc.
Any idea why this happens?
This system is working perfectly for 4 months. I am having this issue since 2 days. Then i tried to upgrade system packages, rebooted the servers but nothing changed. I cleared the arp cache also.
Thank you.
I have example.com on two different servers, I use php curl to connect to one server, do some tasks and then try to connect to another server and do some other tasks, but it doesn't obey the CURLOPT_RESOLVE second time as you can see in the verbose output.
I tried CURLOPT_DNS_CACHE_TIMEOUT set to 0 but still connects to old ip.
Any idea how i can fix it?
I can not connect to ip's directly unfortunately.
* Added example.com:80:207.230.220.255 to DNS cache
* Hostname was found in DNS cache
* Trying 207.97.92.288...
* Connected to example.com (207.97.92.288) port 80 (#0)
> GET /example.php HTTP/1.1
Host: example.com
Accept: */*
< HTTP/1.1 404 Not Found
< Date: Tue, 05 Apr 2016 18:00:58 GMT
* Server Apache is not blacklisted
< Server: Apache
< Accept-Ranges: bytes
< Transfer-Encoding: chunked
< Content-Type: text/html
<
* Connection #0 to host example.com left intact
EDIT *******
Added code :
$check_ping_resolve = ["example.com:80:207.230.220.255"];
$check_ping = "http://example.com/123.php";
$check_ping_verbose = fopen(dirname(__FILE__) . '/ping.txt', 'w');
$check_ping_curl = curl_init();
curl_setopt($check_ping_curl, CURLOPT_SSL_VERIFYHOST,0);
curl_setopt($check_ping_curl, CURLOPT_SSL_VERIFYPEER,0);
curl_setopt($check_ping_curl, CURLOPT_RETURNTRANSFER,1);
curl_setopt($check_ping_curl, CURLOPT_DNS_CACHE_TIMEOUT, 0);
curl_setopt($check_ping_curl, CURLOPT_RESOLVE, $check_ping_resolve);
curl_setopt($check_ping_curl, CURLOPT_URL, $check_ping);
curl_setopt($check_ping_curl, CURLOPT_VERBOSE, 1);
curl_setopt($check_ping_curl, CURLOPT_STDERR, $check_ping_verbose);
$check_ping_curl_result = curl_exec($check_ping_curl);
I had the same problem, I solved it by adding this option:
curl_setopt($ch, CURLOPT_DNS_USE_GLOBAL_CACHE, false);
As mentioned in the official documentation for the CURLOPT_RESOLVE option:
Remove names from the DNS cache again, to stop providing these fake resolves, by including a string in the linked list that uses the format "-HOST:PORT". The host name must be prefixed with a dash, and the host name and port number must exactly match what was already added previously.
So when you send a request to the second server you have to remove mapping for the previously used server IP address as well:
$check_ping_resolve = [
'-example.com:80:207.97.92.288',
'example.com:80:207.230.220.255'
];
=====
UPDATE
I am having a degree of success connecting to the server with fsockopen. However, I would like to use cUrl if possible.
=====
Original
I am trying to make a request to flask using curl in the form of a post. The goal is not to use jQuery but to achieve more separation and validation of information. Despite being able to make a request from my desktop, I cannot make a request from my web server. I have iptables running and have listed the rules below. Also, I have listed the PHP code. I am not receiving any output from flask at all.
Technology:
Go Daddy Hosting account where the request is made
Go Daddy vps for the flask server
Iptables:
ACCEPT all -- c-xxx.xxx.xxx anywhere
ACCEPT all -- ip-xxx.xxx.xxx anywhere
The ip listing is from the web server but the c listing is from my desktop.
Curl Request from PHP
$ch=curl_init();
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_TIMEOUT, 400);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, True);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, True);
curl_setopt($ch, CURLOPT_PORT, 8888);
curl_setopt($ch, CURLOPT_POST, count($data));
curl_setopt($ch, CURLOPT_POSTFIELDS, array('url'=>'login','user'=>$user,'pass'=>$pass,'auth'=>$authstring,'ip'=>$ip));
$result = curl_exec($ch);
This request is not even registering with tcpdump -i eth0 port 8888 unlike the requests from my desktop. The ip addresses are correct.
The request is working in Aptana.
The specific error is:
* Rebuilt URL to: xxx.xx.xx.xx/
* Hostname was found in DNS cache
* Trying xxx.xx.xx.xx...
* connect to xxx.xx.xx.xx port xx failed: Connection refused
* Failed to connect to xxx.xx.xx.xx port xx: Connection refused
* Closing connection 8
This is after completely flushing my iptables. In Aptana with the same php version, I get:
Trying xxx.xx.xx.xx...
* Adding handle: conn: 0x278fa30
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x278fa30) send_pipe: 1, recv_pipe: 0
* Connected to XXX.XX.XX.XX (XXX.XX.XX.XX) port XXXX (#0)
> POST / HTTP/1.1
Host: xxx.xx.xx.xx:xx
Accept: */*
Content-Length: 29
Content-Type: application/x-www-form-urlencoded
* upload completely sent off: 29 out of 29 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 4
< Server: Werkzeug/0.10.4 Python/2.7.5
< Date: Wed, 24 Jun 2015 07:04:28 GMT
<
Unfortunately, a hosting representative from GoDaddy told me that only the following ports are open to me. 80, 21 ftp, 22 SFTP, 445 ssl, 3306 mysql. Basically, I cannot send anything. I had to switch hosting services. A VPS from Go Daddy works too.