I've been scouring around trying to understand curl and building headers but it seems no matter what I do I get the following error :
HTTP/1.1 400 Bad Request Date: Tue, 12 Apr 2016 18:15:33 GMT Server: Apache/2.2.29 (Unix) mod_wsgi/3.5 Python/2.7.10 PHP/5.6.10 mod_ssl/2.2.29 OpenSSL/0.9.8zh DAV/2 mod_fastcgi/2.4.6 mod_perl/2.0.9 Perl/v5.22.0 Content-Length: 226 Connection: close Content-Type: text/html; charset=iso-8859-1
Bad Request
Your browser sent a request that this server could not understand.
The following is the function that I am using to send post variables and fetch the contents via curl. Both the file doing the fetching and the file whose contents are fetched are being hosted locally on MAMP
$url = "./lib/otherpage.php";
$data = array("url"=>$_POST["url"],"format"=>"json");
function tryCurl($baseurl,$data)
{
$bodydata = array(json_encode($data));
$bodystr = http_build_query($bodydata);
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_URL,$baseurl);
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$bodystr);
curl_setopt($ch, CURLOPT_PORT,8888);
curl_setopt($ch, CURLOPT_HEADER,1);
curl_setopt($ch, CURLOPT_HTTPHEADER,array(
"POST / HTTP/1.0",
"Content-Type: application/x-www-form-urlencoded",
"Content-Length: ".strlen($bodystr),
"Host: localhost:8888/gt_dev/",
"User-Agent: My-User-Agent 1.0",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Cache-Control: no-cache",
"Accept-Language: en;q=0.7,en-us;q=0.3",
"Connection: close",
));
// Execute
$result=curl_exec($ch);
// Printing any errors..
echo curl_error($ch);
curl_close($ch);
return $result;
}
Ultimately, the request should send two post variables ("url" and "format") to the receiving file, and expect a json string in return.
This isn't so much an answer to the original issue but turns out I was thinking about my problem the wrong way.
I wanted to use page A as a proxy to page B. And to pass the same variables page B expected from Page A.
Turns out the same effect can be achieved by simply including page B, not curling the expected parameters to it. So the answer is just
include("./lib/otherpage.php");
Related
I'm trying to access this website natura.com.br to get info about some products, I'm trying to make a request the same way I do in the browser but I receive Access denied.
<?php
$url = "https://www.natura.com.br/";
$curl = curl_init($url);
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$headers = array(
"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:89.0) Gecko/20100101 Firefox/89.0",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3",
"Connection: keep-alive",
"Cookie: dtPC=11$302937546_809h-vKVNUNMUJAHDKTQPPAHFQIMSCQPPRKILC-0e0; rxvt=1625712010903|1625710210903; _abck=141C18ABCC9C79DFA1F4D45EF108501A~-1~YAAQk9j2SGYWZX56AQAAtC1zgwbF2BpCRBBWxNKDpMPWjPEYtw7XLvfB2g+dxkZSpfXBRCTvbE4fZTzGxjqtGVa7tZqz9pWp/hYSf65phFg2VW06IioCwpy7tox8su7QlWIt3eZUeTFIJ5S7nRtSv3Te859lzghM7a2lLnZOjpjS5eYLrJzYzyKFsRdc8Kpgj+sGCZA9iKiMvfqtJlI5e+90UfUDDzTQB2sV2U5i2yavjB+GsgBZ1qcKIfmo41PYfXtS8efG+3C4F4cw3sePIjj4vQIex1pQdg3twIbyqAMsEClNaqbGJ29NVu/orK7dAJ7rteSqWvyMuMu9MCltlPoYaYYi/O0Uzv18AiK9hSMYYWod7U/IqUkfE/touhQz6OT0S2KPjG0wJVJ8dghA8LdDJFH4+Y3YX4+oDw==~0~-1~-1; dtCookie=v_4_srv_11_sn_44IDRCETVPHGS7OCQ5F7DV8IS6T6HDI3_perc_100000_ol_0_mul_1_app-3A2a1999229615da92_1_app-3A58ea22250e34b990_1_rcs-3Acss_0; rxVisitor=1625332161003TGNBC4CH08VFEACK6NT19CQ14GERSQEA; dtSa=-; dtLatC=6; RT="z=1&dm=www.natura.com.br&si=a6979d10-a558-4bb3-a962-ccb19999271b&ss=kqu5jycl&sl=0&tt=0"; JSESSIONID=IUGNrH_B8h7DHZlVAsWr38FYiZq_I13dabE1sdnURzTRvdlsps50!-405357702; verifyFirstRequest=true; ORDER_ID=%3B%20; ORDER_NUMBER=%3B%20; X-Oracle-BMC-LBS-Route=93a9c3aed1d29d32ea0391633407edd696daed8327da03a11a2ff120e313e9b656c62fd8a7c42ae86da5fb4c73ec2333f092dbf8c9611add8055dec1; undefined; GTMUtmTimestamp=1625702943894; GTMCampaignReferrer=https%3A%2F%2Fwww.google.com%2F; GTMCampaignLP=https%3A%2F%2Fwww.natura.com.br%2Fp%2Fdesodorante-Col%25C3%25B4nia-kaiak-urbe-masculino-100ml%2F34075%3Futm_content%3DSP_Resp_MCKaiak_2020_Kaiak_Urbe%26cnddefault%3Dtrue%26gclid%3DEAIaIQobChMIxrmerJfS8QIV7R-tBh01uwM5EAAYASAAEgIZU_D_BwE; _gcl_au=1.1.467484977.1625332166; _ga_15QCH7XEDY=GS1.1.1625874531.3.0.1625874531.60; _ga=GA1.3.1744316957.1625332166; GTMBrowserSessionEntranceTimestamp=1625332166546; GTMBrowserSessionEntranceLP=https%3A%2F%2Fwww.natura.com.br%2Facesso-consultor; GTMBrowserSessionEntranceReferrer=https%3A%2F%2Fwww.google.com%2F; GTMLastEntranceTimestamp=1625702945066; GTMLastEntranceLP=https%3A%2F%2Fwww.natura.com.br%2Fp%2Fdesodorante-Col%25C3%25B4nia-kaiak-urbe-masculino-100ml%2F34075%3Futm_content%3DSP_Resp_MCKaiak_2020_Kaiak_Urbe%26cnddefault%3Dtrue%26gclid%3DEAIaIQobChMIxrmerJfS8QIV7R-tBh01uwM5EAAYASAAEgIZU_D_BwE; GTMLastEntranceReferrer=https%3A%2F%2Fwww.google.com%2F; _fbp=fb.2.1625332167173.86579534; GTMUtmSource=google; GTMUtmMedium=cpc; GTMUtmCampaign=auto; GTMGclid=EAIaIQobChMIxrmerJfS8QIV7R-tBh01uwM5EAAYASAAEgIZU_D_BwE; _gcl_aw=GCL.1625702944.EAIaIQobChMIxrmerJfS8QIV7R-tBh01uwM5EAAYASAAEgIZU_D_BwE; _ttgclid=EAIaIQobChMIxrmerJfS8QIV7R-tBh01uwM5EAAYASAAEgIZU_D_BwE; _ttgclid=EAIaIQobChMIxrmerJfS8QIV7R-tBh01uwM5EAAYASAAEgIZU_D_BwE; __bid=4dbbd24a-fc0e-42aa-a664-43c865cae2ef; _gac_UA-35236522-3=1.1625703004.EAIaIQobChMIxrmerJfS8QIV7R-tBh01uwM5EAAYASAAEgIZU_D_BwE; cto_bundle=qVL1jF8yb3h2cW01cmFaUjF5S2VnMkdGcDEyNHdrWWM3ZXpZcFFIMkowdmZqOFZpNTE3VGIlMkJibkZZZXpMJTJGYXBqRElFT0FwdFRia3ZwQ1JZSzZ2ZEx1RSUyQnBXUW54MmM1S3UlMkY3WjlqenBXU1dDZFlRaTYlMkZucVdPM2c5MTJUcERYc2pLcEdyeURWQU9ieDhmaTR1WGJoN1FhaHR4Nm5MZWdJUUNmZUI0MkdNYk50VXpFRGg0Z2hUSlNxS25xZGFNVjJZTGpC; GTMVisitSession=1625702947208; GTMVisitPermanent=1625702947208; _uetvid=b67fe410df8011eb85947b37dce069f8; smeventsclear_d13b2682b72e42cc9203ee1f0a20b68d=true; personNumber=141692103; smCloseBounce=true",
"Upgrade-Insecure-Requests: 1",
);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);
$resp = curl_exec($curl);
curl_close($curl);
var_dump($resp);
?>
I get this HEADER when I do the request
"HTTP/1.1 403 Forbidden Server: AkamaiGHost Mime-Version: 1.0 Content-Type: text/html Content-Length: 271 Cache-Control: max-age=1800 Expires: Sat, 10 Jul 2021 21:35:46 GMT Date: Sat, 10 Jul 2021 21:05:46 GMT Connection: close Server-Timing: cdn-cache; desc=HIT Server-Timing: edge; dur=1 Timing-Allow-Origin: true Access-Control-Max-Age: 86400 Access-Control-Allow-Credentials: true Access-Control-Allow-Headers: * Access-Control-Allow-Methods: GET,POST,OPTIONS Access-Control-Allow-Origin: *
First of all, within "" PHP tries to convert variables to strings. So
$foo = "Hello, ";
echo "$foo World" . PHP_EOL;
would output
Hello, World
You are using "" and have the $-sign in your cookie (not to mention the fact that the cookie-string itself contains the "-sign). So PHP is not sending that "as-is" in the cookie but instead trying to find that variable and turn it into a string. If you don't want this behavior use single quotes ' instead.
So that could be your issue as long as the information in the cookie is valid and should allow you access.
Besides this it is hard to know what happens in the backend of natura.com.br to block your access. It could be any number of things. Do you need to login, do you have a valid session etc.
Instead of working with curl directly you could use Guzzle which is a great PHP based HTTP-Client that simplifies a lot of stuff. It can keep track of cookies and append them automatically etc.
I'm trying to set up an API call through php using cURL. The API documentation gives an example for a call to this API from Java:
HttpResponse<JsonNode> response = Unirest.get("https://api2445582011268.apicast.io/games/1942?fields=*")
.header("user-key", "*******")
.header("Accept", "application/json")
.asJson();
This is my attempt at converting it to a cURL call in php:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://api2445582011268.apicast.io/games/1942?fields=*");
curl_setopt($ch, CURLOPT_HEADER, array(
"user-key: *******",
"Accept: application/json"
));
$output = curl_exec($ch);
echo $output;
curl_close($ch);
However my php is echoing the following output:
HTTP/1.1 403 Forbidden Content-Type: text/plain; charset=us-ascii Date: Sat, 02 Sep 2017 02:52:40 GMT Server: openresty/1.9.15.1 Content-Length: 33 Connection: keep-alive Authentication parameters missing1
Does anyone know how to solve this?
You used CURLOPT_HEADER (which is a Boolean to indicate you want the headers in the output), but you need CURLOPT_HTTPHEADER (which is used to pass an array with the headers for the request):
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
"user-key: *******",
"Accept: application/json"
));
My goal is to send a POST request to a server and get the proper response.
Note: Angled brackets represent placeholders.
In Terminal, using the following code will provide me the desired response.
curl -u <user>:<pass> -H 'Content-Type: application/xml' -X POST https://<rest of url>
My current PHP looks something like this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $uri); //$uri is the same that I use in terminal
curl_setopt($ch, CURLOPT_USERPWD,
sprintf('%s:%s', $user, $pass)); //same as terminal user & pass
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$headers = array(
'Content-Type: application/xml', //expect an xml response
);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$curl_result = curl_exec($ch);
Using this PHP, I get a 400 Bad Request error.
The verbose information:
> POST <same url> HTTP/1.1
Authorization: Basic YWRtaW5Ac3Bhcms0NTEuY29tOnNwYXJrc29tZXRoaW5n
Host: <correct host>
Accept: */*
Content-Type: application/xml
Content-Length: -1
Expect: 100-continue
* HTTP 1.0, assume close after body
< HTTP/1.0 400 Bad request
< Cache-Control: no-cache
< Connection: close
< Content-Type: text/html
Why am I getting a 400 Bad Request error when I use PHP, but not when I use command line? How can I fix this issue so that I get my desired response using PHP?
curl_setopt($ch, CURLOPT_POSTFIELDS, array());
After adding this line, I resolved my problem. In a way, this solution makes sense; but I don't understand why CURLOPT_POSTFIELDS is required. In the PHP documentation, this part should be included under CURLOPT_POST, unless this just accidentally works.
I don't know if this can help you, but for me the Expect: 100-continue looks strange. Take a look at this comment:
http://php.net/manual/en/function.curl-setopt.php#82418
So maybe you can fix it like in the example:
<?php
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Expect:'));
?>
I'm trying to connect to the Twitter api server to make an "Application-only Autentication".
I don't care any other way to connect to Twitter. I need this specific method.
I need to go from localhost through my corporation's proxy to api.twitter.com which needs ssl
Following the instruction of this twitter developer's page https://dev.twitter.com/docs/auth/application-only-auth, i tried with:
cUrl:
try {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
if ($this->proxy != '') {
curl_setopt($ch, CURLOPT_PROXY, $this->proxy);
curl_setopt($ch, CURLOPT_PROXYPORT, $this->port);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, $this->userpwd);
}
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic ".base64_encode(urlencode($consumer_key).":".urlencode($consumer_secret))."
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
"));
$response = curl_exec($ch);
if (FALSE === $response) throw new Exception(curl_error($ch), curl_errno($ch));
curl_close($ch);
var_dump(json_decode($response));
}
catch(Exception $e) {
trigger_error(sprintf('Curl failed with error #%d: %s', $e->getCode(), $e->getMessage()), E_USER_ERROR);
}
Which gives me
Fatal error: Curl failed with error #35: Unknown SSL protocol error in connection to api.twitter.com
file_get_contents:
$context = stream_context_create(array(
"http" => array(
"method"=>"CONNECT",
"proxy" => $this->proxy.":".$this->port,
"request_fulluri" => true,
"header" => "
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Proxy-Authorization: Basic ".base64_encode(urlencode($this->userpwd))."
Authorization: Basic ".base64_encode(urlencode($consumer_key).":".urlencode($consumer_secret))."
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
",
),
));
$response = file_get_contents($url, False, $context);
var_dump(json_decode($response));
Which gives me
Warning: file_get_contents(https://api.twitter.com/oauth2/token) [function.file-get-contents]: failed to open stream: Cannot connect to HTTPS server through proxy
fsockopen:
$fp = fsockopen($this->proxy, $this->port);
fputs($fp, "
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic ".base64_encode(urlencode($consumer_key).":".urlencode($consumer_secret))."
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
");
$data="";
while (!feof($fp)) $data .= fgets($fp,1024);
fclose($fp);
var_dump($data);
Which gives me
HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 727
I am sure that the 443 port is open and it's not a problem of the localhost (I got the same error trying on an online server).
I tried even using CONNECT method instead of POST.
I tried tunneling the proxy, but I'm neither sure I made it nor that that's the problem.
I'm running out ideas..
Try to remove this:
curl_setopt($ch, CURLOPT_SSLVERSION, 3);
You only have one value in this array, it's wrong.
curl_setopt($ch, CURLOPT_HTTPHEADER, array("
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic ".base64_encode(urlencode($consumer_key).":".urlencode($consumer_secret))."
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
"));
Change the above for this:
$consumer_key = base64_encode(urlencode($consumer_key);
$consumer_secret = urlencode($consumer_secret);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
"Host: api.twitter.com",
"User-Agent: My Twitter App v1.0.23",
"Authorization: Basic $consumer_key:$consumer_secret",
"Content-Type: application/x-www-form-urlencoded;charset=UTF-8",
"Accept-Encoding: gzip",
"grant_type=client_credentials"
));
if you want to include the Content-Length: xx, you need to use strlen() to get string length of the post, ex;
$length = strlen($post_content);
Then add it to the CURLOPT_HTTPHEADER array:
"Content-Length: $length"
Found the problem. There's no need (maybe only in this case, i'm not sure) to base64 encode the credentials. They'll be encoded by the server.
I don't know the reason of that different error responses, but was in fact a problem of double encoding, because of which the server was not able to verify my credentials.
I am not that experienced with cURL and I have spent a couple of days trying to sort this problem: I have an issue with cURL not appending the query string to my URL in the headers when I submit a POST request; hence no 'payload' is received by the server and I get returned an error status code by the service I'm accessing which indicated it didn't receive the appropriate data.
I think the POST should start with the full domain name, but I'm not sure. If I'm posting data, shouldn't Content-Length be '0' instead of what I am getting?
The outgoing header looks like this:
POST /rest/v1/oliver/groups/ORIGINNUMBER/member? HTTP/1.1
Accept: application/xml
Content-type: text/plain
User-Agent: Custom PHP Script
Host: campaign.oventus.com
Cookie: JSESSIONID=SECRETCOOKIE
Content-Length: 95
My php code looks like this:
$fields_string = "firstName=$fname&secondName=$sname&password=$pass&deviceAddress=$phonenumber¬es=testing";
$url = "http://campaign.oventus.com/rest/v1/ACCOUNTNAME/groups/ORIGINNUMBER/member?";
$header[] = "Accept: application/xml";
$header[] = "Content-type: text/plain";
$header[] = "User-Agent: Custom PHP Script";
$header[] = "Host: campaign.oventus.com";
$header[] = "Cookie: ".$cookie;
$cx = curl_init();
curl_setopt($cx, CURLOPT_URL,$url);
curl_setopt($cx, CURLOPT_POST, 5);
curl_setopt($cx, CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($cx, CURLOPT_HTTPHEADER, $header);
curl_setopt($cx, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($cx, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($cx, CURLOPT_RETURNTRANSFER, true);
curl_setopt($cx, CURLOPT_FOLLOWLOCATION, FALSE);
curl_setopt($cx, CURLOPT_TIMEOUT, 15);
curl_setopt($cx, CURLOPT_HEADER, 1);
curl_setopt($cx, CURLINFO_HEADER_OUT, TRUE);
$final = curl_exec($cx);
$errors = curl_error($cx);
$errornos = curl_errno($cx);
$headcut2 = explode ("n/xml", $final);
$headstring2 = $headcut2[0]."n/xml";
$xmlstring2 = $headcut2[1];
echo "<h2>Add to Group result: </h2>";
echo "<p>RAW header: <code>$final</code></p>";
//echo "<p>Response header: <code>".htmlentities($headstring2)."</code></p>";
echo "<p>XML response: <code>".htmlentities($xmlstring2)."</code></p>";
//echo "<p>".print_r($info)."</p>";
//echo "<p>CURL info: $info</p>";
//echo "<p>Curl error: $errors</p>";
echo "<p>Curl error num: $errornos</p>";
print "<pre>\n";
print_r(curl_getinfo($cx)); // get error info
print "</pre>\n";
curl_close($cx);
And the header returned by the server is this:
HTTP/1.1 200 OK Date: Fri, 26 Aug 2011 17:30:12 GMT
Server: Apache/2.2.17 (Unix) DAV/2 Content-Length: 144
X-Powered-By: Servlet/2.5 JSP/2.1 Cache-Control: no-store, no-cache
Vary: Accept-Encoding,User-Agent Pragma: no-cache
Content-Type: application/xml 202
With the returned XML:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<status xmlns="http://jaxb.rest.pageone.com" description="No Devices to add">202</status>
As far as I can tell, I'm definitely hitting the server, but it doesn't seem to receive the data I'm sending it..
Stumped. Hope someone can point me in the right direction!
Cheers,
Looks like your hitting the server, and likely the data is going to... I think the answer lies in 202:No Devices to add... which the REST interface documentation should explain (perhaps you're missing a required field?) {202 FYI means accepted but no processing was completed, could also mean the user exists}
By the way you should be escaping those arguments you're putting into the POST payload ($fname,$sname,$pass,$phonenumber)... otherwise a weird value (say name) could cause the post to act completely differently to the way you expected. You can do that using urlencode, or by instead building the POST string with http_build_query
<?php
$fields_string=http_build_query(array(
"firstName"=>$fname,
"secondName"=>$sname,
"password"=>$pass,
"deviceAddress"=>$phonenumber,
"notes"=>"testing"));
//...
?>