How can I "correctly" upload the content of a file with cURL?
There are a lot of explanations and questions here where tmp files are send by curl with a #prefix on the file or using the curlFile api. In My use case I don't want to create a tmp file, and just post the content of file_get_contents('php://input') as body of the post request. The server does not accept multipart form requests.
The Following snipped will work in PHP7:
$body = file_get_contents('php://input');
$ch = curl_init();
curl_setopt($ch, CURLOPT_POSTFIELDS, $body);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
It feels ugly to set the content type to json, no matter what the actual content of the stream may holds. But it seems to work. Sadly I need to support a really old version of PHP and feel like running out of options.
My question is:
Is the above example correct, even for PHP 7 or can it be made better?
Additionally: Is there any improvement or option that can be used to make this work with PHP 5.3.10? In this old version the post body always seems to be empty.
Finally I found and answer on this blog
Adding curl_setopt($curl, CURLOPT_POSTREDIR, 3); was the required option to acutally make the post request work without having an empty body.
Edit & heads up: This option is only needed when you expect your request to be redirected. I made a mistake while asking this question and posting the answer. Please check if your requests is redirected form http to https or the other way around and fix this first to prevent problems.
Copied from blog post of Evert Pot, linked above:
Using CURLOPT_POSTFIELDS you can supply a request body as a string.
Lets try to upload our earlier failed request using that method:
<?php
$curl = curl_init('http://example.org/someredirect');
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($curl, CURLOPT_POSTFIELDS, file_get_contents('largefile.json'));
curl_exec($curl);
?>
This also will not work exactly as you expect. While the second
request to /someredirect will still be a POST request, it will be sent
with an empty request body.
To fix this, use the undocumented CURLOPT_POSTREDIR option.
<?php
$curl = curl_init('http://example.org/someredirect');
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($curl, CURLOPT_POSTFIELDS, file_get_contents('largefile.json'));
curl_setopt($curl, CURLOPT_POSTREDIR, 3);
curl_exec($curl);
?>
According to the PHP changelog, this was added in PHP 5.3.2, and
according to PHP bug #49571 there are four possible values:
0 -> do not set any behavior 1 -> follow redirect with the same type
of request only for 301 redirects. 2 -> follow redirect with the same
type of request only for 302 redirects. 3 -> follow redirect with the
same type of request both for 301 and 302 redirects.
Related
Here's the situation. On one side, we have a server, with a RESTful service. One possible query to it is a POST query to create an object. When that is done, the server returns a 'Location' header, to indicate where information on the newly created object can be found.
However, said server is anal about having the correct Content-Type for each request. For instance, POST requires 'application/json', and GET requires this to be unset (make sense, since GET doesn't have a body).
To sum up, we have:
www.example.com/articles/ ; one can send a POST request with 'Content-Type: application/json', and server will return 'Location: www.example.com/articles/123' if 123 is the id of the new object ;
www.example.com/articles/123 ; one can send a GET request with no 'Content-Type' and server will return a description of the new article object.
On client side, we use PHP with cURL. We use the CURLOPT_FOLLOWLOCATIONsetting so we can read the description of the newly created object. Obviously, we also set 'Content-Type: application/json' for our POST request:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_POSTFIELDS, '{"name": "test"}');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_URL, "https://www.example.com/Articles/");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, ['Content-Type: application/json']);
$result=curl_exec($ch);
var_dump($result);
curl_close ($ch);
?>
This is what we get:
string(101) "{ "errorNo": 415, "error": "Unsupported Media Type Content-Type should not be set in a GET request" }"
I looked at the log of the server, and indeed, 'Content-Type: application/json' is sent to GET www.example.com/articles/123.
Is this an expected behaviour?
If yes, what is then best approach:
remove the 'Content-Type' check on GET requests, server-side?
(sounds silly)
forget about CURLOPT_FOLLOWLOCATION, and make 2 clearly separated curl requests, so I have control over the headers? (but then what's the point of CURLOPT_FOLLOWLOCATION?)
something else?
For control and testing, I also use Postman, and I have no problem with it, it follows the location, doesn't send the 'Content-Type' on the GET part (apparently) after the redirection and so I don't have an error.
EDIT:
There seems to be nothing useful in the PHP doc. But I found something interesting in the command line man page:
https://curl.haxx.se/docs/manpage.html
It says:
"WARNING: headers set with this option will be set in all requests - even after redirects are followed, like when told with -L, --location."
So I guess it probably is the expected behaviour for PHP too. May someone suggest best practices then?
Have you tried using
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
to set the post type
I have been given an example piece of code from a company I'm dealing with for how to post XML data to a URL then read the response. Unfortunately for me this in VBS which I don't have a good working knowledge of:
This is the section of code that I'm interested in. This should pass over the XML file that was read in to oXML then post it and read the response:
set oHTTP = CreateObject("Microsoft.XMLHTTP")
oHTTP.open "POST", "http://www.ophub.net/opxml/response.asp", false,00092,QW 'file url - with dealers Account number, Password
oHTTP.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
oHTTP.setRequestHeader "Content-Length", Len(sRequest)
oHTTP.send oXML
From what I understand of this in PHP this can be done with cUrl and I have come up with the following from bits that I have read online but this doesn't work and I'm not sure why.
$ch = curl_init();
curl_setopt($ch, CURLOPT_HTTPHEADER, Array("Content-Type: application/x-www-form- urlencoded"));
curl_setopt($ch, CURLOPT_URL, "http://www.ophub.net/opxml/response.asp");
curl_setopt($ch, CURLOPT_USERPWD, "00092:QW");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, "XML=" . $xml);
$content=curl_exec($ch);
echo $content;
I'm sure I can't be far off what I need but I can't seem to get there so any hep would be very much appreciated.
In your post fields just set the xml no need to set xml=
curl_setopt($ch, CURLOPT_POSTFIELDS, $xml);
you also need return transfer true
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
It looks to me like that is precisely what you need. It seems to be an exact translation.
The only two potential issues that I can see are:
You PHP code seems to have a space in the middle of the Content-Type header value, this will most likely break things
You will, if you haven't already done so, need to URL encode the $xml data before sending it as part of a application/x-www-form-urlencoded message (clue's in the type name :-P), which you can do with urlencode().
It might be a good idea to build the body data as a string and echo it out, to ensure that the data is correct:
echo 'XML=' . urlencode($xml);
For the record wrapping XML messages in application/x-www-form-urlencoded is a horrible way to do things. But I've come across more than one API that does it, so I'm going to assume that your code is correct in this regard for the API you are trying to consume.
I have following URL
http://www.davesinclairstpeters.com/auto2_inventorylist?i=37647&c=12452&npg=1&ns=50&echo=2
I want to retrieve content of this url using curl but everytime I make this request it is showing me error, as it is not passing required parameters
Below is my code
$ch = curl_init(); // start CURL
curl_setopt($ch, CURLOPT_URL, $json_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPGET, true);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
$response = curl_exec($ch);
That page doesn't give any information stating that the information isn't being passed properly. In fact, it tells you that the information has been recieved - by viewing the source, you can see:
<!--
javax.servlet.forward.request_uri = /auto2_inventorylist
...
javax.servlet.forward.servlet_path = /auto2_inventorylist
...
javax.servlet.forward.query_string = i=37647&c=12452&npg=1&ns=50&echo=2
-->
Which tells you the information has infact been recieved.
Therefore, it's no problem with your code, but with the website itself. You should make sure the URL you are using is valid, or contact that website to get more information.
With regards to your code itself - the curl_setopt($ch, CURLOPT_HTTPGET, true); isn't necessary, as this is already set by default, and you can also pass the URL as an argument of the curl_init function. Doesn't impact performance, but makes for neater code.
$ch = curl_init($json_url); // start CURL
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, true);
$response = curl_exec($ch);
You code is perfectly fine and if there's something wrong returned, simply paste this URL to your web browser and check the result. In this case website simply failed for some reasons. There's nothing you can do about that as problem is NOT on your side.
This URL yields a page of cars with links to more cars. Looks like the URL you're starting with is old, or has some sort of expiration factor that's not obvious.
Not knowing which sort of filtering parameters you're shooting for.. hard to say what else my be wrong, other than your starting URL be bad.
working url:
http://www.davesinclairlincolnstpeters.com/all-inventory/index.htm?listingConfigId=auto-new%2Cauto-used&compositeType=&year=&make=&start=0&sort=&facetbrowse=true&quick=true&preserveSelectsOnBack=true&searchLinkText=SEARCH&showInvTotals=false&showRadius=false&showReset=true&showSubmit=true&facetbrowseGridUnit=BLANK&showSelections=true&dependencies=model%3Amake%2Ccity%3Aprovince%2Ccity%3Astate&suppressAllConditions=false
I have a codeigniter application that has a checkout system through authorize.net. The authorize.net library that I uses preforms a curl to make the payment, but after it is done I cannot redirect because
headers already sent by (output started at /Users/phil/Sites/Medbridge/httpdocs/application/libraries/AuthorizeCimLib.php:1
That is what the log says. If I comment out the payment thing it will redirect fine. I don't know if I am not understand a curl and that is why it is doing something or if I need to change some curl settings.
Thank you
EDIT
Here is the link to the library I am using, it is big and didn't want to repost the whole code
http://www.communitymx.com/content/article.cfm?page=4&cid=FDB14
Here is the curl part and maybe someone could see if this is doing the output to the header
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $this->_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, Array("Content-Type: text/xml"));
curl_setopt($ch, CURLOPT_HEADER, FALSE);
curl_setopt($ch, CURLINFO_HEADER_OUT, FALSE);
curl_setopt($ch, CURLOPT_POSTFIELDS, $this->_xml);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$this->_response = curl_exec($ch);
Thank you
This is because the server has output something to your browser, and can't redirect through php after that.
As mentioned in the comments, you're most likely including some whitespace somewhere in your code. For practice and less troubleshooting, you don't need to include the ending ?> at the bottom of your php files. This is simply not required and sometimes there might end up a space after that ?> which causes an echo during the execution of your code.
Are you includeing any file in your code with might have a at the end of the file?
Another "solution" if you may, is to echo
<script type="text/javascript">
top.location = '<?=$str_redirect_url?>';
</script>
EDIT:
If you're using Codeigniter, you should check your model, helper and library files for whitespaces.
Perhaps you can use AJAX to issue the payment call, get the response and redirect using Javascript?
I'm using a web-service from a provider who is being a little too helpful in anticipating my needs. They have given me a HTML snippet to paste on my website, for users to click on to trigger their services. I'd prefer to script this process, so I've got a php script which posts a cURL request to the same url, as appropriate. However, this provider is keeping tabs on my session, and interprets each new request as an update of the first one, rather than each being a unique request.
I've contacted the provider regarding my issue, and they've gone so far as to inform me that their system is working as intended, and that it's impossible for me to avoid using the same ASP.NET session for each subsequent cURL request. While my favored option would be to switch to a different vendor, that doesn't appear to be an option right now. Is there a reliable way to get a new ASP.NET session with each cURL request?
I've tried the following set of CURLOPT's, to no avail:
//initialize curl
$ch = curl_init($url);
//build a string out of the post_vars
$post_str = http_build_query($post_vars);
//set the necessary curl options
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_str);
curl_setopt($ch, CURLOPT_FAILONERROR, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, 1);
curl_setopt($ch, CURLOPT_FRESH_CONNECT, 1);
curl_setopt($ch, CURLOPT_FORBID_REUSE, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "UZ_".uniqid());
curl_setopt($ch, CURLOPT_REFERER, CURRENT_SITE_URL."index.php?newsession=".uniqid());
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Pragma: no-cache", "Cache-Control: no-cache"));
//execute the call to the backend script, retrieve the results
$xmlstr = curl_exec($ch);
If cURL isn't helping much, why not try other methods to call the services from your script, like php's file() function, or file_get_contents().
If you see do not see any difference at all, then the service provider might be using your ip to track your requests. Try using some proxy for a test.
Normal Asp.net session is tracked by a cookie called ASP.NET_SessionId. This cookie is sent within the response to your first request. So as long as your curl requests don't send back this asp.net cookie, each of your requests will have no connection to each other. Use the curl -c option to see what cookies are flying in-between you and them. Overriding this cookie with a cookie file should work if you confirm that it is normal asp.net session being used here.
It is quite poor for a service to use session (http has much cleaner ways of maintaining state which ReST exploits) so I wouldn't completely rule out the vendor switch option.
Well given the options you are using, it seems you have covered your basics. Can you find out how their sessions are setup?
If you know how they setup a session, IE what they use (if it is IP or what not) and then you can figure out a work around. Another option is trying to set the cookies in a different cookie file:
CURLOPT_COOKIEFILE - The name of the file containing the cookie data. The cookie file can be in Netscape format, or just plain HTTP-style headers dumped into a file.
But if all they do is check cookies your current code should work. If you can figure out what the cookie's name is, you can pass a custom cookie that is blank with the request to see if that works. But if you can get information out of them on how their session's work, that would be best.
use these two line to handle the session:
curl_setopt($ch, CURLOPT_COOKIEJAR, "path/to/cookies.txt"); // cookies.txt should be writable
curl_setopt($ch, CURLOPT_COOKIEFILE, "path/to/cookies.txt");