I'm creating a SublimeText 2 plugin that posts data to a server I run. I wrote the basics while on the train using my phone as a WiFi hot spot and everything worked fine. Then when I got home I got a 400 Bad Request response from my server no matter what I tried to post. I put my laptop back on my personal hot spot and the error went away. Here are the details:
The "client" in this case is Sublime Text 2. If you don't already know, their plugins are written in Python and I'm using urllib, urllib2, and httplib to handle requests. Here is the relevant Python that makes the request:
params = urllib.urlencode({'title': 'ST2 Note', 'content': data, 'user': user, 'pass': pswd})
headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}
conn = httplib.HTTPConnection("staging.myserver.me:80")
conn.request("POST", "staging.myserver.me/st2", params, headers)
response = conn.getresponse()
data2 = response.read()
print data2
conn.close()
The code above sends a POST request just fine. If I'm on a certain connection my server definitely understands it because I set it to just echo back what I send in the POST data. The request is being made to part of a PHP (Codeigniter) application and yes, I've specifically set it up so that CSRF protection is off for this particular URL so I know that's not the issue. The PHP code itself is rather uninteresting but I set it to echo back the server headers and this is what it sent when I made the request from the connection that works:
Host: staging.myserver.me
Accept-Encoding: identity
Content-Length: 56
Content-type: application/x-www-form-urlencoded
Accept: text/plain
Via: HTTP/1.1 akrmspsrvz9ts212.wnsnet.attws.com
Any ideas why the server understands requests from some connections but not others?
Looks like I got it working. I KNOW there are others with my problem out there so here's what happened...
For reasons that I'm too much of a Python/HTTP1.1 newbie to understand, a plan hostname without "http://" in front of it works some of the time. Must have something to do with ISPs and how they route traffic, not sure. So the fix was do modify this line:
conn.request("POST", "staging.myserver.me/st2", params, headers)
The above line caused problems. But changing it to this:
conn.request("POST", "http://staging.myserver.me/st2", params, headers)
Got it to work instantly! I hope this helps someone down the line. 400 errors are almost as mysterious as 500 errors sometimes.
Related
I have a simple rest endpoint built on PHP that works on postman and browsers.
https://someserver.com/api/endpoint.php?name=hello
But the code fails to execute when invoked from Swift.
Upon Investigation, I found out that Postman and Browser sends the Host Header whereas the Swift URLRequest doesn't. When you don't send the HOST Header, the php server refuses to accept the incoming request and I believe the request is rejected at the web server level. I tried using the
request.addValue("127.0.0.1", "Host")
but it still didn't work. Any help is appreciated.
Full Code (Swift 4)
let url = urlInput.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed)!
print("URL is: " + url)
var request = URLRequest(url: URL(string: url)!)
request.httpMethod = "GET"
request.addValue(<#T##value: String##String#>, forHTTPHeaderField: <#T##String#>)
let session = URLSession.shared
let task = session.dataTask(with: request, completionHandler: { data, response, error -> Void in
if let httpResponse = response as? HTTPURLResponse {
let respStr = String(data: data!, encoding: String.Encoding.utf8) as String?
print ("Response is \(respStr ?? "")")
print("statusCode: \(httpResponse.statusCode)")
let data: Data? = respStr?.data(using: .utf8)
}
})
Swift is ignoring the fact that you set the header, as it typically should. See the docs about Swift headers and why it's ignoring your Host header.
The URL Loading System handles various aspects of the HTTP protocol
for you (HTTP 1.1 persistent connections, proxies, authentication,
and so on). As part of this support, the URL Loading System takes
responsibility for certain HTTP headers:
Content-Length
Authorization
Connection
Host
Proxy-Authenticate
Proxy-Authorization
WWW-Authenticate
If you set a value for one of these reserved headers, the system may
ignore the value you set, or overwrite it with its own value, or
simply not send it. Moreover, the exact behavior may change over
time. To avoid confusing problems like this, do not set these headers
directly.
The documentation is here:
https://developer.apple.com/documentation/foundation/nsurlrequest#1776617
I have seen this before with Java - there is a workaround in Java depending on the library you are using, but I don't know about Swift.
I figured out that the issue happens only when you hit a php server. The php web server looks for some header and cant find it when the call is made from Swift. I even copied the working code from Postman and results were the same.
The solution I went with was to create a proxy server using firebase cloud function. Swift code would hit a firebase cloud function which talks to php and sends the response to swift.
I'm posting this on my way home, so forgive the lack of code but I'll try to be as detailed as possible and add code when I can tonight. So essentially I have a react native app using redux and axios. A brief review (code to follow) may explain that I'm doing something wrong.
Serviceapi.js
Creates and exports basic axios with base url.
const ServiceApi = axios.create({
baseURL: BASE_URL,
responseType: 'json'
});
AuthReducer.js
On login sets Authorization header manually using the post method. This works on both android and ios the login is returned and I use the authorization header.
return {
type: PERFORM_LOGIN,
payload: {
user: {
name: username
},
request: {
url: '/login',
method: 'post',
headers: {
'Authorization': 'Basic ' + basicAuth
}
}
}
On login, I return the following redux-axios action, you can see that I set the header: Authorization manually, this works great.
// On login success, set the authInterceptor responsible for adding headers
authInterceptor = ServiceApi.interceptors.request.use((config) => {
console.log(`Attaching Authorization to header ${basicAuth}`);
config.headers.common.Authorization = basicAuth;
return config;
}, (error) => {
Promise.reject(error);
});
On logout I clear the interceptor. I chose to add and remove on login and logout instead of always having it there just because. This could be a problem but it was fine for Android
// Clear the auth interceptor
ServiceApi.interceptors.request.eject(authInterceptor);
Again this is all working great on Android. And it looks to be working on ios. When I debug the interceptor it's getting called and setting the header.
But I get back a 403 on ios. After looking at the request in more detail, there is a big difference between the android header in the request and the ios header in the request. The rest of the request object is the same, only the _header object is different between ios and android.
Android Request
_headers:
accept: "application/json, text/plain, */*"
authorization: "Basic <correct base64 value>"
content-type: "application/json;charset=utf-8"
__proto__: Object
IOS Request
_headers:
accept: (...)
authorization: (...)
content-type: (...)
get accept: ƒ ()
set accept: ƒ ()
get authorization: ƒ ()
set authorization: ƒ ()
get content-type: ƒ ()
set content-type: ƒ ()
__proto__: Object
With the differences, setting a breakpoint at looking at the console for error.request._headers.authorization; I get the same "Basic: " contents as the Android header contains.
index.php
The backend service is a php file that does a $_SERVER['PHP_AUTH_USER'] which fails a 403 if not set which is what's happening. I don't have access to the php, I was just told this is what it's using.
Again I apologize for not providing code but i will when I get a chance later. Is there something maybe I have to set extra for ios? Or maybe php for ios needs an extra header?
Code to follow.
EDIT Updated with code, hopefully I didn't leave in any of the encoded login info.
EDIT 2 Upon further investigation this looks like it's related to apache/PHP rather than react-native/axios. I threw together an express server that simulated the same checking that the PHP does:
- Look for the Authorization header
- Print it
- Return back 403 or 200 w/ data based on that
When running pointing at http://localhost:3000 using the exact same app on the emulator I get back what I'm expecting. To add to this, when I'm on the emulator, I can't actually login to the live URL (even though I could on the regular device), I get the same 403 error but this time a little earlier.
EDIT 3
To provide some more information from the server, here are the three requests that I've been able to log:
1) This is from the IOS Emulator iPhone8 against a an express server:
accept:"application/json, text/plain, */*"
accept-encoding:"gzip, deflate"
accept-language:"en-us"
authorization:"Basic <base 64 encoding>"
connection:"keep-alive"
content-length:"0"
host:"localhost:3000"
user-agent:"MobileApp/1 CFNetwork/978.0.7 Darwin/18.5.
2) This is from the same emulator to apache/PHP (5.3.3), we can see there is no Authorization header.
Accept: application/json, text/plain, */*
User-Agent: MobileApp/1 CFNetwork/978.0.7 Darwin/18.5.0
Accept-Language: en-us
Accept-Encoding: br, gzip, deflate
Connection: keep-alive
3) This is from Android to apache/PHP (5.3.3):
authorization: Basic <Base 64 encoding>
Host: api.serviceurl.com
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: okhttp/3.12.1
Edit 4
So after playing around and googling for some time, it turns out that the issue is with Zend Framework and fastcgi which automatically removes the Authorization header. The weird thing is that it's only doing it from IOS and not from Android, which makes no sense really.
On thing we noticed in the logs, is that it's accepting the Android and Postman as POST but it's logging the IOS requests as GET. I'm not entirely sure what's up with that, but it seems to be another difference. I've updated the task to have zend as a tag. There are a number of SO articles on resolving this with ReWriteMod on apache/zend so I'll give those a go first and see if it fixes the issue.
** Edit 5**
So far we've attempted to follow the SO articles which ask that that that following be added (Authorization header missing in django rest_framework, is apache to blame?):
SetEnvIfNoCase Authorization ^(.*) -e=PHP_HTTP_AUTH
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule .* - [e=HTTP_AUTHORIZATION:%1]
which results in the following:
// IOS
_SERVER[PHP_HTTP_AUTH] = <blank>
_SERVER[HTTP_AUTHORIZATION] = <blank>
// Android
_SERVER[PHP_HTTP_AUTH] = Username
_SERVER[HTTP_AUTHORIZATION] = Basic <Base65 encoded>
_SERVER[PHP_HTTP_PW] = Password
So we know that Header Authorization is getting to Apache, but now it's coming through as blank. There are a few other SO answers I'm researching but the search continues...
Edit 6
Resolved(ish)
Turns out it was a trailing slash required on the request for IOS. I was able to find this link https://github.com/square/retrofit/issues/1037 where the the issue was described as:
For those interested: We are using Django as our backend and by default when you do
not provide a trailing slash on the endpoint Django redirects from the non-slash
endpoint to the slash endpoint.
Now, we aren't using Django, but apparently for our configuration of Zend it was
the same issue - Android was able to re-direct without issue, while IOS was not. Another comment on the task states:
OkHttp strips the "Authorization" header when redirected across hosts (connections)
via a 3xx response from the original host.
Which doesn't seem accurate, since Android was using OkHttp and was working fine. It looked like IOS using Darwin had the issue.
EDIT
I forgot something else from my original post, I also had to change my interceptor from the line config.headers.common.Authorization = ... to config.headers.Authorization = ... which for some reason kept the casing. Original way converted Authorization to authorization, while the latter kept it as Authorization. Not sure if this was an issue, but I made it anyhow.
// On login success, set the authInterceptor responsible for adding headers
authInterceptor = ServiceApi.interceptors.request.use((config) => {
console.log(`Attaching Authorization to header ${basicAuth}`);
config.headers.Authorization = basicAuth;
return config;
}, (error) => {
Promise.reject(error);
});
I cannot believe I spent 5 hours debugging and researching to resolve the issue eventually with a trailing slash! Even when I tried the trailing slash I thought it was a futile attempt but it actually resolved my issue. #kendavidson you're a lifesaver!!
There's a lot to unpack here. First of all, I've edited the title because I realize while eventually my REST request will be implemented into PHP code, right now I've stripped this down to Postman to test JUST the REST, so I've stripped it as low and basic as possible. I can officially say the problem is with my request.
Basically, I'm making a POST request and also testing with a PUT request to Walmart's API using the "new" OAuth authentication. Sounds grand. GET works BEAUTIFULLY in Postman and in my actual PHP code. POST and PUT immediately return the exact same error, no matter what and how I do: 400 Bad Request, Invalid URL. In the case of my PUT test, which I was doing because it's a simpler and faster text with far less XML to try to comb through, here's the exact response in HTML headers:
<HTML>
<HEAD>
<TITLE>Invalid URL</TITLE>
</HEAD>
<BODY>
<H1>Invalid URL</H1>
The requested URL "http://%5bNo%20Host%5d/v3/inventory?", is invalid.
<p>
Reference #9.c9384317.1556319123.8c89b8dc
</BODY>
</HTML>
I have left testing in PHP through my server and moved into Postman to try to locate the exact issue I'm having, and GET requests work beautifully. I am generating a new Token every 15 minutes or so. I have done... SO many minor changes, but the way the Feed examples and requests work, for all that I can tell I'm doing everything right. I honestly think I'm losing my marbles at this point.
What is most frustrating to me is that GET works. My TOKEN is working. My OAuth is working just fine. A lot of the headers that GET uses for the Walmart API are the exact same between PUT/POST/GET. The difference here is ONLY that the link has query parameters AND XML being shoved into the body. Edit: What I mean is that my headers do not change between the GET and the POST; the only thing that changes in what I am supplying is that XML is being sent in the body, and that query params are required. This is the only thing that changes between a successful GET and an unsuccessful 400 bad request PUT/POST. This leads me to believe something is wrong with how I'm processing the query params or my XML, but considering in the below example I've copy/pasted the XML... I'm not sure. It is an existing item in our catalog, I know for a fact.
Something I have noticed that I'm not quite knowledgeable enough to know if it's an issue or not with Postman is that Walmart's API requests that content-type be multipart/form-data. I've noticed it uses the term "example" when stating this, however, it usually says "this or this" if it'll accept something else. If I switch content-type in Postman to multipart/form-data, however, the Body automatically becomes raw: text instead of raw: XML(application/xml) or text/xml. If I try to swap the raw to those types, it flips my content-type automatically to application/xml, so that's a little... hinky.
I am not going through a Proxy. I've turned off Global Proxy Configuration and Use System Proxy. Request timeout is set to 0. There's nothing Client Certificates. I mean, GET works, and my Token is successfully generated via outside PHP code (not in Postman, couldn't get that to work, said heck it).
HEADERS
PUT URL: https://marketplace.walmartapis.com/v3/inventory?sku=0xyz0
AUTHORIZATION
Bearer Token: Bearer Basic --insert token here--
WM_SVC.NAME: Walmart Marketplace
WM_QOS.CORRELATION_ID: randomString123
WM_SEC.ACCESS_TOKEN: --insert token here--
Accept: application/xml
Host: https://marketplace.walmartapis.com
Content-type: multipart/form-data
BODY
raw: XML(application/xml)
<?xml version="1.0" encoding="UTF-8"?>
<inventory xmlns="http://walmart.com/">
<sku>0xyz0</sku>
<quantity>
<unit>EACH</unit>
<amount>7</amount>
</quantity>
<fulfillmentLagTime>1</fulfillmentLagTime>
</inventory>
Exact response
400 Bad Request
<HTML>
<HEAD>
<TITLE>Invalid URL</TITLE>
</HEAD>
<BODY>
<H1>Invalid URL</H1>
The requested URL "http://%5bNo%20Host%5d/v3/inventory?", is invalid.
<p>
Reference #9.c9384317.1556320429.8ca752c4
</BODY>
</HTML>
Please send help, I think I've been staring at this so long I'm going to leave this physical world behind. Walmart relatively recently updated their authentication to OAuth and they've made vague passes at saying their old authentication will be deprecated and phased out, so I obviously want to try to get this to work.I tried to copy paste everything as best as possible. That XML is copy-pasted almost letter for letter from their example, with my own product switched in.
Also, the reference number down there always changes every time I run this, so it's not something I can actually look up. I've only supplied the Postman side of things because frankly if I can get that to work, my PHP will be fine, I've already knocked out some minor issues with the successful GET request.
If it's a semi-colon issue, I'll scream.
API Documentation: https://developer.walmart.com/#/apicenter/marketPlace/latest#updateInventoryForAnItem
Well, I've figured it out.
You'll notice I'm required to supply a "Host" with my headers. That host is replacing my URl that I'm trying to connect to via POST/PUT/GET, so if my Host is https://marketplace.walmartapis.com, then my request URL is https://https://marketplace.walmartapis.com.
Once I took the https:// out of the host, the entire thing granted me a 200 response. The times I got a correct GET response, I had actually copy-pasted the correct HOST without the HTTPS by pure chance, so I completely missed this between my two separate test cases.
I'm trying to create a web hook notification. The documentation of the service i want to use requires that i specify a URL where POST requests can be performed. This URL will receive the following object, in json format, and must respond with a Status Code between 200-299.
{
"type": "ping"
}
I don't know how to proceed making my server on localhost respond with a 200 status code. http_response_code(200) works well on live server but nothing seem to be happening on localhost.
Is there any way i can make it work with localhost?
I've included the link to the documentation here (i hope it's not against the rule).
I am thinking that you wouldn't have to send them the response. The webhook would know about the response. If it reached your URL successfully, it would be a 200 OK right off the bat. If the API is requesting a response back then I imagine that you would have to call it back somehow. Is this a well-known API? Any documentation?
The response code is in the response header, not in the content.
PHP defaults to a response code of 200, so if you don't mess with it at all, you should be good.
If you want to set a different response code (202 for example), just call:
http_response_code(202);
Or set the full header yourself:
header('HTTP/1.1 202 Accepted');
Proper way to explicitly set 200 (or any other) status code with http_response_code function is just as following (don't echo or json_encode it):
http_response_code(200);
It should force webserver to use 200 status code in it's response. However, webserver could possibly ignore it. To check what response code your webserver sends, use telnet or any REST tool like Postman
I am using the ZenDesk API (https://developer.zendesk.com/rest_api/docs/core/introduction) to synchronise a ZenDesk setup with another client database. When I try to delete an organization, I get a response that seems to suggest an update call has been made.
According to the documentation (https://developer.zendesk.com/rest_api/docs/core/organizations#delete-organization) the call should be DELETE /api/v2/organizations/{id}.json where the {id} is the id of the organization.
I have written code that I believe to be correct, and checked this with Fiddler. The call comes through on Fiddler as:
DELETE /api/v2/organizations/39005971.json HTTP/1.1
The raw request view shows (with redactions):
DELETE https://<redacted>.zendesk.com/api/v2/organizations/39005971.json HTTP/1.1
Authorization: Basic <redacted>
Host: <redacted>.zendesk.com
Accept: */*
Content-Type: application/json
and the response comes back as:
{
"error":"RecordInvalid",
"description":"Record validation errors",
"details":{
"name":[
{
"description":"Name: has already been taken",
"error":"DuplicateValue"
}
]
}
}
This is the same response that is given if you try to insert an organization with the same name as an existing one. From the documentation, the basic difference between deleting and updating a record is that delete requests use DELETE and updates use PUT - the endpoint URL is the same.
Does anyone have any suggestions? I can provide upstream code (in PHP) if needed, however as Fiddler is picking up the request as a correctly formatted DELETE, I'm not sure that the code is going to help.
I actually work for Zendesk and figured this out personally. You seemed to have run into a bug having to do with the max characters an organization name can have. You probably had a couple organizations whose names were more than 255 characters long and after getting truncated to 255 were the same name. Now validation issues are popping up. I'm really sorry about that!
I would send a request to https://support.zendesk.com/hc/en-us/requests/new and we'll fix this issue for you!