I've been banging my head against this problem for nearly two days now, and I'm hoping someone on this site can help me.
I live in China and have a server (shared hosting) located in Hong Kong. I've set up a PHP Twitter proxy on the server, and I connect to the proxy using Twitter for iPhone (AKA Tweetie). It's worked beautifully for the past year or so.
Twitter for iPhone was updated yesterday and now requires XAuth authorization to connect to twitter.com (previously, it used Basic Auth). Since the update, I haven't been able to authenticate myself using my proxy, in spite of making (what I believe to be) the appropriate changes to my proxy.
Conceptually, this isn't a very difficult problem to crack. In order to authenticate with twitter.com using XAuth, an app must send a POST request to https://api.twitter.com/oauth/access_token. The POST body must be of the form:
x_auth_username=aUserName&x_auth_mode=client_auth&x_auth_password=aPassword
Additionally, the POST request must have an Authorization header in the form:
OAuth oauth_signature_method="HMAC-SHA1", oauth_consumer_key="IQKbtAYlXsomeLGkey0HUA", oauth_nonce="8B265865-3F57-44FF-BCD6-E009EA7D4615", oauth_signature="sbwblaho64blahr934mZQ+23DYQ=", oauth_timestamp="1277356846", oauth_version="1.0"
So, what I've done is used .htaccess to copy the Auth header to a $_REQUEST variable using this code:
RewriteCond %{HTTP:Authorization} ^OAuth.*
RewriteRule (.*) index.php?OAuth=%{HTTP:Authorization} [QSA,L]
My proxy copies the contents of that $_REQUEST variable to an instance variable called $self->oauthHeader. Then, I make add it as a header to my cURL request using the following code:
if (isset($this->oauthHeader)) {
$headers[] = 'Authorization: '.$this->oauthHeader;
$headers[] = 'Content-Type: application/x-www-form-urlencoded';
$curl_options[CURLOPT_HTTPHEADER] = $headers;
}
I also add the original request's POST body to the cURL's POST body using:
$curl_options[CURLOPT_POST] = true;
$curl_options[CURLOPT_POSTFIELDS] = #file_get_contents("php://input");
I send the cURL request to Twitter. Everything seems to work correctly, but I inevitably receive a response of "Failed to validate oauth signature and token."
I'm at my wit's end, and I can't for the life of me think what I'm doing wrong. Any help would be much appreciated.
You can use this alternative solution, for jailbroken iOS devices only: http://code.google.com/p/gfwinterceptor/
Related
I'm posting this on my way home, so forgive the lack of code but I'll try to be as detailed as possible and add code when I can tonight. So essentially I have a react native app using redux and axios. A brief review (code to follow) may explain that I'm doing something wrong.
Serviceapi.js
Creates and exports basic axios with base url.
const ServiceApi = axios.create({
baseURL: BASE_URL,
responseType: 'json'
});
AuthReducer.js
On login sets Authorization header manually using the post method. This works on both android and ios the login is returned and I use the authorization header.
return {
type: PERFORM_LOGIN,
payload: {
user: {
name: username
},
request: {
url: '/login',
method: 'post',
headers: {
'Authorization': 'Basic ' + basicAuth
}
}
}
On login, I return the following redux-axios action, you can see that I set the header: Authorization manually, this works great.
// On login success, set the authInterceptor responsible for adding headers
authInterceptor = ServiceApi.interceptors.request.use((config) => {
console.log(`Attaching Authorization to header ${basicAuth}`);
config.headers.common.Authorization = basicAuth;
return config;
}, (error) => {
Promise.reject(error);
});
On logout I clear the interceptor. I chose to add and remove on login and logout instead of always having it there just because. This could be a problem but it was fine for Android
// Clear the auth interceptor
ServiceApi.interceptors.request.eject(authInterceptor);
Again this is all working great on Android. And it looks to be working on ios. When I debug the interceptor it's getting called and setting the header.
But I get back a 403 on ios. After looking at the request in more detail, there is a big difference between the android header in the request and the ios header in the request. The rest of the request object is the same, only the _header object is different between ios and android.
Android Request
_headers:
accept: "application/json, text/plain, */*"
authorization: "Basic <correct base64 value>"
content-type: "application/json;charset=utf-8"
__proto__: Object
IOS Request
_headers:
accept: (...)
authorization: (...)
content-type: (...)
get accept: ƒ ()
set accept: ƒ ()
get authorization: ƒ ()
set authorization: ƒ ()
get content-type: ƒ ()
set content-type: ƒ ()
__proto__: Object
With the differences, setting a breakpoint at looking at the console for error.request._headers.authorization; I get the same "Basic: " contents as the Android header contains.
index.php
The backend service is a php file that does a $_SERVER['PHP_AUTH_USER'] which fails a 403 if not set which is what's happening. I don't have access to the php, I was just told this is what it's using.
Again I apologize for not providing code but i will when I get a chance later. Is there something maybe I have to set extra for ios? Or maybe php for ios needs an extra header?
Code to follow.
EDIT Updated with code, hopefully I didn't leave in any of the encoded login info.
EDIT 2 Upon further investigation this looks like it's related to apache/PHP rather than react-native/axios. I threw together an express server that simulated the same checking that the PHP does:
- Look for the Authorization header
- Print it
- Return back 403 or 200 w/ data based on that
When running pointing at http://localhost:3000 using the exact same app on the emulator I get back what I'm expecting. To add to this, when I'm on the emulator, I can't actually login to the live URL (even though I could on the regular device), I get the same 403 error but this time a little earlier.
EDIT 3
To provide some more information from the server, here are the three requests that I've been able to log:
1) This is from the IOS Emulator iPhone8 against a an express server:
accept:"application/json, text/plain, */*"
accept-encoding:"gzip, deflate"
accept-language:"en-us"
authorization:"Basic <base 64 encoding>"
connection:"keep-alive"
content-length:"0"
host:"localhost:3000"
user-agent:"MobileApp/1 CFNetwork/978.0.7 Darwin/18.5.
2) This is from the same emulator to apache/PHP (5.3.3), we can see there is no Authorization header.
Accept: application/json, text/plain, */*
User-Agent: MobileApp/1 CFNetwork/978.0.7 Darwin/18.5.0
Accept-Language: en-us
Accept-Encoding: br, gzip, deflate
Connection: keep-alive
3) This is from Android to apache/PHP (5.3.3):
authorization: Basic <Base 64 encoding>
Host: api.serviceurl.com
Connection: Keep-Alive
Accept-Encoding: gzip
User-Agent: okhttp/3.12.1
Edit 4
So after playing around and googling for some time, it turns out that the issue is with Zend Framework and fastcgi which automatically removes the Authorization header. The weird thing is that it's only doing it from IOS and not from Android, which makes no sense really.
On thing we noticed in the logs, is that it's accepting the Android and Postman as POST but it's logging the IOS requests as GET. I'm not entirely sure what's up with that, but it seems to be another difference. I've updated the task to have zend as a tag. There are a number of SO articles on resolving this with ReWriteMod on apache/zend so I'll give those a go first and see if it fixes the issue.
** Edit 5**
So far we've attempted to follow the SO articles which ask that that that following be added (Authorization header missing in django rest_framework, is apache to blame?):
SetEnvIfNoCase Authorization ^(.*) -e=PHP_HTTP_AUTH
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule .* - [e=HTTP_AUTHORIZATION:%1]
which results in the following:
// IOS
_SERVER[PHP_HTTP_AUTH] = <blank>
_SERVER[HTTP_AUTHORIZATION] = <blank>
// Android
_SERVER[PHP_HTTP_AUTH] = Username
_SERVER[HTTP_AUTHORIZATION] = Basic <Base65 encoded>
_SERVER[PHP_HTTP_PW] = Password
So we know that Header Authorization is getting to Apache, but now it's coming through as blank. There are a few other SO answers I'm researching but the search continues...
Edit 6
Resolved(ish)
Turns out it was a trailing slash required on the request for IOS. I was able to find this link https://github.com/square/retrofit/issues/1037 where the the issue was described as:
For those interested: We are using Django as our backend and by default when you do
not provide a trailing slash on the endpoint Django redirects from the non-slash
endpoint to the slash endpoint.
Now, we aren't using Django, but apparently for our configuration of Zend it was
the same issue - Android was able to re-direct without issue, while IOS was not. Another comment on the task states:
OkHttp strips the "Authorization" header when redirected across hosts (connections)
via a 3xx response from the original host.
Which doesn't seem accurate, since Android was using OkHttp and was working fine. It looked like IOS using Darwin had the issue.
EDIT
I forgot something else from my original post, I also had to change my interceptor from the line config.headers.common.Authorization = ... to config.headers.Authorization = ... which for some reason kept the casing. Original way converted Authorization to authorization, while the latter kept it as Authorization. Not sure if this was an issue, but I made it anyhow.
// On login success, set the authInterceptor responsible for adding headers
authInterceptor = ServiceApi.interceptors.request.use((config) => {
console.log(`Attaching Authorization to header ${basicAuth}`);
config.headers.Authorization = basicAuth;
return config;
}, (error) => {
Promise.reject(error);
});
I cannot believe I spent 5 hours debugging and researching to resolve the issue eventually with a trailing slash! Even when I tried the trailing slash I thought it was a futile attempt but it actually resolved my issue. #kendavidson you're a lifesaver!!
There's a lot to unpack here. First of all, I've edited the title because I realize while eventually my REST request will be implemented into PHP code, right now I've stripped this down to Postman to test JUST the REST, so I've stripped it as low and basic as possible. I can officially say the problem is with my request.
Basically, I'm making a POST request and also testing with a PUT request to Walmart's API using the "new" OAuth authentication. Sounds grand. GET works BEAUTIFULLY in Postman and in my actual PHP code. POST and PUT immediately return the exact same error, no matter what and how I do: 400 Bad Request, Invalid URL. In the case of my PUT test, which I was doing because it's a simpler and faster text with far less XML to try to comb through, here's the exact response in HTML headers:
<HTML>
<HEAD>
<TITLE>Invalid URL</TITLE>
</HEAD>
<BODY>
<H1>Invalid URL</H1>
The requested URL "http://%5bNo%20Host%5d/v3/inventory?", is invalid.
<p>
Reference #9.c9384317.1556319123.8c89b8dc
</BODY>
</HTML>
I have left testing in PHP through my server and moved into Postman to try to locate the exact issue I'm having, and GET requests work beautifully. I am generating a new Token every 15 minutes or so. I have done... SO many minor changes, but the way the Feed examples and requests work, for all that I can tell I'm doing everything right. I honestly think I'm losing my marbles at this point.
What is most frustrating to me is that GET works. My TOKEN is working. My OAuth is working just fine. A lot of the headers that GET uses for the Walmart API are the exact same between PUT/POST/GET. The difference here is ONLY that the link has query parameters AND XML being shoved into the body. Edit: What I mean is that my headers do not change between the GET and the POST; the only thing that changes in what I am supplying is that XML is being sent in the body, and that query params are required. This is the only thing that changes between a successful GET and an unsuccessful 400 bad request PUT/POST. This leads me to believe something is wrong with how I'm processing the query params or my XML, but considering in the below example I've copy/pasted the XML... I'm not sure. It is an existing item in our catalog, I know for a fact.
Something I have noticed that I'm not quite knowledgeable enough to know if it's an issue or not with Postman is that Walmart's API requests that content-type be multipart/form-data. I've noticed it uses the term "example" when stating this, however, it usually says "this or this" if it'll accept something else. If I switch content-type in Postman to multipart/form-data, however, the Body automatically becomes raw: text instead of raw: XML(application/xml) or text/xml. If I try to swap the raw to those types, it flips my content-type automatically to application/xml, so that's a little... hinky.
I am not going through a Proxy. I've turned off Global Proxy Configuration and Use System Proxy. Request timeout is set to 0. There's nothing Client Certificates. I mean, GET works, and my Token is successfully generated via outside PHP code (not in Postman, couldn't get that to work, said heck it).
HEADERS
PUT URL: https://marketplace.walmartapis.com/v3/inventory?sku=0xyz0
AUTHORIZATION
Bearer Token: Bearer Basic --insert token here--
WM_SVC.NAME: Walmart Marketplace
WM_QOS.CORRELATION_ID: randomString123
WM_SEC.ACCESS_TOKEN: --insert token here--
Accept: application/xml
Host: https://marketplace.walmartapis.com
Content-type: multipart/form-data
BODY
raw: XML(application/xml)
<?xml version="1.0" encoding="UTF-8"?>
<inventory xmlns="http://walmart.com/">
<sku>0xyz0</sku>
<quantity>
<unit>EACH</unit>
<amount>7</amount>
</quantity>
<fulfillmentLagTime>1</fulfillmentLagTime>
</inventory>
Exact response
400 Bad Request
<HTML>
<HEAD>
<TITLE>Invalid URL</TITLE>
</HEAD>
<BODY>
<H1>Invalid URL</H1>
The requested URL "http://%5bNo%20Host%5d/v3/inventory?", is invalid.
<p>
Reference #9.c9384317.1556320429.8ca752c4
</BODY>
</HTML>
Please send help, I think I've been staring at this so long I'm going to leave this physical world behind. Walmart relatively recently updated their authentication to OAuth and they've made vague passes at saying their old authentication will be deprecated and phased out, so I obviously want to try to get this to work.I tried to copy paste everything as best as possible. That XML is copy-pasted almost letter for letter from their example, with my own product switched in.
Also, the reference number down there always changes every time I run this, so it's not something I can actually look up. I've only supplied the Postman side of things because frankly if I can get that to work, my PHP will be fine, I've already knocked out some minor issues with the successful GET request.
If it's a semi-colon issue, I'll scream.
API Documentation: https://developer.walmart.com/#/apicenter/marketPlace/latest#updateInventoryForAnItem
Well, I've figured it out.
You'll notice I'm required to supply a "Host" with my headers. That host is replacing my URl that I'm trying to connect to via POST/PUT/GET, so if my Host is https://marketplace.walmartapis.com, then my request URL is https://https://marketplace.walmartapis.com.
Once I took the https:// out of the host, the entire thing granted me a 200 response. The times I got a correct GET response, I had actually copy-pasted the correct HOST without the HTTPS by pure chance, so I completely missed this between my two separate test cases.
I'm making a request to retrieve a JSON file to a server at a particular secure DocuSign uri. However, unless I put in the authorization information (which I do have), I am unable to have the file returned.
<?php
$json = file_get_contents("https://example.docusign.com/sensitiveIDs/moreID");
echo $json
?>
Where would I put in authorization information for the specific server/username/password/other info needed to access the particular DocuSign server using a method like this in PHP? Is there a better method to use for this scenario in PHP?
It depends on how the authorization is implemented. If its basic or digest HTTP authentication then specify it in the URL:
file_get_contents("https://$USER:$PASSWORD#example.docusign.com/sensitiveIDs/moreID");
Cookie based authentication is a lot more difficult (and probably easier to use Curl or even a more complex system like Guzzle. If its oauth2, then you probably want an oauth2 library.
Your call needs to include authentication to make the GET call to retrieve the file.
If your app is initiated by a human use Oauth to retrieve access and refresh tokens. Then included the access token with the GET request.
If your app is a "system app" that wants to autonomously retrieve the file, then you should authenticate by using X-DocuSign-Authentication -- include the following header in your HTTPS request. Since the request is HTTPS, the content is encrypted on the wire:
X-DocuSign-Authentication: <DocuSignCredentials><Username>{name}</Username><Password>{password}</Password><IntegratorKey>{integrator_key}</IntegratorKey></DocuSignCredentials>
Replace {name} with your email address (no braces), etc.
The bottom line is that you can't use the file_get_contents Php method. Instead, you'd do something like the following:
Use https://github.com/rmccue/Requests or a similar library to help with the https request. (http is not allowed due to security issues.)
(untested code)
$url = $base_url . $the_url_section_for_this_call
$headers = array('X-DocuSign-Authentication' =>
'<DocuSignCredentials><Username>your_name</Username><Password>your_password</Password><IntegratorKey>your_integrator_key</IntegratorKey></DocuSignCredentials>');
$request = Requests::get($url, $headers);
# Check that the call succeeded (either 200 or 201 depending on the method
$status_code = $request->status_code;
if ($status_code != 200 && $status_code != 201) {
throw new Exception('Problem while calling DocuSign');
}
$json = $request->body;
The quick question is: is there ane way to change http authorisation header with html / php / javascript?
The goal
I'd like to make an auth service used for user login as well as providing with whole site protection. I want user to be restricted from viewing any file except login page unless authorised. For php I can of course check at the beginning for example session token availability, and redirect if missing, but I can't do it directly for for example jpg images. I thought of creating htaccess file that will verify if user is logged.
The solution
1.
Using Apache I've created .htaccess file, that verifies, if HTTP authorisation is set. If not it redirects any request to login page. With that solution one can not open any file (no matter if it is php script or for example jpg image) except of the login page:
RewriteCond %{HTTP:Authorization} ^$
RewriteCond %{REQUEST_URI} !^/login.php
RewriteRule .* /login.php [L,R]
2.
The login page should display form and when login and password are correct set the http request auth header with proper token.
Unfortunately I can't find a way to create manually http request auth header. The only way I found was to use basic auth:
$auth = new Auth();
if (!empty($_SERVER['PHP_AUTH_USER']) && !empty($_SESSION)) {
$auth->logout();
}
if (!empty($_SERVER['PHP_AUTH_USER']) && empty($_SESSION)) {
$login = $_SERVER['PHP_AUTH_USER'];
$haslo = (!empty($_SERVER['PHP_AUTH_PW']) ? Auth::hashPassword($_SERVER['PHP_AUTH_PW']) : null);
$auth->login($login, $haslo);
}
$auth->getLoginForm();
Where getLoginForm() displays Basic Auth standard form
public function getLoginForm()
{
header(self::HEADER_ERROR_ASK_CLIENT_DATA);
header(self::HEADER_RESPONSE_401);
exit;
}
This solution works, but http request auth header holds the original login data all the time, which is what I want to avoid. I want to inject there "Bearer + token" string, which will help in securing whole system.
What I've tried
I can easily do that with external frontend. For example with windows desktop app I can send one request with basic auth header and the next with bearer header.
The same I can do with JQuery and AJAX - I can retrieve token using url with basic auth, and then use token with custom header for next requests.
I can also make different requests with PHP CURL.
But I can't find a way to force regular browser to login with Basic auth header and after success keep new, custom bearer header for next requests.
There is no standard mechanism to get a User Agent (browser) to provide an Authorization: Bearer header by itself.
What you can do:
make sure all your requests are done via XHR (Ajax), and send the header there. But this will not allow "regular" file loads (new pages, images, scripts, CSS...).
what most people do: send a cookie with your authentication token, and check for that cookie. The browser will automatically send the cookie.
As an aside, note that your .htaccess only checks that there is an Authorization header present, not its value, so it's quite useless. You want to channel all reads via a script that will actually verify the header or cookie before delivering the file.
I have an interesting situation when calling the Shopify API. I use the standard procedure for calling the url and get the data, like this:
define('SHOPIFY_SHOP', 'myteststore.myshopify.com');
define('SHOPIFY_APP_API_KEY', 'xxxx');
define('SHOPIFY_APP_PASSWORD', 'yyy');
$shop_url = 'https://'.SHOPIFY_APP_API_KEY.':'.SHOPIFY_APP_PASSWORD.'#'.SHOPIFY_SHOP;
$response = Requests::get($shop_url.'/admin/products.json');
And I correctly get the response, parse the data and all works great. Now, when I put it to the actual server (Ubuntu 12.04), I noticed a weird message from the Spotify API:
[API] Invalid API key or access token (unrecognized login or wrong password)
I tried creating a new app, but still its the same. So the same file and the same set works on my machine, but not on the server. (only difference in the file is the path to requests library, require_once './Requests/library/Requests.php'; for Linux and require_once '..\Requests\library\Requests.php'; for Windows) As stated, I use the requests library and I assume there has to be some trick where the library (or something else) rewrites the URl and it doesn't get to Shopify correctly.
I tried using CURL with the URL directly, and it works that way as well. Can anyone point me what might be causing this?
Update: I moved to another library which solved the issue, but would like to know what was causing this since I had great experience with Requests up to this point.
I'm starting to use the same lib, and I stumbled upon something relevant right after finding this question:
https://github.com/rmccue/Requests/issues/142#issuecomment-147276906
Quoting relevant part:
This is an intentional part of the API design; in a typical use case,
you won't necessarily need data sent along with a request. Building
the URL for you is just a convenience.
Requests::get is a helper function designed to make GET requests
lightweight in the code, which is why there's no $data parameter
there. If you need to send data, use Requests::request instead
$response = Requests::request( 'http://httpbin.org/get', $headers, $data, Requests::GET, $options );
// GET is the default for type, and $options can be blank, so this can be shortened:
$response = Requests::request( 'http://httpbin.org/get', $headers, $data );
I couldn't figure why is this happening, it appears the Requests library is stripping the parameters from GET requests, so I moved to unirest library and this solved the issue.