I use on my server a Text-to-Speech Synthesis platform (probably written in Java).
While the above application is running on my server, users can get audio as a URL to a wav file using the embedded HTML <audio> tag, as follows:
<audio controls>
<source src=”http://myserver.com:59125/process?INPUT_TEXT=Hello%20world” type=”audio/wav”>
</audio>
In the above ‘src’ attribute, ‘process’ requests the synthesis of some text using local port 59125.
My concern is that I might start seeing performance issues and out of memory errors, which would cause the TTS Synthesis platform server (but not the website) to crash every few days, apparently triggered by one or more entities abusing it as some sort of webservice for their own applications.
I wish to secure the URL requests so that a third party couldn't use my text-to-speech server for audio clips not related to my website.
How to secure the URL service?
I take it this URL is embedded in a public website, so any random public user needs to be able to access this URL to download the file. This makes it virtually impossible to secure as is.
The biggest problem is that you're publicly exposing a useful service which is usable for anyone to do something useful. I.e., just by requesting a URL which I construct, I can get your server to do useful work for me (turn my text into speech). The core problem here is that the input text is fully configurable by the end user.
To take away any incentive for any random person to use your server, you need to take away the ability for anyone to convert any random text. If you are the only one who wants to be in charge of what input texts are allowed, you'll have to either whitelist and validate the input, or identify it using ids. E.g., instead of
http://myserver.com:59125/process?INPUT_TEXT=Hello%20world
your URLs look more like:
http://myserver.com:59125/process?input_id=42
42 is substituted to Hello world on the server. Unknown ids won't be served.
Alternatively, again, validate and whitelist:
GET http://myserver.com:59125/process?INPUT_TEXT=Foo%20bar
404 Not Found
Speech for "Foo bar" does not exist.
For either approach, you'll need some sort of proxy in-between instead of directly exposing your TTS engine to the world. This proxy can also cache the resulting file to avoid repeatedly converting the same input again and again.
The end result would work like this:
GET http://myserver.com/tts?input=Hello%20world
myserver.com validates input, returns 403 or 404 for invalid input
myserver.com proxies a request to localhost:59125?INPUT_TEXT=Hello%20World if not already cached
myserver.com caches the result
myserver.com serves the result
This can be accomplished in any number of ways using any number of different web servers and/or CGI programs which do the necessary steps 2 and possibly 3.
This depends on what server you are using. Possible methods are:
Authentication: Use a username and password combination or ask for a SSH certificate; this could be provided via cURL when one webservice requests another one
IP whitelist: allow only specific IP's to access this server
IP whitelist example in Apache:
Deny from all
# server himself
Allow from 127.0.0.1
Allow from 192.168.1.14 # maybe some additional internal network IP
Allow from 192.168.1.36 # or another machine in the local network
Allow from 93.184.216.34 # or some machine somewhere else on the web
your best bet is using the answer above from feeela to limit the usage of the TTS platform to a said webserver (this will be where the users request the audio from and where your security logic should be implemented)
after that you need to write a "proxy" script that gets a token generated on-the-fly from the page that hosts the audio tag with a logic/method of your choice and check its validity (you can use the session/other user data and a salt), if valid it should call the TTS engine and return the audio, otherwise generate an error/a redirect/whatever you want
It depends what you mean by "securing it".
Maybe you want it to only be accessible to certain users? In that case, you have an easy answer: issue each user with login credentials that they need to enter when they visit the site, and pass those credentials through to the API. Anyone without valid credentials will be unable to use the API. Job done.
Or maybe you want it to work for anyone, but only to be used from specific sites? This is more difficult, because any kind of authentication key you have would need to be within the site's Javascript code, and thus visible to someone wanting to copy it. There isn't a foolproof solution, but the best solution I can suggest is to link each API key to the URL of the site that owns it. Then use the HTTP referrer header to check that calls made using a given API key are being called from the correct site. HTTP requests can be spoofed, including the referrer header, so this isn't foolproof, but will prevent most unauthorised use -- someone would have to go a fair distance out of their way to get around it (they'd probably have to set up a proxy server that forwarded your API requests and spoofed the headers). This is unlikely to happen unless your API is an incredibly valuable asset, but if you are worried about that, then you could make it harder for them by having the API keys change frequently and randomly.
But whatever else you do, the very first thing you need to do to secure it is to switch to HTTPS rather than HTTP.
Related
I have a webserver, and certain users have been retrieving my images using an automated script.I wish to redirect them to a error page or give them an invalid image only if it's a CURL request.
my image resides in http://example.com/images/AIDd232320233.png, is there someway I can route it with .htaccess to my controller index function to where I can check if it's an authentic request?
and my other question, how can I check browser headers to distinguish between most likely authentic ones and ones done with a cURL request?
Unfortunately, the short answer is 'no.'
cURL provides all of the necessary options to "spoof" any browser. That is to say, more specifically, browsers identify themselves via specific header information, and cURL provides all of the tools to set header data in whatever manner you choose. So, directly distinguishing two requests from one another is not possible.*
*Without more information. Common methods to determine if there is a Live Human initiating the traffic are to set cookies during previous steps (attempts to ensure that the request is a natural byproduct of a user being on your website), or using a Captcha and a cookie (validate someone can pass a test).
The simplest is to set a cookie, which will really only ensure that bad programmers don't get through, or programmers who don't want to spend the time to tailor their scraper to your site.
The more tried and true approach is a Captcha, as it requires the user to interact to prove they have blood in their veins.
If the image is not a "download" but more of a piece of a greater whole (say, just an image on your site), a Captcha could be used to validate a human before giving them access to the site as a whole. Or if it is a download, it would be presented before unlocking the download.
Unfortunately, Captchas are are "a pain," both to set up, and for the end-user. They don't make a whole lot of sense for general-purpose access, they are a little overboard.
For general-purpose stuff, you can really only throttle IPs, download limits and the like. And even there, you have nothing you can do if the requests are distributed. Them's the breaks, really...
We are developing in-house web-based application for viewing data reports while targeting on smartphones and tablets. Our customer asked us for possibility that only certain devices could access the content. Hence we use technologies based on javascript/HTML5 we are no capable of reading unique ID like IMEI or device uuid. On the other hand side we could use server technologies like ASP, PHP to gain success.
I have several ideas which dont lead to wanted result (one discussed here: Persistent client-side web storage).
I wonder if you have any idea that allow only certain devices to access web site?
Such access control would only be "secure" if a traditional login method is implemented on top of it, i.e. users (1) need to sign in with username and password, but (2) they can only do so on specific devices.
Step (1) is required to make access basically "secure", while step (2) would only make it just a little harder to break into your app for people who have hardly a clue what they're doing.
(Without the second step, people could attempt to brute force the login form when they know its URL, without sniffing any other network traffic.)
You could certainly fingerprint the user agent (UA) string and possibly other HTTP headers, assuming the mobile browser app isn't constantly updated and therefore doesn't constantly change its UA string (that could be a hassle), and check server-sided.
Your could also create a simple, really simple native mobile app for the target platform(s), consisting only of the platform's default web browser widget, with your app's URL built-in as the default page.
You could then control the URLs and possibly HTTP headers, and add special, secret authentication headers or URL parameters (e.g. device's IMEI), for which you check on the server side.
If you target Android, you don't necessarily need to rely on Google Play; you can also distribute the APK files from one of your own servers, making the app available only to the intended audience.
AFAIK you only have the User Agent to work on, with maybe some Javascript values that you can return as are used when fingerprinting.
The User Agent should give you a lot to go on, but it can easily be spoofed. And so can the Javascript values.
I don't think there is a secure way to do what you want. But then again, I don't know if you really want it that secure.
What you also could do is to not do it 100% browser based, but create a mobile App. (Such as in Apple AppStore / Google Play Store) Here I think you can request access to more variables to identify the machine type.
Try the lightweight php-mobile-detect here: (server side checking is always better) https://code.google.com/p/php-mobile-detect/
I am writing a set of HTML based resources, stored in a mysql db on our server. The resources contain images references, which are stored as a relative paths.
I can login to an admin panel and create and edit resources very easily.
This bit is all done, and working well.
However, we are going to want to provide this resource in two ways: packaged and hosted. The one that is providing real problems is the hosted solution:
We want to host the database and all image resources, however we want to give access to the resources via a set of templates for other users to host on their own site. This is so they can brand the templates accordingly, and have the resource available at their own URL e.g.
http://www.example.com/discojoe
There are two questions I have on this process, which are causing me real headaches:
• I realise the obvious security implication of providing even read-only access to our mysql server. The only alternative I think of is some server side code running on our server, which when requested, squirts the requested data back to the user. This removes the need for them to have any mysql connection.
Are there any examples of the above type of scenario online I can read up on, or can you give some pointers on how I would go about addressing this solution?
• For users whom we stream the data to (if the above solution is sensible), how could I go about updating the image SRC value on the fly within the HTML content. Whilst I want it stored as a relative URL in the database, when I retrieve the data from the database, I want to convert all image srcs from relative to absolute, with the absolute URL that I specify.
I realise the obvious security
implication of providing even
read-only access to our mysql server.
The only alternative I think of is
some server side code running on our
server, which when requested, squirts
the requested data back to the user.
This removes the need for them to have
any mysql connection.
You could create an REST API(I would return JSON) using predefined queries with PDO prepared statements(safe against SQL-injections). With a little bit of care you could make it pretty safe. Ofcourse if the resources should be protected, you must also add authentication to your system using simple API keys for example. I think you could generate these key easily the same way you prevent CSRF($token = md5(uniqid(rand(), TRUE));). Maybe you should add a little bit more entropy, but I think this is going to be sufficient. But if you want to really do it correctly you should use oauth instead.
with a little bit of mod_rewriting you could write pretty URLs.
For users whom we stream the data to
(if the above solution is sensible),
how could I go about updating the
image SRC value on the fly within the
HTML content. Whilst I want it stored
as a relative URL in the database,
when I retrieve the data from the
database, I want to convert all image
srcs from relative to absolute, with
the absolute URL that I specify.
I think you could use any of the many available template languages to achieve this. Even jquery has one built-in
Create a REST-style web service. That is, set up an HTTP server that responds to data requests by using some server code to load up your templates, alter the URLs and other things (relative to absolute), and sends it to the client as fragments of HTML (or even CSS).
Your user, running on another web server, can use an HTTP client package to consume your web service, incorporate the resulting code fragments into her page, and send it out.
Alternatively you could build your code fragments so they function in iframe objects. In that case your user would build her code to deliver iframe objects to her end-users with references to your server in them.
Finally, your web service could deliver XML or JSON, and be consumed by AJAX-style javacscript in the end-user's browsers.
You are absolutely right to prevent direct access to your mySQL table server from random clients.
I have a script that uses JSONP to make cross domain ajax calls. This works great but my question is, is there a way to prevent other sites from accessing and getting data from these URL's? I basically would like to make a list of sites that are allowed and only return data if they are in the list. I am using PHP and figure I might be able to use "HTTP_REFERER" but have read that some browsers will not send this info.... ??? Any ideas?
Thanks!
There really is no effective solution. If your JSON is accessible through the browser, then it is equally accessible to other sites. To the web server a request originating from a browser or another server are virtually indistinguishable aside from the headers. Like ILMV commented, referrers (and other headers) can be falsified. They are after all, self-reported.
Security is never perfect. A sufficiently determined person can overcome any security measures in place, but the goal of security is to create a high enough deterrent that laypeople and or most people would be dissuaded from putting the time and resources necessary to compromise the security.
With that thought in mind, you can create a barrier of entry high enough that other sites would probably not bother making requests with the barriers of entry put into place. You can generate single use tokens that are required to grab the json data. Once a token is used to grab the json data, the token is then subsequently invalidated. In order to retrieve a token, the web page must be requested with a token embedded within the page in javascript that is then put into the ajax call for the json data. Combine this with time-expiring tokens, and sufficient obfuscation in the javascript and you've created a high enough barrier.
Just remember, this isn't impossible to circumvent. Another website could extract the token out of the javascript, and or intercept the ajax call and hijack the data at multiple points.
Do you have access to the servers/sites that you would like to give access to the JSONP?
What you could do, although not ideal is to add a record to a db of the IP on the page load that is allowed to view the JSONP, then on the jsonp load, check if that record exists. Perhaps have an expiry on the record if appropriate.
e.g.
http://mysite.com/some_page/ - user loads page, add their IP to the database of allowed users
http://anothersite.com/anotherpage - as above, add to database
load JSONP, check the IP exists in the database.
After one hour delete the record from the db, so another page load would be required for example
Although this could quite easily be worked around if the scraper (or other sites) managed to work out what method you are using to allow users to view the JSONP, they'd only have to hit the page first.
How about using a cookie that holds a token used with every jsonp request?
Depending on the setup you can also use a variable if you don't want to use cookies.
Working with importScript form the Web Worker is quite the same as jsonp.
Make a double check like theAlexPoon said. Main-script to web worker, web worker to sever and back with security query. If the web worker answer to the main script without to be asked or with the wrong token, its better to forward your website to the nirvana. If the server is asked with the wrong token don't answer. Cookies will not be send with an importScript request, because document is not available at web worker level. Always send security relevant cookies with a post request.
But there are still a lot of risks. The man in the middle knows how.
I'm certain you can do this with htaccess -
Ensure your headers are sending "HTTP_REFERER" - I don't know any browser that wont send it if you tell it to. (if you're still worried, fall back gracefully)
Then use htaccess to allow/deny access from the right referer.
# deny all except those indicated here
order deny,allow
deny from all
allow from .*domain\.com.*
I have a web application that pulls data from my newly created JSON API.
My static HTML pages dynamically calls the JSON API via JavaScript from the static HTML page.
How do I restrict access to my JSON API so that only I (my website) can call from it?
In case it helps, my API is something like: http://example.com/json/?var1=x&var2=y&var3=z... which generates the appropriate JSON based on the query.
I'm using PHP to generate my JSON results ... can restricting access to the JSON API be as simple as checking the $_SERVER['HTTP_REFERER'] to ensure that the API is only being called from my domain and not a remote user?
I think you might be misunderstanding the part where the JSON request is initiated from the user's browser rather than from your own server. The static HTML page is delivered to the user's browser, then it turns around and executes the Javascript code on the page. This code opens a new connection back to your server to obtain the JSON data. From your PHP script's point of view, the JSON request comes from somewhere in the outside world.
Given the above mechanism, there isn't much you can do to prevent anybody from calling the JSON API outside the context of your HTML page.
The usual method for restricting access to your domain is prepend the content with something that runs infinitely.
For example:
while(1);{"json": "here"} // google uses this method
for (;;);{"json": "here"} // facebook uses this method
So when you fetch this via XMLHttpRequest or any other method that is restricted solely to your domain, you know that you need to parse out the infinite loop. But if it is fetched via script node:
<script src="http://some.server/secret_api?..."></script>
It will fail because the script will never get beyond the first statement.
In my opinion, you can't restrict the access, only make it harder. It's a bit like access-restriction by obscurity. Referrers can be easily forged, and even with the short-lived key a script can get the responses by constantly refreshing the key.
So, what can we do?
Identify the weakness here:
http://www.example.com/json/getUserInfo.php?id=443
The attacker now can easily request all user info from 1 to 1.000.000 in a loop. The weak point of auto_increment IDs is their linearity and that they're easy to guess.
Solution: use non-numeric unique identifiers for your data.
http://www.example.com/json/getUserInfo.php?userid=XijjP4ow
You can't loop over those. True, you can still parse the HTML pages for keys for all kinds of keys, but this type of attack is different (and more easily avoidable) problem.
Downside: of course you can't use this method to restrict queries that aren't key-dependent, e.g. search.
Any solution here is going to be imperfect if your static pages that use the API need to be on the public Internet. Since you need to be able to have the client's browser send the request and have it be honored, it's possibly for just about anyone to see exactly how you are forming that URL.
You can have the app behind your API check the http referrer, but that is easy to fake if somebody wants to.
If it's not a requirement for the pages to be static, you could try something where you have a short-lived "key" generated by the API and included in the HTML response of the first page which gets passed along as a parameter back to the API. This would add overhead to your API though as you would have to have the server on that end maintain a list of "keys" that are valid, how long they are valid for, etc.
So, you can take some steps which won't cost a lot but aren't hard to get around if someone really wants to, or you can spend more time to make it a tiny bit harder, but there is no perfect way to do this if your API has to be publically-accessible.
The short answer is: anyone who can access the pages of your website will also be able to access your API.
You can attempt to make using your API more difficult by encrypting it in various ways, but since you'll have to include JavaScript code for decrypting the output of your API, you're just going to be setting yourself up for an arms race with anyone who decides they want to use your API through other means. Even if you use short-lived keys, a determined "attacker" could always just scrape your HTML (along with the current key) just before using the API.
If all you want to do is prevent other websites from using your API on their web pages then you could use Referrer headers but keep in mind that not all browsers send Referrers (and some proxies strip them too!). This means you'd want to allow all requests missing a referrer, and this would only give you partial protection. Also, Referrers can be easily forged, so if some other website really wants to use your API they can always just spoof a browser and access your API from their servers.
Are you, or can you use a cookie based authentication? My experience is based on ASP.NET forms authentication, but the same approach should be viable with PHP with a little code.
The basic idea is, when the user authenticates through the web app, a cookie that has an encrypted value is returned to the client browser. The json api would then use that cookie to validate the identity of the caller.
This approach obviously requires the use of cookies, so that may or may not be a problem for you.
Sorry, maybe I'm wrong but... can it be made using HTTPS?
You can (?) have your API accessible via https://example.com/json/?var1=x&var2=y, thus only authenticated consumer can get your data...
Sorry, there's no DRM on the web :-)
You can not treat HTML as a trusted client. It's a plain text script interpreted on other people's computers as they see fit. Whatever you allow your "own" JavaScript code do you allow anyone. You can't even define how long it's "yours" with Greasemonkey and Firebug in the wild.
You must duplicate all access control and business logic restrictions in the server as if none of it were present in your JavaScript client.
Include the service in your SSO, restrict the URLs each user has access to, design the service keeping wget as the client in mind, not your well behaved JavaScript code.