Is it secure to store user chat messages in database as plain text?
And another question: Where to store page content - in database or in files? Wordpress holds blog entries in database, but it takes 25 requests to database to display a page, so website perfomance decreases.
Summary of comments:
Considering the nature of the application (a chatroom on the Internet), I feel that saving the chats as plain text is acceptable assuming that the users do not chat about private/sensitive/confidential information.
The above assumption can be made given some faith and optimism in the human race (i.e.: that it is smart enough to realize that a chatroom or a PM session is not the time or place to give out passwords, SINs, credit card numbers, etc.).
Concerning the, well... concerns surrounding the situation in which a person betrays the above assumption, I am of the belief that no amount of foolproofing is enough for the most ingenious fool. While encrypting the chat is most definitely more secure than not, the cost of encrypting each and every chat as opposed to the applicability of the added benefit to chats (that is, none whatsoever unless the chat contains sensitive information, which is a rare case at best) provides little incentive to encrypt those chats. A much simpler solution would be to simply disclaim any responsibility for private information leaked from the chats.
One last tidbit on handling sensitive information (this one's for Internet users everywhere): don't do it through email, chat, or any unsecured connection. Try as much as possible to avoid putting sensitive information where it will be logged, unless you have absolute confidence the logs won't be breached.
Separate questions really ought to be posted as separate questions...
And neither question is sufficiently well-defined to give anything but the broadest of answers...
But here we go:
Whether or not something is "secure" depends on your threat model (i.e. your definition of "secure"). But what alternatives are available to you? If you encrypt the messages, where will you store the decryption key?
Where to store data depends on the structure of such data and how you intend for it to be used. If it is "static" and will always be queried in predictable ways, a filesystem may provide sufficient structure for good performance; however, if the data is "dynamic" (i.e. your application will modify it), then a database may offer greater flexibility or better performance. As with most problems in computing, the design decision you take is a trade-off for which the best answer will depend on your own requirements (and indeed, your metric of what is "best").
As long as you keep your database credentials safe with you, It is fine to keep them in database. I do not see any reason to keep them encrypted.
What kind of page content are you talking about ? If it is a CMS kinds stuff, If you keep page contents as Pages, How many pages you are going to keep. God !!!!
You should use a database to keep such things. It makes things easier to update the content in the future etc. And you do not need to worry about Querying your database. Find out items which are not being updated frequently and cache it. when your page needs those contents , get it from the cache layer instead of the DB tables.
Is it secure to store user chat messages in database as plain text?
Depends on how sensible is the information and how secure is the database itself. Example: Can the database be accessed from outside or only from localhost? However if you feel that you need an extra layer of security, then it doesn't hurt to use some simple way of obfuscation.
Where to store html page content - in database or in files?
Most of the times, accessing files is quicker than accessing the Database.
Database should be used to store/access information in a structured way, enabling elaborated searches, data changes, etc... Static HTML is probably better to save in the filesystem. However, sometimes it might be best to store html content in the database.
Examples:
Storing in database makes it easier to change database information from an admin page, than changing the file system (also, it's safer). So for dynamically created pages, or in a CMS (such as Drupal or Wordpress) it might be best to store content and "layouts" in the database.
Forum posts contain "content" as well as markup (styling). You don't usually separate one from the other.
Database storage ends in files too. Compression is in use - less memory for saving in the end.
Related
I am wondering what the best practice way is to store small amounts of data across a site. For example you access an API you retrieve some non sensitive, non user data and you want to use it across your site to add functionality, reference dates of a last event etc.
I have an idea that I could do this via the session variable, to avoid hitting the API every request etc. Is that a good idea or is it bad practice? If it's bad practice what other approaches should I take?
That's fine to do it that way, in fact, that's part of what sessions are all about. If you want to store it long term (after the user destroys the session), store it in the database and reference it when needed.
A nice alternative is Memcached, because putting information in a session will be visible only for the current client.
Free & open source, high-performance, distributed memory object
caching system, generic in nature, but intended for use in speeding up
dynamic web applications by alleviating database load.
Here you can find more information about memcache: http://php.net/manual/en/book.memcache.php
[I hope that this question is not too broad, I think that the subject is very interesting but I incourage you to tell me if it's off-policy.]
My scenario is this:
I have a LAMP website who stores also sensitive data and documents
Only registered users are allowed to operate on the site, and only on certain data and documents. Users are stored in $_SESSION variables
Most of the pages implement a sort of rudimental permission control, but some important DB operations are called via AJAX
AJAX security is implemented very poorly, as anyone that is that smart can tamper with the request sending whatever id they like and delete records with brutal simplicity
Asking for a complete book on security is obviously a bit too much (and I'm already reading and trying a lot on the subject), let's say that my main concern is if AJAX pages should be treated with special regards, as I need to secure the whole software to prevent hacks and other problems.
let's say that my main concern is if AJAX pages should be treated with special regards
Not really. They should be treated almost exactly the same as any other request. All HTTP requests come from outside your system and are under the control of the client (so can consist of, more or less, anything the user can imagine).
You might be returning JSON, you might be returning a complete HTML document, you might be returning XML — but the format doesn't matter, the data does.
If the request is for sensitive data, then you need (on the server) to authenticate the user and then make sure they are authorised to view / edit that data.
The only difference is how you present a "You are not authorised" message. You can't simply return an HTML document with a login form when you expect the browser to load data into XHR. The response needs to be appropriately formatted and the JavaScript needs to be able to handle it.
I have a LAMP website who stores also sensitive data and documents
You should store as little sensitive data as possible. Especially when you are not sure how to keep this information secure/private. Use OpenID or something for your authentication for example. I really like LightOpenID for it's simplicity. I created a little example project/library to see lightopenId in use. It simplifies using OpenID by using openID-selector. When you use OpenID you also use secure OpenID providers the passwords are also not transmitted over the wire in plain-text but protected by https/SSL.
Only registered users are allowed to operate on the site, and only on
certain data and documents. Users are stored in $_SESSION variables
Yup that's what sessions are for.
Most of the pages implement a sort of rudimental permission control,
but some important DB operations are called via AJAX
You should read up on OWASP top 10. at least. (Don’t stop at 10.)
AJAX security is implemented very poorly, as anyone that is that smart
can tamper with the request sending whatever id they like and delete
records with brutal simplicity
See previous section. Read up on OWASP top 10 section at least. Somethings which a lot of people overlook for example are CSRF for example.
This is a beginner question...
In a website, what type of data should or should not be included inside the session? I understand that I should not include any info that needs to remain secure. I'm more interested in programming best practice. For example, it is possible to include into the session some data which would otherwise be sent from page to page as dependency injection. Wouldn't that correspond to creating a global variable?
Generally speaking, what kind of data has or hasn't its place inside a session table?
Thanks,
JDelage
The minimum amount of information needed to maintain needed state information between requests.
You should treat your session as a write-once, read many storage. But one which is rather volatile - e.g. the state of your underlying application data should be consistent (or recoverable) if all the sessions suddenly disappeared.
There are some exceptions to this (normally the shopping basket would be stored in the session - but you might want to perform stock adjustments to 'reserve' items prior to checkout). Here items may be added/edited/changed multiple times - so its not really write-once - but by pre-reserving stock items you are maintaining the recoverabiltiy of the database - but an implication of this is that you should reverse the stock adjustments when the session expires in the absence of completion.
If you start trying to store information about the data relating to individual page turns, you're quickly going to get into problems when the user starts clicking on the forward/back buttons or opens a new window.
In general you can put anything you like in a session. It's bad practice to put information in a session that has to be present to make your page run without (technical) errors.
I suggest to minimize the amount of data in your session as much as possible.
stuff you can save in the session so that you dont have to make another database query for info that isn't going to change. like their username, address, phone number, account balance, security permissions on your site, etc.
(This is perhaps more than you're looking for, but might make for good additional information to add to the good answers already posted.)
Since you mention best practices, you may want to look into some projects/technologies which can be used to take the idea of session state a bit further. One common pitfall with horizontally scaling web applications across multiple servers is maintaining session state between them. (User A logs in to Server A which stores the user's session, but on the next request hits Server B which doesn't know about User A's session, etc.)
One of the things I always end up saying to myself and to colleagues is that session by itself isn't really the best place to store data, even if that data is highly transient in nature. A web server is a request/response system, not a data store. It's highly tuned to the former, but not always so great for the latter.
Thus, there are ways to externalize your application's session data (or any stateful data, which should really be kept to a design minimum in the RESTful stateless nature of the web) from your web server and to another system. Memcached is a very common tool for this. There are also drop-in session replacements (or configurable session options for various frameworks/environments) which store session in a database like SQL or MySQL.
One idea I've been toying with lately is to store session data (well, any transient data where it's OK to lose it in a catastrophe) in a NoSQL database. CouchDB and MongoDB are my current top choices for this, but there's no shortage of other options. CouchDB has excellent horizontal scaling, MongoDB is ridiculously fast when run entirely in-memory, etc.
One of the major benefits of something like this, at least for me, is that deployments can easily become non-events. The web services on any given server can be re-started and the applications therein re-initialized without losing stateful data. If the data is persisted to the disk (that is, not entirely run in-memory) then the server can even be rebooted without losing it. Servers/services can drop in and out of the farm and users would never know the difference.
Additionally, externalizing this data allows you to analyze the data in potentially useful ways. Query it, run metrics on it, interface with it via other web applications or entirely offline tools, etc. It really opens up the options as a project grows in complexity.
(Again, this isn't really intended to answer your question, but rather to just add information that you may find useful. It's something my colleagues and I have been tinkering with as of late and your question seemed like a good place to mention it.)
I am using the Zend Framework but my question is broadly about sessions / databases / auth (PHP MySQL).
Currently this is my approach to authentication:
1) User signs in, the details are checked in database.
- Standard stuff really.
2) If the details are correct only the user's unique ID is stored in the session and a security token (user unique ID + IP + Browser info + salt). The session in written to the filesystem.
I've been reading around and many are saying that storing stuff in sessions is not a good idea, and that you should really only write a unique ID which refers back to the user's details and a security token to prevent session hijacking. So this is the approach i've taken, i use to write the user's details in session, but i've moved that out. Wanted to know your opinions on this.
I'm keeping sessions in the filesystem since i don't run on multiple servers, and since i'm only writting a tiny tiny bit of data to sessions, i thought that performance would be greater keeping sessions in the filesystem to reduce load on the database. Once the session is written on authentication, it really is only read-only from then on.
3) The rest of the user's details (like subscription details, permissions, account info etc) are cached in the filesystem (this can always be easily moved to memory if i wanted even more performance).
So rather than keeping the user's details in session, the user's details are cached in the file system. I'm using Zend_Cache and the unique cache id is something like md5(/cache/auth/2892), the number is the unique id of the user. I guess the benefit of this method is that once the user is logged in, there is essentially not database queries being run to get the user's details. Just wonder if this approach is better than keeping the whole lot in session...
4) As the user moves throughout the site the only thing that is checked is the ID in the session and the security token.
So, overall the first question is 1) is the filesystem more efficient than a database for this purpose 2) have i taken enough security precautions 3) is separating user detail's from the session into a cached file a pointless task?
Thanks.
You're asking a range of things.
Sessions
Sessions in PHP are fast and efficient. Thousands of small disk-based sessions on a moderately up-to-date server is not going to be a performance bottleneck. Neither is writing your own handlers (very easy; the PHP manual has examples) to put it in a database.
About the only best-practice rules about sessions is: only give the web browser one thing, the session ID. Putting just the logged in userid in the session and retrieving those details from the DB when you need them is also best-practice. It also means that user information can be changed and they get it on the next page update.
It doesn't sound like you will have this problem but beware of just throwing a lot of stuff into a session. A few K of data (say, a few dozen scalars) is fine. Tossing many objects and large arrays of data in there will be noticed. If you do this for a specific page, remember to throw it away in the session once the page is done with it.
You may also want to implement your own login timeout with a session variable. The garbage collection settings in php.ini are intended for managing the storage of session data, not for doing login timeouts.
Caching
This is a complex topic and you will probably need to start gathering metrics (generally page load times) before implementing anything.
To implement any sort of caching, you do need to consider the lifetime of the data you're caching and how expensive re-generating it will be on a cache miss. Just throwing memcache at the problem is not a solution; you still need to understand your caching parameters and how memcache interprets them. This also applies to any persistent storage solution, including disk-based sessions, but I'm highlighting memcache because it is high-profile and has quite an aggressive expiry mechanism.
An often overlooked example is loading the same data from the database multiple times in a page: a good ORM will do that for you without relying on MySQL query caching. Another overlooked example are small queries that run on every page: caching these for just a few seconds on a moderately busy server and the database load will drop considerably.
Finally, caching at multiple levels is often much more effective and scalable than once because they can leverage each other's expiries. It also abstracts well: for example, hide it in your ORM and it's theoretically available invisibly and automatically for all your objects.
1) You can easily test which is faster by making a loop script. Anyway, a drawback with using the filesystem is that you need to update the cached file everytime you update the db. Copies of data is in general a bad thing. Also, unless you have millions of visitors I dont think there will be any practical diffrence regarding speed in any of the stratagies. And... not to forget, sessions are also stored in the filesystem. One file for each session.
Is a query faster then the filesystem: Depends. Is query caching enabled. In MySql it is by default, and than you might be lucky and only need a memory access. If not, the db needs to do a filesystem accass anyway. Second, how optimized is your query with index's. How buissy is the server harddisk.
3) Depends on the speed of fetching it from db. In general, caching can do magic to speed performance, but caching in memory would be even better by using memcached or something similar. In general i would avoid copies of the data in files. But of course, if it takes secods to query the data from the db, than go for filesystem caching. Also, if you have many users.. like 10.000+ you have to make some folder system, since putting 10.000 cached files in the same folder slows downs the accesstime...
I currently have a custom session handler class which simply builds on php's session functionality (and ties in some mySQL tables).
I have a wide variety of session variables that best suits my application (primarily kept on the server side). Although I am also using jQuery to improve the usability of the front-end, and I was wondering if feeding some of the session variables (some basics and some browse preference id's) to a JS object would be a bad way to go.
Currently if I need to access any of this information at the front-end I do a ajax request to a php page specifically written to provide the appropriate response, although I am unsure if this is the best practice (actually I'm pretty sure this just creates a excess number of Ajax requests).
Has anyone got any comments on this? Would this be the best way to have this sort of information available to the client side?
I really guess it depends on many factors. I'm always having "premature optimization ..." in the back of my head.
In earlier years I rushed every little idea that came to my mind into the app. That often lead to "i made it cool but I didn't took time to fully grasp the problem I'm trying to solve; was there a problem anyway?"
Nowadays I use the obvious approach (like yours) which is fast (without scarifying performance completely on the first try) and then analyze if I'm getting into problems or not.
In other words:
How often do you need to access this information from different kind of loaded pages (because if you load the information once without the user reloading there's probably not much point in re-fetching it anyway) multiplied by number of concurrent clients?
If you write the information into a client side cookie for fast JS access, can harm be done to your application if abused (modified without application consent)? Replace "JS" and "cookie" without any kind of offline storage like WHATWG proposes it, if #1 applies.
The "fast" approach suits me, because often there's not the big investment into prior-development research. If you've done that carefully ... but then you would probably know that answer already ;)
As 3. you could always push the HTML to your client already including the data you need in JS, maybe that can work in your case. Will be interesting to see what other suggestions will come!
As I side note: I've had PHP sessions stored in DB too, until I moved them over to memcached (alert: it's a cache and not a persistent store so may be not a good idea for you case, I can live with it, I just make sure it's always running) to realize a average drop of 20% of database queries and and through this a 90% drop of write queries. And I wasn't even using any fancy Ajax yet, just the number of concurrent users.
I would say that's definately an overkill of AJAX, are these sessions private or important not to show to a visitor? Just to throw it out there; a cookie is the easiest when it comes to both, to have the data in a javascript object makes it just as easily readable to a visitor, and when it comes down to cookies being enabled or not, without cookies you wouldn't have sessions anyway.
http://www.quirksmode.org/js/cookies.html is a good source about cookie handling in JS and includes two functions for reading and writing cookies.