I was using mysql to check if the user was logged in or not, but I received an email from my hosting company saying that I had exceeded the mysql queries limit and the cost to upgrade the mysql queries limit to 150000 per hour was $240/year.
Now, I need an alternative way to check if the user's logged in or not, like writing the user's timestamp in a file, but I don't know the best way to do it... And I don't even know which tags I put on this post...
So, could you please help me with this problem?
Use $_SESSION. It's 1000 times more efficient and designed for just this purpose. Read about it here
Upgrade (or from cost perspective downgrade) to a root-server and/or use better mechanisms to store sessions. ie memcache, couchdb or mongodb.
In every case, you have to handle semaphores by hand, if you leave PHP's session handler.
It is rather odd that your host has a mysql query limit, but back to your question;
I will take a guess that you are not just trying to see if a user is logged in locally, but rather, having another account see that the user is logged in, which would require a database call, not a $_SESSION (which is maintained on a per-user basis).
If, however, we make the assumption that you want to do this outside of the database, you can generate the end HTML result when a user logs in or out. Keep a table in the database with the status of the users, but whenever a user logs in or out, make ONE query to that table, and generate, say, your HTML list of logged in users. Save the output to a file, and when you need to render a page, include that file instead of making a database call.
It's not completely removing the MySQL database, but it is reasonably reliable, and should significantly reduce your number of database queries.
Related
I have a webshop, user will buy something
When the user visit the my web, their data will be saved temporarly in a variable $user as an array
It will be checked when the user buy something, if their balance is more or equal to the things that they want to buy, but the problem arise when there are some users that try to use two browsers to buy things at the same times, their balance just cut once (it should be twice, since they buy it twice using two browsers)
I know I can just update the $user variable before checking, but I will have to run another query to MySQL, and there is many orders ...
Is there any SQL syntax that can be used to prevent this kind of attack?
for checking their balance and make sure it's correct
Based on your current setup (ie using a variable):
Someone using two browser on the same site trying to use up their balance is going to be fairly rare, with most cases someone trying to game your system.
Just finally check their balance at the point of processing the order and if ok allow it, otherwise don't. For those doing it by accident (which is rare), they'll soon realise the error.
An alternative:
It'd be better to check the real data rather than a variable which isn't reliable and has to be forced to be persistent, and as you know not available in different sessions.
I think a better way would be to use some fast centralised persistent storage like Redis (fairly easy to learn, essentially it's an array stored in memory). You can then store their username (or whatever uniquely IDs them) and while they may have different keys across the two browsers, there will be a common unique ID and you can update their credit value in both (all) sessions by searching for the unique ID.
Then whatever browser that user is logged in to will be updated same as other browsers.
Maybe a better idea:
Unless your application needs it, don't let people log in to different browsers/devices. When they try to login, state "already logged in on another place, want to log that one our and log in here?" etc
On my website, users can currently compare multiple company for a prestation.
For each company, I calculate the price of the prestation.
To calculate this price I have a very big SQL request in order to filter companies based on the user's previous input and get every parameter I need.
Once the query end, I loop througt the company list and calculate the price for each of them and for each additional services the company offer. Then I display those value to the user in HTML.
The user can then add or remove an option and order the company.
Then, when a user choose a price, I send the company_id along with the user's otpion (the different services he chose) to the server and get the price previously calculated from the user SESSION.
Prices are stored in user session in order to avoid the calculation process but, I have like ~6 prices by company and usually ~30 companies for one user request. Which mean that I store in session an array with around 180 different prices for one user.
I found it quite wastefull to store this many variables in session and I was wondering, is there a better way to store those variables ? Should I store them in database ?
By the way, server side, i'm using PHP along with Mysql for the database.
What you're effectively doing is a very primitive form of caching. Sadly, the session is not the best place to do so, for a variety of reasons:
The session can never be shared between users. Some values cached may be the same for every user. It's good to have a cache that allows you to go the "unique" or "shared" routes at will.
Is your session cached data used on every page? If it is, then forget this point. If, however, it isn't, on every page, you're still incurring the cost of fetching (which, depending on your server configuration, may involve a few fs calls, or network calls, or a combination) and deserializing the data, on every request. This, if your session payload is large, can make a significant different to load times.
Another point to consider is the simple fact that, if you are running a straight-out-of-the-box LAMP stack and have not configured a shared session driver, you're going to find a very nasty surprise when you scale out :-)
Before we go any further, ask yourself these questions:
Do the values in the session change on a user-by-user basis? And if they do, is it by a fixed amount of percentage?
Do the values in the session change often?
How are the values calculated?
If #1 is "No"
You are better off caching one copy and use it for every user.
If #2 is "No"
You are better off denormalizing (i.e. pre-calculating) the values in the database
If #3 is a complicated formula
See #2
In every case
MySQL is a very poor cache driver, and should be avoided if you can. Most people replace it with redis, but that's also not a very good way due to its inability to scale by itself (you need nutcracker/twemproxy to make it properly shard). Either way, if your data to be cached is relatively small, MySQL will do. However, plan for the future.
If you do plan on going the proper cache way, consider a key-value store such as Riak, or a document storage driver such as Aerospike.
I'm designing my own session handler for my web app, the PHP sessions are too limited when trying to control the time the session should last.
Anyway, my first tests were like this: a session_id stored on a mysql row and also on a cookie, on the same mysql row the rest of my session vars.
On every request to the server I make a query, get these vars an put them on an array to use the necesary ones on runtime.
Last night I was thinking if I could write the vars on a server file once, on the login stage, and later just include that file instead of making a mysql query on every request.
So, my question is: which is less resource consuming? doing this on mysql or on a file?
I know, I know, I already read several threads on stackoverflow about this issue, but I have something different from all those cases (I hope I didn't miss something):
I need to keep track of the time that has passed since the last time the user used the app, so, in every call to the server not only I request the entire database row, I also update a timestamp on that same row.
So, on both cases I need to write to the session on every request...
FYI: the entire app runs on one server so the several servers scenario when using files does not apply..
It's easier to work with when it's done in a database and I've been using sessions in database mostly for scalability.
You may use MySQL since it can store sessions in it's temporary memory with well-configured MySQL servers, you can even use memory tables to fasten the thing if you can store all the sessions within memory. If you get near your memory limit it's easy to switch to a normal table.
I'd say MySQL wins over files for performance for medium to large sites and also for customization/options. For smaller websites I think that it doesn't make that much of a difference, but you will use more of the hard drive when using files.
the users registered in a web site will have the possibility to send invitations to the friends. I want to add a daily limit for the number of invitations that a user may send.
Initially I've just added a limit (40) in the php file, but then I thought it would be better to give to the administrators the possibility to change this limit, so I've added this limit in the database. But now every time a user want to send invitations the database will be used. Would this affect the performance?
How would you configure this feature?
TL;DR: just put it in the database. :)
Complete story: It should not be a performance hit. Everything (the user itself, the usernames of the recipient, loads of stuff from your page) will be coming from your database. you shouldn't care.
If you have a REALLY big userbase, and it becomes an issue, I'm sure there are other places to do performance updates (like use memcached for all sorts of stuff). But if you want to "cache" it, I guess you could retrieve it once while loggin in and put it in the session. Use this value to substract and check etc. then ALSO check once against the database (in the background) to make sure there isn't any sort of freakish thing going on for this user. But this can be async, and does not have as big of an impact for the user-experience.
In the rare case the session says it's ok, but the database says it isn't, just send the user an error. The other way around, might need the user to re-login. But it will be rare or even impossible if you implement it correctly :)
It depends a bit on hardware but a 400,000 row table (10,000 * 40) isn't that huge in MySQL standards. I think you'll be fine.
Just make sure that you've built it sensibly and from how you've described it that there's an index on the column that stores the unique invite code.
I am thinking about using a noSQL (mongoDB) paired with memcached to store sessions with in my webapp. The idea is that upon each page load, the user data is compared to the data in the memcache and if something has changed, the data would be written to both memcached and mySQL. This way the reads would be greatly reduced and memcached utilized to do what it does best.
However I am a bit concerned about using a non-ACID database for session storage especially with the memcached layer. Let's say something goes wrong while updating the session to the DB and our users got instant headache wondering why their product that they put in the cart doesn't show up...
What's an appropriate approach to this? Should we go for a mySQL session storage or is it fine to keep a non-acid supportive database for sessions?
Thanks!
I'm using MongoDB as session storage currently. It is possible to avoid race conditions mentioned by pilif. I found a class that implements a session handler for MongoDB (http://www.jqueryin.com/projects/mongo-session/) and forked it on github to suit my needs (http://github.com/halfdan/MongoSession).
If you don't want to lose your data, stick with ACID tested databases.
What's the payoff you're looking for?
If you want a secure system, you can't trust anything from the user, save for perhaps selected integers, so letting them store the information is typically a really bad idea.
I don't see the payoff for storing sessions outside of your MySQL database. You can cron cleanup on the tables if that's your concern, but why bother? Some users will shop on a site and then get distracted for a while. They would then come back a day or two later.
If you use cookies or something really temporary to store their session info, there is a really good chance their shopping time was wasted. Users really value their time... so if you stored their session info in the database, you can write something sexy to manage that data.
Plus, the nice side effect of this is that you'll generate a lot of residual information about what people like on your website that wouldn't perhaps be available to you later on. Like you could even consider some of it to be like a poll or something where the items people are adding to their cart could impact how you manage your business, order inventory or focus your marketing.
If you go with something really temporary then you lose out on getting residual benefits.
Without any locking on the session, be really, really careful of what you are storing. Never ever store anything that is dependent on what you have read before as the data might change between you reading and writing - especially in case of ajax where multiple requests can go out at once.
An example what you must not store in a non-locked session would be a shopping cart as, to add a product, you have to read, unserialize, add the product and then serialize again. If any other request does the same thing between the first requests read and write, you lose the second request's data.
Have a look at this article for detail: http://thwartedefforts.org/2006/11/11/race-conditions-with-ajax-and-php-sessions/
Keep Sessions on your filesystem (where PHP locks them for you), in your database (where you have to do manual locking) or never, ever, write anything of value to your session if that value is derived of a previous read.
While using memcached as a cache for database, it is the user who have to ensure the data consistency between database and cache. If you'll want to scale up and add more servers there is a probability to be out of sync with database even if everything seems ok.
Instead you may consider Hazelcast. As of 1.9 it also supports memcache protocol. Compared to memcached Hazelcast wants you to implement Map Persister and only itself updates the database for the updated entries. This way you don't have to handle "check cache, if data changed update database" kind of stuff.
If you write your app so that the user stores all session information client side, then you just verify that information as needed, you won't need to worry about sessions on the server side. This is one of the principles in REST style architecture. For instance, if the user is requesting adding an item to their shopping cart, just store the itemID list and count on the client side. When you hit the cart page, you can easily look up the item information from the list of itemIDs they are telling you are in their cart.
During checkout, go directly against the database with transactions to ensure you aren't getting any race conditions, and check your live inventory. If inventory isn't there when they go to check out, just say, "sorry, we just sold out". Of course, at that point you should go update any caches you have out there that are telling people you have inventory.
I would look at how much the user costs to acquire and then ask what is the cost for implementing a really good system. Keep in mind that users are a biological retry method. "I'm bored... press reload again..." While, this isn't the most perfect solution, it is sometimes acceptable vs the cost comparsion for "not lose anything - ever".
If you want additional security, you can have your sessions cached to a separate set of memcache servers so there are no accidental flushes. :)
There are a number of other systems membase.org, and some other persistent memcache solutions (java implementations) that will persist storage to disk. If you want to modify your client somewhat, or how you access memcache, you can do your own replication of memcache session objects.
-daniel