Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Just curious how other people feel about this. Will appreciate opinions or facts, whatever you got :)
I am working on an application where a lot of info is pulled from MySQL and needed on multiple pages.
Would it make more sense to...
Pull all data ONCE and store it in SESSION variables to use on other pages
Pull the data from the database on each new page that needs it
I assume the preferred method is #1, but maybe there is some downside to using SESSION variables "too much"?
Side question that's kind of related: As far as URLs, is it preferable to have data stored in them (i.e. domain.com/somepage.php?somedata=something&otherdata=thisdata) or use SESSION variables to store that data so the URLs can stay general/clean (i.e. domain.com/somepage.php)?
Both are probably loaded questions but any possible insight would be appreciated.
Thanks!
Your question can't be answered to the point where the answer is applicable everywhere.
Here's why: many web server architectures deal with having HTTP server (Apache, Nginx), serverside language (PHP, Ruby, Python) and RDBMS (MySQL, PostgreSQL) on one and the same machine.
That's one of the most common setups you can find.
Now, this is what happens in your scenario:
You connect to MySQL - you establish a connection from PHP > MySQL and that "costs" a little
You request the data, so MySQL reads it from the hard drive (unless cached in RAM)
PHP gets the data and allocates some memory to hold the information
Now you save that to a session. But by default, sessions are disk based so you just issued a write operation and you spent at least 1 I/O operation of your hard drive
But let's look at what happened - you moved some data from disk (MySQL) to RAM (PHP variable) which then gets saved at disk again.
You really didn't help yourself or your system in that case, what happens is that you made things slower.
On the other hand, PHP (and other languages) are capable of maintaining connections to MySQL (and other databases) so they minimize the cost of opening a new connection (which is really inexpensive in the grand scheme of things).
As you can see, this is one scenario. There's a scenario where you have your HTTP server on a dedicated machine, PHP on dedicated machine and MySQL on dedicated machine. The question is, again, is it cheaper to move data from MySQL to a PHP session. Is that session disk based, redis based, memcache based, database based? What's the cost of establishing the connection to MySQL?
What you need to ask, in any scenario that you can imagine - what are you trading off and for what?
So, if you are running the most common setup (PHP and your database on the same machine) - the answer is NO, it's not better to store some MySQL data in a session.
If you use InnoDB (and you probably are) and if it's optimized properly, saving some data to a session to avoid apparent overhead of querying the db for reads won't yield benefits. It's most likely going to be quite the opposite.
Putting it into the session is almost always a terrible idea. It's not even worth considering unless you've exhausted all other options.
Here's how you tackle these problems:
Evaluate if there's anything you can do to simplify the query you're running, like trim down on the columns you fetch. Instead of SELECT * try SELECT x,y where those are the only columns you need.
Use EXPLAIN to find out why the query is taking so long. Look for any easy wins like adding indexes.
Check that your MySQL server is properly tuned. The default configuration is terrible and some simple one-line fixes can boost performance dramatically.
If, and only if, you've tried all these things and you can't squeeze out any more performance, you want to try and cache the results.
You only pull the pin on caching because caching is one of the hardest things to get right.
You can use something like Memcached or Redis act as a faster store for pre-refetched results. They're designed to automatically expire cached data that's no longer used.
The reason using $_SESSION is a bad idea is because once data is put in there very few take the time to properly expunge it later, leading to an ever growing session. If you're concerned about performance, keep your sessions as small as possible.
Just think about your users(client pc). session takes some spaces to user pc, also session can get lost, may e after closing page, or copying the link and paste it to other browser. God practice there i think just use query, but note something, try as much as possible to reduce number of queries in page, it will slow down your site.
Related
I have searched for a few hours already but have found nothing on the subject.
I am developing a website that depends on a query to define the elements that must be loaded on the page. But to organize the data, I must repass the result of this query 4 times.
At first try, I started using mysql_data_seek so I could repass the query, but I started losing performance. Due to this, I tried exchanging the mysql_data_seek for putting the data in an array and running a foreach loop.
The performance didn't improve in any way I could measure, so I started wondering which is, in fact, the best option. Building a rather big data array ou executing multiple times the mysql_fetch_array.
My application is currently running with PHP 5.2.17, MySQL, and everything is in a localhost. Unfortunatly, I have a busy database, but never have had any problems with the number of connections to it.
Is there some preferable way to execute this task? Is there any other option besides mysql_data_seek or the big array data? Has anyone some information regarding benchmarking testes of these options?
Thank you very much for your time.
The answer to your problem may lie in indexing appropriate fields in your database, most databases also cache frequently served queries but they do tend to discard them once the table they go over is altered. (which makes sense)
So you could trust in your database to do what it does well: query for and retrieve data and help it by making sure there's little contention on the table and/or placing appropriate indexes. This in turn can however alter the performance of writes which may not be unimportant in your case, only you really can judge that. (indexes have to be calculated and kept).
The PHP extension you use will play a part as well, if speed is of the essence: 'upgrade' to mysqli or pdo and do a ->fetch_all(), since it will cut down on communication between php process and the database server. The only reason against this would be if the amount of data you query is so enormous that it halts or bogs down your php/webserver processes or even your whole server by forcing it into swap.
The table type you use can be of importance, certain types of queries seem to run faster on MYISAM as opposed to INNODB. If you want to retool a bit then you could store this data (or a copy of it) in mysql's HEAP engine, so just in memory. You'd need to be careful to synchronize it with a disktable on writes though if you want to keep altered data for sure. (just in case of a server failure or shutdown)
Alternatively you could cache your data in something like memcache or by using apc_store, which should be very fast since it's in php process memory. The big caveat here is that APC generally has less memory available for storage though.(default being 32MB) Memcache's big adavantage is that while still fast, it's distributed, so if you have multiple servers running they could share this data.
You could try a nosql database, preferably one that's just a key-store, not even a document store, such as redis.
And finally you could hardcode your values in your php script, make sure to still use something like eaccelerator or APC and verify wether you really need to use them 4 times or wether you can't just cache the output of whatever it is you actually create with it.
So I'm sorry I can't give you a ready-made answer but performance questions, when applicable, usually require a multi-pronged approach. :-|
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Can't pass mysqli connection in session in php
Many of us have written PHP applications that require databases; mostly MySQL, but I have often used very small MS Access databases for people less technically capable so they can download an tweak them/save backups/etc. on their own (whether this is correct or not, I have no idea).
What I notice, is a lot of time is spent connecting and running some of the same queries. Because of this I had an interesting thought: Storing the connection and possible result sets that are mostly static in a $_SESSION variable to reduce the burden as the user navigates the site.
Obviously doing so requires a lot of consideration. Things like closing the connection when the session is destroyed is just the start.
My question boils down to: Is this really something possible? And if so, what things should I be aware of (besides session fixation, as it is its own problem that applies to all sessions)?
You can't store database connections or result sets in the session, since those are resources, and:
Some types of data can not be serialized thus stored in sessions. It includes resource variables or objects with circular references (i.e. objects which passes a reference to itself to another object).
http://php.net/manual/en/intro.session.php
You can extract a result set into a normal array and store that in the session like any other variable. That would be a fairly typical use case for sessions anyway. Just be careful not to store too much data in the session, as that can become more taxing than fetching it from the database.
Even if you could do this (resource vs. data), this is a bad idea. You'll wind up with lots of concurrent open connections, which will blow your max connections very quickly... especially if its lifecycle is expanded beyond sub 100ms (depending on your queries) to 20 minutes or more. With open connections, something like MySQL also won't be able to reset its memory allocations properly, and the whole system sort of goes to hell. In short, this is not what DBs are for unless the only consumer of your code will be a single user.
As an alternative, I'd highly recommend caching technologies which are designed specifically to reduce database load and obviate connection times. Using something like, at its simplest, memcached will dramatically improve performance all the way around, and you'll be able to specify exactly how many system resources go into the cache -- while letting the database do its job of getting data when it needs to.
You can check permanent connections for the connection part.
http://php.net/manual/en/function.mysql-pconnect.php
You should those in some config file for better use. Sessions are for specific sessions and not for global.
My question is fairly simple; I need to read out some templates (in PHP) and send them to the client.
For this kind of data, specifically text/html and text/javascript; is it more expensive to read them out a MySQL database or out of files?
Kind regards
Tom
inb4 security; I'm aware.
PS: I read other topics about similar questions but they either had to do with other kind of data, or haven't been answered.
Reading from a database is more expensive, no question.
Where do the flat files live? On the file system. In the best case, they've been recently accessed so the OS has cached the files in memory, and it's just a memory read to get them into your PHP program to send to the client. In the worst case, the OS has to copy the file from disc to memory before your program can use it.
Where does the data in a database live? On the file system. In the best case, they've been recently accessed so MySQL has that table in memory. However, your program can't get at that memory directly, it needs to first establish a connection with the server, send authentication data back and forth, send a query, MySQL has to parse and execute the query, then grab the row from memory and send it to your program. In the worst case, the OS has to copy from the database table's file on disk to memory before MySQL can get the row to send.
As you can see, the scenarios are almost exactly the same, except that using a database involves the additional overhead of connections and queries before getting the data out of memory or off disc.
There are many factors that would affect how expensive both are.
I'll assume that since they are templates, they probably won't be changing often. If so, flat-file may be a better option. Anything write-heavy should be done in a database.
Reading a flat-file should be faster than reading data from the database.
Having them in the database usually makes it easier for multiple people to edit.
You might consider using memcache to store the templates after reading them, since reading from memory is always faster than reading from a db or flat-file.
It really doesnt make enough difference to worry you. What sort of volume are you working with? Will you have over a million page views a day? If not I'd say pick whichever one is easiest for you to code with and maintain and dont worry about the expense of the alternatives until it becomes a problem.
Specifically, if your templates are currently in file form I would leave them there, and if they are currently in DB form I'd leave them there.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
Here's the lowdown of the situation. I am creating a role-playing game with PHP. The combat system consists of the various buffs and debuffs. The combat is likely to take several turns (it's against an AI).
I am of two mind about ensuring the correctness of the player's data (buffs and debuffs expire over time. A +5 strength buff may last for only 1 turn). I could
1) Go through a character init sequence which fetch the user's data and changes to his data from items, equipments, passive skills, and then add in the buff, which is stored in session. If the buff exists, I apply it. If not, it's gone. This way the correctness of the data is ensured at the cost of performance...I assume.
2) Store the entire character data in session, and update the buffs. When the buffs is active, I add the modifiers, if the debuff/buff is gone, I have to remember to 'roll-back' or clean-up whatever effects the buff have. I am assuming this is less taxing on DB but will be difficult to ensure correctness as there may be so many different type of buffs.
So in terms of
a) database overhead
b) maintainability of the combat system
c) industry practises on such cases,
how does the 2 solutions fare? Are there more which I don't know about? I was thinking of using a subscriber pattern to implement #2, but as the web is stateless that seems to add more overhead.
I'm pulling the Knuth out: "Premature optimization is the root of all evil."
You don't know what the actual performance impact of this design decision is and you're in the middle of development. Without actual data it's impossible to tell if this will be a bottleneck in the future, there may be something else that is causing your app to be 10X slower than it should that you don't even know.
Measure. Test. Use Real World Data. Optimize.
I'm going to go with "neither" here, and tell you the approach I would take.
I can assume that because of your question, you're obviously planning on having some decent traffic and want to make sure your application performs under load. That being said, you should remember the nature of sessions here. If you ever need to cluster your front end by adding multiple web servers to spread out the load, then PHP's normal file-based session handling becomes relatively useless, as you can't ensure that a web visitor will hit the same front-end server all the time (well, you probably can, but it would be difficult). Storing to cookie might not be the best bet either if you're storing game states, as you might have more than 4Kb of data.
So the next step after that is to move your sessions to a central point with a new handler, which is generally your database. Any database gains you've made by storing to session have an offset now, as you still need to hit the database. It might not be quite as expensive, but it's not ideal. Basically, you're just bundling up data and re-writing it in a different form.
The scalability pattern I would follow here would be closest to your #1. Initially, rebuild the buffs each time and ensure data correctness. Use sessions to alleviate some performance, but don't rely on them heavily. When you start encountering performance issues, you'll have several options. In likely order of preference, they will be:
Install and use memcached. Instead of caching to sessions, cache to memory! Much faster, and a much better performance gain.
Separate the database and the web server onto different servers.
As you continue to grow, change your session handler to be database-oriented, and add more web front-ends
How extensive is the character data? Would it be viable to simply store all of the character's stats and items in the session? If so you could get the best of both worlds.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I want to keep logs of some things that people do in my app, in some cases so that it can be undone if needed.
Is it best to store such logs in a file or a database? I'm completely at a loss as to what the pros and cons are except that it's another table to setup.
Is there a third (or fourth etc) option that I'm not aware of that I should look into and learn about?
There is at least one definite reason to go for storing in the database. You can use INSERT DELAYED in MySQL (or similar constructs in other databases), which returns immediately. You won't get any return data from the database with these kinds of queries, and they are not guaranteed to be applied.
By using INSERT DELAYED, you won't slow down your app to much because of the logging. The database is free to write the INSERTs to disk at any time, so it can bundle a bunch of inserts together.
You need to watch out for using MySQL's built in timestamp function (like CURRENT_TIMESTAMP or CUR_DATE()), because they will be called whenever the query is actually executed. So you should make sure that any time data is generated in your programming language, and not by the database. (This paragraph might be MySQL-specific)
You will almost certainly want to use a database for flexible, record based access and to take advantage of the database's ability to handle concurrent data access. If you need to track information that may need to be undone, having it in a structured format is a benefit, as is having the ability to update a row indicating when and by whom a given transaction has been undone.
You likely only want to write to a file if very high performance is an issue, or if you have very unstructured or large amounts of data per record that might be unweidly to store in a database. Note that Unless your application has a very large number of transactions database speed is unlikely to be an issue. Also note that if you are working with a file you'll need to handle concurrent access (read / write / locking) very carefully which is likely not something you want to have to deal with.
I'm a big fan of log4php. It gives you a standard interface for logging actions. It's based on log4j. The library loads a central config file, so you never need to change your code to change logging. It also offers several log targets, like files, syslog, databases, etc.
I'd use a database simply for maintainability - also multiple edits on a file may cause some getting missed out.
I will second both of the above suggestions and add that file locking on a flat file log may cause issues when there are a lot of users.