Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
Here's the lowdown of the situation. I am creating a role-playing game with PHP. The combat system consists of the various buffs and debuffs. The combat is likely to take several turns (it's against an AI).
I am of two mind about ensuring the correctness of the player's data (buffs and debuffs expire over time. A +5 strength buff may last for only 1 turn). I could
1) Go through a character init sequence which fetch the user's data and changes to his data from items, equipments, passive skills, and then add in the buff, which is stored in session. If the buff exists, I apply it. If not, it's gone. This way the correctness of the data is ensured at the cost of performance...I assume.
2) Store the entire character data in session, and update the buffs. When the buffs is active, I add the modifiers, if the debuff/buff is gone, I have to remember to 'roll-back' or clean-up whatever effects the buff have. I am assuming this is less taxing on DB but will be difficult to ensure correctness as there may be so many different type of buffs.
So in terms of
a) database overhead
b) maintainability of the combat system
c) industry practises on such cases,
how does the 2 solutions fare? Are there more which I don't know about? I was thinking of using a subscriber pattern to implement #2, but as the web is stateless that seems to add more overhead.
I'm pulling the Knuth out: "Premature optimization is the root of all evil."
You don't know what the actual performance impact of this design decision is and you're in the middle of development. Without actual data it's impossible to tell if this will be a bottleneck in the future, there may be something else that is causing your app to be 10X slower than it should that you don't even know.
Measure. Test. Use Real World Data. Optimize.
I'm going to go with "neither" here, and tell you the approach I would take.
I can assume that because of your question, you're obviously planning on having some decent traffic and want to make sure your application performs under load. That being said, you should remember the nature of sessions here. If you ever need to cluster your front end by adding multiple web servers to spread out the load, then PHP's normal file-based session handling becomes relatively useless, as you can't ensure that a web visitor will hit the same front-end server all the time (well, you probably can, but it would be difficult). Storing to cookie might not be the best bet either if you're storing game states, as you might have more than 4Kb of data.
So the next step after that is to move your sessions to a central point with a new handler, which is generally your database. Any database gains you've made by storing to session have an offset now, as you still need to hit the database. It might not be quite as expensive, but it's not ideal. Basically, you're just bundling up data and re-writing it in a different form.
The scalability pattern I would follow here would be closest to your #1. Initially, rebuild the buffs each time and ensure data correctness. Use sessions to alleviate some performance, but don't rely on them heavily. When you start encountering performance issues, you'll have several options. In likely order of preference, they will be:
Install and use memcached. Instead of caching to sessions, cache to memory! Much faster, and a much better performance gain.
Separate the database and the web server onto different servers.
As you continue to grow, change your session handler to be database-oriented, and add more web front-ends
How extensive is the character data? Would it be viable to simply store all of the character's stats and items in the session? If so you could get the best of both worlds.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last year.
Improve this question
Currently, I am working on a website and just started studying backend. I wonder why nobody uses JSON as a database. Also, I don't quite get the utility of php and SQL. Since I could easily get data from JSON file and use it, why do I need php and SQL?
ok! let assume you put the data in a JSON variable and store it in a file for all your projects.
obviously, u need to add a subsystem for getting back up, then you will write it.
you must increase the performance for handling a very large amount of data, just like indexing, hash algorithms, and... , assume u handle it.
if you need some API for working and connecting with a variety of programming languages, u need to write them.
what about functionalities? what if you need to add some triggers, store procedures, views, full-text search and etc? ok, you will pay your time and add them.
ok, good job, but your system will grow up and you need to scale it, can you do it? u will write abilities for clustering across servers, sharding, and ...
now you need to guarantee that your system will compatible with ACID rules, to keep atomicity, Consistency, Isolation, and Durability.
can you always handle all querying techniques (Map/Reduce) and respond with a fast and standard structure?
now it's time to offer very quick write speeds, it brings serious issues for you
ok, now proper your solutions for condition racing, isolation level, locking, relations and ...
after you do all this work plus thousands of many others, probably you will have a DBMS a little bit just like MongoDB or other relational and non-relational databases!
so it's better to use them, however, obviously, you can choose to don't to use them too, I admit that sometimes saving data in a single file has better performance, but only sometimes, in some cases, with some data, for some purpose! if you know what exactly you do, then ist OK to save data in a JSON file.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Just curious how other people feel about this. Will appreciate opinions or facts, whatever you got :)
I am working on an application where a lot of info is pulled from MySQL and needed on multiple pages.
Would it make more sense to...
Pull all data ONCE and store it in SESSION variables to use on other pages
Pull the data from the database on each new page that needs it
I assume the preferred method is #1, but maybe there is some downside to using SESSION variables "too much"?
Side question that's kind of related: As far as URLs, is it preferable to have data stored in them (i.e. domain.com/somepage.php?somedata=something&otherdata=thisdata) or use SESSION variables to store that data so the URLs can stay general/clean (i.e. domain.com/somepage.php)?
Both are probably loaded questions but any possible insight would be appreciated.
Thanks!
Your question can't be answered to the point where the answer is applicable everywhere.
Here's why: many web server architectures deal with having HTTP server (Apache, Nginx), serverside language (PHP, Ruby, Python) and RDBMS (MySQL, PostgreSQL) on one and the same machine.
That's one of the most common setups you can find.
Now, this is what happens in your scenario:
You connect to MySQL - you establish a connection from PHP > MySQL and that "costs" a little
You request the data, so MySQL reads it from the hard drive (unless cached in RAM)
PHP gets the data and allocates some memory to hold the information
Now you save that to a session. But by default, sessions are disk based so you just issued a write operation and you spent at least 1 I/O operation of your hard drive
But let's look at what happened - you moved some data from disk (MySQL) to RAM (PHP variable) which then gets saved at disk again.
You really didn't help yourself or your system in that case, what happens is that you made things slower.
On the other hand, PHP (and other languages) are capable of maintaining connections to MySQL (and other databases) so they minimize the cost of opening a new connection (which is really inexpensive in the grand scheme of things).
As you can see, this is one scenario. There's a scenario where you have your HTTP server on a dedicated machine, PHP on dedicated machine and MySQL on dedicated machine. The question is, again, is it cheaper to move data from MySQL to a PHP session. Is that session disk based, redis based, memcache based, database based? What's the cost of establishing the connection to MySQL?
What you need to ask, in any scenario that you can imagine - what are you trading off and for what?
So, if you are running the most common setup (PHP and your database on the same machine) - the answer is NO, it's not better to store some MySQL data in a session.
If you use InnoDB (and you probably are) and if it's optimized properly, saving some data to a session to avoid apparent overhead of querying the db for reads won't yield benefits. It's most likely going to be quite the opposite.
Putting it into the session is almost always a terrible idea. It's not even worth considering unless you've exhausted all other options.
Here's how you tackle these problems:
Evaluate if there's anything you can do to simplify the query you're running, like trim down on the columns you fetch. Instead of SELECT * try SELECT x,y where those are the only columns you need.
Use EXPLAIN to find out why the query is taking so long. Look for any easy wins like adding indexes.
Check that your MySQL server is properly tuned. The default configuration is terrible and some simple one-line fixes can boost performance dramatically.
If, and only if, you've tried all these things and you can't squeeze out any more performance, you want to try and cache the results.
You only pull the pin on caching because caching is one of the hardest things to get right.
You can use something like Memcached or Redis act as a faster store for pre-refetched results. They're designed to automatically expire cached data that's no longer used.
The reason using $_SESSION is a bad idea is because once data is put in there very few take the time to properly expunge it later, leading to an ever growing session. If you're concerned about performance, keep your sessions as small as possible.
Just think about your users(client pc). session takes some spaces to user pc, also session can get lost, may e after closing page, or copying the link and paste it to other browser. God practice there i think just use query, but note something, try as much as possible to reduce number of queries in page, it will slow down your site.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I am about to create a PHP web project that will be consisting of a large database. The database will be MYSQL and will store more than 30000 records per day. To optimize the DB I thought to use MEMCACHED library with it. Am i going the correct way or some other alternative can be used to overcome the data optimization problem. I just want to provide faster retrieval and insertion. Can somebody advise me which tool should I use and how, as the data will gradually increase at a higher rate ? Should i use object relational mapping concept too ?
You can use Master & Slave technique for this purpose. Basically it would be combination of 2 db first for read operation and other for write operation.
I'd side with #halfer and say he's right about the test data. At least you'll know that you're not trying to optimize something that doesn't need optimizing.
On top of test data you'll also need some test scenarios to mimic the traffic patterns of your production environment, that's the hard part and really depends on the exact application patterns: how many reads versus writes versus updates / per second.
Given your number (30k) you'd average out at about 3 inserts / second which I'd assume even the cheapest machines could handle with ease. As for reads, a years worth of data would be just under 11M records. You may want to partition the data (mysql level or application level) if look ups become slow but I doubt you'd need to with such relatively small volumes. The real difference maker would be if the # of reads is 1000x more than the number of inserts, then you could look into what #ram sharma suggested and set up a replicated master-slave model where the master takes all the writes and the slaves are read-only.
Memcached is a powerful beast when used correctly and can turn a slow DB disk read into a blazing fast memory read. I'd still only suggest you look into it IF the DB is too slow. Adding moving parts to any application also adds potential failure points and increases the overall complexity.
EDIT: as for the use of an ORM, that's your choice and really won't change a thing concerning the DB's speed although it may add fractions of milliseconds to the end user.. usually worth it in my experience.
Cheers --
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
We're taking our functional web-based software that's previously been exclusive to our business and introducing licensing options so that other businesses may use it.
What considerations should be taken into account when choosing between the two approaches:
Modify the code to permit multiple users
Install multiple instances of the code; one for each new user. E.G. completely duplicated, separate databases & PHP.
The software is PHP-based. We intend to offer multiple packages. Server load grows quadratically with increased use per license, due to large amounts of processing that occurs through scheduled cron jobs.
Update: despite the only answer suggesting we should not do this, we are still leaning toward modifying the code to permit multiple users. Does anyone else have any input?
Update 2: for the security reasons, we again changing our position to the multiple-instances solution.
Having done this myself in the last several months, My advice is don't do what we did, which is modify the code to permit multiple users. Turns out that's a rabbit hole and will introduce:
code complexity (adding new simple features will often become difficult)
bugs (due to increased complexity)
security problems (a huge amount of time was spent ensuring clients cannot access each other's data)
performance issues (tables with ~5,000 rows will suddenly grow to ~5,000,000 rows. Performance issues that weren't even noticeable suddenly created ~20 second page load times)
If we could do it again our approach would be something like:
Put each client on a subdomain (maybe even allow them to supply their own full domain name), allowing you to have a separate apache virtual host for each one. Buying a license to something like cPanel is worth serious consideration, and investigate how to automate or semi-automate creating new accounts.
Have a separate database for each client. Each with a different database password. This will provide excellent security and excellent performance (all the databases (and their tables) will be small).
It's up to you whether the actual php source code should be shared between all of these clients, or have a separate copy for each one. A global directory for the files is perfectly reasonable, and will make updates easy, while a separate copy will make customisations easier. Perhaps a hybrid is the right approach here.
One day we might even end up tearing out most of the work done in the last six months, to start again with this approach.
At first glance it seems like this will increase server load, but in reality if you have enough clients for load to even be a consideration, then you will want to be able to spread clients across multiple servers. And that's a piece of cake if it's well segregated.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I want to keep logs of some things that people do in my app, in some cases so that it can be undone if needed.
Is it best to store such logs in a file or a database? I'm completely at a loss as to what the pros and cons are except that it's another table to setup.
Is there a third (or fourth etc) option that I'm not aware of that I should look into and learn about?
There is at least one definite reason to go for storing in the database. You can use INSERT DELAYED in MySQL (or similar constructs in other databases), which returns immediately. You won't get any return data from the database with these kinds of queries, and they are not guaranteed to be applied.
By using INSERT DELAYED, you won't slow down your app to much because of the logging. The database is free to write the INSERTs to disk at any time, so it can bundle a bunch of inserts together.
You need to watch out for using MySQL's built in timestamp function (like CURRENT_TIMESTAMP or CUR_DATE()), because they will be called whenever the query is actually executed. So you should make sure that any time data is generated in your programming language, and not by the database. (This paragraph might be MySQL-specific)
You will almost certainly want to use a database for flexible, record based access and to take advantage of the database's ability to handle concurrent data access. If you need to track information that may need to be undone, having it in a structured format is a benefit, as is having the ability to update a row indicating when and by whom a given transaction has been undone.
You likely only want to write to a file if very high performance is an issue, or if you have very unstructured or large amounts of data per record that might be unweidly to store in a database. Note that Unless your application has a very large number of transactions database speed is unlikely to be an issue. Also note that if you are working with a file you'll need to handle concurrent access (read / write / locking) very carefully which is likely not something you want to have to deal with.
I'm a big fan of log4php. It gives you a standard interface for logging actions. It's based on log4j. The library loads a central config file, so you never need to change your code to change logging. It also offers several log targets, like files, syslog, databases, etc.
I'd use a database simply for maintainability - also multiple edits on a file may cause some getting missed out.
I will second both of the above suggestions and add that file locking on a flat file log may cause issues when there are a lot of users.