Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I need some info on this subject. I've searched around a bit but it seems that it really depends on your situation. My situation is explained below:
We have developed a system where in a company can keep track of their projects and financial situation. They can create orders, divide tasks between employees, send invoices, check if they are paid, etc.
Currently we have 1 domain with 1 database with all the data for this company. We would like to use this system for other company's as well, but on 1 domain with the same files for every company. So we can maintain the files on 1 place and keep everything on our own server.
We want to use multiple databases for the following reason's:
We want the files to be in 1 place, easier to maintain and update
A client can't have acces to another clients financial data by accident
We can make induvidual backups of clients data
Downside's in my opinion are:
If something in a table needs to get updated you have to do that manually in every database
Could MySQL get really slow after 100+ databases?
Am i correct, and are we doing the right thing by giving every Company an induvidual database?
Thanks in advance!
There is technically no limit to the number of databases you can have. A brief search shows a few people have into the 1000+ databases, I don't see a problem with 100+ databases
We want the files to be in 1 place, easier to maintain and update
As you already mention under downsides, what if an update were to require a modification to the database's schema? Having hundreds of databases would be just as problematic to maintain, versus a single database (with client indicator columns in the relevant tables).
A client can't have acces to another clients financial data by accident
But clients can only access the data through your webapp. If that becomes compromised, by accident or otherwise, what is to stop it accessing other databases any moreso than unintended records in the same database?
Views could provide similar security benefit (albeit currently with some performance cost). However, I tend to create stored procedures and force my apps to perform all database actions through them, wherein I can perform my own security checks whilst limiting all database access to only predefined operations.
We can make induvidual backups of clients data
One could still make selective backups e.g. with SELECT ... INTO OUTFILE.
Sometimes it's better to use an own database for each company. Never in theory but often in practise.
Sql commands are cleaner and easier to write.
It's more secure. The companies can't accidentally have access to each other's database if an sql query or a script is faulty. You just have to select the database carefully.
Later if the database gets too busy it's very to separate databases to multiple servers. Big databases or tables may be difficult to split.
The tables stay smaller so the queries are also faster.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed last year.
Improve this question
Currently, I am working on a website and just started studying backend. I wonder why nobody uses JSON as a database. Also, I don't quite get the utility of php and SQL. Since I could easily get data from JSON file and use it, why do I need php and SQL?
ok! let assume you put the data in a JSON variable and store it in a file for all your projects.
obviously, u need to add a subsystem for getting back up, then you will write it.
you must increase the performance for handling a very large amount of data, just like indexing, hash algorithms, and... , assume u handle it.
if you need some API for working and connecting with a variety of programming languages, u need to write them.
what about functionalities? what if you need to add some triggers, store procedures, views, full-text search and etc? ok, you will pay your time and add them.
ok, good job, but your system will grow up and you need to scale it, can you do it? u will write abilities for clustering across servers, sharding, and ...
now you need to guarantee that your system will compatible with ACID rules, to keep atomicity, Consistency, Isolation, and Durability.
can you always handle all querying techniques (Map/Reduce) and respond with a fast and standard structure?
now it's time to offer very quick write speeds, it brings serious issues for you
ok, now proper your solutions for condition racing, isolation level, locking, relations and ...
after you do all this work plus thousands of many others, probably you will have a DBMS a little bit just like MongoDB or other relational and non-relational databases!
so it's better to use them, however, obviously, you can choose to don't to use them too, I admit that sometimes saving data in a single file has better performance, but only sometimes, in some cases, with some data, for some purpose! if you know what exactly you do, then ist OK to save data in a JSON file.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
All the answers about this question assume you're storing all of your user's data in one big file - and so they talk about how that is too slow.
Let's say I have thousands of users and store their data as JSON format in separate files (which I am currently doing), what is the downside to that - as opposed to setting up a proper database like Postgresql - which seems like overkill.
The speed is great on my current setup, but I am advised against doing this.
Since each user has their own separate file, there isn't really an issue of hundreds of people writing to the file at the same time (isolation).
Maybe it only matters for sites with millions of users?
In most systems, the users don't merely have to exist, they have to do stuff. And that stuff would generally be represented in a database. So you want the users to exist in the same system where the things they interact with exist.
What happens if your system crashes (power failure, for example) when a json file is half-way written out? Will you be left with a broken JSON file for that user? With databases, that should be taken care of automatically (you find either the old record, or the new one, not some truncation or mishmash). If you roll your own database, you will have to go some way out of your way to verify that you do this in a safe manner.
How do you name your user files? By the user's name? What if different people have the same name? What if their name has characters that can't be represented in file names? By an account number you assign? What happens if they forgot their account number and need to look it up by their human name? Do you then need to read and parse every user file to identify the correct one? Not that a database will magically make this free, but at least with a database you can just build an index with first having to invent and implement them.
You are basically reimplementing a database system from scratch, one feature at a time, as you discover the need for that feature. You can do it, sure. But why not use one that already exists?
Since each user has their own separate file, there isn't really an issue of hundreds of people writing to the file at the same time (isolation).
What if one person writes to one file at the same time from two different browsers (or tabs)?
There is no absolute right or wrong.
If you will never need to take care of concurrent access to the same record (file) or you don't need to search through your records or scale to multiple servers, the solution is fine and even faster than accessing a database.
I would just recommend to properly escape the user provided data, as JSON
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I've built a school management system for my own needs. The size of tables ranges from ~200 rows to ~30,000 rows /at the end of school year/.
Some of my friends have seen the system and they urge me to make it available to other schools. I'd like to give it a try at least with a few schools for now. Considering my current architecture and shared hosting I'd have to store all schools in single db and so 2 questions bother me:
Can MySql easily handle tables with >300,000 rows?
The system is based on Yii2 at the moment, I've optimized it for max performance - do you think it's wise to try or better work towards solution with a dedicated server and separate db for each school?
I don't know if it's wise to store all students, attendance, payments etc info from my and 10 others schools across shared tables in single db. I'd better ask than cause trouble to myself..
Any advice is more than welcome:)
premature optimization is the root of all evil (or at least most of
it) in programming
You should not worry about this at the moment. Start running your application and as you scale, identify the bottlenecks and then try to figure out a solution for it.
Can MySql easily handle tables with >300,000 rows?
First things first, use the best normlization principles to structure your tables and relations. MySQL is pretty good at handling rows up to 10,000,000. but it also depends on how you are indexing/querying the data. Use proper db indexes on the columns you frequenty use for lookup. A big no to "like" queries but if you must, then use a search engine like (elastic, solr).
The system is based on Yii2 at the moment, I've optimized it for max
performance - do you think it's wise to try or better work towards
solution with a dedicated server and separate db for each school?
I have a very little idea about Yii2, but certainly there are much better frameworks available in php which you can give a try eg. larvel (this will give you a better idea). Ofcourse, best would be to host this application on a dedicated server. why to waste money when you can have private VPS just in 5$ from digitalocean.
I don't know if it's wise to store all students, attendance, payments
etc info from my and 10 others schools across shared tables in single
db. I'd better ask than cause trouble to myself..
There is absolutely no problem storing students, attendance, payments info in the same database, just structure your tables properly.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I have a question for people who have experience working with Oracle and PHP. Please this is not to open a meaningless debate. I just want to make sure everything is considered correctly in my system.
I am working on a project in which there are thousands of users, divided into groups and sub groups. Each group has its different access rights and each subgroup has its own privileges.
I need to have your opinion about these two approaches:
Implementing access rights and privileges in PHP with one big
application user(oracle account),(I am clueless as to the advantages
and disadvantages of this approach).
Implementing access rights and privileges in Oracle database(each
user would be an Oracle account) and use the virtual private
database, caching, secure roles.... from a performance stand point
this is the best approach. Security! well I know it is good but I am
afraid I am missing good things not implementing it in PHP.
I did some research on the net but in vain(I scratched my head a lot). I am new to PHP but I have good knowledge about Oracle.
Any suggestions, Ideas?
As you say you're going to have 1000s of users, i assume your software is going to be used in a big company, which probably means there's not one IT department, but several of them - one providing managed basic hardware (OS level but no applications), another managing databases, and a third one putting it all together (hardware+os, database, application) and providing the managed application as a service for the end user. My decision might be heavily influenced by working for such a department for over 10 years now.
I've seen one application that used the "one database user per OS user" approach (VPM by Dassault Systems, an attachment to the Catia V4 CAD system - and it's a horror to maintain. You want to have your permissions in ONE place, not several. Possibly you have a nice GUI to change them, but when your user calls you saying "i can't do X", and your GUI says he should be able to do X, it's simply too tedious to search everywhere. We've had cases where views didn't have the access roles they should have, views were wrongly defined, users had permissions on some tables but not all of them, all that stuff.
Additionally, our database department has - at the moment - about 600 "databases" that are used by diffent departments. So they are running about 20 real "databases" on several clusters, and they have implemented quite a rigid scheme of database names and corresponding user names. Each database has a base name XXX, with XXX_A the user name DDL statements, and XXX_U for DML. Applications may ONLY use XXX_U, which also means applications may not do DDL. This allows the database admins, in case of load issues on the cluster, to move an entire schema, including all users, roles and tables, to a different instance on a different cluster easily, without knowing too much about the individual databases. Of course, our VPM database doesn't fit into that schema, so we had to argue with the DB people a lot - and our monthly charge by the DB department is much higher than normal, because they have much more trouble administrating it.
Bottom line: Don't expect you can do whatever you want within your database. In a large company, with a department managing the databases, you will have restrictions what your application is allowed to do and what it isn't.
Also, your management might, at one time, decide they want to move to a different database system like DB2 for political reasons. This has much less to do with technical advantages than with who's invited whom to golf. You might, at one time, be asked if your system could be moved to a different database, by people you don't want to say "no" to. I wouldn't want to be dependent on too specific oracle features in this case.
Also keep in mind that requirements change over time, and there might be new, more granular, requirements in a while. This strongly favours doing the permission stuff in software, because it's much easier to add another column to a permission table that specifies something new, instead of having to implement something new in a database permission structure that just isn't meant to handle this kind of thing.
If you were developing a native application that runs on individual users' PCs, using only one oracle account might be a big security hole. But as you're using PHP, it's only the server that's communicating with the DB, so noone can extract login information from userspace anyways.
In my opinion, create an api for permission management first. Do not use oracle users, groups and roles for that; instead, manage your permissions in some SQL tables. Create an api (a collection of check_if_user_X_may_do_Y functions), possibly in pl/sql if you feel more comfortable there, better in PHP if you want to be portable. Build your application on this API. It might be more dev work at the start, but will result (imho) in much less administration work later.
Although Guntram makes some very salient points, he has missed what I consider to be fairly essential considerations:
1) you describe a 3 tier model of authorization which the Oracle permissions model does not accomodate (although it is possible to represent this as a 2-tier model but at the cost of creating work and complexity).
2) using the user supplied credentials to authenticate against the database pretty much precludes the use of persistent database connections - which is rather important when your connection setup time is as expensive as it is with Oracle.
By all means store the representation of the groups/sub-groups/users/permissions in the database but not as Oracle users. But use a single (or small number) of oracle accounts for the access from PHP to the DBMS.
If you want to implement strong security, then you should also store mappings between sessions ids and users (not necessarily the session data itself) in the database, allow no direct access from the designated Oracle accounts to the data - only via stored procedures - and pass the session id as an authentication token to the stored procedure and validate it there (but note that this will be rather expensive if you are using trivial ORM).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I want to keep logs of some things that people do in my app, in some cases so that it can be undone if needed.
Is it best to store such logs in a file or a database? I'm completely at a loss as to what the pros and cons are except that it's another table to setup.
Is there a third (or fourth etc) option that I'm not aware of that I should look into and learn about?
There is at least one definite reason to go for storing in the database. You can use INSERT DELAYED in MySQL (or similar constructs in other databases), which returns immediately. You won't get any return data from the database with these kinds of queries, and they are not guaranteed to be applied.
By using INSERT DELAYED, you won't slow down your app to much because of the logging. The database is free to write the INSERTs to disk at any time, so it can bundle a bunch of inserts together.
You need to watch out for using MySQL's built in timestamp function (like CURRENT_TIMESTAMP or CUR_DATE()), because they will be called whenever the query is actually executed. So you should make sure that any time data is generated in your programming language, and not by the database. (This paragraph might be MySQL-specific)
You will almost certainly want to use a database for flexible, record based access and to take advantage of the database's ability to handle concurrent data access. If you need to track information that may need to be undone, having it in a structured format is a benefit, as is having the ability to update a row indicating when and by whom a given transaction has been undone.
You likely only want to write to a file if very high performance is an issue, or if you have very unstructured or large amounts of data per record that might be unweidly to store in a database. Note that Unless your application has a very large number of transactions database speed is unlikely to be an issue. Also note that if you are working with a file you'll need to handle concurrent access (read / write / locking) very carefully which is likely not something you want to have to deal with.
I'm a big fan of log4php. It gives you a standard interface for logging actions. It's based on log4j. The library loads a central config file, so you never need to change your code to change logging. It also offers several log targets, like files, syslog, databases, etc.
I'd use a database simply for maintainability - also multiple edits on a file may cause some getting missed out.
I will second both of the above suggestions and add that file locking on a flat file log may cause issues when there are a lot of users.