more an approach question, how to deal with the problem:
What methods, approaches, techniques, ideas or terms can be used for the following problem?
Several users should be able to access (read, write, recreate, edit) a web address book.
How do you handle the case that several users are working on one record at the same time?
E.g. editing or one user reads the record while another user changes it (e.g. phone number), then the reader doesn't even notice that his information is no longer correct and it leads to problems.
Are there any known (even new) approaches here?
Bit researched but many entries in forums are already many years old.
Many thanks.
Related
! Actually, I am learning PHP from last couple of months and now I am in a stage where I can program small things like a simple Login Page in PHP and mySQL or a Contact Form. I have wrote a lot of codeblocks like inserting something into a database or selecting something from a database etc. etc. But, I always copy paste my own code-blocks from previous projects while working on a new one. So, I want to know whether this tendency is unique to me only or each of the beginner passes through the same phase once during their journey of being a developer?
Please bear with me because I know this isn't really a programming question and doesn't worth your time as well. I tried finding out in Google as well but this is a snap of what I found:
I mean to say that most of the search results dealt with copy pasting other's code which is not the case of what I am talking about. In order to save time I do copy paste my own code blocks almost everytime. So, how bad is this behaviour of mine?
I again apologize for not posting a question that is worth your time but I am finding it hard to learn to code by myself without having any mentor nearby ( Actually, I searched for a mentor who could teach PHP before giving it a start all by myself, but I found none in my area ) for clearing my doubts and as such Internet is the thing which I mostly depend upon for learning about anything.
This question probably belongs on https://softwareengineering.stackexchange.com but I'll try to give you a decent answer and some guidance.
People re-use their own code all the time. You do not however want to copy/paste if possible. The issue with copy/paste is when you have something used more than a few times - like a MySQL database connection - and it needs updating. I'd rather modify one file (or one small group of files) and have all of my webapps fixed/updated than having to modify 2 or 3 database calls in 9 different web apps...
For things that I use everywhere/all the time - talking with our course management systems API, authenticating a user against our LDAP server, connecting to a MySQL database and running queries, processing forms that are emailed, etc - I've built up my own (or coworkers have) sets of functions, classes, etc. Which I then keep in a single directory, and can include as needed.
If you do this, you want your functions/object methods to be as generic as possible - for example, my MySQL query function takes several arguments - an associative array with connection info (since we have several DB servers based on purpose), a query, and an array of parameters. It returns an array with a status code, and then appropriate data - the record set result for inserts, the ID of the last insert, the count of rows affected (for delete/update). This one function handles 50+ queries and connects to 4 different MySQL servers.
Showing user data on a page by query:
$query = "SELECT * FROM COLLECTIONS WHERE uid = $_GET['user_id']";
But problem is that user can see other users data by changing that uid.
How to solve this problem.
Take your website offline. NOW. Somebody is going to either wipe the data or steal the data or inject malware that's served to all of your customers
Breathe. You've bought yourself some time, assuming it hasn't already been breached.
A small subset of the security measures you NEED to take
These mitigate, in order of "has biggest immediate benefits" to "is probably most important", one problem each. (Apart from number 3, which mitigates anywhere from 4 to 32241 problems of equal or greater magnitude to number 1.)
Look through every instance of every database request, and make sure that you are never using double quotes or the . operator when defining your query string. Rebuild all of your database handling code to use some sort of parametrised SQL query system.
Use an authentication library, or at the very least a crypto library.
Ask about your setup on Security Stack Exchange using an account that is in no way traceable to your website. Not even to your company, if your website is associated with your company.
Why?
Yes, I know, that website is probably important and needs to stay up so people can use it. But try this:
www.badwebsite.com/your/page/here?uid=1 OR 1
All of the data is visible! You are accepting code from the user and running it in your database. Now what if I decided to delete all of your database tables?
That's just covering the first point I made. Please trust that there are bigger problems for your users if you haven't done step 2, the least of which is hundreds of their accounts on other websites (e.g. Gmail, Example Bank) becoming known to cyber criminals.
Take a look at this comic strip:
There's also a unicode-handling bug in the URL request library, and we're storing the passwords unsalted ... so if we salt them with emoji, we can close three issues at once!
This is made to be more funny, but the problem described in this comic strip is probably less bad than the problem you are facing. Please, for the sake of whoever has entrusted you with their data, turn it off for a few days whilst you try to make it something resembling secure.
You might want to bring in a technical consultant; if your developers are not experienced in creating intrusion-proof software then they're probably not up to the task of making insecure software secure (which is orders of magnitude harder, especially if you're new to that sort of thing).
This question already has answers here:
Is it unreasonable to assign a MySQL database to each user on my site?
(10 answers)
Closed 8 years ago.
I have a small file hosting website that I am trying to store the files that users upload in a database. The issue that I am having is that I cannot decide which method would be better:
To store all users in one table
create a new table for each user.
I understand that the second method will slow performance but by how much? I am planning on having 1000+ users eventually. The issue with the first method is listing the files back to the user. What method should I go with and which one would be the most efficient?
Short answer:
No. Use the simplest thing that works: A single table.
Long answer:
You'll know what kind of scaling problems when you have a production system under production loads, and then you can analyze where your bottlenecks are and develop a sharding strategy based on real-world use cases and not hypotheticals.
Right now you're just guessing, and you'll probably guess wrong. Then you're stuck with an awful database structure you'll find impossible to undo.
Try not to store actual files in the MySQL database, this almost always leads to horrible disaster, but instead store them on the filesystem and keep references to them in the database. If you're going to be managing a lot of files, heaps and tons of them, you may want to look at document store database like Riak to help with that.
I suggest creating a table for each entity and having a correct relationship between them.
For example:
Users table will have user_id, user_name, etc.
Files table will have id, url, user_id
In this case, the relationship is created by having the same user_id. So when you upload a file, you attach the user_id to the image.
That means - no, don't create a separate table for each user. Go with method 1 and make sure each important entitiy has its own table.
Down the road, you will probably have more entities and more tables such as Permission, Products, etc etc. By using SQL queries you will be able to get all the data you want.
Hope this helps!
Having 1000 ish users is not a problem for MySQL. But tadman is rigth, save the files on the filesystems instead of the database.
If you know that you will endup with millions of users, I suggested that you read on how Facebook or others big users related sites handle this scalling problems.
Need some ideas/help on best way to approach a new data system design. Basically, the way this will work is there will be a bunch of different database/tables that will need to be updated on a regular (daily/weekly/monthly) basis with new records.
The people that will be imputing the data will be proficient in excel. The input process will be done via a simple upload form. Then the system needs to add what was imported to the existing data in the databases. There needs to be a "rollback" process that'll reset the database to any day within the last week.
There will be approximatively 30 to 50 different data sources. the main primary interface will be an online search area area. so all of the records need to be indexed/searchable.
Ideas/thoughts on how to best approach this? It needs to be built mostly out of php/mysql.
imputing the data
Typo?
What you are asking takes people with several years formal training to do. Conventionally, the approach would be to draw up a set of requirements, then a set of formal specifications, then the architecture of the system would be designed, then the data design, then the code implementation. There are other approaches which tend to shortcut this. However even in the case of a single table (although it does not necessarily follow that one "simple upload form" corresponds to one table), with a single developer there's a couple of days work before any part of the design could be finalised, the majority of which is finding out what the system is supposed to do. But you've given no indication of the usage nor data complexity of the system.
Also what do you mean by upload? That implies they'll be manipulating the data elsewhere and uploading files rather than inputting values directly.
You can't adequately describe the functionality of a complete system in a 9 line SO post.
You're unlikely to find people here to do your work for free.
You're not going to get the information you're asking for in a S.O. answer.
You seem to be struggling to use the right language to describe the facts you know.
Your question is very vague.
There are some other SO questions about concurrency but they don't quite address my scenario.
So let's say that I have a game where users interact with each other, fighting and whatnot. At any given time, a player could potentially be involved in multiple interactions with other players, all of whom can see the event happening. When any one of these players hits the site, it needs to update any data involved and show that to the user.
Example situation: Player A is fighting with player B, and events happen every few minutes in this fight. At the same time, player A is also interacting with player C. By dumb luck, the events for both interactions happen to next be due at the exact same second.
When that second arrives, by dumb luck again, both player B and player C hit the site at the same time, in order to check the status of their fights with player A. Fighting requires updates to information about player A. If I don't code properly, A's data can get messed up.
I have two games with this situation, each with a different solution and different issues. One of them uses a lock, so when a user hits the site, they acquire a lock on a db row, read the data for locks they successfully acquired, then write the changes and release the lock. But sometimes, for reasons still unknown, this fails and the lock gets stuck forever, users complain and we have to fix it manually. My other game uses a daemon to execute these transactions, making the issue (nearly) moot as there is only one process ever making these changes. But players could still do other things at the same time, and potentially cause the same issue.
I've read a bit about different solutions to this, like optimistic or timestamp-based control. I would like to ask:
Which of these is most commonly used for situations like mine, and which is easiest to implement?
My next project is using Kohana (PHP) and its ORM, so my db writes will by default take the form "just overwrite all these fields." Will I need to write my own update queries for this or can I get a solution that is compatible with the ORM?
What about transactions that involve multiple tables? The outcome of a combat has to change the table of combats, and the table of player information, possibly more things too. Which solutions are easier to work with here? Will all of my tables need transaction timestamp columns?
A lot of these solutions say that when there is a conflict, either retry or ignore. What does this mean for me? Does "retry" mean restart my entire script, which would cause additional load time for the user? I don't think ignore is a valid option, since the events have to execute at some point. In the other questions I found, presenting a conflict error to the user was usually a valid option - for me, it isn't.
What are the performance implications of concurrency control - is it even worth it?
I think what you are looking for is already contained in your question : transactions.
If you are using MySQL, you will need to setup your tables with the innoDb engine to be able to use transactions. Some documentation :
http://dev.mysql.com/doc/refman/5.1/en/commit.html
http://www.php.net/manual/en/pdo.begintransaction.php
http://www.php.net/manual/en/mysqli.autocommit.php
Don't try to reinvent the wheel when you can.