An online application we are building (php & mysql) requires users to be able to create their own forms for data capture and record this data in a database, respecting the existing ORM's.
If the forms where "hard coded" then we would simply set the db tables up to store the normalised data ourselves however as our users define the form fields contained in the forms, we're not sure what is the best way to proceed to implement this functionality.
Do we need to think about some kind of meta data or data abstraction layer for our DB? Google hasn't been too much help as we're unsure about how we need to go about this.
Any pointers in the right direction would be gratefully appreciated!
Many content management systems address this problem in different ways.
For example, in Drupal, users can create their own custom content (with custom forms) through the CCK module. The module defines different types of fields that the user can create, then generates tables with specific data types to store the data.
Some tips:
Define your field types - Think about giving the users a choice of different field types (e.g., select box, string, radio).
Create tables for user defined fields - Each field type will have a specific SQL data type. Define a table using these data types. For example, a select box might be mapped to an enum and a input text element might be mapped to a varchar column.
Add data to the new tables - use the new tables to store the data in a somewhat normalized way.
Obviously there are many different approaches, but these are just a few suggestions.
I think I've found a solution to my problem, so for all those people who come along a similar problem have a look at the following artcles -
http://www.adaniels.nl/articles/an-alternative-way-of-eav-modeling/
http://weblogs.sqlteam.com/davidm/articles/12117.aspx
Hope this helps.
Related
I am developing a plugin in WordPress that is about to store configurable entities. They are jobs to be done by Cron. There are plenty of them and each one has name, frequency and some additional data.
There are discussions how to store post-related data in plugins, whether to use postmeta table or own tables. It is officially adviced to use postmeta whenever possible. But what if those entities are not posts? I have 3 possibilities:
Use separate table(s) to store them
Use options table
Make those entities posts with custom post type
I used to keep this kind of information in options table, but how would you do that?
Basically, when you are trying to decide if you need a custom table or not, answer a simple question: does my data need indexes and fast searching? If yes - go with custom tables. If not - e.g.:
this data is being read from database with the post itself - use postmeta
this data is used rarely and is called directly by keys - use options
Else, if you need to store a lot of structured data, perform selects, sorts, or even joins etc - custom db table is your option. Read the docs and do it using core functions and features, like db_delta() function etc. There's nothing wrong in using custom db tables. Just do it right, the "WordPress way".
We are working on a platform which allows the user to create 'promotion' instances, wherein there are an arbitrary amount of pages and 'modules' associated with those pages. Each module has its own customizable collection of attributes. One of the modules I am developing is the entry form, which is the main component.
The form includes some default fields e.g. name, date of birth and email address. The module then allows the user to add as many additional fields of any type that they require (e.g. a '25 words or less' text field, extra opt-in checkboxes, etc).
I'm trying to plan out how I will deal with these X additional fields in terms of storage in MySQL. Sorting and filtering on these fields will be a requirement.
This seems like something that would be a known and solved problem, but I haven't had any luck either wording it correctly or coming across relevant information. I've had some thoughts while searching; but each has a downfall that makes me think there must be a better way:
Create a new table for each form module which contains the additional fields as new rows - this seems really messy / clunky.
Store the additional information in an extra row as JSON (or some other data format). Pull all the data into PHP, expand the JSON and work with all the data in PHP - we envision a high number of entries (5-10k) so I assume this would be too inefficient.
Having an upper limit on additional fields and appending a bunch of rows to the entries table i.e. 'custom1', 'custom2', 'custom3', etc. This seems very messy as well.
Looking again at point 2, I thought there might be a way to take the block of data in the extra row and create a derived table from it, but I haven't had any luck finding information around whether that's possible. For example:
SELECT * FROM( JSON_DECODE(entries.extra) ) ...
If this were possible, that would probably be my preference.
What is the correct way to approach this problem of needing a dynamic amount of additional rows?
The correct solution depends a lot on how you're going to use the data. All solutions have strengths and weaknesses, and you have to understand that you're basically trying to do something that relational databases were not designed for.
I gave a presentation about this topic earlier this year called Extensible Data Modeling, in which I tried to provide a survey of different solutions, and their pros and cons.
You could also throw in the towel, and use a non-relational database to store non-relational data.
I have been tasked with creating an application that allows administrators to alter the content of the user input form (i.e. add arbitrary fields) - the contents of which get stored in a database. Think Modx/Wordpress/Expression Engine template variables.
The approach I've been looking at is implementing concrete tables where the specification is consistent (i.e. user profiles, user content etc) and some generic field data tables (i.e. text, boolean) to store non-specific values. Forms (and model fields) would be generated by first querying the tables and retrieving the relevant columns - although I've yet to think about how I would setup validation.
I've taken a look at this problem, and it seems to be indicating an EAV type approach - which, from my brief research - looks like it could be a greater burden than the blessings it's flexibility would bring.
I've read a couple of posts here, however, which suggest this is a dangerous route:
How to design a generic database whose layout may change over time?
Dynamic Database Schema
I'd appreciate some advice on this matter if anyone has some to give
regards
SWK
I created a very large EVA database years ago (PHP w/ PostgreSQL). It turned out great, but it was large project ($$$). All the forms were completely dynamic, with form/field versioning, publishing workflows, matching dynamic reporting, etc.
EVA basics are easy enough. Getting data in is not the hard part. But form versioning and reporting....you can spend years getting it right.
If I was doing it again today, I would research using one of the newer NoSQL solutions ( http://en.wikipedia.org/wiki/NoSQL#Document_store ). I'd thing about creating a DTO style class that could be passed to a form generator. "Modifying" the form, would actually be modifying the DTO. Then I would persist that DTO into a document/object database.
Also, as you are building your alpha solution, think of how to solve test cases that encompass versioning and reporting needs.
Here is an example of what I mean: A simple "Ask Question form".
Original (version 1): has First,Last,Question
Add email field(Version 2): First,Last,Email,Question
Somebody changes their mind about email: (version 3): First,Last,Question
New marketing guy comes in and changes it: (version 4): Full Name,Email,Question
Now, you need to generate a report (csv). Things get tricky. How do you do it?
We solved this problem with field level versioning with references to their previous versions. Plus the reporting system required the end user to assemble the definition of the report data sources before running. (binding report fields to data fields, etc).
However with the document DB's I'd imagine you can do it differently. I believe the new DB's like CouchDB (others??) have mechanism built in for handling these issues.
Good luck!!
When developing user profiles in my last webapp, I've chosen Key/Value table approach. Here's how my DB design looks:
Users table with fixed columns:
id
login
name
regdate
Users table linked with Profiles table (User HasMany Profile).
Profiles table with different data:
user_id
field
value
This way user can add any additional field to his profile. For example:
user_id = 1
field = 'Facebook'
value = 'http://facebook.com/...'
and
user_id = 1
field = 'Stackoverflow'
value = 'http://stackoverflow.com/user/...'
and so on..
Depending on your needs, it might not be worth even raising the form fields to the "DB fields" level. You could instead serialize these fields in (what is essentially a) dynamic blob and store it in the DB. This is NOT recommended if you have folks who need to query these dynamic fields outside of your app (i.e., the DB design is part of a larger public contract with integrated systems), but if you're just using the app to simply to persist these dynamic fields or if any aggregation/search capabilities within the fields are minor, then I would consider it (esp given CPU capabilities these days). I have used the pattern many times and I have - thus far - never had to refactor. (however, I can understand a case where you might need to).
I'm having some trouble deciding which approach I should take. I want to allow users to create their own html forms by choosing different form elements (textfields, textareas, lists, ratios, etc) I guess something similar to http://wufoo.com/ but much more basic. Should I use database tables or create files?
For tables I was thinking to create a table for each form element, e.g
TEXTFIELD TABLE
ID
TEXTFIELD_NAME
USER
...
TEXTAREA TABLE
ID
TEXTAREA_NAME
USER
etc for all form elements
and then just query all of them with..
WHERE user=$user
Or should I generate php files on the server for each form created?
One approach I used for custom forms on a mobile device was to define a table for field types (eg: text, date, money, custom), a table for form templates (with owner, version, and XML for the template data), and a table for form values (as XML) with a foreign key to the template.
Someone may create a form template with the available field types, and the collected data may be stored for that template, and optionally by template version. If the database supports it, you could query into the XML data to aggregate fields across forms.
For presentation, you could use a fixed style, or transform the template or collected fields however desired.
Should I use database tables or create files?
Storing this stuff in a database has the big advantage that you can easily edit them programmatically, and/or populate the user interface used to edit them.
A generated HTML structure you would have to parse first.
So definitely a database.
I'm currently researching a project for the place that I work in. We are trying to create a system that will allow forms to be set up dynamically from a database.
My question is what database structure would best suit something like this?
I currently have a structure of:
forms_form
forms_formfields
forms_formdata
I don't think this is the most appropriate layout for this. Basically to make is make sense I need to be able to make a form within the database that can have infinite fields all customized and have the data when submitted stored in the database.
The proposed structure looks ok to me. form -> field is clearly 1:M relation, so you'll need forms(id,...) and fields(id, form_id, type, ...). field->data is kind of 1:1, so theoretically a separate table is not needed, unless you allow one form to be used with different sets of data. If this is the case, i'd suggest datasets(id,...) and data(field_id, dataset_id, value).