Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
While I can find the answer to almost any specific question I have concerning PHP + MySQL development, one resource I haven't been able to find is good solutions to advanced problems.
I want to implement a minor messageboard type system that can be queried by AJAX, but it I have too many specifications for it to be filled by a prebuilt library, and I don't need a tutorial about how to implement and use a database.
What I'm looking for is a resource that provides good solutions to advanced users for more complicated problems than "store a message and print the last 10 messages"-style tutorials.
For example: My website will have group pages each with any number of users, and the private group page will have a small "wall"-like message board. What should I take into consideration when designing this particular table? How should I implement locking? Etc.
Of course, I don't expect there to be a tutorial for my exact problem, but I would like perhaps a complete solution to a db-driven website that can be comprehended (unlike WordPress - a little too spread out) and that actually works (unlike the full-solutions you sometimes find at the back of a reference book).
I am a .net guy so I will stay generic is my solution. I don't think the solution you need is language specific any ways. What you need to look into is design patterns and enterprise methods for doing certain things. The key for your success doesn't come (entirely) via database implementation but what and how you use the data. To scale you need to be able to write quickly (write to a queue instead of to the db) and read quickly (read from a cache layer when possible instead of the db) and search quickly (search from a Lucene index instead of the db). Many people get hung up on the db and they are generally correct in doing so. It is the central part of your applications data storage and querying. But it is also usually one of the single biggest bottle necks in any system. Sure, store all your data to the db, just don't do it directly. Sure, read and query your data from the db...but only when absolutely necessary (use an index to locate the data, then just read the data from the database).
The same goes when having your application speak with external services. Take the sending of an email for example. Rather than sending the email by connecting to the SMTP server and packaging and sending the email...stick the message into a queue and create a queue reader to connect to and send the email via SMTP. This way your web app continues running smoothly.
The key to your success is performance oriented research and good architectural design. Look up things such as domain driven design, inversion of control, test driven development, repository pattern, model view controller, memcached, velocity, queue, MSMQ, database mail queue, etc.
You'll find that all of the things you mentioned (presumably as "advanced issues") are all a series of rather simple approaches and technologies built together in a thoughtful way.
Locking a table to keep multiple users from editing simultaneously is a rather simple issue. The issue of locking has been addressed here numerous times.
As for assigning members to groups, that's just a man-to-many relationship between a few tables. You'll have your Users Table, a Groups Table, and a Users-to-Groups table. That's assuming a user can belong to multiple groups.
By doing a couple small joins in your query, you can get posts for a specific group from various different users - and thus fill your group-specific board. Furthermore, you can privatize that board by requring your user exist in that group in order to read that board - this is just a quick query of the Users-to-Groups table to see if that UserID exists alongside that GroupID.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I don't know, if this type of question is already asked or not. Actually I don't know what to search for. Am asking at the right place?
Just as an example, I always wonder how the social media giants like Facebook manages their user settings module... What would be the database design and how they manage to hide the user updates on his friends' timeline if he has chosen not to show his updates on that particular friends timeline. I mean if I had been programming there then I would have loaded all the settings value in an array and there would be many conditional statements to check each and every user setting and accordingly printed data.
But I think this would make that code unmanageable because there would be so many conditions which could lead to undesired results.
So my question is, is there any better approach to do this?
I don't know I am making any sense here, but I tried to explain my question.
Facebook's data is maintained in document repository (Nosql) and efficient indexing is used to quickly find the tags and searches. This approach of search and data storage is markedly different from relational database based data storage and search.
Google also uses similar scheme to map the entire web and promptly give you back the result.
So in simpler terms you data is stored and indexed the way Google indexes messages, only difference is, the data is also lying with Facebook.
The related technologies are Bigdata, Mongodb, Apache Hadoop. And one of the leading index management and search algorithm is Lucene. Apache Elasticsearch is an user friendly package around Lucene.
So facebook treats these security critaria like tag (in simple language) and does google like search and presents you in a pleasing frontend, not sounding like a search engine.
While setting up your system, you can use elasticsearch to have faster search. Elasticsearch is makes implementation of lucene easier. It definitely will have some learning curve. Elasticsearch can also be used along with rdbms, in this case your data is saved in database but indexes also maintained to faster search. Definitely the cost would be disk-space. It makes it possible to have many criteria but still being able to get result quicker.
A quick tutorial on elasticsearch.
There would be many conditions to evaluate, that is correct. But in a SELECT statement you can easily compose all of those conditions in a WHERE clause which is very efficient.
Essentially, as long as you're comparing on equality, the database can easily optimize that, allowing it to quickly search for posts that fit the desired constraints. Even though there are a lot conditions, they don't really affect performance when compared to the fact that there are millions of entries in a table to be searched.
What your asking for is a result of really tough planing.. whenever you need to develop something that has a good potential to be complex you'll have to plan (Engineering) it well using known methodologies.
Usually the DB has many polymorphic relationships with entities, there are guys who are responsible of writing Query Procedures that should retrieve the wanted data fairly for the developers.
It's really not something you could come up with easy solution, the key here is planning, and planning good. there's no one right answer.
If your application is fairly small, you could just implement it your way, then you'll see what can be upgraded.. It's pretty much your only way to go. (BTW that's what most of statups are doing)
I wish you the best of luck.
Regarding facebook's db schema's and how it works and why its a good design, here are some articles that would explain to you why:
The power of the graph
This is posted by facebook and it explains how they are managing data. They use TAO data model and through the application of graph theory and other complicated algorithms and advanced memoray caching and data handling, they can efficiently manage lots of user data..
but regarding to your question: What would be the database design and how they manage to hide the user updates on his friends' timeline if he has chosen not to show his updates on that particular friends timeline?
I think this post would give you some insights on what kind of db structure facebook has and what would be the functionality of it for every user: Social Network Friends Relationship Database Design
Usually, the hiding of user updates on your friends' timeline if you have not shown your update to that particular friend is managed by storing values in the database.. you can create a view_type table in db and that would determine what kind of view the user can see, then issue a where condition in your sqls based on the view the user has selected.. there are still many ways to handle this and a good database structure is needed for this and of course planning for a good and efficient database is a very important and strict procedure..
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Greetings,
I would really love to hear about your broad knowledge about application design on this one!
I'm new to Phalcon, and now building my first MySQL & RESTful-API based application with it, but encountering some challenges along the way.
First, about my application design, the concept is as follows:
An API as a core for the application, wrapped by a "UI shell" of pages and views that utilize it.
The API is supposed to be composed of a set of Phalcon models that represent the DB and business logic, and over them, a component that acts as a layer that makes those models accessible as "HTTP services" - generally by translating requests to model names, and the HTTP verb to the appropriate model action (e.g: GET => $account->find()/findFirst(), PUT => $account->update([params]), etc.).
I was sure that the Phalcon models are gonna rid me of having to write most of my SQL, however, soon I came across some pretty common scenarios that the models couldn't handle the way I would expect:
You have entities like messages for example, and you wish to query them using a column of some other, related entity (like the FIRST NAME of the user that owns those messages). A model can't do this in a single operation.
I want to show a list of messages, each attached with the details of the user that sent it. In Phalcon, the first thought that comes to mind is taking advantage of the model relations feature, but thinking further I came to realize that will perform a full query for every message rendered, which is a disaster performance-wise, rather than retrieving them all together with their user details in some single, JOINed query.
I want to show a list of users, each with a total messages count. Found no other way to achieve that rather than a full query that includes a COUNT() field & GROUP BY or a sub-query.
I tried to look up such use cases and others, for most of them there hasn't seemed to be any elegant solution.
The Point:
What I want to achieve is a API-based architecture that makes sense, scalable and easily customizable to real world scenarios (being able to handle obvious situations like demonstrated above).
What would you do in Phalcon to handle the problems I encounter?
What do you think about the design concept I took? Is it somewhat standard and makes sense?
Most importantly, how would you design a full & flexible API without repeating cumbersome SQL queries everywhere, if at all?
Do you have any references or examples of known companies' approach on this?
That's a big question and any help will be huge!
Thanks!
Dor.
Little disclaimer first: There's no straightforward answer for this question(s), so Stackoverflow isn't the place for them. But since you have put it nicely I'll try to help :)
Almost any modern CMS API has a very similar approach that is very successful and flexible, you can follow their guidelines to implement your own API. I'd recommend storageroom and contentful documentation pages, they have a series of gotchas and patterns that I've successfully added to my own projects eventually.
Phalcon is a good choice for a RESTfull API and a proof of that is the PhalconEye project, a cms built using Phalcon. Once you have designed your API, go check the PhalconEye source code to get some samples on how to implement such things with Phalcon.
Good luck!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I've been using Symfony for close to 2 years now, so far, each project I build is deployed specifically for each client (i.e one client, one codebase, one db).
Lets say I have a Project Management app that I want to deploy for many clients. Assuming the clients will go with whatever features I build into the system, if I deploy a different codebase (therefore, a different db) for each client, here are the problems I foresee:
Pushing bug fixes and upgrades will be painful. I need to push it to every repository that I have deployed. It won't scale well if I have 50 clients using that same app.
Management is painful. How do I build an admin system for myself where I can pull ALL the projects into one HTML table? After all, each client has their own database, right? For me to do anything meaningful with all the records of all my clients, I need a way to look through all their databases at one go, which.. I don't think Symfony allows. (I'm not sure)
User account issues. If a user happens to work for multiple companies, all of them using my Project Management app, that user has to sign up multiple times. (I know this can be circumvented if I use oauth, but I'm trying not to go there if I can)
Here are the solutions I have thought up and tried to a certain extent.
Solution 1
One database and one codebase for ALL my clients. Projects will go under one table, Invoices go under one table, all marked by their own client_id. Users can be assigned to Projects so there is no need to sign up multiple times.
This is not that hard to create. But, what happens if different clients need different columns for their Invoices? My Invoice table will keep expanding (with different fields that different clients want), and each row can potentially contain many null fields. Not to mention, my Invoice entity will grow in file size, and I will have to update the database schema every time a new customization comes in.
Solution 2
One database where each client has their own table prefix. So for Client A, I could use clientA_projects, clientA_invoices, clientA_configuration etc.
This is ideal if each client wants to customize their fields. But, does this mean I need to create new entity and form classes for each new client that comes into the system? It looks like with this solution, I need to update the database schema with every new client I get.
Currently, I am experimenting with schema-less databases (mongo and couch), hoping that without needing to specify the table schema upfront, I can implement Solution 1 without hassle. But I'm still experimenting, and there are ways to go before I dare to deploy a production-ready app, being unfamiliar with the issues of mongo and couch with Symfony.
So, this is where I am stuck at. Being a self-trained programmer, I feel I have a lot of holes in my knowledge that requires filling (as opposed to someone from a CS background). There aren't many places on the web talking about Symfony 2 and multi-tenancy (maybe I'm looking for the wrong thing). If anyone of can point me to a clearer direction, maybe best practices, example projects, I will really appreciate it!
Btw, I plan to execute this in the latest version of Symfony (2.3.2 at this moment).
Thanks in advance guys.
I'm also using Symfony2 for similar amount of time (since one of BETAs) and I advise you to go with solution #1. If you are going SaaS you can't give out code to clients for the reasons you wrote (problems with updates / upgrades mainly). The whole hassle will be on the user management - which user has access to which data, belongs to which group, company and so on. All other things if done properly will be coded the same user-agnostic way. What should you do with different requirements for different companies? Make such features configurable. You can implement this on various levels:
simple entity attributes: have an attributes field in each table and save everything as JSON, YAML or other dynamic-structurable content,
general configuration: have one place where entity base configuration is stored (in the means I wrote above) and allow users to manage new features from there, all changes are propagated to simple entities,
implement something what I call Entity Parameters Pattern - design database tables which would contain parameter types, parameter values and relation to other entities on different levels and then make generic configurable parameter types which can be applied to any place with predefined meaning. For example "preferred_season" being a parameter of type "choice_string" containing configuration "spring,summer,autumn,winter" and when attached to given entity would always render a <select> field with choices and save selected value with relation to both entity and parameter type.
Also the solution #1 has one unbeatable advantage - it could handle more that one company even if you wanted to give out the code at the end. You'd just need to mask the ability to add more. :)
This question is tagged Symfony2 but it really shouldn't. It doesn't matter what framework you're using, you should abstract your application design from code and then use frameworks as a mere tool for doing the job smoothly. I'd like to say that even taking previous sentence into account, I'm absolutely in love with Symfony2. :)
I know this is an older question but can be usefull for others.
I agree with #Tomasz and solution #1 with one database – all tenants in one database. The biggest problem here is proper database design to solve further security issues: access for resources must be controlled by application to prevent unauthorized access between tenants. On the other side we have ease implementation as we are implementing single application with only one database.
Nice article about Symfony2 and moving to SaaS model:
http://www.browserlondon.com/blog/2015/01/moving-to-a-saas-model-with-symfony2/
Also "must read" article about designing database in SaaS platform - patterns that are platform independent:
http://labs.octivi.com/database-design-in-saas-platforms/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I'm trying to make an availability / scheduling system... Basically, I want users to be able to log into this system, and show that they're available during a certain time block, then I want another user to be able to log in and see who's available and when, and to be able to book someones time (so they no longer show up as available). I want to use PHP and SQL. Does anyone know if there are any open source systems out there that do something similar? I feel like there would be and it would be silly to rebuild one from scratch.
Although I want to use PHP and SQL, I'll consider any other open source tools that don't use those technologies, but obviously php and sql are preferred.
Thanks
EDIT: I know this problem can be solved with google calendar... but I need to find another way other than google calendar.
I found this open source project. You can check this: http://supercali.inforest.com/
SuperCali is an event calendar script that supports nested categories of events and multiple moderators, making it a good choice for organizations managing a large number of activities. SuperCali is designed to make data entry as easy and error-free as possible as well as provide a flexible, modular framework for displaying event information. SuperCali works with PHP and MySQL and is free, "open source" software released under the GNU General Public License.
Ok, I'm going to go a little outside the box here and ask if you've considered Gmail's calendar app? It has apis but I think you can skip that and just have the user's share their calendars and they can see them all together in one view.
This has the benefits of taking care of the security for you the advanced scheduling for you.
I know it's not what you're asking for but if you're just wanting the scheduling without the security headaches and coding necessary to bring it up and online, maybe this will work.
mrbs? Any ical server?
Check out http://phpicalendar.net/ as a decent ICalendar client. This makes basically any ICalendar server viable.
More specific to scheduling, there is an open source project using php and mysql called phpMyCal at http://dev.neb.net/phpMyCal/
We do this with MRBS http://mrbs.sourceforge.net/ and it meets your specifications of php/mysql- while it sounds like a room booking system, it is for reserving anything. It is very flexible, uses many different authentication systems, and if I can install it (with help from the user forums), probably anyone can.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
Perhaps, using the word CRM is a bit of a misnomer, essentially, I'm looking for a fast and easy way to create a front-end so that others in my organization can utilize a MySQL database that I've created that manages lists of respondents to marketing research surveys.
I would need to do the following through the proposed front-end:
Update/Add/Remove contact information
Add notes to a contact's "file"
Import/Export data from/to csv
My experience with PHP is not extensive so I would like something that is simple and straightforward (read: I'm not looking for something that tries to do everything or is over complicated).
An answer to this question will be accepted if you can outline the reason(s) for your recommendation.
Database Schema:
Table 1: Contact Information
ID,
Name, address, email, etc...
Table 2: Surveys
Table 1 ID,
Field 1,2,3,4,5 (Logicals 0/1)
Table 3: Notes
Table 1 ID,
Date, Note, etc...
Table 4: Select Survey Data Storage (Demos for easy survey sampling)
Table 1 ID,
Q1,Q2,Q3,Q4...etc
One small framework comes to mind that will help make simple and secure user access and helps you update created database data without much hassel, it is called flourishlib, I've often used and seen used with the small routing controller Moor.
This helps you with:
Simple and safe user access. Supports ACL.
CRUD.
It has an ORM implementation, so you don't need to do much work in the data layer.
Simple page templating.
Posting and updating a record is as simple as $record->populate automatically taking values from a submitted form and populating values.
Great dir and file handling. Reading a CSV.
Check out the how do I page for a quick look on how flourishlib solves common problems.
Update: I'm unsure how the ORM of flourishlib will match your database scheme without any configuration. If you have problems the support for flourish by the lead developer at the forums is great. Anyway you don't have to use the ORM. You could use flourishes fDatabase. Or you could use an other ORM entirely. Two I find interesting is:
Repose
Outlet
Are you sure that you need framework? Your task is very simple and it can be easilly done without framework. Just organise files/folders layout, add some needed classes and you are good to go.
You can always try SugarCRM
It lacks a bit on documentation (mostly in the programming areas), but I think it will accomplish your goals.
In the end I actually went with Django. It's ridiculously easy to create database management edit/update/search tools using their ORM framework.
I think you should give a try to Lime survey.
It an open source application written in PHP with mySql as backend for survey development and managing samples.