In the documentation they mention as an example that we can restrict access to a comment.
But if I didn't get it wrong, the system would be storing a different ACE for each instance. So I guess there would be at least one extra row somewhere in the db for each comment.
Is this suitable for high-traffic sites ? Could this create a bottleneck ?
How do sites like stackoverflow or wikipedia manage their ACL's?(I might have to create another post for this question)
PS: Please don't bring the 'premature optimization' thing, I prefer to make this kind of things right by design. Gradual optimization can always be done in other places.
Related
I'm starting to work on my biggest project yet and if it's succesful it's going to be used by thousands of users. Unfortunately my experience in PHP has been limited to smaller projects only used in limited and closed groups of people or private usage so security never was my biggest concern. I want to create the most secure system possible but I can't find any good reference.
What should I know about securing a login system or storing sensitive data input by users? Is there any book you can recommend or a comprehensive guide? Or should I buy/use a premade system? I realize perfecting it is going to take a lot of time and I'm willing to invest it for it to be as secure as possible. I've already read up a lot about security and what to look for but I want to be sure I didn't miss anything. Also I couldn't find any confirmed secure code example apart from this. Is this all I need to know?
Is there any book you can recommended or a comprehensive guide?
Owasp.org Is a good starting point.
Or should I buy/use some premade system?
Using a pre-made system like WordPress or a framework is a good starting point to achieve a good security (because you use the code that a "more expert coder" write for you).
Is all that I need to know?
Nope. There are a lot of things that come with experience. Also there are a lot of things that are normally out of scope for a coder.
Okay, so this is not a specific "what am I doing wrong" coding question, sorry. If it's better suited to one of the other stack* sites, let me know.
Today I was fired from a contracting gig where they're using Symfony2 and Doctrine. I'm a senior programmer, but not very experienced in either of these two frameworks - though I felt sure I could get up and running fast. After all, it's PHP - what could go wrong?</cynicism>
My assigned teammate (brought on as a "Symfony expert", so I was at first perfectly happy to watch and learn) kept insisting that 'entities should be immutable, and all custom business logic should go in repositories'. Reading the documentation (the getting started guide, no less) this simply seems to be untrue - repositories are for 'custom getters', but data manipulation goes in entities (Doctrine-speak for models). Made perfect sense to me (a database can't know if bitflag 1 means 'active' and 2 'confirmedEmail'), but the guy was adament. When pressed for an example of how I should add custom data editing methods in a repository (since I fooled around for a bit but couldn't get Symfony to see them without going through hoops and manually replacing entities with repositories), he sent a code sample implementing a more or less manual update query. Then why use an ORM in the first place?
He also repeatedly stated that one should never touch entities, since changes might get overwritten when they're regenerated. Apart from the fact that Doctrine seems to normally assume you write entities and generate the database scheme from those - not the other way around - unless my knowledge of the English language is mysteriously failing, the manual seems to clearly state that when (re)generating entities, existing methods are left alone so it's perfectly safe.
End of story, client decided we couldn't work together and chose the other guy. They also had this weird argumentation where I was "the most expensive guy on the project" and I "wasn't providing enough seniorness". I did point out that telling others they're simply wrong and getting yelled at for it (true story, well, they were typing in ALL CAPS which counts as yelling in my book) didn't make me very happy either, so that was that - I wished them the best of luck and we went seperate ways.
Am I losing my mind here? Am I missing something glaringly obvious? Or is the other guy simply misguided (my current working theory) as to what belongs in entities and what in repositories? Anyone experienced in Doctrine/Symfony care to elaborate? I'd be happy to provide more specific examples of what we were supposed to be building. I'm mostly extremely curious to learn the "right way"(tm) of doing things in these frameworks (for future reference, they didn't quite whet my appetite yet), if there indeed is a better way to abstrahate model code away from entities.
Symfony2 is built to allow you to choose how you answer these questions. It's structured to be modular, allowing you to pick and use only the components you need. There are many common design patterns but obviously a team needs to pick one and stick with it. If you guys couldn't do that, it is probably good that you didn't try to complete a project together.
I consider it good practice in Symfony2 to keep the entities and controllers as sparse and easy to read as possible. I put almost nothing in repositories that is not building queries and running them. Pretty much everything else should go in a service of one kind or another. How you enforce separation of concerns among services is left as an exercise to the development team.
Here's an article talking about a set of common design patterns:
http://www.ricardclau.com/2012/02/where-is-the-model-in-symfony2-entities-repositories-and-services/
Another good thing to look at when getting a handle on how to do things and where to put thins, is take a through some of the core/big symfony bundles, see where they put things and how much the different styles of organization make sense for different purposes (some do it better than others).
His specific claims:
1) entities should be immutable
This seems to cause you to lose out of much of the functionality provided by Doctrine's EntityManger and UnitOfWork. If you then update them only using manual queries, you will have to reload your entities to get them to match the updated DB state. I guess this makes sense if you want to load one state at the start of a request and progressively process updates to the DB based solely on the original data. (Doctrine does provide functionality in the EntityManger to mark Entities as read-only so that changes to them will not be persisted.)
2) all custom business logic should go in repositories
In Symfony, replace 'repositories' with 'services' and this would be pretty much 100% correct. Repositories should generally only hold code for re-usable complicated queries beyond what Doctrine's findBy methods get you.
3) He also repeatedly stated that one should never touch entities, since changes might get overwritten when they're regenerated.
True...ish as far as the overwriting goes. Whenever you add changes to your entities from a schema file using doctrine command line tools this 'could' happen so backups are made. However, the tool is fairly smart and I haven't seen any of my changes overwritten.
Your specific claims:
1) but data manipulation goes in entities (Doctrine-speak for models)
Entities are not the 'M' in 'MVC' but are generally just the part of the 'M' where the state is stored. The related logic is generally kept separate in forms, data transformers, validators and other services.
2) he sent a code sample implementing a more or less manual update query. Then why use an ORM in the first place?
Even if you only update data through manual queries rather than using the EntityManger, Doctrine provides many other advantages: Programatic Query Generation, Input Cleaning, Entity Hyrdation, Query/Result Caching, Nested Transactions, ETC.
Just hearing your side of the story, it sounds like a more senior developer with less framework experience working with a less senior developer with more framework experience. I would guess that the other developer had never worked solo/lead on a greenfield Symfony2 project and was just parroting the design decisions made by his prior Senior developer.
Neither of you sounds completely 'right' and both of you were wrong in that you got bogged down in an argument that had to be adjudicated by your client. Maybe if you had been less keen on showing yourself right, and been more willing to utilize and explore your partner's experience by delving into the the reasons behind the practices he was advocating you might have been able to turn this into a success.... Or perhaps the other developer was just a tool and you saved yourself a boatload of stress and drama trying to cover his ass :)
I have the following issue (I am trying to give the full context, if you are in a hurry, skip to the last paragraph):
We need to create a multilanguage support for a project. Our current database is MySQL.
Since the structure would need to change pretty much to allow it in the current DB, we got to the conclusion that the "classic" approach is not optimal.
By classic I mean adding lang tables, linking them with FKs, etc.
So we thought "let's save langs on file on disk". We analyzed that and is easier to implement, and looks ok as a solution. But then we got into the issue "what about concurrent access - we would need to implement something for this".
So we said... why not use a no sql db for this? It deals with concurrent access/lock, is outside our main db, is fast, scalable.
At first glance this looks like a good solution and we are willing to give it a try. But since we lack the experience with no sql... can any1 tell us if this is a good/bad idea and why (I explained how we got here and why we thought is a good idea)?
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I'm rewriting a big website, that needs very solid architecture, here are my few questions, and pardon me for mixing apples and oranges and probably kiwi too:) I did a lot of research and ended up totally confused.
Main question: Which approach would you take in building a big website expected to grow in every way?
Single entry point, pages data in the database, pulled by associating GET variable with database entry (?pageid=whatever)
Single entry point, pages data in separate files, included based on GET variable (?pageid=whatever would include whatever.php)
MVC (Alright guys, I'm all for it, but can't grasp the concept besides checking all tutorials and frameworks out there, do they store "view" in database? Seems to me from examples that if you have 1000 pages of same kind they can be shaped by 1 model, but I'll still need to have 1000 "views" files?)
PAC - this sounds even more logical to me, but didn't find much resources - if this is a good way to go, can you recommend any books or links?
DAL/DAO/DDD - i learned about these terms by diligently reading through stack overflow before posting question. Not sure if it belongs to this list
Sit down and create my own architecture (likely to do if nobody enlightens me here:)
Something not mentioned...
Thanks.
Scalability/availability (iow. high-traffic) for websites is best addressed by none of the items you mention. Especially points 1 and 2; storing the page definitions in a database is an absolute no-no. MVC and other similar patterns are more for code clarity and maintenance, not for scalability.
An important piece of missing information is what kind of concurrent hits/sec are you expecting? Sometimes, people who haven't built high-traffic websites are surprised at the hit rates that actually constitute a "scalability nightmare".
There are books on how to design scalable architectures, so an SO post will not be able to the topic justice, but some very top-level concepts, in no particular order, are:
Scalability is best handled first by looking at hardware-based solutions. A beefy server with an array of SSD disks can go a long way.
Make static anything that can be static. Serve as much as you can from the web server, not the DB. For example, a lot of pages on websites dynamically generate data lists out of databases from data stores that very rarely or never really change.
Cache output that changes infrequently, and tune the cache refresh.
Build dynamic pages to be stateless or asynchronous. Look into CQRS and Event Sourcing for patterns that favor/facilitate scaling.
Tune your queries. The DB is usually the big bottleneck since it is a shared resource. Lots of web app builders use ORMs that create poor queries.
Tune your database engine. Backups, replication, sweeping, logging, all of these require just a little bit of resource from your engine. Tuning it can lead to a faster DB that buys you time from a scale-out.
Reduce the number of HTTP requests from clients. Each HTTP connect has overhead. Check your pages and see if you can increase the payload in each request so as to reduce the overall number of individual requests.
At this point, you've optimized the behavior on one server, and you have to "scale out". Now, things get very complicated very fast. Load-balancing scenarios of various types (sharding, DNS-driven, dumb balancing, etc), separating read data from write data on different DBs, going to a virtualization solution like Google Apps, offload static content to a big CDN service, use a language like Erlang or Scala and parallelize your app, etc...
Single entry point, pages data in the
database, pulled by associating GET
variable with database entry
(?pageid=whatever)
Potential nightmare for maintenance. And also for development if you have team of more than 2-3 people. You would need to create a set of strict rules for everyone to adhere to - effort that would be much better spent if using MVC. Same goes for 2.
MVC (Alright guys, I'm all for it, but
can't grasp the concept besides
checking all tutorials and frameworks
out there, do they store "view" in
database? Seems to me from examples
that if you have 1000 pages of same
kind they can be shaped by 1 model,
but I'll still need to have 1000
"views" files?)
It depends how many page layouts are there. Most MVC frameworks allow you to work with structured views (i.e. main page views, sub-views). Think of a view as HTML template for the web page. How many templates and sub-templates inside you need is exactly how many view's you'll have. I believe most websites can get away with up to 50 main views and up to 100 subviews - but those are very large sites. Looking at some sites I run, it's more like 50 views in total.
DAL/DAO/DDD - i learned about these
terms by diligently reading through
stack overflow before posting
question. Not sure if it belongs to
this list
It does. DDD is great if you need meta-views or meta-models. Say, if all your models are quite similar in structure, but differ only in database tables used and your views almost map 1:1 to models. In that case, it is a good time for DDD. A good example is some ERP software where you don't need a separate design for all the database tables, you can use some uniform way to do all the CRUD operations. In this case you could probably get away with one model and a couple of views - all generated dynamically at run-time using meta-model that maps database columns, types and rules to logic of programming language. But, please note that it does take some time and effort to build a quality DDD engine so that your application doesn't look like hacked-up MS Access program.
Sit down and create my own
architecture (likely to do if nobody
enlightens me here:)
If you're building a public-facing website, you're most likely going to do it well with MVC. A very good starting point is to look at CodeIgniter video tutorials. It helped me understand what MVC really is and how to use it way better than any HOWTO or manual I read. And they only take 29minutes altogether:
http://codeigniter.com/tutorials/
Enjoy.
I'm a fan of MVC because I've found it easier to scale your team when everything has a place and is nice and compartmentalized. It takes some getting used to, but the easiest way to get a handle on it is to dive in.
That said definitely check your local library to see if they have the O'Reilley book on scaling: http://oreilly.com/catalog/9780596102357 which is a good place to start.
If you're creating a "big" website and don't fully grasp MVC or a web framework then a CMS might be a better route since you can expand it with plugins as you see fit. With this route you can worry more about the content and page structure rather than the platform. As long as you pick the appropriate CMS.
I would suggest to create a mock app with some of the web mvc frameworks in the wild and pick one, with which your development was smooth enough. Establishing your code on a solid basis is fundamental, if you want to grasp concepts of mvc and be ready to add new functionality to your web easily.
I have basic knowledge of PHP and had done a few small personal projects before. Since my programs were small and usually consisted of less than five classes, I jumped into coding right away -- thinking of my database tables and user interface design as I went along. The problem was that I often found myself lost in my own ideas at the middle of my project. Hence, I thought that some form of formal planning would be practical to begin with.
I have been reading several software engineering books lately. One book said that, in web engineering, agile processes such as extreme programming and scrum should be used. I was introduced to tools such as use cases, CRC cards, and UML -- all I believe are too complicated and impractical for the simple projects I have in mind.
On the other hand, a web article I read simply suggested a rough sketch of the UI, a sitemap, and a simple workflow diagram. They seemed practical but too rudimentary and informal.
What would be a practical but formal approach to building a simple web application?
Don't be so quick to discard use cases, or some similar concept, e.g. 'user stories'. The benefit these tools bring is to make you think about what the site needs to do. That can guide all the further technical work, and help you focus on what's important instead of getting lost in interesting but low-value work.
Think about the top n things the site needs to do, and for each of these list out:
the ultimate goal for the user in this particular case
the kinds of people and other systems involved (actors, in use-case terminology)
what needs to be in place before starting this activity (the preconditions)
the basic steps the primary actor needs to perform to successfully achieve the goal
what should happen if one of those steps can't be completed
what the system looks like if the case is successfully completed (the postconditions)
what the system should look like if there were errors along the way
With a handful of these use cases, you can get an overall feel for what needs to be built to fulfill the goals. If it starts to look like too much work, you can look at which goals are of greater or lesser importance, and cut or defer those that provide less value. And if you think of new things the site should do, the existing use cases either:
provide a home for that feature, so modify the appropriate use case to incorporate the feature,
don't provide a home, indicating you missed a use case or discovered a new one, so you should elaborate on it as described above, or
don't provide a home, and therefore indicate the isn't necessary and so you shouldn't spend time on it.
The trick is to pick the proper level of abstraction and formality. Work at a high level, write informally, and you can knock these out quickly. When they're no longer useful (you have most of the application sketched out and are now heads-down working on details), put them aside. When you think of new things, see if they fit. Lather, rinse, repeat.
You have to ask yourself a few simple questions first:
1) Do I want to re-use and extend this code later?
If so, it may be a good idea to use some basic pattern such as MVC (Model-View-Controller) or use one of the many PHP frameworks available. The already established frameworks are more format in their design and how you use it. You could always use them as a basis to design your own as well.
2) Is the site you are making going to grow or change frequently? If the site isn't going to change much you could probably make something very simple and not worry too much about good design principles.
3) Do you want to learn and apply some of the techniques you mentioned? If so, then go to town and try the different techniques and see which ones appeal to you.
These kinds of questions will help you decide how you want to decide you build your website.
Start small, mock up a few simple screens, their database structure, etc. Use whatever method works for you - if that is UML diagrams, then fine. Get something finished, evaluate, and iterate from there.
This may not be quite what you are looking for, but also do yourself a favor and choose a framework such as CodeIgniter to make your life easier writing your server-side code. And similarly, consider using a client-side framework such as jQuery. This will make it easier for you when you are ready to actually sit down and write the code.
The Bare Essentials
I find there is a (roughly) four-step process that make application development a lot easier, and helps me avoid getting lost along the way. I make no pretense about using agile development methods. This is just the way I work.
Decide what the application needs to do. By "need", I mean, absolutely must do and nothing else. If it's a "nice to have" feature, write it down and come back to it later.
Figure out what data you need to keep track of to make everything work. Slugs, titles, datetimes, etc. Design the database. (I use MySQL Workbench for this.)
With your feature shortlist, prototype a user-interface with Balsamiq. Remember to add only the features you need right now.
You might consider doing HTML / CSS templates in this step.
Now that you have the data model and the UI, connect them. Business logic should focus on two things:
Abstract away from the data model so you can easily retrieve and store information from step 2.
Make the UI work exactly as you designed in step 3. Don't get fancy.
Focus on Small Goals
There are two keys to the process. The first is to to separate the design of various components. You don't want to be thinking about implementation details when designing the UI, etc. The second key is to iterate. Focus on exactly what you need to add right now and ignore everything else.
Once you're done, you can iterate and tweak, repeat steps 1-4, etc. But don't get stuck in any step for too long. You can always iterate again later if things don't turn out exactly as you like. (I think agile calls the iteration "sprints", but I'm not up on the theory.)
Practical Considerations
It helps if you have a light framework (at minimum) to assist with 4.1 (ORM, like Propel) and 4.2 (JQuery, etc.). You want to keep both the UI and data access as modular and compartmentalized as possible so it's easy to change later.
If you mix everything into one chunk of code, then you'll have to change every part of the program just to (for instance) add a new column to the database or add a new button to your app. You've had some experience, so you should have an idea of pitfalls to avoid.