I have the following issue (I am trying to give the full context, if you are in a hurry, skip to the last paragraph):
We need to create a multilanguage support for a project. Our current database is MySQL.
Since the structure would need to change pretty much to allow it in the current DB, we got to the conclusion that the "classic" approach is not optimal.
By classic I mean adding lang tables, linking them with FKs, etc.
So we thought "let's save langs on file on disk". We analyzed that and is easier to implement, and looks ok as a solution. But then we got into the issue "what about concurrent access - we would need to implement something for this".
So we said... why not use a no sql db for this? It deals with concurrent access/lock, is outside our main db, is fast, scalable.
At first glance this looks like a good solution and we are willing to give it a try. But since we lack the experience with no sql... can any1 tell us if this is a good/bad idea and why (I explained how we got here and why we thought is a good idea)?
Related
I am currently working on a 'simple' social network style in PHP. I call it 'simple' because it's not the classic example-type website; I just try and keep it as simple is possible;
So I basically am on MongoDB and have a two collection structure: the users table, which is something like
{ user:'test', pass:'123456', name:'foobar', info:'hello' }
and a posts table:
{ user:'test', message:'this is a test post' }
I simplified it all to make or more readable at first sight.
As said I am using MongoDB currently, working great and all but, I just read a bit about Cassandra and was wondering if it could be a better choice, given its simpler scaling features (I could basically do it myself instead of having to rely on MongoLab or MongoHQ);
But I am not sure if this could be a good idea as I am making use of dynamic queries.
Messaging and Relationships are not taken into consideration, I am using Redis for this; The switch would only be related to the registration of the users, search, and their posts.
Any ideas if I should keep with Mongo, or if I could consider a change?
Thanks in advance.
You can best decide this by forking your current code-base and clone the environment.
Then start with the implementation of the alternative data-store.
You already name certain points you want to look into, so take care of these one after the other.
You will then experience if the way you outlined/prospected the change would actually work or not and with which price that comes.
Keep a worklog in which you write down the steps you do and what your experiences are.
In the documentation they mention as an example that we can restrict access to a comment.
But if I didn't get it wrong, the system would be storing a different ACE for each instance. So I guess there would be at least one extra row somewhere in the db for each comment.
Is this suitable for high-traffic sites ? Could this create a bottleneck ?
How do sites like stackoverflow or wikipedia manage their ACL's?(I might have to create another post for this question)
PS: Please don't bring the 'premature optimization' thing, I prefer to make this kind of things right by design. Gradual optimization can always be done in other places.
So I have recently taken over maintenance of an in-house Content Management System, and database optimization is not really my area of expertise.
Anyway a couple of things fell out to my eye when I was looking over the code.
The php code is a little bit "spaghetti with meatballs" with little to no comments.
But the biggest thing: As far as I can tell, the original programmer decided to forgo table joins entirely in the database code (both implicit and explicit).
For example here is the process to display a page column:
Fetch element list from the database
For each element call a subroutine to check display permissions and if successful, call another subroutine to fetch the element's html data.
Each of the subroutines effectively calls a separate query for each element. And the permissions step, I believe, involves querying two separate tables.
Performance isn't really a problem at the moment, and I wasn't asked to look into this. Although the page requests are a bit slow in my opinion.
Is it worth trying to rewrite the SQL stuff? I am thinking that the increase in maintainability will be worth it in the end, and that it will make things easier for me should scalability become an issue in the future.
Or is it not really as bad as I think? Maybe I am just overreacting. An expert opinion would be appreciated.
Refactoring is an important part of development process. Ignoring this fact means more problems in the future. The part of the problem is that not many managers understand the importance of continuous refactoring.
I'd recommend you to read "Refactoring to Patterns" by Joshua Kerievsky that has many good examples of how to safely change existing code by implementing new design approaches.
As of your question about SQL queries, it may or may not be the first thing you need to change.
If its about making your job in the future easier, I would start refactoring the spaghetti code before fixing the queries. Once you have a nice design, it should also be more straight forward how to integrate (and maybe even eleminate) some of those manual joins.
I think it would be worth rewriting while you have the time to do so. You don’t want to put it off until it becomes a problem. I say, let the database do what it does best. In this case, table joins would certainly perform better than multiple queries in a loop.
Your question is a bit too generic to give a good answer to and you haven't provided enough information to make educated guesses either, for your particular situtaion.
Things to take in to account:
Will the system grow over time? And how?
If there will be more users online, more preassure on the system - then there's cause for alarm. Systems with bad design does not scale well.
Bad code is one thing, but bad database- and general system design is worse. I think the key might be your comment about "next few years". If you intend to stick with the sytem for that long - then a serious look at the basics is a good idea. If your system might be up for replacement by something else (inhouse CMS's tend to be replaced) then you can patch along while shopping for the right thing.
But as your question stands - rewrite some of the questions, make sure your DB is normalized and refactor your code. Tell your boss it's the right thing to do if you're going to stick with the system.
I have been using ezSQL for the last few years but feel it is outdated. Though I like the simplicity and I like the file based caching ability with json, for small result sets that is.
So starting a new project I was looking for suggestions on a better mysql class for php. I know the db will only be mysql so portability is not a requirement. I read about mysqli extension, pdo etc but just dont know which one would be best for my situation. The site does a lot more reads than writes, though there are times where there are a lot of writes in the admin tool to the db. I looked at doctrine but dont know if that is too "bloated" for what I need. Hopefully this isnt to vague. Any suggestions?
EDIT
The site isnt small, I would consider it a high traffic site with a lot of db queries.
What don't you like about ezSQL? I often wish there was something like it for other protocols/languages I encounter. Every syntax should be written like ezSQL, in my opinion.. It describes the operation to be performed, in as few words as is possible, in the clearest and most logical order. Do you actually have performance problems, or are you just worried that something better has come along? I agree that ezSQL is rarely mentioned, but I have yet to find anything that matches it's simplicity, conciseness, and function...
From what I know of ezSQL (via it's wordpress pendant) I would consider Doctrine as well as too much for the moment because it's a complete data mapper for the database whereas you might be more looking to how to move away from your recent use of ezSQL which I think is a good idea.
Bascially you might be interested in a data-access abstraction layer. That could be PDO as it's build in into PHP. Even if you don't need to change the database server, it will give you defined interfaces how to query and access the data.
As you build the site from scratch, I can suggest you consider using some lightweight framework. A good introduction in my eyes is When Flat PHP meets Symfony which shows how a webapp can generally benefit from patterns and a flexible design.
From experience:
Doctrine - very easy to use I love doctrine query language - I never had to do initial setup though so im not sure how hard it is. It has very good community and lots of tutorials.
Propel - used for a bit. Does the job, very similar to doctrine. However, the documentation is very crap and community is very slack. I found that when I didn't know something it was quite hard to find an answer and often I had to post on Google forums.
Note: If you are starting from scratch you might want to look at some of the frameworks such as symfony+doctrine is a good combination, makes development a lot easier.
Links:
- http://www.doctrine-project.org/
- http://www.propelorm.org/
This is my first time posting something like this. I'm a complete n00b at programming/php. I was trying to grasp all the tutorials out there and wanted to create a sort of framework to base projects off of for web db/applications. I came up with a few files for crud operations that I know must really be horrible code. I was wondering if I could get some advice in advancing this 'framework' to the next level. I'm not great at OO, but I understand the concepts, so I went more functional instead. I'm aware that the songs are gross, I don't need help recognizing that.
I would like some help on the database calls. I understand I should be using PDO (to help with injection), but I'm not sure how to convert this code to that structure. I also realize that at the least I should be using mysqli functions. Again, n00b.
I am also aware of validating and sanitizing data, but again, being so new to this, I don't know where to begin. Especially if I wish to keep it tight and small. The comments are funny from a programmers aspect, but hey, you work with what you got. I'm trying to grasp MVC and be able to tie in AJAX and templates/skins, but the crux of it is to work with databases.
I do like how there is one config file that works with all the other files, and how that results in small files for index, read and delete. The others get a little longer. Not that including files is makes me cool or anything.
I think I fail horribly at understanding MVC and structuring the project so that maintaining it and adapting it is easier. I also realize that there is a lot of html in those php files which is a no-no too. How do I fill in the gaps from what I do know (I know what every line of every file is doing) to what I should know.
The code can be found here (probably not sourceforge worthy, but I want to help others learn as I do).
https://sourceforge.net/projects/eleete/files/
Building such a thing takes a good amount of time, before you go doing so I would suggest looking into open source CMSs such as Wordpress to see if they would suit you better than putting in hours of work creating something yourself, the reason I say this is because to begin with you may think it won't be too hard to create one for yourself, but as you get into it you will realize there are many avenues that need to be explored in terms of security and features.
However, if you still want to build something the most important thing is to be creating reusable, and easily understandable code. For example if you want to create db entries for things like pages, you'll want to create functions to do so, to which you can pass variables such as titles etc. A good way to begin is to create functions for collecting data from your db, for example siteinfo() could be a function that returns an array full of site info. To make sure every file has access to such functions you will need a global file like functions.php for example which you will need to include in each of your other files.
Remember, building a framework is hard work, but if you do it properly and thouraghly you should increase your productivity a good amount - this is something you need to consider when building it, make sure you're not doing things in a roundabout way. Make it simple, make it reusable, and make it powerful.
First off, welcome to the community! I hope you find here people who are excited to bring your more into the programming fold. It's an exciting place to be.
I get the impression you're biting off quite a bit. I fear you're also under the assumption you can write the perfect program from the get-go, which is a flat-out myth no matter how long you've been in the industry.
As to the former observation, my recommendation would be to pick a simple problem and try to solve it. My favorite recommendation here are simple card games - blackjack, for instance. Dice games like craps also make for short, achievable, and interesting programs. The intent of these projects will be not to build a code framework as much as your skills in dissecting problems and turning them into a solution the computer can understand and solve for you- your code.
Starting small will keep you from getting overwhelmed early. Starting fun will keep you motivated through the inevitable snags that will tempt you to throw your computer out of a closed window.
All that said, we're still here to help! Specific questions to specific problems tend to get the best feedback here, as well as questions that demonstrate you've done at least some homework on your own in terms of trying to solve it. The Stack Overflow community will be happy to give you a hand up, but not a handout. Good luck!
You're going to be much better off using a clean, simple framework. As you're so new with this stuff, I'd recommend CodeIgniter for sheer simplicity and speed. It helps enforce the MVC framework you're not totally familiar with yet, has all the built-ins you're likely looking for, and has a decent community. Being a Zend guy, I should probably suggest that, but it's definitely not the easiest to get started with.
MVC and OO are concepts you really should learn well out of the gate before going forward. This tutorial rocks. There are dozens of MVC primers out there, but in essence it boils down to separating your data layer, view layer, and the like.
PDO is not necessary to be safe, you just need to escape your sql with Mysql_escape_string or the like. I'm actually not a fan of PDO due to the monkey wrenches it throws into debugging.
Since you're new, I'd also recommend Xdebug on your server for troubleshooting. It cuts down debug time immensely.
In my experience, unless you've had to fix the shortfalls in someone else's attempt at a framework, you won't really understand how to build one yourself.
One of the key qualities of any sort of API or framework is the concept of abstraction. In a nutshell, as you find yourself repeating code, look for where the repetition could be abstracted away. The trick about this when you're building a framework is to do it in an extensible, usable fashion so that you have pieces that build on each other. In other words, abstractions can hide other abstractions. For example, It's all very well to abstract away the job of assembling an SQL UPDATE statement from a list of fields, but why does the application page have to check the list of fields is correct? Put that behind another layer that can be taught what the valid fields are for a certain object.
Another key quality that comes up at this point is refactoring. The phrase "be prepared to throw the first one away" is sometimes bandied about in programming circles. What it really means is that you should always be open to the idea of re-writing code because you've thought of a better way to do it. Even if you have to rebuild the code that calls it.