I'm staring to build a system for working with native languages, tags and such data in Yii Framework.
I already choose MongoDB for storing my data as I think it feets nicelly and will get better performance with less costs (the database will have huge amounts of data).
My question regards user authentication, payments, etc... This are sensitive bits of information and areas where I think the data is relational.
So:
1. Would you use two different db systems? Should I need them or I'm I complicating this?
2. If you recommend the two db approach how would I achieve that in Yii?
Thanks for your time!
PS: I do not intend this question to be another endless discussion between the relational vs non-relational folks. Having said that I think that my data feets mongo but if you have something to say about that go ahead ;)
You might be interested in this presentation on OpenSky's infrastructure, where MongoDB is used alongside MySQL. Mongo was utilized mainly for CMS-type data where a flexible schema was useful, and they relied upon MySQL for transactions (e.g. customer orders, payments). If you end up using the Doctrine library, you'll find that the ORM (for SQL databases) and MongoDB ODM share a similar API, which should make the experimentation process easier.
I wouldn't shy away from using MongoDB to store user data, though, as that's often a record that can benefit from embedded document storage (e.g. storing multiple billing/shipping addresses within a single user document). If anything, Mongo should be flexible enough to enable you to develop your application without worrying about schema changes due to evolving product requirements. As those requirements become more clear, you'll be able to make a decision based on the app's performance needs and types of database queries you end up needing.
There is no harm in using multiple databases (if you really need), many big websites are using multiple databases so go a head and start your project.
Related
Is there currently any technology that would separate storage from business logic and allow me, to easily switch from MySQL do MongoDB? (I assume I migrate the data myself, or start with an empty database after the switch).
I would like the change to be as easy as changing the configuration, the driver, and the db connection data.
I understand it is possible for PHP with Doctrine, switching between different RDBMS, but I'm interested in switching from any RDBMS to a NoSQL system.
I am focusing on PHP now - but if you know any solutions for other programming languages - I will be happy to learn about them.
I am assuming not a complex database, no transactions, no complex relations.
More background/details
I am writing a simple crawler that will visit websites, and read some data and save it to the DB. It is super simple and I might go with pdo_mysql for PHP only. I am considering an extra layer only to cover the situation in case I want to switch from MySQL to MongoDB one day - and I asking if this is even possible.
Update
I think that Laravel with its Eloquent supports MySQL out of the box, and with an extra plugin: https://github.com/jenssegers/laravel-mongodb supports MongoDB - I will check if this is truly transparent from the programmer's perspective. Unless someone has experience and knows right away?
A typical approach here is the DAO or Repository design pattern. In this pattern you provide a library which is responsible for persistence and returns business objects. The objects do not have persistence logic and get stashed in the Repository or retrieved from the repository.
This being said, that only works for business objects. The problems will come up when you want to redo reporting or the like.... Here the differences between an RDBMS and a NoSQL solution will bite you hard.
To be sure I don't know what a use case is given that "what do you want to do with your data" is a massive concern in selecting between these.
I'm having a hard time implementing a search feature for a web based system I’m working on, I first use MySQL Like with %wildcards%, but it not searching what I want to display, then I come upon Full Text index search, it search very good but has an issue on displaying joined multiple tables with foreign key which I don’t know workarounds, then I came along with MySQL with sphinx,
may I ask for any advice the best way/technologies to implement a search feature to search a Complex database tables
Check Apache Solr search server
Apache Solr official website
this technology will solve all your searching related problems
I guess the general answer here is you want a 'search index' - an index specifically for running searches. A repository that has all the required data to answer queries.
A RDBMS (like MySQL) is very good for Normalizing data, setting data up in a compact and easy to update format (ie minimise duplicate) - thats great for storage. But queries suffer as they have to do much more work to 'join' all the required data back.
... but for searching a denormalizaed structre may be best. (bigger, but easier - therefore quicker to 'search'.
There are many ways of doing that.
A materialized view as noted in your other thread php mysql full text search multiple table joined by id - keeps it all in mysql.
Using a external application. There are many examples, Lucene (variants include Solr and ElasticSearch), SphinxSearch, and many more.
This generally work in a similar way - setting up a dedicated copy of the data to make queries easier.
Use an external provider. Ther are many 'search as a service' systems (basically wrappers around the software mentioned in previous posts)
Building your own! Its possible to build a system yourself using just normal mysql tables. Basically an implementation of an inverted index will probably be the easiest.
Which you use is down to personal preference (eg, an external app is more work to setup, but overall is more powerful)
I recently started working with Yii PHP MVC Framework. I'm looking for advice on how should I continue working with the database through the framework: should I use framework's base class CActiveRecord which deals with the DB, or should I go with the classic SQL query functions (in my case mssql)?
Obviously or not, for me it seems easier to deal with the DB through classic SQL queries, but, at some point, I imagine there has to be an advantage in using framework's way.
Some SQL queries will get pretty complex pretty often. I just can't comprehend how the framework could help me and not make things more complicated than they actually are.
Very General rule from my experience with Yii and massive databases:
Use Yii Active Record when:
You want to retrieve and post single to a few rows in the database (e.g. user changing his/her settings, updating users balance, adding a vote, getting a count of users online, getting the number of posts under a topic, checking if a model exists)
You want to rapidly design a hierarchical model structure between your tables, (e.g. $user->info->email,$user->settings->currency) allowing you to quickly adjust displayed currency/settings per use.
Stay away from Yii Active Record when:
You want to update several 100 records at a time. (too much overhead for the model)
Yii::app()->db->command()
allows you to avoid the heavy objects and retrieves data in simple arrays.
You want to do advanced joins and queries that involve multiple tables.
Any batch job!! (e.g. checking a payments table to see which customers are overdue on their payments, updating database values etc.)
I love Yii Active Record, but I interchange between the Active Record Model and plain SQL (using Yii::app()->db) based on the requirement in the application.
At the end I have the option whether I want to update a single users currency
$user->info->currency = 'USD';
$user->info->save();
or if I want to update all users currencies:
Yii::app()->db->command('UPDATE ..... SET Currency="USD" where ...');
In any language when dealing with the database a framework can help you by providing an abstraction over the database.
Here is a scenario I know I found myself in many times during my earlier development days:
I have an application that needs a database.
I write a ton of code.
I put the SQL statements in the code along with everything else.
The database changes somehow.
I'm stuck with having to go back and make 100 changes to all my SQL statements.
It's very frustrating.
Another scenario I found:
I write a ton of code against a database.
Bugs come in. Lots of bugs. I can't figure them all out.
I'm asked to write tests for my code.
This is impossible because all my code relies on a direct implementation of the database. How do you test SQL statements when they're with the actual code?
So my advice is to use the framework because it can provide an abstraction over the database. This gives you two really big advantages:
You can potentially swap out the database later and your code stays the same! If you're using interfaces/some framework, then most likely you're dealing with objects and not SQL statements directly. A given implementation might know how to write to MySQL or SQL Server, but in general your code just says "Write this object", "Read that list."
You can test your code! A good framework that deals with data will let you mock the database so you can test it easily.
Try to avoid writing SQL statements directly in the application. It'll save you pain later.
I'm unfamiliar with the database system bundled with Yii, but would advise you to use it a little bit to start with. My experience is with Propel, a popular PHP ORM. In general, ORM systems have a class per table (Propel has three per table).
Now, there'll probably be a syntax to do lookups and joins etc, but the first thing to do is to work out how to use raw SQL in your queries (for any of the CRUD operations). Put methods to do these queries in your model classes, so at least you will be benefitting from centralisation of code.
Once you've got that working, you can migrate to the recommended approach at a later time, without getting overwhelmed with the amount of material you have to learn in one go. Learning Yii (especially how to share code amongst controllers, and to write maintainable view templates) takes a while, so it may be sensible not to over-complicate it with many other things as well.
Why to use Yii:
Just imagine that you have many modules and for each module you have to write a pagination code; writing in old fashion style, will need a lot of time;
Why not use Yii ClistView widget? Oh, and this widget comes with a bonus: the data provider and the auto checking for the existance of the article that is about to be printed;
When using Yii CListView with results from ... Sphinx search engine, the widget will check if the article do really exists, because the result may not be correct
How long will it take for you to write a detection code for non existing registration?
And when you have different types of projects will you addapt the methods?
NO! Yii does this for you.
How long would it take for you to write the code in crud style ? create, read, update, delete ?
Are you going to adapt the old code from another project ?
Yii has a miracle module, called Gii, that generates models, modules, forms, controllers, the crud ... and many more
at first it might seem hard, but when you get experienced, it's easy
I would suggest you should use CActiveRecord.It will give many advantages -
You can use many widgets within yii directly as mentioned above.(For paginations,grids etc)
The queries which are generated by the Yii ORM are highly optimized.
You dont need to put the results extracted from SQLs in your VO objects.
If the tables for some reason modified(addition/deletion of column,changing data type), you just need to regenerate the models using the tool provided by yii.Just make sure you try to avoid doing any code changes in the models generated by yii, that will save your merging efforts.
If you plan to change the DB from MYSQL to other vendor in futur, it would be just config change for you.
Also you and your team would save your precious development time.
A friend of mine and I just started an own project to learn about the newest and most future relevant web techniques: we decided to build an own object-oriented content management system with enhanced functionality like the community-points function on this site.
Using PHP as developing language and MySQL as database – which seems to be the standard database solution for everything on the internet – we quickly got into trouble saving our objects into the database, especially the references between objects. As a first solution we worked with table that - as we know now - are called "joints".
But then I started a research on how to solve this problem. Here's what I found:
there are object-oriented mapping solutions for relational databases, also for MySQL & PHP
there is a special database model designed for solving this particular problem: the object-oriented database
the object-relational database, a hybrid model that basically is a relational database with special handling for object savings
Since I don't want to start one more best-database-model-debate, here's my specific question:
Which of all solutions I have mentioned and you know from elsewhere is the one that's well supported from enough and not too expensive web-storage providers and is good enough for "basic" object-oriented functionality and thus be good choice implementing an object-oriented content managment system? (Again, the most painful thing we have crossed til now is the lack of referencing other objects in the database.)
Recommandations on particular database types are highly welcome!
You can try any "NoSQL" database, they are not relational. For example, MongoDB. Mongo stores data in BSON format which is like JSON and in PHP you can easily handle JSON data. You can find information about NoSQL databases at http://nosql-database.org/ and take a look about MongoDB as well http://en.wikipedia.org/wiki/MongoDB
I am thinking about writing a quick chat application for a client to help them solve some of their communication needs. Clearly, writing a simple chat is no brainer, but the company have serious scaling needs, so it is probably a good idea to build the service on a noSQL storage from the beginning.
Besides the obvious lack of transactions, which isn't one of our concerns, is it a good idea to use a noSQL storage for a chat?
MongoDB should be good enough if you're after scalability and performance. Most SQL engines would be overkill for this stuff. I doubt if you need complex data aggregation and other queries for chat data. Even with that, MongoDB has map-reduce capability to help you along.
NoSQL ist used if you have no fixed data model, this applies to document oriented applications where you have to store objects and documents where each one may have a different structure.
I don't think this is the case in your situation, since a chat log has a well defined fixed data model for example (user, time, text). I think a traditional SQL database may be the right fit for you. If used on client side only, SQLite will be the best fit, since there is no need to install or configure, simply redistribute the SQLite dll. Also the footprint is very small.
I would say no. SQLite is included in PHP... why not just use that? Or better still, why not use one of the hundreds of chat applications that already exist, and save yourself a whole load of development time.