I am working on a project that use MongoDB to house transactional data with MySQL housing all other data. For various reasons, we are looking at the possibility of moving from MongoDB to the MySQL xDevApi with the 8.0 version. In preparation for this possibility as well as to ease the learning curve as well as other database considerations, I am looking at creating a wrapper that will allow us to switch the database backend without having to update all the places in the code that interfaces with MongoDB.
I have an outline of one already, but am not sure it is the best way to do it. I think it is a decent start, I am just not certain of the file/folder structure.
The current file/folder structure is as follows:
\DocumentStore
abstract class DocumentStoreQueryBuilder
interface IDocumentStore
interface IDocumentStoreConnection
\DocumentStore\Mongo
class Connection implements IDocumentStoreConnection
class Query implements IDocumentStore
class QueryBuilder extends BuilderAlias
My thought is to use language similar to a relational database in order to help with the initial learning curve of those coming from a RDB background (the majority of those that will be coming onto the project).
I am sure that there is a better way to organize things, but to be honest, I am not too terribly familiar with Document Storage generally.
This is code that works with what I have so far.
In the connection file that is called from all files that need the connection.
$mgdbdoc = new DocumentStore\Mongo\Connection();
$connectionString = $mgdbdoc->buildConnectionString($settings);
$mgdbdoc->connect($connectionString);
$collection = new DocumentStore\Mongo\Query($mgdbdoc);
Defines the collection for action on. This could, theretically, be saved to a unique class name for each collection if necessary.
$collection->setCollection('transactions');
To be called whenever necessary.
$result = $collection->insert($document);
$result = $collection->filter($filter)->limit(2)->descending('_id')->select();
I haven't put in the update and remove functionality yet, and the and/or functionality is proving difficult, but I can get that.
My question is for any advice regarding this project. Any thoughts on proceeding forward? I haven't been able to locate a wrapper for multiple NoSQL databases like there is for PDO. I would appreciate any thoughts or advice.
I don't want to use drupal db_ rules.. I'd rather use my own pdo techniques for database queries and updates etc.. But I've managed to be able to get pdo to work, the only issue is that I have to use a connect on every custom page, some users in my community can edit the pages and will be able to see the connection information.. require & require_once do not seem to work.
Has anyone else had this issue and knows how to avoid it?
You could create your own module and use the init hook:
https://api.drupal.org/api/drupal/modules%21system%21system.api.php/function/hook_init/7
function yourmodulename_init() {
// This code will be executed on every page
}
However you should try to use the drupal database mechanics as it comes with a lot of best practices and powerful tools. E.g. views integration:
https://api.drupal.org/api/views/views.api.php/function/hook_views_data/7
In Drupal 7, Database::getConnection allows you to retrieve DatabaseConnection objects for all configured databases. And DatabaseConnection extends PDO so if your really want to bypass Drupal's Database API (which is itself based on PDO and either re-use or extends PDO classes all over the place), you can use it as you would use your own PDO object.
But if user of your community are able to edit your custom page, I'm guessing that your are actually embedding PHP code inside Drupal nodes using the PHP filter. Which is itself a very bad idea (worse than ignoring Drupal's Database API). You should instead consider writing custom modules tailored for your needs.
I have heard, (thanks to SO) that mysql_* family of functions is vulnerable to SQL injections, so it is always recommended to use mysqli_* OR PDO approach.
So, I tried to trace out which of these approaches is used in Codeigniter as I have been using CI since 4 months. But I could not get it out.
Can anybody tell me which of these or any other approach is used in CI?
Thanks in advance.
CodeIgniter uses whichever method you choose, based on your config/database.php file. For example, if you choose mysqli as your driver, it will use the mysqli family of functions. If you choose mysql, it will use the mysql family, and so on.
There is a PDO driver, but it's not the most stable thing in the world in the current releases.
If you're using the "active record" functions in CodeIgniter (which should really be called "query builder" functions, since they don't follow the Active Record pattern), then all of your data is automatically sanitized with the appropriate functions. If you say, for example, $this->db->where('field', $value), and you're using the mysqli driver, CodeIgniter will automatically call mysqli_real_escape_string on the $value input you pass to it, rendering the query safe.
I should also note that this automatic sanitization only applies to the active record helper functions, like $this->db->get and the like. If you try to run your own query using $this->db->query("SELECT * FROM table WHERE field = '" . $field . "'") then you need to take care to protect yourself, since you're bypassing CodeIgniter's security mechanisms. Running a query directly with the query() method is like saying "don't worry, I know what I'm doing." You need to specifically ask CodeIgniter to sanitize certain values for you by calling $this->db->escape() or its related functions. If you just concatenate variables into your query, then you're going to open yourself up to SQL injections.
Accoring to the Symfony 2 Documentation, you have to use the following PHP code to connect to the database and execute a query...
$conn = $this->get('database_connection');
$users = $conn->fetchAll('SELECT * FROM users');
I'm a complete novice with Symfony 2 but I am experienced with OOP. I was wondering if it was possible to have a globally available $conn variable that I can access from any bundle. The $conn variable would contain the value of $this->get('database_connection') so I don't have to retype $conn = $this->get('database_connection');it every time I want to make a new query.
Thanks!
global variables are most of the time NOT something you want in OOP. They are confusing when it comes to a method which deals with multiple variables and they might even be hidden by local variables. For me, working with statements like
$anything = $this->get('what.the.hell.why.arent.those.identifiers.shorter');
is as annoying as for you so I ended up in creating one subclass of Symfony\Bundle\FrameworkBundle\Controller\Controller per project which provides methods which call get with the actual identifiers. In your case I would create a method
public function getDatabaseConnection()
{
return $this->get('database_connection');
}
In general - why don't you use Doctrine for managing the DB connection? Most of the queries can be done by the ORM and this is the way to work with a real object-oriented interface to the database. Think about it, I'm also playing with Symfony2/Doctrine since some days and it really feels good. In the beginning, it might look like a hell of configuration, but once you've done the basic configs, the development is really fast! :)
I'm creating a small framework for my web projects in PHP so I don't have to do the basic work over and over again for every new website. It is not my goal to create a second CakePHP or Codeigniter and I'm also not planning to build my websites with any of the available frameworks as I prefer to use things I've created myself in general.
I have had no problems in designing and coding the framework when it comes to parts like the core structure, request handling, and so on, but I'm getting stuck with designing the database interface for my modules.
I've already thought about using the MVC pattern but found out that it would be a bit of an overkill for my rather small project(s).
So the exact problem I'm facing is how my frameworks modules (viewCustomers could be a module, for example) should interact with the database.
Is it (still) a good idea to mix in SQL directly into PHP code? (Would be "old way": mysql_query( 'SELECT firstname, lastname(.....))?
How could I abstract a query like the following?
SELECT firstname, lastname FROM customers WHERE id=X
Would MySQL "helper" functions like
$this->db->customers->getBy( 'id', $x );
be a good idea?
I'm not really sure because they tend to become useless when dealing with more complicated queries like the pretty much trivial one above.
Is the "Model" pattern from MVC my only real option to solve this?
What do you currently use to solve the problems shown above?
I believe you just want to get access to your DB from your module. I'd avoid using mysql_query directly from the code. Rather, going for simple model with abstracted DB access would be easy and straight-forward.
For example, you can have a file like models/Customers.php with this code:
<?php
class Customers {
public function getById($id) {
$sql = "SELECT first_name, last_name FROM customers WHERE id='$id'";
$res = $DB::getRow($sql);
return ($res);
}
}
I am assuming some kind of DB helper is already instantiated and available as $DB. Here is a simple one which uses PDO.
Now, you should include this in your module and use the following way:
<?php
include_once "models/Customers.php";
$customers = new Customers();
$theCustomer = $customers->getById(intval($_REQUEST['cust_id']));
echo "Hello " . $theCustomer['first_name']
Cheers.
have you looked into http://www.doctrine-project.org/ or other php orm frameworks (zend_db comes to mind)?
If you need speed, then use raw queries (but you should really use PDO with prepared queries).
If you want something more OOP, you can —as you suggest it— design this with helpers.
Once, I've designed something similar which had the following concept:
DB connection/handler classes (handling multi-connections to different databases and different servers such as MySQL, Oracle, etc.);
A class per action (ie. SELECT, DELETE, etc.);
Filter classes (eg. RangeFilter);
The code looked something like this:
$select = new Select('field1', 'field2', );
$result = $select->from('myTable')
->addFilter(SQLFilter::RangeFilter, 'field2')
->match(array(1, 3, 5))
->unmatch(array(15, 34))
->fetchAll();
It's a simple example of how you can build it.
You can go further and implements automated handling of table relations, field type check (using introspection on your tables), table and field alias support, etc.
It might seem to be a long and hard work, but actually, it won't take you that much time to make all these features (≈1 month).
Three tips:
Use Stored Procedures (so you can separate the php from the db)
Use PDO/MySQLi for prepared statements CALL NEWS_LIST(?, ?)
Use a Static Class for your DB. Allows you to access it within any module.
Raw SQL is still the winner for me, I like to control what I send to the server (for cases like index usage, complex JOIN clauses and etc) so I generally stay away from helper functions.
You should use PDO which already provides a lot of power and if that's not enough, you can extend that (possibly with your own functions, such as checking for hits on Memcached/APC before actually querying the database). You can also extend the class to implement your own SQL functions like:
function getUser($user_id) {
return $this->query("SELECT * FROM users WHERE id = " . (int) $user_id);
}
Of course that, from the model you should still be able to send:
$this->db->query("SELECT * FROM users WHERE id = " . (int) $user_id);
and get the same result. The functions should act merely as a shortcut and the extended class should not be included with the framework as it will be site-dependant.
The MVC pattern will fit nicely into this because you can use the database merely as a driver and your model can then transform the data into what you need. It's not hard to create a simple MVC structure and it will bring you benefits later.
You sound like me. Have you seen http://github.com/Xeoncross/micromvc and the one file ORM in http://github.com/Xeoncross/database? Dig through my code and I think you will find what you're looking for.
The solution is to use the full raw power of some queries - while still allowing ORM and query builders (like codeigniter's AR) for other things.
Both are good.
Not that i know the definitive answer (nor do i think it exists), but i thought i can just share what i have here. I use my own db 'framework', lightweight (~1000 lines currently) and easy to use. My main goal was to simplify the use of sql, not to 'hide' it from the programmer (me:). Some examples:
// row() is 'query' + 'fetch' in one
$user = $db->row("select * from users where id=25");
// the same, injection safe
$user = $db->row("select * from users where id=?", $_GET['id']);
// ? placeholders are smart
$someUsers = $db->rows("select * from users where id IN(?)", array(1, 2, 10));
// ...and even smarter
$data = array('name' => 'Joe', 'age' => 50);
$id = 222;
$db->exec("update users set ?a where id=?", $data, $id);
// 'advanced' fetch functions
$topNames = $db->vlist("select name from users order by name limit 10");
$arrayOfIds = $db->nlist("select id from users where age > 90");
// table() returns a Table Gateway
$db->table('users')->delete('where id=?', 25);
// yes, this is safe
$db->table('users')->insert($_POST);
// find() returns a Row Gateway object
$db->table('users')
->find('where name=?', 'Joe')
->set('status', 'confirmed')
->save();
Understand this: database interaction is a solved problem.
So unless you really want to do it a) for the experience or b) because you're OCD and want to know every character of the code you'll be using, then I'd choose an existing solution.
And there are many: PEAR::MDB2, Zend::Db, Creole, Doctrine, Propel, just to name a few.
I've just come off the "helper functions" path and the one thing that bugged me was that I continued adding functions in one file which grew and grew with identical or similar or defunct functions. I think the line count was at 600 and that is way to much for a single file in my opinion. This has not put me off the idea but I'll be more organised for the next trek. I'll probably split the db functions into multi files according to the db operation (select, insert etc...).
So my advice is to go try the "helper functions" and be as organized as you can.
Also, I used PDO for the first time and quite liked it. Its not as low tech as the mysql() functions or as bloat tech like some we could mention but won't. I'll be using PDO again.
It seems like there are many different opinions on this topic and as I haven't found a really satisfying answer here yet and the bounty is nearly over, I'll just write what I have come up in the last days after some trial and error:
I'm using a singleton MySQL class to handle the connection and the very basic queries as well as errors that may occur.
Single pages like /users/show/1 (using mod_rewrite) don't use raw SQL but some kind of lightweight ORM that works like in the following example:
$user = $this->db
->users
->getBy( 'id', $id );
$this->db is an instance of a Database Abstraction class with a __get( $tableName ) method. Accessing the undefined users property then triggers it. The rest explains itself; A query is formed from the arguments passed to getBy( ) (SQL escaping is also handled by it) and its results are returned as an array.
I haven't finished the whole idea yet, but adding a new user to the database could look like the following:
$user = $this->db
->users
->new;
$user->id = 2;
$user->name = 'Joe';
$user->save( );
As I said the concept isn't really completed and may have (huge) flaws in it. Yet I think that it may be easier to write, more secure and easier to maintain than plain MySQL.
Some other good sides of the whole "thing" would be that it is small, therefore rather fast and also pretty straightforward.
I know that this can't compete with the extremely powerful ORMs and frameworks already out there but I'm still creating this for some reasons mentioned in one of my comments above.
If you do plan on making a database class it may be an idea looking into making it a singleton, allowing you to use it without declaring it/creating it, as...
global $db;
$db = new db;
$db->query ('... sql ...');
is kinda redundant when you can do
db::query ('... sql ...');
I have a set of SQL functions that I use on a near regular basis to reduce what used to be a multi-line escaped lot of SQL to a single call, for example:
get_element ($table, $element, $value, $column='id');
get_row ($table, $value, $column='id');
So if you just want to get the name from a table 'customers' where the id is 4 you:
$name = db::get_element ('customers', 'name', 4);
There are also accompanying functions query_element and query_row, where you just pass it an SQL string and it returns a single element/row.
Along with the functions for insert/update, e.g.
$array = array (
'name' => 'bob jones',
'age' => 28
);
$insert_id = db::insert_array ('customers', $array);
$customer_details = db::get_row ('customers', $insert_id);
$customer_details['age'] = 30;
db:update_array ('customers, $customer_details);
Would create a new row, pull the details back out, update the age, then re-write it to the database.
Creating custom SQL access modules on a per-table basis is generally a mistake I have always found - it's better to just generically query the database using sensible functions.
If you do have to use anything with complex joins then it is always best to create function for it getCustomerInfo () for example, but if you just want a generic table value lookup making lots of custom methods just increases the chances of mistakes in one of them. Plus escaping data is very important - if you can use non-sql as much as possible and funnel it through a few core functions you can ensure everything is properly escaped fairly easily.
If you want to look at my custom database class let me know.