I am developing an MVC framework (please don't question that, I know..) and currently designing a translation mechanism to maximally ease translation of applications, so far I have a lang folder which contains translation files for different pages
./lang/en/system.php
./lang/es/system.php
./lang/fr/system.php
and so on. Lets say this file contains translations of system messages such as
./lang/en/system.php
<?php
return array(
'yourIP' => 'Your IP address is :1'
);
To access that in a page I will use a facade class Lang, which will fetch the file based on the selected language (stored in session) and give me the translations.
Controller
public function index() {
return new View('index', ['translations' => Lang::get('system')]);
}
View
<h1><?= $translations->get('yourIP', System::getClientIP()) ?></h1>
This seems to work pretty fast as I can group translations efficiently in separate files for separate modules/pages.
The problem I am trying to solve now is with translating models. For example lets say I'm building a multilingual blog and saving posts in a database. Each post will need translations of its own, but theoretically there can be an unlimited amount of posts. The current method I'm using does not seem very practical.
What I will have to do is create a child directory and store translations like
./lang/en/posts/post-1.php
./lang/en/posts/post-2.php
...
./lang/en/posts/post-n.php
And that would be for every language, where in this file I will store all translatable (is this a word?) fields of the model and will load it in the model's constructor.
Problems regarding this solution:
The filesystem will get stuffed with lots of very small files - I'm not really a filesystem expert and I would like to ask if having a large amount of small files like that can cause harm to the filesystem itself including slowdowns of reads and such.
There will be n filesystem reads when retrieving a set of models, where n is the number of models. The hard drive is the slowest component in a computer, performing lots of FS reads in a script will present a significant slowdown, now with SSDs maybe not that much but still not a minor problem.
The other solution I came up with is use an additional system database table, which will store translations by table and primary key, something like
table INT
model_pk INT
lang INT
translations TEXT
where table will be a crc32 encoded number of the name of the table which the translations belong to, model will be the PK (id) of the model, lang no need to explain and translations will be a serialized string containing all translatable properties.
Problems with this approach:
Forces developer to use a database and obligates them to have a certain table (currently the framework does not require you to have a database and thus there are no system tables when you actually use one).
models with composite primary keys will not be able to benefit from this since the model column can not store a composite key, so only models with a single column primary key will be translatable.
These are just my observations and thoughts, I may be wrong or I may be missing something. I'm posting this question to get advice on which solution will be less problematic from someone with greater experience or propose a completely different one, I'm open to everything.
If you need to build something great first you must understand that what you make until now is called localization (setting app language which will case translation of static data) but you still need to make translation (which mean translate dynamic data like data came from database)
For more details http://content.lionbridge.com/the-difference-between-translation-and-localization-for-multilingual-website-projects-definitions/
For more about localization: https://laravel.com/docs/5.3/localization
and see also https://github.com/mcamara/laravel-localization
For more about translation you can find this package For Laravel Framework interested https://github.com/dimsav/laravel-translatable
Related
I'm creating a language app that currently only features Mandarin Chinese and Spanish.
Currently, I have self-created dictionary simply loaded as JSON without storing in the DB, but I've found full downloadable dictionaries, such as CEDICT for Chinese to do the definitions for me. That being said, this file is 115k rows long, with 6 columns per row.
I also need to do this for Spanish, and then every other language I plan on including.
Notes:
MySQL DB
Laravel ORM (PHP)
That being said, what's the best way to store this data?
I'm assuming as separate tables, dictionary_zh, dictionary_es, but I could also store each dictionary in a dictionary table, with an added column for locale and query based on that. This SO answer states that 1m records isn't "too much" for a table to handle, it simply defines on how you index the table.
Btw, anyone have a recommendation for a good downloadable Spanish - English dictionary?
Note: I'm downloading the dictionary and cutting it up into something I can load into a CSV
Traditional Simplified Pinyin Meaning Level Quest
佟 佟 Tong2 surname Tong 1 2
...
I'm translating it by simply passing in the identifying character, in this case佟, and grabbing its Meaning.
I would store each dictionary in a separate table to abstract how I fetch the definition for a word depending on the locale, without the need to know how a dictionary (mapped as Dictionary type in the diagram below) operates its translation. This is useful when you might have dictionaries which don't reside in your DB, such as ones translating via an API.
The method translate() is implemented differently for each type of Dictionary (in your case ChineseDictionary or SpanishDictionary).
Another advantage of this approach from a data management point of view is that you will not have to make a lot of operations on the data when new versions of your dictionary are released, which makes it cheap to maintain.
I want to store some user settings and I thought of 3 options, since it's my first project and I want to start it the right way, I can't figure out which is the the optimal way of doing this..
Storing single settings directly in columns in the user table
Having a user_settings table with the columns setting_name, setting_value and user_id
Storing a JSON string in the user table, in a column named something like "user_settings_json"
On a design analysis, I noticed wordpress stores it in a separate table, but I'm not sure that's necessary for every application (since mine does not have nearly as many user settings as wp does)
I'm using Laravel, PHP, javascript/jquery.
Which do you guys think would be, most useful, overall better, in terms of design, serviceability and performance?
Storing a JSON string ? NO !
You want to be able to isolate the settings, just query what you need. Therefore, keep it in separate columns!
As for the question if you have to make a separate table, no, you don't have to. When you just got some simple settings you can just add the columns to the existing table of users. Be aware of the limitations here. If you do need advanced settings, i would recommend using a separate table. Better do it too early. Not every setting might apply to every user. For example when you've got premium accounts who can have more settings. So, keeping it separate is what I would do.
Btw, I wouldn't let the columns start with 'setting_' if they are already in a separate table containing 'setting' in the name.
Conclusion: Option 2 :)
Recently I've made package for my project which uses Laravel 9 and it allows you to add settings to any Laravel model. It can cast values to primitive types like bool, int, but also to custom classes. Eg.:
$user->settings->get('is_gamer');
$user->settings->set('games_count', 10);
// or global site scoped
Settings::get('display_annoucement');
// more advanced usage with definition of custom class
$address = $user->settings->get('address');
$address->country = 'Poland';
$address->zip = '11-222';
$address->city = 'Warsaw';
$user->settings->put('address', $address);
// any model that implements trait
$article->settings->get('show_breadcrumbs');
$post->settings->get('allow_replies');
You can find it here: https://github.com/npabisz/laravel-settings
I have a Postgres DB containing some configuration data spread over several tables.
This configurations need to be tested before they get deployed to the production system.
Now I'm looking for a way to
store single configuration objects with their child entities in SVN, and
to deploy this objects with child entities to different target DB's
The point is that the relations between the objects needs to be somehow maintained without the actual id's which would cause conflicts when copying the data to another DB.
For example, if the database would contain data about music artists, albums and tracks with a simple tree table schema like artist -> has albums -> has tracks, then the solution I'm looking for would allow to export e.g. one selected album with all tracks (or one artist with all albums with all tracks) into one file which could be stored to SVN and later be 'deployed' to whatever DB which has the same schema.
I was thinking of implementing something myself, e.g. to have config file describing dependencies, and an export script which replaces id's with PHP variables and generates some kind of PHP-SQL INSERT or UPDATE script.
But then I thought it would be really silly not to ask before to double check if something like this already exists :o)
This is one of the arguments for Natural Keys. An album has an artist and is made up of tracks. No "id" necessary to link these pieces of information together, just use the names. Perl-esque example of a data file:
"Bob Artist" => {
"First Album" => ["My Best Song", "A Slow Song",],
"Comeback Album" => ["One-Hit Wonder", "ORM Blues",],
}, "Noname Singer" => {
"Parse This Record!" => ["Song Named 'D'",],
}
To add the data, just walk the tree creating INSERT statements based on each level of parent data and if you must have one, use "RETURNING id" (PostgreSQL extension) at the end of each INSERT statement to get the auto-generated ids to pass to the next level down in the tree.
I second Matthew's suggestion. As a refinement of that concept, you may want to create "derived natural keys", for example "bob_artist" for "Bob Artist". The derived natural key would be well suited as a filename when storing the record into svn, for example.
The derived natural key should be generated such that any two different natural keys result in different derived natural keys. That way conflicts can't happen between independent datasets.
The concept of Rails migrations seems relevant although it aims mainly on performing schema updates: http://guides.rubyonrails.org/migrations.html
The idea has been transferred into PHP with the name Ruckusing, but seem to support only mySQL at this point: http://code.google.com/p/ruckusing/wiki/BigPictureOverview
Doctrine also provides migrations functionality but seems again to focus on schema transformations rather than on migrating or deploying data: http://www.doctrine-project.org/projects/migrations/2.0/docs/en
Possibly Ruckusing or Doctrine could be used (abused?) or if needed modified / extended to do the job?
lacking a fellow programmer to talk over the right approach for my problem, I decided to ask you. What is your preferred approach of mapping dictionary tables to a model in MVC paradigm, regardless of the MVC framework / environment you are using?
My problem is I have a couple of database tables that only serve as dictionaries and are related to other tables as foreign keys. A good example would be a table request having a status_id where statuses are kept in a separate status table.
Now, the latter table needs to be mapped to a model on the code-side of the application. I can either:
Define all the statuses as constants so they can be referenced in the code without poking those dreaded 'magic numbers' here and there. However, any change to the dictionary (database-side) would require a code modification.
Omit the `status` table at all and just define meaningful constant to be used across the code. Pros: one place to rule them all. Cons: all changes require diving into the code, now the database features 'magic numbers' not really being foreign keys
Try to translate statuses into the model automagically, adding a field like 'const_name' to the 'statuses' table and them creating the constants on the fly while loading the model. This one seems to have the most sense for me.
Would you mind to share your usual approach to this issue?
Best,
Bartek
If it's just going to be a set of constants that are contained in the database instead of code, you could have a static class load the status constants for everyone else to use. That way there's no duplication between db and code, and no magic numbers.
edit: since it's a static class, you could have it lazy load the constants. Don't hit the database until the first time someone asks for a status value.
I'd say if you going to change it often it's better to go with table. Otherwise static class is fine (for example no point having table to store sex, or list of states).
What I really like about Entity framework is its drag and drop way of making up the whole model layer of your application. You select the tables, it joins them and you're done. If you update the database scheda, right click -> update and you're done again.
This seems to me miles ahead the competiting ORMs, like the mess of XML (n)Hibernate requires or the hard-to-update Django Models.
Without concentrating on the fact that maybe sometimes more control over the mapping process may be good, are there similar one-click (or one-command) solutions for other (mainly open source like python or php) programming languages or frameworks?
Thanks
SQLAlchemy database reflection gets you half way there. You'll still have to declare your classes and relations between them. Actually you could easily autogenerate the classes too, but you'll still need to name the relations somehow so you might as well declare the classes manually.
The code to setup your database would look something like this:
from sqlalchemy import create_engine, MetaData
from sqlalchemy.ext.declarative import declarative_base
metadata = MetaData(create_engine(database_url), reflect=True)
Base = declarative_base(metadata)
class Order(Base):
__table__ = metadata.tables['orders']
class OrderLine(Base):
__table__ = metadata.tables['orderlines']
order = relation(Order, backref='lines')
In production code, you'd probably want to cache the reflected database metadata somehow. Like for instance pickle it to a file:
from cPickle import dump, load
import os
if os.path.exists('metadata.cache'):
metadata = load(open('metadata.cache'))
metadata.bind = create_engine(database_url)
else:
metadata = MetaData(create_engine(database_url), reflect=True)
dump(metadata, open('metadata.cache', 'w'))
I do not like “drag and drop” create of data access code.
At first sight it seems easy, but then you make a change to the database and have to update the data access code. This is where it becomes hard, as you often have to redo what you have done before, or hand edit the code the drag/drop designer created. Often when you make a change to one field mapping with a drag/drop designer, the output file has unrelated lines changes, so you can not use your source code control system to confirm you have make the intended change (and not change anything else).
However having to create/edit xml configuring files is not nice every time you refractor your code or change your database schema you have to update the mapping file. It is also very hard to get started with mapping files and tracking down what looks like simple problem can take ages.
There are two other options:
Use a code generator like CodeSmith that comes with templates for many ORM systems. When (not if) you need to customize the output you can edit the template, but the simple case are taken care of for you. That ways you just rerun the code generator every time you change the database schema and get a repeatable result.
And/or use fluent interface (e.g Fluent NHibernate) to configure your ORM system, this avoids the need to the Xml config file and in most cases you can use naming conventions to link fields to columns etc. This will be harder to start with then a drag/drop designer but will pay of in the long term if you do match refactoring of the code or database.
Another option is to use a model that you generate both your database and code from. The “model” is your source code and is kept under version control. This is called “Model Driven Development” and can be great if you have lots of classes that have simpler patterns, as you only need to create the template for each pattern once.
I have heard iBattis is good. A few companies fall back to iBattis when their programmer teams are not capable of understanding Hibernate (time issue).
Personally, I still like Linq2Sql. Yes, the first time someone needs to delete and redrag over a table seems like too much work, but it really is not. And the time that it doesn't update your class code when you save is really a pain, but you simply control-a your tables and drag them over again. Total remakes are very quick and painless. The classes it creates are extremely simple. You can even create multiple table entities if you like with SPs for CRUD.
Linking SPs to CRUD is similar to EF: You simply setup your SP with the same parameters as your table, then drag it over your table, and poof, it matches the data types.
A lot of people go out of their way to take IQueryable away from the repository, but you can limit what you link in linq2Sql, so IQueryable is not too bad.
Come to think of it, I wonder if there is a way to restrict the relations (and foreign keys).