database: best practice countries, country codes, country phone codes - php

I am looking for a "best practice" if you store country codes in a database but couldn't find a "this is the right way" for that. I want to store the 2 chars country code and also the country phone codes (eg Germany would be "DE" and "+49").
Actually my plan is as follows: create one table countriesand one table with country_codes. Something like this:
TABLE: countries
id INT(11)
code CHAR(2)
TABLE: country_codes
id INT(11)
country_id INT(11) FORGEIGN KEY (countries -> id)
phone_code VARHAR(6)
I think I need to split them because some countries have more than one phone code. This way a country can have multiple phone codes.
But to my question: is that the "best practice" to do that? No only from that point "that will work" also more from that view if I want to rollout my application in "all" countries or if I want to translate the app in multiple languages (in that case I wanted to use the countries table also for the different languages.
What is your way to do thing like taht, if you want to able to translate your app in any language without the need of re-coding stuff and if you also need a list of all countries in you app?
If it should matter: I am planing to go with laravel for this app.

Country codes are standardized to two letters by ISO 3166-1-alpha-2, so storing them that way will work. It's often helpful to include a country name in the table, so a user can choose the right country without having to know all the codes.
Telephone numbers are far less standardized. The ITU offers recommendation E.164 for representing actual telephone numbers (called "directory numbers" in telephony jargon). Country codes are defined as one to three digits. North America (including USA, Canada and many Carribean nations) all are part of the North American Numbering Plan and share the country code 1.
Directory numbers are typically preceded by + and punctuated by dots. So, for example, the published New York City directory assistance number is (or was when they still had such a service) +1.212.555.1212. If you called that number from someplace in Europe, you would see the + and substitute your local international prefix. In NANP, multiple nationalities have the same country code.
But, UK is strange. Calling from outside the country, it's +44.exchange.number. But calling long distance from within the country it's (0) exchange.number.
My point: it's hard to get it right if you try to compose directory numbers with a country code in your software. You're probably better off asking users to provide their telephone numbers with the international prefix.
You should definitely not tie E.164 country codes to ISO 3166 two-letter country codes by putting them as different columns on the same row of a table. You need two separate tables to be future proof. The standardization organizations are different and do their own things, so your data model should reflect that.
Read this: Falsehoods Programmers Believe About Telephone Numbers.

My DB looks like this:
> id int(11) Auto Increment (Just an ID (primary key))
> iso char(2) (2-letters ISO code)
> name varchar(80) (normalized name (all uppercase))
> nicename varchar(80) (Nicely formatted name)
> iso3 char(3) NULL (3-letters ISO code)
> numcode smallint(6) NULL (numeric ISO code)
> phonecode int(5) (phone code like '1' for USA, without '+')
It should be more then enough. You get user's phone number, remove zeroes at the beginning, remove any non-numerical characters, add a country code from DB and you are good to go!
Example:
1) User input (045) 111-22-33, Germany
2) You convert it to 451112233
3) Add code of Germany (49) from DB. You get 49451112233. Add '+' if you wish.
4) Now you can make a call or send SMS with Twilio or any other service.
If you want to "easily" translate the site to other languages, store all of your text in database and pull the right version depending on user's language preferences.

Based on the answers I would do the following:
DB Tables:
+------------------------------------------------------------+
| Table: countries |
+--------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+-------+
| id | int(11) | NO | | NULL | |
| iso_code2 | char(2) | NO | | NULL | |
| iso_code3 | char(3) | NO | | NULL | |
| num_code | int(3) | NO | | NULL | |
| name | varchar(48) | NO | | NULL | |
| nicename | varchar(48) | NO | | NULL | |
+--------------+--------------+------+-----+---------+-------+
// will store all countries available
+------------------------------------------------------------+
| Table: country_phonecodes |
+--------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+-------+
| id | int(11) | NO | | NULL | |
| country_id | int(11) | NO | | NULL | |
| phonce_code | int(6) | NO | | NULL | |
+--------------+--------------+------+-----+---------+-------+
// based on this page: https://countrycode.org/ there are
// countries with more than one code
// and also codes can be 6 chars long
+------------------------------------------------------------+
| Table: languages |
+--------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+-------+
| id | int(11) | NO | | NULL | |
| code | char(2) | NO | | NULL | |
| locale | char(5) | NO | | NULL | |
| name | varchar(50) | NO | | NULL | |
| native_name | varchar(50) | NO | | NULL | |
| flag | varchar(10) | NO | | NULL | |
+--------------+--------------+------+-----+---------+-------+
// table for available translations of the app
+------------------------------------------------------------+
| Table: country_languages |
+--------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+--------------+------+-----+---------+-------+
| id | int(11) | NO | | NULL | |
| country_id | int(11) | NO | | NULL | |
| language_id | int(11) | NO | | NULL | |
+--------------+--------------+------+-----+---------+-------+
// table for language suggestions for a given country
And some example inserts:
+---------------------------------------------------------------------------------------+
| Inserts: countries |
+-----+------------+------------+-----------+---------------------+---------------------+
| id | iso_code2 | iso_code3 | num_code | name | nicename |
+-----+------------+------------+-----------+---------------------+---------------------+
| 1 | de | deu | 276 | GERMANY | Germany |
| 2 | do | dom | 214 | DOMINICAN REPUBLIC | Dominican Republic |
| 3 | be | bel | 056 | BELGIUM | Belgium |
+-----+------------+------------+-----------+---------------------+---------------------+
+----------------------------------+
| Inserts: country_phonecodes |
+-----+-------------+--------------+
| id | country_id | phonce_code |
+-----+-------------+--------------+
| 1 | 1 | 49 |
| 2 | 2 | 1809 |
| 3 | 2 | 1829 |
| 4 | 2 | 1849 |
| 5 | 3 | 32 |
+-----+-------------+--------------+
+----------------------------------------------------------+
| Inserts: languages |
+-----+-------+---------+---------+--------------+---------+
| id | code | locale | name | native_name | flag |
+-----+-------+---------+---------+--------------+---------+
| 1 | de | de_DE | German | Deutsch | de.svg |
| 2 | do | es_DO | Spanish | Español | es.png |
| 3 | be | fr_BE | French | Français | fr.jpg |
| 4 | be | nl_BE | Dutch | Nederlands | nl.png |
| 5 | be | de_BE | German | Deutsch | de.svg |
+-----+-------+---------+---------+--------------+---------+
+----------------------------------+
| Inserts: country_languages |
+-----+-------------+--------------+
| id | country_id | language_id |
+-----+-------------+--------------+
| 1 | 1 | 1 |
| 2 | 2 | 2 |
| 3 | 3 | 3 |
| 4 | 3 | 4 |
| 5 | 3 | 5 |
+-----+-------------+--------------+
I think this should work and be useable for any project where a country list and/or i18n is needed.
If a user comes from Belgium, he can choose from the list of available languages/translations. He will get a suggestion for FR, NL and DE but will still be able to choose es_DO as prefered language.
Think this should cover all needs - but if anyone sees a problem in that or has ideas/comments: I would be happy if I can improve this solution :)

Related

Sorting on data from language file

I want to have translations for a number of languages coming from the default Language files (as provided by CI). I'd also like to do some sorting on these values (e.g. to insert them in a dropdown list, I want them to be alphabetically sorted).
I have a project running in CodeIgniter and am in the process of moving all hardcoded text to the language files. This also includes some database tables.
For example, I have the following table:
+-------------+---------------------+--------------------+---------------+
| language_id | language_nl | language_en | language_code |
+-------------+---------------------+--------------------+---------------+
| 1 | Afrikaans | Afrikaans | AF |
| 2 | Albanese | Albanian | SQ |
| 3 | Arabisch | Arabic | AR |
| 4 | Armeens | Armenian | HY |
| 5 | baskisch | Basque | EU |
| 6 | Bengalees | Bengali | BN |
| | | | |
+-------------+---------------------+--------------------+---------------+
Now I would like to transfer this table to my language file.
I will first start by deleting the language_nl and language_en columns.
A new column will have to be introduced as well, language_key.
So it should look like this:
+-------------+--------------+---------------+
| language_id | language_key | language_code |
+-------------+--------------+---------------+
| 1 | language_af | AF |
| 2 | language_sq | SQ |
| 3 | language_ar | AR |
| 4 | language_hy | HY |
| 5 | language_eu | EU |
| 6 | language_bn | BN |
+-------------+--------------+---------------+
Then in my language file I will add the following lines:
$lang["language_af"] = "African";
$lang["language_sq"] = "Albanian";
$lang["language_ar"] = "Arabic";
And so on...
I will then create a manager class to help me in finding the translations for each necessary language for the key.
It would look something like this:
public function getLanguageName($key) {
return lang(key);
}
Now I have two main questions:
1) Is this a good way of approaching the issue I have?
2) Let's say I need the languages to be shown in a drop-down. I want this dropdown to be alphabetically sorted (order will be different based on the language selected, of course). How do I approach this issue? Do I load in all the values from the language file, put them in an array and then sort that array?

Database structure for multiple locations of an organisation

I'm creating an application using PHP (Codeigniter/MySQL) and within the application are organisations.
Each organisation can have multiple locations, regions, departments, etc (I'm calling these areas)
Each area has an administrator, and sometimes I will need to escalate things to a higher area.
I've currently got all the data in 1 table, and I am using a parent_area_id and area_level to determine the parents,children etc.
But I think this is very inefficient, and I've been pointed towards closure loops, which I have no knowledge of.
Here the database table, is this ok, will it be efficient or is there a better way to do it?
+----------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+-------------+------+-----+---------+----------------+
| area_id | int(12) | NO | PRI | NULL | auto_increment |
| area_title | varchar(40) | NO | | NULL | |
| area_name | varchar(40) | NO | | NULL | |
| address1 | varchar(40) | YES | | NULL | |
| address2 | varchar(40) | YES | | NULL | |
| address3 | varchar(40) | YES | | NULL | |
| town | varchar(20) | YES | | NULL | |
| county | varchar(20) | YES | | NULL | |
| post_code | varchar(10) | YES | | NULL | |
| has_ra | varchar(1) | YES | | 0 | |
| org_id | int(12) | NO | MUL | NULL | |
| parent_area_id | int(8) | YES | | NULL | |
| area_level | int(1) | YES | | NULL | |
+----------------+-------------+------+-----+---------+----------------+
EDIT:
(better explanation of how this is being used)
1) Areas relate to customers of the business only.
2) The areas are different area(region,location,department) that a customer might have. (South region, Oxford Office, Accounts Dept).
3) Each area may have many employees allocated.
SO
If I had a regional administrator for example, they might have the following areas under them: e.g:
South Region
Oxford office
Sales Department
Accounts Department
London Office
Marketing
Planning
SO
If I wanted to get the user_id's of all employees under the regional administrator, using the above database structure, i would need to:
1) Query the db to get all area_id's that have a parent_area_id of the regional administrator.
2) Loop through each returned area_id, and query the db and get all area_id's that have a parent_area_id of the returned area_id
3) Continue looping through returned area_id's until we get to the bottom level
4) Query the db to get all user_id's that have an area_id of all above returned records
SO
That doesn't seem very efficient, and needs multiple SQL queries and programming loops to get a list of users associated with a regional manager.
If thats the most efficient way to do it then fine I just don't seem convinced, and im sure there must be an easier way?
There's no serious problem here if you're dealing with a situation where you're escalating one level at a time. I've got no idea how "closure loops" would factor in here, that's programming related, not a database schema concern, and is largely a matter of personal preference.
So long as you don't violate the Zero, One or Infinity Rule of design, you should be okay. Your multiple address fields here skirt the line, that might be better represented as a single field that accepts multiple lines of text, but that is also how a lot of databases traditionally represent arbitrary street addresses.

Reports in Laravel 5.2

I want to generate a report from a table, like
+-------------+------------------+------+-----+------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------------+------+-----+------------+----------------+
| productID | int(10) unsigned | NO | PRI | NULL | auto_increment |
| productCode | char(3) | NO | | | |
| name | varchar(30) | NO | | | |
| quantity | int(10) unsigned | NO | | 0 | |
| price | decimal(7,2) | NO | | 99999.99 | |
+-------------+------------------+------+-----+------------+----------------+
and show with some graphic the the top sellers. I'm lost in this subject.
Is there a package that make this reports?
Thanks for the info in advance.
I don't think there is a package to generate the reports. Reports are all about getting data from DB, analyze and send output to the client/browser. What I would suggest is that get the data from DB and send to the client as JSON. In client side, you can use graph plotting packages like Highchart, D3JS etc to plot the graph.

Store data in MySQL or a PHP file?

I am working on a project and I ended up with the table below:
+---------------+--------------+------+-----+--------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+--------------------+-------+
| id | int(11) | NO | MUL | NULL | A_I |
| user _id | int(11) | NO | | NULL | |
| info | varchar(255) | NO | | NULL | |
| country | tinyint(3) | NO | | NULL | |
| date_added | timestamp | NO | | 0000-00-00 00:00:00| |
+---------------+--------------+------+-----+--------------------+-------+
Because I wanted to avoid storing countries as varchar all the time I thought I should use number IDs instead. My question is, would it be better to store the country IDs in a table where I would give a name to each one of them or do that in a php file? Countries won't change or anything. It will be a list of around 100 countries.
Thanks!
Use a seperate country table.
countries table
---------------
id
name
Then you can relate to the country ID in your table. That way you make sure only countries from your list are added and you don't need to store strings everywhere and you can easily change country names or addnew ones.

Mysql query optimization Multi Column Index solves this slowness?

+----------------------------+------------------------------------------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------------------+------------------------------------------------------------------------------+------+-----+---------+----------------+
| type | enum('Website','Facebook','Twitter','Linkedin','Youtube','SeatGeek','Yahoo') | NO | MUL | NULL | |
| name | varchar(100) | YES | MUL | NULL | |
| processing_interface_id | bigint(20) | YES | MUL | NULL | |
| processing_interface_table | varchar(100) | YES | MUL | NULL | |
| create_time | datetime | YES | MUL | NULL | |
| run_time | datetime | YES | MUL | NULL | |
| completed_time | datetime | YES | MUL | NULL | |
| reserved | int(10) | YES | MUL | NULL | |
| params | text | YES | | NULL | |
| params_md5 | varchar(100) | YES | MUL | NULL | |
| priority | int(10) | YES | MUL | NULL | |
| id | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| status | varchar(40) | NO | MUL | none | |
+----------------------------+------------------------------------------------------------------------------+------+-----+---------+----------------+
select * from remote_request use index ( processing_order ) where remote_request.status = 'none' and type = 'Facebook' and reserved = '0' order by priority desc limit 0, 40;
This table receives an extremely large amount of writes and reads. each remote_request ends up being a process, which can spawn anywhere between 0 and 5 other remote_requests depending on the type of request, and what the request does.
The table is currently sitting at about 3.5 Million records, and it goes to a snail pace when the site itself is under heavy load and I have more then 50 or more instances running simultaneously. (REST requests are the purpose of the table just in case you were not sure).
As the table grows it just gets worse and worse. I can clear the processed requests out on a daily basis but ultimatly this is not fixing the problem.
What I need is for this query to always have a very low response ratio.
Here are the current indexes on the table.
+----------------+------------+----------------------------------+--------------+----------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+----------------+------------+----------------------------------+--------------+----------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| remote_request | 0 | PRIMARY | 1 | id | A | 2403351 | NULL | NULL | | BTREE | | |
| remote_request | 1 | type_index | 1 | type | A | 18 | NULL | NULL | | BTREE | | |
| remote_request | 1 | processing_interface_id_index | 1 | processing_interface_id | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | processing_interface_table_index | 1 | processing_interface_table | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | create_time_index | 1 | create_time | A | 160223 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | run_time_index | 1 | run_time | A | 343335 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | completed_time_index | 1 | completed_time | A | 267039 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | reserved_index | 1 | reserved | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | params_md5_index | 1 | params_md5 | A | 2403351 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | priority_index | 1 | priority | A | 716 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | status_index | 1 | status | A | 18 | NULL | NULL | | BTREE | | |
| remote_request | 1 | name_index | 1 | name | A | 18 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | processing_order | 1 | priority | A | 200 | NULL | NULL | YES | BTREE | | |
| remote_request | 1 | processing_order | 2 | status | A | 200 | NULL | NULL | | BTREE | | |
| remote_request | 1 | processing_order | 3 | type | A | 200 | NULL | NULL | | BTREE | | |
| remote_request | 1 | processing_order | 4 | reserved | A | 200 | NULL | NULL | YES | BTREE | | |
+----------------+------------+----------------------------------+--------------+----------------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Any idea how i solve this? Is it not possible to make some sort of complicated index that would automatic order them with priority, then take the first 40 that match the 'Facebook' type? It currently is scanning more then 500k rows of the table before it returns a result which is grossly inefficient.
Some other version of the query that I have been tinkering with are:
select * from remote_request use index ( type_index,status_index,reserved_index,priority_index ) where remote_request.status = 'none' and type = 'Facebook' and reserv ed = '0' order by priority desc limit 0, 40
It would be amazing if we could get the rows scanned to under 1000 rows depending on just how many types of requests enter the table.
Thanks in advance, this might be a real nutcracker for most except the most experienced mysql experts?
Your four-column index has the right columns, but in the wrong order.
You want the index to first look up matching rows, which you do by three columns. You are looking up by three equality conditions, so you know that once the index finds the set of matching rows, the order of these rows is basically a tie with respect to those first three columns. So to resolve the tie, add as the fourth column the column by which you wanted to sort.
If you do that, then the ORDER BY becomes a no-op, because the query can just read the rows in the order they are stored in the index.
So I would create the following index:
CREATE INDEX processing_order2 ON remote_request
(status, type, reserved, priority);
There's probably not too much significance to the order of the first three columns, since they're all in equality terms combined with AND. But the priority column belongs at the end.
You may also like to read my presentation How to Design Indexes, Really.
By the way, using USE INDEX() shouldn't be necessary if you have the right index, MySQL's optimizer will choose it automatically most of the time. But USE INDEX() can block the optimizer from considering a new index that you create, so it becomes a disadvantage for code maintenance.
This isn't a complete answer but it was too long for a comment:
Are you actually searching on all of those indexes? If not get rid of some. Extra indexes slow down writes.
Secondly use EXPLAIN on your query and don't specify an index when you do. See how MySQL wants to process it rather than forcing an option (Generally it does the right thing).
Finally sorting is likely what hurts you the most. If you don't sort it probably gets the records pretty quickly. It has to scan and sort every row that meets your criteria before it can return the top 40.
Options:
Try creating a VIEW (not as familiar with VIEWS but it might work)
Split this table into smaller tables
use a third party tool such as
Sphinx or Lucene to create specialized indexes to search on. (I've
used Sphinx for something like this before. You can find it at
http://sphinxsearch.com/).
Or look into using a NoSQL solution where you can use a Map function to do it.
Edit I read a bit about using VIEW and I don't think it will help you in your case because you have such a large table. See the answer in this thread: Using MySQL views to increase performance

Categories