Please note: While my original issue was not possible to be solved in the way I expected, #Bamar solution marked in this post is an alternative that reaches the same goal and works perfectly. What I proposed in this post to be done doesn't seem to be viable if the databases are located in different hosts.
I've been searching for a while and I seem to be unable to solve my issue.
THE DATA I HAVE
My service provider is 1&1. In the current contract I have with them I could create up to 100 databases with a maximun size of 2GB each.
Each database that is created, is assingned a random hostname, port and username (the only item which I can choose is the password).
I've got two different databases, lets call them DB_1 and DB_2.
In the DB_1 I've got a table called T_USERS which fields of interest for this particular problem are:
ID: The ID of the record.
userName: The user name registered on the database.
In the DB_2 I've got a table called T_SCORES which fields of interest for this particular problem are:
ID_User: it's a foregin key that refers to the ID of a particular user in DB_1.T_USERS
score: a numeric value that indicates the score of that user.
It is important to take into account that to access both databases each of them needs different credentials!
WHAT I WANT TO ACHIEVE
What I want to achieve seems simple at a first glance but I was unable to find any documentation or solution online on how to do this using PHP and PDO.
I just want to perform a join with DB_2.ID_USER and DB_1.ID
My final result should look something like this:
DB_1.userName
DB_2.score
Alex
237
Peter
120
Mark
400
...
...
WHERE I'M CURRENTLY STUCK
This is what I've currently tried.
First of all I perform the connection to my databases as follows (I normally use a try/catch when connecting to a DB but I will omit it here):
//Connection to the DB1
$db1_hostName = "hostnameofDB1";//The host name of the database 1
$db1_name = "db1";//The name of the database 1
$db1_userName = "user1";//The username in the database 1
$db1_password = "pw1";//The password for the database 1
$pdo_db1Handle = new PDO("mysql:host=$db1_hostName; dbname=$db1_name;", $db1_userName, $db1_password);
$pdo_db1Handle->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
//Connection to the DB2
$db2_hostName = "hostnameofDB2";//The host name of the database 2
$db2_name = "db2";//The name of the database 2
$db2_userName = "user2";//The username in the database 2
$db2_password = "pw2";//The password for the database 2
$pdo_db2Handle = new PDO("mysql:host=$db2_hostName; dbname=$db2_name;", $db2_userName, $db2_password);
$pdo_db2Handle->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
So basically up to this point what I've done is very simple, create a pdo_db1Handle and pdo_db2Handle. Now to the tricky part...
If I now want to perform a join my SQL syntax should be something like this:
SELECT DB_1.T_USERS.userName, DB_2.T_SCORES.score
FROM DB_2.T_SCORES
LEFT JOIN DB_1.T_USERS
ON (DB_2.T_SCORES.ID_User=DB_1.T_USERS.ID)
ORDER BY DB_2.T_SCORES.score ASC 'The ordering is optional, I'm interested in the join part first
But as far as I'm aware and with all the information I was able to find, you execute the SQL statement against one of the two handles I previously defined in the following way:
$stmt=$pdo_db1Handle->prepare($mySQLStatement);
$stmt->execute();
When I try to do this, an error shows up telling me missing credentials for the DB_2. It happens the opposite (missing credentials of DB_1) if I try to execute it against pdo_db2Handle.
How should I proceed? any solution using PDO for this?
Thanks in advance :)
You can't join if you have to use separate PDO connections, so use nested loops and join the data in PHP.
$stmt_user = $pdo_db1Handle->query("SELECT id, username FROM t_users");
$stmt_score = $pdo_db2Handle->prepare("SELECT score FROM t_scores WHERE id_user = :userid");
$results = [];
while ($row_user = $stmt_user->fetch(PDO::FETCH_ASSOC)) {
$scores = [];
$stmt_score->execute(':userid' => $row_user['id']);
while ($row_score = $stmt_score->fetch(PDO::FETCH_ASSOC)) {
$scores[] = $row_score['score'];
}
$results[$row_user['username']] = $scores;
}
This will create an associative array whose keys are usernames and values are an array of their scores.
Depending on your use case, a work around may be to copy the table from one database to another temporarily and the perform your sql once you have both tables in a single database:
$pdo1 = new PDO('mysql:host=$db1_hostName; dbname=$db1_name', $db1_userName, $db1_password);
$pdo2 = new PDO('mysql:host=$db2_hostName; dbname=$db1_name', $db2_userName, $db2_password);
$insert_stmt = $pdo2->prepare("INSERT INTO T_SCORES (col1, col2, col3, ...) VALUES (:col1, :col2, :col3, ...) ON DUPLICATE KEY IGNORE");
$select_results = $pdo1->query("SELECT * FROM T_SCORES");
while ($row = $select_results->fetch(PDO::FETCH_ASSOC)) {
$insert_stmt->execute($row);
}
-- now work with the tables as you usually would.
You can create the table in the target database before hand and truncate the data before and/or after performing the insert.
I am using an MVC framework (Zend) for my application and I want to find the total size of a table in PostgreSQL (including index). The table name is "V5TableName" - quotes included because table name is case sensitive. I have made sure that there is NO typo involved.
My code to get the table size is shown below:
public function getMyTableSize()
{
$sql = "SELECT pg_size_pretty(pg_total_relation_size( '\"V5TableName\"' ) );";
/* Custom_Db is a custom library in my application which makes the PostgreSQL connection
and queries the database
*/
$tableSize = Custom_Db::query($sql)->fetchColumn();
return $tableSize;
}
When my application calls this function it returns the following error in my logs :
[22-Apr-2020 09:42:37] PID:30849 ERR: SQLSTATE[42P01]: Undefined table: 7 ERROR: relation "V5TableName" does not exist
LINE 1: SELECT pg_size_pretty(pg_total_relation_size( '"V5TableName...
^
query was: SELECT pg_size_pretty(pg_total_relation_size( '"V5TableName"' ) );
If I run this same query in pgAdmin4 it works perfectly fine returning the table size (for instance: 104Mb).
I have tried:
Removing and adding quotes to the table name in the code.
Appending the schema as prefix to the table name (example: 'public."V5TableName"').
NONE of the above seem to work. I am not sure what is going wrong over here.
I also tried to find the total database size in my application (db name: MyDbName - with mixed case spelling) and my query looked something like below:
$sql = "SELECT pg_size_pretty(pg_database_size('MyDbName'))"; // this DID NOT WORK
So I changed it to the one shown below: (it worked)
$sql = "SELECT pg_size_pretty(pg_database_size( current_database() ))"; // this WORKED
I was wondering if there is something similar that could be done to find the table size.
Your query should work. The use of double-quotes seems correct.
SELECT pg_size_pretty(pg_total_relation_size('"V5TableName"'));
First make sure you are connecting to the right database cluster (a.k.a. "server"). It's defined by its data directory, or equally unambiguous by hostname and port number. Read the manual here and here.
Then make sure you are connecting to the right database within that database cluster. A Postgres database cluster consists of 1-n databases. When connecting without specifying the actual database, you end up in the maintenance database named postgres by default. That's the most likely explanation. Check with:
SELECT current_database();
Then check for the right table and schema name:
SELECT * FROM pg_tables
WHERE tablename ~* 'V5TableName'; -- ~* matches case-insensitive
The first riddle should be solved at this point.
Check your DB spelling and possible near-duplicates with:
SELECT datname FROM pg_database;
The call is without double-quotes (like you tried correctly), but requires correct capitalization:
SELECT pg_size_pretty(pg_database_size('MyDbName'));
Note the subtle difference (as documented in the manual):
pg_database_size() takes oid or name. So pass the case-sensitive database name without double-quotes.
pg_total_relation_size() takes regclass. So pass the case-sensitive relation name with double-quotes if you need to preserve capitalization.
pg_database_size() has to differ because there is no dedicated object identifier type for databases (no regdatabase).
The gist of it: avoid double-quoted identifiers in Postgres if at all possible. It makes your life easier.
I am building a custom artisan command that needs to be able to access the database's default values for certain columns. I cannot use the attributes array. So instead I need another way.
I have tried to use Schema. I have been able to get the table DB::table($table) and the column names Schema::getColumnListings($table) but not the default values.
Is there some other way to get the default values?
The Laravel Schema Builder only returns column names by design. But you can use the same approach Laravel uses internally by executing a database statement:
$results = DB::select('
select column_default
from information_schema.columns
where
table_schema = ?
and table_name = ?
', [$database, $table]);
// Flatten the results to get an array of the default values
$defaults = collect($results)->pluck('column_default'))
The above example works for a MySQL database, but you can see the approaches for other databases in the Illuminate\Database\Schema\Grammars namespace of the Laravel source code by searching for the method compileColumnListing.
In Laravel 5+ (including 6 and 7), you can get the db table column metadata (ie type, default value etc) in the following way:
use Illuminate\Support\Facades\Schema;
For all columns:
$columns = Schema::getConnection()->getDoctrineSchemaManager()->listTableColumns('table_name');
For a single column:
$column = Schema::getConnection()->getDoctrineColumn('table_name'', 'column_name');
getDoctrineSchemaManager method returns a array of \Doctrine\DBAL\Schema\Column Class Instances. By using this you can get everything about a db table column.
getDoctrineColumn method returns the instance of \Doctrine\DBAL\Schema\Column class.
Couple of methods from \Doctrine\DBAL\Schema\Column class:
$column->getName();
$column->getNotnull(); // returns true/false
$column->getDefault();
$column->getType();
$column->getLength();
I have multiple sources, like this (say)
source src1{
...
}
source src2{
...
}
AND
index src1{
...
}
index src2{
...
}
src1 has sql query from one individual table and src2 has sql query based on another individual table.
Now, in the PHP script, how do I specify, which indexer to use?
Normally, in the PHP script, we write it this way
$ss = new SphinxClient;
$ss->setServer("localhost", 9312);
$ss->setMatchMode(SPH_MATCH_ANY);
Since, there is no mention about the indexer being used. It's useless to search both indexes (i.e., both tables). I want to search the index src2(say) i.e., data from the second table. So, how do I specify this in my php script, that sphinx should search only that particular indexer.
The Query call includes the index(s) to search
$res = $cl->Query($query,"src1");
For one index (per Barry Hunter)
$res = $cl->Query($query,"src1");
or
For multiple indexes for one query.
$res = $cl->Query($query,"src1 src2 src3 src4");
I would like to create an advanced search form much like a job site would have one that would include criteria such as keyword, job type, min pay, max pay, category,sub category etc...
My problem is deciding on how best to set this up so if I have to add categories to the parameters I'm not having to modify a whole bunch of queries and functions etc...
My best guess would be to create some sort of associative array out of all of the potential parameters and reuse this array but for some reason I feel like it's a lot more complex than this. I am using CodeIgniter as an MVC framework if that makes any difference.
Does anybody have a suggestion as how best to set this up?
Keep in mind I will need to be generating links such as index.php?keyword=designer&job_type=2&min_pay=20&max_pay=30
I hope my question is not to vague.
I don't know if it's what you need, but I usually create some search class.
<?php
$search = new Search('people');
$search->minPay(1000);
$search->maxPay(4000);
$search->jobType('IT');
$results = $search->execute();
foreach ($results as $result)
{
//whatever you want
}
?>
You can have all this methods, or have some mapping at __set() between method name and database field. The parameter passed to the constructor is the table where to do the main query. On the methods or mapping in the __set(), you have to take care of any needed join and the fields to join on.
There are much more 'enterprise-level' ways of doing this, but for a small site this should be OK. There are lots more ActiveRecord methods you can use as necessary. CI will chain them for you to make an efficient SQL request.
if($this->input->get('min_pay')) {
$this->db->where('min_pay <', $this->input->get('min_pay'));
}
if($this->input->get('keyword')) {
$this->db->like($this->input->get('keyword'));
}
$query = $this->db->get('table_name');
foreach ($query->result() as $row) {
echo $row->title;
}
To use Search criterias in a nice way you should use Classes and Interfaces.
Let's say for example you define a ICriteria interface. Then you have different subtypes (implementations) of Criteria, TimeCriteria, DateCriteria, listCriteria, TextSearch Criteria, IntRange Criteria, etc.
What your Criteria Interface should provide is some getter and setter for each criteria, you'll have to handle 3 usages for each criteria:
how to show them
how to fill the query with the results
how to save them (in session or database) for later usage
When showing a criteria you will need:
a label
a list of available operators (in, not in, =, >, >=, <, <=, contains, does not contains) -- and each subtypes can decide which part of this list is implemented
an entry zone (a list, a text input, a date input, etc)
Your main code will only handle ICriteria elements, ask them to build themselves, show them, give them user inputs, ask them to be saved or loop on them to add SQL criteria on a sql query based on their current values.
Some of the Criteria implementations will inherits others, some will only have to define the list of operators available, some will extends simple behaviors to add rich UI (let's say that some Date elements should provide a list like 'in the last day', 'in the last week', 'in the last year', 'custom range').
It can be a very good idea to handle the SQL query as an object and not only a string, the way Zend_Db_Select works for example. As each Criteria will add his part on the final query, and some of them could be adding leftJoins or complex query parts.
Search queries can be a pain sometimes, but not as big of a pain as pagination. Luckily, CodeIgniter helps you out a bit with this with their pagination library.
I think you're on the right track. The basic gist, I would say, is:
Grab your GET variables from the URL.
Create your database query (sanitize the GET values).
Generate the results set.
Do pagination.
Now, CodeIgniter destroys the GET variable by default, so make sure you enable http query strings in your config file.
Good luck!
I don't know anything about CodeIgniter, but for the search application I used to support, we had drop-down combo-boxes with category options stored in a database table and would rely on application and database cacheing to avoid round-trips each time the page was displayed (an opportunity for learning in itself ;-). When you update the table of job_type, location, etc. the new values will be displayed in your combo-box.
It depends on
how many categories you intend to have drop-down lists
how often you anticipate having to update the list
how dynamic you need it to be.
And the size of your web-site and overall activity are factors you will have to consider.
I hope this helps.
P.S. as you appear to be a new user, if you get an answer that helps you please remember to mark it as accepted, or give it a + (or -) as a useful answer
A pagination class is a good foundation. Begin by collecting query string variables.
<?php
// ...in Pagination class
$acceptableVars = array('page', 'delete', 'edit', 'sessionId', 'next', 'etc.');
foreach($_GET as $key => $value) {
if(in_array($key, $acceptableVar)) {
$queryStringVars[] = $key . '=' . $value;
}
}
$queryString = '?' . implode('&', $queryStringVars);
$this->nextLink = $_SEVER['filename'] . $queryString;
?>
Duplicate the searchable information into another table. Convert sets of data into columns having two values only like : a search for color=white OR red can become a search on 10 columns in a table each containing one color with value 1 or 0. The results can be grouped after so you get counters for each search filter.
Convert texts to full text searches and use MATCH and many indexes on this search table. Eventually combine text columns into one searchable column. The results of a seach will be IDs which you can then convert into the records with IN() condition in SQL
Agile Toolkit allows to add filters in the following way (just to do a side-by-side comparison with CodeIgniter, perhaps you can take some concepts over):
$g=$this->add('Grid');
$g->addColumn('text','name');
$g->addColumn('text','surname');
$g->setSource('user');
$conditions=array_intersect($_GET, array_flip(
array('keyword','job_type','min_pay'));
$g->dq->where($conditions);
$g->dq is a dynamic query, where() escapes values passed from the $_GET, so it's safe to use. The rest, pagination, column display, connectivity with MVC is up to the framework.
function maybeQuote($v){
return is_numeric($v) ?: "'$v'";
}
function makePair($kv){
+-- 7 lines: $a = explode('=', $kv);
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
function makeSql($get_string, $table){
+-- 10 lines: $data = explode('&', $get_string);
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
}
$test = 'lloyd=alive&age=40&weather=hot';
$table = 'foo';
print_r(makeSql($test, $table));