I want to empty a table in hbase... eg: user. Is there any command or function to empty the table without deleting it...
My table structure is :
$mutations = array(
new Mutation( array(
'column' => 'username:1',
'value' =>$name
) ),
new Mutation( array(
'column' => 'email:1',
'value' =>$email
) )
);
$hbase->mutateRow("user",$key,$mutations);
Can someone help me?
If you execute this in HBase shell:
> truncate 'yourTableName'
Then HBase will execute this operations for 'yourTableName':
> disable 'yourTableName'
> drop 'yourTableName'
> create 'yourTableName', 'f1', 'f2', 'f3'
Another efficient option is to actually delete the table then reconstruct another one with all the same settings as the previous.
I don't know how to do this in php, but I do know how to do it in Java. The corresponding actions in php should be similar, you just need to check how the API looks like.
In Java using HBase 0.90.4:
// Remember the "schema" of your table
HBaseAdmin admin = new HBaseAdmin(yourConfiguration);
HTableDescriptor td = admin.getTableDescriptor(Bytes.toBytes("yourTableName");
// Delete your table
admin.disableTable("yourTableName");
admin.deleteTable("yourTableName");
// Recreate your talbe
admin.createTable(td);
Using hbase shell, truncate <table_name> will do the task.
The snapshot of truncate 'customer_details' command is shown below:
where customer_details is the table name
truncate command in hbase shell will do the job for you:
Before truncate:
After truncate:
HBase thrift API (which is what php is using) doesn't provide a truncate command only deleteTable and createTable functionality (what's the diff from your point of view?)
otherwise you have to scan to get all the keys and deleteAllRow for each key - which isn't a very efficient option
For the purposes of this you can use HAdmin. It is an UI tool for Apache HBase administration. There are "Truncate table" and even "Delete table" buttons in alter table page.
Using alter command
alter '<table_name>', NAME=>'column_family',TTL=><number_of_seconds>
here number_of_seconds stands for duration after which data will be automatically deleted.
There's no single command to clear Hbase table, but you can use 2 workarounds: disable, delete, create table, or scan all records and delete each.
Actually, disable, delete and create table again takes about 4 seconds.
// get Hbase client
$client = <Your code here>;
$t = "table_name";
$tables = $client->getTableNames();
if (in_array($t, $tables)) {
if ($client->isTableEnabled($t))
$client->disableTable($t);
$client->deleteTable($t);
}
$descriptors = array(
new ColumnDescriptor(array("name" => "c1", "maxVersions" => 1)),
new ColumnDescriptor(array("name" => "c2", "maxVersions" => 1))
);
$client->createTable($t, $descriptors);
If there's not a lot of data in table - scan all rows and delete each is much faster.
$client = <Your code here>;
$t = "table_name";
// i don't remember, if list of column families is actually needed here
$columns = array("c1", "c2");
$scanner = $client->scannerOpen($t, "", $columns);
while ($result = $client->scannerGet($scanner)) {
$client->deleteAllRow($t, $result[0]->row);
}
In this case data is not deleted physically, actually it's "marked as deleted" and stays in table until next major compact.
Perhaps using one of these two commands:
DELETE FROM your_table WHERE 1;
Or
TRUNCATE your_table;
Regards!
Related
I currently have this code below, however when adding around 2000 rows this runs too slow due to being in an foreach loop.
foreach($tables as $key => $table) {
$class_name = explode("\\", get_class($table[0]));
$class_name = end($class_name);
$moved = 'moved_' . $class_name;
${$moved} = [];
foreach($table[0]->where('website_id', $website->id)->get() as $value) {
$value->website_id = $live_website_id;
$value->setAppends([]);
$table[0]::on('live')->updateOrCreate([
'id' => $value->id,
'website_id' => $value->website_id
], $value->toArray());
${$moved}[] = $value->id;
}
// Remove deleted rows
if ($table[1]) {
$table[0]::on('live')->where([ 'website_id' => $live_website_id ])
->whereNotIn('id', ${$moved})
->delete();
}
}
What is happening is basically users will add/update/delete data in a development server, then when they hit a button this data needs to be pushed into the live table, retaining the ID's as auto incremental id's on the live won't work due to look-up tables and multiple users launching data live at the same time.
What is the best way to do this? Should I simply remove all the data in that table (there is a unique identifier for chunks of data) and then just insert?
I think can do:
You can create a temp table and fill just like your development server and just rename table to published table
Use Job and background service(async queue)
You can set your db connection as singleton, because connecting multiple times can waste time
Can use some sql tools for example if use postgresql you can use Foreign Data Wrapper(fdw) to connect db to each other and update and create in a faster way!
i have a code like this ,
$request = Yii::$app->request;
$post = $request->post();
$filesettingid = $post['filesettingid'];
$checkboxValue = $post['selection'];
for($i=0;$i<sizeof($checkboxValue);$i++) {
$store = Yii::$app->db->createCommand('SELECT id FROM store WHERE port='.$checkboxValue[$i])->queryAll();
$storeid = $store[$i]['id'];
Yii::$app->db->createCommand()->insert('tes',
[
'id' => $i+1,
'filesetting_id' => $filesettingid,
'store_id' => $storeid
])->execute();
}
what i want is, each i insert the new data, id will generate automaticly like 1,2,3,4.
the problem in above code is, the ID always be 1.
is it possible to make it real?
so what i want is :
First time insert, id = 1, second is id = 2 , and that is happen automatically.
Have you considered setting database engine to auto increment values with each insert?
Take as an example Yii2 default user table. ID filed is auto incremented, and you don't have to worry about setting it problematically. Every time you send a new insert engine increments ID filed by itself.
See default migration under "advanced template"\console\migrations\m130524_201442_int. (your file name might be different depending on the Yii2 version)
$this->createTable('{{%user}}', [
'id' => $this->primaryKey(),
'username' => $this->string()->notNull()->unique(),
'auth_key' => $this->string(32)->notNull(),
'password_hash' => $this->string()->notNull(),
'password_reset_token' => $this->string()->unique(),
'email' => $this->string()->notNull()->unique(),
'status' => $this->smallInteger()->notNull()->defaultValue(0),
.........
], $tableOptions);
When setting 'id' to primary key database automatically knows to auto increment it. If you already have a table the ID field is not primary key you can use the followign migration:
$this->alterColumn('{{%databaseName}}', 'columnName', $this->integer()->notNull().' AUTO_INCREMENT');
You can also set it from management console, or run a SQL query. Depending on database engine you are using this might look a little different but the concept is the same.
MYSQL:
In MySQL workbench right click on table in question, select Alter Table and check NNm and AI next to column you want auto increment. See Screenshot
Or run command:
ALTER TABLE `dbName`.`nameOfTheTable` MODIFY `columnName` INT AUTO_INCREMENT NOT NULL;
I am a bit rusty on my SQL, so if it does not work let me know I will get you right command.
Hope this helps. Good luck.
There is a problem in inserting to MongoDB database. It is not insert to database in right order.
We read the writing concern in MongoDB:
http://www.php.net/manual/en/mongo.writeconcerns.php
We use mongoDB 2.6 and PHP driver 1.6 with following sample code:
set_message_sample('1');
set_message_sample('2');
set_message_sample($id) {
$connecting_string = sprintf('mongodb://%s:%d/%s', $hosts, $port,$database), // fill with right connection setting
$connection= new Mongo($connecting_string,array('username'=>$username,'password'=>$password)); // fill with proper authentication setting
$dbname = $connection->selectDB('dbname');
$collection = $dbname->selectCollection('collection');
$post = array(
'title' => $id,
'content' => 'test ' . $id,
);
$posts->insert($insert,array("w" => "1"));
Sometimes the result is inserting "2" before "1" to database. We want to inserting in right order (first "1" and next "2") all the times. I also notice that we order the collection with mongoID which automatically set by mongoDB.
We check many options but the problem not solved. How we could solve it? ( How we could disable something like cache or isolate the insert queue to MongoDB.)
So, i think you could insert the second only after the confirmation of the first one. since the insert is asynchronous, you can't be sure who goes first. So you must insert only after the confirmation of the first one.
I'm using the mongoDB to store the log of user. In my real-time report, I need to count the distinct user of the table in a specific type. In the beginning, it runs fast, but it become slower when the table becomes bigger.
Here is the code I used:
$connection = new MongoClient();
$result = $collection->distinct('user', array('type' => $type, 'ctime' => array('$gte' => $start)));
$total = count($result);
$total is the total number of unique user
Can anyone suggest me how to improve the query to get the better performance?
Many thanks.
use $collection->ensureIndex(array('user' => 1)); to create index on user field.
I'm using PostgreSQL & Codeigniter. There is a table called folio in the database. It has few columns containing remarks1, remarks2, remarks3 as well. Data for the all the other columns are inserted when the INSERT statement executes for the 1st time.
When I try to execute below UPDATE statement later for the below 3 columns, remarks1 column get updated correctly. But remarks2, remarks3 columns are updated with ''.
UPDATE "folio" SET "remarks1" = 'test remark', "remarks2" = '', "remarks3" = '' WHERE "id" = '51';
Given that remarks1, remarks2, remarks3 columns data type is character varying. I'm using Codeigniter active records. At a time all 3 columns could be updated else single column could be updated depending on the user input.
What could be the issue? How can I fix this? Why columns are updated with ''?
As requested the php array in CI would be below
$data = array(
'remark1' => $this->input->post('remark1'),
'remark2' => $this->input->post('remark1'),
'remark3' => $this->input->post('remark1')
);
Function which saves the data contains below two lines only
$this->db->where('id', $folio_id);
$this->db->update('folio', $data);
Those columns are updated with '' because you tell them to?
Let's take a closer look at the query
UPDATE "folio"
SET
"remarks1" = 'test remark',
"remarks2" = '',
"remarks3" = ''
WHERE
"id" = '51';
First you select the table folio for the update.
Then you tell it to update remarks1 through remarks3 with new values. For remarks2 and remarks3 you specify to set them to an empty string. And that's what's going to happen.
Last but not least, you tell it to only apply this update to rows where id equals 51.
So, in order to only update remarks1 you can simply remove the other columns from your update:
UPDATE "folio"
SET
"remarks1" = 'test remark'
WHERE
"id" = '51';
Update:
I'm by far not a CI expert, but from what I see, I'd change the $data array to only contain information for remark1:
$data = array(
'remark1' => $this->input->post('remark1')
);
And (from my understanding) it should only update this single column.