I am integrating a bigQuery in my Google Cloud project. I have settle up all the requirements which required to integrate the big query. Now I want to perform the update operation through my PHP file. I have created a dataset and table in bigQuery.
Dataset Name - count
Table name - companies
I want to update in this table through my PHP file. Before this, here is my code for updating the values in cloud datastore:
$dataset = $bigQuery->dataset('count');
$table = $dataset->table('companies');
if ($check) {
$updateResponse = $table->update(['name' => 'A friendly name.']);
if ($updateResponse->isSuccessful()) {
print('Data updated into BigQuery successfully' . PHP_EOL);
}
}
But this code is not working ?
The code seems to be ok, but name is not a valid metadata. friendlyName is a valid one, check other table metadata resources. I'm not quite sure if isSuccessful exists, so, to verify whether the operation was successful or not, I would apply:
$info = $table->info();
echo $info['friendlyName'];
If you want to update data in the BigQuery table, you can execute DML statements just as you would a SELECT statement:
$bigQuery = new BigQueryClient(['projectId' => $projectId,]);
$query = 'UPDATE statement';
$jobConfig = $bigQuery->query($query);
$job = $bigQuery->startQuery($jobConfig);
In this case you can ask the progress of the DML operation with $job->isComplete(), see the complete example here.
You mentioned that "... updating the values in cloud datastore"; however, your code is for BigQuery, if there is something related to datastore, please clarify.
Related
Using PHP i'd like to check the next available number to use as an id after comparing it to the query that lists all my id's
In theory I can do this;
$clientid = '0';
$getid_query = "SELECT clientid FROM clients";
$response = mysqli_query($conny, $getid_query);
while($data = mysql_fetch_assoc($response)){
$row[] = $data;
}
$freeid = False;
while($freeid == false){
if($row[clientid].contains($clientid){
$clientid = $clientid + 1;
}
else {
$freeid = true;
}
}
This leaves $clientid as an unique id ready to be used for the next created client
there might be a few syntax errors but in general I've tried most combinations and seem to get it right, I've been testing the different outputs such as
echo "Error:" . $row[clientid];
and sometimes (more often than not, nothing displays).
Edit 2:
Hold up! I wanted to do it php side because the associated username is generated from the clientid (all in php). So im going to follow the links from liridyn and see if i can do something.
Would there be a safe way to query the database again as soon as a new client has been registered in order to get their allocated clientid, so i can then update their row with the generated username?
Thanks
So, SELECT MAX(clientId) + 1 AS clientId FROM clients would replace your mixed PHP/SQL with a single query - but I would recommend against that, as adding AUTO_INCREMENT to clientId would instruct the SQL client to manage that for you automatically; if you then need that id, instead of generating it from PHP or calling SELECT MAX(clientID) FROM clients (which still has a race condition), you can follow the advice in this answer or call mysqli_insert_id.
I've ended up using Auto_increment on the db side. Thanks
I have a set that have three bins ( PK, cat_id and data ). List index is applied on cat_id . I am able to select record by query:
SELECT * FROM test.myset IN LIST where cat_id = '1'
It is working fine for me. Now I need to delete this record by same condition. But as I have read that PK is necessary to delete any record . In my case i have cat_id to delete this record.
Help me to delete this record by using bin element not PK. I am using PHP for it. AQL also work for me.
You would use an asynchronous background query that applies a tiny (record UDF) Lua function to every record matched by that query's predicate. In the PHP client, you'd use the queryApply() method for this.
This Lua function simply says 'die':
function del_rec(rec)
aerospike:remove(rec)
end
I used the Lua UDF API reference for this 'complex' function, mainly the aerospike object reference. If you wanted to do more logic, like check on other bins, you'd be using the record methods. There's more info on UDFs and developing Record UDFs at the Aerospike website.
Once you register UDF module (the file containing this function) with the cluster, you can call it from your PHP code:
$where = Aerospike::predicateContains("cat_id", Aerospike::INDEX_TYPE_LIST, 1);
$status = $client->queryApply("test", "mytest", $where, "my_udfs", "del_rec", [], $job_id);
if ($status === Aerospike::OK) {
var_dump("The background job ID is $job_id");
} else {
echo "An error occured while initiating the background query [{$client->errorno()}] ".$client->error();
}
I am trying to source the structure and data from a sage line 50 database but am having trouble with my update/create script.
Basically, I am writing a tool so that the local intranet site can display data sourced from sage to normal employees without using up a sage login (orders due in/stock levels etc). I am having trouble with this because it seems that the Sage50 database was developed by total morons. There are no Unique keys in this database, or, more accurately, very very few. The structure is really old school you can find the structure on pastebin HERE (bit too big for here). You'll notice that there are some tables with 300+ columns, which I think is a bit stupid. However, I have to work with this and so I need a solution.
There are a few problems syncing that I have encountered. Primarily it's the fact ODBC can't limit statements to 1 row so I can check data type, and secondly, with there being no IDs, I can't check if it's a duplicate when doing the insert. At the moment, this is what I have:
$rConn = odbc_connect("SageLine50", "user", "password");
if ($rConn == 0) {
die('Unable to connect to the Sage Line 50 V12 ODBC datasource.');
}
$result = odbc_tables($rConn);
$tables = array();
while (odbc_fetch_row($result)){
if(odbc_result($result,"TABLE_TYPE")=="TABLE") {
$tables[odbc_result($result,"TABLE_NAME")] = array();
}
}
This produces the first level of the list you see on pastebin.
A foreach statement is then run to produce the next level with the columns within the table
foreach($tables as $k=> $v) {
$query = "SELECT * FROM ".$k;
$rRes = odbc_exec($rConn,$query);
$rFields = odbc_num_fields ($rRes);
$i = 1;
while($i <= $rFields) {
$tables[$k][] = odbc_field_name($rRes, $i);
$i++;
}
CreateTableandRows($k,$tables[$k]);
}
At the moment, I then have a bodged together function to create each table (not that I like the way it does it).
Because I can't automatically grab back one row (or a few rows), to check the type of data with get_type() to then automatically set the row type, it means the only way I can figure out to do this is to set the row type as text and then change them retrospectively based upon a Mysql query.
Here is the function that's called for the table creation after the foreach above.
function CreateTableandRows($table,$rows) {
$db = array(
"host" => '10.0.0.200',
"user" => 'user',
"pass" => 'password',
"table" => 'ccl_sagedata'
);
$DB = new Database($db);
$LocSQL = "CREATE TABLE IF NOT EXISTS `".$table."` (
`id` int(11) unsigned NOT NULL auto_increment,
PRIMARY KEY (`id`),";
foreach($rows as $k=>$v) {
$LocSQL .= "
".$v." TEXT NOT NULL default '',";
}
$LocSQL = rtrim($LocSQL, ',');
$LocSQL .= "
) ENGINE=MyISAM DEFAULT CHARSET=utf8";
echo '<pre>'.$LocSQL.'</pre>';
$DB->query($LocSQL);
}
I then need/want a function that takes each table at a time and synchronizes the data to the ccl_sagedata database. However, it needs to make sure it's not inserting duplicates, i.e. this script will be run to sync the sage database at the start or end of each day and without ID numbers INSERT REPLACE won't work. I am obviously implementing auto inc primary ID's for each new table in the ccl_sagedata db. But I needs to be able to reference something static in each table that I can identify through ODBC (I hope that makes sense). In my current function, it has to call the mysql database for each row on the sage database and see if there is a matching row.
function InsertDataFromSage($ODBCTable) {
$rConn = odbc_connect("SageLine50", "user", "pass");
$query = "SELECT * FROM ".$ODBCTable;
$rRes = odbc_exec($rConn,$query);
$rFields = odbc_num_fields ($rRes);
while( $row = odbc_fetch_array($rRes) ) {
$result[] = $row;
}
$DB = new Database($db);
foreach($result as $k => $v) {
$CHECKEXISTS = "SELECT * FROM ".$ODBCTable." WHERE";
$DB->query($CHECKEXISTS);
// HERE WOULD BE A PART THAT PUTS DATA INTO THE DATABASE IF IT DOESN'T ALREADY EXIST
}
}
The only other thing I can think to note is that the 'new Database' class is simply just a functionalised standard mysqli database class. It's not something I'm having problems with.
So to re-cap.
I am trying to create a synchronization script that creates (if not exists) tables within a mysql database and then imports/syncs the data.
ODBC Can't limit the output so I can't figure out the data types in the columns automatically (can't do it manually because it's a massive db with 80+ tables
I can't figure out how to stop the script creating duplicates because there are no IDs in the sage source database.
For those of you not in the UK, Sage is a useless accounting package that runs on water and coal.
The Sage database only provides data, it doesn't allow you to input data outside of csv files in the actual program.
I know this is a bit late but Im already doing the same thing but with MS SQL.
Ive used a DTS package that truncates known copies of the tables (ie AUDIT_JOURNAL) and then copies everything in daily.
I also hit a bit of a wall trying to handle updates of these tables hence the truncate and re-create. Sync time is seconds so its not a bad option. It may be a bit of a ball ache but I say design your sync tables manually.
As you rightly point out, sage is not very friendly to being poked, so id say don't try to sync it all either.
Presumably you'll need reports to present to users but you don't need that much to do this. I sync COMPANY,AUDIT_JOURNAL, AUDIT_USAGE, CAT_TITLE,CAT_TITLE_CUS, CHART_LIST,CHART_LIST_CUS, BANK,CATEGORY,CATEGORY_CUS,DEPARTMENT, NOMINAL_LEDGER,PURCHASE_LEDGER,SALES_LEDGER.
This allows recreation of all the main reports (balance sheet, trial balance, supplier balances, etc all with drill down). If you need more help this late on let me know. I have a web app called MIS that you could install locally but the sync is a combo of ODBC and the DTS.
OK you do not need to create a synchronisation script you can query ODBC in real time you can even do joins like you do in SQL to retrieve data from multiple tables. The only thing you cannot do is write data back to sage.
I am in the process of coding a cloud monitoring application and coudnt find useful logic of getting performance counters from AZURE php SDK documentation(such as CPU utilization, disk utilization, ram usage).
can anybody help ??
define('PRODUCTION_SITE', false); // Controls connections to cloud or local storage
define('AZURE_STORAGE_KEY', '<your_storage_key>');
define('AZURE_SERVICE', '<your_domain_extension>');
define('ROLE_ID', $_SERVER['RoleDeploymentID'] . '/' . $_SERVER['RoleName'] . '/' . $_SERVER['RoleInstanceID']);
define('PERF_IN_SEC', 30); // How many seconds between times dumping performance metrics to table storage
/** Microsoft_WindowsAzure_Storage_Blob */
require_once 'Microsoft/WindowsAzure/Storage/Blob.php';
/** Microsoft_WindowsAzure_Diagnostics_Manager **/
require_once 'Microsoft/WindowsAzure/Diagnostics/Manager.php';
/** Microsoft_WindowsAzure_Storage_Table */
require_once 'Microsoft/WindowsAzure/Storage/Table.php';
if(PRODUCTION_SITE) {
$blob = new Microsoft_WindowsAzure_Storage_Blob(
'blob.core.windows.net',
AZURE_SERVICE,
AZURE_STORAGE_KEY
);
$table = new Microsoft_WindowsAzure_Storage_Table(
'table.core.windows.net',
AZURE_SERVICE,
AZURE_STORAGE_KEY
);
} else {
// Connect to local Storage Emulator
$blob = new Microsoft_WindowsAzure_Storage_Blob();
$table = new Microsoft_WindowsAzure_Storage_Table();
}
$manager = new Microsoft_WindowsAzure_Diagnostics_Manager($blob);
//////////////////////////////
// Bring in global include file
require_once('setup.php');
// Performance counters to subscribe to
$counters = array(
'\Processor(_Total)\% Processor Time',
'\TCPv4\Connections Established',
);
// Retrieve the current configuration information for the running role
$configuration = $manager->getConfigurationForRoleInstance(ROLE_ID);
// Add each subscription counter to the configuration
foreach($counters as $c) {
$configuration->DataSources->PerformanceCounters->addSubscription($c, PERF_IN_SEC);
}
// These settings are required by the diagnostics manager to know when to transfer the metrics to the storage table
$configuration->DataSources->OverallQuotaInMB = 10;
$configuration->DataSources->PerformanceCounters->BufferQuotaInMB = 10;
$configuration->DataSources->PerformanceCounters->ScheduledTransferPeriodInMinutes = 1;
// Update the configuration for the current running role
$manager->setConfigurationForRoleInstance(ROLE_ID,$configuration);
///////////////////////////////////////
// Bring in global include file
//require_once('setup.php');
// Grab all entities from the metrics table
$metrics = $table->retrieveEntities('WADPerformanceCountersTable');
// Loop through metric entities and display results
foreach($metrics AS $m) {
echo $m->RoleInstance . " - " . $m->CounterName . ": " . $m->CounterValue . "<br/>";
}
this is the code I crafted to extract processor info ...
UPDATE
Do take a look at the following blog post: http://blog.maartenballiauw.be/post/2010/09/23/Windows-Azure-Diagnostics-in-PHP.aspx. I realize that it's an old post but I think this should give you some idea about implementing diagnostics in your role running PHP. The blog post makes use of PHP SDK for Windows Azure on CodePlex which I think is quite old and has been retired in favor of the new SDK on Github but I think the SDK code on Github doesn't have diagnostics implemented (and that's a shame).
ORIGINAL RESPONSE
Since performance counters data is stored in Windows Azure Table Storage, you could simply use Windows Azure SDK for PHP to query WADPerformanceCountersTable in your storage account to fetch this data.
I have written a blog post about efficiently fetching diagnostics data sometime back which you can read here: http://gauravmantri.com/2012/02/17/effective-way-of-fetching-diagnostics-data-from-windows-azure-diagnostics-table-hint-use-partitionkey/.
Update
Looking at your code above and source code for TableRestProxy.php, you could include a query as the 2nd parameter to your retrieveEntities call. You could something like:
$query = "(CounterName eq '\Processor(_Total)\% Processor Time` or CounterName eq '\TCPv4\Connections Established')
$metrics = $table->retrieveEntities('WADPerformanceCountersTable', $query);
Please note that my knowledge about PHP is limited to none so the code above may not work. Also, please ensure to include PartitionKey in your query to avoid full table scan.
Storage Analytics Metrics aggregates transaction data and capacity data for a storage account. Transactions metrics are recorded for the Blob, Table, and Queue services. Currently, capacity metrics are only recorded for the Blob service. Transaction data and capacity data is stored in the following tables:
$MetricsCapacityBlob
$MetricsTransactionsBlob
$MetricsTransactionsTable
$MetricsTransactionsQueue
The above tables are not displayed when a listing operation is performed, such as the ListTables method. Each table must be accessed directly.
When you retrieve metrics,use these tables.
Ex:
$metrics = $table->retrieveEntities('$MetricsCapacityBlob');
URL:
http://msdn.microsoft.com/en-us/library/windowsazure/hh343264.aspx
I'm writing a small application that will write the contents of a csv file to a database in CodeIgniter, but I'm having difficulty checking for dupes before write. Since this is a fairly common task, I was wondering how some of the professionals (you) do it. For context and comparison, here is my (currently non-working) code:
$sql = "SELECT * FROM salesperson WHERE salesperson_name='" . $csvarray['salesperson_name'][$i] . "'";
$already_exists = $this->db->query($sql);
if (sizeof($already_exists) > 0) {
//add to database
}
Any input at all short of "You suck!" (I already hear that from my own brain twice an hour at this point) will be greatly appreciated. Thanks.
What you're doing will work, but as above the comparison should be "== 0" not "> 0".
You can also use CodeIgniters built in num_rows method instead of using sizeof, like so:
$sql = "your query";
$query = $this->db->query($sql);
if ($query->num_rows() == 0) {
// no duplicates found, add new record
}
As a side note, if you're using CodeIgniter then it's worthwhile to use Active Record to build your queries instead of writing SQL. You can learn more about it here: CodeIgniter - Active Record
Firstly you probably want (rather than >0)
if (sizeof($already_exists) == 0) {
//add to database
}
Also could just push all your data to a temporary staging table, which you could then clean out with some spiffy sql- and then push back to your staging table.
Taking a punt on you using mysql (either that or postgres if you are phping) You could try this (taken from http://www.cyberciti.biz/faq/howto-removing-eliminating-duplicates-from-a-mysql-table/)
You will have to change your .ID to your primary key
delete from salesperson_temptable
USING salesperson_temptable, salesperson_temptable as vtable
WHERE (NOT salesperson_temptable.ID=vtable.ID)
AND (salesperson_temptable.salesperson_name=vtable.salesperson_name)
then when you review the contents of salesperson_temptable you can then insert it back to the salesperson table