Data Import via Management API - php

I'm experiencing an issue with getting the Google Analytics Managment API to work with Data Import.
When I send the data to the API via Google's PHP library, the uploaded file is visible in the "Manage Uploads" section, and appears to be correctly formatted CSV data, although the name of the file is just a random string without ".csv" at the end. See code same of this method below:
$client = new \Google_Client();
$client->setAuthConfig($config);
$client->setScopes(['https://www.googleapis.com/auth/analytics', 'https://www.googleapis.com/auth/analytics.edit']);
$analytics = new Google_Service_Analytics($client);
try {
$analytics->management_uploads->uploadData(
*****, // Account ID
*****, // Property ID
*****, // Data Set ID
[
'data' => $csvData, // contents of tmpfile, created with fputcsv, gathered with fread
'mimeType' => 'application/octet-stream',
'uploadType' => 'media',
]
);
} catch (\Exception $e) {
$response['error'] = 'Analytics Upload Error';
}
Though the data set appeared to be uploaded correctly, there was no corresponding data to be found in any reports. After downloading the file and changing the name to something ending in ".csv" and manually uploading via the web interface, the data became available. I assumed, then, that it meant that despite having the correct contents, Google wasn't handling the file correctly without the ".csv" extension.
So for my second attempt, I found references to using curl instead of their own library; doing so allowed me to pass an actual csv file, instead of just the contents of one. Example below:
$url = 'https://www.googleapis.com/upload/analytics/v3/management/accounts/' . ** Account ID ** . '/webproperties/' . ** Property ID ** . '/customDataSources/' . ** Data Set ID ** . '/uploads?access_token=' . ** Access Token **;
$file = function_exists('curl_file_create') ? curl_file_create($filename, 'application/octet-stream', 'csvData.csv') : '#' . realpath($filename);
$curl = curl_init();
$postFields = ['file_contents' => $file];
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $postFields);
Again, the resulting (named) file shows up in the upload management view with a ".csv" extension and accurate data, yet it's still not visible to reporting tools. And same as the first time: downloading, renaming, and uploading via web interface works as intended, with the data being accessible in reporting.
The documentation doesn't mention any additional steps that should be taken regarding formatting, headers, etc. I'm not sure what further steps to try, since there appears to be no useful information as to what's going wrong - in fact everything looks to be working properly, yet the data is unavailable to reports when not manually uploaded.

Related

Google indexing API wrong json file

Hi I am trying to use Google indexing API. I used this tutorial - https://developers.google.com/search/apis/indexing-api/v3/prereqs
I used this php library - different branch - v1-master because I dont use composer so I had to use autoload.php
https://github.com/googleapis/google-api-php-client/tree/master
Here is my code:
require_once($_SERVER["DOCUMENT_ROOT"] . '/assets/google-api-php-client/src/Google/autoload.php');
require_once($_SERVER["DOCUMENT_ROOT"] . '/assets/google-api-php-client/examples/templates/base.php');
$client = new Google_Client();
// service_account_file.json is the private key that you created for your service account.
$client->setAuthConfig($_SERVER["DOCUMENT_ROOT"] . '/assets/google-api-php-client/serviceAccount');
$client->addScope('https://www.googleapis.com/auth/indexing');
// Get a Guzzle HTTP Client
$httpClient = $client->authorize();
$endpoint = 'https://indexing.googleapis.com/v3/urlNotifications:publish';
// Define contents here. The structure of the content is described in the next step.
$content = '{
"url": "http://example.com/jobs/42",
"type": "URL_UPDATED"
}';
$response = $httpClient->post($endpoint, [ 'body' => $content ]);
$status_code = $response->getStatusCode();
I created service account
downloaded service account json file
This is how the json looks
added path to json as you can see above
I enabled Indexing API in console.developers.google.com for my project
json loads
But I got error Uncaught Google_Exception: Invalid client secret JSON file.
This topic is related to - google-api-php-client: Invalid client secret JSON file
But i just could not resolve it and I am stuck
Issue solved, according to issue here - https://github.com/googleapis/google-api-php-client/issues/2025#issuecomment-762634717, autoload has been removed and whole package depends on autoloader from composer (if you are using it).
If you are not using it, author pointed to the last usable version with autoloader here
https://github.com/googleapis/google-api-php-client/releases/tag/v2.8.2
Download this version and include only one autoload.php in your code
require_once($_SERVER["DOCUMENT_ROOT"] . '/assets/google-api-php-client/vendor/autoload.php');
Whole code is here
require_once($_SERVER["DOCUMENT_ROOT"] . '/assets/google-api-php-client/vendor/autoload.php');
$client = new Google_Client();
// service_account_file.json is the private key that you created for your service account.
$client->setAuthConfig($_SERVER["DOCUMENT_ROOT"] . '/assets/google-api-php-client/mykey-pre-10b793743427.json');
$client->addScope('https://www.googleapis.com/auth/indexing');
// Get a Guzzle HTTP Client
$httpClient = $client->authorize();
$endpoint = 'https://indexing.googleapis.com/v3/urlNotifications:publish';
// Define contents here. The structure of the content is described in the next step.
$content = '{
"url": "https://my-site.sk/news.php?ID=157",
"type": "URL_UPDATED"
}';
$response = $httpClient->post($endpoint, [ 'body' => $content ]);
$status_code = $response->getStatusCode();
return $status_code;
Also dont forget to add client_email from your json into Search Console owners by clicking on Settings - Users and Permission - click on dots next to you account - click Manage owners - add there new Owner (adding it to users in the previous site is not enough) - you would get 403 code.
If operation successful you would get Code 200, but in Search Console nothing would be changed because you initiated index job, that might run in few hours

PHP SOAP-ERROR: Parsing Schema: element already defined When Connecting to D365 WSDL

I am connecting to dynamics 365. It used to work perfectly, i curl to get the token then i use it as an authorization header along with php soapclient and it works, i connect i create a client and i can call my methods.
All of a sudden it decided not to work, and where it used to connect as SOAP 1.1 now it enforced SOAP 1.2
After changing from SOAP 1.1 to SOAP 1.2 ( because I got the error of binding mismatch where it said expecting application/soap+xml and text/xml was found ) So I changed versions and that error disappeared and got replaced with ERROR Fetching HTTP Headers.
That error got stuck for the longest time, people suggested to increase timeout but i put it as high as 500 800 5000 all the same.
Then all of a sudden, it started giving me SOAP ERROR Parsing schema element already defined. I did not change my code, i played for awhile with the headers but to no avail, I even removed the authorization header just to see what is going on and that did nothing i kept getting the same error.
SOAP-ERROR: Parsing Schema: element 'http://schemas.datacontract.org/2004/07/Microsoft.Dynamics.Ax.Xpp:XppObjectBase' already defined [string:Exception:private]
everytime I try to connect I get different kind of parsing schema error even though i am not changing anything in my code:
SOAP-ERROR: Parsing Schema: element 'http://schemas.microsoft.com/2003/10/Serialization/:anyType' already defined [string:Exception:private]
and another
SOAP-ERROR: Parsing Schema: element 'http://schemas.datacontract.org/2004/07/Microsoft.Dynamics.AX.KernelInterop:ProxyBase' already defined [string:Exception:private]
and then sometimes it does get through for a second but with fetching http header error again..
so i can not create a client instance anymore now..
where before i was able to create a client instance but i get an error when I call the method of "Error Fetching HTTP Headers"
something is definitely not stable because my errors are not one.
now some stated the wsdl could be faulty, but this is microsoft and the person i am in contact keeps saying he can not doing anything about it.
Help is this a PHP problem or a dynamics problem or wsdl custom made problem .
And how to solve this.
Thank you.
UPDATE
I'm sorry I mentioned earlier it is Dynamics AX , it turns out it is Dynamics 365 D365. I will keep dynamics ax tag in case it helps someone who needs the solutions provided.
UPDATE
Following is the connection code I am using:
function getAuthenticationHeader()
{
//Each variable has the values for our server
//resource
$appResource = urlencode($appADResource);
//clientID
$appClientID = urlencode($appADClientId);
//appSecret
$appSecret = urlencode($appADSecret);
//username
$appUserID = urlencode($appUserID);
// Password
$appUserPassword = urlencode($password);
// Construct the body for the STS request
$authenticationRequestBody = 'resource='.$appResource.'&client_id='.$appClientID.'&client_secret='.$appSecret.'&grant_type=password&username='.$appUserID.'&password='.$appUserPassword.'&scope=openid';
//Using curl to post the information to STS and get back the authentication response
$ch = curl_init();
// set url
$stsUrl = 'https://login.microsoftonline.com/'.$appTenantId.'/oauth2/token';
curl_setopt($ch, CURLOPT_URL, $stsUrl);
// Get the response back as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// Set the parameters for the request
curl_setopt($ch, CURLOPT_POSTFIELDS, $authenticationRequestBody);
// By default, HTTPS does not work with curl.
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// read the output from the post request
$output = curl_exec($ch);
// close curl resource to free up system resources
curl_close($ch);
// decode the response from sts using json decoder
$tokenOutput = json_decode($output);
return $tokenOutput->{'token_type'}.' '.$tokenOutput->{'access_token'};
}
try
{
//WSDL Link
$url = "https://urlToOurServer/services/webservice?wsdl";
$authorizationToken = getAuthenticationHeader();
$context = stream_context_create(array(
'ssl' => array(
'verify_peer' => false,
'verify_peer_name' => false,
'allow_self_signed' => true
),
'https' => array(
'curl_verify_ssl_peer' => false,
'curl_verify_ssl_host' => false
),
'http' => array(
'header' =>'Authorization: '.$authorizationToken
)
));
//Create array of Soap Options
$arrOpt = array(
"soap_version" => SOAP_1_2,
"cache_wsdl" => WSDL_CACHE_NONE,
"exceptions" => true,
'trace' => true,
'encoding' => 'UTF-8',
'stream_context' => $context
);
}catch(Exception $e)
{
print_r($e);
}
I also found this in my wsdl
<sp:IssuedToken sp:IncludeToken="http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702/IncludeToken/AlwaysToRecipient">
<sp:RequestSecurityTokenTemplate>
<trust:TokenType xmlns:trust="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0
</trust:TokenType>
<trust:KeyType xmlns:trust="http://docs.oasis-open.org/ws-sx/ws-trust/200512">
http://docs.oasis-open.org/ws-sx/ws-trust/200512/Bearer
</trust:KeyType>
</sp:RequestSecurityTokenTemplate>
<wsp:Policy>
<sp:RequireInternalReference/>
How can I connect to SAML for Token ?
If everything is pretty much the same, but it's not working, the first thing to do is rule out the most basic AX issues. These may not solve your issue but will be a good first step.
now some stated the wsdl could be faulty, but this is microsoft and the person i am in contact keeps saying he can not doing anything about it.
Whomever that person is, you need to confirm they've done the following:
Confirm the environment and specifically the CIL is fully compiled. Do a full AXBuild and a full CIL to be sure during non-business hours and ensure the output is good. It's basically saying "recompile everything".
Refresh the WCF configuration in the client configuration you are using to connect to AX. This client configuration may be a *.axc file or it may just be the active one. Also refresh the business connector WCF. This is separate and may be what you are using to connect to AX. This is what most people are talking about.
Here's a little article that talks about creating a configuration, but I'll discuss below.
An AX client configuration ultimately is a bunch of text. It's either stored in an .axc file or stored in the registry in a few locations. The Business Connector client config may be the one that is getting missed in your scenario.
If you follow the link above and create a new .axc configuration file and ensure you've clicked "Refresh Configuration" before exporting, when you open the file up in Notepad, you'll see wcfconfig and a bunch of XML following it. That XML is what you're trying to get updated. Creating a new AXC here is just an exercise to help you understand what it is. You can delete the file after you're done looking.
Now, you've basically created a specific configuration file, but that doesn't mean anything is using it. If you call AX32.exe it will default to the one that is loaded in that config screen. Using a file is a way to very specifically choose one. Your code is probably using a specific AXC somewhere that needs either replaced or refreshed OR it's using one that's saved in this window:
It is very likely it is using one of the two that are saved in that configuration window. When you refresh in that window, it ultimately saves the WCF XML in the windows registry on the machine that is hosting the client and/or the AOS in subfolders in HKLM\SOFTWARE\Microsoft\Dynamics\6.0\Configuration. The key(s) is wcfconfig paired with wcfconfigversionid, which just stores a GUID to see if it's up-to-date.
When I say two, I mean most people don't even bother to look at the Business Connector AXC. It's what is highlighted in yellow in my image, and you need to specifically choose and refresh it. This could be important for you. In my image, I do not have it chosen. You need to drop the menu down and choose it.
On a dev machine, you can just clear both of those keys and refresh and you should see whatever configuration you're working on update.
This is a long post, but it's important to rule this part out first. If you have someone who's reasonably experienced administering AX they should know how to ensure these are refreshed.
Since you're saying this is not Dynamics AX, but one of the Dynamics 365 versions. The AX version used to be called Dynamics 365 for Finance and Operations Enterprise Edition but they've changed the licensing/naming again, so I don't even know what it's technically called. Most people call it Dynamics 365 for Operations or some variant.
Either way, you should test the service following the below method. We would need to see more information about the service details and call, so following the below is most likely best.
https://learn.microsoft.com/en-us/dynamics365/fin-ops-core/dev-itpro/data-entities/third-party-service-test

Slack Slash commands - Not working in private channels or direct messages

I have programmed several slash commands that show a response in public channels without any problems, but they don't show any response in private channels or direct messages.
As shown below, I am using the in_channel response type. Is there any other response type I can use or a workaround so that it works everywhere?
$data = array(
"username" => "My_user",
"channel" => $channel_id,
"response_type" => "in_channel",
"text" => $text,
"mrkdwn" => true,
"icon_url" => $icon_url
);
$json_string = json_encode($data);
$slack_call = curl_init($slack_webhook_url);
curl_setopt($slack_call, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($slack_call, CURLOPT_POSTFIELDS, $json_string);
curl_setopt($slack_call, CURLOPT_CRLF, true);
curl_setopt($slack_call, CURLOPT_RETURNTRANSFER, true);
curl_setopt($slack_call, CURLOPT_HTTPHEADER, array(
"Content-Type: application/json",
"Content-Length: " . strlen($json_string))
);
$result = curl_exec($slack_call);
curl_close($slack_call);
Thanks in advance!
I talked to the Slack team, which was extremely helpful, and we figured out what the problem was. I am sharing it here in case anybody else runs into the same problem.
The problem was not about the commands being used in public or private channels. The person who created the webhook set it so that it would work on a private channel (our testing channel), so it would only work in that channel or in any channels that she was a part of (so, all the public channels). As soon as I added her to a private channel, it would work.
The solution was for the creator of the webhook to edit it (not the code, just the webhook) and set it by default to a public channel (any) instead of a private channel. This made it work in every channel, even direct messages.
This way, I was able to use my original code, which also allows me to change the user icon dynamically, instead of sending a message back.
I hope that helps other people as well!
That does not look like the right approach.
Responding to slash commands
For responding to a slash command you must not send a new message back (e.g. via webhook as in your code example). Instead just respond to the request from Slack with the content of your message.
Example
$message = array (
'response_type' => 'in_channel',
'text' => $text
);
header ('content-type: application/json');
echo json_encode ($message);
that is all you need.
response_type defines if the response can be seen by all members of a channel "in_channel" or only by the issuer of the slash command "ephemeral"
Please see the official documentation for more details and options.
Sending additional messages
You can of course also send a message from your script in response to the slash command. However, if you want to send a message to a private channel please note that the slash command request from Slack will not include the correct channel ID if it is used in a non-public channel. I don't think there is currently any solution or workaround for this.
You can however always send a direct message to the user by using the ID of the user as channel ID.

Cannot sent cookie with Guzzle request

I have a PHP script that needs to fetch a CSV file from an application. There is an API for the application that allows the script to lot in, which gives the script a session cookie for authentication. I then need to doa GET request to fetch the CSV file (which the API does not support).
Using curl directory works:
$c = curl_init($url);
curl_setopt($c, CURLOPT_COOKIE, 'PHPSESSID=' . $session_id_from_api);
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
$csv_file = curl_exec($c);
echo $csv_file;
That fetches the CSV file using the session ID obtained from the API login and passed through a coookie.
Now, I would like to do the same thing using Guzzle, but I just get the login page back instead. This is the code I'm using:
$client = new Guzzle\Http\Client();
$request = $client->get(
$url,
[
'cookies' => ['PHPSESSID' => $session_id_from_api],
]
);
$response = $client->send($request);
echo $response->getBody(true);
That gives me the login page, so the GET to the application is not recognising the session defined by the cookie.
Is there anything else I need to do to ensure the cookie and value I specify is sent to the remote application?
Edit: looking at $request->getRawHeaders(), I see this line in the headers:
cookies: vec6nb1egvvui6op7qr7b0oqf6
That obviously isn't right. The documentation for my version of Guzzle gives this example:
// Enable cookies and send specific cookies
$client->get('/get', ['cookies' => ['foo' => 'bar']]);
which looks to me to be consistent with what I am passing to Guzzle.
Just to be clear, I am not trying to manage cookies in both directions over multiple requests, so there is no need to store any cookies. I have a cookie name and its value (from another source), and I just want to make sure that name and value gets sent to the destination for a single GET request. I'm not trying to "maintain a session", but in a way I am having a session passed to me from another part of the application (not Guzzle) and need to set my Guzzle request up to use it.
Well, this seems to work. Guzzle was not sending the cookie without being sure the domain it was sending it to was correct:
// Set up a cookie - name, value AND domain.
$cookie = new Guzzle\Plugin\Cookie\Cookie();
$cookie->setName('PHPSESSID');
$cookie->setValue($session_id_from_api);
$cookie->setDomain($domain_of_my_service_url);
// Set up a cookie jar and add the cookie to it.
$jar = new Guzzle\Plugin\Cookie\CookieJar\ArrayCookieJar();
$jar->add($cookie);
// Set up the cookie plugin, giving it the cookie jar.
$plugin = new Guzzle\Plugin\Cookie\CookiePlugin($jar);
// Register the plugin with the client.
$client->addSubscriber($plugin);
// Now do the request as normal.
$request = $client->get($url);
$response = $client->send($request);
// The returned page body.
echo $response->getBody(true);

Download a series of Google Docs (.csv files) to my site using PHP

My conundrum is as follows. I have a lot of data stored in various spreadsheets on Google Docs. Compiling all of the data into a single spreadsheet (for analysis purposes) hit the size limit. Seeing as that is out, I needed another method.
I wrote an Apps Script to take the data and export it all to a series of corresponding .csv files. I now need to get the data from those 30 .csv files stored in Google Docs and import it into a MySQL server. Is this the most efficient method?
Right now, I need a way to import all .csv files from my Google Docs List to the server, where my PHP script can import them to the database. Can you point me in the right direction?
I discovered the following post during my research, which seems pretty close, but it's unfortunately way above my head when it comes to adapting it to my purposes...
http://gdatatips.blogspot.com/2009/07/download-google-doc-using-php-library.html
function download($client, $url, $format=null) {
$sessionToken = $client->getHttpClient()->getAuthSubToken();
$opts = array(
'http' => array(
'method' => 'GET',
'header' => "GData-Version: 3.0\r\n".
"Authorization: AuthSub token=\"$sessionToken\"\r\n"
)
);
if ($url != null) {
$url = $url . "&exportFormat=$format";
}
return file_get_contents($url, false, stream_context_create($opts));
}
// TODO 1: setup a Zend_Gdata_Docs client in $docs_client
// TODO 2: fetch a $feed or $entry
$contentLink = $feed->entries[0]->content->getSrc();
$fileContents = download($docs_client, $contentLink, 'txt');
echo 'Contents of document "' . $feed->entries[0]->title . '":<hr>';
echo "<pre>$fileContents</pre>";
Thanks very much! Any assistance would be greatly appreciated! :)
I would download the .csv files using a batch file that is kicked off from scheduled tasks -(i.e. once per day) - then in the batch file kick off SQL c:\sql\update_timekeeping.sql
The SQl commands stored in the update_timekeeping.sql text file. Also provides history of running the timekeeping update....
It seems to me that file_get_contents() should be similar to the PHP version of this script
function file_get_contents(){
//This function generates a pdf of your current spreadsheet and emails it to yourself as attachment
//Make sure your spreadsheet is not too big. The pdf size tends to be 200kb/page and if too large
//if the pdf is too large the urlFetch might timeout
var AUTH_TOKEN = "xxxxxxxxxxxxxx"; //Enter your AUTH_TOKEN
//You can receive it from https://appscripts.appspot.com/getAuthToken
var ssID=SpreadsheetApp.getActiveSpreadsheet().getId()+"&gid="+SpreadsheetApp.getActiveSpreadsheet().getActiveSheet().getSheetId();
var url = "https://spreadsheets.google.com/feeds/download/spreadsheets/Export?key="
+ ssID + "&exportFormat=csv";
//Add &gid=x at the end of above url if you only want a particular sheet
//gid of a sheet can be obtained by the undocumented function getSheetId()
//ex: SpreadsheetApp.getActiveSpreadsheet().getActiveSheet().getSheetId();
//exportFormat=xls did not work when I tried. I dont know why
var auth = "AuthSub token=\"" + AUTH_TOKEN + "\"";
var res = UrlFetchApp.fetch(url, {headers: {Authorization: auth}});
var content=res.getContentText();
return content
}

Categories