As per the issued solution on Azure/azure-event-hubs-for-kafka/issues/51, I tried to use and consume a message but it came out with no luck. Please have a look at the code and guide on what needs to be done?
I have installed the rdkafka.so on my ubuntu system as per php manual for rdkafka and then used this piece of code :
$conf = new RdKafka\Conf();
$conf->set('group.id', '$Default');
$conf->set('metadata.broker.list', 'NAMESPACE.servicebus.windows.net:9093');
$conf->set('security.protocol', 'SASL_SSL');
$conf->set('sasl.mechanisms', 'PLAIN');
$conf->set('sasl.username', '$ConnectionString');
$conf->set('sasl.password', 'Endpoint=sb://NAMESPACE.servicebus.windows.net/;SharedAccessKeyName=syn-wk;SharedAccessKey=7XXX/XXXE=;EntityPath=someStringHere');
$conf->set('ssl.ca.location', '/absolute_path_to_cert');
$conf->set('enable.partition.eof', 'true');
$conf->set('api.version.request', 'false');
$conf->set('log_level', (string) LOG_DEBUG);
$conf->set('debug', 'all');
$rk = new RdKafka\Consumer($conf);
$topicConf = new RdKafka\TopicConf();
$topicConf->set('auto.commit.interval.ms', 100);
$topicConf->set('offset.store.method', 'broker');
$topicConf->set('auto.offset.reset', 'earliest');
$topic = $rk->newTopic("test", $topicConf);
$topic->consumeStart(0, RD_KAFKA_OFFSET_STORED);
while (true) {
$message = $topic->consume(0, 120*10000);
switch ($message->err) {
case RD_KAFKA_RESP_ERR_NO_ERROR:
var_dump($message);
break;
case RD_KAFKA_RESP_ERR__PARTITION_EOF:
echo "No more messages; will wait for more\n";
break;
case RD_KAFKA_RESP_ERR__TIMED_OUT:
echo "Timed out\n";
break;`
default:
throw new \Exception($message->errstr(), $message->err);
break;
}
}
My connection string already has produced events in it that I can confirm by azure-sdk-for-js, using JS SDK I am able to produce and consume events.
When I try to read the $message from $topic it returns as null
PS:
I am currently using the pricing tier as 'Standard' on the azure event hub and it has Kafka Surface 'enabled'
I have tried setting api.version.request as 'true' as well
Related
This code sometimes works, but frequently runs for ~20s then fails with the "503 Service Unavailable" message when I call getPage(). The authentication/token request seems to be working fine.
I can't seem to identify any pattern of when it fails/succeeds. I don't believe it's a throttling error, as there is no "Retry-After" header returned, and the script only runs once per day at night with <200 records returned. I've also tried removing the $filter and changing parameter order as described here, with no clear benefit.
Can someone please help find the cause here? Happy to share any additional info. Any help is much appreciated, thanks!
<?php
define('DEBUG', true);
session_start();
// set config vars
$ms_url_base = "https://login.microsoftonline.com/d3523db7-f84a-4a24-a815-cd4ba4691c9c";
$ms_client_id = '<client id>';
$ms_redirect_uri = "https://example.com/path";
$ms_scope = "calendars.readwrite user.read";
$ms_client_secret = '<secret>';
$ms_auth_endpoint = '/oauth2/v2.0/authorize';
$ms_token_endpoint = '/oauth2/v2.0/token';
$query_numdays = 100;
if (DEBUG) error_reporting(E_ALL);
date_default_timezone_set("America/Detroit");
require_once __DIR__.'/vendor/autoload.php';
use Microsoft\Graph\Graph;
use Microsoft\Graph\Model;
class EventMod extends \Microsoft\Graph\Model\Event {
// custom functions here
public function getNextLink() {
parent::getNextLink();
}
}
class ResponseMod extends \Microsoft\Graph\Http\GraphResponse {}
// authorization
$provider = new Stevenmaguire\OAuth2\Client\Provider\Microsoft([
'clientId' => $ms_client_id,
'clientSecret' => $ms_client_secret,
'redirectUri' => $ms_redirect_uri,
'urlAuthorize' => $ms_url_base.$ms_auth_endpoint,
'urlAccessToken' => $ms_url_base.$ms_token_endpoint,
'urlResourceOwnerDetails' => 'https://graph.microsoft.com/v2.0/me',
]);
if (!isset($_GET['code'])) {
$options = ['scope' => $ms_scope, 'aud' => 'Graph'];
$authUrl = $provider->getAuthorizationUrl($options);
$_SESSION['oauth2state'] = $provider->getState();
header('Location: '.$authUrl);
exit;
} elseif (empty($_GET['state']) || ($_GET['state'] !== $_SESSION['oauth2state'])) {
unset($_SESSION['oauth2state']);
exit('Invalid state');
} else {
try {
$token = $provider->getAccessToken('authorization_code', ['code' => $_GET['code']]);
} catch (League\OAuth2\Client\Provider\Exception\IdentityProviderException $e) {
exit ("Get access token exception: ".$e->getMessage());
}
if (DEBUG) {
echo 'Access Token: ' . $token->getToken() . "<p>";
echo 'Refresh Token: ' . $token->getRefreshToken() . "<p>";
echo 'Expired in: ' . $token->getExpires() . "<p>";
echo 'Already expired? ' . ($token->hasExpired() ? 'expired' : 'not expired') . "<p>";
}
// start calendar query
$start=new DateTimeImmutable("yesterday 0:0:1");
$end = $start->add(new DateInterval("P".$query_numdays."D"));
$url='/me/calendarview'
.'?startdatetime='.$start->format('c')
.'&enddatetime='.$end->format('c')
.'&$filter=isOrganizer+eq+false'
.'&$select=subject,responseStatus,start,categories'
.'&$orderby=start/dateTime';
$graph = new Graph;
$graph->setAccessToken($token->getToken());
$data = array();
try {
$iterator = $graph->createCollectionRequest("GET", $url)
->setReturnType(EventMod::class)
->addHeaders(["Prefer" => 'outlook.timezone="America/Detroit"'])
->setPageSize(25);
do {
$page = $iterator->getPage(); /*************** THIS IS WHERE THE EXCEPTION HAPPENS ************/
if (DEBUG) echo "<pre>".print_r($page, true)."</pre>";
$data = array_merge($data, $page);
} while (!$iterator->isEnd());
}
catch (\Microsoft\Graph\Exception\GraphException $e) {
if (DEBUG) echo "GraphException Message: ".$e->getMessage();
exit;
}
catch (Exception $e) {
if (DEBUG) echo "Unk Exception getting data: ".$e->getMessage();
exit;
}
if (DEBUG) print_r($data);
}
?>
composer.json
{
"require": {
"microsoft/microsoft-graph": "^1.29",
"stevenmaguire/oauth2-microsoft": "^2.2"
}
}
I played with your above API call and i ended up noticing the issue - in the case of having bigger larger dates and lot of data at my end (not with smaller time window or less records). It tells me that the failure is due to the client timeout. We need to understand that Calendar view is an expensive operation that too when you deal with calendars and filters added to it. So i went ahead in this scenario, reduce/minimize time window for calendar view by client so smaller segments of time are scanned for matching calendar events. It helped me to get the records as i expected and make use of effective usage of Calendarview API call too.
Apparently the issue was not in the code at all. I discovered that the 503 error was thrown only when a specific date's events were being read by the API. One of the events was rather large and I deleted it; after this the script works well.
This particular event has 16536 attendees listed, which I believe was the source of the error. Still unexplained is why the error was intermittent. I was eventually able to get graph explorer to successfully read the event, so perhaps the bug is in the Microsoft Graph SDK for PHP. I will post there to see if the devs want to capture this error condition.
I created API for my android application and when I made multiple requests from my android application server is not responding for a while even the website is can't be reached.
This happens when I made another request before the first request is completed. I create a simple PHP API.
Thanks in advance.
My API Code:
switch ($_POST['method']) {
case 'say_hello':
$data="Hello There..Welcome";
echo json_encode($data);
break;
case 'breakfast':
$data="Hello There..Breakfast is ready";
echo json_encode($data);
break;
default:
$msg = "Please Select any category This is default!";
$mainArr = array(
'status' => "0",
'msg' => $msg,
);
echo json_encode($mainArr);
exit();
break;
}
So, I'm using php-ews library to connect to my Microsoft Office 365 Exchange Email account to read emails. I've connected successfully to it and I have managed to retrieve a list of emails that I need.
Now the problem is that I cannot get message body. Reading documentation about Exchange Web Services it says that body cannot be fetched with FindItem(), only with GetItem(), and that's okay.
Now the problem I'm seeing is following:
I tried all possible examples I could find about this, and the code doesn't have any errors, it just says "Soap client returned status of 404".
If anyone has any idea where to look for the solution, please tell me.
EDIT:
$ews = new Client('outlook.office365.com/EWS/OData/Me/Inbox/Messages', '###', '###', ClientEWS::VERSION_2010_SP2);
$request = new FindItemType();
$request->ItemShape = new ItemResponseShapeType();
$request->ItemShape->BaseShape = DefaultShapeNamesType::DEFAULT_PROPERTIES;
$request->ItemShape->BodyType = BodyTypeResponseType::BEST;
$request->Traversal = ItemQueryTraversalType::SHALLOW;
$request->ParentFolderIds = new NonEmptyArrayOfBaseFolderIdsType();
$request->ParentFolderIds->DistinguishedFolderId = new DistinguishedFolderIdType();
$request->ParentFolderIds->DistinguishedFolderId->Id = DistinguishedFolderIdNameType::INBOX;
// sort order
$request->SortOrder = new NonEmptyArrayOfFieldOrdersType();
$request->SortOrder->FieldOrder = array();
$order = new FieldOrderType();
// sorts mails so that oldest appear first
// more field uri definitions can be found from types.xsd (look for UnindexedFieldURIType)
$order->FieldURI = new PathToUnindexedFieldType();
$order->FieldURI->FieldURI = 'item:DateTimeReceived';
$order->Order = 'Ascending';
$request->SortOrder->FieldOrder[] = $order;
try{
//getting list of all emails - works perfectly
$result = $ews->FindItem($request);
if ($result->ResponseMessages->FindItemResponseMessage->ResponseCode == 'NoError' && $result->ResponseMessages->FindItemResponseMessage->ResponseClass == 'Success') {
$count = $result->ResponseMessages->FindItemResponseMessage->RootFolder->TotalItemsInView;
$request = new GetItemType();
$request->ItemShape = new ItemResponseShapeType();
$request->ItemShape->BaseShape = DefaultShapeNamesType::ALL_PROPERTIES;
for ($i = 0; $i < $count; $i++){
$message_id = $result->ResponseMessages->FindItemResponseMessage->RootFolder->Items->Message[$i]->ItemId->Id;
$messageItem = new ItemIdType();
$messageItem->Id = $message_id;
$request->ItemIds->ItemId[] = $messageItem;
}
// Here is your response
// It throws an error here with the message "Soap client returned status of 404"
$response = $ews->GetItem($request);
print_r($response);
}
//print_r($result);
} catch(\Exception $e) {
echo $e->getMessage();
}
It looks like your trying to use the new REST endpoint for Office365
'outlook.office365.com/EWS/OData/Me/Inbox/Messages'
But your trying to make and EWS SOAP Request, the endpoint you should be using for EWS SOAP is
https://outlook.office365.com/EWS/Exchange.asmx
You might want to consider using the new REST interface as an alternative to EWS/SOAP but you then need to use a REST library.ouauth etc as per https://dev.outlook.com/restapi.
I'd suggest you use a newer version of this library that's maintained much more and has more features (In this case, it support OAuth logins for Office 365), garethp/php-ews. When using it, you can either use the endpoint provided by Glen Scales, or just use outlook.office365.com.
I have tool using the REST API SDK for PHP that captures payments for multiple companies in PayPal. If I Post one or more transactions to capture from different companies. The transactions are marked 'Captured' within PayPal everything is great. The issue is trying to capture multiple transactions from the same company within the same Post. What happens is only the first transaction in the array gets marked as 'Captured' in PayPal but any other transaction in that same array does not. Seemingly they get ignored.
The PayPal logs with the application do not show any errors. Here is the method that grabs the POST transaction Ids:
if(isset($_POST['toCapture']) && ! empty($_POST['toCapture'])){
// Array of transaction ids to capture
$idsToCapture = $_POST['toCapture'];
for($i = 0; $i < $arrayCount; $i++) {
// Transaction id for current iteration in the loop
$theId = $idsToCapture[$i]; //transactionId
// Function call to get information for transaction id being processed in current iteration of loop
$infoToCapture = getInfoToCapture($theId);
$theCompany = $infoToCapture['Company'];
$theAmt = number_format($infoToCapture['DocumentAmount'], 2); // DocAmt formatted to two decimal places
$ohId = $infoToCapture['OrderHeaderId'];
try {
// Setting $theId to $authId
$authId = $theId;
// Maing sure the ids to be processed do not have an acknoledgement from PayPal
$checkAckNull = isAckNull($authId);
// Setting the amount we want to capture
$amt = new Amount();
$amt->setCurrency("USD")
->setTotal($theAmt);
// If there is no acknowledgment from PayPal, process the capture
if($checkAckNull == NULL){
### Capture
$capture = new Capture();
$capture->setId($authId)
->setAmount($amt);
// it is the $apiContexts that are different for each company
switch ($theCompany) {
case "BLL":
allTheImportantStuff( $apiContextBLL, $authId, $ohId, $capture, $theCompany );
break;
case "LOT":
allTheImportantStuff( $apiContextLOT, $authId, $ohId, $capture, $theCompany );
break;
case "SNA":
allTheImportantStuff( $apiContextSNA, $authId, $ohId, $capture, $theCompany );
break;
case "FAB":
allTheImportantStuff( $apiContextFAB, $authId, $ohId, $capture, $theCompany );
break;
case "STS":
allTheImportantStuff( $apiContextSTS, $authId, $ohId, $capture, $theCompany );
break;
case "MIA":
allTheImportantStuff( $apiContextMIA, $authId, $ohId, $capture, $theCompany );
break;
}
} // end if
else {
}
} catch (PayPal\Exception\PayPalConnectionException $ex) {
echo "Exception: " . $ex->getMessage() . PHP_EOL;
echo '<pre>'; var_dump( $ex->getData() ); echo '</pre>';
$fail = "failure";
insertFail($theCompany, $ohId, $theId, $fail);
} // end try/catch
} // end for loop
}
Here is the allTheImportantStuff method:
function allTheImportantStuff( $apiContext, $authId, $ohId, $capture, $theCompany ){
global $mode;
// Lookup the authorization.
$authorization = Authorization::get($authId, $apiContext);
// Perform a capture
$getCapture = $authorization->capture($capture, $apiContext);
// Get acknowledgment from capture
$ack = $getCapture->getId();
// Get state from capture
$state = $getCapture->getState();
return insertAck($theCompany, $ohId, $ack, $authId, $state);
}
Up until (roughly) a few weeks ago this code worked. I am not using the latest version of the REST API SDK for PHP but trying to avoid updating because other tools are using the SDK as well and work fine. I hope I have explained the issue thoroughly enough. Any thoughts or ideas on what may be going on or how to fix would be greatly appreciated.
i'm creating a PHP script to integrate with Podio.. I think i have all right, but i can't verify the webhook from the Podio website.. Can you tell me what i'm doing wrong? This is my code:
require_once '../lib/Podio/PodioAPI.php';
require_once '../lib/Podio/__cfg.php';
Podio::setup(CLIENT_ID, CLIENT_SECRET);
if (!Podio::is_authenticated()) {
Podio::authenticate('app', array('app_id' => APP_ID, 'app_token' => APP_TOKEN));
}
if ($_POST) {
switch ($_POST['type']) {
case 'item.create' :
$item = PodioItem::get($_POST['item_id']);
doMyStuff('create',$item);
break;
case 'item.update' :
$item = PodioItem::get($_POST['item_id']);
doMyStuff('update',$item);
break;
}
}
You need to add the case to verify your webhook:
case 'hook.verify':
PodioHook::validate($_POST['hook_id'], array('code' => $_POST['code']));
break;
Try this, i'm sure it will work!