Issue with accents and encoding with Bing Spell Check API v7 - php

Question

So I am trying to work with Bing's Spell Check API in PHP, but I'm having an issues where accents and other special characters aren't decoded properly, creating many errors that aren't in the original text and messing with the offsets.
My implementation is quite simple - it's heavily based on the example they give in their documentation. I'm not sure if I am supposed to be doing something differently or if it is an issue on their side with how they decode those special characters (which seems highly unlikely - me messing something up is much more probable..!)
Here's the code:
$host = 'https://api.cognitive.microsoft.com';
$path = '/bing/v7.0/spellcheck?';
$data = array (
'mkt' => $lang,
'mode' => 'proof',
'text' => urlencode($text)
);
$encodedData = http_build_query($data);
$key = 'subscription key redacted for obvious reasons';
$headers = "Content-type: application/x-www-form-urlencoded\r\n" .
"Ocp-Apim-Subscription-Key: $key\r\n";
if (isset($_SERVER['REMOTE_ADDR']))
$headers .= "X-MSEdge-ClientIP: " . $_SERVER['REMOTE_ADDR'] . "\r\n";
$options = array (
'http' => array (
'header' => $headers,
'method' => 'POST',
'content' => $encodedData
)
);
$context = stream_context_create ($options);
$result = file_get_contents ($host . $path, false, $context);
if ($result === FALSE) {
# Handle error
}
$decodedResult = json_decode($result, true);
If, for example, I try to spell check the following string:
d'institution
$encodedData becomes the following:
mkt=fr-CA&method=proof&text=d%25E2%2580%2599institutions
And the results I get from the API are the following:
array(2) {
["_type"]=>
string(10) "SpellCheck"
["flaggedTokens"]=>
array(1) {
[0]=>
array(4) {
["offset"]=>
int(8)
["token"]=>
string(14) "99institutions"
["type"]=>
string(12) "UnknownToken"
["suggestions"]=>
array(2) {
[0]=>
array(2) {
["suggestion"]=>
string(15) "99 institutions"
["score"]=>
float(0.93191315174102)
}
[1]=>
array(2) {
["suggestion"]=>
string(14) "99 institution"
["score"]=>
float(0.6518044080768)
}
}
}
}
}
As you can see, the decoding seems to be problematic, as the % gets encoded twice, and is only decoded once apparently. Now, if I remove the url_encode() when setting the value of 'text' in $data, it'll work fine for the apostrophe, but it doesn't work with accents. For example, the following string:
Responsabilité
is interpreted by the API as
Responsabilité
which returns an error.
This could very well be something simple that I'm overlooking, but I've been struggling with this for quite a while and would appreciate any help I can get.
Thanks,
- Émile
[ Edit ] Well, as always... when in doubt, assume you're wrong. The API recommended to change all of the accents for regular letters because even if the specified language was French, it still gave suggestions in English instead of returning an empty array. As for the accents that didn't seem to be decoded, well... I was var_dump-ing that data without any doctype set, so of course it would show without the proper encoding. Sorry about that - in the end, simply removing the urlencode() does the trick!

Answer 1

As per the docs:
The API supports two proofing modes, Proof and Spell. The default mode is Proof. The Proof spelling mode provides the most comprehensive checks, but it's available only in the en-US (English-United States) market. For all other markets, set the mode query parameter to Spell. The Spell mode finds most spelling mistakes but doesn't find some of the grammar errors that Proof catches (for example, capitalization and repeated words).