Convert html entities to unicode characters via php mysql csv export - php

Question

In my MySQL database this is a sample of an HTML entity that I have:
Ú
When I export it through my script this is what I get:
ú
As you can see in my script I already have 'html_entity_decode' which should convert it appropriately to this (which is what I want):
Ú
Obviously, I am doing something wrong. I have exhausted other various scripts, solutions and otherwise have been trying to resolve this issue for over a day. Here is my PHP code:
$link = mysqli_connect("localhost", "user", "pass", "db");
$sql="SELECT * FROM wtf";
$result=mysqli_query($link,$sql);
if (!$result) die('Couldn\'t fetch records');
$fp = fopen('php://output', 'w');
if ($fp && $result) {
header("Content-type: application/vnd.ms-excel");
header("Content-Encoding: UTF-8");
header('Content-Disposition: attachment; filename="results.csv"');
header('Pragma: no-cache');
header('Expires: 0');
fputcsv($fp, array('Nome'));
while ($row = $result->fetch_array(MYSQLI_NUM)) {
fputcsv($fp, array_map('html_entity_decode',array_values($row)), ',', '"');
}
die;
}
mysqli_close($link);
exit;
Could someone please help or at least point me in the right direction? Having taken on a project that requires European characters in the CSV results, it has been nothing less then a nightmare...

Answer 1

It sounds like you're probably using a newer version of PHP, which will default to "UTF-8" when html_entity_decode() is called. Maybe try something like this:
Instead of this:
fputcsv($fp, array_map('html_entity_decode',array_values($row)), ',', '"');
Try this:
fputcsv($fp, call_user_func_array('html_entity_decode', array(array_values($row), ENT_COMPAT, 'ISO-8859-1')), ',', '"');

Answer 2

The problem is caused by Excel misinterpreting the character encoding of your output.
An output like ú is an indication that a multi-byte character is being interpreted as two separate single byte characters. When instead of writing a string as CSV, you echo that same string, it is rendered correctly, so this means the problem is not in the string, as stored by PHP.
The header Content-Encoding: UTF-8 does not find its way to Excel, so in order to make Excel aware of the UTF-8 encoding, output a Byte Order Mark at the start of the output:
$fp = fopen('php://output', 'w');
if ($fp && $result) {
header("Content-type: application/vnd.ms-excel");
header("Content-Encoding: UTF-8");
header('Content-Disposition: attachment; filename="results.csv"');
header('Pragma: no-cache');
header('Expires: 0');
fwrite($fp, "\xEF\xBB\xBF"); // <--- add this
Secondly, things tend to also work better when you use a TAB character as separator instead of a comma, as in Europe some regional settings define the semi-colon as the separator (the comma being taken as decimal separator), and this will make all columns collapse into one. So write:
fputcsv($fp, array('Nome'), "\t", '"');
while ($row = $result->fetch_array(MYSQLI_NUM)) {
fputcsv($fp, array_map('html_entity_decode',array_values($row)), "\t", '"');
}