I have the following value from a field in a database that I would like to output exactly like this...
Use the term “fully accredited”-there is no such thing as a partial accreditation.
I need to include the quotes around the term "fully accredited".
Here's my output in PHP...
echo "<p><strong>Never:</strong> <span id=\"nevermsg\">".$results['never1']."</span></p>";
But, when I render the data on the page, it's showing these little diamond shapes with question marks inside them.
*(The 'span id' is there for styling and isn't relevant)
I don't think escaping would work here because quotes are not used in all the data values.
Not sure what to do...
The quotes in your string are extended characters.
You could fix this problem pretty quickly by simply replacing them with standard " quote characters rather than the curly quotes “ ” you've got now.
However, in the long term, you probably need to be able to handle extended characters, as it includes all kinds of things you're likely to need in your text, not just curly quote marks.
To fix this problem properly, you need to ensure that your system uses UTF-8 encoding at all levels. This includes within the database, your PHP code files, and the data that is sent to the browser.
I suggest reading up further on this here: UTF-8 all the way through
Related
I was wondering if anybody knew how to get around this problem.
I am gathering user input from a HTML form which is then posted using htmlspecialchars into PHP to avoid issues when using quotes/etc...
However, I also want to run server-side validation checks on the data being gathered through regular expressions - though I'm not sure how to go about this.
So far, I have thought of decoding the htmlspecialchars - but because I am going to be using the Strings straight away, this means that the code could break after I run this conversion. e.g: Let's say the user inputted a single quote, " into a field. This would be converted to ", then if I decode this and use it in a variable, it could end up like: $string = """; which is going to give me issues.
Any advice on this would be greatly appreciated!
You seem to misunderstand the difference between data and how this data is altered to be parseable in a certain context.
A php string can contain any data. What is stored in this string is the "raw" form: the form in which we want to manipulate the data if needed.
In certain contexts, not all characters are valid. For example, in a html textarea, the < and > characters may not be used, because they are special characters. We still want to be able to use these characters. To use special characters in a context, we escape these characters. By escaping a special character it looses its special meaning. In the context of a html textarea, the < character is escaped as the sequence <. Unlike the < character, this escaped sequence does not have a special meaning in html, and thus if we send the following sequence to the browser, it knows how to parse that sequence and display the right thing: <textarea><</textarea>. When we talk about what the data is that this textarea contains, we do not say that it contains <, but instead we say that it contains <.
As you said, in a php script, in a double quoted string, the " character has a special meaning. This has only to do with parsing. PHP simply does not know how to parse a sequence $str = """;. If we would want to have the double quote in such a double quoted string, we would need to escape it. We escape a double quote in a php double quoted string by prepending it with a \. To make a string containing a single double quote, using the double quoted notation, you would write $str = "\"";.
However, none of this matters.. You are taking input from a html form. When you click the submit button, the browser reads what is in the textarea(, and decodes it as html?). The browser then encodes it in a way as dictated by the form tag, and sends it to the server. The server then decodes the blob of text back in it's raw data form. That data is passed to PHP, and it is this form you will encounter in $_POST['myTextarea'].
In conclusion: If data is encoded, realize for which context it was encoded and decode it based on that context. You do not need to escape for php quoted strings, because you are working on internal strings. There is nothing to parse. Remind yourself that when you are going to use the data somewhere, that you should take care that all special characters in your data for that particular context are escaped.
I suppose that htmlspecialchars() function is called after posting the form to PHP. Simplest solution then will be to match against regular expression first and then do htmlspecialchars().
Also, if you have string encoded with htmlspecialchars(), after decoding with htmlspecialchars_decode(), PHP internal representation will be "\"", so you break nothing. There is big difference how you write strings by hand to PHP file and how PHP internally handle them. You really don't need to be bothered by this.
I am translating with poedit. However poedit seems to be ignoring apostrophes. For example shouldn't is coming through as shouldnt. I am encoding in utf-8. Does anyone know why this is the case and if there is a solution ?
I assure you that Poedit isn't somehow ignoring or eating apostrophes — that's preposterous. It's just an editor that puts whatever you wrote, exactly as you wrote it (yes, including ' or any Unicode characters), into your PO and MO files.
Your problem is in your PHP code where you incorrectly escape the (translated) strings before printing them — how and in what context you do that is unfortunately something you didn't share.
But this is why e.g. WordPress has functions like esc_attr_e that do any necessary escaping and do it correctly, so that you don't have to do anything ridiculous (and painful to work with!) like substituting ' with ’ in all your translations (which wouldn't even work when using untranslated text…).
You need to use the html entity: ’
Source: http://geektnt.com/tag/poedit
Some text characters need to be converted into html entities otherwise they will not display correctly. A very common example is a word containing an apostrophe or single quote (‘) which needs to be replaced with ’ — for example, Chloe O’Brian should be written as Chloe O’Brian. For a complete list of html entities, visit W3Schools.
I'm allowing some user input on my website, that later is read in XML. Every once in a while I get these weird single or double quotes like this ”’. These are directly copied from the source that broke my XML. I'm wondering if there is an easy way to correct these types of characters in my xml. htmlentities did not seem to touch them.
Where do these characters come from? I'm not even sure how I'd go about typing them out unintentionally.
EDIT- I forgot to clarify these quotes are not being used in attributes, but in the following way:
<SomeTag>User’s Input</SomeTag>
Don't disallow and/or modify foreign characters; that's just annoying for your users! This is just an encoding issue. I don't know what parser you're using to read the XML, but if it's reasonably sophisticated, you can solve your problem by including the following encoding pragma at the top of your XML files:
<?xml version="1.0" encoding="UTF-8"?>
There may also be a UTF-8 option in the parser's API.
Edit: I just read that you're reading the XML directly in a browser. Most browsers listen to the encoding pragma!
Edit 2: Apparently, those quotes aren't even legal in UTF-8, so ignore what I said above. Instead, you might find what you're looking for here, where a similar problem is being discussed.
Are these quotes being used in text content, or to delimit attributes? For attribute delimiters, XML requires typewriter quotes (single or double). Microsoft and other word-processing applications often try to be smart and replace typewriter quotes with typographical quotes, which is almost certainly the answer to the question "where are they coming from?".
If you need to get rid of them, a simple global replace using a text editor will do the job fine.
But you might try to work out first why they are causing a problem. Perhaps your data flow can't handle ANY non-ASCII characters, in which case that's a deeper problem that you really ought to fix (it would typically imply some unwanted transcoding is happing somewhere along the line).
If the input string is UTF-8 encoded, maybe you need to specify that to htmlentities(), for example:
$html = htmlentities( '”’', ENT_COMPAT, "utf-8" );
echo $html;
For me gives:
”’
whereas
$html = htmlentities( '”’' );
echo $html;
gets confused:
â??â??
If the input string is non-UTF-8, then you'd need to adjust the encoding arg for htmlentities() accordingly.
Stay away from MicroSoft Office apps. Word, Excel etc. have a nasty habit of replacing matching pairs of single quotes and double quotes with non-standard "smart-quotes".
These quote characters are truly non-standard and never made it into the official latin-1 character set. All the MS Office apps "helpfully" replace standard quote characters with these abominations.
Just google for "undoing smatquotes" or "convert smartquotes back" for hints tips and regexes to get rid of these.
Use
$s = 'User’s Input';
$descriptfix = preg_replace('/[“”]/','\"',$s);
$descriptfix = preg_replace('/[‘’]/','\'',$descriptfix);
echo "<SomeTag>htmlentities($s)</SomeTag>";
I've run into an interesting behavior in the native PHP 5 implementation of json_encode(). Apparently when serializing an object to a json string, the encoder will null out any properties that are strings containing "curly" quotes, the kind that would potentially be copy-pasted out of MS Word documents with the auto conversion enabled.
Is this an expected behavior of the function? What can I do to force these kinds of characters to covert to their basic equivalents? I've checked for character encoding mismatches between the database returning the data and the administration page the inserts it and everything is setup correctly - it definitely seems like the encoder just refuses these values because of these characters. Has anyone else encountered this behavior?
EDIT:
To clarify;
MSWord will take standard quotation marks and apostraphes and convert them to more aesthetic "fancy" or "curly" quotes. These characters can cause problems when placed in content managers that have charset mistmatches between their editing interface (in the html) and the database encoding.
That's not the problem here, though. For example, I have a json_object representing a person's profile and the string:
Jim O’Shea
The UTF code for that apostraphe being \u2019
Will come out null in the json object when fetched from database and directly json_encoded.
{"model_name":"Bio","logged":true,"BioID":"17","Name":null,"Body":"Profile stuff!","Image":"","Timestamp":"2011-09-23 11:15:24","CategoryID":"1"}
Never had this specific problem (i.e. with json_encode()) but a simple - albeit a bit ugly - solution I have used in other places is to loop through your data and pass it through this function I got from somewhere (will credit it when I find out where I got it):
function convert_fancy_quotes ($str) {
return str_replace(array(chr(145),chr(146),chr(147),chr(148),chr(151)),array("'","'",'"','"','-'),$str);
}
json_encode has the nasty habit of silently dropping strings that it finds invalid (i.e. non-UTF8) characters in. (See here for background: How to keep json_encode() from dropping strings with invalid characters)
My guess is the curly quotes are in the wrong character set, or get converted along the way. For example, it could be that your database connection is ISO-8859-1 encoded.
Can you clarify where the data comes from in what format?
If I ever need to do that, I first copy the text into Notepad and then copy it from there. Notepad forces it to be normal quotes. Never had to do it through code though...
My friend has been playing around with some language stuff on our site and our file names are being out put with these characters now. Usually I'd wait for him to wake up but this is a pretty big issue as we are getting e-mails through about the weird characters in the file names.
You don't see the characters when echoed in HTML, but we have the names being output to a header, which does show the characters, like so:
header('Content-Disposition: attachment; filename="'.$title.'.'.strtolower($type).'";');
How can we avoid these characters from displaying? They are also being input to our database, file names such as asdfmovie - I have googled the codes but I can't find any results for them.
Does anyone know what they are? and how to avoid them?
Thank you
html_entity_decode()
http://php.net/manual/en/function.html-entity-decode.php
These are html entities that are valid in HTML. Your email client is actually encoding them into HTML entities (a double effect), which means that the actual entities are what you're seeing. Just make sure that anything passed into the email runs through the html_entity_decode() function.
These are HTML entities which can be decoded using html_entity_decode, like echo html_entity_decode($str, ENT_COMPAT, 'UTF-8').
It's wrong to store such values in the database though, as you are seeing. The values should be stored in their original form and only HTML entity encoded when necessary for outputting to HTML. Figure out where they're being HTML encoded and fix that. If you already have a database full of this nonsense... um, have fun reversing it. :o)