My MySQL database has some fields that sometimes include an apostrophe, so I take care to encode html entities. For example "Cote d'Or" is stored in the database as "Cote d&039;Or".
When a MySQL query populates an href I get something like this in my source code.
Text link
However when I click on the link I get a 403 "Forbidden" error. On checking, hovering on the link says it is reading &039; as an apostrophe. That seems to be the cause of the page error as putting an apostrophe in the database produces the same error and having nothing in there works correctly.
My question now is, how can I have the html entity in the database and still get the link to work correctly?
For URLs, you don't want to use htmlentities() as that's for displaying HTML.
Instead, you'll want to use urlencode():
$link = '/page.php?location=' . urlencode($location);
If your data is already HTML encoded, you'll need to decode it before passing it through urlencode(). A good function for this is html_entity_decode():
$location = html_entity_decode($row['location']);
$link = '/page.php?location=' . urlencode($location);
You might be better using filter_var($var, FILTER_SANITIZE_URL)
http://php.net/manual/en/filter.filters.sanitize.php
While not the best approach, since you are using htmlentities() you can use html_entity_decode()
From: http://php.net/html_entity_decode
html_entity_decode() is the opposite of htmlentities() in that it converts all HTML entities in the string to their applicable characters.
Well, it turns out the real problem wasn't the html entities. It was some of the security stuff written into Apache. A quiet word with my hosting company and they did something that made everything work (presumably removed a mod_security rule).
Moral of the story: if you get a 403 error, suspect Apache security first.
Thanks to the guys wh provided answers.
Related
I'm trying to output the name of a project i.e. "David's Project" in a form, if a user does not correctly input all data in the form, to save the user having to input the name again.
If I var_dump $name I see David's project. But if I echo $name I see David"'" Project. I realise that ' (single quote) becomes "'"; but I have tried using ENT_NOQUOTES and ENT_COMPAT to avoid encoding the single quote but neither works.
$name = trim(filter_input(INPUT_POST, 'name0', FILTER_SANITIZE_STRING));
<form method="post" class="form" />
Title: <input type="text" name="name0" value="<?php echo
htmlspecialchars($name, ENT_NOQUOTES); ?>">
Am I doing something wrong or should the ENT_NOQUOTES work? I tried using str_replace to replace with ' with an \' but this didn't work either.
The only way round this I have found is to use this:
htmlspecialchars_decode(htmlspecialchars($name, ENT_NOQUOTES));
Is that acceptable?
Sorry I realise this is probably a really stupid question but I just can't get my head around it.
Thanks for any replies.
You can accept a simple answer if it solves your problem BUT you should really understand that what you have delved into is a much larger issue you or someone has created for you.
Databases should not contain HTML encoded characters unless they are specifically meant for storing HTML. I highly doubt this is the case as it very rarely is.
Someone is inserting HTML into your database (html encoding data on insert). This means if you ever want to use a mobile app that is not HTML based, or a command line, or anything at all that might use the data and isn't HTML based, you are going to run into a weird problem where the HTML encoded characters have to be removed on output. This is typically kind of the backwards way to do it and can often cause issues.
You rarely need to "sanitize" your inputs. If anything, you should reject input that is not allowed OR simply escape it in the proper way while inserting it into the database. Sanitizing is only a thing in very special circumstances, which you don't appear to have right now. You're simply inputting and outputting text.
You should pretty much never change users input
My suggestion, if possible, is to fix your INSERT code first so it isn't html encoding data. This html encoding should happen when you output the data TO AN HTML FORMAT. You would use htmlspecialchars() to do this.
I'm using a 3rd party API that seems to return its data with the entity codes already in there. Such as The Lion’s Pride.
If I print the string as-is from the API it renders just fine in the browser (in the example above it would put in an apostrophe). However, I can't trust that the API will always use the entities in the future so I want to use something like htmlentities or htmlspecialchars myself before I print it. The problem with this is that it will encode the ampersand in the entity code again and the end result will be The Lion’s Pride in the HTML source which doesn't render anything user friendly.
How can I use htmlentities or htmlspecialchars only if it hasn't already been used on the string? Is there a built-in way to detect if entities are already present in the string?
No one seems to be answering your actual question, so I will
How can I use htmlentities or htmlspecialchars only if it hasn't already been used on the string? Is there a built-in way to detect if entities are already present in the string?
It's impossible. What if I'm making an educational post about HTML entities and I want to actually print this on the screen:
The Lion’s Pride
... it would need to be encoded as...
The Lion&;#8217;s Pride
But what if that was the actual string we wanted to print on the string ? ... and so on.
Bottom line is, you have to know what you've been given and work from there – which is where the advice from the other answers comes in – which is still just a workaround.
What if they give you double-encoded strings? What if they start wrapping the html-encoded strings in XML? And then wrap that in JSON? ... And then the JSON is converted to binary strings? the possibilities are endless.
It's not impossible for the API you depend on to suddenly switch the output type, but it's also a pretty big violation of the original contract with your users. To some extent, you have to put some trust in the API to do what it says it's going to do. Unit/Integration tests make up the rest of the trust.
And because you could never write a program that works for any possible change they could make, it's senseless to try to anticipate any change at all.
Decode the string, then re-encode the entities. (Using html_entity_decode())
$string = htmlspecialchars(html_entity_decode($string));
https://eval.in/662095
There is NO WAY to do what you ask for!
You must know what kind of data is the service giving back.
Anything else would be guessing.
Example:
what if the service is giving back & but is not escaping ?
you would guess it IS escaping so you would wrongly interpret as & while the correct value is &
I think the best solution, is first to decode all html entities/special chars from the original string, and then html encode the string again.
That way you will end up with a correctly encoded string, no matter if the original string was encoded or not.
You also have the option of using htmlspecialchars_decode();
$string = htmlspecialchars_decode($string);
It's already in htmlentities:
php > echo htmlentities('Hi&mom', ENT_HTML5, ini_get('default_charset'), false);
Hi&mom
php > echo htmlentities('Hi&mom', ENT_HTML5, ini_get('default_charset'), true);
Hi&;mom
Just use the [optional]4th argument to NOT double-encode.
I am currently working on a replacement tool that will dynamically replace certain strings (including html) in a website using a smarty outputfilter.
For the replacement to take place, I am using PHP's str_ireplace method, which reads the code that is supposed to be replaced and the replacement code from a database, and then pass the result to the smarty output (using an output filter), in a similar way as the below.
$tpl_source = str_ireplace($replacements['sourceHTML'], $replacements['replacementHTML'], $tpl_source);
The problem is, that although it works great on my dev server, once uploaded to the live server replacements occasionally fail. The same replacements work just fine on my dev version though. After some examinations and googling there was not much I could find out regarding this issue. So my question is, what could influence str_replace's behavour?
Thanks
Edit with replacement example:
$htmlsource = file_get_contents('somefile.html');
$newstr = str_replace('Some text', 'sometext', $htmlsource); // the text to be replaced does exist in the html source
fails to replace. After some checking, it looks like the combination of "> creates a problem. But just the combination of it. If I try to change only (") it works, if I try to change only (>) it works.
It might be that special chars like umlauts do not display on the live server correctly and so str_replace() would fail, if there are specialchars inside the string you want to replace.
Is the input string identical on both systems? Have you verified this? Are you sure?
Things to check:
Are the HTML attributes in the same order?
Are the attribute values using the same kind quote marks? (eg <a href='#'> vs <a href="#">)
Is there any other stray HTML code getting in there?
Is the entity encoding the same? (eg vs - same character; different HTML)
Is the character-set the same? (eg utf-8 vs ISO 8859-1: Accented characters will be encoded differently)
Any of these things will affect the result and produce the failures you're describing.
This was a trikcy problem, and it ended up having nothing to do with the str_replace method itself;
We are using smarty as a tamplating system. The str_replace method was used by a smarty ouput filter in order to change the html in some ocassions, just before it was delivered to the user.
Here is the Smarty outputfilter Code:
function smarty_outputfilter_replace($tpl_source, &$smarty)
{
$replacements = Content::getReplacementsForPage();
if (is_array($replacements))
{
foreach ($replacements as $replacementData)
{
$tpl_source = str_replace($replacementData['sourcecode'], $replacementData['replacementcode'], $tpl_source);
}
}
return ($tpl_source);
}
So this code failed now and then for now apparent reason... until I realized that the HTML code in the smarty template was being manipulated by an Apache filter.
This resulted into the source code in the browser (which we were using as the code to be replaced by something else) not being identical to the template code (which smarty was trying to modify). Result? str_replace failed :)
I'm using more than one ampersand in my url, see my link below
http://www.theonlytutorials.com/video.php?cat=55&vid=3975&auth=many
When i try to validate in w3c validator it showed hundreds of error because of this & (ampersand).
After that i read some post in here and i got the solution too.
Instead of using (&) If i use (&) w3c validates fine.
But the problem now is in search Engine. Instead of taking (&). it is taking like the below link
http://www.theonlytutorials.com/video.php?cat=55&vid=3975&auth=many
if you copy paste the above link in the address bar it will take you to the wrong page!. Please help how can i solve it.
There must be an error in your code but since we cannot see any of it I think the most important bit is to understand why the W3C validator complaints about raw &.
The HTML syntax contains two basic elements: tags (e.g. <strong>) and entities (e.g. €). Everything else is displayed as-is.
Browsers are expected to ignore errors.
When you type unknown or invalid tags, the browser will do its best to guess and fix it (you are probably aware of that already):
<p>Hello <i>world</b>!</p>
... will render as:
<p>Hello <i>world</i>!</p>
But the same happens when you type an unknown or invalid entity. In your example, there are two invalid entities:
http://www.theonlytutorials.com/video.php?cat=55&vid=3975&auth=many
^^^^ ^^^^^
However, it works because the browser is clever enough to figure out the real URL. Only the validator complaints because it is a tool specifically designed to find errors.
Now, imagine I want to use HTML to write an HTML tutorial and I want to explain the <strong> tag. If I just type <strong>example</strong>, the browser will display example. I need to encode the < symbol so it no longer has a special meaning:
<strong>example</strong>
Now the browser displays <strong>example</strong>, which is precisely the content I want to show.
The same happens with your URL. Since & is part of the entity syntax, when I want to insert a literal & I need to encode it as well:
Barnes & Noble
... will render as Barnes & Noble. Please note that this is only a syntactic trick to insert plain text into a HTML document. Your document shows Barnes & Noble. to all effects, no matter how you encode it. So when you replace & with & in your URL, you are not changing your URL, you are just encoding it.
If search engines are spidering the wrong URL, that means you have actually changed your URL rather than just encoding it, so the source code is:
http://www.theonlytutorials.com/video.php?cat=55&vid=3975&auth=many
... and renders as:
http://www.theonlytutorials.com/video.php?cat=55&vid=3975&auth=many
This can happen, for instance, if you encode twice:
<?php
$url = 'http://www.theonlytutorials.com/video.php?cat=55&vid=3975&auth=many';
$url = htmlspecialchars($url);
$url = htmlspecialchars($url);
echo $url;
... or:
<?php
$url = 'http://www.theonlytutorials.com/video.php?cat=55&vid=3975&auth=many';
$url = htmlspecialchars($url); // Oops: URL is already encoded!
echo $url;
Seems that you made a typo error, it must be & not &amp;
I am working with an XML feed that has, as one of it's nodes, a URL string similar to the following:
http://aflite.co.uk/track/?aid=13414&mid=32532&dl=http://www.google.com/&aref=chris
I understand that ampersands cause a lot of problems in XML and should be escaped by using & instead of a naked &. I therefore changed the php to read as follows:
<node><?php echo ('http://aflite.co.uk/track/?aid=13414&mid=32532&dl=http://www.google.com/&aref=chris'); ?></node>
However when this generates the XML feed, the string appears with the full &
and so the actual URL does not work. Apologies if this is a very basic misunderstanding but some guidance would be great.
I've also tried using %26 instead of & but still getting the same problem.
If you are inserting something into XML/HTML you should always use the htmlspecialchars function. this will escape your strings into correct XML syntax.
but you are running into a second problem.
your have added a second url to the first one.
this need also escaped into url syntax.
for this you need to use urlencode.
<node><?php echo htmlspecialchars('http://aflite.co.uk/track/?aid=13414&mid=32532&aref=chris&dl='.urlencode('http://www.google.com/')); ?></node>
& is correct for escaping ampersands in an XML document. The example you've given should work.
You state that it doesn't work, but you haven't stated what application you're using, or in what way it doesn't work. What exactly happens when you click the link? Do the & strings end up in the browser's URL field? If that's the case, it sounds like a fault with the software you've viewing the XML with. Have you tried looking at the XML in another application to see if the problem is consistent?
To answer the final part of your question: %26 would definitely not work for you -- this would be what you'd use if your URL parameters needed to contain ampersands. Say for example in aref=chris, if the name chris were to an ampersand (lets say the username was chris&bob), then that ampersand would need to be escaped using %26 so that the URL parser didn't see it as starting a new URL parameter.
Hope that helps.