Why do we not encode = and & in query strings? I am referencing RFC 3986 but cannot find where it says that we should not encode these characters. Using Guzzle, it doesn't seem they encode anything really.
Take for example the query string: key1='1'&key2='2', shouldn't this be encoded as key1%3D%271%27%26key2%3D%272%27? If I plug key1='1'&key2='2' into chrome as a query string (e.g. www.google.com?key1='1'&key2='2'), it appears as key1=%271%27&key2=%272%27, which does not match guzzle. Guzzle outputs key1='1'&key2='2'. Guzzle's encoding algorithm is below:
private static $charUnreserved = 'a-zA-Z0-9_\-\.~';
private static $charSubDelims = '!\$&\'\(\)\*\+,;=';
public function encode()
{
return preg_replace_callback(
'/(?:[^' . self::$charUnreserved . self::$charSubDelims . '%:#\/\?]++|%(?![A-Fa-f0-9]{2}))/',
function ($match) {
return urlencode($match[0]);
},
$str
);
}
= and & don't have any special meaning as part of URL syntax. As far as URL syntax is concerned, they're just ordinary characters.
However, when used in query strings, there's a convention implemented by most application frameworks to use them to delimit parameters and values. If you want to use these characters literally in a parameter name or value, you need to encode them. See escaping ampersand in url
Related
I am sending an HTTP GET request with urlencoded value from a client application and on the server side I am using $_GET["Value"] to grab the value.
this is what the request looks like on the client side https://example.com/validate.php?Value=+MqZjrRvtvFdcC3GCRRnnQ== but on the server side the result of $_GET["Value"] is MqZjrRvtvFdcC3GCRRnnQ== without + in the beginning of MqZjrRvtvFdcC3GCRRnnQ== How can I grab the value as it is including all the special characters(if any)
I tried htmlspecialchars($_GET["Value"]) but this didnt work either.
You can't inject any random character in a URL, you need to use proper escaping functions. In PHP you have rawurlencode():
$encoded = 'https://example.com/validate.php?Value=' . rawurlencode('+MqZjrRvtvFdcC3GCRRnnQ==');
https://example.com/validate.php?Value=%2BMqZjrRvtvFdcC3GCRRnnQ%3D%3D
(Demo)
In particular, + is some old encoding for whitespace character (U+0020) and = is often used to separate argument name from argument value.
The + is a special char which will be escaped by parse_str().
You need to parse the query string by yourself.
Note: If there are multiple values you need to split by & first.
Calling
http://localhost:4000/?Value=+MqZjrRvtvFdcC3GCRRnnQ==
[$key, $value] = explode('=', $_SERVER['QUERY_STRING']);
will give a $value of
+MqZjrRvtvFdcC3GCRRnnQ==
So when I use PHP's urlencode on the following string, there seems to be a technicality coming up which I think is on a reserved PHP word "¬".
The original string:
cancel_url=https://example.com/payment_cancelled¬ify_url=https://example.com/order_notify
I get the following result using urlencode:
cancel_url=https%3A%2F%2Fexample.com%2Fpayment_cancelled¬ify_url=https%3A%2F%2Fexample.com%2Forder_notify
As you notice above, the '¬' special character it creates (just after the word 'cancelled'). So to me it seems the "¬" portion of "¬ify_url" is an operator reserved operator word ("¬" in PHP?).
I have tried PHP's str_replace function after url encoding as follows:
$paramUrlString = str_replace('¬', '¬', $paramUrlString);
$paramUrlString = str_replace('ª', '¬', $paramUrlString);
(trying the ASCII code for that special character too)
I've run out of ideas now. Please assist, thank you.
urlencode does not usually replace ¬ at all, but does replace & with %26. See example here: http://sandbox.onlinephpfunctions.com/code/e9d62797d01f8162170e5ad5181e14fc339faa52
You could try replacing & with %26 before urlencode.
$urlString = str_replace('&', '%26', $urlString);
It's not that anything in PHP is replacting the string ¬ with ¬, it's that whatever you're using to view/display the data is doing that.
Given that the closing ; on the entity is not required, I would wager that you're putting the URL into XML without properly escaping the entities. While & is the entity that conflicts between URLs and XML, there are more than that.
The simplest solution is if you're embedding a raw string in an XML document you need to call:
$string = htmlspecialchars($string, ENT_XML1 | ENT_COMPAT);
The best solution, on the other hand, is to not create XML documents by hand at all. Use a library like DOMDocument or XMLWriter. This handles not only the escaping/encoding of your data, but all of the other subtle complexities of creatings proper XML documents.
I am doing a post HTTP request in swift 4.2 and in one of my Strings I put in the parameters contain "&" but apparently the requests gets cut off after this symbol. I thought about replacing every "&" symbol with a unique placeholder and convert it back in PHP.
But is there are more elegant or easy way of doing this?
URL encode your data (and decode it when you need to use it), that will make the ampersand into %26 which will stop it cutting off in your GET request.
You could replace the "&" with "%26" and then it's have to work :)
All Precent-encoding characters:
https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters
You should probably minimize how much manual percent escaping you do. You might, for example, use URLComponents to build your URL and percent escape it for you:
guard var components = URLComponents(string: "http://example.com") else { return }
components.queryItems = [URLQueryItem(name: "foo", value: "bar&baz")]
let url = components.url
That will result in:
http://example.com?foo=bar%26baz
The ampersand, as well as a few other characters, need to be encoded if they are within a query parameter otherwise they could be recognized as a delimiter of some sort.
You can encode a string for a query param in Swift like this:
let value = string.addingPercentEncoding(withAllowedCharacters: .urlQueryAllowed)
let urlString = "https://example.com/?query=\(value)"
On the other side, your server will receive the encode param value but will need to decode it.
PHP includes the urlencode() and urldecode() functions, and stift includes the .addingPercentEncoding function.
This means you can replace with the encoded version of the '&' symbol which is '%26', or you can use swift's function
Then when you recieve this value you can use urldecode( $escapedString ), or just replace '%26' with '&', or just pull the values stright from the request with $_GET.
I don't now if this is the place to ask this kind of question so I will give it a try. I was wondering what does the following php user defined function do in the code example below? If someone explain it to me in detail thanks.
function decode_characters($info)
{
$info = mb_convert_encoding($info, "HTML-ENTITIES", "UTF-8");
$info = preg_replace('~^(&([a-zA-Z0-9]);)~',htmlentities('${1}'),$info);
return($info);
}
The function is a little odd. The first function call transforms a string encoded in UTF-8 to an ASCII encoded string where the non-mapped characters are converted to HTML entities (named entities if they exist in HTML 4, otherwise numeric entities). For instance:
echo mb_convert_encoding("foo\"é⌑'&", "HTML-ENTITIES", "UTF-8");
yields
foo"é⌑'&
So this differs from htmlentities in that 1) numerical entities are used in the circumstances given and 2) special characters such as &, " or < are not touched.
The second function call, however, is more strange. It finds if a named entity with only one ASCII alphanumeric character starts the input, and, if so, calls htmlentities on this input (actually it doesn't because the e modifier is not used and the function name is not in a string, so it's executed when the arguments are evaluated). This call has no effect because htmlentities('${1}') is '${1}' and the backreference 1 encompasses the whole match, so, even if the expression matches, there's no substitution.
I used the solution accepted for this question for encrypting by id for example in /index.php?id=3 . The problem is I cannot send the encrypted value as an url, example /index.php?id=dsf13f3343f23/23=. Because sometimes it will have weird characters in the url e.g. notice the = sign in the end
The weird characters in the values passed in the URL should be escaped, using urlencode().
For example, the following portion of code :
echo urlencode('dsf13f3343f23/23=');
would give you :
dsf13f3343f23%2F23%3D
Which works fine, as an URL parameter.
And if you want to build aquery string with several parameters, take a look at the http_build_query() function.
For example :
echo http_build_query(array(
'id' => 'dsf13f3343f23/23=',
'a' => 'plop',
'b' => '$^#test',
));
will give you :
id=dsf13f3343f23%2F23%3D&a=plop&b=%24%5E%40test
This function deals with escaping and concatenating the parameters itself ;-)
Use PHP's urlencode() function to encode the value before you put it into a URL.
string urlencode ( string $str )
This function is convenient when
encoding a string to be used in a
query part of a URL, as a convenient
way to pass variables to the next
page.
This function converts "weird" characters, such as =, into a format safe to put into a URL. You can use it like this:
Header('Location: /index.php?id=' . urlencode($id))
If you use Base64 to encode the binary value for the URL, there is also a variant with URL and filename safe alphabet.
You can use the strtr function to translate one from alphabet to the other:
$base64url = strtr($base64, '+/', '-_');
$base64 = strtr($base64url, '-_', '+/');
So you can use these functions to encode and decode base64url:
function base64url_encode($str) {
return strtr(base64_encode($str), '+/', '-_'));
}
function base64url_decode($base64url) {
return base64_decode(strtr($base64url, '-_', '+/'));
}
See also my answer on What is a good way to produce an short alphanumeric string from a long md5 hash?
There is no use in encrypting parameters.
Send it as is:
/index.php?id=3
nothing wrong with it.