It was noted in another question that wrapping the result of a PHP function call in parentheses can somehow convert the result into a fully-fledged expression, such that the following works:
<?php
error_reporting(E_ALL | E_STRICT);
function get_array() {
return array();
}
function foo() {
// return reset(get_array());
// ^ error: "Only variables should be passed by reference"
return reset((get_array()));
// ^ OK
}
foo();
I'm trying to find anything in the documentation to explicitly and unambiguously explain what is happening here. Unlike in C++, I don't know enough about the PHP grammar and its treatment of statements/expressions to derive it myself.
Is there anything hidden in the documentation regarding this behaviour? If not, can somebody else explain it without resorting to supposition?
Update
I first found this EBNF purporting to represent the PHP grammar, and tried to decode my scripts myself, but eventually gave up.
Then, using phc to generate a .dot file of the two foo() variants, I produced AST images for both scripts using the following commands:
$ yum install phc graphviz
$ phc --dump-ast-dot test1.php > test1.dot
$ dot -Tpng test1.dot > test1.png
$ phc --dump-ast-dot test2.php > test2.dot
$ dot -Tpng test2.dot > test2.png
In both cases the result was exactly the same:
This behavior could be classified as bug, so you should definitely not rely on it.
The (simplified) conditions for the message not to be thrown on a function call are as follows (see the definition of the opcode ZEND_SEND_VAR_NO_REF):
the argument is not a function call (or if it is, it returns by reference), and
the argument is either a reference or it has reference count 1 (if it has reference count 1, it's turned into a reference).
Let's analyze these in more detail.
First point is true (not a function call)
Due to the additional parentheses, PHP no longer detects that the argument is a function call.
When parsing a non empty function argument list there are three possibilities for PHP:
An expr_without_variable
A variable
(A & followed by a variable, for the removed call-time pass by reference feature)
When writing just get_array() PHP sees this as a variable.
(get_array()) on the other hand does not qualify as a variable. It is an expr_without_variable.
This ultimately affects the way the code compiles, namely the extended value of the opcode SEND_VAR_NO_REF will no longer include the flag ZEND_ARG_SEND_FUNCTION, which is the way the function call is detected in the opcode implementation.
Second point is true (the reference count is 1)
At several points, the Zend Engine allows non-references with reference count 1 where references are expected. These details should not be exposed to the user, but unfortunately they are here.
In your example you're returning an array that's not referenced from anywhere else. If it were, you would still get the message, i.e. this second point would not be true.
So the following very similar example does not work:
<?php
$a = array();
function get_array() {
return $GLOBALS['a'];
}
return reset((get_array()));
A) To understand what's happening here, one needs to understand PHP's handling of values/variables and references (PDF, 1.2MB). As stated throughout the documentation: "references are not pointers"; and you can only return variables by reference from a function - nothing else.
In my opinion, that means, any function in PHP will return a reference. But some functions (built in PHP) require values/variables as arguments. Now, if you are nesting function-calls, the inner one returns a reference, while the outer one expects a value. This leads to the 'famous' E_STRICT-error "Only variables should be passed by reference".
$fileName = 'example.txt';
$fileExtension = array_pop(explode('.', $fileName));
// will result in Error 2048: Only variables should be passed by reference in…
B) I found a line in the PHP-syntax description linked in the question.
expr_without_variable = "(" expr ")"
In combination with this sentence from the documentation: "In PHP, almost anything you write is an expression. The simplest yet most accurate way to define an expression is 'anything that has a value'.", this leads me to the conclusion that even (5) is an expression in PHP, which evaluates to an integer with the value 5.
(As $a = 5 is not only an assignment but also an expression, which evalutes to 5.)
Conclusion
If you pass a reference to the expression (...), this expression will return a value, which then may be passed as argument to the outer function. If that (my line of thought) is true, the following two lines should work equivalently:
// what I've used over years: (spaces only added for readability)
$fileExtension = array_pop( ( explode('.', $fileName) ) );
// vs
$fileExtension = array_pop( $tmp = explode('.', $fileName) );
See also PHP 5.0.5: Fatal error: Only variables can be passed by reference; 13.09.2005
Related
Is there any way to easily fix this issue or do I really need to rewrite all the legacy code?
PHP Fatal error: Call-time pass-by-reference has been removed in ... on line 30
This happens everywhere as variables are passed into functions as references throughout the code.
You should be denoting the call by reference in the function definition, not the actual call. Since PHP started showing the deprecation errors in version 5.3, I would say it would be a good idea to rewrite the code.
From the documentation:
There is no reference sign on a function call - only on function definitions. Function definitions alone are enough to correctly pass the argument by reference. As of PHP 5.3.0, you will get a warning saying that "call-time pass-by-reference" is deprecated when you use & in foo(&$a);.
For example, instead of using:
// Wrong way!
myFunc(&$arg); # Deprecated pass-by-reference argument
function myFunc($arg) { }
Use:
// Right way!
myFunc($var); # pass-by-value argument
function myFunc(&$arg) { }
For anyone who, like me, reads this because they need to update a giant legacy project to 5.6: as the answers here point out, there is no quick fix: you really do need to find each occurrence of the problem manually, and fix it.
The most convenient way I found to find all problematic lines in a project (short of using a full-blown static code analyzer, which is very accurate but I don't know any that take you to the correct position in the editor right away) was using Visual Studio Code, which has a nice PHP linter built in, and its search feature which allows searching by Regex. (Of course, you can use any IDE/Code editor for this that does PHP linting and Regex searches.)
Using this regex:
^(?!.*function).*(\&\$)
it is possible to search project-wide for the occurrence of &$ only in lines that are not a function definition.
This still turns up a lot of false positives, but it does make the job easier.
VSCode's search results browser makes walking through and finding the offending lines super easy: you just click through each result, and look out for those that the linter underlines red. Those you need to fix.
PHP and references are somewhat unintuitive. If used appropriately references in the right places can provide large performance improvements or avoid very ugly workarounds and unusual code.
The following will produce an error:
function f(&$v){$v = true;}
f(&$v);
function f($v){$v = true;}
f(&$v);
None of these have to fail as they could follow the rules below but have no doubt been removed or disabled to prevent a lot of legacy confusion.
If they did work, both involve a redundant conversion to reference and the second also involves a redundant conversion back to a scoped contained variable.
The second one used to be possible allowing a reference to be passed to code that wasn't intended to work with references. This is extremely ugly for maintainability.
This will do nothing:
function f($v){$v = true;}
$r = &$v;
f($r);
More specifically, it turns the reference back into a normal variable as you have not asked for a reference.
This will work:
function f(&$v){$v = true;}
f($v);
This sees that you are passing a non-reference but want a reference so turns it into a reference.
What this means is that you can't pass a reference to a function where a reference is not explicitly asked for making it one of the few areas where PHP is strict on passing types or in this case more of a meta type.
If you need more dynamic behaviour this will work:
function f(&$v){$v = true;}
$v = array(false,false,false);
$r = &$v[1];
f($r);
Here it sees that you want a reference and already have a reference so leaves it alone. It may also chain the reference but I doubt this.
Is there any way to easily fix this issue or do I really need to rewrite all the legacy code?
PHP Fatal error: Call-time pass-by-reference has been removed in ... on line 30
This happens everywhere as variables are passed into functions as references throughout the code.
You should be denoting the call by reference in the function definition, not the actual call. Since PHP started showing the deprecation errors in version 5.3, I would say it would be a good idea to rewrite the code.
From the documentation:
There is no reference sign on a function call - only on function definitions. Function definitions alone are enough to correctly pass the argument by reference. As of PHP 5.3.0, you will get a warning saying that "call-time pass-by-reference" is deprecated when you use & in foo(&$a);.
For example, instead of using:
// Wrong way!
myFunc(&$arg); # Deprecated pass-by-reference argument
function myFunc($arg) { }
Use:
// Right way!
myFunc($var); # pass-by-value argument
function myFunc(&$arg) { }
For anyone who, like me, reads this because they need to update a giant legacy project to 5.6: as the answers here point out, there is no quick fix: you really do need to find each occurrence of the problem manually, and fix it.
The most convenient way I found to find all problematic lines in a project (short of using a full-blown static code analyzer, which is very accurate but I don't know any that take you to the correct position in the editor right away) was using Visual Studio Code, which has a nice PHP linter built in, and its search feature which allows searching by Regex. (Of course, you can use any IDE/Code editor for this that does PHP linting and Regex searches.)
Using this regex:
^(?!.*function).*(\&\$)
it is possible to search project-wide for the occurrence of &$ only in lines that are not a function definition.
This still turns up a lot of false positives, but it does make the job easier.
VSCode's search results browser makes walking through and finding the offending lines super easy: you just click through each result, and look out for those that the linter underlines red. Those you need to fix.
PHP and references are somewhat unintuitive. If used appropriately references in the right places can provide large performance improvements or avoid very ugly workarounds and unusual code.
The following will produce an error:
function f(&$v){$v = true;}
f(&$v);
function f($v){$v = true;}
f(&$v);
None of these have to fail as they could follow the rules below but have no doubt been removed or disabled to prevent a lot of legacy confusion.
If they did work, both involve a redundant conversion to reference and the second also involves a redundant conversion back to a scoped contained variable.
The second one used to be possible allowing a reference to be passed to code that wasn't intended to work with references. This is extremely ugly for maintainability.
This will do nothing:
function f($v){$v = true;}
$r = &$v;
f($r);
More specifically, it turns the reference back into a normal variable as you have not asked for a reference.
This will work:
function f(&$v){$v = true;}
f($v);
This sees that you are passing a non-reference but want a reference so turns it into a reference.
What this means is that you can't pass a reference to a function where a reference is not explicitly asked for making it one of the few areas where PHP is strict on passing types or in this case more of a meta type.
If you need more dynamic behaviour this will work:
function f(&$v){$v = true;}
$v = array(false,false,false);
$r = &$v[1];
f($r);
Here it sees that you want a reference and already have a reference so leaves it alone. It may also chain the reference but I doubt this.
In PHP, objects are effectively passed by reference (what’s going on under the hood is a bit more complicated). Meanwhile, parameters to call_user_func() not passed by reference.
So what happens with a piece of code like this?
class Example {
function RunEvent($event) {
if (isset($this->events[$event])) {
foreach ($this->events[$event] as $k => $v) {
//call_user_func($v, &$this);
// The above line is working code on PHP 5.3.3, but
// throws a parse error on PHP 5.5.3.
call_user_func($v, $this);
}
}
}
}
$e = new Example;
$e->events['example'][] = 'with_ref';
$e->events['example'][] = 'without_ref';
$e->RunEvent('example');
function with_ref(&$e) {
$e->with_ref = true;
}
function without_ref($e) {
$e->without_ref = true;
}
header('Content-Type: text/plain');
print_r($e);
Output:
Example Object
(
[events] => Array
(
[example] => Array
(
[0] => with_ref
[1] => without_ref
)
)
[without_ref] => 1
)
Note: Adding error_reporting(E_ALL); or error_reporting(-1); to the top of the file makes no difference. I’m seeing no errors or warnings, and of course php -l on the command line shows no errors.
I was actually expecting it to work both with and without references in the callback functions. I thought that removing the ampersand before $this in call_user_func() would be enough to fix this for the latest version of PHP. Obviously, the version with the reference doesn’t work, but equally it doesn’t throw any linting errors, so it’s hard to track down such situations (which may occur many times in the codebase I’m working with).
I’ve got a practical question here: Is there any way to make the with_ref() function work? I’d like to modify only the RunEvent() code, not every single function which uses it (the majority of which do use references).
I’ve also got a curiosity question, as the behaviour I see here makes no sense to me. The opposite would make more sense. What’s actually going on here? It seems startlingly counter-intuitive that a function call without an ampersand can modify the object, while one with the ampersand cannot.
Parameters passing
The main issue is - that parameters, passed to call_user_func() will be passed as values - so they will be copy of actual data. This behavior overrides the fact, that
objects are passed by reference. Note:
Note that the parameters for call_user_func() are not passed by
reference.
Tracking error
You're not fully correct about "silent agreement" in such cases. You will see error with level E_WARNING in such cases:
Warning: Parameter 1 to with_ref() expected to be a reference, value given in
So - you will be able to figure out that you're mixing reference and values passing
Fixing the issue
Fortunately, it's not too hard to avoid this problem. Simply create reference to desired value:
class Example {
function RunEvent($event) {
if (isset($this->events[$event])) {
foreach ($this->events[$event] as $k => $v) {
$obj = &$this;
call_user_func($v, $obj);
}
}
}
}
-then result will be quite as expected:
object(Example)#1 (3) {
["events"]=>
array(1) {
["example"]=>
array(2) {
[0]=>
string(8) "with_ref"
[1]=>
string(11) "without_ref"
}
}
["with_ref"]=>
bool(true)
["without_ref"]=>
bool(true)
}
Obviously, the version with the reference doesn’t work, but equally it doesn’t throw any linting errors, so it’s hard to track down such situations (which may occur many times in the codebase I’m working with).
It throws an error: Warning: Parameter 1 to with_ref() expected to be a reference....
Error_reporting while developing should be error_reporting(-1);.
I’ve got a practical question here: Is there any way to make the with_ref() function work? I’d like to modify only the RunEvent() code, not every single function which uses it (the majority of which do use references).
You can replace call_user_func($v, $this); with $v($this);.
I’ve also got a curiosity question, as the behaviour I see here makes no sense to me. The opposite would make more sense. What’s actually going on here?
call_user_func can only pass parameters by value, not by reference.
Why does the error "expected to be a reference, value given" appear?
Here’s the modified code (taken from the accepted answer) and comment (taken from the other answers and a comment on the accepted answer). The answer to part 2 of this question is buried in a comment on the accepted answer. If the whole answer were in one place I’d just accept it and have done with it, but since I’ve stitched it together I’m throwing it in here.
function RunEvent($event) {
if (isset($this->events[$event])) {
foreach ($this->events[$event] as $v) {
$obj = &$this;
call_user_func($v, $obj);
// The user func *should* receive the object by value, not
// by reference. What's *actually* passed is a pointer to
// the location of the object, so modifications to the object
// in that func will actually be applied to the real object.
// In that case, a simple call_user_func($v, $this) will
// work.
//
// However, some of the existing user funcs receive the
// object by reference. That can and should be changed,
// but there are quite a lot to track down, and they don't
// throw linting errors. In that case, call_user_func will
// pass by value, and they're expecting to recive it by
// reference, so you get a run-time warning. (In theory,
// anyway. When I was practising on a standalone script
// I saw no warnings at all.) That's not good.
//
// One way around this would be to use $v($this) instead
// of call_user_func, but the user func may sometimes be
// a class method, so that's not going to work either. Instead,
// the above compromise method seems to work without problems.
//
// We may at some future point switch to call_user_func($v, $this),
// and track down the remaining warnings as we hit them.
//
// https://stackoverflow.com/q/20683779/209139
}
}
}
First of all, "objects" are not "passed by reference" or "effectively passed by reference" (that's a made-up term). "Objects" are not values in PHP5, and cannot be "passed" at all. The value of $e, $this, etc., is a pointer to an object.
In PHP, things are passed by reference when there is a &, and passed by value otherwise. Always.
call_by_func is just a function. In its declaration, its parameter does not have a &. Therefore, that parameter is pass-by-value. Therefore, what you passed to call_by_func is always passed by value.
You were using "call-time pass-by-reference" in PHP 5.3, which overrode a pass-by-value parameter into pass-by-reference, and was really bad practice and was removed in PHP 5.4.
Neither the functions with_ref and without_ref assign to their parameter ($e), so actually there was no point to pass it by reference. But since you declared the parameter to with_ref as pass-by-reference, there is a problem when using it with call_user_func, because as we discussed before, call_user_func takes its parameter by value, so it already lost the "reference", so there's no way it can pass-by-reference to your function. The documentation of call_user_func says it results in a warning and the call returns FALSE.
One solution, of course, it just to use $v($this);. i.e. not use call_user_func at all -- just use the name to call it directly. There is rarely a need to use call_user_func anyway.
If you must use the call_user_func* family of functions, you can use call_user_func_array with an array with elements that are by reference (Remember that you can put references into an array). That preserves the "reference" so that it can be passed to the pass-by-reference function:
call_user_func_array($v, array(&$this));
Note: Before PHP 5.4, this used to do call-time pass-by-reference. However, since PHP 5.4, this is not a call-time pass-by-reference. When we use it to call by reference a function that was meant to be called by reference, it works. When we use it to call a pass-by-value function, it works as pass-by-value.
Is there any way to easily fix this issue or do I really need to rewrite all the legacy code?
PHP Fatal error: Call-time pass-by-reference has been removed in ... on line 30
This happens everywhere as variables are passed into functions as references throughout the code.
You should be denoting the call by reference in the function definition, not the actual call. Since PHP started showing the deprecation errors in version 5.3, I would say it would be a good idea to rewrite the code.
From the documentation:
There is no reference sign on a function call - only on function definitions. Function definitions alone are enough to correctly pass the argument by reference. As of PHP 5.3.0, you will get a warning saying that "call-time pass-by-reference" is deprecated when you use & in foo(&$a);.
For example, instead of using:
// Wrong way!
myFunc(&$arg); # Deprecated pass-by-reference argument
function myFunc($arg) { }
Use:
// Right way!
myFunc($var); # pass-by-value argument
function myFunc(&$arg) { }
For anyone who, like me, reads this because they need to update a giant legacy project to 5.6: as the answers here point out, there is no quick fix: you really do need to find each occurrence of the problem manually, and fix it.
The most convenient way I found to find all problematic lines in a project (short of using a full-blown static code analyzer, which is very accurate but I don't know any that take you to the correct position in the editor right away) was using Visual Studio Code, which has a nice PHP linter built in, and its search feature which allows searching by Regex. (Of course, you can use any IDE/Code editor for this that does PHP linting and Regex searches.)
Using this regex:
^(?!.*function).*(\&\$)
it is possible to search project-wide for the occurrence of &$ only in lines that are not a function definition.
This still turns up a lot of false positives, but it does make the job easier.
VSCode's search results browser makes walking through and finding the offending lines super easy: you just click through each result, and look out for those that the linter underlines red. Those you need to fix.
PHP and references are somewhat unintuitive. If used appropriately references in the right places can provide large performance improvements or avoid very ugly workarounds and unusual code.
The following will produce an error:
function f(&$v){$v = true;}
f(&$v);
function f($v){$v = true;}
f(&$v);
None of these have to fail as they could follow the rules below but have no doubt been removed or disabled to prevent a lot of legacy confusion.
If they did work, both involve a redundant conversion to reference and the second also involves a redundant conversion back to a scoped contained variable.
The second one used to be possible allowing a reference to be passed to code that wasn't intended to work with references. This is extremely ugly for maintainability.
This will do nothing:
function f($v){$v = true;}
$r = &$v;
f($r);
More specifically, it turns the reference back into a normal variable as you have not asked for a reference.
This will work:
function f(&$v){$v = true;}
f($v);
This sees that you are passing a non-reference but want a reference so turns it into a reference.
What this means is that you can't pass a reference to a function where a reference is not explicitly asked for making it one of the few areas where PHP is strict on passing types or in this case more of a meta type.
If you need more dynamic behaviour this will work:
function f(&$v){$v = true;}
$v = array(false,false,false);
$r = &$v[1];
f($r);
Here it sees that you want a reference and already have a reference so leaves it alone. It may also chain the reference but I doubt this.
Why don't the function handling functions like call_user_func() support passing parameters by reference?
The docs say terse things like "Note that the parameters for call_user_func() are not passed by reference." I assume the PHP devs had some kind of reason for disabling that capability in this case.
Were they facing a technical limitation? Was it a language design choice? How did this come about?
EDIT:
In order to clarify this, here is an example.
<?php
function more(&$var){ $var++; }
$count = 0;
print "The count is $count.\n";
more($count);
print "The count is $count.\n";
call_user_func('more', $count);
print "The count is $count.\n";
// Output:
// The count is 0.
// The count is 1.
// The count is 1.
This is functioning normally; call_user_func does not pass $count by reference, even though more() declared it as a referenced variable. The call_user_func documentation clearly says that this is the way it's supposed to work.
I am well aware that I can get the effect I need by using call_user_func_array('more', array(&$count)).
The question is: why was call_user_func designed to work this way? The passing by reference documentation says that "Function definitions alone are enough to correctly pass the argument by reference." The behavior of call_user_func is an exception to that. Why?
The answer is embedded deep down in the way references work in PHP's model - not necessarily the implementation, because that can vary a lot, particularly in the 5.x versions. I'm sure you've heard the lines, they're not like C pointers, or C++ references, etc etc... Basically when a variable is assigned or bound, it can happen in two ways - either by value (in which case the new variable is bound to a new 'box' containing a copy of the old value), or by reference (in which case the new variable is bound to the same value box as the old value). This is true whether we're talking about variables, or function arguments, or cells in arrays.
Things start to get a bit hairy when you start passing references into functions - obviously the intent is to be able to modify the original variables. Quite some time ago, call-time pass-by-reference (the ability to pass a reference into a function that wasn't expecting one) got deprecated, because a function that wasn't aware it was dealing with a reference might 'accidentally' modify the input. Taking it to another level, if that function calls a second function, that itself wasn't expecting a reference... then everything ends up getting disconnected. It might work, but it's not guaranteed, and may break in some PHP version.
This is where call_user_func() comes in. Suppose you pass a reference into it (and get the associated the call-time pass-by-reference warning). Then your reference gets bound to a new variable - the parameters of call_user_func() itself. Then when your target function is called, its parameters are not bound where you expect. They're not bound to the original parameters at all. They're bound to the local variables that are in the call_user_func() declaration. call_user_func_array() requires caution too. Putting a reference in an array cell could be trouble - since PHP passes that array with "copy-on-write" semantics, you can't be sure if the array won't get modified underneath you, and the copy won't get detached from the original reference.
The most insightful explanation I've seen (which helped me get my head around references) was in a comment on the PHP 'passing by reference' manual:
http://ca.php.net/manual/en/language.references.pass.php#99549
Basically the logic goes like this. How would you write your own version of call_user_func() ? - and then explain how that breaks with references, and how it fails when you avoid call-time pass-by-reference. In other words, the right way to call functions (specify the value, and let PHP decide from the function declaration whether to pass value or reference) isn't going to work when you use call_user_func() - you're calling two functions deep, the first by value, and the second by reference to the values in the first.
Get your head around this, and you'll have a much deeper understanding of PHP references (and a much greater motivation to steer clear if you can).
See this:
http://hakre.wordpress.com/2011/03/09/call_user_func_array-php-5-3-and-passing-by-reference/
Is it possible to pass parameters by reference using call_user_func_array()?
http://bugs.php.net/bug.php?id=17309&edit=1
Passing references in an array works correctly.
Updated Answer:
You can use:
call_user_func('more', &$count)
to achieve the same effect as:
call_user_func_array('more', array(&$count))
For this reason I believe (unfoundedly) that call_user_func is just a compiler time short cut. (i.e. it gets replaced with the later at compile time)
To give my view on you actual question "Why was call_user_func designed to work this way?":
It probably falls under the same lines as "Why is some methods strstr and other str_replace?, why is array functions haystack, needle and string functions needle, haystack?
Its because PHP was designed, by many different people, over a long period of time, and with no strict standards in place at the time.
Original Answer:
You must make sure you set the variable inside the array to a reference as well.
Try this and take note of the array(&$t) part:
function test(&$t) {
$t++;
echo '$t is '.$t.' inside function'.PHP_EOL;
}
$t = 0;
echo '$t is '.$t.' in global scope'.PHP_EOL;
test($t);
$t++;
echo '$t is '.$t.' in global scope'.PHP_EOL;
call_user_func_array('test', array(&$t));
$t++;
echo '$t is '.$t.' in global scope'.PHP_EOL;
Should output:
$t is 0 in global scope
$t is 1 inside function
$t is 2 in global scope
$t is 3 inside function
$t is 4 in global scope
Another possible way - the by-reference syntax stays the 'right' way:
$data = 'some data';
$func = 'more';
$func($more);
function more(&$data) {
// Do something with $data here...
}