With PHP 8 The concept of an unfound array key was promoted from a NOTICE error level to a WARNING error level.
This causes us huge code and error_log bloats where we turn off notices and retain warnings on error logs which end up being filled with repetative warnings about array key existance.
For example; Some code to check if a SESSION value exists and if so then use it in some local code. SESSION value is set in some other part of the website and is dynamically set.
<?php
// This code is server side action example
include whatever_header.php //sets session start etc.
if($_SESSION['action'] === "submit"){
...
}
$localPageDataArray = $_SESSION['core']['data']?:['default' => 'no'];
Previous to PHP 8, if $_SESSION action was not set it didn't matter, because the if condition only executed if it was set to a specific value. To be clear; if $_SESSION['action'] was never set then the code would work in exactly the same way as if $_SESSION['action'] === "sleep" , ie various conditionals would still fire or not fire accordingly.
NOW, the error logs blow up with various warnings such as:
PHP Warning: Undefined array key "action" in /home/public_html/session_test_script.php on line 5
PHP Warning: Undefined array key "data" in /home/public_html/session_test_script.php on line 9
There are a couple of issues with this warning feedback:
It blows up and hugely inflates the error logs
It causes huge code bloat to workaround this issue:
Example of code bloat being to resolve the above warning would be:
<?php
// This code is server side action example
include whatever_header.php //sets session start etc.
if(array_key_exists('action',$_SESSION) && $_SESSION['action'] === "submit"){
...
}
Or even worse:
<?php
// This code is server side action example
include whatever_header.php //sets session start etc.
if(array_key_exists('action',$_SESSION) && $_SESSION['action'] === "submit"){
...
}
$localPageDataArray = ['default' => 'no']; //default value if below IF is false
if(array_key_exists($_SESSION,'core') && is_array($_SESSION['core'])
&& array_key_exists($_SESSION['core'],'data') {
$localPageDataArray = $_SESSION['core']['data']?:['default' => 'no'];
}
This is a huge bloat on the code for absolutely zero benefit aside from removing tedious warning messages from the error logs. Because the issue has been promoted to WARNING level then we can't realistically turn of WARNING logs as that's a bad idea for genuine warnings.
While we can slightly streamline the above to being an isset check thus:
if(isset($_SESSION['core']['data']) && is_array($_SESSION['core']['data'])) {
$localPageDataArray = $_SESSION['core']['data']?:['default' => 'no'];
}
This still greatly inflates the code size for zero practical improvment.
Question
Are there any workarounds to how to easily and efficiently avoid these Warning error log messages without having to deal with multiple lines of hard coding checking processes?
Note that these issues are due to $_SESSION (and sometimes other superglobal data) being dynamically generated so we can't always ensure that a value exists and that's built into the default checking mechanisms as illustrated here.
If we can do the above with one line, this would be a great compromise:
$localPageDataArray = isset($_SESSION['core']['data'])?$_SESSION['core']['data']:['default' => 'no'];
BUT This only sets the default value (['default' => 'no']) if the $_SESSION['core']['data'] is not set, rather than if it is set and is empty
Not a Dupe
This question is not a dupe because I'm not caring about speed (although it's nice isset is so fast) but asking about how to avoid the error_log Warning messages found in PHP 8+ which are a different issue to speed.
With the help of Cid a minimal coded solution would be to use ?? Null coalescing operator and/or isset to check values are set without getting strung up on intermediate array keys in multidimensional arrays.
e.g:
if(isset($_SESSION['action']) && $_SESSION['action'] === "submit"){
...
}
$localPageDataArray = $_SESSION['core']['data']??['default' => 'no'];
Neither of these tests will now produce Warnings about undefined array keys
I came across some strange session behavior today while testing some things out. First off, I've been tasked to create a registration feature for a WordPress website. I've done a fair amount of work in WordPress in the past, but I've never done anything involving any server-side scripting languages.
I was testing out my work environment, which involves making sure that I'm able to connect to MySQL, making sure sessions are working, and basically just familiarizing myself with the setup.
When testing sessions, I did the following:
<?php
$timer = time();
$_SESSION[$timer] = $timer;
print_r($_SESSION);
?>
I was expecting on the first load of the page to see something like this:
Array
(
[1396745563] => 1396745563
)
... which is exactly what happened. On the second refresh, I wanted to see:
Array
(
[1396745563] => 1396745563
[1396745570] => 1396745570
)
... But instead, I saw:
Array
(
[1396745570] => 1396745570
)
Wow! The original session variable for timestamp 1396745563 was gone, and there was a new one in it's place.
My immediate reaction was that maybe a session_start() was missing, but that was not the case. Just to further convince myself that something was weird about this, I altered my code to look like this:
<?php
$timer = time();
$_SESSION['time_' . $timer] = $timer;
print_r($_SESSION);
?>
I thought that perhaps having a session variable with a timestamp for an index had some sort of special behavior (which might be the answer to this question for all I know).
This time, the behavior was exactly as I had expected. Each refresh of the page added on to the array print-out. After a few refreshes of the page, it looked something like this:
Array
(
[time_1396745663] => 1396745663
[time_1396745667] => 1396745667
[time_1396745671] => 1396745671
[time_1396745675] => 1396745675
[1396745570] => 1396745570
)
That oddball variable is the session data that carried over from my original attempt (I'm just being precise).
Can someone explain this extremely odd behavior? I can't think of anything obvious that I could have done that would have caused this.
Note: I doubt that this has anything to do with WordPress; I only included it to give a motive behind doing this.
The keys of $_SESSION adhere to the same rules as valid variable names in PHP. They must begin with a letter or an underscore, followed by any number of letters, numbers or underscores. Therefore, the output of time() can’t serve as a key unless it’s prefixed by at least one letter or underscore.
Add the line error_reporting(E_ALL); to your code and PHP will throw a notice like the following:
Notice: Unknown: Skipping numeric key 1396747512 in Unknown on line 0
I am having a script which analyses XML data and fills same arrays with information.
For some (huge) input, the script crashed.
There is a foreach loop which is run around 180 times without problems (memory_get_usage() in iteration 180 around 20 MB, each loop adds around 0.1 MB)
Then it happens that with each new loop, the memory usage just doubles.
With the use of lots of logging I was able to track the problem down to the following line in a foreach.
$fu = $f['unit']
$f has the following structure:
array (
'name' => 'Test',
'value' => '4',
'unit' => 'min-1',
)
But in some (many) cases (but also before the 180th iteration), the key unit was not existing in the array.
I was able to eliminate the problem by replacing the line with:
$fu = (isset($f['unit']) ? $f['unit'] : '');
Then the iteration runs until finished (totally 370 iterations).
Is there any explanation for the phenomena?
PHP version: PHP 5.3.3-1ubuntu9.10 with Suhosin-Patch (old...)
Your problem might come from the PHP error handler and not from your actual loop.
like you said, not every "unit" key is existing and will therefore rise an error (or Exception depending on your error handlers). This might also include a stack trace and further debugging information depending on what extensions (xdebug?) you installed.
Both will consume memory.
It's allways a good practice to check for variables existance before using it. Allways enable E_NOTICE errors in your development system to see any such problems.
Can someone please confirm that the following is a bug with PHP 5.2.13: (thank you)
<?php
session_start();
if (!is_array($_SESSION["breadcrumb"]["trail"]))
{
$_SESSION["breadcrumb"]["trail"][] = "trail";
}
foreach ($_SESSION["breadcrumb"]["trail"] as $breadcrumb)
{
echo $breadcrumb;
}
?>
The above PHP script crashes the 3rd time it is run. The foreach() loop seems to have an (improper) side effect which wipes out the nested $_SESSION array because the internal variable used in the loop matches the name of the nested $_SESSION array. Simply changing the name of the internal foreach() variable to something different fixes the problem.
Note: clear session variables before running script 3 times.
Again, changing "$breadcrumb" to "$the_breadcrumb" fixes the problem. But the foreach() loop should have no side effects. Note: since the scope of $breadcrumb is not the same as the scope of $_SESSION["breadcrumb"], there should be no collision.
Note that doing a print_r() on the array shows the array as (correctly) empty the first time, (correctly) populated the second time, and erroneously set as "Array ( [breadcrumb] => trail )" the third time (the nested array has been wiped out).
The error in the PHP error log from the 3rd run:
PHP Fatal error: Cannot use string offset as an array on line 5
The issue is not a problem on PHP 5.3 - only PHP 5.2.13. I could not find any note regarding this issue in the PHP changelog on the PHP site (php.net), and I must use 5.2.13 on my live site, so I'm posting here hoping that someone can confirm. I've also posted a bug report on php.net.
Thanks,
Dan Nissenbaum
Expected result:
No PHP 5.2.13 crash on line 5.
Actual result:
PHP 5.2.13 crashes on line 5.
Solved. notJim points out the register_globals php.ini setting. It was set to On. Turn to Off to separate the scope, as expected. Note: register_globals is deprecated as of (at least as far back as) PHP 5.3 - probably further back.
Yes this is a definitely a bug. I changed the variable names in my foreach statements to something different than my session variables in order to get both to work properly. This problem does not occur in php version 5.3.0.
I use 5.2.6 and works greet, no error.
In a PHP program, I sequentially read a bunch of files (with file_get_contents), gzdecode them, json_decode the result, analyze the contents, throw most of it away, and store about 1% in an array.
Unfortunately, with each iteration (I traverse over an array containing the filenames), there seems to be some memory lost (according to memory_get_peak_usage, about 2-10 MB each time). I have double- and triple-checked my code; I am not storing unneeded data in the loop (and the needed data hardly exceeds about 10MB overall), but I am frequently rewriting (actually, strings in an array). Apparently, PHP does not free the memory correctly, thus using more and more RAM until it hits the limit.
Is there any way to do a forced garbage collection? Or, at least, to find out where the memory is used?
it has to do with memory fragmentation.
Consider two strings, concatenated to one string. Each original must remain until the output is created. The output is longer than either input.
Therefore, a new allocation must be made to store the result of such a concatenation. The original strings are freed but they are small blocks of memory.
In a case of 'str1' . 'str2' . 'str3' . 'str4' you have several temps being created at each . -- and none of them fit in the space thats been freed up. The strings are likely not laid out in contiguous memory (that is, each string is, but the various strings are not laid end to end) due to other uses of the memory. So freeing the string creates a problem because the space can't be reused effectively. So you grow with each tmp you create. And you don't re-use anything, ever.
Using the array based implode, you create only 1 output -- exactly the length you require. Performing only 1 additional allocation. So its much more memory efficient and it doesn't suffer from the concatenation fragmentation. Same is true of python. If you need to concatenate strings, more than 1 concatenation should always be array based:
''.join(['str1','str2','str3'])
in python
implode('', array('str1', 'str2', 'str3'))
in PHP
sprintf equivalents are also fine.
The memory reported by memory_get_peak_usage is basically always the "last" bit of memory in the virtual map it had to use. So since its always growing, it reports rapid growth. As each allocation falls "at the end" of the currently used memory block.
In PHP >= 5.3.0, you can call gc_collect_cycles() to force a GC pass.
Note: You need to have zend.enable_gc enabled in your php.ini enabled, or call gc_enable() to activate the circular reference collector.
Found the solution: it was a string concatenation. I was generating the input line by line by concatenating some variables (the output is a CSV file). However, PHP seems not to free the memory used for the old copy of the string, thus effectively clobbering RAM with unused data. Switching to an array-based approach (and imploding it with commas just before fputs-ing it to the outfile) circumvented this behavior.
For some reason - not obvious to me - PHP reported the increased memory usage during json_decode calls, which mislead me to the assumption that the json_decode function was the problem.
There's a way.
I had this problem one day. I was writing from a db query into csv files - always allocated one $row, then reassigned it in the next step. Kept running out of memory. Unsetting $row didn't help; putting an 5MB string into $row first (to avoid fragmentation) didn't help; creating an array of $row-s (loading many rows into it + unsetting the whole thing in every 5000th step) didn't help. But it was not the end, to quote a classic.
When I made a separate function that opened the file, transferred 100.000 lines (just enough not to eat up the whole memory) and closed the file, THEN I made subsequent calls to this function (appending to the existing file), I found that for every function exit, PHP removed the garbage. It was a local-variable-space thing.
TL;DR
When a function exits, it frees all local variables.
If you do the job in smaller portions, like 0 to 1000 in the first function call, then 1001 to 2000 and so on, then every time the function returns, your memory will be regained. Garbage collection is very likely to happen on return from a function. (If it's a relatively slow function eating a lot of memory, we can safely assume it always happens.)
Side note: for reference-passed variables it will obviously not work; a function can only free its inside variables that would be lost anyway on return.
I hope this saves your day as it saved mine!
I've found that PHP's internal memory manager is most-likely to be invoked upon completion of a function. Knowing that, I've refactored code in a loop like so:
while (condition) {
// do
// cool
// stuff
}
to
while (condition) {
do_cool_stuff();
}
function do_cool_stuff() {
// do
// cool
// stuff
}
EDIT
I ran this quick benchmark and did not see an increase in memory usage. This leads me to believe the leak is not in json_decode()
for($x=0;$x<10000000;$x++)
{
do_something_cool();
}
function do_something_cool() {
$json = '{"a":1,"b":2,"c":3,"d":4,"e":5}';
$result = json_decode($json);
echo memory_get_peak_usage() . PHP_EOL;
}
I was going to say that I wouldn't necessarily expect gc_collect_cycles() to solve the problem - since presumably the files are no longer mapped to zvars. But did you check that gc_enable was called before loading any files?
I've noticed that PHP seems to gobble up memory when doing includes - much more than is required for the source and the tokenized file - this may be a similar problem. I'm not saying that this is a bug though.
I believe one workaround would be not to use file_get_contents but rather fopen()....fgets()...fclose() rather than mapping the whole file into memory in one go. But you'd need to try it to confirm.
HTH
C.
Call memory_get_peak_usage() after each statement, and ensure you unset() everything you can. If you are iterating with foreach(), use a referenced variable to avoid making a copy of the original (foreach()).
foreach( $x as &$y)
If PHP is actually leaking memory a forced garbage collection won't make any difference.
There's a good article on PHP memory leaks and their detection at IBM
There recently was a similar issue with System_Daemon. Today I isolated my problem to file_get_contents.
Could you try using fread instead? I think this may solve your problem.
If it does, it's probably time to do a bugreport over at PHP.