Max Size of value in an array PHP - php
Is there a limit on a value string in php Associative array?
I have a query that returns value of GROUP_CONCAT as CSV for an array element. When I run the query directly, I see all values being returned. When I print the array in PHP, the string is truncated.
Not sure how get around it?
Any ideas?
More info:
I just var_dump the results I got from query. Here it is.
It shows 1024. Looks like there is a limit. What is it?
array(5) {
["FIELD_ID"]=>
string(3) "232"
["answers"]=>
string(1024) "48,50,52,54,56,989,990,991,992,993,994,995,996,997,998,999,1000,1001,1002,1003,1004,1005,1006,1007,1008,1009,1010,1011,1012,1013,1014,1015,1016,1017,1018,1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029,1030,1031,1032,1033,1034,1035,1036,1037,1038,1039,1040,1041,1042,1043,1044,1045,1046,1047,1048,1049,1050,1051,1052,1053,1054,1055,1056,1057,1058,1059,1060,1061,1062,1063,1064,1065,1066,1067,1068,1069,1070,1071,1072,1073,1074,1075,1076,1077,1078,1079,1080,1081,1082,1083,1084,1085,1086,1087,1088,1089,1090,1091,1092,1093,1094,1095,1096,1097,1098,1099,1100,1101,1102,1103,1104,1105,1106,1107,1108,1109,1110,1111,1112,1113,1114,1115,1116,1117,1118,1119,1120,1121,1122,1123,1124,1125,1126,1127,1128,1129,1130,1131,1132,1133,1134,1135,1136,1137,1138,1139,1140,1141,1142,1143,1144,1145,1146,1147,1148,1149,1150,1151,1152,1153,1154,1155,1156,1157,1158,1159,1160,1161,1162,1163,1164,1165,1166,1167,1168,1169,1170,1171,1172,1173,1174,1175,1176,1177,1178,1179,1180,1181,1182,1183,1184,1185,1186,1187,1188,1189,1190,1191,1192,"
["ANSWER_TYPE"]=>
string(2) "SS"
["FIELD_TYPE"]=>
string(2) "DA"
["main_field_name"]=>
string(0) ""
}
More Info:
The issue still exists even after setting the variable group_concat_max_len to 5000. Any other ideas? Any issues with PDO?
After restarting MySQL, it works. Thanks!
You might need to read up on the length of the max concat here.
Command-Line Format --group_concat_max_len=#
Option-File Format group_concat_max_len
Option Sets Variable Yes, group_concat_max_len
Variable Name group_concat_max_len
Variable Scope Global, Session
Dynamic Variable Yes
-- Permitted Values
Platform Bit Size 32
Type numeric
Default 1024
Range 4 .. 4294967295
-- Permitted Values
Platform Bit Size 64
Type numeric
Default 1024
Range 4 .. 18446744073709547520
Edit: I find it rather amusing that the string that is returned to you is 1024 in length - which just happens to be the default max length in mysql. Coincedence?
When you're using GROUP_CONCAT in mysql with the default length limit of 1024, anything above that in your results will get cut off. This happened to me as well.
To determine if this is the issue, try this:
// echo string length to determine what your variable value length is
echo strlen($variable_here);
Then check what the concat limit is set to in mysql by executing this query:
SHOW VARIABLES LIKE 'group_concat_max_len'
I ran it in phpMyAdmin. If you get the same value from strlen and the SHOW VARIABLES query, you'll need to update your my.conf file or ask your host to do it. In WHM/cPanel, it is located in /etc/my.conf and in that file you'll need to have something like this:
[mysqld] #<-- Under that
group_concat_max_len=5120 #<-- add this
I chose 5x1024 to make it 5120 just to not have to deal with it again. Once you've saved the conf file back to the server, you'll need to restart the mysql service. If you're using WHM, you can do that under Home » Restart Services »
SQL Server (MySQL)
Once I did the above, the strlen did in fact echo out 3519 and the value was no longer cut off.
Your question is somewhat tricky to answer becuase:
1. you give very little information on the actual size of your string.
2. You're asking several questions (array size and string size).
You should check out this question and then supply us with extra information like which version of php you're using and also as #FLuffeh says in his comment please show us some code.
Related
Does PHP do any parsing on the php.ini file?
Running PHP Version 7.1.30 under RHEL 7.7. I'm wanting to bump memory_limit, but wasn't sure if I had the syntax right (i.e. 256M or 256MB). So to start with I put a bad value "Hugo" in as the memory_limit setting. The trouble with this is the result of phpinfo() (run under httpd) literally has the string "Hugo" in place, i.e.: So this has me somewhat concerned that PHP doesn't actually do any sanity checking for the value(s). (If the value provided was bad I would expect it to revert to a default, e.g.) Can anyone comment on this - in particular, how do you know whether PHP will be enforcing things (if an arbitary string can be provided).
The confusing thing here is that the setting looks like an integer with some special syntax, but is internally defined as a string. The string is then parsed into a separate global variable whenever the value is changed. Crucially, the result of parsing the string to an integer isn't saved back to the settings table, so when you call phpinfo(), you see the original input, not the parsed value. You can see this in the source: The setting is defined with a callback for when it is changed The callback is fed the raw string value The callback for that particular setting parses the string to an integer using a function called zend_atol, which handles the special suffixes It then calls a function which sets a global variable ("AG" means "Allocation Manager Global Variable", the macro is used to manage thread-safety if that's compiled in) The supported syntax is ultimately defined in zend_atol, which: parses the string for a numeric value, ignoring any additional text looks at the last character of the string, and multiplies the preceding value if it is g, G, m, M, k, or K A value with no digits at the start will be parsed as zero. When setting the global variable, this will set the memory limit to the minimum allowed, based on the constant ZEND_MM_CHUNK_SIZE. You can see the effect by setting the memory limit, then running a loop that quickly allocates a large amount of memory and seeing what comes out in the error message. For instance: # Invalid string; sets to compiled minimum php -r 'ini_set("memory_limit", "HUGO"); while(true) $a[]=$a;' # -> PHP Fatal error: Allowed memory size of 2097152 bytes exhausted # Number followed by a string; takes the number php -r 'ini_set("memory_limit", "4000000 HUGO"); while(true) $a[]=$a;' # -> PHP Fatal error: Allowed memory size of 4000000 bytes exhausted # Number followed by a string, but ending in one of the recognised suffixes # This finds both the number and the suffix, so is equivalent to "4M", i.e. 4MiB php -r 'ini_set("memory_limit", "4 HUGO M"); while(true) $a[]=$a;' # -> PHP Fatal error: Allowed memory size of 4194304 bytes exhausted
First thing first, We first need to understand how PHP.ini work in the way of interpretation workflow. memory_limit is directives for PHP. when using with PHP function you have to do something like this ini_set(‘memory_limit’,’256MB’). So, this function will temporarily set your value to the interpreter variable. If you see closer then you can get the two columns One is for the Local and One is for global. That shows the capability of the values to the individual respectively. But, When you defined for global you need to set as a suffix with K, M, G respectively. If we exceed this value using apache .htaccess it requires the same for the PHP fpm.
PHP Memcached with Binary Protocol — Junk data returned after `increment()`
I've started using the increment() method of the PHP Memcached client, and with that switched to the binary protocol. Apparently, increment() is only supported on the binary protocol. Occasionally, I'm seeing garbage results come back from incremented keys. For example: $memcached = new \Memcached(); $memcached->setOption(\Memcached::OPT_BINARY_PROTOCOL, TRUE); … $this->cache->increment($key,1,1); … $this->cache->get($key); Output: "1\u0000ants1 0 1\r\n1\r\n1\r\n25\r" Given that the key did not exist before it was incremented at first, and an initial value of 1 was given to the increment() call, I'd expect the value returned to be an integer. Instead, the strings returned look like left-over junk, e.g. the ants part of that string has no relevance. Other (possibly) pertinent info: I'm seeing this on a range of different keys Our Memcached server is an AWS Elasticache instance Other clients using the same cache node are not using the binary protocol. All clients are running the same OS (CentOS), PHP and Memcached versions.
tl;dr; This is a bug in the PHP extension code... I dug into the PHP extension code that wraps libmemcached and the libmemcached API code itself, but I think I've found the possible underlying cause of your problem... If you take a look at the PHP Memcached::increment() implementation you'll see on line 1858 of php_memcached.c status = memcached_increment_with_initial(m_obj->memc, key, key_len, (unsigned int)offset, initial, expiry, &value); The problem here is that offset may or may not be 64 bits wide. The libmemcached API tells us that the memcached_increment_with_initial function signature expects a uint64_t for offset whereas here offset is declared long and then cast to unsigned int. So if we were do something like this... $memcached = new memcached; $memcached->addServer('127.0.0.1','11211'); $memcached->setOption(\Memcached::OPT_BINARY_PROTOCOL, TRUE); $memcached->delete('foo'); // remove the key if it already exists $memcached->increment('foo',1,1); var_dump($memcached->get('foo')); You'd see something like... string(22) "8589934592 " as the output from that script. Note this only works if the key foo does not already exist on that memcached server. Also note the length of that string at 22 characters, when clearly it's not supposed to be anywhere near that. If you look at the hex representation of that string.... var_dump(bin2hex($memcached->get('foo'))); The result is clear garbage at the end... string(44) "38353839393334353932000d0a000000000000000000" The object that was being stored was clearly corrupted between the casts. So you may end up getting the same result as me or you may end up getting completely broken data as you have demonstrated above. It depends on how the cast effected the chunk of memory being stored at the time (which is falling into undefined behavior here). Also the only seemingly root cause for this is using an initial value with increment (using increment subsequently after this does not demonstrate that problem or if the key already exists). I guess the problem of this stems from the fact that the libmemcached API has two different size requirements for offset parameter between memcached_increment and memcached_increment_with_initial memcached_increment(memcached_st *ptr, const char *key, size_t key_length, uint32_t offset, uint64_t *value) The former takes uint32_t whereas the later takes uint64_t and PHP's extension code casts both to unsigned int, which would be equivalent to uint32_t pretty much. This discrepency in the width of the offset parameter is likely what causes the key to be corrupted somehow between the calling PHP extension code and the API code.
Treating MySQL's BIT datatype on different OS
I had no problems working with MySQL's BIT data type on a Windows environment with PHP and PDO. The only trick that I used was to cast it to int. But when I uploaded the code to a Linux server, then I wasn't able to see those BIT values: I was getting a square character. After searching for this, I've found this answer by Marc B, which coupled with Stanimir Stoyanov's comment I ended up doing this for both Windows and Linux and it's working great CODE FOR DISPLAY //linux OR windows $display = ($value_from_db === chr(0x01) || $value_from_db == 1) ? 1 : 0; CODE FOR SAVING //both platforms $value_for_db = ($display == 1) ? 0x01 : 0x00; So my questions are: Are both platforms indeed treating this data type differently? Or is it a specific programm issue? (php, apache, mysql) Is this the correct way to address this issue? Maybe there's another approach Windows 7: PHP 5.6.10, Apache 2.4.12, MySQL 5.6.26 Debian Wheezy: PHP 5.5.30, Apache 2.2.22, MySQL 5.6.25
You need to use the PHP functions bindec and decbin: code for display //linux OR windows $display = bindec($value_from_db); Code for saving //both platforms $value_for_db = decbin($display);
BIT is not a single bit, but it is a fixed-width bit-field value that ranges from 1 to 64. The value you store in a column of this datatype is stored as BINARY. Many client libraries, Command Line client and other GUI Clients do not know how to handle this value. It is also transmitted as BINARY value. How it is being displayed depends on the width of column. For example, if column is 32 bit wide, it is treated as CHAR(4) and in case of 8 bit wide, it is treated as CHAR(1). That is why you need to cast it as an INTEGER value in order to display it properly. In case of Microsoft SQL Server, BIT is a single-bit column and stored as a single bit within INTEGER datatype. You can also try BOOL or BOOLEAN datatype for this. These datatypes are similar to TINYINT(1) which can store values other than 0 and 1. TINYINT(1) will consume full byte, but BIT(1) also does the same. According to this page, zero value is considered false and non-zero values are considered true. So, you assign it a value 2, it will be considered true. Therefore, if you want to allow only 0 and 1, then you should use BIT and cast the value to display it properly. According to this page, one should cast BIT field to INT before displaying it. For this, with reference to this link, try following. $bitvalue = ($bitvalue == 0x01) ? 'TRUE' : 'FALSE' OR SELECT CAST(bitfield AS unsigned int)
Searching for hex string in a file in php?
I'm currently using the following two methods in my class to get the job done: function xseek($h,$pos){ rewind($h); if($pos>0) fread($h,$pos); } function find($str){ return $this->startingindex($this->name,$str); } function startingindex($a,$b){ $lim = 1 + filesize($a) - strlen($b)/2; $h = fopen($a,"rb"); rewind($h); for($i=0;$i<$lim;$i++){ $this->xseek($h,$i); if($b==strtoupper(bin2hex(fread($h,strlen($b)/2)))){ fclose($h); return $i; } } fclose($h); return -1; } I realize this is quite inefficient, especially for PHP, but I'm not allowed any other language on my hosting plan. I ran a couple tests, and when the hex string is towards the beginning of the file, it runs quickly and returns the offset. When the hex string isn't found, however, the page hangs for a while. This kills me inside because last time I tested with PHP and had hanging pages, my webhost shut my site down for 24 hours due to too much cpu time. Is there a better way to accomplish this (finding a hex string's offset in a file)? Is there certain aspects of this that could be improved to speed up execution? I would read the entire contents of the file into one hex string and use strrpos, but I was getting errors about maximum memory being exceeded. Would this be a better method if I chopped the file up and searched large pieces with strrpos? edit: To specify, I'm dealing with a settings file for a game. The settings and their values are in a block where there is a 32-bit int before the setting, then the setting, a 32-bit int before the value, and then the value. Both ints represent the lengths of the following strings. For example, if the setting was "test" and the value was "0", it would look like (in hex): 00000004746573740000000130. Now that you mention it, this does seem like a bad way to go about it. What would you recommend? edit 2: I tried a file that was below the maximum memory I'm allowed and tried strrpos, but it was very much slower than the way I've been trying. edit 3: in reply to Charles: What's unknown is the length of the settings block and where it starts. What I do know is what the first and last settings USUALLY are. I've been using these searching methods to find the location of the first and last setting and determine the length of the settings block. I also know where the parent block starts. The settings block is generally no more than 50 bytes into its parent, so I could start the search for the first setting there and limit how far it will search. The problem is that I also need to find the last setting. The length of the settings block is variable and could be any length. I could read the file the way I assume the game does, by reading the size of the setting, reading the setting, reading the size of the value, reading the value, etc. until I reached a byte with value -1, or FF in hex. Would a combination of limiting the search for the first setting and reading the settings properly make this much more efficient?
You have a lot of garbage code. For example, this code is doing nearly nothing: function xseek($h,$pos){ rewind($h); if($pos>0) fread($h,$pos); } because it reads everytime from the begining of the file. Furthemore, why do you need to read something if you are not returning it? May be you looke for fseek()? If you need to find a hex string in binary file, may be better to use something like this: http://pastebin.com/fpDBdsvV (tell me if there some bugs/problems). But, if you are parsing game's settings file, I'd advise you to use fseek(), fread() and unpack() to seek to a place of where setting is, read portion of bytes and unpack it to PHP's variable types.
Long query string variable missing
I've got a pretty long query string variable (value is approximately 600 chars) that I'm trying to reach from a PHP script. The long one is not present in $_REQUEST or $_GET but another variable with shorter value length is. I've shaved off some of it to test and it starts going missing when the value is at 549 chars or more. Fortunately the long variable is present in $_SERVER["QUERY_STRING"] so for now I'm just picking it up from there, but was curious as to if this is a server setting (Apache 2) or something with my PHP setup (5.2.17) or script? BTW, the query string itself is not something I control so unfortunately I can't change how that works, for ex. sending via POST.
Maybe you have a hardend/suhosin version of php. It provides configuration directives to limit the length of names and values for input parameters. suhosin.get.max_name_length Type: Integer Default: 64 Defines the maximum length of variable names for variables registered through the URL. For array variables this is the name in front of the indices. suhosin.get.max_totalname_length Type: Integer Default: 256 Defines the maximum length of the total variable name when registered through the URL. For array variables this includes all indices. suhosin.get.max_value_length Type: Integer Default: 512 Defines the maximum length of a variable that is registered through the URL.
I just ran into this in PHP 5.4 in Debian Wheezy, which does not use suhosin. I finally found this user note buried on the parse_str() page that max_input_vars in php.ini setting controls the number of variables parse_str() will read in this version of PHP. Also note that max_input_vars applies even if you are using the CLI runtime.