Can JavaScript string store 100K characters?
I've written a script where a string from PHP is passed to a variable in JavaScript. It works fine when it is cut short to almost ten thousand characters but breaks the script when attempting to pass the entire string whose length is a bit greater than 100K. No errors could be found though. Is there any solution for this as to any way of increasing character limit of JavaScript variable? I'm just a beginner. Would appreciate is some one could find a solution for this.
The ECMAScript Standard ECMA-262 (6th Edition, June 2015) says
6.1.4 The String Type
The String type is the set of all ordered sequences of zero or more 16-bit unsigned integer values ("elements") up to a maximum length of 253-1 elements.
So don't plan on using more than 9,007,199,254,740,991 or about 9 quadrillion characters. Of course, you should be prepared for systems which cannot allocate 18 PB chunks of memory, as this is not required for conforming ECMAScript implementations.
I think the question is asking about the practical limit, not the spec limit. And, no, it is not always the amount of RAM you have. I have x86_64 24GB PC running Linux Mint with x86_64 Firefox and x86_64 Chrome, and the limits I ran into were:
1,073,741,822 limit in Firefox 84
536,870,888 limit in Chrome 87
Any higher and Firefox throws a Uncaught RangeError: repeat count must be less than infinity and not overflow maximum string size, whereas Chrome throws Uncaught RangeError: Invalid string length. Use the following snippet to run a binary search for the max string length in your browser:
for (var startPow2 = 1; startPow2 < 9007199254740992; startPow2 *= 2)
try {" ".repeat(startPow2);} catch(e) {
break;
}
var floor = Math.floor, mask = floor(startPow2 / 2);
while (startPow2 = floor(startPow2 / 2))
try {
" ".repeat(mask + startPow2);
mask += startPow2; // the previous statement succeeded
} catch(e) {}
console.log("The max string length for this browser is " + mask);
There is no theorical limit to JS or PHP on the size of their strings.
I think there are a few possible situations.
Firstly, check that you are not sending your string via HTTP GET. There is a maximum size for GET and i think its dependent on your web server.
Secondly, if you do use POST, check in php.ini for post_max_size and see if it is smaller than the string size you are sending to it as well as your .htacccess file to see if php_value post_max_size is not too small.
Thirdly, check that in php.ini that your memory_limit does not restrict the size of memory that your script can use.
hope this helps.
Related
In Playback, Background, Read we can combine audio files with &, like file1&file2.
In PHPAgi instead of Read, I use $agi->get_data, but its look like accept only one file!
I also tried to play audio with Background before calling get_data:
$multipleAudioPath = 'file1&file2';
$agi->exec('Background', $multipleAudioPath);
$rawInput = $agi->get_data('blankAudioFile', $timeout, $digits);
But first character goes for breaking Background audio and didn't catch in get_data.
What can I do? Can I do it alone with $agi->get_data ? Or is there any other solution ?
You can use multiple files using Read asterisk command.
$agi->exec("Read","variable,filename&filename2")
[Syntax]
Read(variable[,filename[&filename2[&...]][,maxdigits[,options[,attempts[,timeout]]]]])
[Arguments]
variable
The input digits will be stored in the given <variable> name.
filename
file(s) to play before reading digits or tone with option i
maxdigits
Maximum acceptable number of digits. Stops reading after <maxdigits> have
been entered (without requiring the user to press the '#' key).
Defaults to '0' - no limit - wait for the user press the '#' key. Any value
below '0' means the same. Max accepted value is '255'.
options
s: to return immediately if the line is not up.
i: to play filename as an indication tone from your "indications.conf".
n: to read digits even if the line is not up.
attempts
If greater than '1', that many <attempts> will be made in the event no data
is entered.
timeout
The number of seconds to wait for a digit response. If greater than '0',
that value will override the default timeout. Can be floating point.
[See Also]
SendDTMF()
Other option is mix it together by SOX or via libsox.
I had an issue with the wordpress bacip plugin to compress the files and folders which had the names which were not utf8 encoded, where in we ended up with an observation that the post_max_size was greater than memory_limit, the solution provider says it is the reason for failure in compression..
Please help me understand the reason behind this, if it is so..
post_max_size is the limit of post when you receive a post request for example 50mb of post and stored in a global variable as $_POST['message'] and you only have memory_limit of 32mb it will error it's because you can only allocate a total of 32mb per process. In this case you are using 50mb+ in single request.
real example if int have a size of 4 byte and you have an array of ints which lenght is 8 you are consuming 32byte in memory_limit
<?php
$intArray = range(1, 8); // array(1, 2, 3, ..., 8) consumes 32bytes
$_POST['message'] = range(1, 8); // consumes 32bytes also
If your input is larger than your space to store it, than PHP will throw a fatal error.
It's akin to someone handing you a textbook and telling you to copy its contents on to a small piece of paper. You don't have enough room; you have to give up.
Creating bzip2 archived data in PHP is very easy thanks to its implementation in bzcompress. In my present application I cannot in all reason simply read the input file into a string and then call bzcompress or bzwrite. The PHP documentation does not make it clear whether successive calls to bzwrite with relatively small amounts of data will yield the same result as when compressing the whole file in one single swoop. I mean something along the lines of
$data = file_get_contents('/path/to/bigfile');
$cdata = bzcompress($data);
I tried out a piecemeal bzcompression using the routines shown below
function makeBZFile($infile,$outfile)
{
$fp = fopen($infile,'r');
$bz = bzopen($outfile,'w');
while (!feof($fp))
{
$bytes = fread($fp,10240);
bzwrite($bz,$bytes);
}
bzclose($bz);
fclose($fp);
}
function unmakeBZFile($infile,$outfile)
{
$bz = bzopen($infile,'r');
while (!feof($bz))
{
$str = bzread($bz,10240);
file_put_contents($outfile,$str,FILE_APPEND);
}
}
set_time_limit(1200);
makeBZFile('/tmp/test.rnd','/tmp/test.bz');
unmakeBZFile('/tmp/test.bz','/tmp/btest.rnd');
To test this code I did two things
I used makeBZFile and unmakeBZFile to compress and then decompress a SQLite database - which is what I need to do eventually.
I created a 50Mb filled with random data dd if=/dev/urandom of='/tmp.test.rnd bs=50M count=1
In both cases I performed a diff original.file decompressed.file and found that the two were identical.
All very nice but it is not clear to me why this is working. The PHP docs state that bzread(bzpointer,length) reads a maximum length bytes of UNCOMPRESSED data. If my code below is woring it is because I am forcing the bzwite and bzread size to 10240 bytes.
What I cannot see is just how bzread knows how to fetch lenth bytes of UNCOMPRESSED data. I checked out the format of a bzip2 file. I cannot see tht there is anything there which helps easily establish the uncompressed data length for a chunk of the .bz file.
I suspect there is a gap in my understanding of how this works - or else the fact that my code below appears to perform a correct piecemeal compression is purely accidental.
I'd much appreciate a few explanations here.
To understand how the decompression get the length of bytes you have to understand first the compression. It seems that you don't know any thing about compression algorigthim.
BZIP2
Crucial algorithm of BZIP2 is the Burrows Wheeler transformation (BWT), that converts the original data into a suitable form for following coding. The current version applies a Huffman code. Compression algorithm processes the data in blocks totally independent from each block. Block sizes can be set in a range from 1-9 (100,000 - 900,000 bytes).
BZIP2 Data Structure
The first two character of compressed string start with letter 'BZ' and thereafter 1 byte for algorigthim used. Thereafter identification of the block size immediately follows, being valid for the entire file (h1, h2, h3 to h9). The parameter indicates the block size in units from 1-9 (100,000 - 900,000 bytes).
Actual original data are stored in blocks according to the selected size and will be protected individually with a CRC32 checksum. Additionally a 48 bit identifier introduces each block. This block structure allows a partial reconstruction of damaged files.
GZIP/BZIP
Gzip and bzip2 are functionally equivalent. One advantage of GZIP is that it can compress a stream, a sequence where you can't look behind. This makes it the official compressor of http streams. GZZIP DEFLATE RFC 1951 Compressed Data Format Specification and GUNZIP RFC 1952 File Format Specification are published documents.
GIP explained
When decompressing with gzinflate, I found that - under certain
circumstances - the following code results in out-of-memory errors. Tested with PHP 5.3.20 on an 32bit Linux (Amazon Linux AMI on EC2).
$memoryLimit = Misc::bytesFromShorthand(ini_get('memory_limit')); // 256MB
$memoryUsage = memory_get_usage(); // 2MB in actual test case
$remaining = $memoryLimit - $memoryUsage;
$factor = 0.9;
$maxUncompressedSize = max(1, floor($factor * $remaining) - 1000);
$uncompressedData = gzinflate($compressedData, $maxUncompressedSize);
Although, I calculated the size of $maxUncompressedSize conservatively, hoping to give gzinflate sufficient memory, I still get:
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 266143484 bytes) in foo.php on line 123
When changing the value of $factor from 0.9 to 0.4, then the error goes away, in this case. In other cases 0.9 is OK.
I wonder:
Is the reason for the error really that gzinflate needs more than double the space of uncompressed data? Is there possibly some other reason? Is $remaining really the remaining memory at disposal to the application?
It is indeed possible. IMHO, the issue lies with memory_get_usage(true).
Using true should give a higher memory usage value, because should take everything into account.
I have just bought a GPS Tracker, it can send SMS to cellphone just fine. It also supports reporting to a server via GPRS.
I have setup the device to contact my own server on port 8123, it's a FreeBSD server and i have checked that i recieve packets on that port.
I successfully have setup a listener server written in PHP, and i can receive data from the device. But how do i convert the partial HEX data to something usefull (ASCII)?
Example data string:
$$^#T^#E Y'^WÿU210104.000,A,5534.4079,N,01146.2510,E,0.00,,170411,,*10|1.0|72|0000á
Unfortunately i don't know how i can copy-paste the HEX parts
Now how do i get the ID part out? I have tried echo hexdec(mb_substr($data, 4, 7));
The data is following this protocol
From the document:
Command format of GPRS packets are as follows:
From server to tracker:
##\r\n
From tracker to server:
$$\r\n
Note:
Do NOT input ‘’ when writing a command.
All multi-byte data complies with the following sequence: High byte prior to low byte.
The size of a GPRS packet (including data) is about 100 bytes
Item Specification
## 2 bytes. It means the header of packet from server to tracker.
It is in ASCII code (Hex code: 0x40)
$$ 2 bytes. It is the header of packet from tracker to server.
It is in ASCII code (Hex code: 0x24)
L 2 bytes. It means the length of the whole packet including
the header and ending character and it is in hex code
ID 7 bytes, ID must be digit and not over 14 digits, the unused byte
will be stuffed by ‘f’ or ‘0xff’. It is in the format of hex code.
For example, if ID is 13612345678, then it will be shown as
follows: 0x13, 0x61, 0x23, 0x45, 0x67, 0x8f, 0xff.
If all 7 bytes are 0xff, it is a broadcasting command. ID is in hex code
command 2 bytes. The command code is in hex code. Please refer to the
command list below.
data Min 0 byte and max 100 bytes. See Annex 1 for description of ‘data’.
checksum 2 bytes. It indicates CRC-CCITT (default is 0xffff) checksum of
all data (not including CRC itself and the ending character).
It is in hex code.
For example: 24 24 00 11 13 61 23 45 67 8f ff 50 00 05 d8 0d 0a
0x05d8 = CRC-CCITT (24 24 00 11 13 61 23 45 67 8f ff 50 00)
\r\n 2 bytes. It is the ending character and in hex code
(0x0d,0x0a in hex code)
Update
With the answer from Anomie, i was able to piece this together
$arr = unpack('H4length/H14id/H4cmd/H4crc/H4end', mb_substr($data, 2, 11) . mb_substr($data, -4));
var_dump($arr);
This will out put something like
array(5) {
["length"]=>
string(4) "0054"
["id"]=>
string(14) "004512345678ff"
["cmd"]=>
string(4) "9955"
["crc"]=>
string(4) "c97e"
["end"]=>
string(4) "0d0a"
}
It sounds like you are needing to convert binary data to integers or strings. The most straightforward way is to use unpack.
For example, to extract the length you already know you can use
$length_bin = substr($string, 2, 2);
To convert it to an integer, you can use something like
$length = unpack('v', $length_bin); $length = $length[1];
The 'v' code will work for the length and the checksum; if you have a number stored as 4 bytes use 'V', and for the ID you can use 'H*' to get it as a string of hex digits. Other codes are listed in the documentation.
A somewhat less straightforward way is to do the bit manipulation manually, after using unpack with C* to get an array of all the byte values. For example,
$bytes = unpack('C*', $length_bin);
$length = ($bytes[0] << 8) | $bytes[1];
You need to know the format of the messages you are going to receive from the device. You can get this info from the manufacturer. Then, depending on that, you have to create a proper listener in the server side.
I've been working with several devices like that and normally you have to create a process in the server listening to the port with a Socket (or similar). You may have an authentication process also to differentiate between devices (you can have more than one). After that, you simply get the data from the device, you parse it and you store it. Depending on the device you can also send requests or configurations.
Hope this helps
*Edit 26 April:* I have changed the question a bit, thus this seems out of place. Initial question was more on how to read the data from TCP.
I found some great articles on writing a TCP/socket server in PHP (/me slaps PHP around a bit with a large trout)
http://devzone.zend.com/article/1086
http://kevin.vanzonneveld.net/techblog/article/create_daemons_in_php/
Can't wait to get this going :)