I have an IP-camera that on trigger uploads JPG picture to my apache-php web server.
camera sends POST request with multipart data separated with boundaries, and it is ok in most cases.
but sometimes (every ~5th picture) camera sends wrong content-length (very big number, 139TB). that is some bug in camera software. $_REQUEST and $_FILES arrays are empty when this problem occurs.
except wrong content-length, everything else is fine inside request,
i listen-ed it with NETCAT and parsed picture well, it was ok (14KB only)
so if i can just tell php (php.ini or something) to ignore content-length and read post vars by boundary, everything will be fine.
is there such switch for php?
update:
after analyse, it seems that in case of this camera,
wrongContentLength is always == correctLength + 139736760975360
== correctLength + 0x00007f1700000000
which is maybe caused by some 32bit - 64bit mixup.
But only way to ignore the problem is manualy parsing php://input which is readable when problem ocurres ...reading boundary from getallheaders(), and parsing file-data between two of them.
Related
I seem to be stuck at sending the compressed messages from PHP to NodeJS over Amazon SQS.
Over on the PHP side I have:
$SQS->sendMessage(Array(
'QueueUrl' => $queueUrl,
'MessageBody' => 'article',
'MessageAttributes' => Array(
'json' => Array(
'BinaryValue' => bzcompress(json_encode(Array('type'=>'article','data'=>$vijest))),
'DataType' => 'Binary'
)
)
));
NOTE 1: I also tried putting compressed data directly in the message, but the library gave me an error with some invalid byte data
On the Node side, I have:
body = decodeBzip(message.MessageAttributes.json.BinaryValue);
Where message is from sqs.receiveMessage() call and that part works since it worked for raw (uncompressed messages)
What I am getting is TypeError: improper format
I also tried using:
PHP - NODE
gzcompress() - zlib.inflateraw()
gzdeflate() - zlib.inflate()
gzencode() - zlib.gunzip()
And each of those pairs gave me their version of the same error (essentially, input data is wrong)
Given all that I started to suspect that an error is somewhere in message transmission
What am I doing wrong?
EDIT 1: It seems that the error is somewhere in transmission, since bin2hex() in php and .toString('hex') in Node return totally different values. It seems that Amazon SQS API in PHP transfers BinaryAttribute using base64 but Node fails to decode it. I managed to partially decode it by turning off automatic conversion in amazon aws config file and then manually decoding base64 in node but it still was not able to decode it.
EDIT 2: I managed to accomplish the same thing by using base64_encode() on the php side, and sending the base64 as a messageBody (not using MessageAttributes). On the node side I used new Buffer(messageBody,'base64') and then decodeBzip on that. It all works but I would still like to know why MessageAttribute is not working as it should. Current base64 adds overhead and I like to use the services as they are intended, not by work arounds.
This is what all the SQS libraries do under the hood. You can get the php source code of the SQS library and see for yourself. Binary data will always be base64 encoded (when using MessageAttributes or not, does not matter) as a way to satisfy the API requirement of having form-url-encoded messages.
I do not know how long the data in your $vijest is, but I am willing to bet that after zipping and then base64 encoding it will be bigger than before.
So my answer to you would be two parts (plus a third if you are really stubborn):
When looking at the underlying raw API it is absolutely clear that not using MessageAttributes does NOT add additional overhead from base64. Instead, using MessageAttributes adds some slight additional overhead because of the structure of the data enforced by the SQS php library. So not using MessageAttributes is clearly NOT a workaround and you should do it if you want to zip the data yourself and you got it to work that way.
Because of the nature of a http POST request it is a very bad idea to compress your data inside your application. Base64 overhead will likely nullify the compression advantage and you are probably better off sending plain text.
If you absolutely do not believe me or the API spec or the HTTP spec and want to proceed, then I would advise to send a simple short string 'teststring' in the BinaryValue parameter and compare what you sent with what you got. That will make it very easy to understand the transformations the SQS library is doing on the BinaryValue parameter.
gzcompress() would be decoded by zlib.Inflate(). gzdeflate() would be decoded by zlib.InflateRaw(). gzencode() would be decoded by zlib.Gunzip(). So out of the three you listed, two are wrong, but one should work.
I use WordPress 4.1.1.
I tried to install the JSON API plugin.
Strange letters are displayed above the JSON content. And they update after refresh of the page.
I tried to bring another letter under the code of plugin. These letters appeared under these figures, so is the problem in the WordPress system?
Please help me to understand and to remove them, because I can't parse my JSON.
On localhost it works fine with the same properties and data...
The letter are: 7b00c, 78709, 6eb3d... and they change with updates..
The strange characters is probably a chunk-size.
Content-Length
When a server-side process sends a response through an HTTP server, the data will typically be stored in a buffer before it is transmitted to the client (browser). If the entire response fits in the buffer in a timely manner, the server will declare the size in a Content-Length: header, and send the response as-is to the client.
Chunked Transfer Coding
If the response does not fit in the buffer, or the server decides to vacate the buffer for other reasons before the full size is known, it will instead send the response in chunks. This is indicated by the Transfer-Encoding: chunked header. Each chunk is preceeded by its length in hexadecimal (followed by a CRLF-pair). The end of the response is indicated by a 0 chunk-size. The exact syntax is detailed below.
Solution
If you are parsing the HTTP response yourself, there are all sorts of intricacies that you need to consider. Chunked encoding is one of them. You need to check for the Transfer-Encoding: chunked header and assemble the response by parsing and stripping out the chunk-size parts.
It's much easier to use a library such as cURL which will handle all the details for you.
One hack to avoid chunks is to send the response using HTTP/1.0 rather than HTTP/1.1. In HTTP/1.0, the length is indicated either by the Content-Length: header, or by closing the connection.
Syntax
This is the syntax for chunked bodies specified in RFC 7230 - "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing" (ABNF notation):
4.1. Chunked Transfer Coding
chunked-body = *chunk
last-chunk
trailer-part
CRLF
chunk = chunk-size [ chunk-ext ] CRLF
chunk-data CRLF
chunk-size = 1*HEXDIG
last-chunk = 1*("0") [ chunk-ext ] CRLF
chunk-data = 1*OCTET ; a sequence of chunk-size octets
trailer-part = *( header-field CRLF )
I have just a question about the way I need to set the Content-Length header using curl in PHP to make multipart/form-data POST HTTP requests.
For example, I have a PHP script which makes a curl POST uploading an image to the server, the content of the HTTP entity body of the request would look like this:
------Boundary123
Content-Disposition: form-data; name="files[]"; filename="image.png"
Content-Type: image/png
... contents of image.png
------Boundary123--
I do something like this:
$bodiEntity = "--".$boundary."\r\n".
'Content-Disposition: form-data; name="files[]"; filename="'.$fileName.'"'."\r\n".
'Content-Type: img/png'."\r\n\r\n".
file_get_contents($fileName)."\r\n". // binary contents of file
"--".$boundary."--\r\n";
But then, which value should I set for the Content-Length header? I know, it is the number of bytes of the entity body, but how should I handle the "\r\n"?
I mean, if I do strlen("\r\n") I get 2 as return value, but I guess it should be 1 (meaning) one byte when dealing with sending multipart/form-data, am I right?
So should I just use strlen() for the strings like boundary string, Content-Disposition, Content-Type and count "\r\n" as a single char and use filesize() to get the filesize of the file in bytes and then sum all the values to get the final value for the Content-Length header? Or should I just count "\r\n" as 2 char as PHP does?
Thanks for the attention!
Count "\r\n" as two bytes. The CRLF pair is two bytes long. The entire body including
- the boundary prefix ("--"),
- boundary,
- the content bytes,
- the interim boundaries and their prefixes,
- the final boundary prefix,
- boundary and boundary post fix,
all should be accounted for in the content-length.
As for the content-length. There is no content-length defined for each file part in a multi-part request as of HTTP 1.1.
However, gross Content-length header should be defined in the beginning of the request.
You may wish to read the RFC 7230 Section 3.3.2. It has been written by one of the principle web and HTTP standards author Roy Thomas Fielding.
But there are numerous other ways in which a file can be sent as a request, it doesn't necessarily need to be bounded in a boundary based multipart request. The other mechanism in POST method itself is raw content type request. The content is transmitted just after the headers end in the request, followed by an empty line (read "\r\n").
Other mechanisms include PUT requests which can also work in the similar fashion. Both of these have differences that separate them wide apart, technically, although I don't think going into that detail has any concern with the question, so I'll just skip it.
My C program needs to transmit a block of zlib compressed data from a C program to a PHP program via CURL library. The compressed data seems to have NULL characters in between. I do a memcpy to send the data to PHP program. But, when the data is received in the PHP program, it seems to retrieve only the data until the first NULL character is noticed.
How do I send the block of memory that has such multiple NULL characters and receive it as a whole data in PHP program?
I compress and transmit the data from C program as below:
ret = compress (out, &dest_len, str, src_len);
sprintf (output, "data=");
char *temp = output;
temp += 5;
memcpy (temp, out, dest_len);
curl_easy_setopt(curl, CURLOPT_URL, "http://localhost/post.php");
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, output);
From the PHP program, if I dump the data as below, I get only the first block of data, until NULL is encountered. I do not have much knowledge in PHP. Please help out.
Thanks,
Alexander
You have to set the size of the data, otherwise curl will use strlen(). From the curl documentation:
http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTPOSTFIELDSIZE
CURLOPT_POSTFIELDSIZE
If you want to post data to the server without letting libcurl do a strlen() to measure the data size, this option must be used. When this option is used you can post fully binary data, which otherwise is likely to fail. If this size is set to -1, the library will use strlen() to get the size.
A string is terminated by the first nul character. You need to transmit a binary block of data. You can use the --data-binary flag to indicate that "all the data should be sent".
I'm trying to mimick an application that sends octet streams to and from a server. The data contained in the body looks like raw bytes, and I'm fairly certain the data being sent for each command is static, so I'm hoping to map the bytes to something more readable in my application. For example, I'll have an array that does: "test" => "&^D^^&#*#dgkel" So I can call "test" and get the real bytes that need to be sent. Trouble is, PHP seems to convert these bytes. I'm not sure if it is an encoding problem or what, but what has been happing is I'll give it some bytes (for example, �ھ����#�qs��������������������X����������������������������) which has a length of 67 I believe, but PHP will say (when I do a var_dump of the HTTP request) that the headers sent contained "Content-Length: 174" or something close to that and the bytes will look like �ھ����#�qs��������������������X����������������������������
So I'm not really sure how to fix this.. Anyone have any ideas? Cheers!
Edit, a little PHP:
$request = new HttpRequest($this->GetMessageURL(), HTTP_METH_POST);
$request->addHeaders($headers);
$request->addRawPostData($buttonMapping[$button]);
$request->send();