Translate Code: C Socket to PHP Socket - php

I looked through past threads on here to find any relevant topics regarding C sockets being translated to PHP sockets, and I've ready extensively on php.net and C tutorials to figure out how to convert some C source code I have into PHP source code involving a remote socket connection.
The source I am going to post is in C. This code is already working and confirmed to work in a compiled .exe that one of my programmers wrote. He doesn't know PHP, which I am creating this new program in that requires this snippet.
The program does this: It creates a socket connection to a remote server/port, sends the filesize of an image that it needs to send to the server, then I am guessing when the server knows the filesize, the program sends the binary image data and the filesize of the image again (like an upload feature to the server via sockets). Then it uses the recv(); function in C to receive the bytes back of a specific length.
Basically it is sending a picture with something encrypted inside of it. The server already has a program running on it on the specified port that is decrypting the image. Then the socket is sending back that decrypted text. I do not have access to the algorithm for decryption otherwise I would not be using sockets obviously.
Here is the C source I was sent, and my subsequent attempts in PHP to translate it properly.
// In the this C code, there is the long size; variable that is used at the bottom but is never initialized. I don't know what to do with it. Some other variables are never used either.
function picBin()
assume curlBinaryData variable pic is filled with binary data from picture download.
pic->currentSize is set to the size of the image
it will return 0 if successful. also char *word in the function's params will be set to the pic's decryption
//bin data structure i use for a quick ghetto download function, just so you know how it looks
struct curlBinaryData{
char *data;
int currentSize;
int maxSize;
};
int picBin(curlBinaryData *pic, char *word, int threadNum,
char *host, unsigned short port)
{
char data1[1000], data2[1000],
temp[1000], printBuf[1000], buffer[1000], *p, *p2;
int num, a, totalBytes;
long size;
char *pSize;
SOCKET sock;
while ((sock = connectSocket(host, port)) == INVALID_SOCKET)
{
sprintf(printBuf, "Could not connect(picBin) %s:%d\n", host, port);
print_ts(printBuf, "red");
//Sleep(15000);
}
buffer[0]='\0';
send(sock, buffer, 1, 0);
pSize=(char *)&(pic->currentSize);
send(sock, pSize, 4, 0);
send(sock, pic->data, pic->currentSize, 0);
totalBytes=0;
do{
if ( (num=recv(sock, data1+totalBytes, 1, 0)) > 0)
totalBytes+=num;
} while( (totalBytes<4) && (num>0) );
pSize=(char *)&size;
if (totalBytes==4){ //got 4 bytes for dword
memcpy(pSize, data1, 4);
if (size==1)
{
totalBytes=0;
do
{
if ( (num=recv(sock, data1+totalBytes, 1, 0)) > 0)
totalBytes+=num;
} while( (totalBytes<4) && (num>0) );
memcpy(pSize, data1, 4);
if (totalBytes==4)
{ //got 4 bytes for dword
totalBytes=0;
for (a=0; ( (a<size) && (num>0) ); a++)
{
if ( (num=recv(sock, data1+totalBytes, 1, 0)) > 0)
totalBytes+=num;
}
if (totalBytes==size)
{
closesocket(sock);
data1[totalBytes]='\0';
strcpy(word, data1);
return 0; //0 on success
}
}
}
}
closesocket(sock);
return -1; //something errord
}
Now here is my PHP code I attempted:
if (($sock = socket_create(AF_INET, SOCK_STREAM, SOL_TCP)) === false)
{
echo "socket_create() failed: reason: " . socket_strerror(socket_last_error()) . "\n";
}
if (socket_connect($sock, $host, $port) === false)
{
echo "socket_connect() failed: reason: " . socket_strerror(socket_last_error($sock)) . "\n";
}
socket_send($sock, '\0', 1, MSG_EOF);
$ci = file_get_contents($picURL);
$ciwrite = fopen('pic.jpg', 'w+');
fwrite($ciwrite, $ci);
fclose($ciwrite);
$picFileSize = filesize('pic.jpg');
socket_send($sock, $picFileSize, 4, MSG_EOF);
socket_send($sock, $ci, $picFileSize, MSG_EOF);
$num = socket_recv($sock, $totalBytes, 1, MSG_DONTWAIT);
print $num; // this doesn't even Print anything to my console when I execute it via CLI
/*do{
if (($num = socket_recv($sock, $totalBytes, 1)) > 0)
{
$totalBytes += $num;
}
} while(($totalBytes < 4) && ($num > 0));*/

buffer[0]='\0';
send(sock, buffer, 1, 0);
socket_send($sock, '\0', 1, MSG_EOF);
Backslash escape sequences other than \' and \\ are not expanded in single quoted strings; thus, '\0' is a string of the two characters \ and 0, and the above socket_send() sends the character \.
MSG_EOF is without effect at best and harmful at worst; better don't use it.
A correct translation is:
socket_send($sock, "\0", 1, 0);
pSize=(char *)&(pic->currentSize);
send(sock, pSize, 4, 0);
socket_send($sock, $picFileSize, 4, MSG_EOF);
The above socket_send() sends the first 4 characters of the ASCII string representation of $picFileSize, because socket_send() expects a string as its second argument and thus the given integer is coerced into a string. To send a 4-byte binary integer:
socket_send($sock, pack("L", $picFileSize), 4, 0);
$num = socket_recv($sock, $totalBytes, 1, MSG_DONTWAIT);
print $num; // this doesn't even Print anything to my console when I execute it via CLI
It's no wonder that you get no data if you DONTWAIT for it.
A working translation for the receive part of the C program is:
$totalBytes = socket_recv($sock, $data1, 4, MSG_WAITALL);
if ($totalBytes==4)
{ //got 4 bytes for dword
$size = unpack("L", $data1);
if ($size[1]==1)
{
$totalBytes = socket_recv($sock, $data1, 4, MSG_WAITALL);
if ($totalBytes==4)
{ //got 4 bytes for dword
$size = unpack("L", $data1);
$totalBytes = socket_recv($sock, $data1, $size[1], MSG_WAITALL);
if ($totalBytes==$size[1])
{
echo $data1, "\n";
}
}
}
}

regarding C sockets being translated to PHP sockets
Sockets are implemented by the OS - not the language. There is no such thing as a C socket nor a PHP socket.
When you invoke socket_send from PHP it converts the message to a string (if it's not already one).
pSize=(char *)&(pic->currentSize);
send(sock, pSize, 4, 0);
WTF? Assuming that pic->currentSize contains a string represenation if the image why is it hard coded to 4 bytes!
But here:
send(sock, pic->data, pic->currentSize, 0);
pic->currentSize must contain an integer value. Shoving (char *) in front of a pointer to an integer (or a long, since that appears to be how 'size' is declared) DOES NOT MAKE IT a string. It's still a pointer. OK a 32 bit pointer will be 4 bytes - but WTF is a pointer being sent over a socket??? Unless there's some weird overloading going on here and this is actually C++ rather than C.
Reading further on it sends the pointer to the remote end then reads back a 4 byte value and considers the operation successful if it matches the pointer....!! AAARRGGHHH! even if it were sending the size, this level of integrity checking is entirely redundant over a Unix or INET socket.
I'd dig some more but it's going to be a waste of time trying to make sense of this C code. Use a network sniffer to work out what's really happenning here; do not use this C code as any sort of model of how to write your own client.

Related

Encoding puzzles with sockets in different languages

I have this below code written in PHP responsible for the server socket, specifically by writing messages to certain sockets:
header('Content-Type: text/html; charset=utf-8');
const PAYLOAD_LENGTH_16 = 126;
const PAYLOAD_LENGTH_63 = 127;
const OPCODE_CONTINUATION = 0;
for ($i = 0; $i < $frameCount; $i++) {
// fetch fin, opcode and buffer length for frame
$fin = $i != $maxFrame ? 0 : self::FIN;
$opcode = $i != 0 ? self::OPCODE_CONTINUATION : $opcode;
$bufferLength = $i != $maxFrame ? $bufferSize : $lastFrameBufferLength;
// set payload length variables for frame
if ($bufferLength <= 125) {
$payloadLength = $bufferLength;
$payloadLengthExtended = '';
$payloadLengthExtendedLength = 0;
}
elseif($bufferLength <= 65535) {
$payloadLength = self::PAYLOAD_LENGTH_16;
$payloadLengthExtended = pack('n', $bufferLength);
$payloadLengthExtendedLength = 2;
} else {
$payloadLength = self::PAYLOAD_LENGTH_63;
$payloadLengthExtended = pack('xxxxN', $bufferLength); // pack 32 bit int, should really be 64 bit int
$payloadLengthExtendedLength = 8;
}
// set frame bytes
$buffer = pack('n', (($fin | $opcode) << 8) | $payloadLength).$payloadLengthExtended.substr($message, $i * $bufferSize, $bufferLength);
And below I have the code in Objective-C responsible for receiving these messages from the socket server:
NSInteger len = 0;
uint8_t buffer[4096];
while ([inputStream hasBytesAvailable]) {
len = [inputStream read:buffer maxLength:sizeof(buffer)];
if (len > 0) {
[self.data appendBytes:buffer length:len];
[self.log insertText:[NSString stringWithFormat:#"Log: Received a message from server:\n\n"]];
NSLog(#"Received a message from server...");
}
}
when all bytes are received I run the following command to turn the data into a file:
[self.data writeToFile:#"dataComes.txt" options:NSDataWritingAtomic error:nil]
The Problem
We will send a large file in JSON format for objective-c, with that he will receive that information and will generate a file called dataComes.txt, I can see the JSON file normally but except for some strange characters such as:
~ or ~Â or â-Û
These strange characters always shows at the beginning of each block messages that Objective-C receives (Yes, the socket server and TCP divide large messages into blocks of messages).
What is the cause of this problem and how it could solve this?
SOLUTION 1: Filtering
I can filter out unwanted characters that may come, but it will also filter out some words that have accentuation:
NSCharacterSet *notAllowedChars = [[NSCharacterSet characterSetWithCharactersInString:#"0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ[]{}:,'"] invertedSet];
NSString *resultString = [[total componentsSeparatedByCharactersInSet:notAllowedChars] componentsJoinedByString:#" "];
SOLUTION 2: Stop using sockets
I have tried many ways to send data to my app, the only one that worked was to send the data separately (a loop of one JSON), but to works I had to put my code (PHP) to sleep using sleep(1) (and I believe this is not good) because if not Objective-C recognizes that this data is a single package.
In this case, or my code have problems, or the programming of socket in objective-c was not very well done and has inconsistencies (bug). What remains for me to do with my connections through normal requests via web server (which I do not think it's a good idea, since I have to do this every 3 seconds in a 5 minute time interval).
SOLUTION 3: FILTERING + UNICODE
On the server side I can filter all special characters and create a specific combination for it example:
Hello é world to Hello /e001/ world
And in my app I can filter this combination and change to the real format....

How to read/write 64-bit unsigned little-endian integers?

pack doesn't support them, so how to read and write 64-bit unsigned little-endian encoded integers?
You could consider 64 bit numbers as an opaque chunk of 8 bytes of binary data and use the usual string manipulation functions to handle them (i.e. substr and the dot operator above all).
Whenever (and if) you need to perform arithmetics on them, you can use a couple of wrapper functions to encode/decode that chunk in its hexadecimal representation (in my implementation I call this intermediate type hex64) and use the external library you prefer to do the real work:
<?php
/* Save as test.php */
function chunk_to_hex64($chunk) {
assert(strlen($chunk) == 8);
return strrev(bin2hex($chunk));
}
function hex64_to_chunk($hex64) {
assert(strlen($hex64) == 16);
return strrev(pack('h*', $hex64));
}
// Test code:
function to_hex64($number) {
$hex64 = base_convert($number, 10, 16);
// Ensure the hex64 is left padded with '0'
return str_pad($hex64, 16, '0');
}
for ($number = 0.0; $number < 18446744073709551615.0; $number += 2932031007403.0) {
$hex64 = to_hex64($number);
$hex64_reencoded = chunk_to_hex64(hex64_to_chunk($hex64));
assert($hex64_reencoded == $hex64, "Result is $hex64_reencoded, expected $hex64");
}
$data = file_get_contents('test.php');
// Skip the last element because it is not 8 bytes
$chunks = array_slice(str_split($data, 8), 0, -1);
foreach ($chunks as $chunk) {
$hex64 = to_hex64($number);
$chunk_reencoded = hex64_to_chunk(chunk_to_hex64($chunk));
assert($chunk_reencoded == $chunk, "Result is $chunk_reencoded, expected $chunk");
}
I wrote a helper class for packing/unpacking 64-bit unsigned ints.
The relevant bit is just two lines:
$ints = unpack("#$offset/Vlo/Vhi", $data);
$sum = Math::add($ints['lo'], Math::mul($ints['hi'], '4294967296'));
(Math::* is a simple wrapper around bcmath)
And packing:
$out .= pack('VV', (int)bcmod($args[$idx],'4294967296'), (int)bcdiv($args[$idx],'4294967296'));
It splits the 64-bit ints into two 32-bit ints, which 64-bit PHP should support :-)

Process socket data with a leading length value

This is a follow-up question about how to process prefixed messages received from a network socket. What I try to do is:
Read the first 4 bytes (which is the $prefix and represents the length of the message)
Check if $prefix has really the size of 4 bytes and if it's an integer
Read the complete $message using the length from $prefix
Check if message has really the size of $prefix bytes
So far I've the following two lines of code:
$prefix = socket_read($socket, 4, PHP_BINARY_READ); //No 1.
//No 2: how to do the checks?
$message = socket_read($socket, $prefix, PHP_BINARY_READ); //No 3.
//No 4: how to do the checks?
How can I do the mentioned checks?
A little side note: all data sent through the network socket connection is in UTF8, little-endian
You can validate the length of the binary string you have received by simply using strlen:
$prefix = socket_read($socket, 4, PHP_BINARY_READ);
if (strlen($prefix) != 4) {
// not 4 bytes long
}
According to your previous question, this binary string represents a 32-bit long. Unpack it as such (with the same format specifier you use when pack-ing it), then fetch the message and use strlen again to validate the length:
$length = current(unpack('l', $prefix));
$message = socket_read($socket, $length, PHP_BINARY_READ);
if (strlen($message) != $length) {
// $message not the size of $length
}

Using zlib filter with a socket pair

For some reason, the zlib.deflate filter doesn't seem to be working with socket pairs generated by stream_socket_pair(). All that can be read from the second socket is the two-byte zlib header, and everything after that is NULL.
Example:
<?php
list($in, $out) = stream_socket_pair(STREAM_PF_UNIX,
STREAM_SOCK_STREAM,
STREAM_IPPROTO_IP);
$params = array('level' => 6, 'window' => 15, 'memory' => 9);
stream_filter_append($in, 'zlib.deflate', STREAM_FILTER_WRITE, $params);
stream_set_blocking($in, 0);
stream_set_blocking($out, 0);
fwrite($in, 'Some big long string.');
$compressed = fread($out, 1024);
var_dump($compressed);
fwrite($in, 'Some big long string, take two.');
$compressed = fread($out, 1024);
var_dump($compressed);
fwrite($in, 'Some big long string - third time is the charm?');
$compressed = fread($out, 1024);
var_dump($compressed);
Output:
string(2) "x�"
string(0) ""
string(0) ""
If I comment out the call to stream_filter_append(), the stream writing/reading functions correctly, with the data being dumped in its entirety all three times, and if I direct the zlib filtered stream into a file instead of through the socket pair, the compressed data is written correctly. So both parts function correctly separately, but not together. Is this a PHP bug that I should report, or an error on my part?
This question is branched from a solution to this related question.
I had worked on the PHP source code and found a fix.
To understand what happens I had traced the code during a
....
for ($i = 0 ; $i < 3 ; $i++) {
fwrite($s[0], ...);
fread($s[1], ...);
fflush($s[0], ...);
fread($s[1], ...);
}
loop and I found that the deflate function is never called with the Z_SYNC_FLUSH flag set because no new data are present into the backets_in brigade.
My fix is to manage the (PSFS_FLAG_FLUSH_INC flag is set AND no iterations are performed on deflate function case) extending the
if (flags & PSFS_FLAG_FLUSH_CLOSE) {
managing FLUSH_INC too:
if (flags & PSFS_FLAG_FLUSH_CLOSE || (flags & PSFS_FLAG_FLUSH_INC && to_be_flushed)) {
This downloadable patch is for debian squeeze version of PHP but the current git version of the file is closer to it so I suppose to port the fix is simply (few lines).
If some side effect arises please contact me.
Looking through the C source code, the problem is that the filter always lets zlib's deflate() function decide how much data to accumulate before producing compressed output. The deflate filter does not create a new data bucket to pass on unless deflate() outputs some data (see line 235) or the PSFS_FLAG_FLUSH_CLOSE flag bit is set (line 250). That's why you only see the header bytes until you close $in; the first call to deflate() outputs the two header bytes, so data->strm.avail_out is 2 and a new bucket is created for these two bytes to pass on.
Note that fflush() does not work because of a known issue with the zlib filter. See: Bug #48725 Support for flushing in zlib stream.
Unfortunately, there does not appear to be a nice work-around to this. I started writing a filter in PHP by extending php_user_filter, but quickly ran into the problem that php_user_filter does not expose the flag bits, only whether flags & PSFS_FLAG_FLUSH_CLOSE (the fourth parameter to the filter() method, a boolean argument commonly named $closing). You would need to modify the C sources yourself to fix Bug #48725. Alternatively, re-write it.
Personally I would consider re-writing it because there seems to be a few eyebrow-raising issues with the code:
status = deflate(&(data->strm), flags & PSFS_FLAG_FLUSH_CLOSE ? Z_FULL_FLUSH : (flags & PSFS_FLAG_FLUSH_INC ? Z_SYNC_FLUSH : Z_NO_FLUSH)); seems odd because when writing, I don't know why flags would be anything other than PSFS_FLAG_NORMAL. Is it possible to write & flush at the same time? In any case, handling the flags should be done outside of the while loop through the "in" bucket brigade, like how PSFS_FLAG_FLUSH_CLOSE is handled outside of this loop.
Line 221, the memcpy to data->strm.next_in seems to ignore the fact that data->strm.avail_in may be non-zero, so the compressed output might skip some data of a write. See, for example, the following text from the zlib manual:
If not all input can be processed (because there is not enough room in the output buffer), next_in and avail_in are updated and processing will resume at this point for the next call of deflate().
In other words, it is possible that avail_in is non-zero.
The if statement on line 235, if (data->strm.avail_out < data->outbuf_len) should probably be if (data->strm.avail_out) or perhaps if (data->strm.avail_out > 2).
I'm not sure why *bytes_consumed = consumed; isn't *bytes_consumed += consumed;. The example streams at http://www.php.net/manual/en/function.stream-filter-register.php all use += to update $consumed.
EDIT: *bytes_consumed = consumed; is correct. The standard filter implementations all use = rather than += to update the size_t value pointed to by the fifth parameter. Also, even though $consumed += ... on the PHP side effectively translates to += on the size_t (see lines 206 and 231 of ext/standard/user_filters.c), the native filter function is called with either a NULL pointer or a pointer to a size_t set to 0 for the fifth argument (see lines 361 and 452 of main/streams/filter.c).
You need to close the stream after the write to flush it before the data will come in from the read.
list($in, $out) = stream_socket_pair(STREAM_PF_UNIX,
STREAM_SOCK_STREAM,
STREAM_IPPROTO_IP);
$params = array('level' => 6, 'window' => 15, 'memory' => 9);
stream_filter_append($out, 'zlib.deflate', STREAM_FILTER_WRITE, $params);
stream_set_blocking($out, 0);
stream_set_blocking($in, 0);
fwrite($out, 'Some big long string.');
fclose($out);
$compressed = fread($in, 1024);
echo "Compressed:" . bin2hex($compressed) . "<br>\n";
list($in, $out) = stream_socket_pair(STREAM_PF_UNIX,
STREAM_SOCK_STREAM,
STREAM_IPPROTO_IP);
$params = array('level' => 6, 'window' => 15, 'memory' => 9);
stream_filter_append($out, 'zlib.deflate', STREAM_FILTER_WRITE, $params);
stream_set_blocking($out, 0);
stream_set_blocking($in, 0);
fwrite($out, 'Some big long string, take two.');
fclose($out);
$compressed = fread($in, 1024);
echo "Compressed:" . bin2hex($compressed) . "<br>\n";
list($in, $out) = stream_socket_pair(STREAM_PF_UNIX,
STREAM_SOCK_STREAM,
STREAM_IPPROTO_IP);
$params = array('level' => 6, 'window' => 15, 'memory' => 9);
stream_filter_append($out, 'zlib.deflate', STREAM_FILTER_WRITE, $params);
stream_set_blocking($out, 0);
stream_set_blocking($in, 0);
fwrite($out, 'Some big long string - third time is the charm?');
fclose($out);
$compressed = fread($in, 1024);
echo "Compressed:" . bin2hex($compressed) . "<br>\n";
That produces:
Compressed:789c0bcecf4d5548ca4c57c8c9cf4b57282e29cacc4bd70300532b079c
Compressed:789c0bcecf4d5548ca4c57c8c9cf4b57282e29cacc4bd7512849cc4e552829cfd70300b1b50b07
Compressed:789c0bcecf4d5548ca4c57c8c9cf4b57282e29ca0452ba0a25199945290a259940c9cc62202f55213923b128d71e008e4c108c
Also I switched the $in and $out because writing to $in confused me.

How to calculate CRC of a WinRAR file header?

Talking about this: http://www.win-rar.com/index.php?id=24&kb_article_id=162
I'm able to calculate the correct CRC of an archive header (MAIN_HEAD) by doing:
$crc = crc32(mb_substr($data, $blockOffset + 2, 11, '8bit'));
$crc = dechex($crc);
$crc = substr($crc, -4, 2) . substr($crc, -2, 2);
$crc = hexdec($crc);
The first line will read "CRC of fields HEAD_TYPE to RESERVED2" as states in the documentation. As I noted, it works fine for the archive header.
When I try to calculate the CRC of a file header it always spits out the wrong CRC for unknown reason. I did as the documentation says - "CRC of fields from HEAD_TYPE to FILEATTR" but it simply doesn't work. I've also tried different read-length variations in case the documentation is wrong and it might actually be from HEAD_TYPE to FILE_NAME. Everything without success.
Anyone can give me a hint? I've also checked the unrar source code but it doesn't make me smarter, probably because I don't know C language at all...
I wrote some code that does the same thing. Here is it with some additional snippets for a better understanding:
$this->fh = $fileHandle;
$this->startOffset = ftell($fileHandle); // current location in the file
// reading basic 7 byte header block
$array = unpack('vheaderCrc/CblockType/vflags/vheaderSize', fread($this->fh, 7));
$this->headerCrc = $array['headerCrc'];
$this->blockType = $array['blockType'];
$this->flags = $array['flags'];
$this->hsize = $array['headerSize'];
$this->addSize = 0; // size of data after the header
// -- check CRC of block header --
$offset = ftell($this->fh);
fseek($this->fh, $this->startOffset + 2, SEEK_SET);
$crcData = fread($this->fh, $this->hsize - 2);
// only the 4 lower order bytes are used
$crc = crc32($crcData) & 0xffff;
// igonore blocks with no CRC set (same as twice the blockType)
if ($crc !== $this->headerCrc && $this->headerCrc !== 0x6969 // SRR Header
&& $this->headerCrc !== 0x6a6a // SRR Stored File
&& $this->headerCrc !== 0x7171 // SRR RAR block
&& $this->blockType !== 0x72 // RAR marker block (fixed: magic number)
) {
array_push($warnings, 'Invalid block header CRC found: header is corrupt.');
}
// set offset back to where we started from
fseek($this->fh, $offset, SEEK_SET);
I tested it on a couple of SRR files and it works as expected. I started with reading the basic 7 byte header. The size of the header can be found there. I used this to grab the correct amount of data for the crc32 function. I noticed that when you convert it to hexadecimal, you can get false positives when comparing: '0f00' != 'f00'. You would need to pad it with zeros. This is why I kept the decimal representations of crc32() and unpack() for the comparison. Also, the number of fields of a file block can vary if some header flags are set: it is possible you took a wrong size.

Categories