$ip="****"; //Set the TCP IP Address to connect too
$port="8088"; //Set the TCP PORT to connect too
$command="hi"; //Command to run
$req['path'] = $path;
$post = json_encode($req);
//Connect to Server
$socket = stream_socket_client("tcp://{$ip}:{$port}", $errno, $errstr, 30);
if($socket) {
//Start SSL
stream_set_blocking ($socket, true);
stream_socket_enable_crypto ($socket, true, STREAM_CRYPTO_METHOD_SSLv3_CLIENT);
stream_set_blocking ($socket, false);
//Send a command
fwrite($socket, $post);
$buf = null;
//Receive response from server. Loop until the response is finished
while (!feof($socket)) {
$buf .= fread($socket, 20240);
}
//close connection
fclose($socket);
echo "<pre>";
print_r($buf); exit;
//echo our command response
return json_decode($buf);
}
This is my code.
this code is working fine for below 8192 bytes value. but
it can't get the above 8192 bytes what i need to get above this bytes.
because i need get the more bytes of data here
Please provide one example
Thanks in advance
When you write to a network stream, a single call to fwrite() does not necessary writes the whole data.
There is a note at http://php.net/manual/en/function.fwrite.php which says:
Writing to a network stream may end before the whole string is
written. Return value of fwrite() may be checked:
<?php
function fwrite_stream($fp, $string) {
for ($written = 0; $written < strlen($string); $written += $fwrite) {
$fwrite = fwrite($fp, substr($string, $written));
if ($fwrite === false) {
return $written;
}
}
return $written;
}
?>
You may wonder, where does this 8192 number come from.
It seems like this is the default chunk size for the stream.
You can check and change chunk size via stream_set_chunk_size() function
You have set the socket in non-blocking mode:
stream_set_blocking ($socket, false);
In non-blocking mode, you should wait for data availability on the socket before trying to perform read operations. PHP provides stream_select function for this purpose.
So you should whether make the socket blocking, or handle events with stream_select.
Related
I've coded a non-evil, non-spammy IRC bot in PHP, using fsockopen and related functions. It works. However, the problem is that I need to support proxies (preferably SOCKS5, but HTTP is also OK if that is somehow easier, which I doubt). This is not supported by fsockopen.
I've gone through all search results for "PHP fsockopen proxy" and related queries. I know of all the things that don't work, so please don't link to one of them.
The PHP manual page for fsockopen mentions the function stream_socket_client() as
similar but provides a richer set of options, including non-blocking connection and the ability to provide a stream context.
This sounded promising at first, supposedly allowing me to just replace the fsockopen call with stream_socket_client and specify a proxy, maybe via a "stream context"... but it doesn't. Or does it? I'm very confused by the manual.
Please note that it must be a PHP code solution; I cannot pay for "Proxifier" or use any other external software to "wrap around" this.
All the things I've tried seem to always result in me getting a bunch of empty output from the server, and then the socket is forcefully closed. Note that the proxy I'm trying with works when I use HexChat (a normal IRC client), with the same network, so it's not the proxies themselves that are at fault.
As far as I know there is no default option to set a SOCKS or HTTP proxy for fsockopen or stream_socket_client (we could create a context and set a proxy in HTTP options, but that doesn't apply to stream_socket_client). However we can establish a connection manually.
Connecting to HTTP proxies is quite simple:
The client connects to the proxy server and submits a CONNECT request.
The server responds 200 if the request is accepted.
The server then proxies all requests between the client and destination host.
<!- -!>
function connect_to_http_proxy($host, $port, $destination) {
$fp = fsockopen($host, $port, $errno, $errstr);
if ($errno == 0) {
$connect = "CONNECT $destination HTTP/1.1\r\n\r\n";
fwrite($fp, $connect);
$rsp = fread($fp, 1024);
if (preg_match('/^HTTP\/\d\.\d 200/', $rsp) == 1) {
return $fp;
}
echo "Request denied, $rsp\n";
return false;
}
echo "Connection failed, $errno, $errstr\n";
return false;
}
This function returns a file pointer resource if the connection is successful, else FALSE. We can use that resource to communicate with the destination host.
$proxy = "138.204.48.233";
$port = 8080;
$destination = "api.ipify.org:80";
$fp = connect_to_http_proxy($proxy, $port, $destination);
if ($fp) {
fwrite($fp, "GET /?format=json HTTP/1.1\r\nHost: $destination\r\n\r\n");
echo fread($fp, 1024);
fclose($fp);
}
The communication protocol for SOCKS5 proxies is a little more complex:
The client connects to the proxy server and sends (at least) three bytes: The first byte is the SOCKS version, the second is the number of authentication methods, the next byte(s) is the authentication method(s).
The server responds with two bytes, the SOCKS version and the selected authentication method.
The client requests a connection to the destination host. The request contains the SOCKS version, followed by the command (CONNECT in this case), followed by a null byte. The fourth byte specifies the address type, and is followed by the address and port.
The server finally sends ten bytes (or seven or twenty-two, depending on the destination address type). The second byte contains the status and it should be zero, if the request is successful.
The server proxies all requests.
<!- -!>
More details: SOCKS Protocol Version 5.
function connect_to_socks5_proxy($host, $port, $destination) {
$fp = fsockopen($host, $port, $errno, $errstr);
if ($errno == 0) {
fwrite($fp, "\05\01\00");
$rsp = fread($fp, 2);
if ($rsp === "\05\00" ) {
list($host, $port) = explode(":", $destination);
$host = gethostbyname($host); //not required if $host is an IP
$req = "\05\01\00\01" . inet_pton($host) . pack("n", $port);
fwrite($fp, $req);
$rsp = fread($fp, 10);
if ($rsp[1] === "\00") {
return $fp;
}
echo "Request denied, status: " . ord($rsp[1]) . "\n";
return false;
}
echo "Request denied\n";
return false;
}
echo "Connection failed, $errno, $errstr\n";
return false;
}
This function works the same way as connect_to_http_proxy. Although both functions are tested, it would be best to use a library; the code is provided mostly for educational purposes.
SSL support and authentication.
We can't create an SSL connection with fsockopen using the ssl:// or tls:// protocol, because that would attempt to create an SSL connection with the proxy server, not the destination host. But it is possible to enable SSL with stream_socket_enable_crypto and create a secure communication channel with the destination, after the connenection with the proxy server has been established. This requires to disable peer verification, which can be done with stream_socket_client using a custom context. Note that disabling peer verification may be a security issue.
For HTTP proxies we can add authentication with the Proxy-Authenticate header. The value of this header is the authentication type, followed by the username and password, base64 encoded (Basic Authentication).
For SOCKS5 proxies the authentication process is - again - more complex. It seems we have to change the authentication code fron 0x00 (NO AUTHENTICATION REQUIRED) to 0x02 (USERNAME/PASSWORD authentication). It is not clear to me how to create a request with the authentication values, so I can not provide an example.
function connect_to_http_proxy($host, $port, $destination, $creds=null) {
$context = stream_context_create(
['ssl'=> ['verify_peer'=> false, 'verify_peer_name'=> false]]
);
$soc = stream_socket_client(
"tcp://$host:$port", $errno, $errstr, 20,
STREAM_CLIENT_CONNECT, $context
);
if ($errno == 0) {
$auth = $creds ? "Proxy-Authorization: Basic ".base64_encode($creds)."\r\n": "";
$connect = "CONNECT $destination HTTP/1.1\r\n$auth\r\n";
fwrite($soc, $connect);
$rsp = fread($soc, 1024);
if (preg_match('/^HTTP\/\d\.\d 200/', $rsp) == 1) {
return $soc;
}
echo "Request denied, $rsp\n";
return false;
}
echo "Connection failed, $errno, $errstr\n";
return false;
}
$host = "proxy IP";
$port = "proxy port";
$destination = "chat.freenode.net:6697";
$credentials = "user:pass";
$soc = connect_to_http_proxy($host, $port, $destination, $credentials);
if ($soc) {
stream_socket_enable_crypto($soc, true, STREAM_CRYPTO_METHOD_ANY_CLIENT);
fwrite($soc,"USER test\nNICK test\n");
echo fread($soc, 1024);
fclose($soc);
}
I have the following (stripped-down) piece of code:
function curl_request_async($url, $params)
{
foreach ($params as $key => $val) {
$post_params[] = $key.'='.urlencode($val);
}
$post_string = implode('&', $post_params);
$parts=parse_url($url);
$fp = fsockopen($parts['host'],
isset($parts['port'])?$parts['port']:80,
$errno, $errstr, 30);
fwrite($fp, "$type ".$parts['path']." HTTP/1.1\r\n");
fwrite($fp, "Host: ".$parts['host']."\r\n");
fwrite($fp, "Content-Type: application/x-www-form-urlencoded\r\n");
fwrite($fp, "Content-Length: ".strlen($post_string)."\r\n");
fwrite($fp, "Connection: Close\r\n\r\n");
$bytes_written = fwrite($fp, $post_string);
var_dump($bytes_written, strlen($post_string));
// fread($fp, 1);
// fflush($fp);
fclose($fp);
}
The problem with this code is that I found no evidence the request reached the server called. The line var_dump($bytes_written, strlen($post_string)); outputted int(493) int(493), so it should have received all data, yet it didn't.
If I uncomment fread($fp, 1); it works without a problem. That could be working solution, but it doesn't seem to make sense. There has to be a better way!
My question then is two-fold: why does fread($fp, 1); fix my problem and is there a better solution?
your problem is probably that you wrote the server code in PHP, and you dont have ignore_user_abort=true by default (see http://php.net/manual/en/misc.configuration.php#ini.ignore-user-abort ), so when you close the connection, your server stop executing your php code, thus fread(fp,1) fix your problem - connection dont close before php start writing a response
you can use this code to make a server to test if its actually connecting or not -
<?php
error_reporting(E_ALL);
ini_set('display_errors',1);
$sck=socket_create(AF_INET,SOCK_STREAM,SOL_TCP);
if($sck===FALSE){
die('socket_create failed!');
}
if(!socket_set_block($sck)){
die("socket_set_block failed!");
}
if(!socket_bind($sck, '0.0.0.0',1337)){
die("FAILED to bind to port 1337");
}
if(!socket_listen($sck,0)){
die("socket_listen failed!");
}
$fullFile='';
while((print('listening for connections!'.PHP_EOL)) && false!==($conn=socket_accept($sck))){
echo "new connection!".PHP_EOL;
echo "generating crypto iv..";
while(false!==($buffi=socket_recv($conn,$buff,1024,MSG_WAITALL))){
if($buffi===0){
break;//socket_recv's way of
//saying that the connection closed,
//apparently. the docs say it should return
// false, but it doesn't, it just infinitely returns int(0).
// at least on windows 7 x64 sp1.
}
$fullFile.=$buff;
echo "recieved ".strlen($fullFile)." bytes...".PHP_EOL;
$buff='';//do i need to clear it? or wiill recv do it for me?
}
echo "all bytes recieved (i guess, todo, socket_last_error confirm).";
echo PHP_EOL;var_dump($fullFule);
echo "done!".PHP_EOL;
}
die("should never reach this code...");
it will make a netcat-style server on http://127.0.0.1:1337
fread needs two parameters: a resource and a length number of bytes to read.
Right now you are only reading 1 byte. fread($fp, 1);
If you want to read the complete result, loop it until readed completely:
while(!feof($fp)){
echo fread($fp, 128);
}
We're using a system at the moment that takes an incoming JSON request over TCP and responds using JSON too. Currently I've set up my socket like so in PHP:
$socket = fsockopen($host, $port, $errno, $errstr, $timeout);
if(!$socket)
{
fwrite($socket, $jsonLoginRequest); // Authentication JSON
while(json_decode($loginResponse) == false) // We know we have all packets when it's valid JSON.
{
$loginResponse .= fgets($socket, 128);
}
// We are now logged in.
// Now call a test method request
fwrite($socket, $jsonMethodRequest);
while(json_decode($methodResponse) == false) // We know we have all packets when it's valid JSON.
{
$methodResponse .= fgets($socket, 128);
echo $methodResponse; // print response out
}
// Now we have the response to our method request.
fclose($socket);
}
else
{
// error with socket
}
This works at the moment, and the server responds to the method request. However, some methods will respond like this to acknowledge the call, but will also respond later on with the results I'm after. So what I really need is a TCP listener. Could anyone advise how I could write a TCP listener using fsock like I have above?
Thanks
To create a listening socket use the following functions:
socket_create_listen (this one is cool)
socket_accept
I'm not shure if fwrite()/fread() are working with those sockets otherwise you have to use the following functions:
socket_recv
socket_send
Message-loop
I have now written some function to read a single JSON responses with the assumption that multiple responses are separated by CRLF.
Here's how I would do it (assuming your php-script has unlimited execution time):
// ... your code ...
function readJson($socket) {
$readData = true;
$jsonString = '';
while(true) {
$chunk = fgets($socket, 2048);
$jsonString .= $chunk;
if(($json = json_decode($jsonString)) !== false) {
return $json;
} elseif(empty($chunk)) {
// eof
return false;
}
}
}
// ....
// Now call a test method request
fwrite($socket, $jsonMethodRequest);
$execMessageLoop = true;
while($execMessageLoop) {
$response = readJson($socket);
if($response === false) {
$execMessageLoop = false;
} else {
handleMessage($socket, $response);
}
}
function handleMessage($socket, $response) {
// do what you have to do
}
Now you could implement the "handleMessage" function which analyses the response and acts to it.
I have a simple download function in a class that might be dealing with files of many hundreds of megabytes at a time from an Amazon Web Services bucket. The whole file cannot be loaded into memory at once, so it must be streamed directly to a file pointer. This is my understanding as this is the first time I've dealt with this issue and I'm picking things up as I go along.
I've ended up with this, based on a 4 KB file buffer which simple testing showed was a good size:
$fs = fsockopen($host, 80, $errno, $errstr, 30);
if (!$fs) {
$this->writeDebugInfo("FAILED ", $errstr . '(' . $errno . ')');
} else {
$out = "GET $file HTTP/1.1\r\n";
$out .= "Host: $host\r\n";
$out .= "Connection: Close\r\n\r\n";
fwrite($fs, $out);
$fm = fopen ($temp_file_name, "w");
stream_set_timeout($fs, 30);
while(!feof($fs) && ($debug = fgets($fs)) != "\r\n" ); // ignore headers
while(!feof($fs)) {
$contents = fgets($fs, 4096);
fwrite($fm, $contents);
$info = stream_get_meta_data($fs);
if ($info['timed_out']) {
break;
}
}
fclose($fm);
fclose($fs);
if ($info['timed_out']) {
// Delete temp file if fails
unlink($temp_file_name);
$this->writeDebugInfo("FAILED - Connection timed out: ", $temp_file_name);
} else {
// Move temp file if succeeds
$media_file_name = str_replace('temp/', 'media/', $temp_file_name);
rename($temp_file_name, $media_file_name);
$this->writeDebugInfo("SUCCESS: ", $media_file_name);
}
}
In testing it's fine. However I have got into a conversation with someone who is saying that I am not understanding how fgets() and feof() work together, and he's mentioning chunked encoding as a more efficient method.
Is the code generally OK, or am I missing something vital here? What is the benefit that chunked encoding will give me?
Your solution seems fine to me, however I have a few comments.
1) Don't create a HTTP packet yourself, i.e. don't send the HTTP request. Instead use something like CURL. This is more fool proof and will support a wider range of responses the server might reply with. Additionally CURL can be setup to write directly to a file, saving you doing it yourself.
2) Using fgets may be a problem if you are reading binary data. Fgets reads to the end of a line, and with binary data this may corrupt your download. Instead I suggest fread($fs, 4096); which will handle both text and binary data.
2) Chunked encoding is a way for a webserver to send you the response in multiple chunks. I don't think this is very useful to you, however, a better encoding that the webserver might support is the gzip encoding. This would allow the webserver to compress the response on the fly. If you use a library like CURL, it will tell the server it supports gzip, and then automatically decompress it for you.
I hope this helps
Don't deal with sockets, optimize your code and use the cURL library, PHP cURL. Like this:
$url = 'http://'.$host.'/'.$file;
// create a new cURL resource
$fh = fopen ($temp_file_name, "w");
$ch = curl_init();
// set URL and other appropriate options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FILE, $fh);
//curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// grab URL and pass it to the browser
curl_exec($ch);
// close cURL resource, and free up system resources
curl_close($ch);
fclose($fh);
And the final result in case it helps anyone else. I also wrapped the whole thing in a retry loop to decrease the risk of a completely failed download, but it does increase the use of resources:
do {
$fs = fopen('http://' . $host . $file, "rb");
if (!$fs) {
$this->writeDebugInfo("FAILED ", $errstr . '(' . $errno . ')');
} else {
$fm = fopen ($temp_file_name, "w");
stream_set_timeout($fs, 30);
while(!feof($fs)) {
$contents = fread($fs, 4096); // Buffered download
fwrite($fm, $contents);
$info = stream_get_meta_data($fs);
if ($info['timed_out']) {
break;
}
}
fclose($fm);
fclose($fs);
if ($info['timed_out']) {
// Delete temp file if fails
unlink($temp_file_name);
$this->writeDebugInfo("FAILED on attempt " . $download_attempt . " - Connection timed out: ", $temp_file_name);
$download_attempt++;
if ($download_attempt < 5) {
$this->writeDebugInfo("RETRYING: ", $temp_file_name);
}
} else {
// Move temp file if succeeds
$media_file_name = str_replace('temp/', 'media/', $temp_file_name);
rename($temp_file_name, $media_file_name);
$this->newDownload = true;
$this->writeDebugInfo("SUCCESS: ", $media_file_name);
}
}
} while ($download_attempt < 5 && $info['timed_out']);
I am working on a PHP script that makes an API call to a external site. However, if this site is not available or the request times out, I would like my function to return false.
I have found following, but I am not sure on how to implement it on my script, since i use "file_get_contents" to retrieve the content of the external file call.
Limit execution time of an function or command PHP
$fp = fsockopen("www.example.com", 80);
if (!$fp) {
echo "Unable to open\n";
} else {
fwrite($fp, "GET / HTTP/1.0\r\n\r\n");
stream_set_timeout($fp, 2);
$res = fread($fp, 2000);
$info = stream_get_meta_data($fp);
fclose($fp);
if ($info['timed_out']) {
echo 'Connection timed out!';
} else {
echo $res;
}
}
(From: http://php.net/manual/en/function.stream-set-timeout.php)
How would you adress such an issue? Thanks!
I'd recommend using the cURL family of PHP functions. You can then set the timeout using curl_setopt():
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT,2); // two second timeout
This will cause the curl_exec() function to return FALSE after the timeout.
In general, using cURL is better than any of the file reading functions; it's more dependable, has more options and is not regarded as a security threat. Many sysadmins disable remote file reading, so using cURL will make your code more portable and secure.
<?php
$fp = fsockopen("www.example.com", 80);
if (!$fp) {
echo "Unable to open\n";
} else {
stream_set_timeout($fp, 2); // STREAM RESOURCE, NUMBER OF SECONDS TILL TIMEOUT
// GET YOUR FILE CONTENTS
}
?>
From the PHP manual for File_Get_Contents (comments):
<?php
$ctx = stream_context_create(array(
'http' => array(
'timeout' => 1
)
)
);
file_get_contents("http://example.com/", 0, $ctx);
?>
<?php
$fp = fsockopen("www.example.com", 80, $errno, $errstr, 4);
if ($fp) {
stream_set_timeout($fp, 2);
}