How to measure the network bandwidth used between client and server? - php

I have a client which GETs a JSON response from a server. I want to calculate how many requests/responses will consume my allotted transfer allowance from a web hosting company (e.g. 100GB per month).
How do I measure/calculate this?
I assume I only need to measure once because the msgs are of a consistent length and format.
I have control over client/server/network. All can be uniquely dedicated to the task. The client is an IOS App and the server is a PHP REST Web Service (on Windows). Both on my LAN.
I know nothing about this and so far have just got the size of the JSON using strlen(). Is that even heading in the right direction?

I would recommend using Charles Proxy. It is an invaluable tool for debugging all kinds of information exchanged via HTTP. You may use it for tracing all HTTP/s communication from and to your iPhone/iPod/iPad as well as the simulator.
It does unfortunately not work too well with most Android devices as those do not really support configuring a system-wide HTTP proxy. For those cases and non HTTP-based communication, I would recommend using WireShark.
In some rare cases, for reasons that are still unclear to me, Charles rarely fails on iOS devices for HTTP-based connections - a typical case would be GoogleAnalytics. For those, again I would recommend WireShark.

The length of JSON string gives you only the size of the payload field in the transferred network packets. This data field may be encapsulated within an HTTP packet and HTTP packet should be put into an IP packet before transmission. Each of these packets have header fields which contributes to total transmission length.
So, for a precise estimate you should first find the real length of the response packet by using Wireshark or an equivalent tool. If this is the only request type for your application you can divide your bandwidth to the response size of your server application to get maximum number of requests to reach the limit. However, this is usually not the case if you have a web application which has several web pages that are accessible from clients, since any access (browsing) will cause a data transfer from server to client.

Your Apache logs will have the number of bytes of each request, but assuming you want a completely PHP solution, add this to the beginning of your scripts:
<?php
function log_input() {
# get raw post data
$length = strlen(file_get_contents('php://input'));
if (function_exists('apache_request_headers')) {
# quick, but not accurate
$length += strlen(serialize(apache_request_headers()));
} else {
# add fudge for HTTP/1.1 etc
$length += strlen($_SERVER['QUERY_STRING']) + 14;
foreach ($_SERVER as $k => $v) {
if (preg_match('/^HTTP/i', $k)) {
$length += strlen($k) + strlen($v);
}
}
}
$s = sprintf("%s\t%s\n", date('c'), $length);
file_put_contents('/tmp/input.log', $s, FILE_APPEND);
}
function log_output() {
$s = sprintf("%s\t%s\n", date('c'), ob_get_length());
file_put_contents('/tmp/output.log', $s, FILE_APPEND);
}
log_input();
register_shutdown_function('log_output');
ob_start();
?>
<html> ....

Related

C++ WinSock Get Data From PHP Page?

Hello I'm quite new to using sockets and am not that familiar with them yet, Basically all i am trying to do is pass a string variable to a web address (e.g. www.example.com/index.php?Example=StringExample) and then get a response, so for example it would return "Test Example" if index.php looked like this:
<?php
if($_GET['Example'] == "StringExample")
{
echo "Test Example";
}
?>
Here is what I've tried in c++:
struct sockaddr_in SocketAddress;
hostent* addr = gethostbyname("www.example.com/index.php?Example=StringExample");
int sizeofaddr = sizeof(addr);
SocketAddress.sin_addr.s_addr = inet_addr(addr->h_name);
SocketAddress.sin_port = htons(80);
SocketAddress.sin_family = AF_INET;
SOCKET Connection = socket(AF_INET, SOCK_STREAM, NULL);
if (connect(Connection, (SOCKADDR*)&addr, sizeofaddr) != 0)
{
return 0; //Failed to Connect
}
char buffff[256];
recv(Connection, buffff, sizeof(buffff), NULL);
//"Test Example" now stored in buffff
What am i doing wrong?
Btw in my case i would not like to use any libraries like boost or anything like that. Thanks for the help :)
gethostbyname("www.example.com/index.php?Example=StringExample");
"www.example.com/index.php?Example=StringExample" is not a valid server name. This is an entire URL; a server name would be "www.example.com". gethostbyname() takes the name of a server, and not a URL, and returns its IP address. Additionally, gethostbyname() has been obsoleted. New code should use the getaddrinfo(3) function, instead.
This is obviously an HTTP URL. To download a document via HTTP it is a lot more work than just connecting a socket. Establishing a socket connection is just the first step in the process of downloading a document from an HTTP server. This must be followed by sending a valid HTTP request, and then receiving an HTTP response from the server.
There are many libraries, such as curl, that implement the entire client-side process needed to download an HTTP document, that will handle the socket connection themselves.
But there's nothing wrong with trying to implement this yourself, either. It's a good programming excersize.
So, after resolving www.example.com's IP address, you will need to
1) Connect to the server's port 80, the default HTTP port.
2) Send an HTTP request for "/index.php?Example=StringExample".
3) Parse the HTTP response.
The specification for HTTP requests and responses is defined by RFC 2616, which you can consult for complete documentation of how HTTP requests and responses are structured.
If you want to access a web server with sockets, you have to keep in mind:
You can open a tcp/ip connection to your web server
BUT afterwards you have to do the http protocol by yourself
In case of your example:
hostent* addr = gethostbyname("www.example.com");
//...
const char* request = "GET index.html"
send(Connection, request, strlen(request), NULL)
//fetch index.html with a recv and parse it
To be more precise, if you want to access your server, you have to take a look how GET, PUT, POST, etc. are implemented in the http protocol, send the proper commands to your web server and recv() the replies

connect to a server with PHP and see if that server can ping another

I personally don't see how this can be done even though these are my servers. But I want to know if my servers can reach external sites--ping a generic website for example--have outgoing communication. That is, I want to use execute a PHP script on one server, connecting to another of my servers, and test if the second server can ping a website, for example. So I know how to use PHP on the server my script is executing from to ping a website with fopensocket. I just don't know how to set this up to test another server's pingability. I mean I have the credentials but the only way is to have my script on each and every server and then reach the script and execute them. That is not what I want. I want to do this from the one/external server and just feed my script the ip/port/uid/pwd of the server I want to test.
An easy API would look something like:
SERVER1:
// get response from server2
$response = file_get_contents('http://www.server2.com/api.php?method=ping&ip=IP&port=PORT&uid=UID&pwd=PWD');
// do json_decode() if response is json string
SERVER2 (api.php):
// respond to API call
if (isset($_GET['method']) && $_GET['method'] == 'ping') {
// get other params and do your ping function
echo $pingresult; // perhaps a json encoded array
exit;
}
There is no security so you could send an API password or do it with OAuth or HMAC

How to receive a file via HTTP PUT with PHP

This is something that has been bugging me for a while.. I'm building of a RESTful API that has to receive files on some occasions.
When using HTTP POST, we can read data from $_POST and files from $_FILES.
When using HTTP GET, we can read data from $_GET and files from $_FILES.
However, when using HTTP PUT, AFAIK the only way to read data is to use the php://input stream.
All good and well, untill I want to send a file over HTTP PUT. Now the php://input stream doesn't work as expected anymore, since it has a file in there as well.
Here's how I currently read data on a PUT request:
(which works great as long as there are no files posted)
$handle = fopen('php://input', 'r');
$rawData = '';
while ($chunk = fread($handle, 1024)) {
$rawData .= $chunk;
}
parse_str($rawData, $data);
When I then output rawData, it shows
-----ZENDHTTPCLIENT-44cf242ea3173cfa0b97f80c68608c4c
Content-Disposition: form-data; name="image_01"; filename="lorem-ipsum.png"
Content-Type: image/png; charset=binary
�PNG
���...etc etc...
���,
-----ZENDHTTPCLIENT-8e4c65a6678d3ef287a07eb1da6a5380
Content-Disposition: form-data; name="testkey"
testvalue
-----ZENDHTTPCLIENT-8e4c65a6678d3ef287a07eb1da6a5380
Content-Disposition: form-data; name="otherkey"
othervalue
Does anyone know how to properly receive files over HTTP PUT, or how to parse files out of the php://input stream?
===== UPDATE #1 =====
I have tried only the above method, don't really have a clue as to what I can do else.
I have gotten no errors using this method, besides that I don't get the desired result of the posted data and files.
===== UPDATE #2 =====
I'm sending this test request using Zend_Http_Client, as follows:
(haven't had any problems with Zend_Http_Client so far)
$client = new Zend_Http_Client();
$client->setConfig(array(
'strict' => false,
'maxredirects' => 0,
'timeout' => 30)
);
$client->setUri( 'http://...' );
$client->setMethod(Zend_Http_Client::PUT);
$client->setFileUpload( dirname(__FILE__) . '/files/lorem-ipsum.png', 'image_01');
$client->setParameterPost(array('testkey' => 'testvalue', 'otherkey' => 'othervalue');
$client->setHeaders(array(
'api_key' => '...',
'identity' => '...',
'credential' => '...'
));
===== SOLUTION =====
Turns out I made some wrong assumptions, mainly that HTTP PUT would be similar to HTTP POST. As you can read below, DaveRandom explained to me that HTTP PUT is not meant for transferring multiple files on the same request.
I have now moved the transferring of formdata from the body to url querystring. The body now holds the contents of a single file.
For more information, read DaveRandom's answer. It's epic.
The data you show does not depict a valid PUT request body (well, it could, but I highly doubt it). What it shows is a multipart/form-data request body - the MIME type used when uploading files via HTTP POST through an HTML form.
PUT requests should exactly compliment the response to a GET request - they send you the file contents in the message body, and nothing else.
Essentially what I'm saying is that it is not your code to receive the file that is wrong, it is the code that is making the request - the client code is incorrect, not the code you show here (although the parse_str() call is a pointless exercise).
If you explain what the client is (a browser, script on other server, etc) then I can help you take this further. As it is, the appropriate request method for the request body that you depict is POST, not PUT.
Let's take a step back from the problem, and look at the HTTP protocol in general - specifically the client request side - hopefully this will help you understand how all of this is supposed to work. First, a little history (if you're not interested in this, feel free to skip this section).
History
HTTP was originally designed as a mechanism for retrieving HTML documents from remote servers. At first it effectively supported only the GET method, whereby the client would request a document by name and the server would return it to the client. The first public specification for HTTP, labelled as HTTP 0.9, appeared in 1991 - and if you're interested, you can read it here.
The HTTP 1.0 specification (formalised in 1996 with RFC 1945) expanded the capabilities of the protocol considerably, adding the HEAD and POST methods. It was not backwards compatible with HTTP 0.9, due to a change in the format of the response - a response code was added, as well as the ability to include metadata for the returned document in the form of MIME format headers - key/value data pairs. HTTP 1.0 also abstracted the protocol from HTML, allowing for the transfer of files and data in other formats.
HTTP 1.1, the form of the protocol that is almost exclusively in use today is built on top of HTTP 1.0 and was designed to be backwards compatible with HTTP 1.0 implementations. It was standardised in 1999 with RFC 2616. If you are a developer working with HTTP, get to know this document - it is your bible. Understanding it fully will give you a considerable advantage over your peers who do not.
Get to the point already
HTTP works on a request-response architecture - the client sends a request message to the server, the server returns a response message to the client.
A request message includes a METHOD, a URI and optionally, a number of HEADERS. The request METHOD is what this question relates to, so it is what I will cover in the most depth here - but first it is important to understand exactly what we mean when we talk about the request URI.
The URI is the location on the server of the resource we are requesting. In general, this consists of a path component, and optionally a query string. There are circumstances where other components may be present as well, but for the purposes of simplicity we shall ignore them for now.
Let's imagine you type http://server.domain.tld/path/to/document.ext?key=value into the address bar of your browser. The browser dismantles this string, and determines that it needs to connect to an HTTP server at server.domain.tld, and ask for the document at /path/to/document.ext?key=value.
The generated HTTP 1.1 request will look (at a minimum) like this:
GET /path/to/document.ext?key=value HTTP/1.1
Host: server.domain.tld
The first part of the request is the word GET - this is the request METHOD. The next part is the path to the file we are requesting - this is the request URI. At the end of this first line is an identifier indicating the protocol version in use. On the following line you can see a header in MIME format, called Host. HTTP 1.1 mandates that the Host: header be included with every request. This is the only header of which this is true.
The request URI is broken into two parts - everything to the left of the question mark ? is the path, everything to the right of it is the query string.
Request Methods
RFC 2616 (HTTP/1.1) defines 8 request methods.
OPTIONS
The OPTIONS method is rarely used. It is intended as a mechanism for determining what kind of functionality the server supports before attempting to consume a service the server may provide.
Off the top of my head, the only place in fairly common usage that I can think of where this is used is when opening documents in Microsoft office directly over HTTP from Internet Explorer - Office will send an OPTIONS request to the server to determine if it supports the PUT method for the specific URI, and if it does it will open the document in a way that allows the user to save their changes to the document directly back to the remote server. This functionality is tightly integrated within these specific Microsoft applications.
GET
This is by far and away the most common method in every day usage. Every time you load a regular document in your web browser it will be a GET request.
The GET method requests that the server return a specific document. The only data that should be transmitted to the server is information that the server requires to determine which document should be returned. This can include information that the server can use to dynamically generate the document, which is sent in the form of headers and/or query string in the request URI. While we're on the subject - Cookies are sent in the request headers.
HEAD
This method is identical to the GET method, with one difference - the server will not return the requested document, if will only return the headers that would be included in the response. This is useful for determining, for example, if a particular document exists without having to transfer and process the entire document.
POST
This is the second most commonly used method, and arguably the most complex. POST method requests are almost exclusively used to invoke some actions on the server that may change its state.
A POST request, unlike GET and HEAD, can (and usually does) include some data in the body of the request message. This data can be in any format, but most commonly it is a query string (in the same format as it would appear in the request URI) or a multipart message that can communicate key/value pairs along with file attachments.
Many HTML forms use the POST method. In order to upload files from a browser, you would need to use the POST method for your form.
The POST method is semantically incompatible with RESTful APIs because it is not idempotent. That is to say, a second identical POST request may result in a further change to the state of the server. This contradicts the "stateless" constraint of REST.
PUT
This directly complements GET. Where a GET requests indicates that the server should return the document at the location specified by the request URI in the response body, the PUT method indicates that the server should store the data in the request body at the location specified by the request URI.
DELETE
This indicates that the server should destroy the document at the location indicated by the request URI. Very few internet facing HTTP server implementations will perform any action when they receive a DELETE request, for fairly obvious reasons.
TRACE
This provides an application-layer level mechanism to allow clients to inspect the request it has sent as it looks by the time it reaches the destination server. This is mostly useful for determining the effect that any proxy servers between the client and the destination server may be having on the request message.
CONNECT
HTTP 1.1 reserves the name for a CONNECT method, but does not define its usage, or even its purpose. Some proxy server implementations have since used the CONNECT method to facilitate HTTP tunnelling.
I've never tried using PUT (GET POST and FILES were sufficient for my needs) but this example is from the php docs so it might help you (http://php.net/manual/en/features.file-upload.put-method.php):
<?php
/* PUT data comes in on the stdin stream */
$putdata = fopen("php://input", "r");
/* Open a file for writing */
$fp = fopen("myputfile.ext", "w");
/* Read the data 1 KB at a time
and write to the file */
while ($data = fread($putdata, 1024))
fwrite($fp, $data);
/* Close the streams */
fclose($fp);
fclose($putdata);
?>
Here is the solution that I found to be the most useful.
$put = array();
parse_str(file_get_contents('php://input'), $put);
$put will be an array, just like you are used to seeing in $_POST, except now you can follow true REST HTTP protocol.
Use POST and include an X- header to indicate the actual method (PUT in this case). Usually this is how one works around a firewall which does not allow methods other than GET and POST. Simply declare PHP buggy (since it refuses to handle multipart PUT payloads, it IS buggy), and treat it as you would an outdated/draconian firewall.
The opinions as to what PUT means in relation to GET are just that, opinions. The HTTP makes no such requirement. It simply states 'equivalent' .. it is up to the designer to determine what 'equivalent' means. If your design can accept a multi-file upload PUT and produce an 'equivalent' representation for a subsequent GET for the same resource, that's just fine and dandy, both technically and philosophically, with the HTTP specifications.
Just follow what it says in the DOC:
<?php
/* PUT data comes in on the stdin stream */
$putdata = fopen("php://input", "r");
/* Open a file for writing */
$fp = fopen("myputfile.ext", "w");
/* Read the data 1 KB at a time
and write to the file */
while ($data = fread($putdata, 1024))
fwrite($fp, $data);
/* Close the streams */
fclose($fp);
fclose($putdata);
?>
This should read the whole file that is on the PUT stream and save it locally, then you could do what you want with it.

Decoding network chars (HTML5 Websocket)

I'm trying to develop a webchat with HTML5 websocket (with hybi-17 protocol) but I've some problems with chars decoding.
This is what I send through the client (user-agent: Firefox 7):
var socket = new MozWebSocket ('ws://localhost/server.php');
socket.onopen = function () {
alert ('Opened!');
}
Then, I send these data:
socket.send ('Hello');
socket.send ('World');
And this is the server-side code:
$bytes = #socket_recv ($socket, $buffer, BUFSIZE, 0);
if (($bytes == 0) || ($bytes == 2)) {
this->disconnect ($socket);
}
else {
echo $buffer;
}
While this is the data recevied echoed:
��6S~g?Y (Hello)
���~����� (World)
As you can see, the socket is opened and data travels from the client to the server.
The server works with PHP5 and uses normal socket functions to build the connection.
How can I decode that unreadable string in a human readable one?
Thanks in advance.
You have made one of the most common errors people make when they first start writing code that uses TCP -- you forgot to implement the protocol!
In your case, you forgot it in the server. The client already has a WebSocket implementation, and you request it by creating a 'MozWebSocket' object. The WebSocket specification says, "Each frame starts with a 0x00 byte, ends with a 0xFF byte, and contains UTF-8 data in between." Where's the code in the server to find the start of a frame and the end of a frame? Where the code to discard the 0xFF byte?
You actually have to implement the protocol. The protocol specification tells you how to decode the received data. (In your case, the data you are seeing as junk is most likely part of the protocol handshake -- the part that looks like ^n:ds[4U in this description of the handkshake phase.)
I'm pretty sure your method of invoking a PHP script to handle the WebSocket call through the web server will not work. That is, unless your web server knows how to do this -- WaterSpout and phpdaemon do.
Using the base server.php and client.html from http://code.google.com/p/phpwebsocket/ along with modification from HTML5 WebSocket with hybi-17 to deal with the single key standard I can successfully send one message to the server before it closes unexpectedly. It's at least a step closer just not sure as of yet why it closes after one successful message.

PHP web service, send response before end of script execution

I have a web service written in PHP to which an iPhone app connects to. When the app calls the service, a series of notification messages are sent to Apple's APNs server so it can then send Push Notifications to other users of the app.
This process can be time consuming in some cases and my app has to wait a long time before getting a response. The response is totally independent of the result of the notification messages being sent to the APNs server.
Therefore, I would like the web service to send the response back to the app regardless of whether the messages to APNs have been sent.
I tried using pcntl_fork to solve the problem:
<?php
...
$pid = pcntl_fork();
if($pid == -1)
{
// Could not fork (send response anyway)
echo "response";
}
else if($pid)
{
// Parent process - send response to app
echo "response";
}
else
{
// Child process - send messages to APNs then die
sendMessageAPNs($token_array);
die();
}
?> // end of script
Unfortunately, the parent process seems to wait for the child process to end before sending the response even though I do not use pcntl_wait in the parent process. Am I doing something wrong or is this normal behaviour? If this is normal then is there another way I can solve this problem?
Thank you!
If you're hosting the PHP process in Apache then you really shouldn't use this: see this for the section that says *Process Control should not be enabled within a web server environment and unexpected results may happen if any Process Control functions are used within a web server environment. *.
You should probably set up a separate daemon in your preferred language of choice and hand the APNS communication tasks off to that. If you really really really must try using ob_flush().
I think you can send the response back before doing the "long" process. Take a look at the flush() function of PHP it'll maybe help

Categories