I am working on this prototype and i have used the php built in web server, and all works perfect except handling a range request from Safari for a mp4 file. i have the following bootstrap code that gets executed for every incoming request for the php built in server
$extensions = array("html");
$path = parse_url($_SERVER["REQUEST_URI"], PHP_URL_PATH);
$ext = pathinfo($path, PATHINFO_EXTENSION);
if (!in_array($ext, $extensions)) {
return isRangeRequest();
}
require_once(__DIR__ . $path);
function isRangeRequest()
{
$filelength = filesize(__DIR__ . '/video/video.mp4');
if (isset($_SERVER['HTTP_RANGE'])) {
$ranges = explode(',', substr($_SERVER['HTTP_RANGE'], 6));
$parts = explode('-', $ranges[0]);
$start = $parts[0]; // If this is empty, this should be 0.
$end = $parts[1]; // If this is empty or greater than than filelength - 1, this should be filelength - 1.
if (intval($end) == 0) return false;
$readBytes = $end - $start + 1;
header('HTTP/1.1 206 Partial Content');
header("Content-Length: " . $readBytes);
header('Content-Type: video/mp4');
header('Content-Range: bytes '.$start.'-'.$end.'/'.$filelength);
ini_set('memory_limit', "512M");
$fp = fopen(__DIR__ . '/video/video.mp4', 'rb');
if ($readBytes < 4096) {
print(fread($fp, $readBytes));
flush();
} else {
fpassthru($fp);
}
fclose($fp);
exit;
}
return false;
}
i have verified using tcpdump that the code above sends all the correct headers and i manage to get safari to start playing the video. however safari only is able to play the first few frames and then video stops.
here is response headers from a nginx server for the same request from safari
GET /video/video.mp4 HTTP/1.1
Host: job.local
Accept-Language: en-us
X-Playback-Session-Id: CD33782F-F257-4506-95C4-94ABD4E200CE
Cookie: PHPSESSID2=avrunqgk9rkb3941fe24q9asd6
Range: bytes=0-1
Accept: */*
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko) Version/9.0.3 Safari/601.4.4
Referer: http://job.local/test.html
Accept-Encoding: identity
Connection: keep-alive
HTTP/1.1 206 Partial Content
Server: nginx/1.6.2 (Ubuntu)
Date: Fri, 27 May 2016 03:29:16 GMT
Content-Type: video/mp4
Content-Length: 2
Last-Modified: Thu, 26 May 2016 17:53:49 GMT
Connection: keep-alive
ETag: "5747382d-8c6c509"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Cache-Control: max-age=315360000
Content-Range: bytes 0-1/147244297
GET /video/video.mp4 HTTP/1.1
Host: job.local
Accept-Language: en-us
X-Playback-Session-Id: CD33782F-F257-4506-95C4-94ABD4E200CE
Cookie: PHPSESSID2=avrunqgk9rkb3941fe24q9asd6
Range: bytes=0-147244296
Accept: */*
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko) Version/9.0.3 Safari/601.4.4
Referer: http://job.local/test.html
Accept-Encoding: identity
Connection: keep-alive
HTTP/1.1 206 Partial Content
Server: nginx/1.6.2 (Ubuntu)
Date: Fri, 27 May 2016 03:29:16 GMT
Content-Type: video/mp4
Content-Length: 147244297
Last-Modified: Thu, 26 May 2016 17:53:49 GMT
Connection: keep-alive
ETag: "5747382d-8c6c509"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Cache-Control: max-age=315360000
Content-Range: bytes 0-147244296/147244297
and here is the headers from the php built in server using the bootstrap code above
GET /video/video.mp4 HTTP/1.1
Host: localhost:8005
Accept-Language: en-us
X-Playback-Session-Id: 740A36CB-C116-4F5E-A3BD-5B4B3A27B543
Range: bytes=0-1
Accept: */*
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko) Version/9.0.3 Safari/601.4.4
Referer: http://localhost:8005/index.html
Accept-Encoding: identity
Connection: keep-alive
HTTP/1.1 206 Partial Content
Host: localhost:8005
Connection: close
X-Powered-By: PHP/5.5.30
Content-Length: 2
Content-Type: video/mp4
Content-Range: bytes 0-1/147244297
GET /video/video.mp4 HTTP/1.1
Host: localhost:8005
Accept-Language: en-us
X-Playback-Session-Id: 740A36CB-C116-4F5E-A3BD-5B4B3A27B543
Range: bytes=0-147244296
Accept: */*
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/601.4.4 (KHTML, like Gecko) Version/9.0.3 Safari/601.4.4
Referer: http://localhost:8005/index.html
Accept-Encoding: identity
Connection: keep-alive
HTTP/1.1 206 Partial Content
Host: localhost:8005
Connection: close
X-Powered-By: PHP/5.5.30
Content-Length: 147244297
Content-Type: video/mp4
Content-Range: bytes 0-147244296/147244297
apart from Connection: keep-alive not being supported by the built-in php server i don't see any other difference in the response. Do you see any potential issues here? The mp4 files is about 100MB and if i use a data-url instead of a file reference would that be ok?
This project is for a prototype that used only on a local machine, so there is no performance or security concerns and please avoid making such comments. the core concern is the reason why safari only plays the first few frames. (i also observed if the range request is not handled correctly safari will not even play the first few frames. So there is some other issue that maybe not so obvious.
Related
I get the request headers from the browser like this.
<?php
$DataFromBrowser='';
$DataToBrowser='';
$headers = getallheaders();
foreach($headers as $key=>$val){
$DataFromBrowser = $DataFromBrowser . $key . ': ' . $val . "\n";
}
//echo get_raw_http_request();
echo "Data from browser";
echo '<textarea rows="12" style="width:100%;">' . $DataFromBrowser . '</textarea>';
?>
And the output will look like this.
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:101.0) Gecko/20100101 Firefox/101.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Cookie: foo=bar
Upgrade-Insecure-Requests: 1
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Now I want to do exactly the same but for the response headers from the server. I'm aware I can do the following but it will create another connection I need the response headers of the current served page.
<?php
$URL = 'https://www.google.com';
$headers = get_headers($URL);
foreach($headers as $value) {
echo $value;
echo "<br>";
}
?>
If I change the above to http://127.0.0.1 It get's it self in a deadlock and will loop until it bombs out.
The response headers should look like this.
HTTP/1.1 200 OK
Date: Wed, 08 Jun 2022 16:53:45 GMT
Server: Apache/2.4.46 (Win64) OpenSSL/1.1.1j PHP/8.0.3
X-Powered-By: PHP/8.0.3
Content-Length: 589
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
I am making a request through Guzzle 3.8.1 for a Jasper report (via the Jasper Server API) that is over 2MB and I'm getting a response with the correct Content-Length header but no response body.
Guzzle request:
GET /jasperserver/rest_v2/reports/projects/i3app_suite/Resource/BulkShiftExport.csv?ACCOUNT_ID=2&START_DATETIME=2015-01-01&END_DATETIME=2015-01-31 HTTP/1.1
Host: jasper.i3app:8080
User-Agent: Guzzle/3.8.1 curl/7.19.7 PHP/5.5.8
Authorization: Basic ***=
Response:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Cache-Control: private
Expires: Wed, 31 Dec 1969 17:00:00 MST
P3P: CP="ALL"
Set-Cookie: JSESSIONID=F0B0F72B65A8145B45DA9DB2BACE53D8; Path=/jasperserver/; HttpOnly, userLocale=en_US;Expires=Fri, 13-Feb-2015 18:56:44 GMT;HttpOnly
Content-Disposition: attachment; filename="BulkShiftExport.csv"
output-final: true
Content-Type: application/vnd.ms-excel
Content-Length: 2173897
Date: Thu, 12 Feb 2015 18:57:02 GMT
If I make this request through curl on the command line (or request it in a browser) I get the report as expected
GET /jasperserver/rest_v2/reports/projects/i3app_suite/Resource/BulkShiftExport.csv?ACCOUNT_ID=2&START_DATETIME=2015-01-01&END_DATETIME=2015-01-30 HTTP/1.1
Authorization: Basic ***=
User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Host: jasper.i3app:8080
Accept: */*
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Cache-Control: private
< Expires: Wed, 31 Dec 1969 17:00:00 MST
< P3P: CP="ALL"
< Set-Cookie: JSESSIONID=AF1BF885354AF3E352DD9E18FA044A4B; Path=/jasperserver/; HttpOnly
< Set-Cookie: userLocale=en_US;Expires=Fri, 13-Feb-2015 19:03:42 GMT;HttpOnly
< Content-Disposition: attachment; filename="BulkShiftExport.csv"
< output-final: true
< Content-Type: application/vnd.ms-excel
< Content-Length: 2113902
< Date: Thu, 12 Feb 2015 19:03:49 GMT
<
{ [data not shown]
The only difference I could see was Accept: */* in the curl request. I tried adding that header to the guzzle request and got the same result.
When making the request through the Guzzle client it appears to take the same amount of time (5-6 seconds) to receive the response, and it sets the Content-Length header, but the response body is empty. Why am I getting an empty response body though Guzzle which is using curl but not when using curl on the command line? Is there an option I need to set to make this work?
$request = $this->getGuzzleClient()->createRequest('GET');
$config = $this->getConfig();
$url = new Url(
$config['scheme'],
$config['host'],
$config['user'],
$config['pass'],
$config['port'],
$config['path'] . $reportPath . '.' . $format,
new QueryString($parameters)
);
$request->setUrl($url);
$response = $request->send();
...
public function getGuzzleClient()
{
if (!$this->restClient) {
$client = new GuzzleClient();
$this->setRestClient($client);
}
return $this->restClient;
}
In my case, I was using MockHandler mistakenly.
I am trying to download xml file from one polish website. For first days it worked but then I could download this file to my server (but I can open and download it on my computer). In file on my server in which there should be xml content is html content telling me that I have been blocked.
I was trying to contact with webmaster from website from which I want to get xml and he told me that I am not blocked by IP address. So the question is what I should sent in headers or what to download this file?
My code to download xml file is below and here is the xml which I want to download: http://www.polskatimes.pl/rss/fakty_kraj.xml
$headers[] = "User-Agent:Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13";
$headers[] = "Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$headers[] = "Accept-Language:pl-PL,pl;q=0.8";
$headers[] = "Accept-Encoding:gzip,deflate,sdch";
$headers[] = "Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$headers[] = "Keep-Alive:115";
$headers[] = "Connection:keep-alive";
$headers[] = "Cache-Control:max-age=0";
$xml_data = file_get_contents($xml,false,stream_context_create(
array("http" => array('header' => $headers)))); // your file is in the string "$xml" now.
file_put_contents($xml_md5, $xml_data); // now your xml file is saved.
Request the URL in verbose mode (-v):
* About to connect() to www.polskatimes.pl port 80 (#0)
* Trying 195.8.99.38... connected
* Connectede to www.polskatimes.pl (195.8.99.38) port 80 (#0)
> GET /rss/fakty_kraj.xml HTTP/1.1
> User-Agent: curl/7.21.0 (x86_64-pc-linux-gnu) libcurl/7.21.0 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
> Host: www.polskatimes.pl
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 18 Apr 2013 10:40:15 GMT
< Content-Type: text/html; charset=utf8
< Transfer-Encoding: chunked
< Connection: close
< Vary: Accept-Encoding
< Expires: Thu, 18 Apr 2013 10:40:15 GMT
< Cache-Control: max-age=0
(html page with message that I am temporary blocked)
* Closing connection #0
To inspect what happens behind the scene (and which headers you actually need or not) you need to analyze a little. That is nothing magic, you can do it on the commandline with a software called curl. It is available for many (even all?) computer platforms.
First step most often is to request the URL in verbose mode (-v):
$ curl -v http://www.polskatimes.pl/rss/fakty_kraj.xml
* About to connect() to www.polskatimes.pl port 80 (#0)
* Trying 195.8.99.38... connected
* Connected to www.polskatimes.pl (195.8.99.38) port 80 (#0)
> GET /rss/fakty_kraj.xml HTTP/1.1
> User-Agent: curl/7.21.1 (i686-pc-mingw32) libcurl/7.21.1 OpenSSL/0.9.8r zlib/1.2.3
> Host: www.polskatimes.pl
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Wed, 17 Apr 2013 17:39:51 GMT
< Server: Apache
< Set-Cookie: sprawdz_cookie=1; expires=Thu, 17-Apr-2014 17:39:51 GMT
< Location: http://www.polskatimes.pl/rss/fakty_kraj.xml?cookie=1
< Vary: Accept-Encoding
< Content-Length: 0
< Connection: close
< Content-Type: text/html; charset=iso-8859-2
<
* Closing connection #0
That shows you the request (prefixed with >) and response (prefixed with <) headers and the response body (empty in this case). As you can see the status is 302 Found which means as 3xx a redirect and the location header tells where to:
Location: http://www.polskatimes.pl/rss/fakty_kraj.xml?cookie=1
As the query parameter suggests, this is a cookie-check. The cookie itself is set as well:
Set-Cookie: sprawdz_cookie=1; expires=Thu, 17-Apr-2014 17:39:51 GMT
So in the next step we will replay the last command but this time setting the cookie which can be done with the -b argument:
$ curl -v -b prawdz_cookie=1 http://www.polskatimes.pl/rss/fakty_kraj.xml
* About to connect() to www.polskatimes.pl port 80 (#0)
* Trying 195.8.99.38... connected
* Connected to www.polskatimes.pl (195.8.99.38) port 80 (#0)
> GET /rss/fakty_kraj.xml HTTP/1.1
> User-Agent: curl/7.21.1 (i686-pc-mingw32) libcurl/7.21.1 OpenSSL/0.9.8r zlib/1.2.3
> Host: www.polskatimes.pl
> Accept: */*
> Cookie: prawdz_cookie=1
>
< HTTP/1.1 200 OK
< Date: Wed, 17 Apr 2013 17:43:52 GMT
< Server: Apache
< Set-Cookie: sesja_gratka=e38fa0eb93705c8de7ae906198494439; expires=Wed, 24-Apr-2013 17:43:52 GMT; path=/; domain=polskatimes.pl
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Vary: Accept-Encoding
< Connection: close
< Transfer-Encoding: chunked
< Content-Type: text/xml; charset=utf-8
<
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title><![CDATA[Fakty - Kraj]]></title>
<link>http://www.polskatimes.pl/fakty/kraj/</link>
<atom:link href="http://www.polskatimes.pl/rss/fakty_kraj.xml" rel="self" type="application/rss+xml"/>
<description><![CDATA[Materiały z działu Kraj]]></description>
... (cutted)
So this is immediately successful. And now the real good part: You know that you need to set the cookie for the request and curl shows you already all headers it used:
> GET /rss/fakty_kraj.xml HTTP/1.1
> User-Agent: curl/7.21.1 (i686-pc-mingw32) libcurl/7.21.1 OpenSSL/0.9.8r zlib/1.2.3
> Host: www.polskatimes.pl
> Accept: */*
> Cookie: prawdz_cookie=1
Most of them you do not need to care about with file_get_contents, the first line as well as the Host: and the Accept: line.
The User-Agent: header does not look like it really plays a role as curl is accepted.
So all what is left is the Cookie: header. Let's try in PHP:
$ php -r "echo file_get_contents('http://www.polskatimes.pl/rss/fakty_kraj.xml', null,
stream_context_create(['http'=>['header'=>['Cookie: prawdz_cookie=1']]]));"
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title><![CDATA[Fakty - Kraj]]></title>
<link>http://www.polskatimes.pl/fakty/kraj/</link>
<atom:link href="http://www.polskatimes.pl/rss/fakty_kraj.xml" rel="self"
type="application/rss+xml"/>
... (cutted)
And this is the direct test that only the Set-Cookie: prawdz_cookie=1 header is needed.
I'm trying to connect to a exchange 2003 server, and read the mails. The server has form based auth.
It all works fine on the login POST I get the two cookies, but when I try the second request the server sends a 440 status code.
I have included the request/respond serie (Server, pass, username and domain has been replased ), hope that anyone can see where I'm making a mistake.
**** REQUEST ******
>[url] => https://<SERVER>/exchweb/bin/auth/owaauth.dll
>[request_header] => POST /exchweb/bin/auth/owaauth.dll HTTP/1.1
> User-Agent: Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1)
> Accept: */*
> Accept-Encoding: gzip
> Cookie: cadata="2CurvKH0Qt2fgnqjSl4/bOTmdobQWv581GKUu5IxzspI+BN525+gHJdhkX8hmWWkLK9KdGWNd5Jd9Fz9/"; sessionid=8d0f4db8-0f3e-4b6d-90d1-bec8cff4fe5f
> Connection: keep-alive
> Host: <CALLING SERVER>
> Content-type: application/x-www-form-urlencoded;charset=UTF-8
> Content-Length: 152
> destination=https%3A%2F%2F<SERVER>%2Fexchange%2F<USER>%2F&username=<DOMAIN>%5C<USER>&password=<PASSWORD>&SubmitCreds=Log+On&forcedownlevel=1&trusted=1
*** RESPONSE ****
< string(421) "HTTP/1.1 302 Moved Temporarily
< Content-Length: 0
< Location: https://<SERVER/exchange/<USER>/
< Server: Microsoft-IIS/6.0
< MicrosoftOfficeWebServer: 5.0_Pub
< X-Powered-By: ASP.NET
< Set-Cookie: sessionid=3456b6f5-abd1-4bf9-8da3-539900f7f10d; path=/
< Set-Cookie: cadata="17DhqZvs6837xRRMiNH2lBcCzo/AnK8Qbqj1mH791xfgUqy+TpnB201UvxcD9IePzaYLkZQfpjR2nOW3D"; HttpOnly; secure; path=/
< Date: Wed, 05 Sep 2012 07:53:12 GMT
*** REQUEST ***
> [url] => https://<SERVER>/exchange/<USER>/Indbakke
> [request_header] => SEARCH /exchange/<USER>/Indbakke HTTP/1.0
> User-Agent: Mozilla/5.0 (Windows NT 6.0; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1
> Cookie: cadata="17DhqZvs6837xRRMiNH2lBcCzo/AnK8Qbqj1mH791xfgUqy+TpnB201UvxcD9IePzaYLkZQfpjR2nOW3D"; sessionid=3456b6f5-abd1-4bf9-8da3-539900f7f10d
> Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
> Accept-Language: da-DK,da;q=0.8,en-US;q=0.6,en;q=0.4
> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
> Connection: keep-alive
> Accept-Encoding: gzip,deflate,sdch
> Host: <CALLING SERVER>
> Depth: 0
> Translate: f
> Content-type: application/xml;
> Content-Length: 297
> <?xml version="1.0"?><a:searchrequest xmlns:a="DAV:" xmlns:s="http://schemas.microsoft.com/exchange/security/">
> <a:sql>
> SELECT "DAV:displayname"
> ,"urn:schemas:httpmail:subject"
> FROM "https://<SERVER>/exchange/USER/Indbakke/"
> </a:sql>
> </a:searchrequest>
*** RESPONSE ***
< HTTP/1.1 440 Login Timeout
< Set-Cookie: sessionid=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
< Set-Cookie: cadata=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
< Content-Type: text/html
< Connection: close
< Content-Length: 43
"440 Login Timeout"
http://support.microsoft.com/kb/941201
Your login details have expired and you need to login again. This is a OWA thing.
In Safari and Firefox, the response part of the code is not working (i.e. from PHP-->Ajax-->jQuery). The variables definitely make it to the PHP fine (tested using mail() ), so it's probably some small error on my behalf!
jQuery:
$.ajax({
type: "POST",
dataType: "json",
data: postData,
url: "http://www.kudiclub.com/test/login/?loginsub",
success: function(data){
if(data.success==false){
$("#login .error").html(data.reply).show();
$("#login-email").val(data.email);
$("#password").val("");
}else{
window.location = data.ref;
}
}
});
PHP:
$data = array('success' => false, 'reply' => 'Username and password did not match.', 'email' => $email);
print json_encode($data);
return;
Hoping somebody can help. Thanks, Nick.
SOLUTION
After much fiddling about, it turns out that it doesn't see a full URL as a relative path. Changing the url to '/test/login/?loginsub' did the trick.
The server says: Content-Type: text/html. Is not a json document (application/json).
http://www.kudiclub.com/test/login/?loginsub
GET /test/login/?loginsub HTTP/1.1
Host: www.kudiclub.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: es-es,es;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Cookie: PHPSESSID=060b8210adfb3c67ff792b9471c7fa1c
Cache-Control: max-age=0
HTTP/1.1 200 OK
Date: Thu, 02 Aug 2012 22:12:10 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Keep-Alive: timeout=2, max=200
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html