I have a website with some images sent from php script.
Every time ETag is sent and checked with next request. That works fine. On first page open all files are downloaded with 200 Ok and with the next with 304 Not Modified.
But I noticed that some file sent directly from http server are event not checked for beeing modified if I re-open the page but are while reloading (F5).
If I open the page for the first time:
Files from web server: 200 Ok
Files from php script: 200 Ok
If I press F5:
Files from web server: 304 Not Modified
Files from php script: 304 Not Modified
If I reopen the page:
Files from web server: NO REQUEST
Files from php script: 304 Not Modified
What I want to do is to make browser skip checking whether my files where modified or not and display the cached version.
So I checked respones that http and php script are sending to browser:
Static content from http server:
Accept-Ranges bytes
Connection keep-alive
Content-Length 22033
Content-Type image/png
Date Sat, 15 Mar 2014 10:42:06 GMT
Etag "523c1617-5611"
Last-Modified Fri, 20 Sep 2013 09:32:07 GMT
Server nginx
Content served by php script:
Connection keep-alive
Content-Type image/jpeg
Date Sat, 15 Mar 2014 10:45:16 GMT
Etag c4ca4238a0b923820dcc509a6f75849b
Server nginx
Transfer-Encoding chunked
So the differences are:
Accept-Ranges, Content-Length, Last-Modified, Transfer-Encoding
But the only one which I THINK could make browser not check for update is Last-Modified. So if I add Last-Modified to my php script - would browser skip checking for update?
Ok, found an anwser.
Last-Modified: with firefox made image-loading something like random.
Cache-Control: max-age=... completly stopped checking whether image was modified
Related
I am trying to send a GET request to a PHP file uploaded on web by ESP8266, but I want do it directly by serial monitor. Every thing is right and the GET request sends correctly but the PHP file does not execute.
When I try to call the PHP file in browser and run http://readsensor.epizy.com/esp_post.php?temperature=11&humidity=11 it executes and data changes correctly on web.
Here is my serial monitor
AT+CIPSTART="TCP","readsensor.epizy.com",80
CONNECT
OK
AT+CIPSEND=85
OK
>
Recv 85 bytes
SEND OK
+IPD,1099:HTTP/1.1 200 OK
Server: nginx
Date: Sun, 15 Jul 2018 12:59:01 GMT
Content-Type: text/html
Content-Length: 870
Connection: keep-alive
Vary: Accept-Encoding
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Cache-Control: no-cache
<html><body><script type="text/javascript" src="/aes.js" ></script>
<script>function toNumbers(d){var e=[];d.replace(/(..)/g,function(d)
{e.push(parseInt(d,16))});return e}function toHex(){for(var d=
[],d=1==arguments.length&&arguments[0].constructor==Array?
arguments[0]:arguments,e="",f=0;f<d.length;f++)e+=
(16>d[f]?"0":"")+d[f].toString(16);return e.toLowerCase()}var
a=toNumbers("f655ba9d09a112d4968c63579db590b4"),b=toNumbers
("98344c2eee86c3994890592585b49f80"),c=toNumbers
("248813c2e30a32e656e03e29a770b794");document.cookie="__test=
"+toHex(slowAES.decrypt(c,2,a,b))+"; expires=Thu, 31-Dec-37 23:55:55 GMT;
path=/"; location.href="http://readsensor.epizy.com/esp_post.php?
temperature=40&humidity=40&i=1";</script><noscript>This site requires
Javascript to work, please enable Javascript in your browser or use a browser
with Javascript support</noscript></body></html>CLOSED
Because of this part of response
This site requires Javascript to work, please enable Javascript in your browser or use a browser with Javascript support
I checked my web browser to enable script and even changed it but still I facing this error.
My server is NGINX. Is there a problem with the server?
Look at the error message.
This site requires Javascript to work, please enable Javascript in your browser or use a browser with Javascript support
Then look at the code that is returned by the server:
document.cookie="__test="+toHex(slowAES.decrypt(c,2,a,b))+"; expires=Thu, 31-Dec-37 23:55:55 GMT;
It sets a cookie…
location.href="http://readsensor.epizy.com/esp_post.php?
temperature=40&humidity=40&i=1";
… then redirects the browser.
So either:
You are requesting a URL which isn't http://readsensor.epizy.com/esp_post.php, and then the JavaScript doesn't run, and you never request the PHP or
The PHP checks to see if a cookie is set, and if it is not, it outputs JavaScript to set it, and then redirect to run the script again.
If the problem is the former, then you just need to change your code to request the first URL in the first place.
If it is the latter, then you'll need to send the cookie data. You might be able to hard code that into your program, or you might need to parse the response to extract the right values for the cookie from the script.
I have following problem that's bugging my mind completely. I have to take over this cms from someone who doesn't want to care for it anymore and is giving no support whatsoever.
Situation is as follows: on the site there are several photo albums which are populated by reading a directory in php. All is good there, pictures are shown in the order they are read. In the management system, these pictures can be changed in order by an up or down-button. The way this is done is by swapping the image's filenames. This works, when I change the order for an image i can see server-side the filenames have actually been swapped.
This is however not the case on the site, at least not immediately: it takes an average of 10 minutes to see the images swapped there. Ofcourse, my client can't work like this, and he claims it has always worked before. I have tried to turn off caching browser-side, this hasn't helped. I can also note the changes take effect on the same time in IE and FF. I tried several ways of turning off cache server-side in php too, also to no avail.
Is there any other place where I should be looking or could there be another reason why these changes don't take effect immediately?
In addition, changes i make to javascript don't get picked up immediately too. I installed fiddler and this is the request header for that js file:
GET http://www.nobel-country-gite.be/admin/modules/Photoalbum/js/album.js HTTP/1.1
Accept: application/javascript, /;q=0.8
Referer: http://www.nobel-country-gite.be/admin/index.php?page=pic&album=24
Accept-Language: nl-BE
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
If-Modified-Since: Wed, 27 May 2015 15:55:12 GMT
If-None-Match: "ba1248f5-138b-5171244a92f66"
DNT: 1
Host: www.nobel-country-gite.be
Pragma: no-cache
Cookie: __utmc=39679548; __utma=39679548.1608184058.1429963360.1432662247.1432664636.7; __utmz=39679548.1429963360.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmc=1; PHPSESSID=7uge1ltg2rc11q63untthrc5s1; __utma=1.459796341.1429963360.1432662247.1432664636.7; __utmz=1.1429963360.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
Response header is as follows:
HTTP/1.1 304 Not Modified
Server: Apache
Last-Modified: Wed, 27 May 2015 15:55:12 GMT
ETag: "ba1248f5-138b-5171244a92f66"
Vary: Accept-Encoding
Content-Type: application/javascript
Date: Wed, 27 May 2015 16:57:55 GMT
X-Varnish: 1826689067 1825041752
Age: 556
Via: 1.1 varnish
Connection: keep-alive
I would expect the answer to be different instead of 'not modified'?
Edit - upon waiting a few minutes and refreshing the page again, the response for this file is what is expected:
HTTP/1.1 200 OK
Server: Apache
Last-Modified: Wed, 27 May 2015 16:57:30 GMT
ETag: "ba1248f5-1387-51713237ac28e"
Vary: Accept-Encoding
Content-Type: application/javascript
Transfer-Encoding: chunked
Date: Wed, 27 May 2015 17:03:43 GMT
X-Varnish: 1827728442
Age: 0
Via: 1.1 varnish
Connection: keep-alive
I couldn't help but notice you are using Varnish (indicated by the X-Varnish response header). Varnish is a caching reverse proxy, which means your pages are not just being cached by the browser, but also on the server (by Varnish). Your browser connects to Varnish, and Varnish connects to your Apache backend.
The first response header includes "Age: 556" - that's the cached version's age in seconds (almost 10 minutes). Then the age comes across as "0" when the page refreshes - that's because Varnish has updated its cache. Probably you can access the page over HTTPS to see your changes immediately reflected (Varnish doesn't work for HTTPS and most people don't bother setting up an HTTPS cache), or you can generally add garbage GET parameters to your URL (e.g. "?bogus=123") to force Varnish to re-fetch the page (this won't make other users see the new version, since they'll be accessing via the normal URLs).
Fixes: You can use varnishadm to ban (expire) certain URLs in Varnish when you've made a change; you can modify the "Cache-Control" or "Expires" headers your CMS/Apache (via PHP, .htaccess, etc.) produces to reduce cache time (Varnish completely respects cache control headers in its caching strategy); you can change Varnish's behavior by editing the relevant VCL (usually "default.vcl"); or you can accept that caches are generally good (they save a lot of time and resources in generating the response), and maybe a 10 minute delay is an acceptable trade-off.
When I did a networks course I learned about HTTP Request and Response messages and I know how to code in php reasonably enough to get around. Now my question is, the PHP has to have some link to HTTP request and response message but how. I can't seem to see the link between the two. My reasoning for asking this is that I am using the Twitter API console tool to query their api. The tool sends the following HTTP request:
GET /1.1/search/tweets.json?q=%40twitterapi HTTP/1.1
Authorization:
OAuth oauth_consumer_key="DC0se*******YdC8r4Smg",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1410970037",oauth_nonce="2453***055",oauth_version="1.0",oauth_token="796782156-ZhpFtSyPN5K3G**********088Z50Bo7aMWxkvgW",oauth_signature="Jes9MMAk**********CxsKm%2BCJs%3D"
Host:
api.twitter.com
X-Target-URI:
https://api.twitter.com
Connection:
Keep-Alive
and then I get a HTTP response:
HTTP/1.1 200 OK
x-frame-options:
SAMEORIGIN
content-type:
application/json;charset=utf-8
x-rate-limit-remaining:
177
last-modified:
Wed, 17 Sep 2014 16:07:17 GMT
status:
200 OK
date:
Wed, 17 Sep 2014 16:07:17 GMT
x-transaction:
491****a8cb3f7bd
pragma:
no-cache
cache-control:
no-cache, no-store, must-revalidate, pre-check=0, post-check=0
x-xss-protection:
1; mode=block
x-content-type-options:
nosniff
x-rate-limit-limit:
180
expires:
Tue, 31 Mar 1981 05:00:00 GMT
set-cookie:
lang=en
set-cookie:
guest_id=v1%3A14109******2451388; Domain=.twitter.com; Path=/; Expires=Fri, 16-Sep-2016 16:07:17 UTC
content-length:
59281
x-rate-limit-reset:
1410970526
server:
tfe_b
strict-transport-security:
max-age=631138519
x-access-level:
read-write-directmessages
So how do these HTTP request and response messages fit into PHP? Does PHP auto generate this? How do I add authorization to PHP requests etc? I'm confused about the deeper workings of PHP
When the client sends the HTTP request to the server, there has to be something to receive the HTTP request, which is called a web server. Examples of web servers are Apache, IIS, Nginx, etc. You can also write your own server, which can handle input however it wants. In this case, I'll assume that you are requesting a PHP file.
When the web server captures the HTTP request, it determines how it should be handled. If the file requested is tweets.json, it will go make sure that file exists, and then pass control over to PHP.
PHP then begins its execution, and performs any logic that the script needs to do, meaning it could go to the database, it reads, writes and makes decisions based cookies, it does math, etc.
When the PHP script is done, it will return a HTML page as well as a bunch of headers back to the web server that called it. From there, the web server turns the HTML page and headers back into a HTTP request to respond.
That is a pretty simple overview, and web servers can work in many different ways, but this is a simple example of how it could work in a introductory use-case. In more complex scenarios, people can write their own web servers, which perform more complex logic inside of the web server software, rather than passing it off to PHP.
When it comes down to it, PHP files are just scripts that the web server executes when they are called, they provide the HTTP request as input, and get a web page and headers as output.
I made a curl request to a PHP file on my server from my machine and it returns the following response.
HTTP/1.1 200 OK
Content-Length: 317
Content-Type: application/json
Server: Microsoft-IIS/7.5
Last-Modified: Wed, 19 Feb 2014 11:30:16 GMT
Via: 1.1 SC10100_83_75
Connection: keep-alive
Date: Wed, 19 Feb 2014 12:00:47 GMT
The problem is, this seems to be caching my request and is not returning the latest values. I suspect it is due to the Via header (read that it is about proxies). What is it and how can I remove it?
Looks like via - indicates that you have been routed via a proxy - however - this is information only - you need to make sure you caching headers are being set correctly so that the proxy takes the correct action.
Currently there is no caching headers being set on your request.
If you do not want it to be cached - then set the following header:
Cache-Control: no-cache
See http://www.w3.org/Protocols/HTTP/Issues/cache-private.html
I am trying to use curl to download a torrent file the url is
http://torcache.net/torrent/006DDC8C407ACCDAF810BCFF41E77299A373296A.torrent
You will notice that upon getting to the page the download of the file is blocked for a few seconds via javascript, I was wondering if there is anyway to bypass this while using curl and php?
Thanks
The file is not blocked via javascript, that's just an informal message if you request that file. The redirect then is done via javascript.
You can simulate the request your own, the important part here is that you add the HTTP Referrer request header. Example:
$ curl -I -H 'Referer: http://torcache.net/torrent/006DDC8C407ACCDAF810BCFF41E77299A373296A.torrent' http://torcache.net/torrent/006DDC8C407ACCDAF810BCFF41E77299A373296A.torrent
HTTP/1.1 200 OK
Server: nginx/1.3.0
Date: Sun, 10 Jun 2012 17:13:59 GMT
Content-Type: application/x-bittorrent
Content-Length: 10767
Last-Modified: Sat, 09 Jun 2012 22:17:03 GMT
Connection: keep-alive
Content-Encoding: gzip
Accept-Ranges: bytes
Referrer is one thing to check, mind the typo in the HTTP specs, see Wikipedia.