This question already has answers here:
What requests do browsers' "F5" and "Ctrl + F5" refreshes generate?
(6 answers)
Closed 8 years ago.
Here are the request headers:
GET /url/ HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
DNT: 1
Cookie: data=ABC;
Connection: keep-alive
Cache-Control: max-age=0
And here is response from the server:
HTTP/1.1 200 OK
Date: Fri, 11 Oct 2013 23:28:32 GMT
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By: PHP/5.3.2-1ubuntu4.18
Expires: Wed, 16 Oct 2013 03:28:32 GMT
Cache-Control: max-age=360000
Pragma: cache
Content-Language: en
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 844
Keep-Alive: timeout=15, max=92
Connection: Keep-Alive
No matter how many times I refresh the page and no matter how, it always makes a request (which takes 200ms+ since it's over HTTPS). Am I overseeing something? Should the response headers contain more parameters in order to prevent Firefox from making a server request?
I normally do something like add example.com/url/?nocache=<?php echo rand(); ?> which will add a random number to the url, making it a different url each time to prevent caching.
Related
When I browse to a page with Firefox and click a download link, the following headers are shown when I inspect the request in network inspector:
Connection: keep-alive
Content-Disposition: attachment; filename="example_file.mp3"
Content-Length: 35181829
Content-Transfer-Encoding: binary
Content-Type: audio/mpeg
Date: Fri, 19 Aug 2016 18:19:02 GMT
Keep-Alive: timeout=60
Server: nginx
X-Powered-By: PHP/5.4.45
However, when I use cURL to visit the same address, I get this:
Connection: keep-alive
Content-Length: 1918
Content-Type: text/html; charset=UTF-8
Date: Fri, 19 Aug 2016 20:46:23 GMT
Keep-Alive: timeout=60
Server: nginx
X-Powered-By: PHP/5.4.45
How can I form a request with cURL that gives me the same response as Firefox?
In Firefox, open up the Net tab in the developer options(F12) and open the URL of the page you need.
Take note of all the Request Headers in the request sent to the server:
Example:
Accept
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding
gzip, deflate
Accept-Language
nl,en-US;q=0.7,en;q=0.3
Connection
keep-alive
Cookie
_ga=GA1.2.598213448.1471644637; _gat=1
Host
mariannesdelights.be
User-Agent
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
Put all the headers in an array in this way
$headers = array('HeaderName:HeaderValue','HeaderName2:HeaderValue2');
Use the php function curl_setoption() to set the headers in the request:
curl_setopt($ch,CURLOPT_HTTPHEADER,$headers);
That should produce the exact same HTTP-Response headers.
I use redirects for all of my outbound links, which work fine with the exception of Amazon.
BUT, if I have the actual Amazon link in the HREF it works fine.
Here is an example:
When I redirect the link in the HREF looks something like this:
http://domain.com/buy-web/1425
which goes via an internal PHP script that gets the actual Amazon link, which looks like:
http://www.amazon.com/gp/search?ie=UTF8&tag=AFF_ID&index=aps&linkCode=ur2&camp=CAMP&creative=CREATIVE&keywords=tory-burch-amanda-crossbody-bag
and does:
header('Location: ' . $outURL);
when I redirect I am sent to this page on Amazon instead of the right one:
http://www.amazon.com/ref=nb_sb_noss_null
I have double checked that $outURL has the right link in it.
Anybody got any ideas why?
Thanks everyone.
PS: Here are the raw headers:
http://andynew/buy-web/1026
GET /buy-web/1026 HTTP/1.1
Host: andynew
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:30.0) Gecko/20100101 Firefox/30.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Cookie: __atuvc=1%7C28; andynew=a%3A10%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%227cb2ce95595fdf811ba5e2163b5f1d24%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A9%3A%22127.0.0.1%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A81%3A%22Mozilla%2F5.0+%28Macintosh%3B+Intel+Mac+OS+X+10.8%3B+rv%3A30.0%29+Gecko%2F20100101+Firefox%2F30.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1406480767%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3Bs%3A8%3A%22discount%22%3Bs%3A1%3A%220%22%3Bs%3A10%3A%22gridOrList%22%3Bs%3A4%3A%22grid%22%3Bs%3A11%3A%22displayData%22%3Bs%3A3%3A%22rel%22%3Bs%3A8%3A%22currency%22%3Bs%3A1%3A%22%24%22%3Bs%3A9%3A%22productId%22%3Bs%3A0%3A%22%22%3B%7D81d96834d2c29c51fc5169a3b4a3b489
Connection: keep-alive
HTTP/1.1 302 Found
Date: Sun, 27 Jul 2014 18:00:10 GMT
Server: Apache/2.2.25 (Unix) mod_ssl/2.2.25 OpenSSL/0.9.8y DAV/2 PHP/5.4.4
X-Powered-By: PHP/5.4.4
Set-Cookie: andynew=a%3A10%3A%7Bs%3A10%3A%22session_id%22%3Bs%3A32%3A%227de1b339301f44415c2d6e9b6bb4123a%22%3Bs%3A10%3A%22ip_address%22%3Bs%3A9%3A%22127.0.0.1%22%3Bs%3A10%3A%22user_agent%22%3Bs%3A81%3A%22Mozilla%2F5.0+%28Macintosh%3B+Intel+Mac+OS+X+10.8%3B+rv%3A30.0%29+Gecko%2F20100101+Firefox%2F30.0%22%3Bs%3A13%3A%22last_activity%22%3Bi%3A1406484010%3Bs%3A9%3A%22user_data%22%3Bs%3A0%3A%22%22%3Bs%3A8%3A%22discount%22%3Bs%3A1%3A%220%22%3Bs%3A10%3A%22gridOrList%22%3Bs%3A4%3A%22grid%22%3Bs%3A11%3A%22displayData%22%3Bs%3A3%3A%22rel%22%3Bs%3A8%3A%22currency%22%3Bs%3A1%3A%22%24%22%3Bs%3A9%3A%22productId%22%3Bs%3A0%3A%22%22%3B%7D576acce0b2310f850aea22ec8c28ae79; expires=Sun, 27-Jul-2014 20:00:10 GMT; path=/
Location: http://www.amazon.com/gp/search?ie=UTF8&tag=AFF-ID&index=aps&linkCode=ur2&camp=CAMP&creative=CREATIVE&keywords=ugg-classic-bow-shorty-womens-sized-accessory-grey
Content-Length: 0
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
----------------------------------------------------------
http://www.amazon.com/gp/search?ie=UTF8&tag=AFF-ID&index=aps&linkCode=ur2&camp=CAMP&creative=CREATIVE&keywords=ugg-classic-bow-shorty-womens-sized-accessory-grey
GET /gp/search?ie=UTF8&tag=AFF-ID&index=aps&linkCode=ur2&camp=CAMP&creative=CREATIVE&keywords=ugg-classic-bow-shorty-womens-sized-accessory-grey HTTP/1.1
Host: www.amazon.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:30.0) Gecko/20100101 Firefox/30.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Cookie: session-id-time=2082787201l; session-id=185-3040520-0718910; ubid-main=184-2208389-3838529; session-token="Q34hrPBvyfBn/m8gsaJOC185MUqzRj+6pViKhkOotL7DNO+KI3+yGaNFG65xvuN79/agGpPsGKWGN5fDBbt+KAnyq++5PFQSpAkNQnMAsJwMqR+hNzNXYZYr/pwBLe5RbsEF3mjVsACMNNMuzeVKw1OXUhkSO4XNxp+Z6LtlmyWy62KX0x5Qnz2AWy+pgKVFjLfDmHQAe1RMt82gDA0hMbgBZB3dHrko1dKm9o8BZ6I="; x-main="4g66HOBViU1sjppUYDkyRt5qEx7xXo?2"; __utma=125759317.321611390.1405706148.1406480645.1406480704.15; __utmz=125759317.1406478568.13.11.utmccn=(referral)|utmcsr=amazon.com|utmcct=/ap/signin|utmcmd=referral; __utmv=125759317.AFF-ID; x-wl-uid=1eBl7bcTv1V/h74WHTIZP+Hvnsr/oVfw2gl4r2f4jsJRBO2JdOf8BaddaGBLw/itrjEKvX1dbb0YAZxGDfP8eBA==; s_pers=%20s_vnum%3D1408288972388%2526vn%253D1%7C1408288972388%3B%20s_invisit%3Dtrue%7C1405698772388%3B%20s_nr%3D1405696972390-Repeat%7C1413472972390%3B; s_fid=12639358825850B3-1B157A0114E15FF1; s_dslv=1396942038784; s_vn=1418980489687%26vn%3D4; aws-ubid-main=182-0303093-2027858; aws-x-main="?6#eyI2zA2v9U3hUThKr9ptYKZDEnL1u"; regStatus=registered; csm-hit=s-10PZ2HKQV6RQT3SNRG82|1406478566454; __utmc=125759317
Connection: keep-alive
HTTP/1.1 302 Found
Date: Sun, 27 Jul 2014 18:00:10 GMT
Server: Server
x-amz-id-1: 1ENW70JDY98QP5G7R23W
x-amz-id-2: Cgjt+l8Pxxwl5A0t0tAla6b7y5Yobfh45Yq+kRDS4BPgrqyzZMzUmI5YVe3zF4lQej9X7ieHSTw=
X-Frame-Options: SAMEORIGIN
Location: /ref=nb_sb_noss_null
Content-Type: text/html;charset=UTF-8
Content-Length: 0
Set-Cookie: ubid-main=184-2208389-3838529; Domain=.amazon.com; Expires=Sat, 22-Jul-2034 18:00:11 GMT; Path=/
Vary: User-Agent
You aren't sending the URL you claim to be sending. All of your ampersands are being encoded as &, as if this were HTML. Stop doing that, and your problem will go away.
We couldn't tell you what part of your code is doing this unnecessary encoding, since the code you show in your question will not have this problem.
I'm trying to use jQuery's $.ajax() to GET a PHP page, on a based interval, which prints the latest enteries from a MySQL table. If there are new enteries, they are printed on the page, if not, "NaN" gets printed.
This is my PHP function which checks for new enteries: http://pastebin.com/R6NHDU3t
This is the HTML + jQuery which checks for response from the above function(got it from some answer here on Stack): http://pastebin.com/myT7evAx
The idea is this: I access the HTML page, the jQuery inside is GET-ing the PHP page every 3 seconds, the PHP checkes for new enteries inside MySQL; if there are new enteries, they are printed on the PHP page and get fetched by jQuery, then appended into #messages, if not, "NaN" is printed(because of JSON, doesen't matter).
This works fine if no new enteries are available: every 3 seconds "NaN" gets printed on the page; some Live HTTP Headers:
http://localhost/handler.class.php?Q=comments&_=1348715334458
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
X-Requested-With: XMLHttpRequest
Referer: http://localhost/live.html
HTTP/1.1 200 OK
Date: Thu, 27 Sep 2012 03:08:54 GMT
Server: Apache/2.2.22 (Win32) PHP/5.3.13
X-Powered-By: PHP/5.3.13
Content-Length: 15 <---- the "NaN", no new data
Keep-Alive: timeout=5, max=97
Connection: Keep-Alive
Content-Type: text/html
If new enteries are made, the script stopps appending "NaN"(doh, new data), and it hould append my new enteries, but the script stopps, no new data is appended and it stopps making requests. Here is the latest Live HTTP Headers log, showing the last request:
http://localhost/handler.class.php?Q=comments&_=1348715337473
GET /handler.class.php?Q=comments&_=1348715337473 HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:14.0) Gecko/20100101 Firefox/14.0.1
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
X-Requested-With: XMLHttpRequest
Referer: http://localhost/live.html
HTTP/1.1 200 OK
Date: Thu, 27 Sep 2012 03:08:57 GMT
Server: Apache/2.2.22 (Win32) PHP/5.3.13
X-Powered-By: PHP/5.3.13
Content-Length: 3257 <-- new data, so it got the response, but stopped here. no appending, no more requests.
Keep-Alive: timeout=5, max=96
Connection: Keep-Alive
Content-Type: text/html
Why does it stop when handler.class.php?Q=comments returns new data?
I have many redirections within a VM webserver, which work when browsing the server with the embedded navigator (iceweasel). But that does not work when accessing the server from the hosting machine's browsers (tested with FF4/IE8/Chrome/Opera11).
All experienced redirecting methods are driving to a "server not available or overloaded" in the hosting machine browsers.
If you could have a look to the headers from the apache logs and give some hints about the differences (main one looks to be the GET url, provided that the same code is operating):
Working request leads to this log :
cat /var/log/apache2/access.log | grep 127 | grep random | tail -n1
127.0.0.1 - authuserid [26/Jun/2011:11:11:52 +0200]
"GET /index.php?page=100 HTTP/1.1" 200 49151
"https://www.mydomain.foo/index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788&random=c0117685e7e65a307989c219efc587b4&sid=n7en2it41h2gumrcq3kmmil3c0&sidf=.ps_AWDkIY"
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.19) Gecko/2011050718 Iceweasel/3.0.6 (Debian-3.0.6-3)"
Non working request leads to this log :
cat /var/log/apache2/access.log | grep 192 | grep random | tail -n1
www.mydomain.org:80 192.168.X.Y - authuserid [26/Jun/2011:11:08:07 +0200]
"GET /index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788&random=685de8bcd4d198d6ad7f3cf4b23de5b7 HTTP/1.1" 302 -
"http://www.mydomain.foo/index.php?page=xyz"
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1"
I can't show the header response as I don't get a response and no error reported by apache (loglevel=error).
Thx
Controls done :
I have increased the browsers timeout (FF: network.http.keep-alive.timeout to 3600s : no change.
I checked that no headers were sent previously to the redirection : ok (a dump of headers_sent() shows no headers sent nor blank line or space in the includes, )
I have increased the Apache server timeout just in case: no change
I made sure of using an absolute url as of HTTP/1.1.
I tried php, html meta and js redirect: no change
EDIT 1:
Here are the headers as seen by LiveHTTPHeaders in the "non working" case :
http://www.mydomain.org/menus/noeud4.php
POST /menus/noeud4.php HTTP/1.1
Host: www.mydomain.org
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-15,utf-8;q=0.7,*;q=0.7
Keep-Alive: 3600
DNT: 1
Connection: keep-alive
Referer: http://www.mydomain.org/index.php?page=890
Cookie: PHPSESSID=4bge5gg1rgkit78k3seqlfcbq2
Authorization: Basic aW52aXRlZEBjYW1hY2FzYTp5b3VybXlndWVzdEB0b2RheQ==
Content-Type: application/x-www-form-urlencoded
Content-Length: 98
login=my_superlogin1&pwd1=vbigpass3xqz%40A2L&captcha=91690& source=noeud4.php&>formulaire_valide=SOUMETTRE
HTTP/1.1 302 Found
Date: Sun, 26 Jun 2011 14:17:27 GMT
Server: Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 mod_fastcgi/2.4.6 mod_python/3.3.1 Python/2.5.2 >mod_ssl/2.2.9 OpenSSL/0.9.8g PHP/5.3.3
X-Powered-By: PHP/5.3.3
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Location: http://www.mydomain.org/index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788
Content-Length: 0
Keep-Alive: timeout=60
Connection: Keep-Alive
Content-Type: text/html
http://www.mydomain.org/index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788
GET /index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788 HTTP/1.1
Host: www.mydomain.org
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-15,utf-8;q=0.7,*;q=0.7
Keep-Alive: 3600
DNT: 1
Connection: keep-alive
Referer: http://www.mydomain.org/index.php?page=890
Cookie: PHPSESSID=4bge5gg1rgkit78k3seqlfcbq2
Authorization: Basic aW52aXRlZEBjYW1hY2FzYTp5b3VybXlndWVzdEB0b2RheQ==
HTTP/1.1 302 Found
Date: Sun, 26 Jun 2011 14:19:59 GMT
Server: Apache/2.2.9 (Debian) DAV/2 SVN/1.5.1 mod_fastcgi/2.4.6 mod_python/3.3.1 Python/2.5.2 >mod_ssl/2.2.9 OpenSSL/0.9.8g PHP/5.3.3 X-Powered-By: PHP/5.3.3
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Location: https://www.mydomain.org/index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788&sid=4bge5gg1rgkit78k3seqlfcbq2&sidf=.ps_Z5wRio
Content-Length: 0
Keep-Alive: timeout=60
Connection: Keep-Alive
Content-Type: text/html
EDIT2:
Comparing both cases of request/responses (working/not working), I isolated the following 2 main differences among others :
On the "working" responses :
Status : 200
which I don't have on the "non working" reponse, but I do not understand why.
on the "NON Working" response :
DNT:1
which stands for the option Do Not Track (me) from FF4.
So I tried to deactivate this option, but same result.
I may miss sthg for sure. All looks as if the server was down. Maybe the session cookie (76 kb) is too big. I also tried downgrading firefox 4 to 3.6 as this another changed parameter, but I still get the same response with FF3.6 as FF4.
As you can see in the requests you posted you try to hit:
http://www.mydomain.org/menus/noeud4.php
but you get redirected to http://www.mydomain.org/index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788 and then again to https://www.mydomain.org/index.php?page=100&new_session=a4da9106dba2ffd40345a5eb624d7788&sid=4bge5gg1rgkit78k3seqlfcbq2&sidf=.ps_Z5wRio
Does it keep sending out 302 headers?
I'm guessing the noeud4.php script is some login script that will likely create a session and probably set some cookies. My guess would be to check if that is being done correctly - and figure out why it's throwing the 302.
Firebug shows a request which causes a huge delay to
http://reboltutorial.com/wp-content/themes/minaflow/none
Details below but I don't understand why it says it comes from xmlrpc and the stylesheet:
Date Sun, 04 Apr 2010 16:10:02 GMT
Server Apache
X-Powered-By PHP/5.2.13
X-Pingback http://reboltutorial.com/xmlrpc.php
Expires Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control no-cache, must-revalidate, max-age=0
Pragma no-cache
Set-Cookie wordpress_test_cookie=WP+Cookie+check; path=/; domain=.reboltutorial.com
Last-Modified Sun, 04 Apr 2010 16:10:03 GMT
Vary Accept-Encoding
Content-Encoding gzip
Keep-Alive timeout=2, max=94
Connection Keep-Alive
Transfer-Encoding chunked
Content-Type text/html; charset=UTF-8
RequĂȘtemise en page impression
GET /wp-content/themes/minaflow/none HTTP/1.1
Host: reboltutorial.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; fr; rv:1.9.2) Gecko/20100115 Firefox/3.6
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://reboltutorial.com/wp-content/themes/minaflow/style.css
1) Please remove SESSION_ID and all cookies from post (it's quote easy for hacker to access to your site with that)
2) In CSS you have 3 times next code:
background: url(none);
That's why it goes to 404 error!