I am looking to put together a website that displays the full HTTP Request Headers and HTTP Response Headers for the loading of the page itself. For instance. If someone browses to http://example.com/index.php , I want the following to display:
HTTP Request
GET /index.php HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
HTTP Response
HTTP/1.1 200 OK
Date: Mon, 21 Dec 2011 10:20:46 GMT
Server: Apache/2.2.15 (Red Hat)
X-Powered-By: PHP/5.3.3
Content-Length: 1169
Connection: close
Content-Type: text/html; charset=UTF-8
We were able to get the Request header to display fairly simply using the following PHP code:
print $_SERVER['REQUEST_METHOD']." ".$_SERVER['REQUEST_URI']." ".$_SERVER['SERVER_PROTOCOL']."<br>";
foreach (apache_request_headers() as $name => $value)
echo "$name: $value<br>";
But are having some difficulties with the HTTP Response header. Anyone have any ideas of how we can do this? It does not have to be PHP if you have a method that works in Perle or CGI or whatever.
To be clear, I don't mean to set the HTTP Response to anything specific, only display the response served by the web server to load the page.
You want to use headers_list()
http://www.php.net/manual/en/function.headers-list.php
headers_list() will return a list of headers to be sent to the browser / client. To determine whether or not these headers have been sent yet, use headers_sent().
Well here is the issue, the response header is generated after the PHP (or any server-side language for that matter) has already completed its job.
To put it in english its like the post man handing you a letter and you asking him to explain how the process of handing you the letter went. He will probably just look at you dumb.
You will need a client-side language (ie. JavaScript) to perform this task.
Use PHP to get the headers sent to the web Server.
http://www.php.net/manual/en/function.apache-request-headers.php
Use JavaScript to get headers sent by the web server. I would suggest using jQuery for that.
http://api.jquery.com/jQuery.ajax/#jqXHR
This way you are sure that you get all the headers which are either received by the web server or the browser.
Check out get_headers in PHP Manual
Related
I'm experiencing a strange error in my phorum server - seemingly at random, some users will be swapped with each other, and find themselves fully logged in as one another, with complete access to one another's accounts. I've experienced this bug myself once but cannot recreate it.
Phorum is configured to track sessions via cookie, with the session hashes also stored in the user database. I've confirmed the database is intact and there aren't session ID collisions happening.
You can see the source for phorum's authentication and session management here, it's fairly straightforward. Sessions are only created via log-in or from existing cookies so my working theory (after talking to a phorum developer) is that there is some sort of caching issue on the server. There's a known caching issue that affects ASP (see here or here for examples) but my server is a Linux server running Apache 2.4, MySQL (technically MariaDB 10.1 + InnoDB) and PHP 5.6. Is anyone aware of how this might be happening? I've been plugging away at this for over a week and have made little progress other than confirming the lack of bugs in the phorum session code.
The only clue I have to go on is that the session swapping started on the same day my hosting service took down (and restored) their file servers. They said they don't see how that could be responsible, however.
EDIT #1: I am adding some request and response headers.
Here is the response for an initial GET to get a list of forums.
General
Request URL: https://www.example.com/forum/list.php?11 Request
Method: GET Status Code: 200 Remote Address: x.x.x.x:443
Referrer Policy: no-referrer-when-downgrade
Response Headers
content-encoding: gzip content-type: text/html;
charset=UTF-8 date: Mon, 07 May 2018 20:23:08 GMT server: Apache
set-cookie: phorum_session_v5=35%3A412b7c329cc8741de88532342df9; expires=Tue, 08-May-2018 20:23:08 GMT; Max-Age=86400; path=/
status: 200 vary: Accept-Encoding via: e3s
Request Headers
:authority: www.example.com
:method: GET
:path: /forum/list.php?11
:scheme: https
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
accept-encoding: gzip, deflate, br accept-language: en-US,en;q=0.9
cache-control: max-age=0
cookie: phorum_session_v5=35%3A412b7c329cc8741de88532342df9;
referer: https://www.example.com/forum/addon.php?11,module=user_list
upgrade-insecure-requests: 1 user-agent: Mozilla/5.0 (Windows NT 10.0;
Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/66.0.3359.139 Safari/537.36
And here is a follow-up Ajax call:
General
Request URL: https://www.example.com/forum/ajax.php?client
Request Method: GET Status Code: 200 (from memory cache)
Remote Address: x.x.x.x:443 Referrer Policy: no-referrer-when-downgrade
Response Headers
age: 734
cache-control: must-revalidate
content-encoding: gzip
content-length: 2862
content-type: text/javascript;charset=UTF-8
date: Mon, 07 May 2018 18:29:37 GMT
expires: Tue, 08 May 2018 02:29:37 GMT
pragma: cache
server: Apache
status: 200
vary: Accept-Encoding
via: e2s
#Sammitch was correct - phorum was not setting cache-control in the headers for most of my pages. This was causing my hosting service's own caching system to cache responses that included session cookies.
I fixed this by changing my .htaccess file as described here: https://stackoverflow.com/a/7664157/1411376
This seems to work for my server config, since the php code (phorum) had no code that set cache-control (except for ajax requests).
Okay. I've found an answer to this on another StackOverflow post.
jQuery Ajax returning 404 Error, but correct Response
While that specific post assumes that my Wordpress installation may have been updated, the proposed solution worked to resolve my issue perfectly. Loading wp-load.php instead has resulted in the ajax calls working perfectly.
It's a shame that I found that answer only after creating this post!
I swear did my research :(
Below is the original question:
---------------------------------------------
A few months back, I have created a simple jQuery ajax call that pokes a php file to receive a small chunk of HTML back as response.
The call looks like this.
function ajaxTable() {
var request = $.ajax({
url: "/wp-content/themes/itul.theme.wp.abc/includes/download-ajax.php",
type: "POST",
dataType: "html",
data: { model: $('#download-model-select').find(':selected').data('id')}
});
request.done(function(msg) {
$("#download-table tbody").html(msg);
});
request.fail(function(jqXHR, textStatus) {
alert( "Request failed: " + textStatus );
});
}
The code has been working perfectly fine until about two days ago when someone noticed that the page had stopped working.
There had been absolutely no interaction with the code behind it, but all the jQuery ajax calls that I've been calling on the website had stopped working at that point. The code I've posted above is the simplest implementation of the various ajax calls hosted on the website.
For some reason, the calls kept flagging 404 errors despite the fact that going into the URL manually returns the results just fine! I had tried changing call type from POST to GET with no success either. Try going into the following URL, you'll see that the .php file is in place and pumping out the necessary response.
http://digital-watchdog.com/wp-content/themes/itul.theme.wp.abc/includes/download-ajax.php?model=856
After much frustration I went in to check the response headers, and viola! It seems that my ajax calls had been receiving the correct information all along! But why does it keep tagging 404 error?
I will paste the header information below. Please let me know if you've seen something like this happen before.
Thank you.
Remote Address:50.62.69.1:80
Request URL:http://digital-watchdog.com/wp-content/themes/itul.theme.wp.abc/includes/download-ajax.php?model=569
Request Method:GET
Status Code:404 Not Found
Request Headersview parsed
GET /wp-content/themes/itul.theme.wp.abc/includes/download-ajax.php?model=569 HTTP/1.1
Host: digital-watchdog.com
Connection: keep-alive
Accept: text/html, */*; q=0.01
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36
Referer: http://digital-watchdog.com/support/download/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,ko;q=0.6,es;q=0.4
Cookie: hsfirstvisit=http%3A%2F%2Fdigital-watchdog.com%2Fsupport%2Ftraining%2Fdw-spectrum-certification-san-francisco%2F|http%3A%2F%2Fdigital-watchdog.com%2Fwp-admin%2Fpost.php%3Fpost%3D1497%26action%3Dedit%26message%3D6|1392226058154; language=en; currency=USD; wp-settings-1=hidetb%3D1%26editor%3Dtinymce%26urlbutton%3Dnone%26imgsize%3Dfull%26align%3Dleft%26wordblock_category_tab%3Dpop%26libraryContent%3Dbrowse%26wplink%3D1%26ed_size%3D1042; wp-settings-time-1=1403654552; __utma=143756192.1546910067.1390588707.1403647017.1403708197.349; __utmb=143756192.6.10.1403708197; __utmc=143756192; __utmz=143756192.1401822249.292.9.utmcsr=hs_email|utmccn=(not%20set)|utmcmd=email; __hstc=74344461.64d6fa8253755804582ef98e557366d7.1392226058157.1403647017540.1403708204775.313; __hssrc=1; __hssc=74344461.6.1403708204775; hubspotutk=64d6fa8253755804582ef98e557366d7
Query String Parametersview parsed
model=569
Response Headersview parsed
HTTP/1.1 404 Not Found
Date: Wed, 25 Jun 2014 15:24:08 GMT
Server: Apache
X-Pingback: http://digital-watchdog.com/xmlrpc.php
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
Pragma: no-cache
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
How is it possible for client browser data to be saved in an array in PHP?
PHP runs on the server side, so I don't understand how it has access to information about the client's browser.
User agent data is usually sent with every HTTP requests, in the User-Agent HTTP header field. You might want to read up on HTTP message formats in general. For example, this is part of the HTTP request that my browser sent to load jQuery on this very page:
GET http://ajax.googleapis.com/ajax/libs/jquery/1.5.2/jquery.min.js HTTP/1.1
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Connection: keep-alive
If-Modified-Since: Fri, 01 Apr 2011 21:23:55 GMT
Accept-Charset: UTF-8,*;q=0.5
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.24 (KHTML, like Gecko) Chrome/11.0.696.60 Safari/534.24
Accept: */*
PHP reads the client browser data that you're asking about from the User-Agent header field.
The client sends data to the server which the server uses to build the array (I'm assuming you're talking about $_GET, $_POST, $_SERVER, etc.)
You will find it here
$_SERVER['HTTP_USER_AGENT']
You may need to parse this by regex to get the browser name and version separately.
$_REQUEST
An associative array that by default contains the contents of $_GET, $_POST and $_COOKIE.
The data is submited by the browser when a new page is requested, PHP just puts it into an array for your convenience.
You should start by reading a bit about HTTP (GET and POST to begin with), and HTTP headers.
Hi all,
i'm testing out a service for internal ads on our website...
I need to save internal session (PHP) to know what ads have been displayed and some other protection stuff...
The problem is that if i access to the API throught webbrowser GET or POST it does session (and saves the cookie of php session), but if i use jQuery.ajax() method it doesn't save it...
My link for tests is http://search.microdual.com
(this is because i wanted a customized google search on my laptop :p)
I suggest you to use firebug to take a look at the javascript code. (PHP code isn't needed because it is working on clicks...)
To simplify the debugging, i print out on response json array from server the session id on the var {id_sessao:"..."}...
Thanks in advance,
José Moreira
EDIT:
Headers from server:
Date Wed, 08 Sep 2010 11:24:48 GMT
Server Apache/2.2.8 (Ubuntu)
P3P CP="NOI NID ADMa OUR IND UNI COM NAV"
Cache-Control private, must-revalidate
Access-Control-Allow-Orig... *
Access-Control-Allow-Cred... true
Access-Control-Allow-Meth... OPTIONS, GET, POST
Access-Control-Allow-Head... Content-Type, Depth, User-Agent, X-File-Size, X-Requested-With, If-Modified-Since, X-File-Name, Cache-Control
Set-Cookie SN4b55935921bde=7ad280272050b4b44f17769909fd6f34; path=/ SN4b55935921bde=7ad280272050b4b44f17769909fd6f34; path=/
Keep-Alive timeout=15, max=97
Connection Keep-Alive
Transfer-Encoding chunked
Content-Type text/html; charset=UTF-8
Headers from the jQuery Ajax request:
Host www.microdual.com
User-Agent Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; pt-PT; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Accept application/json, text/javascript
Accept-Language pt-pt,pt;q=0.8,en;q=0.5,en-us;q=0.3
Accept-Encoding gzip,deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive 115
Connection keep-alive
Content-Type application/x-www-form-urlencoded; charset=UTF-8
Referer http://search.microdual.com/
Content-Length 29
Origin http://search.microdual.com
Pragma no-cache
Cache-Control no-cache
Where is the cookie control on the jQuery Ajax Request?
The problem is probably that your parent document is on search.microdual.com, and you are requesting a script from www.microdual.com. The session cookie isn't transported across domains.
If you are not using document.domain - I don't see it anywhere? - you will also have the additional problem that the AJAX call fails due to the Single Origin Policy.
To fix the cookie issue, you could either
set the cookie's domain to .microdual.com so it works in both sub-domains
carry the session across manually in the URL (?sessionid=xxxxx) and continue the session using session_id("xxxxx");.
put both scripts on the same sub-domain (search.microdual.com)
The latter would be the easiest way.
This is a two part question.
Q1: Can cURL based request 100% imitate a browser based request?
Q2: If yes, what all options should be set. If not what extra does the browser do that cannot bee imitated by cURL?
I have a website and I see thousands of request being made from a single IP in a very short time. These requests harvest all my data. When looked at the log to identify the agent used, it looks like a request from browser. So was curious to know if its a bot and not a user.
Thanks in advance
This page has all the answers to your questions. You can imitate the things mostly.
R1 : I suppose, if you set all the correct headers, that, yes, a curl-based request can imitate a browser-based one : after all, both send an HTTP request, which is just a couple of lines of text following a specific convention (namely, the HTTP RFC)
R2 : The best way to answer that question is to take a look at what your browser is sending ; with Firefox, for instance, you can use either Firebug or LiveHTTPHeaders to get that.
For instance, to get this page, Firefox sent those request headers :
GET /questions/1926876/can-a-curl-based-http-request-imitate-a-browser-based-request-completely HTTP/1.1
Host: stackoverflow.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2b4) Gecko/20091124 Firefox/3.6b4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://stackoverflow.com/questions/1926876/can-a-curl-based-http-request-imitate-a-browser-based-request-completely/1926889
Cookie: .......
Cache-Control: max-age=0
(I Just removed a couple of informations -- but you get the idea ;-) )
Using curl, you can work with curl_setopt to set the HTTP headers ; here, you'd probably have to use a combination of CURLOPT_HTTPHEADER, CURLOPT_COOKIE, CURLOPT_USERAGENT, ...