I'm not sure if this function in CURL just strips the response body out but still load it fully.
Is that true? I don't want to waste bandwidth, I just want the headers.
CURLOPT_NOBODY will send a HEAD request to web server. The server should respond with just the HTTP headers and no body content.
http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.
It will only load the headers, it won't load the body of the requested document.
As you can see in official doc, It will not download the body if you enable it
https://curl.haxx.se/libcurl/c/CURLOPT_NOBODY.html
DESCRIPTION A long parameter set to 1 tells libcurl to not include the
body-part in the output when doing what would otherwise be a download.
For HTTP(S), this makes libcurl do a HEAD request. For most other
protocols it means just not asking to transfer the body data.
Enabling this option means asking for a download but without a body.
Related
I'm trying to retrieve a remote file (6MB text file) with PHP and I noticed that with fopen the speed is limited to 100KB/s and with file_get_contents is 15KB/s.
Howewer with wget from the server the speed is above 5MB/s.
What controls these speeds?
I checked the live speeds with nethogs.
wget is great on it's own to mirror sites it can actually parse links from pages and download files.
file_get_contents doesn't send a "connection" HTTP header, so the remote web server considers by default that's it's a keep-alive connection and doesn't close the TCP stream until 15 seconds (It might not be a standard value - depends on the server conf).
A normal browser would consider the page is fully loaded if the HTTP payload length reaches the length specified in the response Content-Length HTTP header. File_get_contents doesn't do this and that's a shame.
SOLUTION
SO, if you want to know the solution, here it is:
$context = stream_context_create(array('http' => array('header'=>'Connection: close\r\n')));
file_get_contents("http://www.something.com/somepage.html",false,$context);
The thing is just to tell the remote web server to close the connection when the download is complete, as file_get_contents isn't intelligent enough to do it by itself using the response Content-Length HTTP header.
I'm working on a twilio project with PHP which will be playing back a frequently changing audio file.
Twilio's TwiML Voice documentation states to:
make sure your web server is sending the proper headers to inform us
that the contents of the file have changed
Which headers are these and how do I set them in PHP.
Which headers are these?
This is how caching works on Twilio
Twilio requests a .mp3 from your server using a GET request. Your
server sends back a 200 OK, and also sends back an E-Tag header.
Twilio will save the E-Tag header, as well as the mp3 file, in its
database.
The next time Twilio sends a GET request to that URL, it will send
along the E-Tag header (it should look like "If-None-Match"). If the
file has not changed since the last time Twilio accesses it, your
server will send back a 304 Not Modified header. Crucially, it will
not send the mp3 file data. Twilio will use the mp3 file it has
stored in its database. It's much faster for Twilio to read the mp3
file from its database than it is for your server to send it (and it
also saves your server bandwidth).
If you change the content of the mp3 that is being served at the URL,
and Twilio makes a GET request to the URL, then your server will send
back a 200 OK, with a new E-Tag. Twilio will download the file from
your server, and cache it.
How do I set them in PHP?
header("ETag: \"uniqueID\");
When sending a file, web server attaches ID of the file in header called ETag. When requesting file, browser checks if the file was already downloaded. If cached file is found, server sends the ID with the file request to server. Server checks if the IDs match and if they do, sends back header("HTTP/1.1 304 Not Modified"); else Server sends the file normally.
One easy way to check is by adding some fake key-value pairs to the end of the URL, like http://yoururl.com/play.mp3?key=somevalue. Your website should still serve the same mp3 as it would if you loaded example.com/test.mp3, but to Twilio it will appear to be a new URL (uncached).
Twilio uses Squid to cache MP3. You can control how long an item is cached using the cache control header.
cache-control: max-age=3600
http://wiki.squid-cache.org/SquidFaq/InnerWorkings#How_does_Squid_decide_when_to_refresh_a_cached_object.3F
I would like to see if an external image is sending back the 304 not modfied header, is this able to be done on my server or will this violate the same origin policy?
The same origin policy can only make problems when using ajax. Images, javascripts etc. aren't affected by this policy, which only exists in browsers.
Simply send an curl request with the headers and then read the headers. See also: Can PHP cURL retrieve response headers AND body in a single request?
I'm writing a script that will use cURL to check a number of links. I know I can use curl_getinfo() to get the http status code, but I'm not sure if that requests the entire page or just the response with the headers. Is there any curl option or setting I can use to only request the headers of the URL (i.e. 404 not found, moved, etc) ?
You can instruct cURL to not download the contents (body) of the request by setting the CURLOPT_NOBODY option to TRUE - see
Does CURLOPT_NOBODY still download the body - using bandwidth
p.s. curl_getinfo() does not make requests - it gets information from requests that have already been executed.
Yesterday , I asked a question about socket post file data to php page.
Link is here
Upload and POST file to PHP page
When I follow the way , I changed my code.
char *str="POST /path/upload_file.php HTTP/1.0 \n Host: 00.00.00.00 \n Content-Disposition: name=2.jpg;filename=2.jpg\r " ;
write(sockfd, buf, filestat.st_size);
sprintf(send1,"%s%s\r\n",str,buf);
retval= send(sockfd,send1,sizeof(send1),0);
When I execute the Program , can get
result:
501 Method Not Implemented
Method Not Implemented
to /index.html not supported.
Invalid method in request
Apache/1.3.39 Server at localhost Port 80
I guess it's possible :
1. apache can't support something?
(my apache don't support ssl)
2. http protocol is not details?
3. other?
Thanks a lot.
There are multiple issues with your code.
First, the Host header is part of the HTTP 1.1 specification, and does not apply to HTTP 1.0.
Second, the delimiter for separating headers as well as body is \r\n, not \n.
Third, you are using the Content-Disposition in a request, the way it should be used in a response.
Lastly, you need to send the request as multipart/form-data. The answer you've linked to had it right. The request has to respect that format.
I've written a detailed example not so long ago (although in PHP, but the request format stays the same). You can find it here.