I've used a simple file_get_contents function but that didn't get the actual contents (output) of that..
I could not figure the error!!!
Code:
<?php
// $url = $_GET['url'];
// $flv_http_path = urlencode($url);
$flv_http_path = 'http://r12.bhartibb-maa1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dXSlBSUl9FSkNNN19ITFZB&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1285074000&key=yt1&signature=3E1E4994130745C392FA479F6ACCE5F40E703A2C.A87325A1DCB178B04FD89A9DEEE811CDCB08157C&factor=1.25&id=8b2fd4fd9ac2f09f&st=lc';
echo "----$flv_http_path------";
$data = file_get_contents($flv_http_path);
echo "$data";
if($data)
echo "data is avail";
else
echo "data not available";
// $new_flv_path = dirname(_FILE_).'/flvs/sample.flv' ;
$new_flv_path = '/home/public_html/temp/sample.flv' ;
if(file_put_contents($new_flv_path, $data))
return $new_flv_path ;
else
{
echo "else part ";
return false;
}
?>
I got that url from the response headers of the youtube video
and the headers which i got is
http://v3.lscache1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dXSlBTVl9FSkNNN19ITVpF&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1285088400&key=yt1&signature=536A81F10AA43A4E015BB05FA182A9A966047C3C.C22269E2E1ECFC2C2DE7A8A45BA2C3DF7CF1EC08&factor=1.25&id=fd61d32bbbd1be5e&
GET /videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dXSlBTVl9FSkNNN19ITVpF&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1285088400&key=yt1&signature=536A81F10AA43A4E015BB05FA182A9A966047C3C.C22269E2E1ECFC2C2DE7A8A45BA2C3DF7CF1EC08&factor=1.25&id=fd61d32bbbd1be5e& HTTP/1.1
Host: v3.lscache1.c.youtube.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090616 Firefox/3.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: VISITOR_INFO1_LIVE=9CH-GUrsSEQ; __utma=27069237.1455305642.1275034254.1279868001.1280568792.6; __utmz=27069237.1279868001.5.2.utmcsr=google.com|utmccn=(referral)|utmcmd=referral|utmcct=/landing/youtube/lifeinaday/; watched_video_id_list_kvijayhari=7b1d7ce3852b9aca07a985813b83aaa6WxMAAABzCwAAADFuNzRnSExwU0M4cwsAAAB2ajgxNXlQNDFMQXMLAAAARWNjZ0lLdHVDM1lzCwAAAHFHZFo5elhoQ0ZvcwsAAAB0WXMwTXhvbTRjSXMLAAAAYUdBdDZwNGh0c2NzCwAAAGR2V25wMjdBSGZvcwsAAABtNDBhbG1SQzNzSXMLAAAANjhVT1BhTUtwOTBzCwAAADZnaFUxWDBqdVM4cwsAAABiRy0xYTRsUnlEMHMLAAAAWjh5OFFDRFNUQ29zCwAAADY0T0w3NzhBeUlFcwsAAABzQkl1OWpnSWtwQXMLAAAASllYM08wWEEteWdzCwAAAF95WGxpc0g4dkF3cwsAAABzcXZCSXdDMWxtWXMLAAAAaEMzd09EU0U5MHdzCwAAAGZaODhxaHduTVow; auto_translation=b901c47ed36700682e23d64062529856cwQAAAB0cnVl; PREF=f1=50000000&f2=2000&emt=iceberg&ftuc=32&ems=hd720&HIDDEN_MASTHEAD_ID=brO_JIa6RTI; use_hitbox=72c46ff6cbcdb7c5585c36411b6b334edAEAAAAw; GEO=489e10e70a42c0dfed7513e1895ffe1bcwsAAAAzSU56spxTTJhEAw==; watched_video_id_list=2aa4a241cbdc35137f13b3513ea3e653WwQAAABzCwAAAF9XSFRLN3ZSdmw0cwsAAABpeV9VX1pyQzhKOHMLAAAAd3ZsTUFKLVU2SEVzCwAAAENaQmpoVGQ0WjlN
HTTP/1.0 200 OK
Last-Modified: Sun, 20 Jun 2010 03:59:10 GMT
Content-Type: video/x-flv
Date: Tue, 21 Sep 2010 10:05:34 GMT
Expires: Tue, 21 Sep 2010 16:55:00 GMT
Cache-Control: public, max-age=24566
Content-Length: 4077907
Accept-Ranges: bytes
X-Content-Type-Options: nosniff
Server: gvs 1.0
X-Cache: MISS from localhost.localdomain
X-Cache-Lookup: MISS from localhost.localdomain:3128
Via: 1.0 localhost.localdomain:3128 (squid/2.6.STABLE6)
Connection: keep-alive
Check your URL.
When I put your url in browser it gives nothing so file_get_contents returns an empty string.
You need to check the output of file_get_contents as:
if($data !== false)
instead of
if($data)
I also get a HTTP Response 500. In order to crawl Youtube, you probably would have to spoof the User-Agent of the call and other measures to prevent Youtube from identifying you as a crawler.
It's youtubes way of preventing you from grabbing their flv files automatically.
You can't get the file from your server because the download link (which you got from your browser, or how did you find the flv link) is locked to your browser.
Which is why when someone other than you try to call the link we all get the 403 HTTP forbidden, even with a spoofed user-agent.
Try to use cURL and show the headers, you'll see what I mean.
I get a HTTP 403 at the follwoing Location:
http://r12.bhartibb-maa1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id,expire,ip,ipbits,itag,algorithm,burst,factor,oc:U0dXSlBSUl9FSkNNN19ITFZB&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1285074000&key=yt1&signature=3E1E4994130745C392FA479F6ACCE5F40E703A2C.A87325A1DCB178B04FD89A9DEEE811CDCB08157C&factor=1.25&id=8b2fd4fd9ac2f09f&st=lc
Response headers:
Content-Type:text/plain
Date:Tue, 21 Sep 2010 09:59:13 GMT
Proxy-Connection:close
Server:gvs 1.0
Via:1.0 proxy3#XXXXX.sch.uk:8080 (squid/2.6.STABLE19), 1.0 wcsproxy.XXXX.org.uk:8080 (squid/2.6.STABLE19)
X-Cache:MISS from proxy3#XXX.sch.uk, MISS from wcsproxy.XXX.org.uk
X-Content-Type-Options:nosniff
Well, when I tried to load the URL you refer to in $flv_http_path I got:
HTTP/1.1 403 Forbidden
Content-Type: text/plain
Connection: close
X-Content-Type-Options: nosniff
Date: Tue, 21 Sep 2010 09:57:19 GMT
Server: gvs 1.0
In return.
That should give you a clue :)
If that was not the acutal file you were trying to open, and you're not actually trying to scrape youtube you should try wrapping the url in urlencode() edit: But the url is already urlencoded (duh!)
"If you're opening a URI with special characters, such as spaces, you need to encode the URI with urlencode()." -- http://www.php.net/manual/en/function.file-get-contents.php
The link is empty. Fire the link in your browser and check the sourcecode. there is no data.
Related
Create a very simple form (no styling required) on your own environment to
integrate into our Webservice API using HTTP POST.
I have written a simple application using Zend Framework 2. The application creates a form that once validated is to be sent to a "Webserver API via HTTP POST" to receive a response.
I would like query the API in the correct way as I am very much a believer in protocols and standards.
My following code works for validating the form:
zf-skeleton/module/MyApplication/src/MyApplication/Controller/IndexController.php
public function submitAction() {
$myForm = new MyForm();
$myForm->get('submit')->setValue('Add');
$request = $this->getRequest();
if ($request->isPost()) {
$myModel = new MyModel();
$myForm->setInputFilter($myModel->getInputFilter());
$myForm->setData($request->getPost());
if ($myForm->isValid()) {
// Form is validated. [1]
Once the form has been validated I would like to know the best way to send the form data to the "Webserver API via HTTP POST" and handle the response
What kind of service am I connecting to?
My request headers:
GET /api?foo=1&bar=2 HTTP/1.1
Host: [theservice]
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:39.0) Gecko/20100101 Firefox/39.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: [my cookies]
Connection: keep-alive
The response headers:
HTTP/1.1 200 OK
Date: Tue, 23 Feb 2016 12:58:18 GMT
Content-Type: text/xml
Content-Length: 343
Connection: keep-alive
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: [Cookie data]
Vary: Accept-Encoding
Content-Encoding: gzip
Server: cloudflare-nginx
If I put into my browser:
http://theservice/api?foo=1&bar=2
I get the response formatted in XML:
<response>
<validresponse>YES</validresponse>
<foo>21</foo>
<bar>21</bar>
</response>
Is it SOAP , REST, neither or unknown?
Rest can generate response both in xml and json. So we cannot rule out REST necessarily.
It is very REST in my opinion.
I'd like to say that it's unknown. You can't say that REST use only JSON format because it can also use XML. IMO you can say that's endpoint which return some data in XML format.
I did R&D on prevention of CRLF injection in php, but i didn't find any solution in mycase, as I'm using a burp suite tool to inject some headers using CRLF characters like the below.
// Using my tool i put CRLF characters at the start of my request url
GET /%0d%0a%20HackedHeader:By_Hacker controller/action
//This generates an header for me like below
HackedHeader:By_Hacker
So i can modify all headers by doing just like above
This tool is just like a proxy server so it catches the request and gives the response and we can modify the response in the way we want.
So i'm just modifying the response by injecting some headers using CRLF characters. Now the Server responds to this request by injecting the CRLF characters in the response.
I'm just worried as header fields like Pragma, Cache-Control, Last-Modified can lead to cache poisoning attacks.
header and setcookie contain mitigations against response/header splitting, But these can't support me in fixing the above issue
Edit
When i request to mysite.com contact us page like below This is the request I captured in my tool like below
Request headers:
GET /contactus HTTP/1.1
Host: mysite.com
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
And i get the Response HTML for the above request
Now for the same request using the tool i'm adding custom headers just like below
Request Headers:
GET /%0d%0a%20Hacked_header:By_Hacker/contactus HTTP/1.1
Host: mysite.com
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Response Headers:
HTTP/1.1 302 Found
Date: Fri, 10 Jul 2015 11:51:22 GMT
Server: Apache/2.2.22 (Ubuntu)
Last-Modified: Fri, 10 Jul 2015 11:51:22 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Location: mysite.com
Hacked_header:By_Hacker/..
Vary: Accept-Encoding
Content-Length: 2
Keep-Alive: timeout=5, max=120
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
You can see the injected header Hacked_header:By_Hacker/.. in the above response
Is there anyway in php or apache server configuration to prevent such kind of headers' hack?
Not sure why all the down votes - infact, it is an interesting question :)
I can see that you have tagged CakePHP - which means your app is using Cake Framework... Excellent! If you are using Cake 3 , it is automatically strip off : %0d%0a
Alternatively, where you receive the response header, just strip off %0d%0a and you are good!
Where things like these could be applied - a 3rd party API response or say.... a Webhook response! or a badly sanitized way to handle intl.. example : lang=en to lang=fr where the GET param is directly set as response header... That would not be a wise move!
Ideally, the responses will be as GET and not in the header but either way just strip the %0d%0a and you are good.
Answering your edit.
You can see the injected header Hacked_header:By_Hacker/.. in the above response
That injected header cannot be controlled or stopped, mate. We do not have control over what the other server does.
The question is.. What do you do with the response header?
The answer is... You sanitize it, as ndm said you need to sanitize the input.. What you get as a response IS an input. As soon as you detect %0d%0a, discard the response.
Need code work?
<?php
$cr = '/\%0d/';
$lf = '/\%0a/';
$response = // whatever your response is generated in;
$cr_check = preg_match($cr , $response);
$lf_check = preg_match($lf , $response);
if (($cr_check > 0) || ($lf_check > 0)){
throw new \Exception('CRLF detected');
}
My Response Header is
Access-Control-Allow-Meth... GET, POST
Access-Control-Allow-Orig... *
Cache-Control no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection Keep-Alive
Content-Length 81
Content-Type text/html
Date Mon, 26 Aug 2013 06:35:53 GMT
Expires Thu, 19 Nov 1981 08:52:00 GMT
Keep-Alive timeout=5, max=99
Pragma no-cache
Server Apache/2.4.3 (Win32) OpenSSL/1.0.1c PHP/5.4.7
X-Powered-By PHP/5.4.7`
And The Request Header is
Accept application/json, text/javascript, */*; q=0.01
Accept-Encoding gzip, deflate
Accept-Language en-US,en;q=0.5
Cache-Control no-cache
Connection keep-alive
Content-Length 31
Content-Type application/x-www-form-urlencoded; charset=UTF-8
Cookie USERNAMEEMAIL=shan%40atlos.com; PHPSESSID=8asm46iltcqc9oahsbaaap1c16
Host localhost
Pragma no-cache
Referer http://localhost/test/
User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0
X-Requested-With XMLHttpRequest`
i am getting error in firefox "Not Well Formed" what is the problem in this.
i am getting the data correctly in json form but it show also error that is very annoying
Java Script Code to Make Request
GetTopNotification
And Uses a Class for make Ajax Request is
Workspace
Your reponse-header is incorrect.
if(headers_sent()) die('Should not output data before json');
header('Content-type: application/json');
echo json_encode($data_for_json);
exit;
Also, nothing should be sent before the json, and nothing after it either.
In response to comment below:
Somewhere in your php-code you're outputting json. However, as stated, your reponse header is incorrect: the Content-type part should be set to application/json; the above code does just that. A line-by line walktrough:
Checks if you did not already send anything and die if you did
Set the Content-type part of your response header to the appropriate mime-type
Output the json (as it currently is, should be fine)
exit;
More update irt comments
You're creating your json string manually: something i can wholeheartedly advice against, use an array or object and then use json_encode to create you json.
I also added output_buffering over your code, just in case.
Give it a try, new code is here
Update three
In work-space.js Replace this line
self.responseValue = self.getValueFromResponse( jqXHR );
With this
if(type != 'xml') self.responseValue = data;
else self.responseValue = self.getValueFromResponse( jqXHR );
save it, clear the cache, and try again.
How can I get the Header Informations like this web page (below) with using php
This web site Check website HTTP Server Header Information
Result:
HTTP Status for: "http://www.abc.com"
The title is: ""
Keywords: ""
Description: ""
HTTP/1.1 200 OK
Date: Fri, 22 Feb 2013 12:00:56 GMT
Server: Apache/2.2.3 (Debian) PHP/4.4.4-8+etch6
X-Powered-By: PHP/4.4.4-8+etch6
Keep-Alive: timeout=300
Connection: Keep-Alive
Content-Type: text/html
Use PHPs function get_headers:
$headers = get_headers($url, 1);
See: http://php.net/manual/en/function.get-headers.php
If you also want the meta keywords and meta description use get_meta_tags():
$tags = get_meta_tags($url);
You can use the PHP function get_header();.
This function will return an array with all the header fields.
For more information see: http://php.net/manual/en/function.get-headers.php
You could use:
print_r(get_headers($url));
I am using PHP http_post_data() call to send data to a cakephp controller.I do it like this:
$response=http_post_data($url, $xml_data_encoded);
The data arrives ok to the destination and I get a response which holds the response status.In my case the status is number 1 which means -data delivered ok.As you can see from the code below I get not only the status number (which is at the bottom of the message) but also the whole http post header.How can I strip this message off the header code so that eventually the response message holds only the status number?
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Thu, 10 Nov 2011 08:34:15 GMT
Server: Apache/2.2.17 (Ubuntu)
X-Powered-By: PHP/5.3.5-1ubuntu7.3
Set-Cookie: CAKEPHP=xxxxxxxxxxxxxxxx; expires=Fri, 18-Nov-2011 16:34:15 GMT; path=/XXXXXXXXXX/xxxxxxxx
P3P: CP="NOI ADM DEV PSAi COM NAV OUR OTRo STP IND DEM"
Vary: Accept-Encoding
Content-Length: 19
Content-Type: text/html
1
Btw, I also tried this:
HttpMessage::getBody(http_post_data($url, $xml_data_encoded));
and got no response at all.
Any help will be highly appreciated.
$response = http_post_data($url, $data);
preg_match_all('~HTTP/1\.[01]\s(\d{3})~', $response, $codes);
$codes would store all matches. Just print_r the array and look for desired keys.
Instead of a regular expression you can use http_parse_message
$response = http_parse_message(http_post_data($url, $data));
print $response->body;