PHP | How to send URL in 'content'=>http_build_query? - php

Im trying to send URL inside the query, But I get an error:
syntax error, unexpected '=' on line 7
How do I send the URL correctly?
UPDATE 1: now problem in line 8
<?php
$opts = array(
'http'=>array(
'method'=>"POST",
'content'=>http_build_query("url=http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/meera-humiliates-the-modis/1000055355")
'header'=>"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0" ,
"Content-Type: application/x-www-form-urlencoded; charset=UTF-8" ,
"Cookie: __cfduid=d1d6556674ee5fe2038c6c516e6ad786f1515596203"
));
$context = stream_context_create($opts);
$url = "http://en.fetchfile.net/fetch/";
$data = file_get_contents($url, false, $context);
echo $data;
UPDATE 2:
I do not know why, but the POST request does not receive the JSON
I'm trying to check what the problem is currently, if possible for help thanks
THE WHOLE POST REQUEST:
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://en.fetchfile.net/?url=http%3A%2F%2Fwww.hotstar.com%2Ftv%2Fsaath-nibhaana-saathiya%2F693%2Fahemgopi-have-a-fight%2F1000055779
Content-Length: 122
Cookie: __cfduid=d1d6556674ee5fe2038c6c516e6ad786f1515596203; _ym_uid=1515754827599813582; _ym_isad=2; bv_DSKskdck_s1d=bvDSKskdcks1d; CoinHiveOptOut=1515754855.247; bvppvarnp0=1
Connection: keep-alive
Body:
url=http%3A%2F%2Fwww.hotstar.com%2Ftv%2Fsaath-nibhaana-saathiya%2F693%2Fahemgopi-have-a-fight%2F1000055779&action=homePure
JSON Response:
{"acodec":"mp4a.40.2","description":"Meera humiliates Gopi at the Modi Bhavan. Later, Kinjal gets suspicious seeing Kokila's behaviour. Ahem is furious at Gopi for using his shirt as a cleaning cloth. Gopi reacts to his accusations. Later, Meera hurts Molu's head and humiliates Paridhi for playing a prank on her.","display_id":"1000055779","duration":1296,"episode":"Ahem-Gopi have a fight","episode_number":1369,"ext":"mp4","extractor":"HotStar","extractor_key":"HotStar","format":"hls-4563 - 1920x1080","format_id":"hls-4563","formats":[{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-64 - 250x140","format_id":"hls-64","fps":null,"height":140,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":64.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_1_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":250},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-121 - 320x180","format_id":"hls-121","fps":null,"height":180,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":121.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_2_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":320},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-241 - 320x180","format_id":"hls-241","fps":null,"height":180,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":241.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_3_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":320},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-461 - 416x234","format_id":"hls-461","fps":null,"height":234,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":461.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_4_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":416},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-861 - 640x360","format_id":"hls-861","fps":null,"height":360,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":861.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_5_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":640},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-1360 - 720x404","format_id":"hls-1360","fps":null,"height":404,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":1360.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_6_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":720},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-2062 - 1280x720","format_id":"hls-2062","fps":null,"height":720,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":2062.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_7_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","width":1280},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-3062 - 1600x900","format_id":"hls-3062","fps":null,"height":900,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":3062.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_8_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","width":1600},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-4563 - 1920x1080","format_id":"hls-4563","fps":null,"height":1080,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":4563.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_9_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","width":1920}],"fps":null,"height":1080,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"id":"1000055779","manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","playlist":null,"playlist_index":null,"preference":null,"protocol":"m3u8","requested_subtitles":null,"series":"Saath Nibhaana Saathiya","tbr":4563.0,"timestamp":1428499800,"title":"Ahem-Gopi have a fight","upload_date":"20150408","url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_9_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","webpage_url":"http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/ahemgopi-have-a-fight/1000055779","webpage_url_basename":"1000055779","width":1920}
Cookies valid until 2019
I send the POST request once again, I get the HTML page of fetchfile, and not the JSON with the video links

You should replace:
http_build_query(url=http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/meera-humiliates-the-modis/1000055355)
By
http_build_query("url=http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/meera-humiliates-the-modis/1000055355"),

Related

how to check if i am having correct file at laravel controller from post request

trying to upload an image at laravel: On echo $request; i am getting this: POST /infoshore0/api/public/v1/product/addProductImg?lang=en-us HTTP/1.1 Accept: application/json, text/plain, */* Accept-Encoding: gzip, deflate, br Accept-Language: en-IN,en-GB;q=0.9,en-US;q=0.8,en;q=0.7 Authorization: Bearer eyJ0eXAiOiJKV1QiLNjc0MTcsIm5iZiI6MTYwMjI2MzgxNywianRpIjoiZmxhdHJ1Mk5BeGdLc0JWRCJ9.lFcZOOSZx_94eZY97L-OfR36XLX_KLqvbZFJo5l2FW8 Connection: keep-alive Content-Length: 209141 Content-Type: text/plain;charset=UTF-8 Cookie: laravel_session=7dab0a4fdba93fbec911f Host: localhost:8080 Origin: http://localhost:8080 Referer: http://localhost:8080/cp/ Sec-Fetch-Dest: empty Sec-Fetch-Mode: cors Sec-Fetch-Site: same-origin User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36 {"file":"ÿØÿà\u0000\u00.......very long text","product_id":499859489}
I have tried if ($request->hasFile('file')) { echo 'File Found'; } and nithing is happeningAlso Passing product_id and image file using angularJS

Cannot retrieve all request headers using PHP slim framework

I'm using PHP slim framework for a personal project. For some reason, the PSR implementation of Request in Slim apparently is filtering some headers. I am trying to set a custom CSRF token and it is not available via $request->getHeaders(). Here's one example that shows the problem:
$app->get('/bar', function ($request, $response, $args) {
echo "PHP's getallheaders() <br>";
foreach (getallheaders() as $name => $value) {
echo "$name: $value <br>";
}
echo "Slim's GetHeaders() <br>";
foreach ($request->getHeaders() as $name => $values) {
foreach ($values as $value) {
echo "$name: $value <br>";
}
}
});
I get this output:
PHP's getallheaders()
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: null
Accept-Encoding: gzip, deflate
csrf_name: csrf56fc038c2f6eb
csrf_value: 4e077c04dadf22377da2aebc1a8caa78
Cookie: PHPSESSID=41016nbag70gi6shq4u2tg0aq1
Connection: keep-alive
Slim's GetHeaders()
Host: localhost
HTTP_USER_AGENT: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
HTTP_ACCEPT_LANGUAGE: null
HTTP_ACCEPT_ENCODING: gzip, deflate
HTTP_COOKIE: PHPSESSID=41016nbag70gi6shq4u2tg0aq1
HTTP_CONNECTION: keep-alive
I am trying to understand why the custom headers:
csrf_name: csrf56fc038c2f6eb
csrf_value: 4e077c04dadf22377da2aebc1a8caa78
are being removed by Slim.
It is not Slim, it is the webserver.
Even though header whose name contains underscore is valid by HTTP spec, both Nginx and Apache silently drop those headers for security reasons. In general you should use only use headers containing a..zA..Z and - characters.
With Apache you can still access header with underscore in their name using getallheaders() which is an alias to apache_request_headers().
With Nginx you can enable headers with underscrore in their name with underscores_in_headers on setting.
Believe it or not, the problem was that Slim does not like an underscore in a user-defined header. Once I changed csrf_name to csrfname it worked:
PHP's getallheaders()
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: null
Accept-Encoding: gzip, deflate
csrfvalue: 4e077c04dadf22377da2aebc1a8caa78
csrfname: csrf56fc038c2f6eb
Cookie: PHPSESSID=5aom8b5q7ottorc9279q9sh4g1
Connection: keep-alive
Slim's GetHeaders()
Host: localhost
HTTP_USER_AGENT: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
HTTP_ACCEPT_LANGUAGE: null
HTTP_ACCEPT_ENCODING: gzip, deflate
HTTP_CSRFVALUE: 4e077c04dadf22377da2aebc1a8caa78
HTTP_CSRFNAME: csrf56fc038c2f6eb
HTTP_COOKIE: PHPSESSID=5aom8b5q7ottorc9279q9sh4g1
HTTP_CONNECTION: keep-alive
So, don't forget, remove underscores!!
EDIT As explained by Mika Tuupola, the root cause is the HTTP server and not slim.

Set cookies login with curl on php

I want to scrap some data, but I need to log in. So, my idea is to copy the cookies when I log in to my program. But I don't know why, but if I'm using my program, I kept redirect to login pages. I already compared it, but the cookies are same.
Here's the header if I login using my google chrome, (Copied it from request header):
GET /example/data/data.jsp?date=01-Jan-2001&_=1439020103330 HTTP/1.1
Host: www.example.com
Connection: keep-alive
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,ms;q=0.6
Cookie: SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA; __utma=77920972.1013585791.1438786361.1438966138.1439020034.5; __utmc=77920972; __utmz=77920972.1438786423.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)
My curl code:
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, true);
$f = fopen('request.txt', 'w');
curl_setopt($ch, CURLOPT_STDERR , $f);
curl_setopt($ch, CURLOPT_HTTPHEADER,array('
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'Accept-Encoding: gzip, deflate, sdch',
'Accept-Language: en-US,en;q=0.8,ms;q=0.6',
'Cookie: SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA; __utma=77920972.1013585791.1438786361.1438966138.1439020034.5; __utmc=77920972; __utmz=77920972.1438786423.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)',
'Upgrade-Insecure-Requests: 1',
'User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36',
'X-DevTools-Emulate-Network-Conditions-Client-Id: 3A45EE97-D41F-45A3-AFCD-1540014377A7
'));
Here's my request.txt used to debug my program header:
* About to connect() to www.example.com port 80 (#0)
* Trying 202.43.163.203... * connected
* Connected to www.example.com (202.43.163.203) port 80 (#0)
> GET /example/data/data.jsp?date=01-Jan-2001&_=1439020103330 HTTP/1.1
Host: www.example.com
Accept: */*
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8,ms;q=0.6
Cookie: SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA; __utma=77920972.1013585791.1438786361.1438966138.1439020034.5; __utmc=77920972; __utmz=77920972.1438786423.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36
X-DevTools-Emulate-Network-Conditions-Client-Id: 3A45EE97-D41F-45A3-AFCD-1540014377A7
< HTTP/1.1 302 Found
< Date: Sat, 08 Aug 2015 09:44:06 GMT
< Server: Apache
< Set-Cookie: SESSIONID=7C8779894A3CE29D4BCED4B4D311E07E.NGXA; Path=/example/; HttpOnly
< Location: http://www.example.com/login.jsp
< Content-Length: 0
< Content-Type: text/plain
<
* Connection #0 to host www.example.com left intact
* Closing connection #0
This my first time scrapping data with login user, but did I miss something?
Cookies can be set via the CURLOPT_COOKIE. In your case
curl_setopt($ch, CURLOPT_COOKIE, 'SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA');
With semicolon space you can add more cookies to the request. See http://php.net/manual/en/function.curl-setopt.php for more information.
If you want to store and re-use the cookies you can also use CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR.

file_get_contents() with modified HTTP headers returning garbage html output

The following code is being used to extract html using SIMPLETHTMLDOM parser for php.
include('simple_html_dom.php');
$context = stream_context_create(array(
'http'=>array(
'method'=>"GET",
'header'=>"Accept: text/html,application/xhtml+xml,application/xml\r\n" .
"Accept-Charset: ISO-8859-1,utf-8\r\n" .
"Accept-Encoding: gzip,deflate,sdch\r\n" .
"Accept-Language: en-US,en;q=0.8\r\n",
'user_agent'=>"User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11\r\n"
)
));
$html = file_get_contents('http://www.nseindia.com/content/equities/cmbhav.htm', false, $context);
echo $html;
foreach($html->find('a') as $e)
echo $e->href . '<br>';
The output generated is Garbage html :
‹µVßsâ6~Ï_¡s§¹ð2¹^:ìNˆÝ†)!ið=äéFØV"K>I˜pMÿ÷®lCšéÝKy#+í­¾ý´òè]t}‘ÜÝÄä2¹š’›Oãéä‚x]J'qò¥Q5š=ŸÒxæ…G£Ü À2,·ÂÙ<&]YsËÁ 8eŒ%ãœU©*·#Úئ“Ù$×°<* ôRc<¢AóänÏ/ã8!v[B`áÑRÔâ&Õ¼´D0¹Z³Þ=«X³è£SuohªŠBÉåZ¦¦wo¼pD“o*¦ 7“88ÉTº.#Ú¢ókà[ålþÓ­`[ÐæÀà,8yö_¸Pãí$ëŸHVñ³J÷XYÎXAð~Ö¤¬„÷ƒXŠÜ³óôTg/ "Wª×'ñÓSmÖgm‡€åJ’œÉL#¬µÒ'ò—»Ö’X½†áßdÃe¦6=%ÁéIðÒzH0«¸ÂìÍIgxô#>W¾|°´xäÃBЖ5ãëèŽ,V©JÞËúççU0½â2ð‰Uå^n„ø*·¨j¦žÙ<𱾯ˆt˜zùº.GxöjjìV AÅ _pÁívó,9$¥2H[%la”X[":zÐ/‡¤Ù´‘¿v5xô‡uªnÄ,}G6Ì3IÎÇÓ˜¤ „)YÊå ³](FÁ-—,Ëšåö4}ßÿ­Æ¿wÝî! 3V‘1²rn™¶†\‚†nõÿu£æœ?ºÔí0p,³aQ{¡¤Åz“s ì_ÛÚ¬$8ó}R1ÁW2Àsï{O§7çQ4™ýŽ§t³ùÍùE3_ßFñ-.7ÕúøËi‹h°?øná£+%î…¬#ðdðáÅ^$̘Àx3—&±¶ä˜‰ÃŸ¡óB‡ÇraÊaó&m§ÉU%¶Ž6//¼Ä™s Ç+;$;+l 5Ú6†Âg.­VÙº¾PÎ9Üõ¸ïô/,—U»^1ý–Lp®æ·²ËÁ{3 ¤;:Z•±ms€6RâVÈ>ÈaÓ­ñp˜à ¶BTs,¹uu‹Úzôý±M¢FñªÀKĤÅÞ]]ÐáAgÇíMèöiö*[ûN×Ç]±ÚJì#ιÁfÈS&hüç§I2‰çôÔïŸÒ«ó[šþÏ8ºù_|$ªÞW^zï¶Ö#Â&ŒÔF Å2²ä—Hƒ¬CdD?ÿ‡\²ÅòÛ¹ ÑA.Ïø¤xÏ#¿Iè£7)à`pi¢¬Ét‰ÖÐæÄæ°gk0jB<=óšé4çºês>¿Ð.]¤Âhš ­›\jvìÒ¦æ”S8>Ѻï9‰V÷±W åu»ixH-[à—º¸ÇÀ͇Å?§m2“
Where am i doing Wrong !! I checked the headers info from fiddler. These are the actual header information when the link is opened with browser !!
[Client]
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11
[Transport]
Host: www.nseindia.com
Proxy-Connection: keep-alive
In the HTTP request you are explicitly declaring that you can handle compressed data, so that is what the server is returning:
Accept-Encoding: gzip,deflate,sdch\r\n
You must now decode that compressed data:
$html = gzuncompress($html);
As piotrekkr mentions in a comment, you can alternatively remove the Accept-Encoding header and the web server should return plain text instead.

Content not being Posted properly?

I am using simpleTest WebBrowser for DataScraping on this URL http://www.magicbricks.com/bricks/agentSearch.html. But though everything seems right I always get the error City Field is required. I guess the problem might be with the fact that values in city field changes dynamically when value of State changes. Any solutions? Here is my code.
<?php
require_once('simpletest/browser.php');
$browser = &new SimpleBrowser();
$browser->addHeader('User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2');
$browser->get('http://www.magicbricks.com/bricks/agentSearch.html');
$browser->setField('source','agentSearch');
$browser->setField('_transactionType','1');
$browser->setField('_propertyType','1');
$browser->setField('resultPerPage','50');
$browser->setField('agentSearchType','B');
$browser->setField('state','520');
$browser->setField('city','4320');
$browser->setField('keyword','');
$browser->setField('country','50');
print $browser->submitFormById('searchFormBean');
print $browser->getResponseCode()
?>
Here are some errors i noticed
Field Missing
Missing agentSearchType field
Missing transactionType ( There is Both transactionType & _transactionType)
missing propertyType ( There is both propertyType & _propertyType)
There are some header information you need to add such as
Referer
Cookie
A typical post test should come this format if you view the headers
POST http://www.magicbricks.com/bricks/agentSearch.html HTTP/1.1
Host: www.magicbricks.com
Connection: keep-alive
Content-Length: 173
Cache-Control: max-age=0
Origin: http://www.magicbricks.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.79 Safari/535.11
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Referer: http://www.magicbricks.com/bricks/agentSearch.html
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: JSESSIONID=nF1UqV3DM2tZC42zByYm6Q**.MBAPP09; __utma=163479907.1423216630.1331970312.1331970312.1331970312.1; __utmb=163479907.1.10.1331970312; __utmc=163479907; __utmz=163479907.1331970312.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _mbRunstats=3k0ilrpcgprh4tea
source=agentSearch&agentSearchType=B&country=51&state=601&city=8417&transactionType=11951&_transactionType=1&propertyType=10001&_propertyType=1&keyword=tesy&resultPerPage=50
I hope this helps
:D

Categories