PHP | How to send URL in 'content'=>http_build_query? - php
Im trying to send URL inside the query, But I get an error:
syntax error, unexpected '=' on line 7
How do I send the URL correctly?
UPDATE 1: now problem in line 8
<?php
$opts = array(
'http'=>array(
'method'=>"POST",
'content'=>http_build_query("url=http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/meera-humiliates-the-modis/1000055355")
'header'=>"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0" ,
"Content-Type: application/x-www-form-urlencoded; charset=UTF-8" ,
"Cookie: __cfduid=d1d6556674ee5fe2038c6c516e6ad786f1515596203"
));
$context = stream_context_create($opts);
$url = "http://en.fetchfile.net/fetch/";
$data = file_get_contents($url, false, $context);
echo $data;
UPDATE 2:
I do not know why, but the POST request does not receive the JSON
I'm trying to check what the problem is currently, if possible for help thanks
THE WHOLE POST REQUEST:
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0
Accept: application/json, text/javascript, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Referer: http://en.fetchfile.net/?url=http%3A%2F%2Fwww.hotstar.com%2Ftv%2Fsaath-nibhaana-saathiya%2F693%2Fahemgopi-have-a-fight%2F1000055779
Content-Length: 122
Cookie: __cfduid=d1d6556674ee5fe2038c6c516e6ad786f1515596203; _ym_uid=1515754827599813582; _ym_isad=2; bv_DSKskdck_s1d=bvDSKskdcks1d; CoinHiveOptOut=1515754855.247; bvppvarnp0=1
Connection: keep-alive
Body:
url=http%3A%2F%2Fwww.hotstar.com%2Ftv%2Fsaath-nibhaana-saathiya%2F693%2Fahemgopi-have-a-fight%2F1000055779&action=homePure
JSON Response:
{"acodec":"mp4a.40.2","description":"Meera humiliates Gopi at the Modi Bhavan. Later, Kinjal gets suspicious seeing Kokila's behaviour. Ahem is furious at Gopi for using his shirt as a cleaning cloth. Gopi reacts to his accusations. Later, Meera hurts Molu's head and humiliates Paridhi for playing a prank on her.","display_id":"1000055779","duration":1296,"episode":"Ahem-Gopi have a fight","episode_number":1369,"ext":"mp4","extractor":"HotStar","extractor_key":"HotStar","format":"hls-4563 - 1920x1080","format_id":"hls-4563","formats":[{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-64 - 250x140","format_id":"hls-64","fps":null,"height":140,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":64.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_1_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":250},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-121 - 320x180","format_id":"hls-121","fps":null,"height":180,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":121.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_2_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":320},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-241 - 320x180","format_id":"hls-241","fps":null,"height":180,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":241.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_3_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":320},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-461 - 416x234","format_id":"hls-461","fps":null,"height":234,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":461.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_4_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":416},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-861 - 640x360","format_id":"hls-861","fps":null,"height":360,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":861.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_5_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":640},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-1360 - 720x404","format_id":"hls-1360","fps":null,"height":404,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":1360.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_6_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.66.30","width":720},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-2062 - 1280x720","format_id":"hls-2062","fps":null,"height":720,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":2062.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_7_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","width":1280},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-3062 - 1600x900","format_id":"hls-3062","fps":null,"height":900,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":3062.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_8_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","width":1600},{"acodec":"mp4a.40.2","ext":"mp4","format":"hls-4563 - 1920x1080","format_id":"hls-4563","fps":null,"height":1080,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","preference":null,"protocol":"m3u8","tbr":4563.0,"url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_9_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","width":1920}],"fps":null,"height":1080,"http_headers":{"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","Accept-Charset":"ISO-8859-1,utf-8;q=0.7,*;q=0.7","Accept-Encoding":"gzip, deflate","Accept-Language":"en-us,en;q=0.5","Cookie":"_alid_=Q7nfOk5ehqKpd9pOQBAIiQ==; hdntl=exp=1515843396~acl=%2f*~data=hdntl~hmac=f1a825da0313c89669ed8f19155bf8410b9f1f6f39068ee04279dd22e7c4e78d","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)","X-Forwarded-For":"117.200.190.41"},"id":"1000055779","manifest_url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/master.m3u8?hdnea=st=1515756695~exp=1515757295~acl=/*~hmac=a524380edf50b440821777604ba55f39e4b12b45212be67d878ad7299ec4402f","playlist":null,"playlist_index":null,"preference":null,"protocol":"m3u8","requested_subtitles":null,"series":"Saath Nibhaana Saathiya","tbr":4563.0,"timestamp":1428499800,"title":"Ahem-Gopi have a fight","upload_date":"20150408","url":"https://staragvod1-vh.akamaihd.net/i/videos/plus/sns/1369/1000055779_,16,54,106,180,400,800,1300,2000,3000,4500,_STAR.mp4.csmil/index_9_av.m3u8?null=0&id=AgDUL+9aAhYWjMSdWFr0pI0ySbRDIOcQ+v8AA5I0zbQ0kTxVW6Fk8MREQkUs6kKfbBZtynevwukFzQ%3d%3d","vcodec":"avc1.77.30","webpage_url":"http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/ahemgopi-have-a-fight/1000055779","webpage_url_basename":"1000055779","width":1920}
Cookies valid until 2019
I send the POST request once again, I get the HTML page of fetchfile, and not the JSON with the video links
You should replace:
http_build_query(url=http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/meera-humiliates-the-modis/1000055355)
By
http_build_query("url=http://www.hotstar.com/tv/saath-nibhaana-saathiya/693/meera-humiliates-the-modis/1000055355"),
Related
how to check if i am having correct file at laravel controller from post request
trying to upload an image at laravel: On echo $request; i am getting this: POST /infoshore0/api/public/v1/product/addProductImg?lang=en-us HTTP/1.1 Accept: application/json, text/plain, */* Accept-Encoding: gzip, deflate, br Accept-Language: en-IN,en-GB;q=0.9,en-US;q=0.8,en;q=0.7 Authorization: Bearer eyJ0eXAiOiJKV1QiLNjc0MTcsIm5iZiI6MTYwMjI2MzgxNywianRpIjoiZmxhdHJ1Mk5BeGdLc0JWRCJ9.lFcZOOSZx_94eZY97L-OfR36XLX_KLqvbZFJo5l2FW8 Connection: keep-alive Content-Length: 209141 Content-Type: text/plain;charset=UTF-8 Cookie: laravel_session=7dab0a4fdba93fbec911f Host: localhost:8080 Origin: http://localhost:8080 Referer: http://localhost:8080/cp/ Sec-Fetch-Dest: empty Sec-Fetch-Mode: cors Sec-Fetch-Site: same-origin User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36 {"file":"ÿØÿà\u0000\u00.......very long text","product_id":499859489} I have tried if ($request->hasFile('file')) { echo 'File Found'; } and nithing is happeningAlso Passing product_id and image file using angularJS
Cannot retrieve all request headers using PHP slim framework
I'm using PHP slim framework for a personal project. For some reason, the PSR implementation of Request in Slim apparently is filtering some headers. I am trying to set a custom CSRF token and it is not available via $request->getHeaders(). Here's one example that shows the problem: $app->get('/bar', function ($request, $response, $args) { echo "PHP's getallheaders() <br>"; foreach (getallheaders() as $name => $value) { echo "$name: $value <br>"; } echo "Slim's GetHeaders() <br>"; foreach ($request->getHeaders() as $name => $values) { foreach ($values as $value) { echo "$name: $value <br>"; } } }); I get this output: PHP's getallheaders() Host: localhost User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: null Accept-Encoding: gzip, deflate csrf_name: csrf56fc038c2f6eb csrf_value: 4e077c04dadf22377da2aebc1a8caa78 Cookie: PHPSESSID=41016nbag70gi6shq4u2tg0aq1 Connection: keep-alive Slim's GetHeaders() Host: localhost HTTP_USER_AGENT: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0 HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 HTTP_ACCEPT_LANGUAGE: null HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_COOKIE: PHPSESSID=41016nbag70gi6shq4u2tg0aq1 HTTP_CONNECTION: keep-alive I am trying to understand why the custom headers: csrf_name: csrf56fc038c2f6eb csrf_value: 4e077c04dadf22377da2aebc1a8caa78 are being removed by Slim.
It is not Slim, it is the webserver. Even though header whose name contains underscore is valid by HTTP spec, both Nginx and Apache silently drop those headers for security reasons. In general you should use only use headers containing a..zA..Z and - characters. With Apache you can still access header with underscore in their name using getallheaders() which is an alias to apache_request_headers(). With Nginx you can enable headers with underscrore in their name with underscores_in_headers on setting.
Believe it or not, the problem was that Slim does not like an underscore in a user-defined header. Once I changed csrf_name to csrfname it worked: PHP's getallheaders() Host: localhost User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: null Accept-Encoding: gzip, deflate csrfvalue: 4e077c04dadf22377da2aebc1a8caa78 csrfname: csrf56fc038c2f6eb Cookie: PHPSESSID=5aom8b5q7ottorc9279q9sh4g1 Connection: keep-alive Slim's GetHeaders() Host: localhost HTTP_USER_AGENT: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0 HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 HTTP_ACCEPT_LANGUAGE: null HTTP_ACCEPT_ENCODING: gzip, deflate HTTP_CSRFVALUE: 4e077c04dadf22377da2aebc1a8caa78 HTTP_CSRFNAME: csrf56fc038c2f6eb HTTP_COOKIE: PHPSESSID=5aom8b5q7ottorc9279q9sh4g1 HTTP_CONNECTION: keep-alive So, don't forget, remove underscores!! EDIT As explained by Mika Tuupola, the root cause is the HTTP server and not slim.
Set cookies login with curl on php
I want to scrap some data, but I need to log in. So, my idea is to copy the cookies when I log in to my program. But I don't know why, but if I'm using my program, I kept redirect to login pages. I already compared it, but the cookies are same. Here's the header if I login using my google chrome, (Copied it from request header): GET /example/data/data.jsp?date=01-Jan-2001&_=1439020103330 HTTP/1.1 Host: www.example.com Connection: keep-alive Cache-Control: max-age=0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36 Accept-Encoding: gzip, deflate, sdch Accept-Language: en-US,en;q=0.8,ms;q=0.6 Cookie: SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA; __utma=77920972.1013585791.1438786361.1438966138.1439020034.5; __utmc=77920972; __utmz=77920972.1438786423.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) My curl code: curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_VERBOSE, true); $f = fopen('request.txt', 'w'); curl_setopt($ch, CURLOPT_STDERR , $f); curl_setopt($ch, CURLOPT_HTTPHEADER,array(' Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Accept-Encoding: gzip, deflate, sdch', 'Accept-Language: en-US,en;q=0.8,ms;q=0.6', 'Cookie: SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA; __utma=77920972.1013585791.1438786361.1438966138.1439020034.5; __utmc=77920972; __utmz=77920972.1438786423.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)', 'Upgrade-Insecure-Requests: 1', 'User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36', 'X-DevTools-Emulate-Network-Conditions-Client-Id: 3A45EE97-D41F-45A3-AFCD-1540014377A7 ')); Here's my request.txt used to debug my program header: * About to connect() to www.example.com port 80 (#0) * Trying 202.43.163.203... * connected * Connected to www.example.com (202.43.163.203) port 80 (#0) > GET /example/data/data.jsp?date=01-Jan-2001&_=1439020103330 HTTP/1.1 Host: www.example.com Accept: */* Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Encoding: gzip, deflate, sdch Accept-Language: en-US,en;q=0.8,ms;q=0.6 Cookie: SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA; __utma=77920972.1013585791.1438786361.1438966138.1439020034.5; __utmc=77920972; __utmz=77920972.1438786423.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided) Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36 X-DevTools-Emulate-Network-Conditions-Client-Id: 3A45EE97-D41F-45A3-AFCD-1540014377A7 < HTTP/1.1 302 Found < Date: Sat, 08 Aug 2015 09:44:06 GMT < Server: Apache < Set-Cookie: SESSIONID=7C8779894A3CE29D4BCED4B4D311E07E.NGXA; Path=/example/; HttpOnly < Location: http://www.example.com/login.jsp < Content-Length: 0 < Content-Type: text/plain < * Connection #0 to host www.example.com left intact * Closing connection #0 This my first time scrapping data with login user, but did I miss something?
Cookies can be set via the CURLOPT_COOKIE. In your case curl_setopt($ch, CURLOPT_COOKIE, 'SESSIONID=BA4BA42C628D5C6EB959D49DB745D94A.NGXA'); With semicolon space you can add more cookies to the request. See http://php.net/manual/en/function.curl-setopt.php for more information. If you want to store and re-use the cookies you can also use CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR.
file_get_contents() with modified HTTP headers returning garbage html output
The following code is being used to extract html using SIMPLETHTMLDOM parser for php. include('simple_html_dom.php'); $context = stream_context_create(array( 'http'=>array( 'method'=>"GET", 'header'=>"Accept: text/html,application/xhtml+xml,application/xml\r\n" . "Accept-Charset: ISO-8859-1,utf-8\r\n" . "Accept-Encoding: gzip,deflate,sdch\r\n" . "Accept-Language: en-US,en;q=0.8\r\n", 'user_agent'=>"User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11\r\n" ) )); $html = file_get_contents('http://www.nseindia.com/content/equities/cmbhav.htm', false, $context); echo $html; foreach($html->find('a') as $e) echo $e->href . '<br>'; The output generated is Garbage html : ‹µVßsâ6~Ï_¡s§¹ð2¹^:ìNˆÝ†)!ið=äéFØV"K>I˜pMÿ÷®lCšéÝKy#+í¾ý´òè]t}‘ÜÝÄä2¹š’›Oãéä‚x]J'qò¥Q5š=ŸÒxæ…G£Ü À2,·ÂÙ<&]YsËÁ 8eŒ%ãœU©*·#Úئ“Ù$×°<* ôRc<¢AóänÏ/ã8!v[B`áÑRÔâ&Õ¼´D0¹Z³Þ=«X³è£SuohªŠBÉåZ¦¦wo¼pD“o*¦ 7“88ÉTº.#Ú¢ókà[ålþÓ`[ÐæÀà,8yö_¸Pãí$ëŸHVñ³J÷XYÎXAð~Ö¤¬„÷ƒXŠÜ³óôTg/ "Wª×'ñÓSmÖgm‡€åJ’œÉL#¬µÒ'ò—»Ö’X½†áßdÃe¦6=%ÁéIðÒzH0«¸ÂìÍIgxô#>W¾|°´xäÃBЖ5ãëèŽ,V©JÞËúççU0½â2ð‰Uå^n„ø*·¨j¦žÙ<𱾯ˆt˜zùº.GxöjjìV AÅ _pÁívó,9$¥2H[%la”X[":zÐ/‡¤Ù´‘¿v5xô‡uªnÄ,}G6Ì3IÎÇÓ˜¤ „)YÊå ³](FÁ-—,Ëšåö4}ßÿÆ¿wÝî! 3V‘1²rn™¶†\‚†nõÿu£æœ?ºÔí0p,³aQ{¡¤Åz“s ì_ÛÚ¬$8ó}R1ÁW2Àsï{O§7çQ4™ýŽ§t³ùÍùE3_ßFñ-.7ÕúøËi‹h°?øná£+%î…¬#ðdðáÅ^$̘Àx3—&±¶ä˜‰ÃŸ¡óB‡ÇraÊaó&m§ÉU%¶Ž6//¼Ä™s Ç+;$;+l 5Ú6†Âg.VÙº¾PÎ9Üõ¸ïô/,—U»^1ý–Lp®æ·²ËÁ{3 ¤;:Z•±ms€6RâVÈ>ÈaÓñp˜à ¶BTs,¹uu‹Úzôý±M¢FñªÀKĤÅÞ]]ÐáAgÇíMèöiö*[ûN×Ç]±ÚJì#ιÁfÈS&hüç§I2‰çôÔïŸÒ«ó[šþÏ8ºù_|$ªÞW^zï¶Ö#Â&ŒÔF Å2²ä—Hƒ¬CdD?ÿ‡\²ÅòÛ¹ ÑA.Ïø¤xÏ#¿Iè£7)à`pi¢¬Ét‰ÖÐæÄæ°gk0jB<=óšé4çºês>¿Ð.]¤Âhš ›\jvìÒ¦æ”S8>Ѻï9‰V÷±W åu»ixH-[à—º¸ÇÀ͇Å?§m2“ Where am i doing Wrong !! I checked the headers info from fiddler. These are the actual header information when the link is opened with browser !! [Client] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 Accept-Encoding: gzip,deflate,sdch Accept-Language: en-US,en;q=0.8 User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11 [Transport] Host: www.nseindia.com Proxy-Connection: keep-alive
In the HTTP request you are explicitly declaring that you can handle compressed data, so that is what the server is returning: Accept-Encoding: gzip,deflate,sdch\r\n You must now decode that compressed data: $html = gzuncompress($html); As piotrekkr mentions in a comment, you can alternatively remove the Accept-Encoding header and the web server should return plain text instead.
Content not being Posted properly?
I am using simpleTest WebBrowser for DataScraping on this URL http://www.magicbricks.com/bricks/agentSearch.html. But though everything seems right I always get the error City Field is required. I guess the problem might be with the fact that values in city field changes dynamically when value of State changes. Any solutions? Here is my code. <?php require_once('simpletest/browser.php'); $browser = &new SimpleBrowser(); $browser->addHeader('User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.2) Gecko/20100101 Firefox/6.0.2'); $browser->get('http://www.magicbricks.com/bricks/agentSearch.html'); $browser->setField('source','agentSearch'); $browser->setField('_transactionType','1'); $browser->setField('_propertyType','1'); $browser->setField('resultPerPage','50'); $browser->setField('agentSearchType','B'); $browser->setField('state','520'); $browser->setField('city','4320'); $browser->setField('keyword',''); $browser->setField('country','50'); print $browser->submitFormById('searchFormBean'); print $browser->getResponseCode() ?>
Here are some errors i noticed Field Missing Missing agentSearchType field Missing transactionType ( There is Both transactionType & _transactionType) missing propertyType ( There is both propertyType & _propertyType) There are some header information you need to add such as Referer Cookie A typical post test should come this format if you view the headers POST http://www.magicbricks.com/bricks/agentSearch.html HTTP/1.1 Host: www.magicbricks.com Connection: keep-alive Content-Length: 173 Cache-Control: max-age=0 Origin: http://www.magicbricks.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.79 Safari/535.11 Content-Type: application/x-www-form-urlencoded Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Referer: http://www.magicbricks.com/bricks/agentSearch.html Accept-Encoding: gzip,deflate,sdch Accept-Language: en-US,en;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3 Cookie: JSESSIONID=nF1UqV3DM2tZC42zByYm6Q**.MBAPP09; __utma=163479907.1423216630.1331970312.1331970312.1331970312.1; __utmb=163479907.1.10.1331970312; __utmc=163479907; __utmz=163479907.1331970312.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _mbRunstats=3k0ilrpcgprh4tea source=agentSearch&agentSearchType=B&country=51&state=601&city=8417&transactionType=11951&_transactionType=1&propertyType=10001&_propertyType=1&keyword=tesy&resultPerPage=50 I hope this helps :D