I need to get the location header the following request:
This is what the request looks like from the chrome network dev tool when I execute it in chrome.
here is the redirect request in chrome network dev tool, just in case its needed for some reason.
I've tried the following things:
this seems to only give me an empty answer:
<?php
$url = $_GET["url"];
if (0 === preg_match('/form_build_id" value="form-([^"]*)"/', file_get_contents('http://anything2mp3.com/'), $matches)) exit("false");
$id = $matches[1];
$params = array('http' => array(
'method' => 'POST',
'content' => array(
'url' => $url,
'op' => "Convert",
'form_build_id' => "form-" . $id,
'form_id' => "videoconverter_convert_form"
),
'max_redirects' => "0",
'header' => "Content-Type:application/x-www-form-urlencoded"
));
stream_context_set_default($params);
$headers = get_headers("http://anything2mp3.com/?q=&mobile_detect_caching_notmobile&mobile_detect_caching_nottablet", false, $ctx);
echo implode("\n", $headers);
?>
i've also tried this:
<?php
$url = $_GET["url"];
if (0 === preg_match('/form_build_id" value="form-([^"]*)"/', file_get_contents('http://anything2mp3.com/'), $matches)) exit("false");
$id = $matches[1];
$params = array('http' => array(
'method' => 'POST',
'content' => array(
'url' => $url,
'op' => "Convert",
'form_build_id' => "form-" . $id,
'form_id' => "videoconverter_convert_form"
),
'max_redirects' => "0",
'header' => "Content-Type:application/x-www-form-urlencoded"
));
$ctx = stream_context_create($params);
$file = fopen("http://anything2mp3.com/?q=&mobile_detect_caching_notmobile&mobile_detect_caching_nottablet", 'rb', false, $ctx);
$headers = $http_response_header;
echo implode("\n", $headers);
?>
the last one gives me the following result:
HTTP/1.1 200 OK
Date: Sun, 26 Jul 2015 18:20:20 GMT
Content-Type: text/html; charset=utf-8
Connection: close
Set-Cookie: __cfduid=db7807d30a61600d31a18cc1a725150811437934820; expires=Mon, 25-Jul-16 18:20:20 GMT; path=/; domain=.anything2mp3.com; HttpOnly
Vary: Accept-Encoding
X-Cookie-Domain: .anything2mp3.com
Expires: Tue, 24 Jan 1984 08:00:00 GMT
Last-Modified: Sun, 26 Jul 2015 18:20:20 GMT
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
ETag: W/"1437934820"
Content-Language: en
X-Device: normal
X-GeoIP-Country-Code: US
X-GeoIP-Country-Name: United States
X-Speed-Cache-Key: /?q=&mobile_detect_caching_notmobile&mobile_detect_caching_nottablet
X-NoCache: Method
X-This-Proto: http
X-Server-Name: anything2mp3.com
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Server: cloudflare-nginx
CF-RAY: 20c21e3333eb23a8-IAD
as you can see, there is no location header.
note that I've seen some similar posts already (like question 6532753, I couldn't link it because I'm already at my 2 link limit), but they don't seem to work for me.
How do I retrieve the location header from request like marked in the original screenshot? What am I missing?
edit: I messed up pretty hard when copying my code, sorry for that
I eventually got it to work using curl like #frz3993 suggested (and after sending in a blank 'cookies' header, the server didn't seem to like it without that)
here is the code that I use now:
<?php
$url = $_GET["url"];
if (0 === preg_match('/form_build_id" value="form-([^"]*)"/', file_get_contents('http://anything2mp3.com/'), $matches)) exit("false");
$longid = $matches[1];
$ch = curl_init("http://anything2mp3.com/?q=&mobile_detect_caching_notmobile&mobile_detect_caching_nottablet");
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query(array(
'url' => $url,
'op' => "Convert",
'form_build_id' => "form-" . $longid,
'form_id' => "videoconverter_convert_form"
)));
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
"Content-Type:application/x-www-form-urlencoded",
"Cookie:"
));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
$data = curl_exec($ch);
if (0 === preg_match('/Location: (.*id=(\d+))/', $data, $matches)) exit("false");
$redirect = $matches[1];
$id = $matches[2];
echo $redirect;
echo " - ";
echo $longid;
die();
?>
note that even tho the second suggestion from my OP gave me headers, it did not give me the location header which curl does now. (don't ask me why :p)
Related
I wrote a small script to verify if an url exists. I am using get_headers to retrieve the headers. The issue is that with some url, example this one: https://forum.obviousidea.com the response is 403 Bad Behavior, while if i open the page with browser it works.
Example output:
$headers = get_headers(https://forum.obviousidea.com);
print_r($headers);
(
[0] => HTTP/1.1 403 Bad Behavior
[Server] => nginx/1.6.2
[Date] => Tue, 04 Jun 2019 21:56:27 GMT
[Content-Type] => text/html; charset=ISO-8859-1
[Content-Length] => 913
[Connection] => close
[Set-Cookie] => Array
(
[0] => bb_lastvisit=1559685385; expires=Wed, 03-Jun-2020 21:56:25 GMT; Max-Age=31536000; path=/; secure
[1] => bb_lastactivity=0; expires=Wed, 03-Jun-2020 21:56:25 GMT; Max-Age=31536000; path=/; secure
[2] => PHPSESSID=cqtkdcfpm0k2s8hl4cup6epa37; path=/
)
[Expires] => Thu, 19 Nov 1981 08:52:00 GMT
[Cache-Control] => private
[Pragma] => private
[Status] => 403 Bad Behavior
)
How can I get the right status code using get_headers ?
Note using the user agent suggested in the answer now this website works.
But for example this url still doesn't work: https://filezilla-project.org/download.php?type=client
You may have changed the UserAgent header in php.ini or by ini_set
check it or set UserAgent like like the example below
ini_set('user_agent', '');
$headers = get_headers('https://forum.obviousidea.com');
I prefer use bellow curl function:
/**
* #param string $url
* #param array $headers
* #return array
* #throws Exception
*/
function curlGetHeaders(string $url, array $headers = [])
{
$url = trim($url);
if (!filter_var($url, FILTER_VALIDATE_URL)) {
throw new Exception("$url is not a valid URL", 422);
}
$url = explode('?', $url);
$curl = curl_init();
curl_setopt_array($curl, [
CURLOPT_URL => $url[0],
CURLOPT_RETURNTRANSFER => true,
CURLOPT_NOBODY => true,
CURLOPT_HEADER => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "GET",
]);
if (isset($url[1])) {
curl_setopt($curl, CURLOPT_POSTFIELDS, $url[0]);
}
if (!empty($headers)) {
foreach($headers as $key => $header) {
$curlHeaders[] = "$key:$header";
}
curl_setopt($curl, CURLOPT_HTTPHEADER, $curlHeaders);
}
$response = rtrim(curl_exec($curl));
$responseCode = curl_getinfo($curl, CURLINFO_RESPONSE_CODE);
curl_error($curl);
curl_close($curl);
$headers = [];
$data = explode("\r\n", $response);
$headers['Response-Code'] = $responseCode;
$headers['Status'] = $data[0];
array_shift($data);
foreach($data as $part) {
$middle = explode(":", $part, 2);
if (!isset($middle[1])) {
$middle[1] = null;
}
$headers[trim($middle[0])] = trim($middle[1]);
}
return $headers;
}
I want to fetch json data from one API but I am getting following error
HTTP/1.1 500 Internal Server Error
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 27 Mar 2019 06:07:40 GMT
Content-Length: 36
{"Message":"An error has occurred."}
Its works with postman.Is there any syntax error while sending request?
This is my code
<?php
// Initiate curl
//$post = "NoofAdult=1&NoofChild=1&NoofInfant=1&FromAirportCode=AMD&ToAirportCode=BOM&DepartureDate=21/06/2019&ReturnDate&TripType=1&FlightClass=Y&SpecialFare=0&AirlineType=A";
$postData = array(
'NoofAdult' => '1',
'NoofChild' => '1',
'NoofInfant' => '1',
'FromAirportCode' => 'AMD',
'ToAirportCode' => 'BOM',
'DepartureDate' => '21/06/2019',
'ReturnDate' => '',
'TripType' => '1',
'FlightClass' => 'Y',
'SpecialFare' => '0',
'AirlineType' => 'A'
);
$header_data = array(
"Content-Type: application/json",
"Accept-Encoding: gzip, deflate",
"InterfaceCode:1",
"InterfaceAuthKey:1",
"AgentCode:",
"Password:"
);
$ch = curl_init();
$curlOpts = array(
CURLOPT_URL => 'http://stagingv2.flightmyweb.com/API/FlightAvailibility',
//CURLOPT_URL => 'http://localhost/akshay/sampleapi.php',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => $header_data,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $postData,
CURLOPT_HEADER => 1,
);
curl_setopt_array($ch, $curlOpts);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length: 0'));
$answer = curl_exec($ch);
// If there was an error, show it
if (curl_error($ch)) {
die(curl_error($ch));
}
curl_close($ch);
echo '<pre>';
print_r($answer);
echo '</pre>';
// Will dump a beauty json :3
//var_dump(json_decode($result, true));
//echo json_encode($outp);
?>
I need json output:
i can't get it to connect my php to my spiders
$url = 'https://app.scrapinghub.com/api/jobs/list.json';
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HEADER => 1,
CURLOPT_URL => $url,
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => array(
'project' => 'projecid',
'spider' => 'spiderid',
),
CURLOPT_USERPWD => "apikey")
);
$response = curl_exec($ch);
curl_close($ch);
echo '<pre>';
print $response;
echo '</pre>';
it returns:
HTTP/1.1 100 Continue
HTTP/1.1 400 method not allowed
Server: nginx/1.10.1
Date: Fri, 07 Jul 2017 07:24:02 GMT
Content-Type: application/json
Content-Length: 57
Connection: keep-alive
Vary: Cookie
Set-Cookie: csrftoken2=67xgaAQB9ytsIYNxseDFAIUzkCivPZMda74Hvg7UNMp6iD3zALRDWP6zhknxiEIP; Domain=.scrapinghub.com; expires=Fri, 06-Jul-2018 07:24:02 GMT; Max-Age=31449600; Path=/; Secure
X-Upstream: dash-master_apiv1
{"status": "badrequest", "message": "method not allowed"}
it says bad request means i am wrong with my url but in the docs it said it should be like these or i am wrong on sending my apikey.
please help me here thanks.
You are sending a POST request to scrapinghub which is the message of your badrequest status as method not allowed. instead you can try sending a GET request through PHP curl like so
$url = 'https://app.scrapinghub.com/api/jobs/list.json';
$ch = curl_init();
$curl_opts = array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HEADER => 1,
CURLOPT_URL => $url."?project=projectid&spider=spiderid",
CURLOPT_USERPWD => "apikey"
);
curl_setopt_array($ch, $curl_opts);
$response = curl_exec($ch);
curl_close($ch);
echo '<pre>';
print $response;
echo '</pre>';
I'm working on attaching files via PHP to confluence (version 5.9.10)
Here is my code
$ch=curl_init();
$headers = array(
'X-Atlassian-Token: no-check'
);
$data = array('file' => '#test.txt');
curl_setopt_array(
$ch,
array(
CURLOPT_URL=>'https://<path_to_confluence>/rest/api/content/<page_id>/child/attachment',
CURLOPT_POST=>true,
CURLOPT_VERBOSE=>1,
CURLOPT_POSTFIELDS=>$data,
CURLOPT_SSL_VERIFYHOST=> 0,
CURLOPT_SSL_VERIFYPEER=> 0,
CURLOPT_RETURNTRANSFER=>true,
CURLOPT_HEADER=>false,
CURLOPT_HTTPHEADER=> $headers,
CURLOPT_USERPWD=>C_USERNAME.":".C_PASSWORD
)
);
$result=curl_exec($ch);
$ch_error = curl_error($ch);
if ($ch_error) {
echo "cURL Error: $ch_error";
} else {
var_dump($result);
}
curl_close($ch);
But after running script I've next error:
>
HTTP/1.1 500 Internal Server Error
Server: nginx/1.5.12
Date: Thu, 03 Nov 2016 10:12:44 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
X-ASEN: SEN-2160053
Set-Cookie: JSESSIONID=EE3116DFC552C7D4571608BFCF410559; Path=/; HttpOnly
X-Seraph-LoginReason: OK
X-AUSERNAME: user
Cache-Control: no-cache, must-revalidate
Expires: Thu, 01 Jan 1970 00:00:00 GMT
X-Content-Type-Options: nosniff
HTTP error before end of send, stop sending
Closing connection 0
string(93) "statusCode":500,"message":"java.lang.IllegalArgumentException:File name must not be null"
What I'm doing wrong? What have I missed?
Try this:
$data = array('file' => '#' . realpath('test.txt'));
...
CURLOPT_POSTFIELDS=>$data
If the filename is required, you can try this instead:
$data = array('file' => '#' . realpath('test.txt') . ';filename=test.txt'));
...
CURLOPT_POSTFIELDS=>$data
I encountered this error when I was using the lib cURL and was doing it the old way, like:
CURLOPT_POSTFIELDS => [
'file' => '#/pathToFile'
]
But since it's deprecated from PHP 5.5, you need to use it the new way:
Imperative way:
CURLOPT_POSTFIELDS => [
'file' => curl_file_create($filePath)
]
Object way:
CURLOPT_POSTFIELDS => [
'file' => new CurlFile($filePath)
]
I'm using the following code to post information to a URL.
$query = http_build_query($myvars);
$options = array(
'http' => array(
'header' => "Content-Type: application/x-www-form-urlencoded\r\n".
"Content-Length: ".strlen($query)."\r\n".
"User-Agent:MyAgent/1.0\r\n",
'method' => "POST",
'content' => $query,
),
);
$context = stream_context_create($options);
$response = file_get_contents($url, false, $context);
Is it possible to show the complete header information of the response.
First I used curl, but this took to much cpu.
With curl I used the following option:
curl_setopt($ch, CURLOPT_HEADER, 1);
And I received the following header information:
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Date: Mon, 21 Sep 2015 07:06:35 GMT
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By: PHP/5.5.9-1ubuntu4.11
Content-Description: File Transfer
Content-Disposition: attachment; filename=File.txt
Content-Transfer-Encoding: binary
Content-Length: 333
Cache-Control: must-revalidate, post-check=0, pre-check=0
Expires: 0
Pragma: public
Vary: Accept-Encoding
Content-Type: text/plain
Is the with file_get_contents also possible?
file_get_contents("http://example.com");
var_dump($http_response_header);
http://php.net/manual/en/reserved.variables.httpresponseheader.php