i can't get it to connect my php to my spiders
$url = 'https://app.scrapinghub.com/api/jobs/list.json';
$ch = curl_init();
curl_setopt_array($ch, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HEADER => 1,
CURLOPT_URL => $url,
CURLOPT_POST => 1,
CURLOPT_POSTFIELDS => array(
'project' => 'projecid',
'spider' => 'spiderid',
),
CURLOPT_USERPWD => "apikey")
);
$response = curl_exec($ch);
curl_close($ch);
echo '<pre>';
print $response;
echo '</pre>';
it returns:
HTTP/1.1 100 Continue
HTTP/1.1 400 method not allowed
Server: nginx/1.10.1
Date: Fri, 07 Jul 2017 07:24:02 GMT
Content-Type: application/json
Content-Length: 57
Connection: keep-alive
Vary: Cookie
Set-Cookie: csrftoken2=67xgaAQB9ytsIYNxseDFAIUzkCivPZMda74Hvg7UNMp6iD3zALRDWP6zhknxiEIP; Domain=.scrapinghub.com; expires=Fri, 06-Jul-2018 07:24:02 GMT; Max-Age=31449600; Path=/; Secure
X-Upstream: dash-master_apiv1
{"status": "badrequest", "message": "method not allowed"}
it says bad request means i am wrong with my url but in the docs it said it should be like these or i am wrong on sending my apikey.
please help me here thanks.
You are sending a POST request to scrapinghub which is the message of your badrequest status as method not allowed. instead you can try sending a GET request through PHP curl like so
$url = 'https://app.scrapinghub.com/api/jobs/list.json';
$ch = curl_init();
$curl_opts = array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_FOLLOWLOCATION => 1,
CURLOPT_HEADER => 1,
CURLOPT_URL => $url."?project=projectid&spider=spiderid",
CURLOPT_USERPWD => "apikey"
);
curl_setopt_array($ch, $curl_opts);
$response = curl_exec($ch);
curl_close($ch);
echo '<pre>';
print $response;
echo '</pre>';
Related
I am trying to get the response code from the response header using cURL PHP.
When I send the request, this is the response header that is returned by MYOB AccountRight API:
HTTP/1.1 200 OK
Access-Control-Expose-Headers: Request-Context
Cache-Control: must-revalidate, private
Content-Encoding: gzip
Content-Type: application/json;charset=utf-8
Date: Thu, 20 May 2021 01:07:56 GMT
ETag: "XXXXXXXXX"
Expires: -1
Request-Context: appId=cid-v1:a4936349-ef26-4f8a-9268-XXXXXXXXX
Server: Microsoft-IIS/10.0
Vary: Accept-Encoding
X-AspNet-Version: 4.0.30319
X-Mashery-Message-ID: 2fc6b494-54e8-43e2-8bc4-XXXXXXXXX
X-Mashery-Responder: prod-j-worker-ap-southeast-2b-33.mashery.com
x-myobapi-elapsed: 1370
x-myobapi-requestid: bb0764c8-f62d-4848-bcae-XXXXXXXXX
X-Powered-By: ASP.NET
Content-Length: 1205
Connection: keep-alive
I have tried the solution from Getting HTTP code in PHP using curl , but I will not get the http code.
This is my code to get the accounts data:
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'https://ar1.api.myob.com/accountright/766d620e-a5eb-41c3-8343-XXXXXXXX/GeneralLedger/Account?$filter=Name%20eq%20\'Inventory\'%20or%20Name%20eq%20\'Cost%20Of%20Sales\'%20or%20Name%20eq%20\'Inventory%20Income\'',
CURLOPT_HEADER => true,
CURLOPT_NOBODY => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_HTTPHEADER => array(
'x-myobapi-version: v2',
'Accept-Encoding: gzip,deflate',
'x-myobapi-key: '.$theAPIKey,
'x-myobapi-cftoken: '.$theCFToken,
'Authorization: Bearer '.$theAccessToken
)
));
$response = curl_exec($curl);
$theInfo = curl_getinfo($response);
$http_code = $theInfo['http_code'];
curl_close($curl);
echo 'http code: ' . $http_code . '<br />';
echo '<pre>';
echo $response;
echo '</pre>';
When I echo the http code, nothing will be printed.
I think you need to pass $curl to the curl_getinfo method, not the $response
$response = curl_exec($curl);
$theInfo = curl_getinfo($curl);
$http_code = $theInfo['http_code'];
You can see the doco here.. https://www.php.net/manual/en/function.curl-getinfo.php
I want to fetch json data from one API but I am getting following error
HTTP/1.1 500 Internal Server Error
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 27 Mar 2019 06:07:40 GMT
Content-Length: 36
{"Message":"An error has occurred."}
Its works with postman.Is there any syntax error while sending request?
This is my code
<?php
// Initiate curl
//$post = "NoofAdult=1&NoofChild=1&NoofInfant=1&FromAirportCode=AMD&ToAirportCode=BOM&DepartureDate=21/06/2019&ReturnDate&TripType=1&FlightClass=Y&SpecialFare=0&AirlineType=A";
$postData = array(
'NoofAdult' => '1',
'NoofChild' => '1',
'NoofInfant' => '1',
'FromAirportCode' => 'AMD',
'ToAirportCode' => 'BOM',
'DepartureDate' => '21/06/2019',
'ReturnDate' => '',
'TripType' => '1',
'FlightClass' => 'Y',
'SpecialFare' => '0',
'AirlineType' => 'A'
);
$header_data = array(
"Content-Type: application/json",
"Accept-Encoding: gzip, deflate",
"InterfaceCode:1",
"InterfaceAuthKey:1",
"AgentCode:",
"Password:"
);
$ch = curl_init();
$curlOpts = array(
CURLOPT_URL => 'http://stagingv2.flightmyweb.com/API/FlightAvailibility',
//CURLOPT_URL => 'http://localhost/akshay/sampleapi.php',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => $header_data,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $postData,
CURLOPT_HEADER => 1,
);
curl_setopt_array($ch, $curlOpts);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length: 0'));
$answer = curl_exec($ch);
// If there was an error, show it
if (curl_error($ch)) {
die(curl_error($ch));
}
curl_close($ch);
echo '<pre>';
print_r($answer);
echo '</pre>';
// Will dump a beauty json :3
//var_dump(json_decode($result, true));
//echo json_encode($outp);
?>
I need json output:
<?php
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => "",
//return the transfer as a string
CURLOPT_RETURNTRANSFER => true,
//enable headers
CURLOPT_HEADER => true,
//get only headers
CURLOPT_NOBODY => true,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_HTTPHEADER => array(
"Accept: application/json",
"password: ",
"username: "
),
));
// $response contains the output string
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
print_r($response);
?>
I am trying to take $response and truncate data from it.My output of $response is
HTTP/1.1 200 OK Server: Apache-Coyote/1.1 access-token: a3581f021476e4fb406c6b7c79e4095cece5d0c6 token-expiration-hours: 24 Content-Type: application/json Content-Length: 0 Date: Tue, 21 Aug 2018 14:12:27 GMT
I would like $response to just be the value of the access-token
a3581f021476e4fb406c6b7c79e4095cece5d0c6
One option is to use regex to find the access token:
preg_match('/access-token: ([^\s]+) /', $response, $matches);
vaR_dump($matches[1]); //a3581f021476e4fb406c6b7c79e4095cece5d0c6
This will match any string after the string access-token, up to any whitespace. Just make sure the access-token is not the last item in the headers as there probably won't be any whitespace...
Assumption: the access token will never contain any white space.
Using this Regular Expression
(?<=access-token: )[a-z0-9]* you can get everything after 'access_token: ' until the next space:
<?php
$header = "HTTP/1.1 200 OK Server: Apache-Coyote/1.1 access-token: a3581f021476e4fb406c6b7c79e4095cece5d0c6 token-expiration-hours: 24 Content-Type: application/json Content-Length: 0 Date: Tue, 21 Aug 2018 14:12:27 GMT";
preg_match('/(?<=access-token: )[a-z0-9]*/', $header, $matches);
$access_token = $matches[0];
echo $access_token;
Another option is to explode by whitespace, find the key for 'access-token', and the value will be the next in the array:
$data = explode(' ', $response);
$key = array_search('access-token:', $data); //5
var_dump($data[$key + 1]); //a3581f021476e4fb406c6b7c79e4095cece5d0c6
Use regex like Inazo suggests as it's a better solution, but you can do:
$data = 'HTTP/1.1 200 OK Server: Apache-Coyote/1.1 access-token: a3581f021476e4fb406c6b7c79e4095cece5d0c6 token-expiration-hours: 24 Content-Type: application/json Content-Length: 0 Date: Tue, 21 Aug 2018 14:12:27 GMT';
$explodedData = explode("access-token: ", $data);
$explodedData2 = explode(" token-expiration-hours:", $explodedData[1]);
echo $explodedData2[0]; //contains access token.
Based on this question, I have extracted, the token of a twitter JS file, without using the library:
https://github.com/divinity76/hhb_.inc.php/blob/master/hhb_.inc.php
I believe, that only with cURL solves, see below my code:
<?php
require_once './system/config.php';
$TwitterUser = null;
if (isset($_SESSION[SITE_TITLE . '_SESSION'])) {
$TwitterUser = $_SESSION[SITE_TITLE . '_SESSION'];
}
$twitter_url_js = 'https://abs.twimg.com/k/pt/init.pt.8a8c7bc568e38012a94b.js';
$getToken = curl_init();
curl_setopt_array($getToken, [
CURLOPT_URL => $twitter_url_js,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => 2,
CURLOPT_USERAGENT => $_SERVER['HTTP_USER_AGENT'],
CURLOPT_HEADER => true,
]
);
$token = curl_exec($getToken);
preg_match('/\"([A-z0-9%]{114})\";/', $token, $matches);
$auth_token = $matches[1];
$friend_post = http_build_query([
'screen_name' => $TwitterUser
]
);
$twitter_friend_url = 'https://api.twitter.com/1.1/friendships/create.json';
curl_setopt_array($getToken, [
CURLOPT_URL => $twitter_friend_url,
CURLOPT_CUSTOMREQUEST => 'POST',
CURLOPT_POSTFIELDS => $friend_post,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => 2,
CURLOPT_CAINFO => ROOT . 'system' . SEPARATOR . 'cacert' . SEPARATOR . 'ca-bundle.pem',
CURLOPT_USERAGENT => $_SERVER['HTTP_USER_AGENT'],
CURLOPT_HEADER => true,
]
);
$friend = curl_exec($getToken);
var_dump($friend);
I received this as an answer:
'HTTP/1.1 400 Bad Request
content-length: 62
content-type: application/json; charset=utf-8
date: Sun, 16 Jul 2017 23:56:55 GMT
server: tsa_d
set-cookie: guest_id=v1%3A150024941521507075; Domain=.twitter.com; Path=/; Expires=Tue, 16-Jul-2019 23:56:55 UTC
strict-transport-security: max-age=631138519
x-connection-hash: 2d6dd7875837513960a72f9bfc09724b
x-response-time: 131
x-tsa-request-body-time: 0
{"errors":[{"code":215,"message":"Bad Authentication data."}]}' (length=472)
How to use this token now? NOTE: I am not using API.
This is against Twitter's developer policy and your application and IP risk being banned from the platform. Furthermore, Twitter's rules explicitly prohibit the kind of application you are building - see https://support.twitter.com/articles/20171936
Only applications using the official Twitter API are supported.
I work with boomerang, but it's more or less the same as postman (i think).
With Boomerang i can do the call normally, but when i tried to do the same with php i can't.
here the two headers:
BOOMERANG / POSTMAN
GET https://example.es/api/bla/bloum/174 HTTP/1.1
Authorization: Basic ajklfsdkjfalksjdflñaskjdflakjdf=
Server: Apache-Coyote/1.1
X-Result-MaxResults: 1
X-Result-CurrentPage: 1
X-Result-MaxPages: 1
Content-Type: application/json;charset=UTF-8
Date: Fri, 28 Oct 2011 08:24:22 GMT
Transfer-Encoding: chunked
MY CODE
GET /api/bla/bloum/174 HTTP/1.1
Host: example.es
Accept: */*
Authorization: Basic ajklfsdkjfalksjdflñaskjdflakjdf=
Server: Apache-Coyote/1.1
X-Result-MaxResults: 1
X-Result-CurrentPage: 1
X-Result-MaxPages: 1
Content-Type: application/json;charset=UTF-8
Date: Fri, 28 Oct 2011 08:24:22 GMT
Transfer-Encoding: chunked
I can't put the url in the same way that Boomerang show, and i don't know if it's the problem.
POSTMAN CODE
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => "https://example.es/api/bla/bloum/174",
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYHOST => false,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_ENCODING => "",
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => "GET",
CURLOPT_HTTPHEADER => array(
"authorization: Basic ajklfsdkjfalksjdflñaskjdflakjdf=",
"cache-control: no-cache",
"content-type: application/json;charset=UTF-8",
"date: Fri, 28 Oct 2011 08:24:22 GMT",
"postman-token: FJKFK",
"server: Apache-Coyote/1.1",
"transfer-encoding: chunked",
"x-result-currentpage: 1",
"x-result-maxpages: 1",
"x-result-maxresults: 1"
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
echo "cURL Error #:" . $err;
} else {
echo $response;
}
UPDATE
$error = curl_error($ch); // "Empty reply from server"
$httpCode = curl_getinfo($ch , CURLINFO_HTTP_CODE); // 0
UPDATE 2.0
I use POSTMAN to genereate the code and now the error is:
"Operation timed out after 30001 milliseconds with 0 bytes received"
without:
CURLOPT_TIMEOUT => 30 //"Empty reply from server"
This code works:
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'https://example.es/api/asd/zxc/123',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_SSL_VERIFYHOST => false,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
CURLOPT_HTTPHEADER => array(
'authorization: Basic ghfgfiytfgfigfkgfkgjkbvjkhgkjhgkjhg',
'cache-control: no-cache',
'content-type: application/json;charset=UTF-8',
'server: Apache-Coyote/1.1',
'x-result-currentpage: 3',
'x-result-maxpages: 1',
'x-result-maxresults: 1'
),
));
$response = curl_exec($curl);
$err = curl_error($curl);
curl_close($curl);
if ($err) {
$this->logger->error($err);
}
But i don't know why, sorry.
I export it with POSTMAN, and the second time it works...