Error on PHP cURL with POST Method [duplicate] - php

Question

I want to extract data from this URL with Php Curl:
https://www.traveloka.com/en/flight/fullsearch?ap=JKTA.DPS&dt=14-10-2017.NA&ps=1.0.0&sc=ECONOMY
But I got the following error:
"{"errorType":"SERVER_ERROR","userErrorMessage":"Unexpected server error occurred. We apologize for the inconvenience. Please try again later.","errorMessage":"Exception occurred in server."}".
How I can extract the data from that link with php curl?
Here is my code
$url = 'https://api.traveloka.com/en/v2/flight/search/oneway';
$params = [
'context' => [
'tvLifetime' => 'eTq6r7InDN+j0vrg5Bujah9yFLWfBGsNGWxzjTBUa/jvVfn8fy/IF40U7OQl0vjmoqMJwuSocopqxISYLLi6YlngzuFViHSWhNHdFgs+49yydWXm5gSjBRwDBFuO0UKHd+B69Ip0Tk1qnKH+oyzW43f2GdS7QOd10yBpqoCOyOk73cVe4oyqCjYUR7X72PoHr14UQNQEUjl1NP5Mcxp+1Gw6RzKF7uV7jMRzmsYbGfGKpYLfsYtxaSx1t35KGWOO605YN9Mj2n5kP5fOD7j2KA9adtfLBtEymWXf6tEt3ug8oBVyzj5c2/pp/hboYilQnDRCih+RwhV5WX7hPTw9IsKapSNtWZ1NX8biH7UyYuhNLgcLK03OS4WNpoO+NphjOPKh09oBpUgrEJ0UqeY+1rfj98lWMAdpMO5rp2E5pvmP7HRuW6CqBwSchPLtVPQAi7ceDGYgYneH+AfodZMd5A==',
'tvSession' => '9tCFUug+5pqBk0WdAmwAbThaxD2lAm75JaxFJenJTB2MkEWW7bwVa5FW83NZnCLnlL2TAAijDDIDfD9YbC7NhRws3r5fKxPj62n1bJ+Nck309g3Rkogk+dtxsoMRpFHbkVkEJbYuNFbd9Ckp9iEBGg==',
'nonce' => '5eebdd23-2574-4465-afa7-cecc94b8f909'
],
'clientInterface' => 'desktop',
'data' => [
'currency' => 'IDR',
'destinationAirportOrArea' => 'DPS',
'flightDate' => [
'day' => '14',
'month' => '10',
'year' => '2017'
],
'isReschedule' => 'false',
'locale' => 'en_ID',
'newResult' => 'true',
'numSeats' => [
'numAdults' => '1',
'numChildren' => '0',
'numInfants' => '0'
],
'seatPublishedClass' => 'ECONOMY',
'seqNo' => 'null',
'sortFilter' => [
'filterAirlines' => [],
'filterArrive' => [],
'filterDepart' => [],
'filterTransit' => [],
'selectedDeparture' => '',
'sort' => 'null'
],
'sourceAirportOrArea' => 'JKTA',
'searchId' => 'null',
'usePromoFinder' => 'false',
'useDateFlow' => 'false'
],
'fields' => []
];
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($params));
curl_setopt($ch, CURLOPT_REFERER, "https://www.traveloka.com/en/flight/fullsearch?ap=JKTA.DPS&dt=14-10-2017.NA&ps=1.0.0&sc=ECONOMY");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
"Host: api.traveloka.com",
"Accept: application/json, text/javascript, */*; q=0.01",
"Accept-Language: en-us,en;q=0.5",
"X-Requested-With: XMLHttpRequest",
"Connection: keep-alive",
"Pragma: no-cache",
"Cache-Control: no-cache")
);
$result = curl_exec($ch);
print_r($result);
curl_close($ch);
Actually I'm not doing screen crawling, but I want to crawl its json data. I open the "Network" Tab in browser and see the XHR section, and then I want to grab the response from that XHR section. So how to do it and what's wrong with my code?

Answer 1

As per the URL you have given using the below code, you would get nothing rather the redirection link:
Replace with the link and parameters with the correct api:
General format of using CURL is given here:
$ch = curl_init();
$params = urldecode('{"ap":"' . 'JKTA.DPS' . '","dt":"' . '14-10-2017.NA' . '","ps":"' . '1.0.0'. '","sc":"' . 'ECONOMY'. '"}');
$url = 'https://www.traveloka.com/en/flight/fullsearch?JsonData=' . $params;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0");
curl_setopt($ch, CURLOPT_HEADER, 0);
// Should cURL return or print out the data? (true = return, false = print)
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
// Timeout in seconds
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
// Download the given URL, and return output
$output = curl_exec($ch);
// Close the cURL resource, and free system resources
curl_close($ch);
if (!empty($output)) {
$x = json_decode($output, true);
var_dump($x);
}else{
return FALSE;
}

Answer 2

What you want to do is called Web scraping, there are plenty of tools that you can use to do that.
For now, go for:
https://github.com/FriendsOfPHP/Goutte
or https://github.com/duzun/hQuery.php.