Recursive call until value is not more on response - php

Question

I have a cURL call to this services/data/v28.0/query/?q=SELECT Id,Name,LastModifiedDate FROM Territory and the response looks like:
{
"totalSize": 6911,
"done": false,
"nextRecordsUrl": "/services/data/v28.0/query/01g8000002eI8dMAAS-2000",
"records": [ ... ]
}
That means complete set of records will have 6911 items but on first request just the first 2000 are return, now if I change a bit the cURL call into this /services/data/v28.0/query/01g8000002eI8dMAAS-2000 I will get the next 2000 items for the first 6911 and so on.
I should call the cURL recursive until done = true or nextRecordsUrl will be NULL or doesn't exists anymore on the response, how? I need some advice or pseudo-code that illustrate me how to achieve this since I am a bit stucked.
This shows the complete set of recursive calls I should perform (this could be dynamic so hard code won't work):
call:
/services/data/v28.0/query/?q=SELECT Id,Name,LastModifiedDate FROM Territory
response:
{
"totalSize": 6911,
"done": false,
"nextRecordsUrl": "/services/data/v28.0/query/01g8000002eI8dMAAS-2000",
"records": [ ... ]
}
call:
/services/data/v28.0/query/01g8000002eI8dMAAS-2000
response:
{
"totalSize": 6911,
"done": false,
"nextRecordsUrl": "/services/data/v28.0/query/01g8000002eI8dMAAS-4000",
"records": [ ... ]
}
call:
/services/data/v28.0/query/01g8000002eI8dMAAS-4000
response:
{
"totalSize": 6911,
"done": false,
"nextRecordsUrl": "/services/data/v28.0/query/01g8000002eI8dMAAS-6000",
"records": [ ... ]
}
call:
/services/data/v28.0/query/01g8000002eI8dMAAS-6000
response:
{
"totalSize": 6911,
"done": true,
"records": [ ... ]
}
UPDATE
I have figured out, more or less, how to achieve this but now the my problem turns in how to merge recursive the values from each response. See code below:
$soqlQuery2 = "SELECT Id,Name,LastModifiedDate FROM Territory";
$soqlUrl2 = $instanceUrl.'/services/data/v28.0/query/?q='.urlencode($soqlQuery2);
$curl = curl_init($soqlUrl2);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array("Authorization: OAuth $veevaToken"));
$jsonResponse = curl_exec($curl);
curl_close($curl);
$soqlObj2 = json_decode($jsonResponse, true);
$this->performPaginateSOQLQuery($veevaToken, $instanceUrl, $tokenUrl, $soqlObj2['nextRecordsUrl']);
Above in $soqlObj2['done'] and $soqlObj2['nextRecordsUrl'] I will have the values I need for the second and recursive calls. So, I built this function:
public function performPaginateSOQLQuery($veevaToken, $instanceUrl, $tokenUrl, $nextRecordsUrl)
{
if (isset($nextRecordsUrl) && $nextRecordsUrl !== null && $nextRecordsUrl !== "") {
$nextRecordsUrlSOQLQuery = $instanceUrl.$nextRecordsUrl;
$curl = curl_init($nextRecordsUrlSOQLQuery);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array("Authorization: OAuth $veevaToken"));
$jsonResponse = curl_exec($curl);
$status = curl_getinfo($curl, CURLINFO_HTTP_CODE);
if ($status != 200) {
$respObj['error'] = "Error: call to token URL $tokenUrl failed with status $status, response $jsonResponse, curl_error ".curl_error(
$curl
).", curl_errno ".curl_errno($curl);
return $respObj;
}
curl_close($curl);
$nextRecordsUrlObj = json_decode($jsonResponse, true);
while ($nextRecordsUrlObj['done'] !== true) {
$this->performPaginateSOQLQuery($veevaToken, $instanceUrl ,$tokenUrl, $nextRecordsUrlObj['nextRecordsUrl']);
}
return $nextRecordsUrlObj;
}
}
But I need to merge the whole $soqlObj2 with $nextRecordsUrlObj from each iteration on performPaginateSOQLQuery() call, how? Any help?
UPDATE 2: Losing values on 1st iteration
I've updated my code to this:
public function performPaginateSOQLQuery(
$veevaToken,
$instanceUrl,
$tokenUrl,
&$nextRecordsUrl,
&$dataToSave = array()
) {
if (isset($nextRecordsUrl) && $nextRecordsUrl !== null && $nextRecordsUrl !== "") {
$nextRecordsUrlSOQLQuery = $instanceUrl.$nextRecordsUrl;
$curl = curl_init($nextRecordsUrlSOQLQuery);
curl_setopt($curl, CURLOPT_HEADER, false);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl, CURLOPT_HTTPHEADER, array("Authorization: OAuth $veevaToken"));
$jsonResponse = curl_exec($curl);
$status = curl_getinfo($curl, CURLINFO_HTTP_CODE);
if ($status != 200) {
$respObj['error'] = "Error: call to token URL $tokenUrl failed with status $status, response $jsonResponse, curl_error ".curl_error(
$curl
).", curl_errno ".curl_errno($curl);
return $respObj;
}
curl_close($curl);
$nextRecordsUrlObj = json_decode($jsonResponse, true);
echo $nextRecordsUrlObj['nextRecordsUrl'] . "\n";
print_r($nextRecordsUrlObj);
$dataToSave[] = $nextRecordsUrlObj;
$i = 0;
while ($nextRecordsUrlObj['done'] !== true) {
echo "time ".$i;
$i++;
$this->performPaginateSOQLQuery(
$veevaToken,
$instanceUrl,
$tokenUrl,
$nextRecordsUrlObj['nextRecordsUrl'],
$dataToSave
);
}
return array('url' => $nextRecordsUrlObj, 'data' => $dataToSave);
}
}
But on iteration time 1 I am getting this error:
[Symfony\Component\Debug\Exception\ContextErrorException]
Notice: Undefined index: nextRecordsUrl
Why is that?

Answer 1

You can use a while loop with a boolean variable.
Once the response you get is the last one (done property is true) change the value of that boolean variable and the loop should stop.
<?php
//Assuming $response contains the response you're getting
//and that it's JSON-encoded.
$keepRequesting = true;
while($keepRequesting){
//Your curl code over here
// ...
$response = json_decode($response);
if($response->done == true) {
$keepRequesting = false;
}
}
Regarding your update:
While I don't think you should use recursive methods for this problem,
just add the data you want to "save" as a reference to the function and use array_merge. Something like that:
public function performPaginateSOQLQuery($veevaToken, $instanceUrl, $tokenUrl, $nextRecordsUrl, &$dataToSave = array()) { //1 new parameters.
//More code here...
$nextRecordsUrlObj = json_decode($jsonResponse, true);
$dataToSave[] = $nextRecordsUrlObj;
while ($nextRecordsUrlObj['done'] !== true) {
$this->performPaginateSOQLQuery($veevaToken, $instanceUrl ,$tokenUrl, $nextRecordsUrlObj['nextRecordsUrl'], $dataToSave);
}
return array('url' => $nextRecordsUrlObj, 'data' => $dataToSave);
The problem is that you should return it, otherwise it will get lost.
You'll have to change the function a bit so it would return the URL and the data you want.
Maybe this post would help you:
php recursion global variable?