Parse.com greater than returning duplicates - php

Question

I'm using this PHP parse.com API library to retrieve rows from a table from Parse.
Due to the limit of maximum 1000 rows per request, I'm retrieving them in a for loop like this:
$lastUpdated = null;
$parse = new parseQuery($tableName);
$parseAllResults = array();
$skip = 0;
do {
$index = count($parseAllResults) - 1;
if($skip === 10000) {
$lastUpdated = $parseAllResults[$index]['updatedAt'];
$skip = 0;
}
$parseResults = queryParseCrawlObjects($lastUpdated, $skip);
if (!empty($parseResults)) {
$skip += 1000;
} else {
$skip = 0;
}
$parseAllResults = array_merge($parseAllResults, $parseResults);
} while ($skip > 0);
function queryParseCrawlObjects($parse, $lastUpdated, $skip) {
global $parse;
date_default_timezone_set('UTC');
$parse->orderBy('updatedAt');
if ($lastUpdated != null) {
$parse->whereGreaterThan('updatedAt', $parse->dataType('date', $lastUpdated));
} else {
$parse->whereNotEqualTo('objectId', '');
}
$parse->setLimit(1000);
$parse->setSkip($skip);
$results = $parse->find();
return $results['results'];
}
I'm using the whereNotEqualTo('objectId', ''); restriction as a workaround for a bug in the library, that practically retrieves all the rows, and the skip parameter ro retrieve them in a batch of 1000 rows.
Another limitation of Parse is that it doesn't allow a skip greater than 10.000. So I use the updatedAt field of the last row in the first 10.000 batch, as a restriction for the next rows.
And after the first 10.000 rows, it calls the whereGreaterThan method, that internally adds the gt (greater than) Parse parameter to the curl request.
The problem is that in the second loop step, it returns the same row as the last one from the previous step, and I get an error for duplicate objectId when I try to insert them into a database.
So the array looks something like this:
// first step
$parseAllResults[0] = array('objectId' => 'ihJikHNkjH', ...);
$parseAllResults[1] = array('objectId' => 'sHJKHfddkO', ...);
...
$parseAllResults[9999] = array('objectId' => 'rukBfcaDFR', ...);
// second step
$parseAllResults[10000] = array('objectId' => 'rukBfcaDFR', ...);
$parseAllResults[10001] = array('objectId' => 'gusFGvQWVs', ...);
...
$parseAllResults[19999] = array('objectId' => 'asHppNVAaD', ...);
with the 9999th and 10000th having the rest of the properties equal, so I'm sure it's the same row from Parse retrieved twice.
I don't understand why it does that, since it has a whereGreaterThanOrEqualTo method too, using the gte (greater than or equal to) Parse parameter.
TL;DR
greater than (gt) parameter behaves exactly like greater than or equal to (gte) parameter.

Answer 1

This might fix your issue. Change this part of your code:
if($skip === 10000) {
$lastUpdated = $parseAllResults[$index]['updatedAt'];
$skip = 0;
}
to:
if($skip === 10000) {
$lastUpdated = $parseAllResults[$index]['updatedAt'];
$skip = 1;
}