Generate multiple guid in PHP for firebase database - php

Question

I have 70,000 rows in a MySQL table. I am trying to upload them in Firebase via import JSON. I have exported the table into JSON format. The output was an array. e.g., JSON is as follows -
[
{"question_id":"99","question":"What is your name?"},
{"question_id":"200","question":"What do you do?"}
]
For the correct use of Firebase in mobile apps, I need to import this JSON data as an object, along with GUID like following -
{
"-Lf64AvZinbjvEQLMzGc" : {
"question_id" : 99,
"question" : "What is your name?"
},
"-Lf64AvZinbjvEQLMzGd" : {
"question_id" : 200,
"question" : "What do you do?"
}
}
Since the number of rows are 70,000 (JSON file of this data is 110 MB); inserting individually into Firebase is not possible. So I was trying for a way to generate 70,000 GUID and editing the JSON file to add each one before every object. But, I am stuck at both places. I am using the following class (PushId.php) for generating GUID -
<?php
/**
* Fancy ID generator that creates 20-character string identifiers with the following properties:
*
* 1. They're based on timestamp so that they sort *after* any existing ids.
* 2. They contain 72-bits of random data after the timestamp so that IDs won't collide with other clients' IDs.
* 3. They sort *lexicographically* (so the timestamp is converted to characters that will sort properly).
* 4. They're monotonically increasing. Even if you generate more than one in the same timestamp, the
* latter ones will sort after the former ones. We do this by using the previous random bits
* but "incrementing" them by 1 (only in the case of a timestamp collision).
*/
class PushId
{
/**
* Modeled after base64 web-safe chars, but ordered by ASCII.
*
* #var string
*/
const PUSH_CHARS = '-0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz';
/**
* Timestamp of last push, used to prevent local collisions if you push twice in one ms.
*
* #var int
*/
private static $lastPushTime = 0;
/**
* We generate 72-bits of randomness which get turned into 12 characters and appended to the
* timestamp to prevent collisions with other clients. We store the last characters we
* generated because in the event of a collision, we'll use those same characters except
* "incremented" by one.
*
* #var array
*/
private static $lastRandChars = [];
/**
* #return string
*/
public static function generate()
{
$now = (int) microtime(true) * 1000;
$isDuplicateTime = ($now === static::$lastPushTime);
static::$lastPushTime = $now;
$timeStampChars = new SplFixedArray(8);
for ($i = 7; $i >= 0; $i--) {
$timeStampChars[$i] = substr(self::PUSH_CHARS, $now % 64, 1);
// NOTE: Can't use << here because javascript will convert to int and lose the upper bits.
$now = (int) floor($now / 64);
}
static::assert($now === 0, 'We should have converted the entire timestamp.');
$id = implode('', $timeStampChars->toArray());
if (!$isDuplicateTime) {
for ($i = 0; $i < 12; $i++) {
$lastRandChars[$i] = floor(rand(0, 64));
}
} else {
// If the timestamp hasn't changed since last push, use the same random number, except incremented by 1.
for ($i = 11; $i >= 0 && static::$lastRandChars[$i] === 63; $i--) {
static::$lastRandChars[$i] = 0;
}
static::$lastRandChars[$i]++;
}
for ($i = 0; $i < 12; $i++) {
$id .= substr(self::PUSH_CHARS, $lastRandChars[$i], 1);
}
static::assert(strlen($id) === 20, 'Length should be 20.');
return $id;
}
/**
* #param bool $condition
* #param string $message
*/
private static function assert($condition, $message = '')
{
if ($condition !== true) {
throw new RuntimeException($message);
}
}
}
Following is the PHP code that I wrote to generate 70,000 GUID but it showed error (while the same code is working if I use it to generate 1 GUID only)
require_once('PushId.php');
$vars = new PushId();
$my_file = 'TheGUIDs.txt';
$handle = fopen($my_file, 'w') or die('Cannot open file: '.$my_file);
$i=1;
while($i <= 70000){
$data = $vars->generate();
fwrite($handle, $data);
echo $data;
$i++;
}
fclose($handle);
Edit -1
The error I am getting in generating GUID is -
Notice: Undefined offset: 11 in C:\wamp64\www\firebase-json\PushId.php on line 65
Notice: Undefined variable: lastRandChars in C:\wamp64\www\firebase-json\PushId.php on line 71
The first GUID is complete, e.g. in last run I got -Lf8WPlkkNTUjYkIP4WT, but then I got incomplete GUID -Lf8WPlk------------

Answer 1

First, you forgot to prepend static:: to $lastRandChars in some places:
for ($i = 0; $i < 12; $i++) {
$lastRandChars[$i] = floor(rand(0, 64)); // << here
}
for ($i = 0; $i < 12; $i++) {
$id .= substr(self::PUSH_CHARS, $lastRandChars[$i], 1); // << and here
}
Next:
When incrementing previous ID, you missed that you should increment not only last char, but all previous chars as well if they are more than 63.
Sample IDs sequence:
...
-Lf64AvZinbjvEQLMzGw
-Lf64AvZinbjvEQLMzGx
-Lf64AvZinbjvEQLMzGy
-Lf64AvZinbjvEQLMzGz
-Lf64AvZinbjvEQLMzH- <<< on this step last char is reset, and previous one is incremented by one
-Lf64AvZinbjvEQLMzH0
-Lf64AvZinbjvEQLMzH1
...
-Lf64AvZinbjvEQLMzzw
-Lf64AvZinbjvEQLMzzx
-Lf64AvZinbjvEQLMzzw
-Lf64AvZinbjvEQLMzzz
-Lf64AvZinbjvEQLN--- <<< on this step last three chars are reset, and previous one is incremented by one
-Lf64AvZinbjvEQLN--0
-Lf64AvZinbjvEQLN--1
...
Modified increment logic:
// If the timestamp hasn't changed since last push, use the same random number, except incremented by 1.
for ($i = 11; $i >= 0; $i--) {
$previousIncremented = false;
for ($j = $i; $j > 0; $j--) {
if (!$previousIncremented) {
static::$lastRandChars[$j]++;
}
if (static::$lastRandChars[$j] == 64) {
static::$lastRandChars[$j] = 0;
static::$lastRandChars[$j - 1]++;
$previousIncremented = true;
} else {
break 2;
}
}
}