Uncaught TypeError: DOMDocument::importNode() - php

Question

I get the following error when running php on my local:
Fri, 25 Mar 2022 03:11:55 +0000---Starting f_contracts with query 1 Fri, 25 Mar 2022 03:12:01 +0000---Starting XML -> JSON conversion
Warning: XMLReader::expand(): /private/tmp/redshift-dump.xml:1109: parser error : Extra content at the end of the document in /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php on line 1605
Warning: XMLReader::expand(): </table_data> in /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php on line 1605
Warning: XMLReader::expand(): ^ in /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php on line 1605
Warning: XMLReader::expand(): An Error Occurred while expanding in /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php on line 1605
Fatal error: Uncaught TypeError: DOMDocument::importNode(): Argument #1 ($node) must be of type DOMNode, bool given in /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php:1605 Stack trace: #0 /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php(1605): DOMDocument->importNode(false, true) #1 /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php(1582): Primary->mysqlDumpXmlToJson() #2 /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php(1509): Primary->dumpData('f_contrac...', 1) #3 /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php(1460): Primary->process('f_contrac...', 1) #4 /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php(46): Primary->processFContracts() #5 /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php(1665): Primary->__construct() #6 {main} thrown in /Users/hm/repo/f_contract_update/data.redshift.sync.hsdp/scripts/primary.php on line 1605
The exact same script runs perfectly when running on the server, but gives me the above error when running on local.
When I look at the XML dump, and compare it to the dump on the server, I notice that the XML doesn't close off properly :
Compared to the server :
This is the script which it is complaining about :
private function mysqlDumpXmlToJson()
{
if (file_exists($this->json_file)) {
unlink($this->json_file);
}
$z = new \XMLReader();
$z->open($this->xml_file);
$doc = new \DOMDocument();
while ($z->read() && $z->name !== 'row') ;
$f = fopen($this->json_file, 'a+');
while ($z->name === 'row') {
$data = [];
$node = simplexml_import_dom($doc->importNode($z->expand(), true));
foreach ($node as $col) {
$value = (string)$col;
$value = str_replace('0000-00-00 00:00:00', '', $value);
$data[(string)$col['name']] = $value;
}
fwrite($f, json_encode($data));
$z->next('row');
}
fclose($f);
}
Could it be that the mysqldump is limiting output size and where are these configurations set?
EDIT********
The following steps gets executed:
private function process($table, $query)
{
$this->info('Starting ' . $table . ' with query ' . $query);
$this->dumpData($table, $query);
$this->info('Compressing JSON');
$this->compressJson();
$this->info('Moving to S3');
$this->moveToS3();
$this->info('Moving data to staging table');
$this->copyData($table);
$this->info('Finished ' . $table . ' with query ' . $query);
}
The failure occurs during dumpData :
private function dumpData($table, $query)
{
if (file_exists($this->xml_file)) {
unlink($this->xml_file);
}
if ($table=='c')
{
system(sprintf('mysql -h rds.sdp.com -u redshift -****** --xml --database onnet --execute "select columns from c" >%s'
, $this->xml_file));
}
else {
system(sprintf(
'mysqldump --single-transaction --no-tablespaces -h rds -u redshift -***** --xml onnet %s --where="%s"> %s',
$table,
$query,
$this->xml_file
));
}
if ($table == 's_m') {
system(sprintf(
'sed -i "s/<..>/__#__/g" %s',
$this->xml_file
));
system(sprintf(
'iconv -c -f utf8 -t ascii < %s > %s',
$this->xml_file,
$this->xml_file . '.tmp'
));
system(sprintf(
'strings %s > %s',
$this->xml_file . '.tmp',
$this->xml_file
));
};
$this->info('Starting XML -> JSON conversion');
$this->mysqlDumpXmlToJson();
$this->info('Finished XML -> JSON conversion');
return (filesize($this->json_file) > 1000);
}
Hope that clarifies.
File systems are:
private $xml_file = '/tmp/rsh-dmp.xml';
private $json_file = '/tmp/rsh-dmp.json';
private $json_file_compressed = '/tmp/rsh-dmp.json.gz';
private $json_file_s3 = 's3://sdp-ew/rsh-dmp.json.gz';

Answer 1

You have stated that the exact same script runs correctly on prod, while it fails on your local. So, the source-code that you have is capable to run correctly. So, the problem must be something else.
Now, we know that $this->xml_file is a file on the server, prod or local and it is XMLReader::expand() that complains first and then, the result is a bool instead of a DOMNode, which strongly suggests that the DOM could not be properly validated.
Since you have shown that the actual file ends unexpectedly, it is highly probable that your file is not properly formed.
First of all, you need a root node. From your example we see that your root node is not being closed. So, you either have some unclosed nodes, or, you do not have a root node in the first place.
If you download the file from prod and test your local code with that file, then you will see that it is being correctly executed. So, the bug if at the place where the file was created/generated. If you have a function/module which generates the file, then you will need to debug on how it is being generated and see why the nodes are not being closed, or why the root node is not being wrapped around your structure.