How to validate data from csv before converting to json? - php

Question

I have a csv file like this:-
+------+------+------------------------+
| name | mark | url |
+------+------+------------------------+
| ABCD | 5 | http://www.example.org |
| BCD | -2 | http://www.example.com |
| CD | 4 | htt://www.c.com |
+------+------+------------------------+
It contains a header for name, mark and url. I am using PHP to convert the data from csv to json. I want to add validation before converting it into json like for the name it should be in UTF-8, the mark should be a positive number and between 0 to 5 and the url should be valid. If the row passes all validation then it gets stored in a errorless.json and if any row has any issues then in error.json with a comment what was wrong. PHP code for csv to json:-
$fh = fopen("names.csv", "r");
$csvdata = array();
while (($row = fgetcsv($fh, 0, ",")) !== FALSE) {
$csvdata[] = $row;
}
$fp = fopen('data.json', 'w');
fwrite($fp, json_encode($csvdata));
fclose($fp);
I wanted to know how can i add these validations for converting the data. I am new to these concepts and unable to think of a way to do it. I would be highly grateful if anyone can help me.

Answer 1

Here is a shell of what you would do:
// to make life easier on yourself create
// a function that checks one row of data
// to make sure it is valid
// if a row is found to be invalid, the
// function will add an `error` field to the
// array explaining the validation error
function validateRow(&$data) {
if (!array_key_exists('mark', $data) && ((int)$data['mark']) >= 0 && ((int)$data['mark']) <= 5) {
$data['error'] = "No 'mark' value found between 0 and 5";
return false;
}
// do validation for $data['name']
// do validation for $data['url']
return true;
}
$fh = fopen("names.csv", "r");
$csvdata = array();
while (($row = fgetcsv($fh, 0, ",")) !== FALSE) {
$csvdata[] = $row;
}
$header = $csvdata[0];
$n = count($header);
$errorless = array();
$haserrors = array();
// here is where we convert the CSV data into
// an associative array / map of key value
// pairs by treating each row and the header
// row as parallel arrays. Start with index 1
// in the for loop to skip over the header row
// in csvdata
for ($row = 1; $row < count($csvdata); ++$row) {
$data = array();
for ($x = 0; $x < $n; ++$x) {
$data[$header[$x]] = $csvdata[$row][$x];
}
// if we encounter no errors,
// add data to the errorless array
if (validateRow($data)) {
$errorless[] = $data;
}
else {
$haserrors[] = $data;
}
}
// you will want to do a parallel write
// for the error.json file
$fp = fopen('errorless.json', 'w');
fwrite($fp, json_encode($errorless));
fclose($fp);

Answer 2

You should use something called JSON schema. You define how your document should be formed. Then your document can be validated automatically.
In case you were looking for another concrete implementation of CSV validation libraries, you should check out packagist.org in the first place.

Answer 3

This may help you,
The standard JSON format doesn't explicitly support file comments. RFC 4627 application/json, It's a lightweight format for storing and transferring data. If the comment is truly important, you can include it as another data field like comments
Input
$ cat test.csv
name,mark,url
ABCD,5,http://www.example.org
BCD,-2,http://www.example.com
CD,4,htt://www.c.com
Script
<?php
function validate_url($url)
{
return in_array(parse_url($url, PHP_URL_SCHEME),array('http','https')) && filter_var($url, FILTER_VALIDATE_URL);
}
function validate_mark($val)
{
return ($val >= 0 && $val <= 5);
}
function errors($field)
{
$errors = array(
'url' => 'URL should be valid',
'mark' => 'Mark should be between 0 to 5'
);
return ( isset($errors[$field]) ? $errors[$field] : "Unknown");
}
function csv2array_with_validation($filename, $delimiter = ",")
{
$header = $row = $c_row = $output = $val_func = array();
if (($handle = fopen($filename, 'r')) !== FALSE)
{
while (($row = fgetcsv($handle, 0, $delimiter)) !== FALSE)
{
if (empty($header))
{
$header = array_map('strtolower', $row);
foreach ($header as $e)
{
$val_func[$e] = function_exists('validate_' . $e);
}
continue;
}
$c_row = array_combine($header, $row);
$index = 'error_less'; $errors = array();
foreach ($c_row as $e => $v)
{
if ($val_func[$e])
{
if (!call_user_func('validate_' . $e, $v))
{
$index = 'error';
$errors[$e] = errors($e);
}
}
}
/*
If the comment is truly important,
you can include it as another data field like errors,
comment below part if you do not wish to create new field (errors) in
json file
*/
if(!empty($errors))
{
$c_row['errors'] = $errors;
}
$output[$index][] = $c_row;
}
fclose($handle);
}
return $output;
}
$output = csv2array_with_validation('test.csv');
// Write error.json
if (isset($output['error']) && !empty($output['error']))
{
file_put_contents('error.json', json_encode($output['error'], JSON_PRETTY_PRINT | JSON_NUMERIC_CHECK));
}
// Write errorless.json
if (isset($output['error_less']) && !empty($output['error_less']))
{
file_put_contents('error_less.json', json_encode($output['error_less'], JSON_PRETTY_PRINT | JSON_NUMERIC_CHECK));
}
?>
Output
$ php test.php
$ cat error.json
[
{
"name": "BCD",
"mark": -2,
"url": "http:\/\/www.example.com",
"errors": {
"mark": "Mark should be between 0 to 5"
}
},
{
"name": "CD",
"mark": 4,
"url": "htt:\/\/www.c.com",
"errors": {
"url": "URL should be valid"
}
}
]
$ cat error_less.json
[
{
"name": "ABCD",
"mark": 5,
"url": "http:\/\/www.example.org"
}
]