Converting CSV table data into YAML file using PHP arrays - php

I have been assigned the following PHP task for Uni:
1) Export the Trades/Crafts table to Excel
2) Make three columns in Excel: Trade-ID, Category-ID, Trade-Name and organize it
3) Export the table as a .CSV file (easier for PHP manipulation)
4) Write a PHP script that creates a YAML file from CSV file that corresponds to the structure of AppBundle / Resources / fixtures / prod / trades.yml
How you do that exactly, is up to you. You are free to use whatever method you prefer, but the YAML file has to have
the right structure.
This is the trades.yml file (it was given as an example for us to follow):
-
ref: trade-1
id: 1
name: Plumber
category: $trade-category-1
-
ref: trade-16
id: 16
name: Electronic Engineer
category: $trade-category-2
So as you might have guessed, I have a table with those 3 columns (Trade-ID, Category-ID and Trade-Name). The table contains about 150 rows with the name of many different kind of jobs, the type/branch of the job as category ID, and the ID of the job itself.
I exported that Excel table as a .csv file, as ordered. Now, I think I should use some function like str_getcsv or fgetcsv, which, how I understand it, reads the data in the CSV table and converts it into PHP arrays. After that, I need to convert those arrays into the YAML syntax format, but I read that's not particularly difficult.
Anyways, the number of 'titles/entries' in the YAML structure (ref, id, name, category) does not equal the number of columns in the CSV table (trade name, cat id, trade id), so I don't know how or where I should even start!
Also, for the "ref" part,I guess I would have to make it look like this: trade-<ID of trade> , but how can I do something like "trade-" . $TradeID inside an array? How can declare $TradeID = <ID of the job>? Can I refer to the CSV table's row and column I want, like SELECT in SQL? Or should I maybe use a WHILE loop which fetches all of the table's rows?
I have tried it like this:
<?php
$file = fopen('MyTable.csv', 'r') or die('error');
$line = array();
while (($line = fgetcsv($file)) !== FALSE) {
//$line is an array of the csv elements
// print_r($line);
foreach ($line as $key => $value) {
# code...
}
}
fclose($file);
But it outputs something like this:
Array ( [0] => 1;1;Bituminiser; ) Array ( [0] => 1;2;Construction Dryer; ) Array ( [0] => 1;3;Concrete Driller and Cutter; ) Array ( [0] => 1;4;Concrete Block and Terrazzo Maker; ) Array ( [0] => 1;5;Well Builder; )
And so on... But I still have the other problem, plus the [0] inside each array.
I really can't figure it out. How can I convert the CSV table into a YAML file with the structure of trades.yml using PHP?

Try this:
fgetcsv($file, 0, ';')
And look for more information in PHP docs for fgetcsv
However, I've made a sample script for you:
<?php
$file = fopen('MyTable.csv', 'r') or die('error');
$yml = '';
$indent = str_repeat(' ', 4);
$keys = [
'ref',
'id',
'name',
'category',
];
while (($values = fgetcsv($file, 0, ';')) !== FALSE) {
$yml .= "-\n";
$arr = array_combine($keys, $values);
foreach ($arr as $key => $value) {
$yml .= "{$indent}{$key}: {$value}\n";
}
}
fclose($file);
echo $yml;

Related

Export/import mysql json object via CSV

I'm working with a pair of PHP scripts. One script reads data from a MYSQL database and exports it to a csv file then a second script uploads the exported csv file to another MySQL database instance using csv. The structure of the database tables A (export) and B (import) are identical.
These scripts work fine for "normal" MySQL tables and column types. However, the import fails when we apply them to a MySQL table that stores a JSON object in one of the columns (MySQL column type is "json").
The script that exports the data works as expected, producing a CSV file with the JSON object surrounded by double quotes...just like the other values in the row.
The row in the exported CSV file looks like this (the last item is a complex json object, abbreviated for simplicity):
"894","Somebody","Related","2020-02-20","{"name1":"value1","name2":"value2","name3":"value3"}","expired"
In the PHP script to export the data it's essentially this:
$rowStr = "894","Somebody","Related","2020-02-20","{"name1":"value1","name2":"value2","name3":"value3"}","expired";
file_put_contents($filepath, trim($rowStr), FILE_APPEND);
No issues with the export. Row appears in the CSV file as expected (same format as above).
My code to read the csv into the other database looks like this:
$allRows = array_map('str_getcsv',file($fp)); // read the exported csv file where $fp is the path to the file
foreach($allRows as $i => $row) {
//get the col_names from the 2nd database table (identical to the first) where $ac-> is the class that handles data queries
$col_names = $ac->get_table_column_names('databasename',$tablename);
$update_arr = array();
foreach($col_names as $i => $cname) {
$update_arr[$cname['COLUMN_NAME']] = $val;
}
//and write the row to the 2nd db's table
$ac->create_update_table($update_arr,$tablename,FALSE);
}
And, if it matters, here are the Queries used in the "get_table_column_names" and "create_update_table" functions:
get_table_column_names //Using PDO
SELECT COLUMN_NAME,COLUMN_DEFAULT,DATA_TYPE FROM information_schema.columns WHERE table_schema = :db AND table_name = :table
create_update_table
INSERT INTO 'tablename' (field1, field2, field3, field4,json_object_column) VALUES ("894","Somebody","Related","2020-02-20","{"name1":"value1","name2":"value2","name3":"value3"}")
The problem is that, when importing, the row is converted to an array like this:
array (
[0] = "894",
[1] = "Somebody",
[2] = "Related",
[3] = "2020-02-20",
[4] = "{name1":"value1",
[5] = "name2:"value2", //should be part of node 4
[6] = "name3:"value3"}", //should be part of node 4
[7] = "expired"
);
What's happening is that the "," inside the JSON object is being treated as a field separator and the JSON is broken up into array nodes. Other than writing a script to detect fields that start with "{ and end with }", how can I read the entire json string as one field (as it is in the database)? or, perhaps, is there a better way to output the string so that it can be read as one item?
If instead of just writing out the data using something like file_put_contents() you use some of the methods designed for CSV files, this will do most of the work for you...
To write the data use fputcsv() and this escapes the delimiter (in this case the " becomes "")...
$row = ["894","Somebody","Related","2020-02-20",'{"name1":"value1","name2":"value2","name3":"value3"}',"expired"];
$fh = fopen($filepath, "a");
fputcsv($fh, $row);
fclose($fh);
which will write to the file
894,Somebody,Related,2020-02-20,"{""name1"":""value1"",""name2"":""value2"",""name3"":""value3""}",expired
and then to read from the file, just read a line at a time and use fgetcsv()...
$fh = fopen($filepath, "r");
print_r(fgetcsv($fh)); // This in a loop to read all lines
fclose($fh);
which shows
Array
(
[0] => 894
[1] => Somebody
[2] => Related
[3] => 2020-02-20
[4] => {"name1":"value1","name2":"value2","name3":"value3"}
[5] => expired
)
One way of solving this is to create a new copy of the array and manipulate the new array
and add json as a sliced part of the original array.
$allRows = array_map('str_getcsv',file($fp));
$new_arr = [];
foreach($allRows[0] as $key=>$item) {
$json = false;
if (substr($item,0,1) == '{') {
$json_start = $key;
$json = true;
}
if (substr($item,-2,2) == '}"') {
$json_stop = $key;
$json = true;
//Slice json-part from original array (in your case 4,5,6)
$sl = array_slice($allRows[0], $json_start, ($json_stop-$json_start)+1);
//Add the sliced part into key where json started
$new_arr[$json_start] = implode('',$sl);
}
if ($json === false) $new_arr[] = $item;
}
And then you have your expected array in $new_arr.

How to ask queries on csv file using php script?

How to prepare csv file so I could use it for running queries.. Do you have any ideas or maybe links that could be helpful
I have mysql database and i run certain sql query on this database lets say "select * from some_database where id=1", and thats everything clear and ok, but, now, i export csv from database and i want to open in and read with php and to sort datas similar as in mysql query. Can someone help me and give me some ideas, points in what way to go
<?php
$csv =<<<CSV
23, Foo, Bar, foo#example.com
47, Baz, Qux, baz#example.com
CSV;
$items = explode("\n", $csv);
$items = array_map('str_getcsv', $items);
$get_by_id = function ($id) use ($items) {
foreach($items as $item)
if($item[0] == $id)
return $item;
};
var_export($get_by_id('47'));
Output:
array (
0 => '47',
1 => ' Baz',
2 => ' Qux',
3 => ' baz#example.com',
)

How do I put a .csv file correctly into an array in PHP?

So I'd like to make a basic login/register page. I got a CSV file which roughly looks like this:
a, b
r,d
login, pass
I am already able to correctly add new combinations to the file. But if I want to put the CSV into an array so that I can check if the username/password combination is true, I only get the first row in the array, so [0] = "a" and [1] = "b". There are similar questions on this site on how to put a csv into an array, but with every solution this problem comes up. How do I get the other elements in the array, too?
Edit: as suggested, the code I used:
$database = fopen("database.csv", "r");
$data = fgetcsv($database, 1000, ",");
print_r($data);
This returns: Array ( [0] => q [1] => w )
Exact data:
q,w
g,h
o,p
t,y
c,d
o,p
o,p
a,b
Hope you can help me.
You can see from the documentation that fgetcsv returns just one line from the file pointer, and NULL or FALSE if it was unable to get another line.
You should put your code in a while loop to get all of the CSV rows.
$credentials = array();
$database = fopen("database.csv", "r");
while (is_array($data = fgetcsv($database, 1000, ','))) {
$credentials[] = $data;
}
fclose($database);
var_dump($credentials); // This contains all of the credentials.

PHP fgetcsv 2 dimensional array

I have seen few similar examples but it is still not working.
csv data file "data1.csv" is as below:
symbol,num1,num2
QCOM,10,100
QCOM,20,200
QCOM,30,300
QCOM,40,400
CTSH,10,111
CTSH,20,222
CTSH,30,333
CTSH,40,444
AAPL,10,11
AAPL,20,22
AAPL,30,33
AAPL,40,44
--end of file ----
$inputsymbol = QCOM ; // $inputsymbol will come from html.works fine.
I want to read the csv file and fetch lines that matches symbol = QCOM. and convert it in to array $data1 to plot line chart for num1 and num2 as below.
$data1 = array (
array(10,100),
array(20,200),
array(30,300),
array(40,400)
);
Note: 1. no comma at the end of each csv lines in csv datafile.
2. Multiple symbols in same file. so the lines that match symbols only
should be included in $data1.
==============
Mark's soluition solves the problem. Now to make the data access faster (for a very large csv file), I have (externally) formatted same data as below. Question is how it can automatically extract headers and then for the data1 array?
symbol,1/1/2015,1/2/2015,1/3/2015,1/4/2015
QCOM,100,200,300,400
CTSH,11,22,33,44
AAPL,10,11,12,13
Note that the number of fields in header is not fixed. (it will increase every month). But the data will also increse accordingly.
Not complicated:
$inputsymbol = 'QCOM';
$data1 = [];
$fh = fopen("data1.csv", "r"));
while (($data = fgetcsv($fh, 1024)) !== FALSE) {
if ($data[0] == $inputsymbol) {
unset($data[0]);
$data1[] = $data;
}
}
fclose($fh);
So where exactly are you having the problem?

How to distinguish data from CSV files

I have two csv files, and both have same data structure.
ID - Join_date - Last_Login
I want to compare and get the exactly matching records numbers based on this example:
the first files has 100 records, of which 20 are not included in the 2nd file.
the 2nd file has 120 records.
I want a script in PHP to compare these two files and build two separate CSV files.
And I want to remove all extra records from the 2nd file which are not included in the first file.
And remove all records from the first file which are not included in the 2nd file.
Thanks
There is a GNU utility comm that will do this really easily. You could exec that through php or just do it directly. If you don't have access to comm, the easiest thing to do would be to store both files in an array (probably via file()) and use array_intersect().
You an try this for limited number of CSV file .. if you have a very large CSV i would advice you import it directly into MySQL
function csvToArray($csvFile, $full = false) {
$handle = fopen ( $csvFile, "r" );
$array = array ();
while ( ($data = fgetcsv ( $handle )) !== FALSE ) {
$array [] = ($full === true) ? $data : $data[0]; // Full array or only ID
}
return $array;
}
$file1 = "file1.csv" ;
$file2 = "file2.csv" ;
$fileData1 = csvToArray($file1);
$fileData2 = csvToArray($file2);
var_dump(array_diff($fileData1,$fileData2));
var_dump(array_intersect($fileData1,$fileData2));

Categories