PHP, inserting CSV from external source into MySQL DB - php

Question

I know this has been asked before -- I tried to read previous Q&A on the topic but I'm still stuck. Probably I read too many Q&A and have a bad mix of techniques as a result.
I don't get an error, I just don't get anything in my table. The echo $i is to help debug and I only ever get a 0 rather than expected 0 1 2 3 ... N rows.
My DB connection credentials are all fine I use them all over my site for Select statements.
$csv_file = str_getcsv("https://ds.data.jma.go.jp/tcc/tcc/products/gwp/temp/list/csv/year_wld.csv");
$csvfile = fopen($csv_file, 'r');
$theData = fgets($csvfile);
$i = 0;
while (!feof($csvfile))
{
echo $i;
$csv_data[] = fgets($csvfile, 1024);
$csv_array = explode(",", $csv_data[$i]);
$yrr = $csv_array[0];
$vals= $csv_array[1];
$sql1 = "INSERT INTO Table1(Year,Value) VALUES(" . $yrr . "," . $vals. ")";
$conn->query($sql1);
$i++;
}

Answer 1

The main problem here is the fact that you are trying to open a text variable as a file:
$csvfile = fopen($csv_file, 'r');
In fact you already have an array from from str_getcsv so your whole code should look like (if you can read the whole file at once):
$csvFile = array_map('str_getcsv', file("https://ds.data.jma.go.jp/tcc/tcc/products/gwp/temp/list/csv/year_wld.csv"));
array_shift($csvFile); //we remove the headers
$i = 0;
/**
* Removes all the "*" and "+" symbols as I assume that you want a float since you are not wrapping it in the sql query
*/
function removeUnwantedChars($string) {
return preg_replace('/[^0-9\\.\\-]/i', '', $string);
}
foreach($csvFile as $csvData) {
echo $i++;
$yrr = $csvData[0];
$vals = removeUnwantedChars($csvData[1]);
$sql1 = "INSERT INTO Table1(Year,Value) VALUES(" . $yrr . "," . $vals. ")";
$conn->query($sql1);
}
If you cannot read it all at once then I suggest to first download the file line by line:
<?php
$url = "https://ds.data.jma.go.jp/tcc/tcc/products/gwp/temp/list/csv/year_wld.csv";
$fileHandle = fopen($url, "r");
/**
* Removes all the "*" and "+" symbols
*/
function removeUnwantedChars($string) {
return preg_replace('/[^0-9\\.\\-]/i', '', $string);
}
$i = 0;
$headersSkipped = false;
while ($csvData = fgetcsv($fileHandle)) {
if (!$headersSkipped) {
$headersSkipped = true;
continue;
}
echo $i++;
$yrr = $csvData[0];
$vals = removeUnwantedChars($csvData[1]);
$sql1 = "INSERT INTO Table1(Year,Value) VALUES(" . $yrr . "," . $vals. ")";
$conn->query($sql1);
}
fclose($fileHandle);
Yet like said by #Shadow above it is always great to be more verbose. So in case query returned false then it would be great to output the last error (if you are using PDO then errorInfo() function.