I am trying to create a script that compare the names of the files in a directory with a row in my database in order to get all the files that are not present in the database list.
The script I have so far prints out the files in the directory however when I try to compare the values with my sql I have a lot of results repeated.
How can I compare the name of the files in order to get all the name files that are not listed in my database?
Directory:
a.mp4
0.mp4
b.mp4
34.mp4
c.mp4
Database rows:
> Date---------------Name---------------video_path
> 01-01-01----------|jonh--------------|a.mp4
> 02-01-01----------|andrea------------|b.mp4
> 03-01-01----------|faith-------------|c.mp4
result should be:
0.mp4
34.mp4
SCRIPT:
$directory = '/var/www/html/EXAMPLE/resources/';
$files1 = scandir($directory, 1);
if ( $files1 !== false )
{
//$filecount = count( $files );
foreach ($files1 as $i => $value) {
$sqlfindTmp = "SELECT * FROM `videos`";
if ($resultTmp = mysqli_query($conn,$sqlfindTmp)) {
while ($row=mysqli_fetch_row($resultTmp)) {
if ($row[3] == $value) {
}else{
echo $value . "<br/>";
}
}
}
}
}
else
{
echo 0;
}
The way you have structured your code right now is that it makes an SQL query for every file and on each of those queries it retrieves the whole database, which might not be a problem right now but when your dataset grows will slow things down.
I would suggest restructuring your code so you first query the database for all filenames, save this to an array and then loop through the files, checking if they are in the database.
// Query database
$sqlFind = 'SELECT `video_path` FROM `videos`';
$result = mysqli_query($conn, $sqlFind);
$db = []; // create empty array
while ($row = mysqli_fetch_row($result))
array_push($db, $row[0]);
// Check files
$files1 = scandir($directory, 1);
if ( $files1 !== false ) {
foreach ($files1 as $i => $value) {
if (in_array($value, $db)) {
// File exists in both
} else {
// File doesn't exist in database
}
}
} else {
echo 0;
}
I agree with Kalkran, get the 'video_path' values into an array and do this once. scandir returns all the filenames into an array as well. So you can use a simple array_diff to get an array of files in the directory that aren't in 'video_path'.
So I would try :
$sqlFind = 'SELECT `video_path` FROM `videos`';
$result = mysqli_query($conn, $sqlFind);
$db = []; // create empty array
while ($row = mysqli_fetch_row($result))
array_push($db, $row[0]);
// Check files
$files1 = scandir($directory, 1);
$filesOfInterest = $files1 ? array_diff($files1,$db) : null;
Related
As you can see I have a majority of the step done. My goal is to create a basic shopping list using 0 for not checked off, and 1 for checked off. If a user inputs the name of the list and the item they wish check off the list the 0 will be replaced by a 1. I know I have to loop over each row in the csv and then edit the item the user inputs in the terminal. However I think I did something slightly wrong in the while loop and need help on where to fix. Here is my code:
<?php
$args = $argv;
if(!isset($args[2])) {
echo 'Not given a shopping list title or item to edit!' . PHP_EOL;
exit(1);
}
$listName = $args[1];
$fileName = $listName . '.csv';
$path = __DIR__ . '/../storage/' . $fileName;
$resource = fopen($path, 'a+');
$item = $args[2];
$rows = [];
while(($row = fgetcsv($resource)) !== false) {
$rows[] = $row;
if($rows === $item) {
$item[1] = 1;
}
while($rows == 0) {
fputcsv($resource, $rows);
}
}
fclose($resource);
Here is my csv:
bananas,0
apples,0
lettuce,0
potatoes,0
pasta,0
rice,0
tofu,0
tempeh,0
berries,0
For example if the user wishes to check off rice the 0 will be replaced with a 1.
Edit: The code works up until if($rows === $item) statement. What I am doing with while(($row=fgetcsv($resource)) !== false) { $rows[] = $row; } is get each row of the csv in one array because when I ran the code I only got the first line of the code. And then I add it to the empty rows array to make each entry (there are 9 of them to be in the same array. So when a user wishes to check an item off the list each item will be in the same array and easier to refer to.
This part
if($rows === $item) {
$item[1] = 1;
}
while($rows == 0) {
fputcsv($resource, $rows);
}
I am unsure about because I want to target the 1st index point of whatever item the user types in and wants to check off.
I am trying to pass an array value into a function, but can't seem to get it to work. Here is what I am trying to do
I have a folder on my server called /Rpt
The /Rpt folder has a bunch of different folders inside of it (30 of them) - these folders contain a bunch of different files in them
I want to check most of the folders in /Rpt, and get the name and date of the latest file (based on last modified or created date) ... i want to store the results into an array, that has (folder path, file name of last file, file date of last file)
This query gets the folders that need to be checked
$sql = "my SQL here";
$stmt = $dbh->prepare($sql);
$stmt->execute();
$arrValues = $stmt->fetchAll(PDO::FETCH_ASSOC);
This is a function that checks the specified folder, and displays the info i need (folder path, file name, file date)
function GetFileNameDate($location, $file_name, $file_date) {
$files = scandir($location);
$path = $location;
foreach ($files as $file) {
if (strpos($file, " ") !== false) {
$filename = $file;
$last_updated = date ("F d Y H:i:s", filemtime($path.'\\'.$file));
$results = array($filename=>$last_updated);
$file_name = key($results);
$file_date = reset($results);
return array($location, $file_name, $file_date);
}
}
}
When i call the function - and enter a specific path link in this example, it works OK and i shows the values i want to see
$FileNameDate = GetFileNameDate('E:\Rpt\FolderA');
echo $FileNameDate[0];
echo $FileNameDate[1];
echo $FileNameDate[2];
echo "<br/><br/>";
This is where I am having problems
I am trying to pass an array (list of folders from my SQL query) into the function, so i can output the (folder path, name of latest file, date of latest file) for each of the folders from SQL query
When i echo $locations (it lists all the folders which i am trying to pass to the function)
foreach ($arrValues as $row){
$locations = array($row['Folder']);
$FileNameDate = GetFileNameDate($locations);
echo $FileNameDate[0];
echo $FileNameDate[1];
echo $FileNameDate[2];
}
As per suggestion from #lovelace I also tried the following, but again just a blank page.
foreach ($arrValues as $row){
$locations = array($row['Folder']);
foreach ($locations as $FolderPath) {
$FileNameDate = GetFileNameDate($FolderPath);
echo $FileNameDate[0];
echo $FileNameDate[1];
echo $FileNameDate[2];
}
}
SOLUTION
see comment below to how above was fixed ... however I ended up implementing what i wanted a different way, sharing it in case anyone else needs similar functionality
function GetFileNameDate($location) {
$files = scandir($location);
$path = $location;
foreach ($files as $file) {
$iterator = new DirectoryIterator($path);
$mtime = 0;
$file = "";
foreach ($iterator as $fileinfo) {
if ($fileinfo->isFile()) {
if ($fileinfo->getMTime() > $mtime) {
$file = $fileinfo->getFilename();
$mtime = $fileinfo->getMTime();
}
}
}
return array($path, $file, $mtime);
}
}
foreach ($arrValues as $row){
$locations = array(rtrim($row['Folder']));
foreach ($locations as $FolderPath) {
$FileNameDate = GetFileNameDate($FolderPath);
echo $FileNameDate[0]; #folder
echo $FileNameDate[1]; #file
echo $FileNameDate[2]; #date
}
}
I need to list all files/folders in a given parent folder, and dump it out to mysql.
So far I have:
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
$dir = '/home/kevinpirnie/www';
function dirToArray( $dir ) {
$result = array();
$cdir = scandir($dir);
foreach ($cdir as $key => $value) {
if (!in_array($value, array(".", ".."))) {
if (is_dir($dir . DIRECTORY_SEPARATOR . $value)){
$result[$value] = dirToArray($dir . DIRECTORY_SEPARATOR . $value);
} else {
$result[] = $value;
}
}
}
return $result;
}
$res = dirToArray($dir);
echo '<hr />';
echo '<pre>';
print_r($res);
echo '</pre>';
What I am stuck on is how I can assign ID's to the directories, and then associate them with their parent ID's.
Right now, this code sort of does what I need it to, I just need to be able to convert it to mysql insert statements, yet keep the associative structure, and I am braindead from a long long week of work.
In the end, I am looking to have a table structure similar to:
FileID, FolderID, ParentFolderID, FileName
How can I do this?
try something like this:
function dirToDb($res, $parentId = 0)
{
foreach ($res as $key => $value) {
if (is_array($value)) {
$db->exec("insert into table (path, parentId) VALUES (?, ?)", [$key, $parentId]);
dirToDb($value, $db->fetch("SELECT LAST_INSERT_ID()"));
} else {
$db->exec("insert into table (path, parentId) VALUES (?, ?)", [$value, $parentId]);
}
}
}
$res = dirToArray($dir);
dirToDb($res);
I have modified your code a little. Now for every directory,
index 0 point to directory index
index 1 point to parent directory index
index 2,3,....n points to files
<?php
ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
$dir = '/home/kevinpirnie/www';
$GLOBALS['I'] = 0; // root folder given index 0
function dirToArray( $dir , $parent) {
$result = array();
$cdir = scandir($dir);
foreach ($cdir as $key => $value) {
if (!in_array($value, array(".", ".."))) {
if (is_dir($dir . DIRECTORY_SEPARATOR . $value)){
$result[$value] = [++$GLOBALS['I']]; // add folder index
$result[$value][] = $parent; // add parent folder index
$result[$value][] = dirToArray($dir . DIRECTORY_SEPARATOR . $value, $GLOBALS['I']);
} else {
$result[] = $value;
}
}
}
return $result;
}
$res = dirToArray($dir, $GLOBALS['I']);
echo '<hr />';
echo '<pre>';
print_r($res);
echo '</pre>';
echo '</pre>';
You can now insert the data into mysql tables directly using a similar recursive loop (if you do not want to use mysql auto generated id)
If you want to use auto generated mysql id, you should do insertion in two passes. In first pass insert the folder data and get the id from mysql insert id function. Then create an associative array map
$array_map[$current_folder_id] = mysqli_insert_id()
Then update this id in the second recursive pass
I think you need to insert folder names too as records into the DB, otherwise you wont get parent ID. But, in your result the folder names are missing as record items. I modified your code with assumption that you don't need the result as array but the SQL statement only.
<?php
$qry = array();
$result = $conn->query("SELECT MAX(FileID) as lastid FROM YOUR-TABLE-NAME-HERE");
$row = $result->fetch_assoc();
$id = $row['lastid'];
function dirToSql($dir, $parent) {
global $qry;
global $id;
$result = array();
$cdir = scandir($dir);
foreach($cdir as $key => $value) {
if(!in_array($value, array('.', '..'))) {
$id++;
$qry[] = "('".$id."', '".$dir."', '".$parent."', '".$value."')";
if(is_dir($dir . DIRECTORY_SEPERATOR . $value)) {
dirToSql($dir . DIRECTORY_SEPERATOR . $value, $id);
}
}
}
return $qry;
}
$dir = '/home/kevinpirnie/www';
$sql = dirToSql($dir, 0);
//Here is your SQL Statement
echo $qry = "INSERT INTO YOUR-TABLE-NAME-HERE (`FileID`, `FolderID`, `ParentFolderID`, `FileName`) VALUES ".implode(',', $sql);
You can optimize the code & the query if needed
Every row in your Mysql directory table has three columns -
Directory Id, Directory name, parent id
For each insert the parent id will specify the parent of current directory in a row. For the Root directory parent id is 0
A really useful way to structure your database is with nested sets. Joomla uses this for things like article categories. These can be infinitely nested under one another.
enter link description here
I have csv file with 1500+ entries in a column.I can able to read csv file's all values of column with this.
$rowcount = 1;
$srcFileName = "input/test.csv";
$file = fopen($srcFileName,"r");
$inputfielscount = count(file($srcFileName, FILE_SKIP_EMPTY_LINES));
while($rowcount < $inputfielscount)
{
$row = fgetcsv($file);
$result=array("id" =>$row[0],"des"=>"I am jhon",salery="10000");
$Final=array("listingsEmp"=>$result);
}
After reading first (1-10) value i will create an array (like array [0] =>$result) and Then wantto repeat same task from (11-20) and create another array (like array [1] =>$Final this time $final array contain information about the next ids whic we read from csv file (11-10)) and so on.
For the above requirment i changed code to this :
$rowcount = 1;
$srcFileName = "input/test.csv";
$file = fopen($srcFileName,"r");
while($rowcount < 20)
{
if(($rowcount % 10 == 0) && ( $rowcount != 0)) {
$rowcount++;
break;
}else{
$row = fgetcsv($file);
// some curl code for fetching data according to csv file field(Id)
$result=array("id" =>$row[0],"des"=>"I am jhon",salery="10000"); //contain 10 array
}
}
$Final=array("listingsEmp"=>$result);
Now i will post this $final array which has (0-10 index array ,each has unique id and corresponding values) using curl and get response which i am save in csv file.
$currenttime=date("Y-m-d-H_i_s");
$opfile='output'.$currenttime.'.csv'; //path wher op csv file exist
if(!#copy($srcFileName,'/output/'.$opfile))
{
$errors= error_get_last();
echo "COPY ERROR: ".$errors['type'];
echo "<br />\n".$errors['message'];
}else { // echo "File copied from remote!";
$fp = fopen('output/output'.$currenttime.'.csv',"a");
$fr = fopen($srcFileName,"r");
$rowcounts=0;
$FinalRES=$Final->response;
while($rowcounts< $inputfielscount) {
$resultBulk=$FinalRES[$rowcounts];
$resultBulkStatus=$FinalRES->status;
$resultBulkErrors=$FinalRES->errors;
$errorMsgArray=$resultBulkErrors[0];
$BulkErrorsMessage=$errorMsgArray->message;
$rows = fgetcsv($fr);
if($resultBulkStatus=='failure'){
$list = array ($rows[0],$rows[1],$resultBulkStatus,$BulkErrorsMessage);
}else {
$list = array ($rows[0],$rows[1],$resultBulkStatus,"successfully");
}
fputcsv($fp,$list);
//$p++;
$rowcounts++;
}
}
This full code runs once and give response for 10 ids ,i want repeat this code again for next 10 id (11-20)and then for (21-30) so on .
Once all response write in output csv file After that it display download output file link,Output file contain full response for all Ids which is in csv file(1500 +)
<?php $dnldfilw='output'.$currenttime.'.csv';?>
<a href='download.php?filename=<?php echo $dnldfilw; ?>'>Download Output file</a>
?>
The easiest method is to just use the file() function you are already using...
So to shorten the code to some pseudocode:
<?php
$indexedArray = array();
$indexedSplit = 10;
$lines = file($srcFileName);
$tempArray = array();
foreach($lines as $line) {
if(count($tempArray) % $indexedSplit === 0) {
$indexedArray[] = $tempArray;
$tempArray = array();
}
$tempArray[] = $line;
}
foreach($indexedArray as $index => $valueArray) {
// do the curl magic
// write results of curl into csv
}
Your question is poorly phrased, but I think this would be your aim, right?
When I ouput these txt files, I am trying to group them by unique county with a count limitation per county file. For example, let's say the query returns 2 unique counties in this accessable result field: $row['county_txt'].. Let's say I set the $per_file limitation to 2500. I have the script working now with the per_file etc but not with the counties grouping. Below is somewhat of a mash of where I am at. Thanks for any guidance in helping me resolve this.
Output examples:
Green County - Total Green county results 2900 output would be 2 files.
Output files would be:
Green-#1-20130627-2500.txt
Green-#2-20130627-400.txt
Red County - Total Red county results 12650 output would be 5 files.
Output files would be:
Red-#1-20130627-2500.txt
Red-#2-20130627-2500.txt
Red-#3-20130627-2500.txt
Red-#4-20130627-2500.txt
Red-#5-20130627-150.txt
... // earlier part of script
// Functions I've been attempting
$county[] = $row['county_txt'];
function unique_county() {
foreach($county as $unq_cnty) {
echo $unq_cnty;
return $unq_cnty;
}
}
function get_unique_county() {
$column = array();
while($row = mysql_fetch_array($result)){
$column[] = array_unique($row['county_txt']);
echo $column;
}
}
get_unique_county();
$file_count = 1;
$recs = 0;
$per_file = 2500;
$footer = "FOOTER";
$default_contents = $contents = array("BODY CONTENT TOP");
while ($row = mysql_fetch_array($result)) {
$line = "...";
$contents[] = $line; // Each array element will be a line in the text file
$i++;
$recs++;
if ($county == $unq_cnty && $i == $per_file) {
$contents[] = $footer; // Add the footer to the end
file_put_contents($unq_county . "-#" . $file_count . "-" . date('Y') . "-" . $recs . '.txt', implode("\r\n", $contents));
$i = 0;
$recs = 0;
$contents = $default_contents;
$file_count++;
} // End of if()
} // End of while()
You need a counter, and then be able to reset it (upon resetting it, you save the file).
Example (untested, example only):
<?php
$rowCounter = 0;
$fileCounter = 1;
$startID = md5(microtime(1));
$fp = fopen("{$startID}.txt", "w");
while ($row = mysql_fetch_array($result)) {
$rowCounter++;
fwrite($fp, $row['county_txt']."\r\n");
if($rowCounter == 2500) {
fclose($fp);
if($startID) {
rename("{$startID}.txt", "Red-#{$fileCounter}-".date("Ymd")."-{$rowCounter}.txt");
$startID = md5(microtime(1));
}
$fp = fopen("{$startID}.txt", "w");
$rowCounter = 0;
$fileCounter++;
}
}
// Save last file
fclose($fp);
rename("{$startID}.txt", "Red-#{$fileCounter}-".date("Ymd")."-{$rowCounter}.txt");
?>
On that note, don't use mysql_* functions. Instead, use mysqli at the very least, or PDO.
Not really sure what you are trying to do here, but it seems you are making things way harder than need be. In essence, it seems that you need to work with a two-dimensional array. So why not just query the database and read the data into a 2-D array right off the bat rather than jump through all these extra hoops (i.e. functions to determine unique array values and such)?
So you code might look something like this:
$county_array = array()
while ($row = [YOUR DATABASE ROW FETCHING MECHANISM HERE]) {
$county_array[$row['county_name']][] = $row; // you can change $row here to whatever data you actually need to store.
}
$limit = 2500;
foreach ($county_array as $county_name => $county_array) {
$temp_array = array();
$i = 0;
foreach ($county_array as $item) {
$temp_array[] = $item;
$i++;
if ($i === $limit) {
// we reached file limit, so write it to file code omitted for this
$temp_array = array();
$i = 0;
}
}
if (count($temp_array) > 0) {
// there are still items in temp array so write them to file code omitted for this
}
}
If you actually order by country name in your query and detect for changes to the value when reading county names out (and thus starting a new file), you could actually write directly into files in your loop that reads from the DB saving yourself memory overhead.