I'm trying to import lot's of CSV files into a MySQL DB programmatically.
After some research, I found LOAD DATA, but it is not a possibility, as the server don't allow it.
Then, I found mysqlimport a viable alternative.
Here's what is happening:
I download lot's of CSV files from a FTP server and, one by one, I execute the folowing:
exec("ln " . $path . $csv_file. " " . $path . "CSVs.txt");
exec("mysqlimport -u root -ppass --local --ignore-lines=1 --columns=column1,column2 " . $path . "CSVs.txt>" . $path . "_.txt");
unlink($path . "CSVs.txt");
First, I create a symlink of the file (as I didn't manage to use the file name different from the table name)
Then, I execute the mysqlimport command, sending the output do a txt file.
Finally, I remove the symlink.
When I run, the CSV temp file is created and deleted, but nothing apear on my DB, nor in the output txt.
If I echo the command and past in my terminal, it works flawlessly, but, in the exec, nothing happens.
I believe it has something to do with the quotes and backslashes, but couldn't manage to fix it.
Any help will be appreciated :P
EDIT: I already chmod**ed the **mysqlimpot bin… still didn't work!
It seem like PHP wasn't finding mysqlimport command. So, I had to use it's full path to make it work!
Related
<?php // BISMILLAHIRRAHMANIRRAHEEM
error_reporting(E_ALL);
ini_set("display_errors", 1);
// tried the following with soffice, soffice.com, soffice.exe, soffice.bin
// MS-DOS: soffice, soffice.com, soffice.bin all work successufully and generate the pdf
$ex = "soffice.com --headless --convert-to html \"" . getcwd() . DIRECTORY_SEPARATOR . "sources" . DIRECTORY_SEPARATOR . "test.docx\" --outdir \"" . getcwd() . DIRECTORY_SEPARATOR . "results\"";// 2>&1";
chdir("C:\Program Files\LibreOffice\program");
echo $ex . "<br />";
exec($ex, $output);
//exec("mkdir rr", $output); // generates directory
//exec("dir soff*", $output);
//exec("a.bat", $output); // doesn't generate pdf
print_r($output);
?>
Hi, I'm using IIS with Windows 7, PHP 5.6.39. I am unable to convert a docx document to pdf by calling LibreOffice via PHP. The command output to screen in the code example works if executed in MS-DOS window but doesn't work in php's exec() or shell_exec(). The folder seems to have the appropriate permissions since my upload scripts and exec("mkdir newdir") are working.
exec("dir"); also outputs directory listings for the mentioned path, so DOS navigation commands seem to work as well.
I'd like to avoid the solution to make another user. Not elegant. This shouldn't be difficult INSHAALLAH.
By not working, I mean no document is generated and no output/ error is generated by exec() either which makes it difficult to debug.
Thank you for your time.
Update:
After some fiddling, I've realized the return code is: -1073741515
Google results show that its apparently a File I/O failure.
Another script I have downloaded which does something similar, also gave the same error. So looking into the causes. As I said before, the directories can be made and files uploaded as it has permissions.
In the link I've posted, there seems to be need for making new user. Is it possible that IIS has permissions to the directory but not the PHP exec()?
On Linux, it gives me error code 77, permission denied?
BISMILLAHIRRAHMANIRRAHEEM
ALHAMDOLILLAH after much search, finally found the answer.
Windows:
I did some permissions etc earlier, should be trivial with IIS. But what finally solved the problem was this.
Apparently there needs to be specified -env:UserInstallation=file:///C:\some\path because you cannot run multiple instances of LibreOffice concurrently without specifying a different profile folder (Reference).
Also, see the same solution here.
When executing a program via PHP over IIS, the user account for Windows is the IUSR account.
So,
Give IUSR write (and modify? I give full) permissions to be able to write to C:\inetpub\wwwroot\path directory.
Change directory to LibreOffice path for soffice.com (Not needed if providing full path when executing):
chdir("C:\\Program Files\\LibreOffice\\program");
Set HOME environment variable, as LibreOffice needs a writable temporary path. IUSR must have permissions to this path. I simply re-use wwwroot (This step could be optional):
putenv("HOME=C:\\inetpub\\wwwroot\\temp");
Execute in PHP, something like:
exec("\"C:\\Program Files\\LibreOffice\\program\\soffice.com\" --headless \"-env:UserInstallation=file:///C:/inetpub/wwwroot/LibreOfficeProfilePath1\" --convert-to pdf:writer_pdf_Export --outdir \"C:\\inetpub\\wwwroot\\result\" \"C:\\inetpub\\wwwroot\\source\\file.docx\"", $output, $ret);
// Displaying the command output
print_r($output);
echo "<br />\n\n<br />"; // Line breaks for view-source
echo "Return code: " . $ret;
Clean up by removing the profile directories for LibreOffice. Since you'd be making unique directories on-the-fly on a server environment for concurrent sessions of LibreOffice, you'd like to delete them too. Following snippet from here:
exec("rmdir /S /Q \"C:\\inetpub\\wwwroot\\LibreOfficeProfilePath1\"");
You could look to generate this profile path automatically, like:
// Do this before executing soffice command and pass it to -env
$profilepath = "C:/inetpub/wwwroot/LibreOfficeProfilePath" . date("YmdHis") . rand(0, 999999);
// Make sure to replace '/' with '\' when deleting
exec("rmdir /S /Q \"" . str_replace("/", "\\", $profilepath) . "\""); // Windows
exec("rm -rf \"" . str_replace("/", "\\", $profilepath) . "\"); // Linux
or instead use PHP to delete directory with files in it as shown here.
LibreOffice running as a service and PHP requesting document conversions may be a better solution but this post relates to invoking LibreOffice each time a document conversion is needed and then letting it close. A separate process method may also be used to avoid waiting for LibreOffice to complete conversion.
Linux:
Similar steps for www-data, as that is the user account for apache in Linux. See the solution here.
www-data does not have $HOME directory so it needs to be specified as indicated in (3) above.
env path would be something like:
\"-env:UserInstallation=file:///var/www/tmp\"
Notice the three / after file: regardless of Windows or Linux, even though paths in Linux start with /, or maybe they ignore the slash in Windows, seems inconsistent to me, though.
INSHAALLAH it'll work after that.
I am trying to make a simple program in Laravel that would allow the user to click a button and a backup SQL for their database would be generated locally.
$command = "/usr/local/mysql/bin/mysqldump --user=" . env('DB_USERNAME') ." --password=" . env('DB_PASSWORD') . " --host=" . env('DB_HOST') . " " . env('DB_DATABASE') . " > " . storage_path() . "/" . $filename;
$returnVar = NULL;
$output = NULL;
exec($command, $output, $returnVar);
This is my current code and as you can see I have specified the entire path of mysqldump, if I don't it just returns an empty .sql. The same command (without the entire path) runs flawlessly when I run it in terminal, now this code is going to run on locally different environments (Windows, Linux, Mac) and the path to mysqldump would be different in each.
Is there a way I can make this happen without specifying the entire path?
You could use another env variable with a default to 'mysqldump'.
However, you shouldn't use env variables directly in your code, load them into config parameters in your config files. env() won't work once the config is cached.
The underlying problem, however, is that /usr/local/mysql/bin is not in your PATH variable for the web user. PHP exec $PATH variable missing elements
I have a PHP code that writes cronjobs into crontab using this code.
$output = shell_exec('crontab -l');
file_put_contents('/tmp/crontab.txt', $output . $command . PHP_EOL . PHP_EOL);
exec('crontab /tmp/crontab.txt');
It works 100% well. But issue is that lots of users would accessing the application so won't files will be over-written? if not over-written I am sure there would be some other issue.
My question is how do I write/append to crontab using single command without writing that cron contents to temp file?
I tried something like echo "cron here" > crontab but didn't work.
Let's be honest, but how many times can these concurence edit occur to crontab? I would prefer some kind of locking mechanism for this case. For example create .lock file when user opens the file and delete the .lock file after editing is done. If another user will want to edit the crontab and .lock will be present, he won't be allowed.
I'm using exec() for execute mysql-dump but it's generating an empty file on apache, but when i use the "php artisan serve" the file is generated correctly, the output file has the same user and group in apache and in the artisan serve.
Using: Ubuntu 14.04 and Xamp 5.6.12
$dir = substr(__DIR__, 0, 24).'database/backups/';
$newBackup = Backup::create();
$command = 'mysqldump -uroot lions > '.$dir.$newBackup->getDateTimeString().'.sql';
exec($command);
Ok, I was facing the same problem with Laravel 5.6 and running windows though. Before anything, you should make sure to include the password argument (either as --password or -p), even if its empty.
$cmd =
"mysqldump -h " . env('DB_HOST') .
" -u " . env('DB_USERNAME') .
" -p\"" . env('DB_PASSWORD') . "\"" .
" --databases " . env('DB_DATABASE');
The main issue is that even though several sources note that exec waits until the process is complete there is no way to know for sure, leaving that aside, since processes could be owned by different users it'd probably be better if you created a temporary file and then flashed the contents of the output array into it rather than hoping for mysqldump to do it for you.
$output = [];
$name = "your_random_file_name.sql";
exec($cmd, $output);
$tmppath = tempnam(sys_get_temp_dir(), $name);
$handle = fopen($tmppath, "w");
fwrite($handle, implode($output, "\n"));
Finally, write the main file to your desired location, in my case I upload it directly to an Amazon S3 bucket, and close the temporary file.
Storage::disk('myS3bucket')
->putFileAs("backups", new File($tmppath), $name);
fclose($handle);
I know it seems redundant but, as I mentioned above, there is no way to know for sure.
Indeed, in a development environment, it is customary not to set a password for the database. And that's why your command does not work.
But in a production environment, you are obliged to set a password for the database.
Try in your production environment (ie with a password!), And you will see that the script works.
I don't have a solution for this problem but I do know it should be possible to do this. If I enter the following command in my terminal the mysqldump works:
mysqldump "--user=root" "--password=" dbname > ~/Desktop/export.sql
But when I use this command in my scheduler as below, I gett an empty file.
$schedule->exec("mysqldump \"--user=".env('DB_USERNAME')."\" \"--password=".env('DB_PASSWORD')."\" ".env('DB_DATABASE')." > export.sql")->everyMinute();
Maybe there is someone who knows why this doesn't work? I'm guessing it's a permission issue, but I leak experience in this.
I have been searching for a while in how to successfully exec a Python script through PHP, just for a test thats all.
I've never actually worked with Python and a while since I programmed in PHP but looked for a simple code to send data from PHP to Python and then in .py send back data to PHP.
I have tried exec in PHP, with and without json enc/dec without any success.
These are the commands that have I tried in PHP:
* $result = shell_exec('pythontest.py' . escapeshellarg(json_encode($data)));
* exec("python pythontest.py", $resultData);
var_dump($resultData);
with no success.
In pythontest.py: a print to send back data.
Tried with full paths of both Python and pythontest.py (PHP).
enc/dec with json works in PHP file so nothing seems to be wrong in the PHP code still I can't run the script!
As I said, I haven't worked with Python before so I wonder if there is more needed than just writing the code in the script in order to make it work?
I have seen many posts about this and tried their code that "would work for them" but not for me^^
Additional info: PHP through WAMP, .py in same map. It is supposed to be web based, if that makes it difference.
Check to make sure that the directory containing python.exe is in the PATH of your PHP script (via getenv('PATH')).
Try running python c:\path\to\pythontest.py directly via a Windows command prompt. Does that work? If not, debug PATHs and issues from there, then try again in PHP.
If your WAMP stack runs PHP with limited permissions, check to make sure that user has "Read and execute" type permissions on the python.exe file, as well as the entire directory which contains it, recursively. This might be overkill, but recursively setting perms for that user on the whole python distribution directory rules out permissions-related problems.
Try qualifying the python executable's name with .exe, e.g. c:\python27\python.exe pythontest.py (you might have tried that anyway, since you say you tried with "full paths", but I'm throwing it out there to make sure you included the .exe).
Try replacing the contents of pythontest.py with something extremely simple (e.g. print 'hello') and check whether or not behavior changes.
Also, please post the contents (or at least first lines) of pythontest.py.
I have now tried changing the path, as python.exe was not in the same directory.
My end result on this was
$output = getenv('PATH');
echo "Real Path: " . $output . "<br>";
$ret = apache_getenv("PATH");
echo "Apache Path: " . $ret . "<br>";
apache_setenv("PATH", "C:/Python33");
$ret = apache_getenv("PATH");
echo "Apache new path: " . $ret . "<br>";
$output = getenv('PATH');
echo "Real path, hopefully changed: " . $output . "<br>";
this was just to see if i could change the path, since i have never done that before.
i have now only
import sys, json
print ("hello")
in my python file and tried to run it through cmd, and it worked.
but when i try run it with php, it fails once again.
i have also looked over the permissions:
admin, user have permission to read and execute, i even added "Everyone" with full permission on folder and sub-folder/files just to see if that made any difference.
something i have missed or did i do something wrong with permission, python code or paths ?