Parellel tasks in Laravel Scheduling - php

I basically have a system that's theoretically just a bot, I need to run it on multi-threads (yes I know, it's PHP so not entirely possible) or somewhere along the lines of being able to run an artisan command parallel based on an Eloquent result.
Currently my system looks something like this:
This is my Kernel.php
protected $commands = [
Commands\RunBuy::class
]
protected function schedule(Schedule $schedule)
{
$schedule->command('buy:cron')->everyMinute();
}
This is my RunBuy.php
public function __construct() {
parent::__construct();
}
public function handle() {
$this->bot = Bots::where('status', '1')->where('in_use', '0')->first();
.......
}
I basically need the same capability but being able to manage multiple Bots depending on how many rows their is? what I'd like to be able to do is something like so:
protected function schedule(Schedule $schedule)
{
foreach(Bots::where('status', '1')->where('in_use', '0')->get() as $bot) {
$schedule->command('buy:cron '.$bot->id)->everyMinute();
}
}
And then handle it inside the RunBuy.php like so:
protected $signature = 'buy:cron {bot_id : The Bot ID we must perform actions with}';
public function __construct() {
parent::__construct();
}
public function handle() {
$bot_id = $this->argument('bot_id');
$this->bot = Bots::find($bot_id);
.......
}
But something like this wouldn't work since Laravel process each command one after another? I need them all to fire off at the same time.

If you are on a linux system you can call a command directly with final & to fork the process
$schedule->exec('php artisan buy:cron ' . $bot->id . ' > /dev/null &')->everyMinute();
On windows you can add a scheduled process to be called to run (a batch which accepts 1 param)
and call it as
$schedule->exec('schtasks /run /tn yourtask.bat ' . $bot->id)->everyMinute();
yourtask.bat should be a simple php artisan call
cd /path/to/project && php artisan buy:cron %1
or you can launch each process in background
function execInBackground($cmd) {
try {
if (substr(php_uname(), 0, 7) == "Windows") {
pclose(popen("start /B " . $cmd, "r"));
} else {
exec($cmd . " > /dev/null &");
}
} catch (\Exception $ex) {
throw $ex;
}
}

Related

How to run a background process in symfony

I have a symfony commad that needes few minutes to be completed. And i want to run it by a http request. The problem is that the server kills the command process just after sending response, he has a timeout.
i tried to use symfony process asynchronously But still not working ( i get always the timeout problem).
I dont want use the kernel.terminate event, since it is not a best practice. Here is the code in my controller :
$commandProcess=new Process('php bin/console app:doSomeThing');
$commandProcess->setWorkingDirectory('./../');
$commandProcess->start();
$commandProcess->setTimeout(50000000);
Any response will be much appreciated.
I had to desibale the process output and add '&' after the command
Here the answer:
new Process('php bin/console app:dosomthing &');
$commandProcess=new Process('php bin/console app:dosomthing &');
$commandProcess->setWorkingDirectory('./../');
$commandProcess->disableOutput();
$commandProcess->setTimeout(1800);
$commandProcess->start();
I write here the solution that has finally worked for me
First create a service:
use Symfony\Component\Process\Process;
use Symfony\Component\HttpKernel\KernelInterface;
class ProcessService {
public function __construct(
private KernelInterface $appKernel
){ }
public function executeProcessAsync($processToExecute) {
$rootDir = $this->appKernel->getProjectDir();
$process = Process::fromShellCommandline($processToExecute . ' &');
$process->setWorkingDirectory($rootDir);
$process->disableOutput();
$process->setTimeout(0);
$process->start();
}
}
Then we use the service from a controller:
class MyController {
public function __construct(
private ProcessService $procesService
){
public function __invoke(Request $request)
{
// (other stuff ...)
$this->procesService->executeProcessAsync('php bin/console mycommands:command ');
// (other stuff...)
}
}

Symfony command running inside another command dont work

Hellow , I need some hel regarding executing symfony commands inside another command. I am not new with this and I created many commands and run them from inside commands, controllers and it always work. But this one I do not understand why It do not working like the others are. I am running one command all the time and from time to time I created some extra workers when there are many jobs to get this one worker some help (one-check option).
I created command to run beanstalk worker with this class:
<?php
namespace AppBundle\Command;
use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Question\Question;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Command\LockableTrait;
/**
* Class BeanstalkWorkerCommand
* Command start the Beanstalk worker. Get the job from queue and preform job
*
* #method configure()
* #method execute(InputInterface $input, OutputInterface $output)
* #method authenticateUser(InputInterface $input, OutputInterface $output)
* #method checkAuthentication($username, $password)
*/
class BeanstalkWorkerCommand extends ContainerAwareCommand
{
use LockableTrait;
protected function configure()
{
$this
->setName('beanstalk:worker:start')
->setDescription('Start the Beanstalk infinitive worker. Get the job from queue and preform job')
->addOption(
'one-check',
'o',
InputOption::VALUE_NONE,
'If set, the worker will check tubes only once and died if there is no jobs in queue'
)
;
}
protected function execute(InputInterface $input, OutputInterface $output)
{
$output->writeln("\n<info>Beanstalk worker service started</info>");
$tubes = $this->getContainer()->get('app.job_manager.power_plant')->getTubes();
if ($input->getOption('one-check')) {
// run once
foreach ($tubes as $tubeName) {
if ($tubeName != "default") {
$this->getContainer()->get('app.queue_manager')->fetchQueue($tubeName);
}
}
$output->writeln("\n<info>Beanstalk worker completed check and stoped</info>");
} else {
// run forever
set_time_limit(0);
ini_set('xdebug.max_nesting_level', 1000);
if (!$this->lock()) {
$output->writeln('The command is already running in another process.');
return 0;
}
while (1) {
foreach ($tubes as $tubeName) {
if ($tubeName != "default") {
$this->getContainer()->get('app.queue_manager')->fetchQueue($tubeName);
}
sleep(0.1);
}
}
$output->writeln("\n<error>Beanstalk worker service has stoped</error>");
}
}
}
Than I run another command to create some extra workers with this functions:
public function startExtraWorkers(OutputInterface $output)
{
$numOfExtraWorkers = $this->getContainer()->getParameter('num_of_extra_beanstalk_workers');
for ($i=0; $i < $numOfExtraWorkers; $i++) {
$payload = [
"--one-check" => TRUE
];
$this->getContainer()->get('app.job_manager.queue')->createAndSendToQueue('beanstalk:worker:start', $payload, 10);
}
$output->writeln("\n<question>".$numOfExtraWorkers." extra benastalk workers started!</question>");
return TRUE;
}
public function createAndSendToQueue($command, $payload, $priority = 65536)
{
$jobData = $this->createJob($command, $payload);
return $this->job->enqueue($command, $jobData, $priority);
}
public function enqueue($job, array $args, $priority = 65536, $delay = 0, $ttr = 120)
{
if (!preg_match('/[a-z0-9\.]+/i', $job)) {
throw new InvalidArgumentException("Invalid job name");
}
$args = json_encode($args);
return $this->pheanstalk->put($args, $priority, $delay, $ttr);
}
And the problem is that if I run this command from terminal or with cron job it forks but if i run it like that with this function it do not work. I see that command has been executed but for some unknown reason it do not work.
If I executed this command i can see all commands has been executed bot they do not perform job like if i run the same command from terminal or with cron job:
ps ax |grep "beanstalk:worker:start --one-check"
Output (first one has been run from this function and second one with cron job. And only second one works):
31934 ? Ss 0:00 /bin/sh -c /usr/bin/php /var/www/mose-base/bin/console beanstalk:worker:start --one-check
31935 ? S 0:00 /usr/bin/php /var/www/mose-base/bin/console beanstalk:worker:start --one-check
Can any one give me some advice why is this not working like other commands? And why the same command run OK if i run it with cron job or inside terminal?
Thanks for help!
Try using:
/bin/sh -c '/usr/bin/php /var/www/mose-base/bin/console beanstalk:worker:start --one-check'
Mind the quotes.

Crontab do not run the php command

I am using Symfony 3 framework with pheanstalk php library. I run the app on server with Linux Debian Jesse. The job creation works ok and if run the worker from terminal it works like it should. But when I added the command to crontab I see that the command do not work. There is not any log in /var/main/user to help me with debuging. I will be very glad for any help.
This is my worker command (only the execute function):
protected function execute(InputInterface $input, OutputInterface $output)
{
$output->writeln("\n<info>Beanstalk worker service started</info>");
$tubes = $this->getContainer()->get('app.job_manager.power_plant')->getTubes();
if ($input->getOption('one-check')) {
// run once
foreach ($tubes as $tubeName) {
if ($tubeName != "default") {
$this->getContainer()->get('app.queue_manager')->fetchQueue($tubeName);
}
}
$output->writeln("\n<info>Beanstalk worker completed check and stoped</info>");
} else {
// run forever
set_time_limit(0);
max_execution_time(0);
while (1) {
foreach ($tubes as $tubeName) {
if ($tubeName != "default") {
$this->getContainer()->get('app.queue_manager')->fetchQueue($tubeName);
}
}
}
$output->writeln("\n<info>Beanstalk worker stoped</info>");
}
}
This is my app.queue_manager function to get job from queue and run command:
public function fetchQueue($tubeName)
{
if ($this->pheanstalk->getConnection()->isServiceListening()) {
while (true === is_object($job = $this->pheanstalk->watch($tubeName)->ignore('default')->reserve(self::WATCH_TIMEOUT))) {
$data = json_decode($job->getData(), true);
$this->worker($data);
$this->pheanstalk->delete($job);
}
}
}
And the worker to run command
public function worker($data)
{
$application = new Application($this->kernel);
$application->setAutoExit(false);
$parameters = [];
$parameters['command'] = $data['command'];
foreach ($data['meta'] as $key => $param) {
$parameters[$key] = $param;
}
$input = new ArrayInput($parameters);
$output = new NullOutput();
return $application->run($input, $output);
}
This is my crontab that do not work:
#reboot /usr/bin/php /var/www/mose-base/bin/console beanstalk:worker:start
I created another cron tab that works ok. It works every 15 min, and the difference is that do not have infinite loop (while(1)) so it goes only once thru tubes and than finished. But it is not what i want. I want infinite loop that works all the time like I created it with first crontab:
*/15 * * * * /usr/bin/php /var/www/mose-base/bin/console beanstalk:worker:start --one-check
If it works in the console, it doesn't work for your file user permissions. Check it.
You can report your error by email with this job
*/15 * * * * /usr/bin/php /var/www/mose-base/bin/console beanstalk:worker:start --one-check 2>&1 | mail -s "mysql_dump" example#mail.example
I used rc.local. It solve the problem. Tnx FreudianSlip

How to run custom Symfony2 command in background

Symfony2 enables developers to create their own command-line commands. They can be executed from command line, but also from the controller. According to official Symfony2 documentation, it can be done like that:
protected function execute(InputInterface $input, OutputInterface $output)
{
$command = $this->getApplication()->find('demo:greet');
$arguments = array(
...
);
$input = new ArrayInput($arguments);
$returnCode = $command->run($input, $output);
}
But in this situation we wait for the command to finish it's execution and return the return code.
How can I, from controller, execute command forking it to background without waiting for it to finish execution?
In other words what would be equivalent of
$ nohup php app/console demo:greet &
From the documentation is better use start() instead run() if you want to create a background process. The process_max_time could kill your process if you create it with run()
"Instead of using run() to execute a process, you can start() it: run() is blocking and waits for the process to finish, start() creates a background process."
According to the documentation I don't think there is such an option: http://api.symfony.com/2.1/Symfony/Component/Console/Application.html
But regarding what you are trying to achieve, I think you should use the process component instead:
use Symfony\Component\Process\Process;
$process = new Process('ls -lsa');
$process->run(function ($type, $buffer) {
if ('err' === $type) {
echo 'ERR > '.$buffer;
} else {
echo 'OUT > '.$buffer;
}
});
And as mentioned in the documentation "if you want to be able to get some feedback in real-time, just pass an anonymous function to the run() method".
http://symfony.com/doc/master/components/process.html

Using shell with Zend Framework

I'm curios if Zend has a component which can make use of the shell. For example I want to do an shell command like this:
mysqldump --compact --uroot --ppass mydatabase mydable >test.sql
from a controller.
If there isn't, do you know a way how to dump data from tables in Zend Framework?
update:
I've found a way here http://www.zfsnippets.com/snippets/view/id/68
There's no direct exec() support in the zend framework. the closest to command line support there is the Zend_Console class, but it's meant for getting arguments from the command line.
I would wrap the exec() function as a process object and work with that. Here's a nice example from the php docs:
<?php
// You may use status(), start(), and stop(). notice that start() method gets called automatically one time.
$process = new Process('ls -al');
// or if you got the pid, however here only the status() metod will work.
$process = new Process();
$process.setPid(my_pid);
?>
<?php
// Then you can start/stop/ check status of the job.
$process.stop();
$process.start();
if ($process.status()){
echo "The process is currently running";
}else{
echo "The process is not running.";
}
?>
<?php
/* An easy way to keep in track of external processes.
* Ever wanted to execute a process in php, but you still wanted to have somewhat controll of the process ? Well.. This is a way of doing it.
* #compability: Linux only. (Windows does not work).
* #author: Peec
*/
class Process{
private $pid;
private $command;
public function __construct($cl=false){
if ($cl != false){
$this->command = $cl;
$this->runCom();
}
}
private function runCom(){
$command = 'nohup '.$this->command.' > /dev/null 2>&1 & echo $!';
exec($command ,$op);
$this->pid = (int)$op[0];
}
public function setPid($pid){
$this->pid = $pid;
}
public function getPid(){
return $this->pid;
}
public function status(){
$command = 'ps -p '.$this->pid;
exec($command,$op);
if (!isset($op[1]))return false;
else return true;
}
public function start(){
if ($this->command != '')$this->runCom();
else return true;
}
public function stop(){
$command = 'kill '.$this->pid;
exec($command);
if ($this->status() == false)return true;
else return false;
}
}
?>
It's also let you stop and check the status of a job.

Categories