Beanstalk/Pheanstalk not working in background - php

Question

I am having troubles in setting up Pheanstalk on Ubuntu server.
I am relative new to programming, I have done all steps:
- installed beanstalk with sudo apt-get install beanstalk
- got pheanstalk from https://github.com/pda/pheanstalk/
And here is my code:
require_once('pheanstalk_init.php');
$pheanstalk = new Pheanstalk_Pheanstalk('127.0.0.1');
$pheanstalk
->useTube('testtube')
->put(exec("cat ../uploads/databases/app_data/Filename.sql | sqlite3 ../uploads/databases/Filename.sqlite"));
$job = $pheanstalk
->watch('testtube')
->ignore('default')
->reserve();
echo $job->getData();
$pheanstalk->delete($job);
The problem is that it takes 4-5 mins to run this code and for some reason the exec command does not run in background.
Any ideas on what am I doing wrong?
Many thanks in advance!

Answer 1

This is an old question, and probably you have solved it already ... anyway here is a short answer and more detailed explanation:
TL;DR : short answer: the problem was, that you run your (cpu/disk io) intensive code inside the put() method, and you needed to run it in the worker after the $job->getData() .
Explanation:
Beanstalk acts as a message queue, and allows you to decouple information producers from consumers. The idea is that you post just a description of the job that you want done to a queue and generating and posting that description to the queue is very quick. The actual resource (CPU/RAM/disk IO/network) consuming processing, happens in the consumer(s) (workers), when there are available ones, meanwhile the producer is free to do other stuff. If there are no workers available, then the jobs will just pile up in the queue.
So in your case, for example, separate your producer and consumers to chief.php and worker.php:
chief.php
<?php
require_once('pheanstalk_init.php');
$pheanstalk = new Pheanstalk_Pheanstalk('127.0.0.1');
$pheanstalk
->useTube('testtube')
->put("cat ../uploads/databases/app_data/Filename.sql | sqlite3 ../uploads/databases/Filename.sqlite");
?>
worker.php:
<?php
require_once('pheanstalk_init.php');
$pheanstalk = new Pheanstalk_Pheanstalk('127.0.0.1');
while(true){
$job = $pheanstalk->watch('testtube')->ignore('default')->reserve();
$cmd = $job->getData();
exec($cmd);
$pheanstalk->delete($job);
}
?>
So, when you run a worker, it will connect to the beanstalkd server, and wait for a job to be posted on the 'testtube' queue/tube . Then it will do the described job, and remove it from the queue, and loop, ready to process another job, when it arrives. You can run several workers in parallel too, if you want.
NB: The beanstalkd server does NOT run your workers; it only distributes jobs from producers to consumers. So you need to manage running worker processes yourself. I personally prefer to run workers as runit services, so that when there are any problem with them, they will be restarted by runit, and their output will be logged. You can of course use any process monitor tool you like: runit, systemd, monit, etc ...
It is also a good idea to run beanstalkd itself, under your favourite process monitor, just in case it crashes, which is unlikely, but can happen (it had happened to me an year ago).