PHP method execution only from one client at a time - php

I wrote a class that syncs the db from an xml file and reports through email any alerts.
The xml contains product prices and stock.
The execution of the method only occurs only if the xml filetime is newer than the last one synced.
Here is the first problem. I suspect that server (randomly) changes the filetime for some reason, becuse the sync method runs although no new xml file produced.
The xml file is exported from a local server and uploads to the remote server through an ftp client
(SyncBack)
Second problem is that on heavy traffic hours, the do_sync method runs more than once because i get the alerts more than once into my email.
I understand why it is called many times, so i created a flag syncing_now, to prevent the execution.
The mistake is that the flag is stored into db and since the first call has to update the db, all other call can run the method.
<?php class Sync extends Model{
public function __construct(){
parent::__construct();
$this->syncing_now = $this->db->get($syncing_now);
}//END constructor
public function index(){
if($this->determine_sync()){
$this->do_sync();
}else{
return FALSE;
}
}
public function determine_sync(){
if( filemtime($file) <= $this->db->last_sync() or !$this->$syncing_now){
return FALSE;
}else{
return TRUE;
}
}
public function do_sync(){
$this->db->update('syncing_now', TRUE);
//the sync code works fine..
$this->db->update('syncing_now', FALSE);
}
}
So what can i do to run the method only once and how can track down why the filetime change occurs?
Thanks all any help appreciated.

I suggest you use a Table that stores your Synchronisations.
id | md5_of_xml_file | synched_date
Now use LOCK_TABLES in order to ensure that only one process at a time may process your sync files.
Lock the synchronisations table. If locking them fails, just quit.
if (!mysqli_query('LOCK TABLES synchronisations WRITE')) {
die();//quit;
}
If an entry with the hash of the XML Sync File already exists, just quit.
$md5Hash = md5_file('yourXmlSyncFile.xml');
$result = null;
$stmt= $mysqli->prepare("SELECT md5_of_xml_file FROM synchronisations
WHERE md5_of_xml_file=?");
$stmt->bind_param("s", $md5Hash);
$stmt->execute();
$stmt->bind_result($result);
$stmt->fetch();
$stmt->close();
if ($result == $md5Hash) {
die();//quit;
}
Else, attempt to sync the file. If that works, add an entry, storing when you did this and a hash of the file used for synchronisation.

Related

Configuration File for Driving Selenium

I have about 500 possible paths to a particular page, and I need to test all of them. Each path to the that page looks similar to this (using PHP web driver; usually has about 10 steps):
// Navigate to form
$driver->get('http://www.domain.com');
$driver->get($driver->findElement(WebDriverBy::xpath("//a[contains(text(),'Foo 1')]"))->getAttribute('href'));
$driver->findElement(WebDriverBy::xpath("//div[#class='countryHeader']//a[contains(text(), 'Bar 1')]"))->click();
$driver->findElement(WebDriverBy::xpath("//form[#name='formDisclaimer']//input[contains(#class, 'button')]"))->click();
I don't want to have to write code for all the steps for all possible paths to the page. I do, however, have all the pertinent details of the steps (e.g. the XPath, the string the node may contain, etc.) in a database.
Is there a way for me to "dynamically" produce some sort of configuration file (either in XML or JSON) that I can feed to the driver as a set of instructions for it to follow?
A long time back at one of my project I had a similar requirement. I tried to create a Robot (or someone may call Web Crawler). As I started navigating through the pages I started maintaining the navigation paths in spreadsheet, so I don't have to click on the paths manually. Once I have the paths, next time whenever a Path changes I will be notified and if it is a valid change then make that change in s/s or raise it as a bug.
As you said you have all relevant details in the database then you just can simply read it and in a foreach loop pass to selenium driver.
or if you don't want to have a reference to the database in your test, just dump data to PHP array and add to your test class.
You just need to write a logic to transform your sql data into test. Don't need to write every test manually.
I don't know which testing framework you are using, but you can execute many tests from a single test for example in PHPUnit that would be something like:
class My_PathsTest extends PHPUnit_Framework_TestCase
{
public function setUp() {
// setup $this->tests here
}
public function testAll() {
// $this->tests would contain info about paths taken from database.
$failures = array();
foreach($this->tests as $paths_set) {
try {
/**
* $driver->get($paths_set['start_point']);
* foreach ($paths_set['paths'] as $path ) {
* $driver->findElement(WebDriverBy::xpath($path));
* }
*
* Important!!!
* If you didn't find something you expected
* just throw the PHPUnit_Framework_ExpectationFailedException exception
* throw new PHPUnit_Framework_ExpectationFailedException('Element missing add some info here about which is missing etc..');
*/
}
catch(PHPUnit_Framework_ExpectationFailedException $e) {
$failures[] = $e->getMessage();
}
}
if (!empty($failures)) {
throw new PHPUnit_Framework_ExpectationFailedException(count($failures) . " assertions failed:\n\t" . implode("\n\t", $failures));
}
}
}
best is to get data from db with odbc as a list (array) xpath locators and then loop over it.
If you don't have a direct access to the db, export the query results as a .csv file (MS db has an option save as, not sure about the others) and then read the file and loop over the array

How to call same name dynamic function from dynamically created files based database inputs

I'm creating a system that would work on call same function names from dynamic created php files.
The file structure is as bellow :
/root/caller.php //{uses pthreads & process multi simultaniously}
/root/rules/mode1.php
/root/rules/mode2.php
/root/rules/mode{n}.php // goes till n , so unlimited
Caller.php reads data from db and calls mode{n} accordingly. Sometimes there are chances to call multiple of rules at a point of time.
Sample of caller.php {written on OOPS}
foreach ($result as $row){
$mode = $rom->modeNum;
include 'rules/mode' . $mode . '.php';
call_fuction_inside_mode();
}
Sample of mode{n}.php {procedural programming}
function call_fuction_inside_mode(){
//..Custom function depending upon mode{n}
}
This won't function properly, because caller.php would include same function name again and again.
Please note :
I do not want to play with rename function with pecl, as all includes would be required
I have tried calling mode{n}.php by introducing another file "proxy.php"
And proxy.php works in below pattern
caller.php -> curl 127.0.0.1/proxy.php?mode={n} -> calls as defined
But this is not efficient way to do this when dealing with >500 requests per sec on micro server. I noticed many requests are killed and there's problem with order number. My target is to achieve this with least memory usage and 100% utilization of provided cpu resources.
I'm confused and so curious to find the solution I should be dealing here.
create an interface called Mode in Mode.php:
interface Mode{
public static function AcivateMode();
}
and include Mode.php into your code, then change the code in your mode{n}.php to encapsulate your function into classes that implements the interface Mode like so:
class Mode1 implements Mode {
public static function AcivateMode() {
//..Custom function depending upon mode{n}
}
}
then modify your foreach to activate the modes like:
foreach ($result as $row){
$mode = $rom->modeNum;
include 'rules/mode' . $mode . '.php';
call_user_func("Mode$mode::AcivateMode");
}
OR
you can use the namespaces approach by adding the namespace to each of your mode{n}.php files like so:
mode1.php
namespace Mode1;
function call_fuction_inside_mode(){....}
and calling the function using:
call_user_func("Mode$mode\call_fuction_inside_mode");
that worked for me, hope it's gonna be helpful :)
Regards,

PHP OO - How should I handle soft errors in multiple classes?

I apologise if this has already been answered somewhere, but I haven't managed to find an answer so far - maybe I'm searching for the wrong thing!
I am trying to figure out how to handle errors in my OO PHP system, which is used to generate web pages. Hopefully this example will explain what I mean.
Imagine I have a Content class, a Form class and a FormObject class, which hold all the information on page content, web forms and form fields. All classes can run multiple MySQL queries via the DB class.
Users can create new content or forms in the back-end. When they do this, I use the classes to create and store the data in the database.
I also have a System class, which is used to generate the web pages. The System class checks what should be displayed on the front-end, builds the appropriate Content and Form objects, then generates the HTML and outputs it to the screen.
I have some checks for serious errors, which stop the code from going any further. However, the problem is that I want to feed back some "soft errors" to the front-end. For example, maybe the System class builds a Form object, which in-turn builds the fields using the FormObject class. The FormObject class queries the database for a field name, but a field name is not found. So the DB class returns an error. I want to be able to feed back a message to the front-end that says the field name has not been found.
What is the best way to get that "soft error" message back to the System class, so it can be outputted to the front-end?
I realise it is fairly simple in this particular example, but as more classes are added and, crucially, more levels are added, the problem becomes a bit bigger.
One way I thought of doing this was to have an Error class. The system would create an Error object and pass it on to each Content and Form object as they are created. The Form class would pass the same Error object to the FormItem class. Whenever an error is found, it is logged via a method in the Error class. The system can then access the original Error object and output all the errors. However, as the system grows, more classes are added, and more objects are created, it could get quite confusing. Is there a better way?
You might want to use either
something global that all classes can access (e.g. a global variable or a Singleton), or
something that is passed in to all instantiations of classses producing what you call 'soft errors'
to collect such errors. You then want to use whatever you collected and add it to the output in your System class somehow.
To be more specific...
This is an example for the solution using a global:
global $softErrorMessages = array();
class A
{
function sampleFunctionA()
{
// [...]
// some code setting $result to some valid value
// or to false if an error occured
if($result === false) // check for validity
{
global $softErrorMessages;
$softErrorMessages[] = "The sample function A caused a soft error";
return;
}
// [...]
// some code requiring a valid $result
}
}
If you use such a global, you can then easily access it from your System class and put its contents into the right places of your output.
However, if you perform unit tests, you might not want to use globals or global-like solutions (like singletons). So here is an example for an 'error collection' approach:
class ErrorCollector
{
private $errors = array();
function addError($error)
{
$this->errors[] = $error;
}
function getErrors()
{
return $this->errors;
}
}
class A
{
private $errorCollector;
function __construct(/* your additional parameters */, ErrorCollector $errorCollector)
{
// [...]
// additional instantiation stuff
$this->errorCollector = $errorCollector;
}
function sampleFunctionA()
{
// [...]
// some code setting $result to some valid value
// or to false if an error occured
if($result === false) // check for validity
{
$this->errorCollector->addError("The sample function A caused a soft error");
return;
}
// [...]
// some code requiring a valid $result
}
}
You would instantiate the ErrorCollector only once and then pass it to all other class instantiations. Then you let your objects perform their duties (and possibly add soft errors to the ErrorCollector). Once they're done, your System class would then get all the error messages and - again - place them at the right place of your output.
Exceptions is a convenient mechanism to handle errors. FormObject can throw an exception of some SoftErrorException class if DB returns an error. And then in System you are catching this exception and render it to front-end.
class System {
public function showFormAction() {
try {
$form = ... // create a form
$this->renderForm($form);
} catch (SoftErrorException $e) {
$this->handleSoftError($e);
}
}
public function handleSoftError(SoftErrorException $e)
{
// Do whatever you want with exceptions: render it
// $this->renderErrorPage($e->getMessage());
// or collect them and show after
// $this->errors[] = $e;
}
}

PHP concurrency issue, multiple simultaneous requests; mutexes?

So I've just realised that PHP is potentially running multiple requests simultaneously. The logs from last night seem to show that two requests came in, were processed in parallel; each triggered an import of data from another server; each attempted to insert a record into the database. One request failed when it tried to insert a record that the other thread had just inserted (the imported data comes with PKs; I'm not using incrementing IDs): SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '865020' for key 'PRIMARY' ....
Have I diagnosed this issue correctly?
How should I address this?
The following is some of the code. I've stripped out much of it (the logging, the creation of other entities beyond the Patient from the data), but the following should include the relevant snippets. Requests hit the import() method, which calls importOne() for each record to import, essentially. Note the save method in importOne(); that's an Eloquent method (using Laravel and Eloquent) that will generate the SQL to insert/update the record as appropriate.
public function import()
{
$now = Carbon::now();
// Get data from the other server in the time range from last import to current import
$calls = $this->getCalls($this->getLastImport(), $now);
// For each call to import, insert it into the DB (or update if it already exists)
foreach ($calls as $call) {
$this->importOne($call);
}
// Update the last import time to now so that the next import uses the correct range
$this->setLastImport($now);
}
private function importOne($call)
{
// Get the existing patient for the call, or create a new one
$patient = Patient::where('id', '=', $call['PatientID'])->first();
$isNewPatient = $patient === null;
if ($isNewPatient) {
$patient = new Patient(array('id' => $call['PatientID']));
}
// Set the fields
$patient->given_name = $call['PatientGivenName'];
$patient->family_name = $call['PatientFamilyName'];
// Save; will insert/update appropriately
$patient->save();
}
I'd guess that the solution would require a mutex around the entire import block? And if a request couldn't attain a mutex, it'd simply move on with the rest of the request. Thoughts?
EDIT: Just to note, this isn't a critical failure. The exception is caught and logged, and then the request is responded to as per usual. And the import succeeds on the other request, and then that request is responded to as per usual. The users are none-the-wiser; they don't even know about the import, and that isn't the main focus of the request coming in. So really, I could just leave this running as is, and aside from the occasional exception, nothing bad happens. But if there is a fix to prevent additional work being done/multiple requests being sent of to this other server unnecessarily, that could be worth pursuing.
EDIT2: Okay, I've taken a swing at implementing a locking mechanism with flock(). Thoughts? Would the following work? And how would I unit test this addition?
public function import()
{
try {
$fp = fopen('/tmp/lock.txt', 'w+');
if (flock($fp, LOCK_EX)) {
$now = Carbon::now();
$calls = $this->getCalls($this->getLastImport(), $now);
foreach ($calls as $call) {
$this->importOne($call);
}
$this->setLastImport($now);
flock($fp, LOCK_UN);
// Log success.
} else {
// Could not acquire file lock. Log this.
}
fclose($fp);
} catch (Exception $ex) {
// Log failure.
}
}
EDIT3: Thoughts on the following alternate implementation of the lock:
public function import()
{
try {
if ($this->lock()) {
$now = Carbon::now();
$calls = $this->getCalls($this->getLastImport(), $now);
foreach ($calls as $call) {
$this->importOne($call);
}
$this->setLastImport($now);
$this->unlock();
// Log success
} else {
// Could not acquire DB lock. Log this.
}
} catch (Exception $ex) {
// Log failure
}
}
/**
* Get a DB lock, returns true if successful.
*
* #return boolean
*/
public function lock()
{
return DB::SELECT("SELECT GET_LOCK('lock_name', 1) AS result")[0]->result === 1;
}
/**
* Release a DB lock, returns true if successful.
*
* #return boolean
*/
public function unlock()
{
return DB::select("SELECT RELEASE_LOCK('lock_name') AS result")[0]->result === 1;
}
Your example code would block the second request until the first is finished. You would need to use LOCK_NB option for flock() to return error immediately and not wait.
Yes you can use either locking or semaphores, either on filesystem level or directly in the database.
In your case when you need each import file to be processed only once, the best solution would be to have a SQL table with row for each import file. At the beginning of import, you insert the info that import is in progress, so other threads will know to not process it again. After import is finished, you mark it as such. (Then few hours later you can check the table to see if the import really finished.)
Also it is better to do such one-time long-lasting things like import on separate scripts and not while serving normal webpages to visitors. For example you can schedule a nightly cron job which would pick up the import file and process it.
It doesn't seem like you are having a race condition, because the ID is coming from the import file, and if your import algorithm is working correctly then each thread would have its own shard of the work to be done and should never conflict with others. Now it seems like 2 threads are receiving a request to create the same patient and get in conflict with eachother because of bad algorithm.
Make sure that each spawned thread gets a new row from the import file, and repeat only on failure.
If you cant do that, and want to stick to mutex, using a file lock doesn't seem like a very nice solution, since now you solved the conflict within the application, while it is actually occurring in your database. A DB lock should be a lot faster too, and overall a more decent solution.
Request a database lock, like this:
$db -> exec('LOCK TABLES table1 WRITE, table2 WRITE');
And you can expect a SQL error when you would write to a locked table, so surround your Patient->save() with a try catch.
An even better solution would be to use a conditional atomic query. A DB query that also has the condition within it. You could use a query like this:
INSERT INTO targetTable(field1)
SELECT field1
FROM myTable
WHERE NOT(field1 IN (SELECT field1 FROM targetTable))
I see three options:
- use mutex/semaphore/some other flag - not easy to code and maintain
- use DB built-in transaction mechanism
- use queue (like RabbitMQ or 0MQ) to write messages into DB in a row

How can I run a function from a different server?

I have a model in codeigniter which contains a function that pings servers and stores data in an external server. I am trying to use my main server to call the function on the slave server. Here is the function:
public function check_online($ip, $port){
if(!$socket = stream_socket_client('tcp://'.$ip.':'.$port, $this->timeout)) {
return FALSE;
} else {
return TRUE;
}
}
Help would be very greatly appreciated.
What you're describing seems to be a remote invocation of a function on a different machine.
Usually, this requires you to set up a function stub at your client end and set up a channel. But since you're using PHP, you can use a simple POST request to accomplish the same.
Here is an example: PHP: Remote Function Call and returning the result?
However, if the remote function only "stores" the data in (say) MySQL, then I would suggest you to re-design your application so that you don't need a separate function on a different server to do the store operation.

Categories