Despite its inadvisability, using PHP's shell commands to interact with non-php system commands remains a common way of quickly achieving certain results in web applications.
Has anyone abstracted out the common use cases into a class library (something in Zend maybe?) that offers a more sane/common way of handling this? Every time I encounter (or have to produce) this kind of code it's a bunch of procedural spaghetti, copy-pasted over and over again. I was wondering if (hoping that) the PHP community had come up with a better way of handling using command line applications in your web/php applications.
Executing commandline applications is nothing dirty. In fact, it's the Unix way. And most mostly it's saner than trying to reimplement e.g. ImageMagick in pure PHP code. (Due to the disparity of its cmdline args, imagemagick is a bad example case if you look for a nice exec() abstraction.)
There isn't much wrapping up you can do. At best you can summarize in-/output to your external binary in a method:
function exec($args) {
$args = implode(" ", array_map("escapeshellcmd", func_get_args()));
$opts = $this->opts();
return `{$this->bin} {$args} {$opts}`;
}
So you just call ->exec("-o", "$file") where needed. Your code can only be gneralized further with specialized exec submethods, if the particular cmdline app has an inherent system in its --argument naming scheme.
Depending on your actual use case, you might be able to stash a few standard options away. I did this for pspell, where you have an almost 1:1 relationship of option names to --cmdline=args:
function opts() {
$map = array(
"--ignore" => $this->ignore,
"--verbose" => $this->verbose,
"--dir={$this->dir}" => isset($this->dir),
);
return implode(" ", array_keys(array_intersect($map, array(1=>1))));
}
A very generic abstraction class for exec/popen (for a wide range of cmdline programs) probably doesn't exist.
Related
Sorry for the very broad question but we have some problems, e.g. Checkmarx is complaining about code injection in something like the following
$accesskey = $_GET['accesskey'] ?? $argv[1] ?? null;
if (!$accesskey || !ctype_alnum($accesskey)) {
throw new RuntimeException(sprintf('Passed accesskey "%s" is invalid', $accesskey));
}
$commandParts = ['echo', $accesskey]
$commandParts = array_map('escapeshellarg', $commandParts);
$command = implode(' ', $commandParts);
$command = escapeshellcmd($command);
system($command);
I think the commands are escaped and everything is fine but why is Checkmarx thinking different?
The application's <?php method calls an OS (shell) command with system, at line 1 of REDACTED, using an untrusted string with the command to execute.
This could allow an attacker to inject an arbitrary command, and enable a Command Injection attack.
The attacker may be able to inject the executed command via user input, _GET, which is retrieved by the application in the <?php method, at line 1 of REDACTED.
I'm also wondering if and how Checkmarx is able to understand library or framework code which is installed via Composer? E.g.
Assert::oneOf($unsafeUserInput, ['foo', 'bar']); // throws an Exception if $unsafeUserInput is not 'foo' or 'bar'
// $unsafeUserInput is now safe
or WP related stuff which is also often falsely flagged as being prone to SQL injections
global $wpdb;
$foo = $wpdb->getVar($wpdb->prepare('SELECT foo FROM bar WHERE baz = %s', $_GET['baz'] ?? ''));
If it checks for sanitisation methods is there a specific way they have to look? I honestly want to avoid changing too much code for Checkmarx.
Your question of how well Checkmarx analyzes PHP code could lean towards a subjective answer and your perception of the tool can be biased given that you are using methods (escapeshellcmd) that are not recognized as sanitizers and the framework that you are inquiring about (Wordpress and Composer) are not technically supported.
In fairness to Checkmarx, they do support a variety of PHP frameworks such as Zend, Kohana, CakePHP, Symfony and Smarty which could end in lesser false positives (note: I'm not suggesting you switch platforms)
Any static analyzers would need some help from it users for it to be effective. I would advice you to exclude the Composer files from the scan.
You don't really don't have to make changes to the code and just argue with your AppSec team that these findings are false positives since the prepare method prevent SQL injection attacks and that escapeshellcmd does encode string. My recommendation however is to use escapeshellarg on $accesskey instead
I want to be able to call the CakeS3 plugin from the Cake Shell. However, as I understand it components cannot be loaded from the shell. I have read this post outlining strategies for overcoming it: using components in Cakephp 2+ Shell - however, I have had no success. The CakeS3 code here is similar to perfectly functioning cake S3 code in the rest of my app.
<?php
App::uses('Folder','Utility');
App::uses('File','Utility');
App::uses('CakeS3.CakeS3','Controller/Component');
class S3Shell extends AppShell {
public $uses = array('Upload', 'User', 'Comment');
public function main() {
$this->CakeS3 = new CakeS3.CakeS3(
array(
's3Key' => 'key',
's3Secret' => 'key',
'bucket' => 'bucket')
);
$this->out('Hello world.');
$this->CakeS3->permission('private');
$response = $this->CakeS3->putObject(WWW_ROOT . '/file.type' , 'file.type', $this->CakeS3->permission('private'));
if ($response == false){
echo "it failed";
} else {
echo "it worked";
}
}
This returns an error of "Fatal error: Class 'CakeS3' not found in /home/app/Console/Command/S3Shell.php. The main reason I am trying to get this to work is so I can automate some uploads with a cron. Of course, if there is a better way, I am all ears.
Forgive me this "advertising"... ;) but my plugin is probably better written and has a better architecture than this CakeS3 plugin if it is using a component which should be a model or behaviour task. Also it was made for exactly the use case you have. Plus it supports a few more storage systems than only S3.
You could do that for example in your shell:
StorageManager::adapter('S3')->write($key, StorageManager::adapter('Local')->read($key));
A file should be handled as an entity on its own that is associated to whatever it needs to be associated to. Every uploaded file (if you use or extend the models that come with the plugin, if not you have to take care of that) is stored as a single database entry that contains the name of the config that was used and some meta data for that file. If you do the line of code above in your shell you will have to keep record in the table if you want to access it this way later. Just check the examples in the readme.md out. You don't have to use the database table as a reference to your files but I really recommend the system the plugin implements.
Also, you might not be aware that WWW_ROOT is public accessible, so in the case you store sensitive data there it can be accessed publicly.
And finally in a shell you should not use echo but $this->out() for proper shell output.
I think the App:uses should look like:
App::uses('CakeS3', 'CakeS3.Controller/Component');
I'm the author of CakeS3, and no I'm afraid there is no "supported" way to do this as when we built this plugin, we didn't need to run uploads from shell and just needed a simple interface to S3 from our controllers. We then open sourced the plugin as a simple S3 connector.
If you'd like to have a go at modifying it to support shell access, I'd welcome a PR.
I don't have a particular road map for the plugin, so I've tagged your issue on github as an enhancement and will certainly consider it in future development, but I can't guarantee that it would fit your time requirements so that's why I mention you doing a PR.
Firstly, I will say that I come from the Java world (this is important, really).
I have been coding PHP for a while, one of the problems that I have encountered is that due to the lack of compilation, sometimes errors that could be easily detected at compilation time (for example, wrong number of parameters for a given function), can silently pass.
That could be easily detected as code coverage increases by adding unit tests. The question is, does it make sense for example to tests constructors in order to check that the passed parameters are correct? I do not mean only the number of parameters, but also the content of such parameters (for example, if a parameter is null, certain objects should launch an exception in order to avoid creating a "dirty" object).
Question is, am I too contaminated by years of Java code? Because after all, increasing the code coverage to "discover" missued functions feels like a (really) primitive way of compiling.
Also, I would like to note that I already use a development environment (PHPStorm), we are also using tools like PHPCodeSniffer.
Any ideas/suggestions?
This is a good question that can be answered on a number of levels:
Language characteristics
Test coverage
CASE tools
1. Language characteristics
As you have pointed out the characteristics of the PHP language differ markedly from the more strongly-typed languages such as Java. This raises a serious issue where programmers coming from the more strongly-typed languages such as Java and C# may not be aware of the implications of PHP's behaviour (such as those you have described). This introduces the possibility of mistakes on the part of the programmer (for example, a programmer who may have been less careful using Java because they know the compiler will catch incorrect parameters may not apply the appropriate care when developing in PHP).
Consequently, better programmer education/supervision is needed to address this issue (such as in-house company coding standards, pair programming, code review). It also (as you have pointed out) raises the question of whether test coverage should be increased to check for such mistakes as would have been caught by a compiler.
2. Test Coverage
The argument for test coverage is very project-specific. In the real world, the level of test coverage is primarily dictated by the error tolerance of the customer (which is dictated by the consequences of an error occuring in your system). If you are developing software that is to run on a real-time control system, then obviously you will test more. In your question you identify PHP as the language of choice; this could apply equally to the ever-increasing number of web-enabled frontends for critical systems infrastructure. On the other side of the coin, if you are developing a simple website for a model railroad club and are just developing a newsletter app then your customer may not care about the possibility of a bug in the constructor.
3. CASE Tools
Ultimately it would be desirable for a CASE tool to be available which can detect these errors, such as missing parameters. If there are no suitable tools out there, why not create one of your own. The creation of a CASE tool is not out of reach of most programmers, particularly if you can hook into an open-source parsing engine for your language. If you are open-source inclined this may be a good project to kick start, or perhaps your company could market such a solution.
Conclusion
In your case whether or not to test the constructors basically comes down to the question: what will the consequences of a failure in my system be? If it makes financial sense to expend extra resources on testing your constructors in order to avoid such failures, then you should do so. Otherwise it may be possible to get by with lesser testing such as pair programming or code reviews.
Do you want the constructor to throw an exception if invalid parameters set? Do you want it to behave that same way tomorrow and next week and next year? Then you write a test to verify that it does.
Tests verify that your code behaves as you want it to. Failing on invalid parameters is code behavior just as much as calculating sales tax or displaying a user's profile page.
We test constructors, as well as the order of the parameters, the defaults when not provided, and then some actual settings. For instance:
class UTIL_CATEGORY_SCOPE extends UTIL_DEPARTMENT_SCOPE
{
function __construct($CategoryNo = NULL, $CategoryName = NULL)
{
parent::__construct(); // Do Not Pass fields to ensure that the array is checked when all fields are defined.
$this->DeclareClassFields_();
$this->CategoryName = $CategoryName;
$this->CategoryNo = $CategoryNo;
}
private function DeclareClassFields_()
{
$this->Fields['CategoryNo'] = new UTIL_ICAP_FIELD_PAIR_FIRST('CCL', 6, ML('Category'), 8);
$this->Fields['CategoryName'] = new UTIL_ICAP_FIELD_PAIR_SECOND('CCL', 32, ML('Name'), 15, array(), array(), NULL, UTIL_ICAP_FIELD::EDIT_DENY, UTIL_ICAP_FIELD::UPDATE_DENY, 'DES');
}
}
We then create our tests to not only check the constructor and its order, but that class and inheritance has not changed.
public function testObjectCreation()
{
$CategoryInfo = new UTIL_CATEGORY_SCOPE();
$this->assertInstanceOf('UTIL_CATEGORY_SCOPE', $CategoryInfo);
$this->assertInstanceOf('UTIL_DEPARTMENT_SCOPE', $CategoryInfo);
$this->assertInstanceOf('UTIL_DATA_STRUCTURE', $CategoryInfo); // Inherited from UTIL_DEPARTMENT_SCOPE
}
public function testConstructFieldOrder()
{
$CategoryInfo = new UTIL_CATEGORY_SCOPE(1500, 'Category Name');
$this->assertEquals(1500, $CategoryInfo->CategoryNo);
$this->assertEquals('Category Name', $CategoryInfo->CategoryName);
}
public function testConstructDefaults()
{
$CategoryInfo = new UTIL_CATEGORY_SCOPE();
$this->assertNull($CategoryInfo->CategoryNo);
$this->assertNull($CategoryInfo->CategoryName);
}
public function testFieldsCreated()
{
$CategoryInfo = new UTIL_CATEGORY_SCOPE();
$this->assertArrayHasKey('CategoryNo', $CategoryInfo->Fields);
$this->assertArrayHasKey('CategoryName', $CategoryInfo->Fields);
$this->assertArrayHasKey('DeptNo', $CategoryInfo->Fields); // Inherited from Parent
$this->assertArrayHasKey('DeptName', $CategoryInfo->Fields); // Inherited from Parent
}
I am just getting started with graph databases. I would like to talk to them in a PHP application. In particular, I am looking at OrientDB, mainly because of its licensing benefits and features over neo4j.
What is the recommended way to interact with OrientDB within PHP? Is there a generalized framework like tinkerpop for the PHP world?
I would like to query my graphs using mainly gremlin and a bit of OrientDB's extended SQL (only if necessary).
I know there are 2 php connectors: Orient and OrientDB-PHP, but have not tried them yet. Has anyone had any experiences with them? What are the pros and cons? I would of course prefer something like tinkerpop for PHP if it exists, but if it doesn't a library geared towards OrientDB is fine too.
I'm Alex, I actually wrote the biggest part of the codebase for Orient (the php library).
In PHP, currently, there is no thing like tinkerpop but I think you should be able to kickstart with Orient. Using it its extremely simple:
<?php
namespace Congow\Orient;
use Congow\Orient\Binding\HttpBinding;
use Congow\Orient\Binding\BindingParameters;
require __DIR__.'/../autoload.php';
$parameters = BindingParameters::create('http://admin:admin#127.0.0.1:2480/friends');
$binding = new HttpBinding($parameters);
$response = $binding->query('select from friends where any() traverse(0,1) ( #rid = #5:3 ) and #rid <> #5:3');
$friends = $response->getResult();
foreach ($friends as $friend) {
echo $friend->name, "\n";
}
Instead of writing the query from scratch you can also use the query builder:
$query = new Select(array('myClass'));
$query->orderBy("name ASC", false);
echo $query->getRaw() // SELECT FROM myClass ORDER BY name ASC
You could have a look at the tests of the library, or at the mini-samples in the example directory.
We are slowly keep going on with the development, so you find a few resources here: http://odino.org/blog/categories/orientdb/
Cheers,
I know It's been a while but I wanted to point out that you can use the Tinkerpop stack in PHP via available drivers.
These will allow you a certain level of abstraction over the underlying database you choose -- thus allowing you to switch DBs when needed (Neo4j, OrientDB, Titan, etc.).
TinkerPop 3
You can use Gremlin server (new name for rexster) via the php driver available:
gremlin-php check this tutorial
Tinkerpop 2 (no longer maintained)
You can use the Rexster server from the stack via the php drivers available:
rexpro-php check this article
Doolittle
rexpro-php-driver (PHP ext.)
I'm looking for a simpler test framework. I had a look at a few PHPUnit and SimpleTest scripts and I find the required syntactic sugar appalling. SnapTest sounded nice, but was as cumbersome. Apache More::Test was too procedural, even for my taste. And Symfony lime-test was ununique in that regard.
BDD tools like http://everzet.com/Behat/#basics are very nice, but even two abstraction levels higher than desired.
Moreover I've been using throwaway test scripts till now. And I'm wondering if instead of throwing them away, there is a testing framework/tool which simplifies using them for automated tests. Specifically I'd like to use something that:
evaluates output (print/echo), or even return values/objects
serializes and saves it away as probe/comparison data
allows to classify that comparison output as passed test or failure
also collects headers, warning or error messages (which might also be expected output)
in addition to a few $test->assert() or test::fail() states
Basically I'm too lazy to do the test frameworks work, manually pre-define or boolean evaluate and classify the expected output. Also I don't find it entertaining to needlessly wrap test methods into classes, plain include scripts or functions should suffice. Furthermore it shouldn't be difficult to autorun through the test scripts with a pre-initialized base and test environment.
The old .phpt scripts with their --expect-- output come close, but still require too much manual setup. Also I'd prefer a web GUI to run the tests. Is there a modern rehersal of such test scripts? (plus some header/error/result evalation and eventually unit test::assert methods)
Edit, I'll have to give an example. This is your typical PHPUnit test:
class Test_Something extends PHPUnit_Test_Case_Or_Whatever {
function tearUp() {
app::__construct(...);
}
function testMyFunctionForProperResults() {
$this->assertFalse(my_func(false));
$this->assertMatch(my_func("xyzABC"), "/Z.+c/");
$this->assertTrue(my_func(123) == 321);
}
}
Instead I'd like to use plain PHP with less intermingled test API:
function test_my_function_for_proper_results() {
assert::false(my_func(false));
print my_func("xyz_ABC");
return my_func(123);
}
Well, that's actually three tests wrapped in one. But just to highlight: the first version needs manual testing. What I want is sending/returning the test data to the test framework. It's the task of the framework to compare results, and not just spoon-feeded booleans. Or imagine I get a bloated array result or object chain, which I don't want to manually list in the test scripts.
For the record, I've now discovered Shinpuru.
http://arkanis.de/projects/shinpuru/
Which looks promising for real world test cases, and uses PHP5.3-style anonymous functions instead of introspection-class wrappers.
Have to say - it isn't obvious how your example of a simplified test case would be possible to implement. Unfortunately the convolutedness is - more or less - something that has to be lived with. That said, I've seen cases where PHPUnit is extended to simplify things, as well as adding web test runners, tests for headers, output etc (thinking SilverStripe here - they're doing a lot of what you want with PHPUnit). That might be your best bet. For example:
evaluates output (print/echo):
enable output buffering and assert against the buffer result
collect headers, warning or error messages
register your own handler that stores the error message
wget against urls and compare the result (headers and all)
Etc.