I have been having some problems with a Magento site recently and am looking for a way to check the integrity of a Magento site at any given point.
Unit Testing jumps out as one method of doing this but I would assume it would be a very big job to write a whole lot of tests to check everything in the site is working as it should.
Can anyone involved in unit testing and magento advise on the following:
Is it possible to test the whole site and not just custom modules -
is so some examples of tests would be amazing.
Given that the site is heavily linked to the database - how would it
be possible to fully test the site without disturbing the database
Are there any better ways to automaticlly check the integrity of a
magento site
When I say integrity i really mean that there are no faults on the site - shipping, payment etc are all working correctly.
This is a big task, however there are Magento community members who have tackled it.
The EcomDev_PHPUnit module provides a framework for unit testing Magento, but it doesn't contain any actual tests. It could (and has) been used to test core functionality or modules that you have developed yourself.
One of the key advantages of the EcomDev module is that it does provide 100% isolation of your database. It creates an exact copy of your database structure, and then uses fixtures (see pg 6 of the manual) to insert data into those tables to create test pre-requisites. This is powerful and best practice, but does require quite a bit of setup.
You can try using phpMyAdmin to export data into YAML in readiness for creating fixtures.
The alternative is to create and automate a comprehensive Selenium test suite for the browser UI. In fact, the best solution is to prepare both unit and UI tests as there will be areas that can only be tested in one functional domain. There is a significant amount of business logic built into Magento's Javascript (all the validation.js for example) that PHPUnit can't easily test, Selenium is your best option here.
There have been early conversations about creating a repository of unit tests to cover the core functionality, however keep in mind that Magento 2.0 (planned for 2012) advertises complete test coverage.
Related
I am developing TYPO3 projects since 2006 now, and projects are getting bigger and more complex. Setting up a simple CMS site with a contact form and news listing is all routine.
Right now, we finished a bigger project: A platform for an international company with countless extensions:
Login & registration, news, listing database records, dynamic contact forms, surveys & statistics, intranet functions: document upload & download, several backend "tweaks" per TCA modifications, etc. .
The project managers got upset at us developers, because sometimes, after we finished on function X and later committed function Y to the dev server, function X was broken. This was related to typoscript settings, extension interdependencies, versioning errors or sometimes simple programming mistakes and typos.
I know how to take care of the latter, but in general:
From your experience:
How can we develop an error-proof system in TYPO3, where everything works in hand and extensions don't get in their way? In other words:
How can we secure and isolate functionalities (extensions) - and avoid those interdepency issues?
We are working in a DEV team with two developers, and we already use:
Subversion Repository
Local DEV server for development & testing
External typoscript configuration files, split into single files for each extension
Edit for Bountyhunters:
What I am looking for is a best-practice-summary that might include these topics:
General workflow habits
General coding habits
Reliability of our subversion commits (or Git)
Unit testing (PHPUnit, Selenium?)
Deployment (I haven't yet figured out how automated deployment can
help us)
Typoscript best practices
Problems we could find in large TYPO3 projects are not to much different from any development project.
General practices :
configure continuous integration platform with continuous deployment tools;
Test Driven Development with automated testing;
robust architecture (dB, URL routing, ...);
performance tests during development;
use versionning with formatted comments;
use powerful IDE as PHPStorm, Eclipse, Netbeans;
Common TYPO3 practices :
use official API;
follow TYPO3 API code guideline;
use TYPO3 hooks where You can if You need to modify the logic of Core or 3rd party extensions;
use TypoScript constants to separate data from logic in Your configuration;
Additional references :
TYPO3 Best Practice Workshop
TYPO3 Best Practices (de)
TDD & Best Practices mit TYPO3 (de)
Extensions could help to manage complex TYPO3 installation :
caretaker
additional reports
Use modern project management methodologies & tools
Scrum, Kanban, lean developement principles
Bugtrackers as Redmine, Trac
Books :
php|architect's Guide to Enterprise PHP Development
I absolutely recommmend start using PHPUnit for unit testing, but remember unit testing is really about how you create the code in the first place, not usually something you add later. But of course, better late than never.
You should consider setting up a build server like jenkins/hudson or atlassian bamboo. The latter is quite nice and integrates with zend studio which in my opinion is the better choice when developing in PHP. In general the atlassian products are widely used for software projects. (Jira + confluence + greenhopper in particular)
I would also recommend setting up phpunit on jenkins - see http://jenkins-php.org/ as a template, although I've read good feedback about Teamcity. Then, depending on the code you write you setup unit tests (for raw php code, maybe a bit with of mocks), integration tests (API and module connectivity) and system tests (selenium).
Once you have it running after each build you can be sure that at least covered functionality is working. The problem however is that you will spend more time on writing tests and their support as well as thinking about testable code. Also keep in mind that you cannot cover everything - thats not the point. You must have critical paths covered.
This is more or less related to project management and also with every developer. How you guys handle this situation when you have developed many features on development site and all are tested by client and ready to go live.
These features have some code in common files ie. One PHP file have the code for one feature as well as one other feature.
But client will ask you to upload only 2 feature out of 10 or 15. Files are common if you upload that file directly will leads to error problems because they have code for other features. If you upload all updated files then all feature will be live.
A possible way is go back and comment out that feature which is not needed live for now from common files. But there is possiblities to forgot to comment anywhere else.
This is also not a good way and at last client will say what happen everything was tested on development server and why these bugs and errors are introduced on live server.
This will reduce the faith on developers.
I faced this problem many times and could not found any good way to avoid these issues. So I am thinking that you guys also facing or faced this problem.
I am thinking versioning system can help here.
How you guys are handling this?
Could you share ideas?
The situation you are describing is impossible to manage sanely. I don't believe it would be possible to make this situation work, but the real question is why would you want to?
There are a number of issues with the scenario you describe, but the core issue is really this. You are testing one thing, and deploying another. You acknowledge in your question the interconnected nature of changes. In reality it is even more difficult than you describe. You simply cannot know how a system will behave when you try and deploy parts of a tested solution. Why test it at all?
The only sensible solution I can see is to have a sandbox environment where new features are demonstrated. However keep your test enviornment only for testing stuff that will go live. So in your example the one or two features are in test, ready to be signed off for prod, and the other featues are locked in the sandbox.
This leads to the next problem, which is managing your source code. I don't see any sane strategy for managing the arbirtrary inclusion of features from a code base. Even under the mostflexible system I know, Perforce, any branching straegy would require awful resolves on merges as you try to move stuff in and out.
I have seen this happen, and believe me it gets very ugly.
I suggest you come up with a better solution. Talk to your client and change the way things are done. It will be better for you, and in the long run better for them.
A solution could be to use cheap version branching as provided by VCS such as Git or Mercurial. The project would consist in many feature branches used to develop said features and build branches where feature branches would be merged and adhoc fixing would take place. When a build branch is ready for test, it is tested, fixed if needed and then the build branch is shipped to production platform.
When features have been validated, the build branch can be merged into remaining feature branches so the branches under development can integrate the "official" changes.
To sum up, the application is custom built from existing feature branches as needed.
One reasonably sane way to manage this on the code level is to isolate each feature into a plugin. Then you can add/remove features on-demand by simply enabling or disabling corresponding plugins.
But this solution has certain costs:
Time to develop and test plugin engine for your app
You need to test every plugin configuration (set of enabled plugins and their versions) that is going to be deployed. Otherwise there's a risk that this specific set is not compatible and end users would be first to see resulting crash, or data loss, or some other horror
Additional time to wirte plugins the way that they're minimally dependent on each other.
It's usually worth it only if you have many clients with different needs. In your case, I'd recommend explaining cost of separately enabling features to your client to see if they really need it this hard. Most likely, they don't
EDIT: to clarify - I am asking for advice on both unit testing and user interface testing.
Currently, I don't use TDD. While I am developing an application I am constantly testing what I am working on. Testing iteration could be anything from minor function changes to entire models. I try not to code too much before I test. I like instant feedback.
Of course, with experience I can see potential problems or bugs occurring as I'm coding.
BUT, after an application is complete I will usually go through the entire app on the frontend and ensure all functions are working as expected. This means literally everything. Every add/edit/delete, sort, filter, even broken links and such.
This can take a lot of time sometimes but it does ensure my work hardly ever contains bugs after deployment.
However, I'm looking for a more standard solution. What do experienced developers do? Assume for a moment I am a single developer and so do not have a testing department etc
Do you hire beta testers (no good if app is sensitive to public use)?
Is it viable to build a series of 'general' unit tests which can for example test ALL sorting, filter functions. One for testing ALL add/edit/delete functions.
Love to hear your feedback. Will be changing the way I develop based on suggestions.
Of course as David said : Unit-testing for models and helpers, of course.
and I need to add Selenium
Selenium is a robust set of tools that supports rapid development of
test automation for web-based
applications. Selenium provides a rich
set of testing functions specifically
geared to the needs of testing of a
web application. These operations are
highly flexible, allowing many options
for locating UI elements and comparing
expected test results against actual
application behavior.
which is amazing :
watch the 2 min intro
http://seleniumhq.org/movies/intro.mov
How Selenium Works
Testing with PHPUnit and Selenium
Unit-testing for models and helpers, of course.
But you can do unit-testing on "page-requests", as well. See, for example:
Content with Style - Unit testing controllers with Zend Framework
That's a step towards integration testing. But for issues of layout and visual aesthetics, you're pretty much stuck with walking-through each request with your browser.
Usually what you code works is not that the problem, very nice if you also write tests for it. The problem is that you need to test the integration of your changes every time you make commit.
If you like instant feedback,as I do, probably you should have a look at Continuous Integration.
I've started using Hudson as CI server and I am not regretting it!
I'm looking for inputs into how I can manage the upgrade process itself of a homegrown php/mysql application. Meaning, if we have a 'stable' version of our php/mysql application working on our production server, and we now want to upgrade it to the next version that we've worked on - how do we go about doing that elegantly? What practices should I be implementing?
What I was planning to do was just to
Ask the developers to stop
checking in code after all stability
/ functionality tests are done
Take the application offline*** (Q: how should I prevent ppl for logging in / accessing public pages? Best practices for that?) but allow access to developers through a secret login page / url
Log onto the production server and check out the latest version
locally***
Have the developers/testers test their code through the secret access page / url***
After that is done, we restore access to all by removing this secret access page / url, removing the site-under-maintenance page and restoring access to all.
***NOTE: A simple way of doing this would be to rename /myapp/ to /myapp.old/ and put the new application version into /myapp.new/ Developers would access /myapp.new/, test to their satisfaction and then after we're done, we would rename this back to /myapp/ (this is just the basic idea)
This is a huge question, and in many ways it will depend on your specific project. But here are some practices to think about:
Put lots of comments in your code. Things that seem perfectly logical now will be confusing when you go back to make changes in a year or two.
Maintain a development version of the site with its own database. You can test changes to the site before publishing to your production site.
Use a PHP framework (such as CakePHP, CodeIgniter, etc). If you are far along on your project, this may be difficult to do. But it will help you write code in a way that is easy to update, and will include a lot of stable, mature functions that you won't have to write from scratch. Using one of these frameworks (and following its best practices) is probably the best way for a beginner to learn to think about writing modular code that's easy to update. This will also encourage you to develop your database in a way that is consistent with the structure of your site.
Write tests (the framework should help you with this) to programatically check your code for errors.
Use a version control system such as Subversion or Git. This allows you to track changes to the site, and easily roll back changes if/when you realize they are buggy.
Comprehensive unit test coverage would be very helpful, as would small, highly cohesive, low-coupled classes. In addition to the unit tests, good coverage from an integration level would be valuable.
I'm curious to see how other developers go about testing their web sites. PHP specifically in my case, but this probably spans multiple languages. I've been working on a site for over a year now, and I'd really like to automate a lot of the regression testing I do between versions.
This specific site is in CodeIgniter, so I have some tests for my models. I'd like to move beyond just testing those though. However, this is an issue even non-MVC developers have had to tackle I'm sure.
Edit: I think the functionality that would satisfy a lot of my test desires is the ability to assert that paramters have a specific value at the end of the script processing. In my case a lot of logic is in the controller, and that's the main area I'd like to test.
For actual unit testing without testing the UI, you should just test the functions in the model. Most of your functionality should be in there anyways.
You might want to have a look at Selenium for testing the UI of your site. It can record your actions and play them back, or you can edit the scripting directly.
(source: seleniumhq.org)
Have you tried Fitnesse ?
It helps on creating Acceptance tests. They are specially useful for websites, which doing this kind of tests are a pain.
There are a couple of videos from unclebob inside the webpage too. The good thing is that Fitnesse is not restricted for website testing, so your knowledge about using it can be used with other apps too.
The project I'm working on is a Desktop APP written in c++ that uses Fitnesse tests.
But if you meant unit testing the models (which I think you didn't), they can be create using the phpunit lib. I think the ZEND framework has a similar lib for that.
You might want to check out PHPUnit
http://www.phpunit.de/manual/current/en/
I have started using it on my PHP projects and it's very easy to work with and very powerful. In particular, learn and use mocks:
http://www.phpunit.de/manual/3.0/en/mock-objects.html
Mocking is especially important when unit testing applications that do database operations.
Take a look at TOAST. It's build specially for CodeIgniter. It uses CI infrastructure, so you can run all test tests via a browser and results are displayed back as a web page (HTML). It's very simple to use.
I suggest you test your Controllers as well. Testing model is ok, but model is just the DB storage. Controllers contain all the "business logic" and are the place where most things go wrong.
One of the best ideas I've heard of, as far as testing web apps go, was to create a script that would go over all the pages in the site and check them for differences from the previous scan, letting you accept changes and fix regressions.
Generally speaking, automatic testing of GUI applications (websites are GUI apps) is difficult and usually unnecessary. Unit tests work best with simple libraries.
I use Canoo WebTest. It is the best free web site unit test framework out there. It is entirely scriptable with XML and requires no browser so it can run from a build server.
We modified Waiter (Ruby). It plays back "scripts" of URLs and Form Filling to IE and we have added a script "command" to take a Screen Capture; the screen capture image is compared against a Known-Good-Image (i.e. a Master Image) and if that image is different it is logged (basically a Web page of such results is prepared) and "a human" does a review of the Master / Test image. Obviously there are two outcomes at that point - "The difference is intentional" or "There is an incorrect change". In the first instance the Master image is replaced with the New Image; in the second we go fix the bug, and the change will be included in the next test run