Till now I've been testing my web (usually written in PHP) as well as desktop applications (normally Java or C#) manually. Now I read somewhere on the net about automated tests. I tried searching to know about it in details but almost all searches end up at things like PHPUnit. Could someone please put some light on the theory behind automated tests? How a software can be tested automatically? Any limitations etc? Or may be you can tell me a place where I can read about this.
Regards
For testing the correctness of code you can use unit testing. This was first explained to me from Dive Into Python: Unit Testing and will do much more justice to the topic then I could here. Now that you know about the term unit testing, you shouldn't be far from other existing explanations that will make sense to you, be it that one doesn't.
You may find test-driven development of interest too.
Your code is not magically tested for you, as you may had the idea of. The code to test your application will be written by you. What packages like PHPUnit offer you is a framework in which you can implement your tests. These packages will provide many conveniences for defining your tests, running them together as a suite, and generating a report. These are the only automated aspects.
These Test Tools are used in the following manner as, If suppose we have to test a webform we will pas the input data into the fields they have provided us with the tool, and the number of users needs to filling the form (their user-name), In this way when this is executed then the form gets filled with provided data with individual usernames, This execution may provide u test data as performance, time of execution and load etc...
WebLoad Testing Tutorial
see the above link for load testing...
See these links as well
Functional testing
Security Testing
Link and HTML tool Testing
Performance Testing
Related
I'm trying to build a full test suite for Joomla and others. There are some docs around but they're quite limited and I'm wondering somebody did wrote already some scripts. By 'full' I do mean really a complete testing :
1. installing an extension on Windows, Max, Linux (Vagrant, VM)
2. configuring the extensions with the CMS option panels
3. doing things in a full Ajax application
I guess I'll end up with lots of bash-scripts, triggering other scripts within a virtual setup, right ?
I must admit, I am not really familiar with all this testing frameworks and products and I'd be already very happy to get pointed to anything. Doing pure unit tests doesn't seem enough given the nature of such systems (namespace collision, interfering plugins,...)
Thank you for any hints
I am not familiar with wordpress and joomla but those are just PHP code, so using PHPUnit can be suitable. With PHPUnit you are not only doing unit testing but you can also do other kinds of tests (it depends on how much time you are willing to spend on testing but I would say that it can cover pretty much any aspects)
As for front-end testing, there are several choices
Using selenium to write tests
Using one of the many headless testing frameworks out there (for instance, casperjs in Javascript, Watir for Ruby etc...)
For setting up virtual environment, apart from vagrant you can check docker
There are many programs you can use to do the normal interaction tests; if you want to handle logins, some scripting will be necessary to grab the token.
But for a full integration test including ajax I guess you'd be better off with a headless browser, take a look at this Real headless browser where the same requirement is discussed in detail
We have a project developed by 2 years with poorly designed architecture. Now a days there are no any unit tests at all.
Current version of system works satisfyingly but we vitally need refactoring of core modules.
The budget is also limited so we can not hire suffisient number of developers to write unit tests.
Is it the possible strategy to generate code automatically for unit tests which covers, for example, interaction with data, in assumpion that now system works fine and current system's output can be converted in XML-fixtures for unit testing?
This approach gives us a possibility to quickly start refactoring of existing code and receieve immediate feedback if some core functionality is corrupted because of changes.
I would be wary of any tools that claim to be able to automatically determine and encode an arbitrary application's requirements into nice unit tests.
Instead, I would spend a little time setting up at least some high-level functional tests. These might be in code, using the full stack to load a predefined set of inputs and checking against known results, for instance. Or perhaps even higher-level with an automation tool like Selenium or FitNesse (depending on what type of app you're building). Focus on testing the most important pieces of your system first, since time is always limited.
Moving forward, I'd recommend getting a copy of Michael Feathers' Working Effectively with Legacy Code, which deals with exactly the problem you face: needing to make updates to a large, untested codebase while making sure you don't break existing functionality in the process.
I've been reading this site here and there and appears as though you guys have a wonderful community.
As for my background, I am a sophomore at university familiar with SQL, C++, Visual Basic, and some PHP. One of my school projects for the summer term involves building a web application that allows users to log in and schedule specific timeslots over the internet. Typically, I have been the only person working on a project, but in this case I will be part of a group. Since we're all relatively new to working as a team, I would like to set up source control for my group so we're not all working off a shared drive somewhere. Additionally, I would like to make sure that all of us are able to test our changes in some sort of development server that hosts an instance of our website.
My actual question is in regards to the toolset that we should use to achieve this. As a group, we are most familiar with PHP and MySQL so we'll end up using that for the code and database. I have used SVN in the past for my own personal use, but my group members aren't very familiar with source control. We'll probably stick with something simple like Excel for the project management and bug tracking side of things. Ideally, we would like the tools to be free and open source.
How as a group should we manage the construction of the actual application? Are there methods out there that I can use that will allow any one of us to move the files to our development machine and keep track of who did it so we don't end up overwriting each other's changes? If this is not possible, one of us will write some scripts to handle it - but I would like to avoid building basically a separate software application that will only be used to manage our project. Another issue I foresee will be updating the database running on the development machine. Are there any standardised methods that we can use to manage our SQL scripts among the four of us?
I do not expect a really long winded answer here (after all, this is our project!), but any helpful tips would be greatly appreciated. Once I return from holiday I am looking forward to getting started! Thanks!
I recommend your group use source control to synchronize your code. You can either setup your own server or just use a free provider such as github, Google code, or bitbucket.
If you do decide to use one of these sites, a nice feature is that they provide free issue tracking as well, so you can use that instead of Excel.
The best way to manage the SQL scripts is to break them out into separate files and place them under source control as well. You can either create .sql files, or use a tool to manage these changes - for example, have a look at Ruby on Rails' Migrations. This may take some effort to setup, but you'll thank yourself later if you are working on a project of any size...
Draw up a plan for how you would do it if it were just you.
Split the plan up into tasks that take around 3-4 hours to complete. Make sure each task has a measurable objective.
Divy out the tasks. Try to sort them if possible to maximize developer efficiency.
Teach them to use source control. Explain to them that they will use this (maybe not svn, but SOMETHING) in a few years, so they might as well learn how now. Additionally, this will help in every group project they do down the road.
Make a script for building and running your tests. Also script your deployment. This will ensure you have the same mechanism going to live as you do going to test, which increases the number of defects found in testing. (This is as opposed to letting them exist but not found in testing.)
You mentioned updating the development database. It would be entirely reasonable to dump the development database often with a refresh from live. You may want to make 3 environments. Development, staging, and production. The development database would contain fabricated test data. The staging database would have a copy of live (recent to within a few days maybe.) And of course live is live.
Excel works fine as a "bug database." Consider putting it in source control that you manipulate and commit. This will give you a good idea of what happened over time, and you can correct mistakes quicker.
As far as source/version control, I would recommend subversion. There are some GUI tools they might use, or even webDAV to access the SVN. This will allow users to edit files collaboratively and also give you details as to who edited what, when, and why... SVN will also do a pretty good job at merging files that happen to be saved at the same time.
It's not the easiest concept to wrap your head around, but its not very complicated once you get running.
I suggest having everyone read the first chapter from: http://svnbook.red-bean.com/en/1.5/
and they should have a good idea of what's happening.
I am also curious to see what people have to say about the database
How as a group should we manage the construction of the actual application? Are there methods out there that I can use that will allow any one of us to move the files to our development machine and keep track of who did it so we don't end up overwriting each other's changes?
It sounds like you're looking for build management. In the case of PHP, a true "build" is as simple as a collection of source files because the language is interpreted; there is no compilation.
It just so happens that I am one of the developers for BuildMaster, a tool which basically solves every problem you have listed in your question... and it also sounds like it would be free in your case under the Community Edition license. I'll try to address some of your individual pain points and how BuildMaster could be used as a solution.
Source Control
As suggested by others, you must use it. The trick when it comes to deployment is to set up some form of continuous integration so that every time someone checks in, a new "build" is created. In BuildMaster, you can set this up for any source control provider you want.
Issue/Bug Tracking
Excel will work, but it's not an optimal solution. There are plenty of free issue tracking tools you can use to manage your bugs and features. With BuildMaster, you can link your bugs and features list with the application by their release number so you could view them within the tool at any time. It can also modify issue statuses and add descriptions automatically if you want.
Deployments
Using BuildMaster, you can create automated deployment plans for your development environment, e.g.:
Get Latest Source Code
Create Artifact
Copy Files To Development Machine
Deploy Configuration Files
Update Database
The best part is, once you set these up for other environments (glowcoder's point #6), pushing all of your code and database updates is as simple as clicking a button.
Another issue I foresee will be updating the database running on the development machine. Are there any standardised methods that we can use to manage our SQL scripts among the four of us?
Database Updates
Not surprisingly, BuildMaster handles these as well by using the change scripts module. When a member of your team creates a script (e.g. ALTER TABLE ADD [Blah] INT NOT NULL) he can upload it into BuildMaster, then run it on any environment you have created.
The best part is that you can add a step in your automated deployment and never worry about it again. As Justin mentions, you can use .sql files for your object code (stored procedures, views, triggers, etc.) and have those executed on every build since they are essentially code anyway. You can keep those in source control.
Configuration Files
One aspect of all this you may have neglected (but will inevitably run into) is dealing with configuration files. With PHP, you may have an .htaccess file, a php.ini file, a prepend.php, or roll your own custom config file. Since by definition configuration files need to change between your personal machine and the development machine, grabbing them from source control wouldn't necessary work without some bit of hacking a la:
if (DEV) {
// do one thing
}
else if (PROD) {
// do another
}
With BuildMaster, you can templatize your configuration files and associate them with an environment so they can be deployed automatically. It will also maintain a history of changes for you.
Automated Testing
If you want the full ALM effect, you can automatically unit test your code during an automated build, and notify you if anything fails so you know as soon as possible that something is broken.
Apologies for the "long winded" response, but I feel like you're already ahead of the game by observing the problems you might run into in the future and really believe BuildMaster will make all of this deployment stuff simple for your team so you can focus on the fun part, coding!
For web development ,
Can u send some sample test case xls report ,
Or is there any good open source plz refer ,
Is there any free automated testing for php web development ,
Thanks
Automated testing tools for PHP:
phpunit for unit tests.
Selenium is a browser-based automated testing tool which can work with any language, but includes the ability to run with phpunit.
Nobody can give you test cases for your web site since every project/application/website is unique and deserves their own unique test cases. There is no "right" or "wrong" in testing. Testing is an art and what you should choose and how well it works out for you depends a lot from project to project and your experience.
But as a professional Test Expert my suggestion is that you have a healthy mix of automated and manual testing.
(Examples below are in PHP but you can easily find the correct examples for what ever langauge/framework you are using)
AUTOMATED TESTING
Unit Testing
Use PHPUnit to test your classes, functions and interaction between them.
http://phpunit.sourceforge.net/
Automated Functional Testing
If it's possible you should automate a lot of the functional testing. Some frame works have functional testing built into them. Otherwise you have to use a tool for it. If you are developing web sites/applications you might want to look at Selenium.
http://www.webinade.com/web-development/functional-testing-in-php-using-selenium-ide
Continuous Integration
Use CI to make sure all your automated tests run every time someone in your team makes a commit to the project.
http://martinfowler.com/articles/continuousIntegration.html
MANUAL TESTING
As much as I love automated testing it is, IMHO, not a substitute for manual testing. The main reason being that an automated can only do what it is told and only verify what it has been informed to view as pass/fail. A human can use it's intelligence to find faults and raise questions that appear while testing something else.
Exploratory Testing
ET is a very low cost and effective way to find defects in a project. It take advantage of the intelligence of a human being and a teaches the testers/developers more about the project than any other testing technique i know of. Doing an ET session aimed at every feature deployed in the test environment is not only an effective way to find problems fast, but also a good way to learn and fun!
http://www.satisfice.com/articles/et-article.pdf
This answer is (almost) identical to one that I gave to another question. Check out that question since it had some other good answers that might help you.
How can we decide which testing method can be used?
My workplace consists of a lot of cowboy coders. Many of them are junior. Which coincidentally contributes to a lot of code quality issues.
I'm looking for suggestions on how to best wane my team into using TDD (we can start with Unit tests, and move into regression tests, and later more automated testing).
Ultimately, I want us to learn more quickly from our mistakes, and produce better code and breed better developers.
I'm hoping there are some practical suggestions for how to introduce TDD to a team. Specifically, what tools are best to choose in the LAMP (php) stack.
Sorry if this question is too open-ended.
After going through this process four times now, I've found that any introduction of TDD will fail without some level of enforcement. Programmers do not want to switch style and will not write their first unit test and suddenly see the light.
You can enforce on a management level, but this is time-consuming for all involved. Some level of this is necessary at the beginning anyway, but ultimately you will need automatic enforcement. The answer to this is to introduce Continuous Integration.
I've found that a CI Server is the ultimate keystone to any TDD environment. Unless developers know that something bad will happen if they don't write the tests, you'll always have cowboys who feel it's beneath them.
Make writing tests easy and the results visible.
Use a TestFramework with good documentation. like SimpleTest
If test depend on database contents, create a reference database that will be dropped and created at the beginning of a script.
Make a script that runs all test and shows the results on a standalone monitor or something that will make the test visible / easily accessable. (Running a command prompt is not an option)
I personally don't write test for every chunk of code in the application.
Focus on the domain objects in the application. In my case these are "price-calculation" and "inventory-changes"
Remind them that they are probably already writing tests, but that they delete their work just after creation.
Example: During the development of a function you'll have a page/testscript in with an echo or var_dump() the result. After a manual validation of the result you'll modify the parameters of the function and check again.
With some extra effort these tests could be automated in a UnitTest. And which programmer doesn't like to automate stuff?
As for the team question as well as universal ideas about software development and testing, I would suggest Joel Spolski's website and books: http://joelonsoftware.com/ I got many insights from him.
SimpleTest - excellent documentation and explanations of testing for php
Another way to start TDD is try to use PHP framework. Without framework, it's hard to implement unit test effectively.