There are quite a few testing techniques and ways to implement test cases for an automated test system. A few that should be implemented for WordPress are listed, but there are others and with better details on other web sites. I’m not the authority on unit testing and I don’t pretend to be. However, what I do know is detailed below and take my word with a grain of salt. I will say that the details are in general what can be applied when coding test cases.
Unit testing is quite what it says, you are testing a single unit or component of the library. This can be function or method. This method usually makes use of mock objects for classes to correctly test the behavior of an individual unit.
This method has the advantage of allowing for testing code coverage and finding out just how accurate your tests are.
I would argue that this is the easiest of the methods, since all you are doing is checking the current behavior and writing tests to make sure that the current behavior continues in later versions.
You’re not actually getting down into the source code and making sure that all of the branches are covered and that the correct results are covered. You are also not always checking that incorrect data has a fallback in place to prevent failures.
Functional Testing is a step up from unit testing, where you are testing a component within the software environment. This is more how the Automattic Automated Testing suite is set up. It does not allow you to test the code coverage, because since the entire software environment is running it will skew the code coverage results.
The primary difference between unit testing and functional testing is that with unit testing, every test in unit testing is self contained and does not interfere with the other tests. With functional tests, you have to be careful, because the tests are not contained and will affect other areas. Other than that, it would be easy to switch from unit testing to functional testing and back again.
Users do this already, but this method is more about automating the process of what users do. It doesn’t completely replace user testing, but it does complement it. It reduces the tedious tasks of inserting content and does that and returns the results. For known problems, it is very useful in order to prevent presentational regressions.
It is also quite frankly, in my humble opinion, one of the most difficult to accomplish for browser based acceptance testing. Selenium is very helpful, but setting that up has been difficult for me. If you can get it set up then PHPUnit can talk to it and it would be as easy as writing any unit or functional test.
WordPress is a community effort and it takes everyone in the community to make WordPress the best it needs to be. Getting involved is as easy as using WordPress and reporting when defects show up. It can get up to creating patches for WordPress Trac defects or enhancements. Everyone can do something and people are doing great things for WordPress every day.
Testing WordPress Trunk
If you use just the versions of WordPress that are released, some bugs that you find could have already been fixed on trunk and just awaiting the next release. If you want to really test WordPress, you must first check out the Trunk and join the WP-Testers mailing list.
When testing make sure you input both invalid and valid information and keep in mind what you are expecting what the actual result is. You are trying to break WordPress! If something doesn’t break, then you aren’t trying hard enough or WordPress already has checks which prevent it from breaking.
Reporting bugs is one of the easiest ways to get involved with quality assurance. If a defect is not reported then the WordPress developers and community will not know of the defect. The sooner a defect is reported and confirmed the sooner it can be fixed.
You can report bugs at the WordPress Trac Site. You can also find more information on reporting bugs at the WordPress Codex: Reporting Bugs. As an reminder, you may need to stay around for the life of the ticket to make sure to answer any questions and to follow up on anything that developers or others may need to satisfy the requirements of the report.
Remember: make sure to check the defect or enhancement hasn’t already been fixed or reported. Doing so, will be appreciated by those who have to mark your ticket a duplicate.
The next step from reporting defects or offering suggestions for enhancements is to provide solutions to existing defects and enhancements. As for most people, you may choose whatever ticket of your ability and liking. You don’t have to start at the top and work your way down.
It should be noted that the tickets of low priority probably won’t be as committed as quickly as those tickets that have a higher priority. Give enough time for the core development team to look over your ticket and see if it worth committing. Keep in touch after you commit the patch to make sure that answer any questions those in the community and core development team may have.
If a reasonable amount of time has past without any resolution with an existing patch, writing another comment asking for a response or set one of the keywords to include ‘dev-feedback’ (without the quotes). It might be that the development team might have forgotten about your ticket. Leaving a friendly reminder might push the ticket ahead in their minds.
When your patch does get committed, rejoice, it will be one less ticket in the queue. Any amount of help is appreciated.
Writing Codex Documentation
The codex is the central place for finding information and help on WordPress. As many features and solutions that WordPress offers, the amount of help is far to broad to be completely covered by any one page or person. If you notice that there is a section that is missing, then sign up and add it.
Writing Inline Documentation
WordPress also needs to have inline or source documentation for those who browse the source for their information. Not everything can be learned from the codex and since the codex can grow outdated, it would be useful to look at the source from time to time. Those that do look at the source need additional information in order to quickly realize what the code is supposed to be doing.
It is also useful for those who can’t read code as clearly as some of those who have been reading and writing code for a long time. In order words, it helps novices of PHP to learn what WordPress is doing. They might not be able to build an application like WordPress, but they can write plugins that are useful for them and/or other people.
There is currently an Inline Documentation Standard that should be used. It is unwise to write a lot for functions since there is a tiny amount of overhead in processing the comments. Not much, but leaving as much detail to only describe what the function does should be enough. Any examples can be located at the codex.
Writing Test Cases
There is currently an Automated Testing Suite for WordPress which you can provide test cases for. Test cases are useful to prove or disprove a defect and whether a patch fixes the defect. Without test cases it would be difficult without manual testing to find out if the defect was fixed. The test cases also can test whether the defect shows up in the future (called a regression).
Writing test cases will be detailed in a later post. Make sure to stay around to learn more!