There are quite a few testing techniques and ways to implement test cases for an automated test system. A few that should be implemented for WordPress are listed, but there are others and with better details on other web sites. I’m not the authority on unit testing and I don’t pretend to be. However, what I do know is detailed below and take my word with a grain of salt. I will say that the details are in general what can be applied when coding test cases.
Unit testing is quite what it says, you are testing a single unit or component of the library. This can be function or method. This method usually makes use of mock objects for classes to correctly test the behavior of an individual unit.
This method has the advantage of allowing for testing code coverage and finding out just how accurate your tests are.
I would argue that this is the easiest of the methods, since all you are doing is checking the current behavior and writing tests to make sure that the current behavior continues in later versions.
You’re not actually getting down into the source code and making sure that all of the branches are covered and that the correct results are covered. You are also not always checking that incorrect data has a fallback in place to prevent failures.
Functional Testing is a step up from unit testing, where you are testing a component within the software environment. This is more how the Automattic Automated Testing suite is set up. It does not allow you to test the code coverage, because since the entire software environment is running it will skew the code coverage results.
The primary difference between unit testing and functional testing is that with unit testing, every test in unit testing is self contained and does not interfere with the other tests. With functional tests, you have to be careful, because the tests are not contained and will affect other areas. Other than that, it would be easy to switch from unit testing to functional testing and back again.
Users do this already, but this method is more about automating the process of what users do. It doesn’t completely replace user testing, but it does complement it. It reduces the tedious tasks of inserting content and does that and returns the results. For known problems, it is very useful in order to prevent presentational regressions.
It is also quite frankly, in my humble opinion, one of the most difficult to accomplish for browser based acceptance testing. Selenium is very helpful, but setting that up has been difficult for me. If you can get it set up then PHPUnit can talk to it and it would be as easy as writing any unit or functional test.