Quality Assurance is ensuring the quality of a product. It is an ongoing process that never ends, or at least not during the life of the product. Quality Assurance in open source is a more open process and anyone can help.
It involves testing, documentation, and automated testing. Testing is the most important factor, but documentation is equally important. Actually, Software Quality Assurance is a paradigm that involves a great deal more, but for the purpose of this discussion, I’m going to limit this to just the above three.
What it means to the User
When you commit to Software Quality Assurance, you are telling the customer and user that you have a policy in place that will hopefully prevent a majority of preventable bugs from occurring before the product is released. In the likely event that a bug is found, there is a policy in place that ensures that it will be fixed and prevented (in most cases) from occurring again in the future.
It also says that there is detailed documentation for both the users and developers which extensively covers the topics that both groups will want to know. Extensively covers the known areas and areas that have been questioned about in previous versions. Covers the more difficult areas better and with more details than “simpler” topics that users and developers should not have any problems with.
Users, testers, and developers all help with the process. The user can make a request for a new feature that will improve the product or report a problem in the product. The testers will usually beta test and test versions of the product before it is released reporting any issues that come up. Developers will take the reports and create patches that fix the issues for the next version.
Developers will also write test cases that check whether the problem is fixed and if the problem reoccurs in a later version. If the problem does reoccur then the problem is immediately known and can be rechecked until the regression is corrected. If the regression was intentional, then documentation would have to be updated and other developers updated on the change.
Writing documentation should be concise, tutorial type explanations in language that an user and/or a developer will understand. For open source, I would just recommend writing something as close as possible to this style as you can get and cut the extraneous details (ex, I don’t want to know about how you used it in your project and the history, unless it further explains the purpose of the function, sometimes it is useful, other times it is not).
Not everyone is a professional writer (myself included), but there are enough people who are that if you give them enough information, they can edit it to where it is better. I write with the assumption that someone is going to look at what I wrote and point out mistakes and hopefully, instead of cussing me, will actually move to fix the mistakes. On Wikis, this is very simple process, on this post however, they would have to send it to be me personally or export it to their blog or wiki.
You are testing that the software works (well, yes you are for those who are disagreeable), you are trying to test whether the software breaks. Therefore it makes more sense to input invalid data to see if what is expected happens or not. Developers (indeterminate amount, I have this problem, so I only speak of myself and the general stereotype of developers with testing their software) have a habit of only using valid data.
You only know that the software works if you enter something invalid and it didn’t break on that input. If it breaks on invalid input, then there should be a check in place that either automatically the problem or informs the user of their mistake. There is also automated testing, which automates the lower level testing and higher level testing which automates some of what users do.
WordPress is a community effort and it takes everyone in the community to make WordPress the best it needs to be. Getting involved is as easy as using WordPress and reporting when defects show up. It can get up to creating patches for WordPress Trac defects or enhancements. Everyone can do something and people are doing great things for WordPress every day.
Testing WordPress Trunk
If you use just the versions of WordPress that are released, some bugs that you find could have already been fixed on trunk and just awaiting the next release. If you want to really test WordPress, you must first check out the Trunk and join the WP-Testers mailing list.
When testing make sure you input both invalid and valid information and keep in mind what you are expecting what the actual result is. You are trying to break WordPress! If something doesn’t break, then you aren’t trying hard enough or WordPress already has checks which prevent it from breaking.
Reporting bugs is one of the easiest ways to get involved with quality assurance. If a defect is not reported then the WordPress developers and community will not know of the defect. The sooner a defect is reported and confirmed the sooner it can be fixed.
You can report bugs at the WordPress Trac Site. You can also find more information on reporting bugs at the WordPress Codex: Reporting Bugs. As an reminder, you may need to stay around for the life of the ticket to make sure to answer any questions and to follow up on anything that developers or others may need to satisfy the requirements of the report.
Remember: make sure to check the defect or enhancement hasn’t already been fixed or reported. Doing so, will be appreciated by those who have to mark your ticket a duplicate.
The next step from reporting defects or offering suggestions for enhancements is to provide solutions to existing defects and enhancements. As for most people, you may choose whatever ticket of your ability and liking. You don’t have to start at the top and work your way down.
It should be noted that the tickets of low priority probably won’t be as committed as quickly as those tickets that have a higher priority. Give enough time for the core development team to look over your ticket and see if it worth committing. Keep in touch after you commit the patch to make sure that answer any questions those in the community and core development team may have.
If a reasonable amount of time has past without any resolution with an existing patch, writing another comment asking for a response or set one of the keywords to include ‘dev-feedback’ (without the quotes). It might be that the development team might have forgotten about your ticket. Leaving a friendly reminder might push the ticket ahead in their minds.
When your patch does get committed, rejoice, it will be one less ticket in the queue. Any amount of help is appreciated.
Writing Codex Documentation
The codex is the central place for finding information and help on WordPress. As many features and solutions that WordPress offers, the amount of help is far to broad to be completely covered by any one page or person. If you notice that there is a section that is missing, then sign up and add it.
Writing Inline Documentation
WordPress also needs to have inline or source documentation for those who browse the source for their information. Not everything can be learned from the codex and since the codex can grow outdated, it would be useful to look at the source from time to time. Those that do look at the source need additional information in order to quickly realize what the code is supposed to be doing.
It is also useful for those who can’t read code as clearly as some of those who have been reading and writing code for a long time. In order words, it helps novices of PHP to learn what WordPress is doing. They might not be able to build an application like WordPress, but they can write plugins that are useful for them and/or other people.
There is currently an Inline Documentation Standard that should be used. It is unwise to write a lot for functions since there is a tiny amount of overhead in processing the comments. Not much, but leaving as much detail to only describe what the function does should be enough. Any examples can be located at the codex.
Writing Test Cases
There is currently an Automated Testing Suite for WordPress which you can provide test cases for. Test cases are useful to prove or disprove a defect and whether a patch fixes the defect. Without test cases it would be difficult without manual testing to find out if the defect was fixed. The test cases also can test whether the defect shows up in the future (called a regression).
Writing test cases will be detailed in a later post. Make sure to stay around to learn more!