Extend participation of the user base in the GeoServer evolution by setting up an official testing team.

Andrea Aime

Hopefully, but not necessarily, in time for GeoServer 2.0.

Choose one of: Under Discussion, In Progress, Completed, Rejected, Deferred

Increasing the GeoServer quality is one of the GeoServer community ongoing concerns. At the moment the Quality Assurance (QA) work is formalized only against the developers work in terms of unit testing and pre-release testing against the official OGC compliance tests. The user base contributes a number of reports too, but they get to the developer when the release is already out. The proposal tries to formalize the existence of a testing team and a pre-release testing period in which users can check and contribute bug reports so that regressions and critical issues can be fixed before reaching a wider user base.

This proposal recognizes a wide variety of testers working in diverse environments and with specific request and load patterns are necessary to keep up the good quality of a software product and thus opens to the user base providing an official testing role for anybody who is interested in improving the GeoServer release quality.

The key elements of the proposal are the testers, the release procedure changes needed to allow their timely contribution, and how to handle the resulting bug reports.

The testing team is open ended, everybody can participate. Nobody is asked to execute specific manual testing scripts, on the contrary, diversity in testing is encouraged and welcomed.

However, an official testing team will be setup that contains well known members. Those people represent the core testers, the ones that are the most engaged in the activity and that as such will be recognized and will drive the evolution of the testing along with the developers. The initial testing team is formed by:

• Jody Garnett (uDig checks)
• Aleda Freeman (SDE and general stability checks)
• Francesco Izzi (general checks)
• Andrea Aime (performance checks)

One week before the release date a freeze is called on the GeoServer repository and no changes are allowed anymore. This week is the testing window, the only commits allowed in the window are related to regressions or critical issues found by the team (developers and community found regressions are of course accepted as well, we’re simply supposing the testing team will discover the most of the issues)

During the window of development bugs are normally fixed. Those should be marked as “resolved” and have the user that reported them verify they are actually gone. If there is no such a verification this could be a good target for the testing team to check.

All regressions and critical issues found during the window will be fixed before the release:

• a regression is a bug that reduces the functionality of GeoServer compared to a previous release.
• a critical bug is some bug that affects a large amount of users and will make GeoServer not usable in production for them. Marking a bug as critical will be the result of a discussion involving the developer team and the official testing team (which thus gets a stronger position than a random tester as a premium for the extra involvement).

Other issues will be scheduled as normal

This section should contain feedback provided by PSC members who may have a problem with the proposal.

There are no backwards compatibility issues

Andrea Aime: +1 Alessio Fabiani: +1 Chris Holmes Jody Garnett: +1 Rob Atkinson: +1 Simone Giannecchini:

Email Discussion: * [http://www.nabble.com/GeoServer-QA:-asking-users-to-help-out--td25136350.html]