On Thu, 2009-04-02 at 11:45 -0700, Adam Williamson wrote:
On Thu, 2009-04-02 at 11:11 -0700, Adam Williamson wrote:
c) make the test cases as automated as possible. Offer sending
smolt
profile (smoltGUI) and run test cases directly from the desktop.
Also We
should add a link (or the document itself) about how to test what
and
why to the desktop. If it's possible for the test generated
results
should also be posted automagically to the results page, or, if
tests
fail, bug reports could be created by bugbuddy.
These would all be nice, but we *are* running on a one-week cycle
here,
it may be hard to get it all done. Assistance welcome. =) it's
pretty
hard to automate some elements of some test cases, but some could certainly be done with scripts etc.
Oh, I forgot to mention another consideration here: I'm trying to write these test cases to be pretty future-proof, so we could use them virtually unmodified for F12, F13, F14...after all, the URLs don't have expire-by dates and the test cases will still be hanging around in 10 years most likely. I suspect scripts are liable to go stale faster than instructions.
My initial concern was the lack of defined test cases. Thankfully that problem is slowly going away (see https://fedoraproject.org/wiki/Category:Test_Cases). I'm always in favor of automation where possible, but I try to make sure we know what we automate before we dive into the shell.
However, with the test case "library" expanding ... it isn't a bad idea to begin thinking as scripts/automation comes online, where/how will we store these tests. Depending on the test target, automation may already exist. So leveraging any off-the-shelf test automation is always a "good thing." Some of these questions will naturally tie into the beaker project as that matures, but there's no reason we can't discuss/debate these issues now.
Thanks, James