----- "Jesse Keating" jkeating@redhat.com wrote:
Perhaps I wasn't clear. It seemed to me that when you were to test pending update A, you would setup your test environment such that it had a install of say F13, plus all the current stable updates of F13, then potential update A installed.
What I'm saying, and what is more true to a user's experience, is that you need to install F13, plus all the current updates, plus all the pending updates, including A. When we push A, we won't be pushing it alone, we'll be pushing it with all the other pending updates at the same time, so that's the environment you need to test in.
Ah, I understand you now. Yes, that seems like a better approach. Of course this requires some time window for accepting pending updates and running test cases only after closing that window. So if there is a set S of pending updates (A, B, C, ...), we will provide test cases with the whole set S. And when test cases claim that pending update A is valid, we must be sure set S doesn't change afterwards. (As was discussed on some past conference call.)
If this is the decided solution, it makes sense that we reflect that in the test plan. I have updated the definition [1]. Does that concur now with your suggestion?
[1] https://fedoraproject.org/wiki/User:Kparal/Proposal:Package_update_acceptanc...
On Mon, 2010-04-26 at 07:12 -0400, Kamil Paral wrote:
----- "Jesse Keating" jkeating@redhat.com wrote:
Perhaps I wasn't clear. It seemed to me that when you were to test pending update A, you would setup your test environment such that it had a install of say F13, plus all the current stable updates of F13, then potential update A installed.
What I'm saying, and what is more true to a user's experience, is that you need to install F13, plus all the current updates, plus all the pending updates, including A. When we push A, we won't be pushing it alone, we'll be pushing it with all the other pending updates at the same time, so that's the environment you need to test in.
Ah, I understand you now. Yes, that seems like a better approach. Of course this requires some time window for accepting pending updates and running test cases only after closing that window. So if there is a set S of pending updates (A, B, C, ...), we will provide test cases with the whole set S. And when test cases claim that pending update A is valid, we must be sure set S doesn't change afterwards. (As was discussed on some past conference call.)
If this is the decided solution, it makes sense that we reflect that in the test plan. I have updated the definition [1]. Does that concur now with your suggestion?
[1] https://fedoraproject.org/wiki/User:Kparal/Proposal:Package_update_acceptanc...
Nice. I like the updated wording ... "the testing should consider that set as a whole, not just individual packages in it."
This also reflects the different repositories that wwoods includes during depcheck verification, so this seems to validate both ways.
Thanks, James