We are going to be having another proventesters meetup tomorrow on IRC in #fedora-meeting at 18:00UTC.
Purpose of meetup: Brainstorm ideas on improving testing and processes for testing updates.
* Intro/gather more agenda items
* Recruiting more proventesters/testers.
* One stop page for updates testing resources
* Your amazing agenda item here.
Please do join us if you are a proventester, want to become one, or have ideas for improving the testing of updates. Note that this is not the venue for changing the updates policy, see FESCo for that, this is an attempt to get things working better within our existing updates policy.
Hope to see folks there!
kevin
_______________________________________________ test-announce mailing list test-announce@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/test-announce
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Kevin,
would it be possible to move the meeting/next meetings from 18:00 UTC to somewhat later? I'd like to participate, but real life tends to make it really difficult for me at 18:00 UTC. I have to wait, until the kids are in bed, sleeping.
Would another mailing list, e.g. proventester-list make sense?
Cheers, Matthias
On 04/10/11 21:09, Kevin Fenzi wrote:
We are going to be having another proventesters meetup tomorrow on IRC in #fedora-meeting at 18:00UTC.
Purpose of meetup: Brainstorm ideas on improving testing and processes for testing updates.
Intro/gather more agenda items
Recruiting more proventesters/testers.
One stop page for updates testing resources
Your amazing agenda item here.
Please do join us if you are a proventester, want to become one, or have ideas for improving the testing of updates. Note that this is not the venue for changing the updates policy, see FESCo for that, this is an attempt to get things working better within our existing updates policy.
Hope to see folks there!
kevin
_______________________________________________ test-announce mailing list test-announce@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/test-announce
- -- Matthias Runge mrunge@matthias-runge.de mrunge@fedoraproject.org
On Wed, 2011-10-05 at 09:06 +0200, Matthias Runge wrote:
Would another mailing list, e.g. proventester-list make sense?
I'm not sure that would really make sense, as much of the practical discussion of proven testing is entirely shared with people testing the same packages who don't happen to be proven testers. I'm not sure there's enough proventesters administrivia to justify a separate list. It can easily enough be discussed here, it's on-topic.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 05 Oct 2011 09:06:45 +0200 Matthias Runge mrunge@matthias-runge.de wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Kevin,
would it be possible to move the meeting/next meetings from 18:00 UTC to somewhat later? I'd like to participate, but real life tends to make it really difficult for me at 18:00 UTC. I have to wait, until the kids are in bed, sleeping.
well, we were hitting 19UTC, but then everyone wanted to move earlier. ;( I'll ask today if a later slot would work...
Would another mailing list, e.g. proventester-list make sense?
Not sure it would. It would be kinda of dividing our resources some...
kevin
=========================================== #fedora-meeting: proventesters (2011-10-05) ===========================================
Meeting started by nirik at 18:00:01 UTC. The full logs are available at http://meetbot.fedoraproject.org/fedora-meeting/2011-10-05/proventesters.201...
Meeting summary --------------- * Intro/Gather more agenda (nirik, 18:00:15) * LINK: http://fedoraproject.org/wiki/Proven_tester#Joining_the_proven_testers (nirik, 18:01:53)
* Recruitment (nirik, 18:05:35) * LINK: https://picasaweb.google.com/103451550643082159750/B#5576235128261512338 <--- I doodled up a t-shirt idea a while back (maxamillion, 18:08:01) * IDEA: short term here: mine existing karma for non proventesters and ask them to join, more press (blog posts, point people on test list), and look at f-e-k improvements (nirik, 18:24:17) * IDEA: longer term: gui for f-e-k and having pk or something like it offer updates-testing (nirik, 18:24:26) * IDEA: longer term: tie into abrt (nirik, 18:24:33)
* One stop page for updates testing resources (nirik, 18:25:26) * will try and create a package updates tester page with links to resources. (nirik, 18:29:44)
* Pending updates (nirik, 18:31:39) * 11 critpath updates pending in f14 (nirik, 18:35:41) * 19 critpath updates pending in f15 (nirik, 18:35:49)
* Open Floor (nirik, 18:35:56)
Meeting ended at 19:00:41 UTC.
Action Items ------------
Action Items, by person ----------------------- * **UNASSIGNED** * (none)
People Present (lines said) --------------------------- * nirik (95) * tflink (30) * maxamillion (18) * mike-c (12) * Cerlyn (8) * abadger1999 (7) * Southern_Gentlem (4) * satellit_ (4) * zodbot (3) * jwb (2) -- 18:00:01 <nirik> #startmeeting proventesters (2011-10-05) 18:00:01 <zodbot> Meeting started Wed Oct 5 18:00:01 2011 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:01 <nirik> #meetingname proventesters 18:00:01 <zodbot> The meeting name has been set to 'proventesters' 18:00:15 <nirik> #topic Intro/Gather more agenda 18:00:23 <nirik> any folks around for a proventesters meetup? 18:00:31 * satellit_ listening 18:00:38 <jwb> decided to show up to figure out how to become one 18:01:25 <nirik> jwb: cool. ;) it's easy. 18:01:35 * tflink is around 18:01:53 <nirik> http://fedoraproject.org/wiki/Proven_tester#Joining_the_proven_testers 18:02:16 <nirik> basically apply, read the docs as to what you are supposed to do, get approved 18:02:28 * maxamillion is here-ish 18:02:57 <jwb> nirik, thx 18:03:30 <nirik> I had 2 items on my plan... recruitment ideas, and one stop shopping page for resources... did any other folks have items? 18:03:44 <tflink> nothing here 18:04:12 <maxamillion> none here 18:04:15 * nirik will wait a min more for folks. 18:04:19 <maxamillion> rgr 18:05:26 <nirik> ok, lets go ahead then... 18:05:35 <nirik> #topic Recruitment 18:05:57 <nirik> so, any ideas on how we can get more proventesters involved? and/or more testers for updates in general? 18:06:37 <nirik> I had a few random ideas: 18:06:44 <tflink> I forget, do we have any step-by-step instructions for installing updates-testing and giving karma? 18:06:53 <nirik> could we mine old updates for people who have given lots of karma but are not proventesters? 18:06:55 <maxamillion> I've actually thought about this in the past ... I think maybe getting a little funding to offer swag of some sort would be good ... like a "Fedora QA Community Member" t-shirt or something (likely a less expensive option to start out with) 18:06:58 * tflink will wait for nirik's ideas before giving mine 18:07:31 <nirik> well, all we have currently is http://fedoraproject.org/wiki/Proven_tester 18:07:40 <nirik> which lists some things, but not sure in how much detail. 18:07:44 <tflink> I remember an interesting conversation about metrics and their human effect 18:08:00 <nirik> I really think most of it is that people don't know how easy it is to become a proventester 18:08:01 <maxamillion> https://picasaweb.google.com/103451550643082159750/B#5576235128261512338 <--- I doodled up a t-shirt idea a while back 18:08:09 <tflink> as soon as there is some sort of reward for doing X based on measurement Y, people will generally start behaving differently 18:08:20 <nirik> maxamillion: that would be cool. dunno if we have funds for such a thing tho 18:08:22 <maxamillion> tflink: well ... true 18:08:25 <maxamillion> nirik: yeah 18:08:25 <tflink> nirik: yeah, and how much they're needed 18:09:03 <nirik> ideally, everyone who provides karma on any kind of regular basis should take the few minutes to become a proventester, IMHO. 18:09:54 <tflink> I wonder if stuff like step by step instructions would help 18:10:00 <tflink> maybe screencasts, too? 18:10:14 <tflink> the other two things that I can think of are: 18:10:25 <tflink> people worried about borking their system and losing data/time 18:10:46 <tflink> make f-e-k a little more user friendly 18:11:12 <tflink> I've thought about trying to make a gui for f-e-k before 18:11:38 <nirik> I'm not sure how active the developer is these days... but he was around not long back 18:11:40 <tflink> to make it easier to find test cases, related packages and skip over stuff that you don't feel right karma-izing 18:12:05 <tflink> I'd be interested in doing it but the issue is available time 18:12:13 <nirik> yeah. 18:12:57 <tflink> is it something that we really think would help? 18:13:04 <nirik> also, folks providing feedback on list could be pointed at becoming proventesters... 18:13:38 <Cerlyn> Would a GUI avoid the "skip" confusion? Or can that be solved at the command line? 18:13:49 <nirik> not sure. I think a bigger win might be to setup something so PK would say "hey, there's an update of foo you have installed in updates-testing, would you like to test it?" 18:14:01 <tflink> just leaving karma would be a start, not even becoming a proven tester 18:14:21 <maxamillion> nirik: that's a cool idea 18:14:34 <tflink> nirik: yeah, that would be good. Give a list of testing updates available and let the user select which ones 18:14:40 <nirik> it's been floated many times before, but has noone working on it. ;) 18:14:41 <tflink> to install 18:14:55 <nirik> a yum plugin might be easier. ;) 18:15:55 <nirik> I don't think there's a magic bullet... 18:15:58 <Cerlyn> More precisely needed is a way to identify which packages a user actually used and for libraries, how they got there 18:16:07 <Cerlyn> A lot of packages are installed by default I would never use 18:16:27 <nirik> Cerlyn: yeah, sometimes its a bit of a puzzle... 18:16:37 <tflink> and updated libraries that come down and I have no idea what they do 18:16:39 <nirik> libfoo is an update. What the heck do I use libfoo for? 18:16:49 * tflink was wondering if a dependency tree + description might help 18:17:14 <tflink> appbar -> libbar -> libfoo (appbar does magical stuff) 18:17:19 <nirik> yeah, it does give summary now, but tree would be nice. 18:17:39 <nirik> not sure how hard that would be to implement. 18:17:52 <Cerlyn> It also makes no sense to tell me about a new ATI Xorg driver is available, but I have nvidia/Intel/unknown-to-me instead. Saying video still works could be useless. 18:17:56 <tflink> me neither, I've never even looked @ f-e-k code 18:18:29 <nirik> Cerlyn: yeah, all the x drivers are installed by default, since they are small and you could move your install to another box and want it to work. 18:18:44 <nirik> but it means you have a number that aren't actually in use. 18:20:15 <nirik> so, I think short term here: mine existing karma for non proventesters and ask them to join, more press (blog posts, point people on test list), and look at f-e-k improvements? 18:21:10 <tflink> yeah 18:21:42 <nirik> longer term: gui for f-e-k and having pk or something like it offer updates-testing 18:21:50 <nirik> another idea was tie it into abrt. ;) 18:22:09 <nirik> foo has just crashed and you reported a bug on it... but foo has a updates-testing update, would you like to test? 18:22:28 <tflink> that's a cool idea 18:24:17 <nirik> #idea short term here: mine existing karma for non proventesters and ask them to join, more press (blog posts, point people on test list), and look at f-e-k improvements 18:24:26 <nirik> #idea longer term: gui for f-e-k and having pk or something like it offer updates-testing 18:24:33 <nirik> #idea longer term: tie into abrt 18:24:40 <nirik> anything more on this? or shall we move on? 18:25:26 <nirik> #topic One stop page for updates testing resources 18:25:41 <nirik> Another idea I had was to try and create a short page with all the resources. 18:25:47 <nirik> we have stuff all over the place now. 18:25:50 <maxamillion> that's probably a good one 18:25:58 <maxamillion> I thought we had that at one point ... or atleast a mockup of one 18:26:00 <tflink> yeah, especially for the short term 18:26:01 <maxamillion> on the wiki somewhere 18:26:04 <nirik> bodhi, fedora-easy-karma, rss feeds, reports to mailing lists, etc 18:26:51 <nirik> would anyone be able to put in some time to make one? ;) 18:27:01 <nirik> I could help out, but not sure how much time I have to devote to it. 18:27:48 <nirik> I can try and make something tho.. or ask on the list. ;) 18:28:10 <maxamillion> oh ... like all in wonder page for QA efforts, that's different than what I had in mind but I like it 18:28:46 <nirik> one of the complaints has been that it's hard to see all the stuff you need... 18:28:57 <nirik> if we can get all the links in one place it might help 18:29:18 <tflink> yeah 18:29:19 <maxamillion> yeah 18:29:40 <maxamillion> and for long term, see if we can't get some of that stuff merged into bits of the revamp to fedora community? 18:29:44 <nirik> #info will try and create a package updates tester page with links to resources. 18:30:03 <nirik> maxamillion: possibly. Not sure if thats a focus for it, but it could be nice for that. 18:30:56 <tflink> lets figure out what's really being used before trying to integrate it :) 18:31:08 <tflink> or what would be the most useful 18:31:10 <nirik> fair enough 18:31:23 <maxamillion> rgr 18:31:39 <nirik> #topic Pending updates 18:32:00 <nirik> so, did we want to try and look at the updates pending in 15/14 currently and see if we could move along the ancient ones? 18:32:06 <nirik> or is that better done out of the meeting? 18:32:53 <nirik> 11 critpath ones in f14 18:33:12 <nirik> 19 in f15 18:33:53 <nirik> some of them are ancient due to almost no one having the hardware. 18:34:00 <nirik> some are just unclear how to test. 18:35:30 <nirik> I'm hoping to have an old laptop setup to help test 14/15 soon... 18:35:41 <nirik> #info 11 critpath updates pending in f14 18:35:49 <nirik> #info 19 critpath updates pending in f15 18:35:56 <nirik> #topic Open Floor 18:36:01 <nirik> ok, anything for open floor? 18:36:15 <nirik> should we continue meeting? or does it seem to be doing any good? 18:36:37 <mike-c> nirik: I came in late - so trying to get the gist of the discussion... 18:36:44 <satellit_> would there be a limited proventester category for testers who mainly do installs to Hardware? 18:36:59 <nirik> mike-c: basically we are trying to increase updates-testing package testing. ;) 18:37:07 <tflink> satellit_: what would the benefit of that be? 18:37:37 <nirik> mike-c: get more proventesters, or general testers involved so packages don't get stuck in updates-testing. 18:37:37 <satellit_> I am not sure I have the skills to do some of the other kinds of testing.... 18:38:00 <mike-c> Any value in asking testers what main kinds of testing and which versions of Fedora they are mainly doing? 18:38:30 <nirik> satellit_: then just don't add karma where you don't feel you have tested enough... ;) 18:38:34 <Southern_Gentlem> satellit_, if you are running a vm to test a package that is good as well 18:38:37 <satellit_> ok 18:38:49 <nirik> mike-c: well, not sure... we want more people particularly for the older releases... 18:38:57 <Southern_Gentlem> i am running f14 so i do test f14 native and f15 in a vm 18:39:49 <mike-c> For example I am very keen to test kernels, nouveau and similar, as well as mail server and similar but also test other things where I can - mainly f16 currently with a little f15 18:40:16 <Southern_Gentlem> thats why i think a proven-testers mailing list that on monday and thursday spam with a list that needs tested would be great 18:40:27 <nirik> in general I think we have more testers on newer releases... fedora is so fast moving, most of the folks involved tend to want to move fast with it. 18:40:58 * nirik has a f14 and f15 vms, and a bunch of f15 machines and a few f16 machines where I do most of my testing. 18:41:31 <mike-c> Makes sense if testers do run older versions as servers and where stability is essential but newer versions where they want to test upcoming performance? 18:42:23 <nirik> Southern_Gentlem: would a web page with links to the current stuff meet your needs? 18:42:34 <nirik> mike-c: yeah, I think some of that is happening. 18:43:13 <mike-c> Southern_Gentlem: I am happy with being spammed on the test list with the list of packages that need testing with one spam shot per fedora version.... 18:44:22 <nirik> Southern_Gentlem: I floated the idea of another list last week, and the reaction was pretty negative on the test list. Do chime in there if you like... 18:45:43 <mike-c> I guess also that for most people availability to run tests varies in time depending on other commitments... for example I do try to join the graphics test days but can't always be free then - similarly during the normal cycle of listed packages for test from the announcements on the list 18:46:48 <nirik> yeah, absoutely. 18:47:13 <nirik> I tend to run fedora-easy-karma once a week or so to collect all the feedback from using the machine... 18:47:22 <nirik> or more often if there's a specific update that needs testing 18:47:26 <mike-c> Would there be any value in asking people who join proventester what their main interest is i.e. testing packages for released N, N and N-1, N and N+1 etc so you could have a list of what the pool of testers are likely to be doing? 18:47:31 <Southern_Gentlem> mike thats why i think another list would be good thing 18:48:12 <nirik> mike-c: well, I'm not sure how that helps us... I guess we would know more who we have out there, but it doesn't help us grow or provide better coverage... 18:49:38 <mike-c> I guess - it would be nice to have a balanced division of effort - though I know that is why this discussion is happening! At the moment it is pot luck what gets tested in any given period... 18:50:03 <Cerlyn> It probably wouldn't be too hard to push a survey to everyone already in the group as well 18:50:13 <nirik> sure, but we can't easily say: 'hey, you, go test f14' (well, we could, but they could say 'no, I don't care about f14') 18:51:30 <mike-c> Sure - testing is dependent on voluntary effort - and there have been several attempts to encourage testers to do things - Adam Will occasionally makes a specific request to the test list 18:51:56 <nirik> yeah, although those are almost always for fN-branched. 18:52:08 <nirik> occasionally a maintainer will ask for testing... 18:52:13 <mike-c> I guess that it only needs one or two people to test any one package so provided the pool of testers is reasonable it "should" work! 18:52:35 <nirik> I guess this is related to another idea I read somewhere (can't recall where): 18:52:58 <nirik> revive the 'qa-contact' field for packages... so people could sign up for specific packages. 18:53:29 <nirik> pkgdb has a 'Initial qa contact' field... which I don't think we currently use for anything. 18:54:00 <mike-c> Interesting! That may well work for some testers who have specific interests and do test the ones they have a specific love for on a regular basis - perhaps across several fN 18:54:07 <maxamillion> nirik: should qa-contact tag everyone in proventesters for all critpath packages? 18:54:28 <nirik> well, I am not sure what actual effect it has. It might be cc'ed on all bugs only. 18:54:29 <maxamillion> or just divvy it up accordingly? 18:54:34 <maxamillion> rgr 18:54:35 <nirik> abadger1999: you happen to be around? 18:54:42 <abadger1999> nirik: yep 18:54:52 <abadger1999> qa-contact is for bugzilla 18:54:57 <abadger1999> per-package 18:55:09 <abadger1999> takes a single fas username 18:55:26 <nirik> ok, so thats not any different than inital cc really? 18:55:32 <abadger1999> yeah 18:55:42 <abadger1999> Just a different field in bugzilla 18:55:52 <abadger1999> (qa contact field instead of CC field) 18:56:11 <Cerlyn> I think it has some purpose if Bugzilla is setup to use a test-in-build style of operation 18:56:37 <mike-c> Perhaps it could be offered for testers to sign up on the bz to "take" the "qa-contact" field? 18:56:40 <nirik> we could perhaps co-opt it some in bodhi to send those people info about updates to that package I guess. 18:56:46 <Cerlyn> I had another idea - could Bodhi notify people who tested the last N versions of a package and gave non-zero karma points that a new one has been released? 18:56:54 <nirik> but I think it's going to require more thought/design. 18:58:11 <Cerlyn> (The QA contact role has its own set of email preferences for users in bugzilla) 18:58:35 <nirik> Cerlyn: we could, but I bet people would complain. 18:59:19 <nirik> ok, we are running low on time. 18:59:36 <nirik> any last minute items? do folks want to try and meet next week? 19:00:31 <nirik> lets continue on list... 19:00:38 <nirik> Thanks for coming everyone! 19:00:41 <nirik> #endmeeting