How about this. Instead of trying to craft a policy right now which applies equally to all parts of the distribution. Let's narrowly define a prioritized list of functionality which we think is critical and deserves to be a priority when doing update QA.
Here's my short list 1) Packaging Updating at the console (rpm and yum) 2) Package Updating in the desktop (PK and friends) 3) python-matplotlib 4) xeyes
Your list with most likely be different than mine. But can we get project wide consensus as to the top 2 are really really important to keep working for 'most' people? Everything else aside.. all the good and bad ideas about how to do a top to bottom restructuring of updates generation project wide off the table for a few seconds. Can we agree that 1 and 2 are critical functionality which deserves extra precautionary effort to reduce the risk of falling over and dying for users compared to other functionality? Maybe more important than security? If an update goes out which could impact rpm,yum,PK and friends can we make it a policy that those updates require a specific level of testing, even if it means holding up a security tagged update until basic functionality of rpm,yum,PK is confirmed?
This is a risk management argument I am making.
-jef"is really thinking about adapting all the Integrated Safety Management training that was beaten into him while at PPPL and re-applying it to Fedora packaging"spaleta
How about this. Instead of trying to craft a policy right now which applies equally to all parts of the distribution. Let's narrowly define a prioritized list of functionality which we think is critical and deserves to be a priority when doing update QA.
Here's my short list
- Packaging Updating at the console (rpm and yum)
- Package Updating in the desktop (PK and friends)
- python-matplotlib
- xeyes
Your list with most likely be different than mine. But can we get project wide consensus as to the top 2 are really really important to keep working for 'most' people? Everything else aside.. all the good and bad ideas about how to do a top to bottom restructuring of updates generation project wide off the table for a few seconds. Can we agree that 1 and 2 are critical functionality which deserves extra precautionary effort to reduce the risk of falling over and dying for users compared to other functionality? Maybe more important than security? If an update goes out which could impact rpm,yum,PK and friends can we make it a policy that those updates require a specific level of testing, even if it means holding up a security tagged update until basic functionality of rpm,yum,PK is confirmed?
This is a risk management argument I am making.
Well made. +1.
But can we really afford to be so slipshod with xeyes? Next thing you know, someone with break KSpaceDuel, and then it's all over. . .;)
-jef"is really thinking about adapting all the Integrated Safety Management training that was beaten into him while at PPPL and re-applying it to Fedora packaging"spaleta
-- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Jon Ciesla wrote:
How about this. Instead of trying to craft a policy right now which applies equally to all parts of the distribution. Let's narrowly define a prioritized list of functionality which we think is critical and deserves to be a priority when doing update QA.
Here's my short list
- Packaging Updating at the console (rpm and yum)
- Package Updating in the desktop (PK and friends)
- python-matplotlib
- xeyes
Your list with most likely be different than mine. But can we get project wide consensus as to the top 2 are really really important to keep working for 'most' people? Everything else aside.. all the good and bad ideas about how to do a top to bottom restructuring of updates generation project wide off the table for a few seconds. Can we agree that 1 and 2 are critical functionality which deserves extra precautionary effort to reduce the risk of falling over and dying for users compared to other functionality? Maybe more important than security? If an update goes out which could impact rpm,yum,PK and friends can we make it a policy that those updates require a specific level of testing, even if it means holding up a security tagged update until basic functionality of rpm,yum,PK is confirmed?
This is a risk management argument I am making.
Well made. +1.
But can we really afford to be so slipshod with xeyes? Next thing you know, someone with break KSpaceDuel, and then it's all over. . .;)
I suggested an time based approach on this matter on #fedora-qa.
Have packages stay in updates-testing for an x amount of time before pushed to update with the ability to be rushed through.
Packages that would cause system breakage would stay for example 5 days. Non system-breakage packages for example 3 days Security and urgent updates for a day them being flagged as urgent and mail sent to the test-list asking testing asap.
That at least guarantee an x time for testing and to provide feedback but of course does not guarantee feedback from testers any more than it is today.
JBG
Jóhann B. Guðmundsson wrote:
Jon Ciesla wrote:
How about this. Instead of trying to craft a policy right now which applies equally to all parts of the distribution. Let's narrowly define a prioritized list of functionality which we think is critical and deserves to be a priority when doing update QA.
Here's my short list
- Packaging Updating at the console (rpm and yum)
- Package Updating in the desktop (PK and friends)
- python-matplotlib
- xeyes
Your list with most likely be different than mine. But can we get project wide consensus as to the top 2 are really really important to keep working for 'most' people? Everything else aside.. all the good and bad ideas about how to do a top to bottom restructuring of updates generation project wide off the table for a few seconds. Can we agree that 1 and 2 are critical functionality which deserves extra precautionary effort to reduce the risk of falling over and dying for users compared to other functionality? Maybe more important than security? If an update goes out which could impact rpm,yum,PK and friends can we make it a policy that those updates require a specific level of testing, even if it means holding up a security tagged update until basic functionality of rpm,yum,PK is confirmed?
This is a risk management argument I am making.
Well made. +1.
But can we really afford to be so slipshod with xeyes? Next thing you know, someone with break KSpaceDuel, and then it's all over. . .;)
I suggested an time based approach on this matter on #fedora-qa.
Have packages stay in updates-testing for an x amount of time before pushed to update with the ability to be rushed through.
Packages that would cause system breakage would stay for example 5 days. Non system-breakage packages for example 3 days Security and urgent updates for a day them being flagged as urgent and mail sent to the test-list asking testing asap.
That at least guarantee an x time for testing and to provide feedback but of course does not guarantee feedback from testers any more than it is today.
Members of the QA group could provide that feedback
There is also an additional problem it's the lack of test cases. ( does not apply to bug fixes since you can inspect the bug report and check if it fixes what was discussed there )
For instance should a tester vote +1 in karma if the application starts? Should he vote +1 if the application does x,y and z?
"Updated to upstream version" is a favor of mine and tells me nothing
Well I have to admit sometimes I bother to go upstream and read the changelog sometimes..
Did that new upstream release bring several bug fixes? ( Dont those fixes need testing)
Did that new upstream release bring enhancements? ( Dont those enhancments need testing)
Hey I start the application it runs should I vote +1 in karma
JBG.
On Fri, Dec 12, 2008 at 09:44:28AM -0900, Jeff Spaleta wrote:
How about this. Instead of trying to craft a policy right now which applies equally to all parts of the distribution. Let's narrowly define a prioritized list of functionality which we think is critical and deserves to be a priority when doing update QA.
Here's my short list
- Packaging Updating at the console (rpm and yum)
- Package Updating in the desktop (PK and friends)
I'd propose, more largely @code, @base and dependencies.
-- Pat
On Fri, Dec 12, 2008 at 10:59 AM, Patrice Dumas pertusus@free.fr wrote:
I'd propose, more largely @code, @base and dependencies.
I think that's too broad a target to start with and you don't have the QA resourses in place to support a policy that broad.
If QA resources are scarce, how about we treat them as scarce and build narrow policies which let the existing resources get used for best benefit on targetted priorities. And as resources grow, we grow the list of priorities downward into more areas.
Right now. I am asking us as a project to suck it up and identify a top functionality priority and to live within our means as it comes to existing QA expectations.
For me, lowering the risk of the potential breakage of update functionality breakage is the number 1 priority in terms of critical functionality. I dare you to suggest an alternative specific functionality. If we can't even settle on one specific functionality as a priority we've no chance a targetting all of @core and @base. None.
-jef
On Fri, Dec 12, 2008 at 11:40:07AM -0900, Jeff Spaleta wrote:
On Fri, Dec 12, 2008 at 10:59 AM, Patrice Dumas pertusus@free.fr wrote:
I'd propose, more largely @code, @base and dependencies.
For me, lowering the risk of the potential breakage of update functionality breakage is the number 1 priority in terms of critical functionality. I dare you to suggest an alternative specific functionality. If we can't even settle on one specific functionality as a priority we've no chance a targetting all of @core and @base.
I think that the boot to console sequence should also be preserved. And networking too (not networking reconfiguration). Reading comps, indeed, @core + @base is too much. Even @core has some non critical packages, like authconfig and ed...
I propose for the console part: yum, rpm, initscripts, mkinitrd, the bootloaders, kbd, shadow-utils, (passwd?).
And basic network: iproute, dhclient
And dependencies of those packages basesystem, bash, coreutils, util-linux-ng, ncurses, libgcc, filesystem, e2fsprogs, cpio, glibc, curl, python...
-- Pat
On Fri, Dec 12, 2008 at 2:40 PM, Jeff Spaleta jspaleta@gmail.com wrote:
On Fri, Dec 12, 2008 at 10:59 AM, Patrice Dumas pertusus@free.fr wrote:
I'd propose, more largely @code, @base and dependencies.
I think that's too broad a target to start with and you don't have the QA resourses in place to support a policy that broad.
If QA resources are scarce, how about we treat them as scarce and build narrow policies which let the existing resources get used for best benefit on targetted priorities. And as resources grow, we grow the list of priorities downward into more areas.
Yes!
Right now. I am asking us as a project to suck it up and identify a top functionality priority and to live within our means as it comes to existing QA expectations.
Based on an informal weighted system, using scores for Technicality, Severity, and Emotional impact, these add up fairly evenly:
*) grub (e.g. Bug 450143) - not so severe, and short to fix (for those adept with repair disk), but it's soooo disheartening to see a system turned to paperweight in an instant.
*) wireless - it just *has* to work, period. Oft-times, even currently wired areas (campus, business, etc.) go wireless. Extra bonus : connect in runlevel 3. I can *remote desktop* to my laptop upstairs when it's running the windowing os from M$, yet I can't even ssh to it when I boot up Fedora. Come on, it's nearly 2009 folks....
*) the stuff between grub and networking (forgive the generalization :-) - wired networking included here as a fallback option and is, generally, easier to implement/fix.
*) wired network - it's still important.
[unless we have the above, we don't have a simple update ability, right?]
*) [pony alert] some sort of 'update from media' feature - like an "Install from" dvd/cd/sd which contains updates instead of base packages... repair-disk-on-drugs... Something to help get the machine back that's easier than going into the office, burning to cd all the packages to fix my breakage (assuming I've figured that out beforehand - of course I'd get it right the first time!), booting it (assuming I can), mount the cd (assuming I can), edit yum.conf for the disk-based updates, and apply them...??
*) Package Updating at the console (rpm and yum)
*) The desktop that, 1. Works, and 2. Looks the same as my previous login [related example is KDE in rawhide - no desktop, but the right click konsole terminal is there at least - even better with a wireless net, see above :-)]
jerry
On Sun, 2008-12-14 at 12:00 -0600, Jerry Amundson wrote:
On Fri, Dec 12, 2008 at 2:40 PM, Jeff Spaleta jspaleta@gmail.com wrote:
On Fri, Dec 12, 2008 at 10:59 AM, Patrice Dumas pertusus@free.fr wrote:
I'd propose, more largely @code, @base and dependencies.
I think that's too broad a target to start with and you don't have the QA resourses in place to support a policy that broad.
Definitely too large.. but not by a lot.
Right now. I am asking us as a project to suck it up and identify a top functionality priority and to live within our means as it comes to existing QA expectations.
The informal Critical Path for current QA testing is: yum and its deps.
No matter how bad *other* things are broken, as long as yum works you can 'yum update' and download fixes. But if yum breaks, you have a Severity CFF issue [1].
Note that by "yum and its deps" I don't mean the actual "Requires:" on the 'yum' package, I mean "everything required to have yum run properly". So this includes stuff like - duh - networking.
So, throw in NetworkManager and its deps. Like it or not, NetworkManager is the default network management system, and if it doesn't work, we are once again registering Severity CFF issues.
Since this discussion is about defining a *formal* critical path, I propose the following:
Bootloader: grub (yaboot on ppc), kernel, and all dependent packages. Networking: NetworkManager and all dependent packages. Update system: yum and all dependent packages.
I *believe* that would pull in everything Patrice mentioned earlier (e.g. kernel requires initscripts, mkinitrd, module-init-tools, etc. mkinitrd requires lvm, e2fsprogs, bash, coreutils, etc.) and all those things that are completely vital that sometimes we don't think about (dbus, cough cough cough).
This will define our Tier-1 / Core / Critical Path / NTBFW[2] package set, which should have some additional sanity checks / scrutiny applied during updates, release testing, and so on.
-w
[1] Completely Fucking Fucked. A common QA term that I just now made up. [2] Not To Be Fucked With. More QA parlance that I just now made up.[3] [3] Us QA guys have dirty mouths. Or, at least, I do.[4] [4] Oh come on. Show me a QA engineer who has *not* shouted curses so vile as to cause scarring and/or spontaneous combustion in lab animals, and I will show you a ticking time bomb of rage. Or a mute. Or someone who is no fun at parties.
On Mon, Dec 15, 2008 at 05:26:26PM -0500, Will Woods wrote:
[4] Oh come on. Show me a QA engineer who has *not* shouted curses so vile as to cause scarring and/or spontaneous combustion in lab animals, and I will show you a ticking time bomb of rage. Or a mute. Or someone who is no fun at parties.
Usually it's the programmers/developers/engineers that shout curses at the QA guy when he tells them their product is fucked.
On Mon, 2008-12-15 at 17:26 -0500, Will Woods wrote:
On Sun, 2008-12-14 at 12:00 -0600, Jerry Amundson wrote:
On Fri, Dec 12, 2008 at 2:40 PM, Jeff Spaleta jspaleta@gmail.com wrote:
On Fri, Dec 12, 2008 at 10:59 AM, Patrice Dumas pertusus@free.fr wrote:
I'd propose, more largely @code, @base and dependencies.
I think that's too broad a target to start with and you don't have the QA resourses in place to support a policy that broad.
Definitely too large.. but not by a lot.
Right now. I am asking us as a project to suck it up and identify a top functionality priority and to live within our means as it comes to existing QA expectations.
The informal Critical Path for current QA testing is: yum and its deps.
No matter how bad *other* things are broken, as long as yum works you can 'yum update' and download fixes. But if yum breaks, you have a Severity CFF issue [1].
Note that by "yum and its deps" I don't mean the actual "Requires:" on the 'yum' package, I mean "everything required to have yum run properly". So this includes stuff like - duh - networking.
So, throw in NetworkManager and its deps. Like it or not, NetworkManager is the default network management system, and if it doesn't work, we are once again registering Severity CFF issues.
Since this discussion is about defining a *formal* critical path, I propose the following:
Bootloader: grub (yaboot on ppc), kernel, and all dependent packages. Networking: NetworkManager and all dependent packages. Update system: yum and all dependent packages.
A followup: here is the complete list of (source) packages in that set, at least for F10-i386:
acl attr audit avahi basesystem bash bzip2 ca-certificates chkconfig compat-db ConsoleKit coreutils cpio cracklib crontabs cryptsetup-luks curl cyrus-sasl db4 dbus dbus-glib device-mapper-multipath dhcp dhcpv6 diffutils dirmngr dmidecode dmraid dnsmasq e2fsprogs elfutils ethtool eventlog expat fedora-logos fedora-release fedora-release-notes file filesystem findutils gamin gawk gcc gdbm glib2 glibc gnupg2 gpgme grep grub gzip hal hal-info hdparm initscripts iproute iputils isomd5sum kbd kernel keyutils krb5 less libcap libdaemon libdhcp libgcrypt libgpg-error libidn libksba libnl libpcap libpng libselinux libsepol libsmbios libssh2 libusb libx86 libxml2 linux-atm logrotate lua lvm2 MAKEDEV mdadm mingetty mkinitrd module-init-tools ncurses net-tools NetworkManager nspr nss openldap openssl pam parted pciutils pcre pinentry pm-utils PolicyKit popt ppp procps psmisc pth pygpgme python python-iniparse python-urlgrabber radeontool readline rpm rsyslog sed setup shadow-utils sqlite sysvinit tar tcp_wrappers texinfo tzdata udev upstart util-linux-ng vbetool wpa_supplicant yum-metadata-parser zlib
No real surprises here, although there's a couple things that I hadn't really thought about before: gamin, for instance. And lua? Oh yeah, RPM has an embedded lua interpreter, doesn't it.
So that would be the proposed list of Really Important Packages.
-w
-
"Jóhann B. Guðmundsson" -
Chuck Anderson -
Jef Spaleta -
Jerry Amundson -
Jon Ciesla -
Patrice Dumas -
Will Woods