Hey all,
we'd like to announce the 'Fit and Finish' initiative for Fedora,
http://fedoraproject.org/wiki/Fit_and_Finish
with the goal to improve the user experience of the Fedora desktop. We want to identify the small (and sometimes large) roadblocks that make everyday computer use harder than it needs to be, and try to fix them.
'Fit and Finish' is meant to be complementary to the work of the Fedora QA team. They do a great job of ensuring the quality of all the new features that land in Fedora each cycle. But when features are developed and tested on their own, the overall experience of the system as a whole can sometimes end up a bit uneven and rough. 'Fit and Finish' will focus on improving the way our users experience Fedora.
To achieve this, we will hold regular test days, each of which will focus on use cases in a certain area. A few ideas for test day topics can be found on the 'Fit and Finish' page already. If you have ideas for other areas that could benefit from this kind of attention, please let us know.
Our first test day will focus on display configuration, and will be held on Tuesday, July 7, from 12:00 to 21:00 UTC (8am -> 5pm EDT), in the fedora-fit-and-finish irc channel on FreeNode. Please come and join us there !
Matthias Clasen for the Desktop team
On Mon, 2009-06-29 at 15:27 -0400, Matthias Clasen wrote:
If you have ideas for other areas that could benefit from this kind of attention, please let us know.
I can think of a number of different cross-component tests...
* Audio Applications - Does the sound hardware work out of the box? Do all applications that produce audio output work out of the box? (For example, I had trouble in Fedora 9 with Rosegarden, which needs the Jack audio server set up in addition to PulseAudio, so it doesn't work out of the box.) Does it work if you try to use the application while another application is already producing audio output? Does volume control work? See:
https://fedoraproject.org/wiki/Bugs/Common#PulseAudio-based_volume_control_c... * Network Audio - Is it easy to play music on one computer and listen to it from the speakers on a different computer on the same subnet? * File Formats - For each given file type in the known universe (.txt, .doc, .jpg, .ogg, .mp3, .iso, etc.) are they handled properly by each application you would expect to do so? Files not handled internally should be passed off to the right application by all web browsers, IRC/instant message clients, Nautilus, etc. All of these applications should use the same universal list of handlers. If the necessary handler is not installed, one of these new features should be triggered: http://fedoraproject.org/wiki/Releases/FeatureCodecBuddy https://fedoraproject.org/wiki/Features/AutoFontsAndMimeInstaller * Help - Does activating the help system in each application work? Does it actually provide helpful information? Is documentation for non-network-oriented applications accessible even without a network connection? * Application Capabilities - Are applications honest about what file types they can open (e.g. when you select File -> Open from the menu, is there a filter which shows you only openable files)? If a given application can handle a given type of file, does that show up in the list of handlers in other applications (like a web browser)? If you open a file from within an application which it cannot handle, does something sensible happen (as opposed to crashing, getting gibberish, etc.)? Do all applications handle non-ASCII filenames properly? * Cut and Paste - Does it work smoothly everywhere, whether using mouse or keyboard shortcuts? Even in the To: and Cc: fields when composing an email? Even in the URL bar? In gnome-terminal? Does complex data (e.g. formatted text, graphics, multi-line text in single-line input boxes) get transferred properly? Most importantly, does cut-and-paste handle non-ASCII text encodings properly (without corrupting characters)? * International Content - Inside documents and in their own menus and GUI, applications should generally be able to render glyphs in any language or Unicode symbol set. If a suitable font is not installed, this should kick in: https://fedoraproject.org/wiki/Features/AutoFontsAndMimeInstaller In some cases, suitable free fonts do not yet exist, but these cases are generally quite exotic. A complex example: https://bugzilla.redhat.com/show_bug.cgi?id=455510 (Undisplayable glyphs on Wikipedia) * Menus - When someone unfamiliar with the layout thinks, "I want to do ___", is it easy to navigate to that function in the Gnome menus? If you install Everything, are all programs filed in the right place? Do all programs have a suitable menu icon? Is menu placement consistent with Add/Remove groups (e.g. Sound & Video vs Multimedia)? * Small Screens - Is everything still usable even when the screen resolution is 800x600 or 648x480? Most importantly, no functionality should be lost. One good test is to open each application and go to Edit -> Preferences; applications with too many preferences will often produce a window that is not scrollable even though it is too big to fit on the screen. Less important but good to test, is whether merely ugly or annoying things happen (like an application starting up with a window that is larger than the screen) which could be handled more gracefully. Is there a Fedora desktop configuration suitable for PDAs or other small mobile devices? * Press All the Buttons - Do all the keys you never use (Print Screen, Pause/Break, Windows, Menu, Play/Stop/Fast Forward, web shortcuts, etc.) actually do something sensible, both on the desktop and in the context of the command line and any given application (e.g. watching a video). Handy links to descriptions of traditional behavior: http://en.wikipedia.org/wiki/Keyboard_(computing) http://en.wikipedia.org/wiki/Category:Computer_keys * Persistence - Do your desktop settings stay in place after you logout? After you reboot? What about numlock, capslock, etc? * Desktop Stress Test - Add lots of applets to a panel. Do sensible things happen when notification icons appear? Do all the applets work as advertised? Are any of them redundant? Open multiple documents in multiple applications (OpenOffice, Firefox, Emacs, Gimp, etc.). Does the default configuration do sensible things (it's easy to find the document you are looking for using the window list)? What about alternate window-navigation tools? What about if you add multiple workspaces to the mix? * Working Remotely - Do applications function properly if you are running them remotely via ssh or other tool? Some known problems are related to: https://bugzilla.redhat.com/show_bug.cgi?id=484171 (ssh doesn't forward dbus) Applications should function acceptably even given this limitation. * Crashes - All crashes should bring up a user-friendly assistant to report the problem. Bug reports should be successfully filed in a suitable bug tracker without any special knowledge on the part of the reporter. Crashes can be induced from the command line with "kill -SEGV". More info: http://fedoraproject.org/wiki/Features/ABRT (Some applications use different crash-handling software.) It would also be nice if applications recovered from crashes without significant data loss. (Can be tested by opening a document and making several changes without saving, waiting a little while to let any periodic auto-save kick in, then inducing a crash.) Obviously any real crashes that are encountered during testing should be reported, since they shouldn't happen in the first place.
As testing days are finished, it might be interesting to output a not-very-detailed but growing list similar to http://fedoraproject.org/wiki/QA/ReleaseCriteria but for "things the user interface should do seamlessly". As in, "if it's not doing this for you, you should file a bug" and also as guide for future re-testing. Or maybe it's more efficient to leave tracking up to Bugzilla and just trust people to know something is wrong (whether it's a bug or a design flaw) when they see it.
-B.
A great list :-)
The only additions I can think of at the moment are:
extension to the autosave test: Does the application find the autosaved file when you try to reopen the original data file, or is the user expected to be able to find it?
Quit application or log out of the desktop with unsaved data. Do you get a dialog offering to save your data?
I'd also add: - Does suspend and resume work? - Does all the hardware continue to work after resume? (ALSA bug #305, I'm talking to you.) - Does power management work for both AC and battery? - Does CPU throttling work? - Does the machine turn off when shut down?
(But I suppose I should read what's already there...)
Chris
On Tue, 2009-06-30 at 13:55 -0400, Christopher Beland wrote:
On Mon, 2009-06-29 at 15:27 -0400, Matthias Clasen wrote:
If you have ideas for other areas that could benefit from this kind of attention, please let us know.
I can think of a number of different cross-component tests...
[...]
Yeah, this is a very nice checklist for 'basic sanity'. And any bug you file about a problem in one of those categories certainly qualifies as a 'fit and finish' issue. But as a test day topic, it might be a bit boring to spend the whole day testing e.g. copy-and-paste between app X and Y to fill a big matrix...
Matthias
Matthias Clasen wrote:
On Tue, 2009-06-30 at 13:55 -0400, Christopher Beland wrote:
On Mon, 2009-06-29 at 15:27 -0400, Matthias Clasen wrote:
If you have ideas for other areas that could benefit from this kind of attention, please let us know.
I can think of a number of different cross-component tests...
[...]
Yeah, this is a very nice checklist for 'basic sanity'. And any bug you file about a problem in one of those categories certainly qualifies as a 'fit and finish' issue. But as a test day topic, it might be a bit boring to spend the whole day testing e.g. copy-and-paste between app X and Y to fill a big matrix...
Matthias
The point of testing is to find issues and get them fixed. When issues are found but ignored because you don't run a pristine system with no questionable modules (closed source drivers) then it gets frustrating to even test.
For me, I am about to test my wife with a F11 install on her laptop. It will be interesting to see.
I have three issues that I see as major problems.
3D support out of the box. I know that this is a major issue. For a home system it is a requirement. No 3D, not use.
My own major headache is how to set up the locale to a format for all users across the system that works. I tried to modify the LC_X variables but I started to get all kinds of error messages, (on F10) so I just gave up. This is a RFE but to me it is a usability issue.
Finally as mentioned before. Codecs. Again the law is in the way.
I will be participating when I can.
-
Birger Monsen -
Chris Schumann -
Christopher Beland -
Matthias Clasen -
Robin Laing