Hi folks!
So Colin and I had a bit of a chat about openQA Atomic testing 'integration', and one of the things that came up was the state of the 'compose check report' emails for Atomic tests.
If you're signed up to test@ or devel@ you may have noticed that a 'compose check report' mail is sent out for every mainline Fedora compose openQA tests, with info on the numbers of passed and failed tests and some other stuff.
When we set up openQA to test the Atomic Host OStree installer image and to test the nightly 'two-week Atomic' composes, I asked whether we should generate those mails, and who they should go to.
The answer I got - and the way it's configured right now - was that people only wanted the mails produced when a test *failed*, and they should be mailed to this list plus to one person directly (I think at first it was Adam Miller, it now seems to be Mike McGrath).
I've just verified that this is actually working: when openQA tests the 'two-week Atomic' composes and a failure occurs, the mail does get sent. However, there hasn't been a failure of the test since - AFAICS - 2016-10-01. So that's why no such mails have been sent out recently. I got Mike to check, and he actually did get a mail on 20167-10-01, when the test last failed.
I think the mails that have been sent to cloud@ were never approved through moderation, so they never actually appeared here. It'd be good if a list moderator could check if they see a few 'compose check' mails hung up in moderation, or something.
So, I just wanted to give a quick heads-up on that, and say that if anyone would like that configuration changed, I can do it easily enough. We could have a report generated for every compose as for the 'main' composes, but it'd be quite dull I think, because there's only one test and it almost always passes. We can also change the list of addresses that receive the mails when they're sent, if it should be changed.
Also do let me know if running more tests on the Atomic installer image would be desirable. For now we just run a single test - a straight- through install test on x86_64 BIOS - since that's all I was asked for initially. We could run the same test on UEFI, if desired, and we could run some of the post-install tests that are run on other images, and we could run some of the install variant tests; I just don't know which ones are relevant / useful for the Atomic installer image.