NOC dashboard idea
by Patrick マルタインアンドレアス Uiterwijk
After the failed mediawiki patch of yesterday, and the downtime
resulting from it, I thought of something that might be useful for our
users: a dashboard to view the current status of services.
When we have problems, we could refer users to this page, so that they
can check whether we know about the downtime yet.
This should be hosted on a server and domain outside of the Fedora
datacenter and fedora main domain, such that this will contain
reachable for users when everything breaks down, maybe OpenShift or
some other public host?
An example of such a dashboard would be: http://status.apps.rackspace.com/.
My suggestion would be to make it very easy to toggle service for the
admins and to submit news (maybe an addition to zodbot?) for the
users.
Kevin suggested a dashboard for admins which combines Nagios and
Collectd information to get a quick and complete overview of status.
Please let me know what you think of this ideas.
11 years, 10 months
Meeting Agenda Item: Introduction Matt Jones
by Jones, Matthew
Hi my name is Matt, I would like to start learning more about what goes on behind the scenes and apply my current skills and gain some new ones, hopefully in web application development. By day I work in systems/network operations for a FOIP company in, mainly doing new physical deployments and backend grunt work (backups, monitoring(Nagios), vulnerability scanning(NMap, Nessus), firefighting. I have a good working knowledge of BASH, Sed/Awk, Python, C/C++, SQL, MediaWiki and some Django. I hope to put the little Django experience I have to more use and expand upon it.
By night I am working on a small script/package collection for Windows admins who refuse to actually use Windows or a heavyweight GUI for that matter. My goal is a single package that will contain everything most admins need including rdesktop, OpenConnect and various scripts for parsing common windows log files into meaningful noise. If I have to do something more than twice, I'm scripting it.
My IRC nick is urlugal and I hope to contribute to the website and the wiki as my skills improve. For now I am ok with watching and learning unless someone has a specific project I can get my feet wet with. I couldn't find anything that looked relatively simple in the Active Tickets list and didn't want to step on toes.
11 years, 10 months
Warning for development and staging instances of different services
by Patrick マルタインアンドレアス Uiterwijk
As of recent, I am busy with a new feature for services, to add a
warning message if the server is a development or staging deployment.
This will mean different things for different services:
FAS: Account System will have a different color of the top bar, and
show a big header "STAGING INSTANCE" or "DEVELOPMENT INSTANCE". This
is live on development, but not yet on staging instances.
pkgdb: Package Database will show a big header "STAGING INSTANCE" or
"DEVELOPMENT INSTANCE". This is already live on both development and
staging instances.
This feature will be extended to other services, so do not be
surprised when you see these messages on other services as well, as I
will continue to work with service maintainers to add it to all
services.
If, however, you see this warning on a production instance, please do
not hesitate to warn me or any of the service developers about this.
11 years, 10 months
using ansible to do status checks on systems from remote
by Seth Vidal
Want to check on a set of hosts but you don't have root/sudo access on
lockbox?
Want to do it in a way that you can hit the host quickly w/o having to do
them one by one?
Here's how you can use ansible from your local computer to do it:
1. check out ansible:
git clone git://github.com/ansible/ansible.git
2. get a copy of our host inventory:
ssh lockbox01 "pushd /srv/web/infra/hosts>>/dev/null; for host in
*.org; do echo \$host; done" >> ~/fedora-inv
3. make sure your ~/.ssh/config file lets you bounce through bastion for
fedora hosts:
Host *.phx2.fedoraproject.org *.vpn.fedoraproject.org *.qa.fedoraproject.org
IdentityFile ~/.ssh/fedora-rsa
VerifyHostKeyDNS yes
ProxyCommand ssh -q bastion.fedoraproject.org nc %h %p
4. cd into the ansible checkout dir:
5. setup the ansible environment:
. ./hacking/env-setup
6. run your commands like this:
ansible -c ssh -u skvidal -i ~/fedora-inv app??.phx2* -a uptime
that reads:
- use the ssh connectory as user skvidal with ~/fedora-inv as the
inventory file. Connect to the hosts app??.phx2* and run the command
module (the default module) with argument of "uptime"
the output looks like this:
app07.phx2.fedoraproject.org | success | rc=0 >>
21:20:34 up 3 days, 23:22, 1 user, load average: 0.00, 0.03, 0.00
app02.phx2.fedoraproject.org | success | rc=0 >>
21:20:34 up 3 days, 22:00, 5 users, load average: 2.41, 2.26, 3.27
app03.phx2.fedoraproject.org | success | rc=0 >>
21:20:34 up 1:27, 3 users, load average: 3.09, 3.89, 6.39
app04.phx2.fedoraproject.org | success | rc=0 >>
21:20:34 up 3 days, 23:24, 2 users, load average: 3.68, 3.11, 12.08
app01.phx2.fedoraproject.org | success | rc=0 >>
21:20:34 up 3 days, 21:39, 7 users, load average: 1.80, 2.03, 8.66
please do try to use this sensibly :)
-sv
11 years, 10 months
New FAS release for after alpha release -- New feature to test
by Toshio Kuratomi
Codeblock is working on a new FAS release for production deployment just
after the Fedora 18 alpha release. puiterwijk has added a security question
feature to this release that could use some testing. The idea of the
security question is that if you enter it into fas and subsequently lose
both your password and access to the email address you used in FAS (For
instance, if you change jobs) you can ask an admin to verify you via your
security question. The admin will ask you the question you asked in FAS and
then see if the answer you give matches what's recorded in fas.
This is a manual process because we want to allow fuzzy matches for the
answers. To mitigate some of the risk of having another means of verifying
you, we've encrypted the answer to the question with a public key. The
admins have access to the private key and will be decrypting and reading
your question and answer when you need to verify. Please be aware of that
when choosing a question and answer.
For testing purposes, if people would like to add questions and answers to
their accounts on the stg server they can do so here:
https://admin.stg.fedoraproject.org/accounts/user/changequestion
Sometime before pushing to production, I'll be testing that I can
***decrypt all the answers*** that have been entered here so that we don't
end up with a production instance that's saving answers we cannot later
read. Please be aware of that and use values you don't care about if this
concerns you.
If you see any issues with this, found a way to retrieve someone else's
question or answer (and aren't an admin), etc, please bring it up with us so
we can fix the issue.
Thanks for any testing!
-Toshio
11 years, 10 months
Plan for tomorrow's Fedora Infrastructure meeting (2012-08-02)
by Kevin Fenzi
The infrastructure team will be having it's weekly meeting tomorrow,
2012-08-02 at 18:00 UTC in #fedora-meeting on the freenode network.
Note that I will be gone for this meeting and the one following.
Smooge will run the meeting.
Suggested topics:
#topic New folks introductions and Apprentice tasks.
If any new folks want to give a quick one line bio or any apprentices
would like to ask general questions, they can do so here.
#topic Applications status / discussion
Check in on status of our applications: pkgdb, fas, bodhi, koji,
community, voting, tagger, packager, dpsearch, etc.
If there's new releases, bugs we need to work around or things to note.
#topic Sysadmin status / discussion
Can note or talk about sysadmin related tasks or items that happened in
the past week or are going to happen.
#topic Upcoming Tasks/Items
#info 2012-07-30 to 2012-08-03 PHX2 trip for smooge
#info 2012-08-07 to 2012-08-21 F18 Alpha Freeze
#info 2012-08-08 drop inactive apprentices.
#info 2012-08-08 hosted03-> hosted01/02 migration (tenative)
#info 2012-08-21 F18 Alpha release.
#info 2012-08-31 end of 2nd quarter
#info 2012-09-11 to 2012-09-25 F18 Beta Freeze
#info 2012-09-25 F18 Beta release
#topic Open Floor
Submit your agenda items, as tickets in the trac instance and send a
note replying to this thread.
More info here:
https://fedoraproject.org/wiki/Infrastructure/Meetings#Meetings
Thanks
kevin
11 years, 10 months
fedmsg status, production, and help with testing?
by Ralph Bean
I've been working on the Messaging SIG work for fedmsg for a few months now and
have been active in IRC, but silent on this list. Here's a status report.
At this point, I've developed an API for sending and receiving messages and have
patched some existing services for only sending. Nothing *depends* on fedmsg
messages arriving at this point, nor will anything so depend for a long while
out. We want to be able to test and make sure that the fedmsg message bus is
reliable and secure before we do anything like that.
Status
------
In staging, the following services are emitting fedmsg messages.
- bodhi
- fas
- mediawiki
- tagger
- scm/gitolite/fedpkg
Additionally, there is a new service running on app01.stg called "fedmsg-relay".
There are also two new commands: "fedmsg-logger" and "fedmsg-tail".
The messages are all signed by service-host specific RSA keys. Documentation
for the CA and generating new cert/key pairs can be found at
http://infrastructure.fedoraproject.org/infra/docs/fedmsg-certs.txt
Motivation
----------
I'd like to move most of the puppet modules I have from "modules-staging" to
"modules" sooner than later. I'm worried about that directory bloating and
getting out of sync with the main modules directory over the long winter of the
freeze. There's much more work to be done and the freeze offers a good
opportunity to assess and commit to a setup. However, the freeze is set for
next Tuesday, so soon!
lmacken is preparing a bodhi1 release before the freeze which will include the
fedmsg work and relrod is working on a fas release. That leaves mediawiki,
tagger, and scm/gitolite/fedpkg on which we'll need to coordinate a push to
production or decide that they stay in staging for the time being.
For anyone with the time/energy, here is a set of instructions for how
to test what I've set up. I've tested it pretty thoroughly, but peer review is
best.
Grotesque Detail
----------------
For all the tests, login to app0[1-7].stg or packages[1-2].stg and run
"fedmsg-tail". There will be spam from the busmon consumer we have
running in staging, so its probably best to run
"fedmsg-tail | grep -v busmon".
Message signing is turned on globally from puppet in /etc/fedmsg.d/ssl.py.
On a given host (say, app01.stg), check /etc/pki/fedmsg to see if the
permissions on public certs and private keys makes sense.
- fedmsg-logger
- $ echo "this is a test" | fedmsg-logger
- bodhi
- Login to https://admin.stg.fedoraproject.org/updates
- Messages are sent when you:
- Make any change to an update.
- fas
- Login to https://admin.stg.fedoraproject.org/accounts
- Messages are sent when you:
- create a new user
- edit your profile
- apply for a group.
- sponsor someone for a group.
- create a group.
- update a group.
- remove a member from a group.
- mediawiki
- Login to https://stg.fedoraproject.org/wiki/Fedora_Project_Wiki
- Messages are sent when you:
- edit an article
- upload something
- tagger
- Login to https://apps.stg.fedoraproject.org/tagger/
- Messages are sent when you:
- Upvote/downvote a tag
- Add a new tag
- Login
- scm/gitolite/fedpkg
- Change your /etc/rpkg/fedpkg.conf to point to stg.fedoraproject.org.
- Messages are sent when you:
- Push a new commit
Post Freeze Plans
-----------------
Whether or not we can vet and push these into production before the freeze, I
will be working on the following bits afterwards:
- Documentation sprint: Some documentation for developers exists, but it
hasn't had careful attention in a few months. It should include both a
How-To for developers and some discussion of how the fedmsg internals work.
- Start in on patches that add hooks to other services.
http://fedmsg.readthedocs.org/en/latest/status.html is my running list. I
think pkgdb, meetbot, and elections are next.
11 years, 10 months
FW: [S3tools-general] s3cmd moving forward - call for a project leader!
by Matt Domsch
Be careful what you start...
s3tools s3cmd is used by the s3-mirror module.
--
Matt Domsch
Technology Strategist
Dell | Office of the CTO
From: Michael Ludvig [mailto:mludvig@logix.net.nz]
Sent: Tuesday, July 31, 2012 10:18 PM
To: s3tools-general(a)lists.sourceforge.net
Subject: [S3tools-general] s3cmd moving forward - call for a project leader!
On 01/08/12 06:30, Matt Domsch wrote:
Still hoping Michal Ludvig will pull all these changes into his tree.
Hi Matt and everyone who sent me pull requests over the past year or so.
I'm really sorry for not following up and for ... basically ... ignoring your efforts. I know this is not the right way the project is meant to be led.
As it's often the case in the open source world these projects are only a hobby for their founders and so was s3cmd my hobby for over 5 years. Not generating any real income but it was fun and a great learning experience. However after such a long time of working on s3cmd, bringing it to the masses, making it one of the best products in its class (ok, enough bragging :) and seeing the community of users and contributors grow I'm afraid I have now became the bottleneck slowing the project down instead of being the leader pushing it forward. And unfortunately it's not about to improve anytime soon - I now have other commitments, family, less free time overall and can't spend as much time on s3cmd as the project deserves.
But I'm pleased to see the community is doing very well even without me being actively involved and I'm more than happy to pass the leadership hat on to someone new. To someone who is keen to bring new life to the project, invest his time in reviewing the pull requests, responding to emails and releasing new versions.
There is indeed some responsibility in this role - in order to keep s3cmd maintainable in the long term there has to be some vision of where the project goes and quality control of the code being merged. Reviewing patches and working with the contributors is what I actually found to be the most energy consuming task, and that's why there are so many pending pull requests in the queue - I didn't have time to review them and eventually work with their authors to bring them up to the required standard. That's probably the biggest challenge of this role.
On the other hand it is by no means a full time position, a couple of hours a month is usually all it takes. Don't feel obliged to keep the role forever, people come and go, that's normal. However if you could spend some time on s3cmd every now and then over the next year or so that'd be great. Shorter engagements probably don't make much sense - you'll have to lead the crowd for a while, not just merge all the outstanding pull requests and disappear.
Is there anyone on the list who is keen to step in and assume the s3cmd leadership role? Raise your hands! One, two, three leaders / maintainers, not a problem :)
Guys? Girls? Cats? Dogs? Parrots? Matt? Anyone?
Needless to say it's your unique opportunity to gain a worldwide fame by being the main figure in such a prominent project (ok, overstating a bit :)
Hope to hear from you soon!
Michal
11 years, 10 months
