Docker Host Image: Requirements?
by Sandro "red" Mathys
Heads-up: I've claimed ownership over the Docker Host Image Change:
https://fedoraproject.org/wiki/Cloud_Changelist#Change:_Docker_Host_Image
Now, while I'm very interested in Docker, I've never seriously /
productively used it nor ever used it to actually implement PaaS on
top of IaaS. Sp, i'm pretty sure some people out there have more
experience with Docker than I do and therefore I'd value your input.
To me it currently seems there's only a very few, very small changes
to add to the cloud base image, i.e.:
1) add docker-io to %packages
2) systemctl enable docker.service
Since we try to compete with CoreOS, do we also want to add etcd out
of the box? That'd double the necessary changes, though! ;)
3) add etcd to %packages
4) systemctl enable etcd.service
What else would you expect from the Docker Host Image? This is way too
easy, so kindly tell me what obvious things I'm clearly missing. Apart
from "you need to actually make sure an image is built and working"...
:)
Thanks,
Sandro
10 years, 2 months
Preparing Change Proposal: Modular Kernel Packaging for Cloud - Feedback?
by Sandro "red" Mathys
In the past ~24h, I've been preparing the "Modular Kernel Packaging
for Cloud" change. Before I submit it to the wrangler, I'm looking for
everyone's feedback. Note this is my first change proposal so I might
have misunderstood things or whatever.
https://fedoraproject.org/wiki/Changes/Modular_Kernel_Packaging_for_Cloud
Note, that I haven't yet reached out to the Anaconda team (regarding
the possibility to install kernel-core instead of kernel) but will do
so now. They don't seem critical to the change, though but just to the
way we're implementing it when creating the images. (Assuming we will
use Anaconda to build F21 images).
Thanks,
Sandro
10 years, 2 months
Re: Modular Kernel Packaging for Cloud
by Josh Boyer
On Thu, Mar 6, 2014 at 9:57 AM, Don Zickus <dzickus(a)redhat.com> wrote:
> On Thu, Mar 06, 2014 at 08:16:00AM +0900, Sandro "red" Mathys wrote:
>> That's the point, we want a reasonably small package while still
>> providing the required functionality. Not sure how providing a fixed
>> size number is helping in this. But most of all, I didn't throw in a
>> number because I have no idea what is reasonably possible. I really
>> only just said "every MB counts" because the question came up before
>> (in Josh's old thread) and I hoped I could stop this discussion from
>> happening again before we have any numbers for this.
>
> Ever work in the embedded space? Every MB counts there too. :-) This was
> solved by creating budgets for size and memory requirements. This helped
> control bloat, which is going to be your biggest problem with cloud
> deployment.
>
> What concerns me is that you don't know what size your cloud deployment is
> but expect everyone to just chop chop chop. How do we know if the kernel
> is already at the right size?
>
> There is s a huge difference between re-architecting the kernel packaging
> to save 1 or 2 MB (off ~143 MB size currently) vs. re-architecting to save
> 50 MB. The former is really a wasted exercise in the bigger picture,
> wherease the latter (if proven needed) accomplishes something.
>
> But again it comes down to understanding your environment. Understanding
> your environment revovles around control. I get the impression you are
> not sure what size your environment should be.
>
> So I was proposing the kernel stay put or maybe create _one_ extras
> package that gets installed in addition to the bzImage. But from the
Right. When I said I had kernel-core and kernel-drivers, I wasn't
being theoretical. I already did the work in the spec file to split
it into kernel-core and kernel-drivers. The kernel package becomes a
metapackage that requires the other two, so that existing installs and
anaconda don't have to change (assuming I did thinks correctly).
Cloud can just specify kernel-core in the kickstart or whatever.
> sound of it, the chopping is really going to get you savings of about
> ~30MB or so.
I spent some time yesterday hacking around on an existing VM and just
removing stuff from /lib/modules/ for an installed kernel. I was able
to get it down from 123MB to 58MB by axing entire subsystems that
clearly didn't apply and running depmod on the results to make sure
there weren't missing dependencies. Some stuff had to be added back
(want virtio_scsi? you need target and libfc), but a lot could be
removed. That brings the total to about 81MB for vmlinuz, initramfs,
and /lib/modules for that particular kernel.
In those 81MB, I still had all of the main GPU drivers, all of the
intel and a few other ethernet drivers, ext4, xfs, btrfs, nfs, the
vast majority of the networking modules (so iptables and netfilter),
scsi, acpi, block, char, etc. The major things missing were bluetooth
and wireless, infiniband, some firewire stuff. Basically it resulted
in a system that boots perfectly fine in a VM for a variety of
different use cases.
I think that's a reasonable start, and it's a significant reduction.
Beyond that, we get into much less reduced savings and having to move
stuff around on a finer level. For the curious, I uploaded the module
list here:
http://jwboyer.fedorapeople.org/pub/modules.small
Again, this was just hacking around on an installed system. Still
work to do at a packaging level, but this is as good as anything to
start from.
josh
10 years, 2 months
systemd-networkd
by Janez Nemanič
Hi,
I decided to contribute to Fedora Cloud. On the Cloud ToDo list this seems
very interesting:
investigate systemd-networkd
- What: new service to do very basic network config
- Where: remove the sysv network scripts, add config for this
- Why: remove very kludgey shell scripts; try the new hotness
- When: Before F21 alpha
- Who: ???
Is it okey if I start working on this? Or is somebody else already working
on this?
--
Janez Nemanic
Lep pozdrav
Best regards
10 years, 2 months
Minutes from today's (13 March 2014) Cloud WG Meeting
by Joe Brockmeier
=======================
#fedora-meeting Meeting
=======================
Meeting started by jzb at 14:06:54 UTC. The full logs are available at
http://meetbot.fedoraproject.org/fedora-meeting/2014-03-13/fedora_cloud_w...
.
Meeting summary
---------------
* possible cloud-init replacements (jzb, 14:08:42)
* ACTION: mattdm to discuss specific heat requirements with sdake
(mattdm, 14:10:57)
*
http://www.slideshare.net/dotCloud/open-stack-heat-docker-unconference-1
(mattdm, 14:15:03)
* Getting the community more involved in the Cloud efforts (jzb,
14:27:11)
* meeting minutes not consumable for outsiders (mattdm, 14:33:07)
* HELP: more blog posts would be great; meeting summaries are a good
start (mattdm, 14:33:20)
* ACTION: jzb to get red_trela files for existing Cloud Brochure
(jzb, 14:44:22)
* ACTION: frankieonuonga to organize something about podcasts etc. on
mailing list (mattdm, 14:47:01)
* New Business (jzb, 14:51:48)
* HELP: changes on the list at
https://fedoraproject.org/wiki/Cloud_Changelist which don't have
owners need owners (mattdm, 14:53:23)
Meeting ended at 14:56:00 UTC.
Action Items
------------
* mattdm to discuss specific heat requirements with sdake
* jzb to get red_trela files for existing Cloud Brochure
* frankieonuonga to organize something about podcasts etc. on mailing
list
Action Items, by person
-----------------------
* frankieonuonga
* frankieonuonga to organize something about podcasts etc. on mailing
list
* jzb
* jzb to get red_trela files for existing Cloud Brochure
* mattdm
* mattdm to discuss specific heat requirements with sdake
* red_trela
* jzb to get red_trela files for existing Cloud Brochure
* **UNASSIGNED**
* (none)
People Present (lines said)
---------------------------
* jzb (66)
* mattdm (58)
* frankieonuonga (39)
* red_trela (18)
* number80 (12)
* zodbot (5)
* walters (5)
* roshi (4)
* gnokii (1)
* geppetto (1)
* samkottler (0)
Generated by `MeetBot`_ 0.1.4
.. _`MeetBot`: http://wiki.debian.org/MeetBot
--
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb | http://dissociatedpress.net/
10 years, 2 months
Modular Kernel Packaging for Cloud
by Sandro "red" Mathys
Heads-up: I've taken ownership over the Cloud SIG's planned
"Cloud-Friendly Kernel Packaging" (aka "Modular Kernel Packaging for
Cloud") Change [0].
(Sorry for cross-posting this on the kernel and cloud lists, but both
the kernel team and the cloud SIG need to be involved in this
discussion).
Obviously, we wish for modular kernel packaging and Josh Boyer already
reached out to the cloud SIG in this regard a while ago when we didn't
really know what we want yet [1]. We do now know that somewhat better
[2] but details could still require additional discussion. Our
motivation is to save space, time and bandwidth (the latter two
referring to required updates) as in the cloud, all three equal to
costs. And yes, we also target things other than the kernel to save
space.
So, in our case hardware drivers are rather unnecessary and the kernel
experts might know other ways to shrink the footprint for our limited
use cases. The kernel we require supports all primary architectures
(i686 and x86_64 right now, ARM likely later) and is able to run on a
hypervisor (primarily KVM and Xen but support for ESXi and Hyper-V are
becoming crucial in private clouds). Only. No bare-metal. We also
require support for Linux Containers (LXC) to enable the use of
Docker.
Now, I heard some people screaming when I said HW drivers are
unnecessary. Some people will want to make use of PCI passthrough and
while I think we don't want to ship the necessary modules by default,
they should be easily installable through a separate package. If some
more granularity is acceptable (and makes sense), I think one package
for SRIOV NICs, one for graphic cards (to enable mathematical
computation stuff) and one for everything else PCI passthrough (plus
one for all other HW drivers for the non-cloud products) would be
totally nice.
What does everybody think about this? Can it be done? How is it best
done? What's the timeframe (we'd really like to see this implemented
in F21 Beta but obviously the earlier modular kernels can be tested,
the better)? Do you require additional input?
Please try to keep the discussion as simple as possible (in terms of
us non-kernel hackers should be able to follow it). Just like Josh
requested when reaching out to us: "Explain it to me like I'm a
child."
Thanks,
Sandro
[0] https://fedoraproject.org/wiki/Cloud_Changelist#Change:_Cloud-Friendly_Ke...
[1] https://lists.fedoraproject.org/pipermail/cloud/2013-October/002862.html
[2] https://fedoraproject.org/wiki/Cloud_PRD
10 years, 2 months
kernel split up
by Josh Boyer
OK, initial builds of the packaging split I talked about last week are now at:
http://copr-fe.cloud.fedoraproject.org/coprs/jwboyer/kernel-core/
Feel free to poke around with the build from today in some VMs you
don't care about. I have installed the x86_64 kernel-core package on
a local VM and booted successfully. The split is roughly what I
talked about last week as well:
[jwboyer@vader kernel]$ rpm -qpi
x86_64/kernel-core-3.14.0-0.rc6.git2.4.fc21.x86_64.rpm | grep Size
Size : 77220258
[jwboyer@vader kernel]$ rpm -qpi
x86_64/kernel-drivers-3.14.0-0.rc6.git2.4.fc21.x86_64.rpm | grep Size
Size : 65418445
So ~74MB for kernel-core installed with a 17MB RPM size, and ~63MB for
kernel-drivers installed with a 16MB RPM size.
I'll be doing some more local testing of various things over the next
couple of days. Once I see that they aren't totally broken, I'll post
the patches to the kernel list.
josh
10 years, 2 months
Reminder + Agenda for Next WG Meeting (new time) on 13 March 2014
by Joe Brockmeier
Hi all,
As promised, here's the agenda and info:
Meeting time: 14:00 UTC
To convert UTC to your local time, take a look at
http://fedoraproject.org/wiki/UTCHowto
Or run:
date -d '2014-03-13 14:00 UTC'
Meeting channel: #fedora-meeting
= Agenda =
== Old Topics ==
* Status of Cantas board for tracking?
== New Topics ==
* Discussion of replacing cloud-init with min-metadata-service as
default or other alternatives
* Also consider CoreOS cloud-init
* Do we want to (try and) kick out Python on the Docker Host image?
This would *require* both replacing cloud-init by min-metadata-service
and yum by ostree.
* Do we want the Docker Host image to use ostree (as kinda yum/dnf
replacement)?
* How can we get the community more involved in our efforts? Changes
need owners.
* Open Floor
Best,
jzb
--
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb | http://dissociatedpress.net/
10 years, 2 months
