On Thu, Apr 11, 2024 at 5:58 AM Tadej Janež tadej.j@nez.si wrote:
On Wed, 2024-04-10 at 11:22 -0400, Neal Gompa wrote:
On Wed, Apr 10, 2024 at 10:54 AM Tadej Janež tadej.j@nez.si wrote:
On Tue, 2024-04-09 at 10:00 -0400, Neal Gompa wrote:
On Tue, Apr 9, 2024 at 9:53 AM Tadej Janež tadej.j@nez.si wrote:
Hey Fedora Cloud WG!
Firstly, thanks for providing Fedora for the popular cloud providers!
Since Fedora 35, the Fedora Cloud images use btrfs by default [1]. For my deployments, I would like to use ext4 or xfs, so my question are:
- Is it possible to change the root file system at deploy
time? 2) If not, how could one change the cloud images to use a different root file system?
It is not possible to change at deployment time, you would need to build your own custom images.
Ok, thanks for clarifying!
Our images are now defined here: https://pagure.io/fedora-kiwi-descriptions/blob/rawhide/f/teams/cloud/cloud....
If you want to use something else, you'd want to have your own version of the definitions and modify that file to use the filesystem of your choice.
I see, one would need to modify the "filesystem" key of the selected Cloud-Base-<provider> image.
Are there instructions on how to build a custom cloud image?
The repository README includes a quickstart section for building your own images.
However, even if I can easily build a custom cloud image, the overhead of maintaining custom cloud images is very high.
For every Fedora release and cloud provider, I would need to build the image and then with IaC (e.g. Terraform), handle uploading the custom image and using it...
Yeah, it is more overhead, for sure.
Thinking out loud... the modification in my case would not be a custom tailored image for my use case but more similar to how Fedora project builds different spins if people want to use a non-GNOME desktop environment or ALT images for people using alternative architectures.
Is there a possibility to reuse the existing image building infrastructure and add tasks for building images with e.g. ext4 as the rootfs?
I think we could consider adding some overlay samples that show different ways to leverage the descriptions for building custom stuff on top of ours in the pagure.io/fedora-cloud namespace. I know David and I were already talking about doing that after we got done with the F40 cycle.
Is there a particular reason you want to use ext4 or xfs for your rootfs? Typically the pattern we see is that people attach a secondary volume or use S3 and put their data on that instead of the rootfs.
Yes, I use that pattern as well. Usually, /srv or /var would be on a separate block storage and formatted with the file system of choice.
The reasons why I would want to use, e.g. ext4 for the rootfs, would be:
- Familiarity. I've mainly been using ext4 or LVM+ext4 (with LUKS
underneath) for the last 2 decades. I know the tools and I know what "care" such filesystems need.
- Maturity. Ext2/3/4 have been round for quite longer than btrfs
and there are very little "unknowns" or "surprises" with it.
- Simplicity. When provisioning machines with a cloud provider, I
actually don't need the LVM+ext4 combination because the cloud provider would typically handle the things LVM would handle for a non-cloud machine, e.g. increasing the block storage size, snapshotting, ...
Please, don't read this as a critique against btrfs, just me trying to explain why I would find it nicer to just use ext4 for the rootfs as well.
That's totally fair. I will point out that one of the big reasons we use Btrfs is actually to enable space efficiency (which can lead to cost savings across the board). The used storage is roughly 40% less than on ext4 due to leveraging both transparent compression and reflinks.
The other reason is to make it easy to do replication for backups and other purposes.
You may also find it to be beneficial for easily avoiding certain kinds of failure cases that can happen on ext4 (such as inode exhaustion with containers), since btrfs dynamically allocates inodes per subvolume.
You can also grow (and shrink!) the volume live while the system is mounted and online safely.
Thanks for enumerating these benefits. And I agree that these benefits alone justify the reason for switching to btrfs by default.
I might be tempted to check out the btrfs status page again and start playing with it.
Give it a shot, I think you'll be pleasantly surprised! :)