An application to help find cloud images (in multiple clouds)

[Major Hayden]

Hey there,

Nearly all of my Fedora usage is in the cloud (I guess I picked the right SIG to join) and one of my frustrations is finding the right image to use. 🥵

Fedora does make this fairly simple on the cloud page[0], but I see two issues here:

1) Sometimes new images are available in AWS, but the Fedora page points to an older version.

2) Sometimes I want an older version of Fedora (or rawhide!) and that's not listed.

When I want a particular version of Fedora in a container, it's really easy to fire up podman and pull some images from known URLs:

quay.io/fedora/fedora:rawhide
  quay.io/fedora/fedora:37
  quay.io/fedora/fedora:36

I'd love to have some kind of tool that would look up this information and give me the appropriate AMI ID at AWS so I could say "launch Fedora 37 latest" and get it where I want it. Frankly, I'd love this for CentOS, RHEL, and other distros, too. 💙

My team at Red Hat came up with the idea of a locator service that would gather data from upstream locations (AWS, Azure, GCP, and others), compile the image data into a common schema, and make it available for API calls and a web frontend.

CoreOS already does something like this and offers a JSON[1] file with a standardized schema that makes it easier to find the current released in various locations.

Would this type of application be useful for others in the Fedora community? We'd like to know if there would be broader interest here in this idea or if there's something else that makes more sense to pursue.

Thanks! 👋

[0] https://alt.fedoraproject.org/en/cloud/
[1] https://builds.coreos.fedoraproject.org/streams/stable.json

--
Major Hayden

[Colin Walters]

On Tue, Jan 31, 2023, at 9:10 PM, Major Hayden wrote:
> My team at Red Hat came up with the idea of a locator service that 
> would gather data from upstream locations (AWS, Azure, GCP, and 
> others), compile the image data into a common schema, and make it 
> available for API calls and a web frontend.

I hope we could also share a common model with CentOS Stream, which doesn't seem to have any answer at all today?
> CoreOS already does something like this and offers a JSON[1] file with 
> a standardized schema that makes it easier to find the current released 
> in various locations.

Yep, would love to think about how we can share code/approaches here with other Fedora images!

[Duncan, David]

Apologies for the top post. I am stuck in Outlook:

I love the idea of having our own DB for managing this. Is this something similar to what Scott Moser did for the Ubuntu Finder with a JSON stream? Do we need to gather it or should we report it back in deploying the images?

On 2/9/23, 3:31 PM, "Colin Walters" <[e-mail] > wrote:

On Tue, Jan 31, 2023, at 9:10 PM, Major Hayden wrote:
> My team at Red Hat came up with the idea of a locator service that
> would gather data from upstream locations (AWS, Azure, GCP, and
> others), compile the image data into a common schema, and make it
> available for API calls and a web frontend.

I hope we could also share a common model with CentOS Stream, which doesn't seem to have any answer at all today?
> CoreOS already does something like this and offers a JSON[1] file with
> a standardized schema that makes it easier to find the current released
> in various locations.

Yep, would love to think about how we can share code/approaches here with other Fedora images!
_______________________________________________
cloud mailing list -- [e-mail] 
To unsubscribe send an email to [e-mail] 
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/[e-mail] 
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue https://pagure.io/fedora-infrastructure/new_issue

[Major Hayden]

On Thu, Feb 9, 2023, at 22:30, Duncan, David wrote:
> Apologies for the top post. I am stuck in Outlook:

You are forgiven. 😆
> I love the idea of having our own DB for managing this. Is this 
> something similar to what Scott Moser did for the Ubuntu Finder with a 
> JSON stream? Do we need to gather it or should we report it back in 
> deploying the images?

Possibly. We're thinking about a system with two main interfaces:

1) A web frontend for quick access to data or access to people who don't need an API

2) An API that provides structured data, including JSON and data formats for specific provisioning tools such as terraform or ansible

The idea would be to:

1) Gather all image data from cloud providers (in all regions)
2) Store that data
3) Transform that data into a common schema that contains the information that clients would need
4) Serve the structured data from an object storage bucket on AWS, Azure, Backblaze, or some other service

Steps 1-4 would be automated, of course.