urgent(ish): heat-cfntools -> python-boto -> python3 -> sad

[Matthew Miller]

hey all. heat-cfntools requires python-boto, which requires python3,
which shouldn't happen before we're ready to transition to python3
overall.

This has a _significant_ impact on image size now that the python-boto
dep is gone from cloud init. Having _one_ python requirement is bad
enough but it's egregious to pull in both 2 and 3.

Options:

1. does heat-cfntools hard-require python-boto?
2. can we use a python-boto that isn't python3 before we're ready?
   (there was supposed to be a coherent plan for this!)
3. should we drop heat-cfntools for f21?

Agents are a pain. :(

[Steven Hardy]

On Sat, Nov 08, 2014 at 01:39:47PM -0500, Matthew Miller wrote:
> hey all. heat-cfntools requires python-boto, which requires python3,
> which shouldn't happen before we're ready to transition to python3
> overall.

When you say "requires" python3, you're saying that boto now works only
with python3?  I've probably missed some context if so, can you provide any
links to clarify why that's happened?

On the face of it either upstream or the package maintainer forcing python3
vs providing Py3k compatibility seems like a pretty impressively
user-hostile change :(
> This has a _significant_ impact on image size now that the python-boto
> dep is gone from cloud init. Having _one_ python requirement is bad
> enough but it's egregious to pull in both 2 and 3.
>
> Options:
> does heat-cfntools hard-require python-boto?

Right now, yes, but if we have to it can be reworked to break that
requirement.
> can we use a python-boto that isn't python3 before we're ready?
> (there was supposed to be a coherent plan for this!)

Yes, this sounds like the best short-term fix while we work on (1).
> should we drop heat-cfntools for f21?

We'd really prefer it if you didn't, please :)

We (the heat community) are happy to help drive resolution of this, it's
just something of an unexpected issue tbh :(

Historically boto has broken us more regularly than any other dependency
though, so it may not be a bad thing to break that dependency anyway.  I'm
not sure if we can do it in the next 7 days, so planning for (2) sounds
like the best plan right now, if possible.
> Agents are a pain. :(

I appreciate the frustration, but it feels a bit like heat-cfntools is the
messenger here, with boto being the real problem?

Let us know how you'd like to proceed and we'll try to get it sorted for
the change deadline.

Thanks,

Steve

[Matthew Miller]

On Mon, Nov 10, 2014 at 04:34:50PM +0000, Steven Hardy wrote:
> > hey all. heat-cfntools requires python-boto, which requires python3,
> > which shouldn't happen before we're ready to transition to python3
> > overall.
>
> When you say "requires" python3, you're saying that boto now works only
> with python3?  I've probably missed some context if so, can you provide any
> links to clarify why that's happened?

I mean, the package pulls it in. I haven't investigated the details.

Oooh. Hmmmmm maybe this is just a problem which will go away if
python-boto is pulled in _explicitly_ -- I see that python3-boto is
actually a subpackage. Let me look into that more.
> We (the heat community) are happy to help drive resolution of this, it's
> just something of an unexpected issue tbh :(

Yeah. We need to get "dependency explosion detection" into our nightly
checks so we know immediately.
> I appreciate the frustration, but it feels a bit like heat-cfntools is the
> messenger here, with boto being the real problem?

Probably :). Or possibly the real problem is "implicit depsolving
chosing the wrong thing".
> Let us know how you'd like to proceed and we'll try to get it sorted for
> the change deadline.

Thanks Steve!

[Matthew Miller]

On Mon, Nov 10, 2014 at 02:27:11PM -0500, Matthew Miller wrote:
> I mean, the package pulls it in. I haven't investigated the details.
> Oooh. Hmmmmm maybe this is just a problem which will go away if
> python-boto is pulled in _explicitly_ -- I see that python3-boto is
> actually a subpackage. Let me look into that more.

Looks like that's because gholms fixed it, saving the day. :)

[Matthew Miller]

On Mon, Nov 10, 2014 at 02:35:56PM -0500, Matthew Miller wrote:
> > I mean, the package pulls it in. I haven't investigated the details.
> > Oooh. Hmmmmm maybe this is just a problem which will go away if
> > python-boto is pulled in _explicitly_ -- I see that python3-boto is
> > actually a subpackage. Let me look into that more.
>
> Looks like that's because gholms fixed it, saving the day. :)

Ugh except

$ rpm -qRp python-boto-2.34.0-3.fc21.noarch.rpm
/usr/bin/python2
python(abi) = 2.7
python-requests
python-rsa
python3-requests
python3-rsa

$ rpm -qRp python3-boto-2.34.0-3.fc21.noarch.rpm 
python(abi) = 3.4

Looks like the requirements meant for the subpackage are ending up in
the main package. :(

[Matthew Miller]

Okay! Let's get some karma added to
https://admin.fedoraproject.org/updates/FEDORA-2014-14606/python-boto-2.34.0...

[Ryan Brown]

Just talked to gholms, and the python3 dep is a bug and he's removed it
in this build[1].

I'd still be in favor of a longer term plan to drop the boto dep, but we
should probably do more evaluation on what functionality we'd lose. This
could be a candidate for an optional dependency down the road.

[1]: http://koji.fedoraproject.org/koji/buildinfo?buildID=591757

On 11/10/2014 11:34 AM, Steven Hardy wrote:
> On Sat, Nov 08, 2014 at 01:39:47PM -0500, Matthew Miller wrote:
> > hey all. heat-cfntools requires python-boto, which requires python3,
> > which shouldn't happen before we're ready to transition to python3
> > overall.
>
> When you say "requires" python3, you're saying that boto now works only
> with python3?  I've probably missed some context if so, can you provide any
> links to clarify why that's happened?
>
> On the face of it either upstream or the package maintainer forcing python3
> vs providing Py3k compatibility seems like a pretty impressively
> user-hostile change :(
> > This has a _significant_ impact on image size now that the python-boto
> > dep is gone from cloud init. Having _one_ python requirement is bad
> > enough but it's egregious to pull in both 2 and 3.
> >
> > Options:
> > does heat-cfntools hard-require python-boto?
>
> Right now, yes, but if we have to it can be reworked to break that
> requirement.
> > can we use a python-boto that isn't python3 before we're ready?
> > (there was supposed to be a coherent plan for this!)
>
> Yes, this sounds like the best short-term fix while we work on (1).
> > should we drop heat-cfntools for f21?
>
> We'd really prefer it if you didn't, please :)
>
> We (the heat community) are happy to help drive resolution of this, it's
> just something of an unexpected issue tbh :(
>
> Historically boto has broken us more regularly than any other dependency
> though, so it may not be a bad thing to break that dependency anyway.  I'm
> not sure if we can do it in the next 7 days, so planning for (2) sounds
> like the best plan right now, if possible.
> > Agents are a pain. :(
>
> I appreciate the frustration, but it feels a bit like heat-cfntools is the
> messenger here, with boto being the real problem?
>
> Let us know how you'd like to proceed and we'll try to get it sorted for
> the change deadline.
>
> Thanks,
>
> Steve
> _______________________________________________
> cloud mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/cloud
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct