Handling packages with missing dependencies provided by HA-RS repos?

[Diego Herrera]

I've been checking the packages that won't install on EPEL [1] and found out that drbd-pacemaker cant get installed
because of a missing dependency (pacemaker). While researching why, I saw that pacemaker exists on EPEL7 because it's 
provided by the HighAvailability repo, but by policy [2] that repo is not a base for EPEL8 nor EPEL9.

When I asked on how to handle this cases on the steering meeting, some proposed ideas were:

* Rebuild the dependencies as -epel
* Retire the packages 
* Bringing back HA & RS repo

The only other package that i've found also has this problem is resalloc-aws that depends on awscli.

Is there a policy on this cases? Are EPEL packages allowed to require packages outside of the policy approved?
I would like more feedback on how to proceed so we can file bugs for this packages correctly.

Package: drbd-pacemaker-9.20.2-1.el9
Error: Problem: conflicting requests - nothing provides pacemaker needed by drbd-pacemaker-9.20.2-1.el9.x86_64

Package: resalloc-aws-1.1-1.el9
Error: Problem: conflicting requests - nothing provides awscli needed by resalloc-aws-1.1-1.el9.noarch

Package: drbd-pacemaker-9.17.0-1.el8
Error: Problem: conflicting requests - nothing provides pacemaker needed by drbd-pacemaker-9.17.0-1.el8.x86_64

[1] https://tdawson.fedorapeople.org/epel/willit/epel8/status-wont-install.html
[2] https://docs.fedoraproject.org/en-US/epel/epel-policy/#_policy

[Troy Dawson]

On Thu, Mar 10, 2022 at 2:17 PM Diego Herrera  wrote:
> I've been checking the packages that won't install on EPEL [1] and found
> out that drbd-pacemaker cant get installed
> because of a missing dependency (pacemaker). While researching why, I saw
> that pacemaker exists on EPEL7 because it's
> provided by the HighAvailability repo, but by policy [2] that repo is not
> a base for EPEL8 nor EPEL9.
>
> When I asked on how to handle this cases on the steering meeting, some
> proposed ideas were:
> Rebuild the dependencies as -epel
> Retire the packages
> Bringing back HA & RS repo
>
> The only other package that i've found also has this problem is
> resalloc-aws that depends on awscli.
>
> Is there a policy on this cases? Are EPEL packages allowed to require
> packages outside of the policy approved?
> I would like more feedback on how to proceed so we can file bugs for this
> packages correctly.
>
> Package: drbd-pacemaker-9.20.2-1.el9
> Error: Problem: conflicting requests - nothing provides pacemaker needed
> by drbd-pacemaker-9.20.2-1.el9.x86_64
>
> Package: resalloc-aws-1.1-1.el9
> Error: Problem: conflicting requests - nothing provides awscli needed by
> resalloc-aws-1.1-1.el9.noarch
>
> Package: drbd-pacemaker-9.17.0-1.el8
> Error: Problem: conflicting requests - nothing provides pacemaker needed
> by drbd-pacemaker-9.17.0-1.el8.x86_64
>
> [1]
> https://tdawson.fedorapeople.org/epel/willit/epel8/status-wont-install.html
> [2] https://docs.fedoraproject.org/en-US/epel/epel-policy/#_policy

Someone can tell me I'm wrong, but I believe if something is in HA or RS in
RHEL8 or 9 it is fair game for EPEL8 or EPEL9.
Those packages are currently not being excluded when you request a package
for epel8 or epel9.

So, my opinion, build pacemaker in EPEL.

Troy

[Carl George]

On Mon, Mar 14, 2022 at 4:23 PM Troy Dawson  wrote:
> On Thu, Mar 10, 2022 at 2:17 PM Diego Herrera  wrote:
> > I've been checking the packages that won't install on EPEL [1] and found out that drbd-pacemaker cant get installed
> > because of a missing dependency (pacemaker). While researching why, I saw that pacemaker exists on EPEL7 because it's
> > provided by the HighAvailability repo, but by policy [2] that repo is not a base for EPEL8 nor EPEL9.
> >
> > When I asked on how to handle this cases on the steering meeting, some proposed ideas were:
> > Rebuild the dependencies as -epel
> > Retire the packages
> > Bringing back HA & RS repo
> >
> > The only other package that i've found also has this problem is resalloc-aws that depends on awscli.
> >
> > Is there a policy on this cases? Are EPEL packages allowed to require packages outside of the policy approved?
> > I would like more feedback on how to proceed so we can file bugs for this packages correctly.
> >
> > Package: drbd-pacemaker-9.20.2-1.el9
> > Error: Problem: conflicting requests - nothing provides pacemaker needed by drbd-pacemaker-9.20.2-1.el9.x86_64
> >
> > Package: resalloc-aws-1.1-1.el9
> > Error: Problem: conflicting requests - nothing provides awscli needed by resalloc-aws-1.1-1.el9.noarch
> >
> > Package: drbd-pacemaker-9.17.0-1.el8
> > Error: Problem: conflicting requests - nothing provides pacemaker needed by drbd-pacemaker-9.17.0-1.el8.x86_64
> >
> > [1] https://tdawson.fedorapeople.org/epel/willit/epel8/status-wont-install.html
> > [2] https://docs.fedoraproject.org/en-US/epel/epel-policy/#_policy
>
> Someone can tell me I'm wrong, but I believe if something is in HA or RS in RHEL8 or 9 it is fair game for EPEL8 or EPEL9.
> Those packages are currently not being excluded when you request a package for epel8 or epel9.
>
> So, my opinion, build pacemaker in EPEL.
>
> Troy
>
> epel-devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]...
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

Typically EPEL inherits policy from Fedora, diverging when necessary.
Here is the corresponding section of Fedora policy.

"All package dependencies (build-time or runtime, regular, weak or
otherwise) MUST ALWAYS be satisfiable within the official Fedora
repositories."

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_package_dependen...

We don't consider HA or RS part of the base RHEL distribution
(referred to in policy as the "Target Base").  However, I don't think
we should strictly forbid any dependency on HA or RS packages, because
that would require unnecessary duplication of HA/RS packages in EPEL
(which is allowed, but shouldn't be required IMO).  I suggest a
compromise that we can make the policy:

"All EPEL package dependencies (build-time or runtime) MUST ALWAYS be
satisfiable within the Target Base or EPEL itself.  Weak package
dependencies are allowed on packages from additional RHEL channels
that are not part of the Target Base, such as the HighAvailability
channel."

[Carl George]

On Wed, Mar 23, 2022 at 2:54 PM Carl George  wrote:
> On Mon, Mar 14, 2022 at 4:23 PM Troy Dawson  wrote:
> > On Thu, Mar 10, 2022 at 2:17 PM Diego Herrera  wrote:
> > > I've been checking the packages that won't install on EPEL [1] and found out that drbd-pacemaker cant get installed
> > > because of a missing dependency (pacemaker). While researching why, I saw that pacemaker exists on EPEL7 because it's
> > > provided by the HighAvailability repo, but by policy [2] that repo is not a base for EPEL8 nor EPEL9.
> > >
> > > When I asked on how to handle this cases on the steering meeting, some proposed ideas were:
> > > Rebuild the dependencies as -epel
> > > Retire the packages
> > > Bringing back HA & RS repo
> > >
> > > The only other package that i've found also has this problem is resalloc-aws that depends on awscli.
> > >
> > > Is there a policy on this cases? Are EPEL packages allowed to require packages outside of the policy approved?
> > > I would like more feedback on how to proceed so we can file bugs for this packages correctly.
> > >
> > > Package: drbd-pacemaker-9.20.2-1.el9
> > > Error: Problem: conflicting requests - nothing provides pacemaker needed by drbd-pacemaker-9.20.2-1.el9.x86_64
> > >
> > > Package: resalloc-aws-1.1-1.el9
> > > Error: Problem: conflicting requests - nothing provides awscli needed by resalloc-aws-1.1-1.el9.noarch
> > >
> > > Package: drbd-pacemaker-9.17.0-1.el8
> > > Error: Problem: conflicting requests - nothing provides pacemaker needed by drbd-pacemaker-9.17.0-1.el8.x86_64
> > >
> > > [1] https://tdawson.fedorapeople.org/epel/willit/epel8/status-wont-install.html
> > > [2] https://docs.fedoraproject.org/en-US/epel/epel-policy/#_policy
> >
> > Someone can tell me I'm wrong, but I believe if something is in HA or RS in RHEL8 or 9 it is fair game for EPEL8 or EPEL9.
> > Those packages are currently not being excluded when you request a package for epel8 or epel9.
> >
> > So, my opinion, build pacemaker in EPEL.
> >
> > Troy
> >
> > epel-devel mailing list -- [e-mail]
> > To unsubscribe send an email to [e-mail]
> > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]...
> > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>
> Typically EPEL inherits policy from Fedora, diverging when necessary.
> Here is the corresponding section of Fedora policy.
>
> "All package dependencies (build-time or runtime, regular, weak or
> otherwise) MUST ALWAYS be satisfiable within the official Fedora
> repositories."
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#_package_dependen...
>
> We don't consider HA or RS part of the base RHEL distribution
> (referred to in policy as the "Target Base").  However, I don't think
> we should strictly forbid any dependency on HA or RS packages, because
> that would require unnecessary duplication of HA/RS packages in EPEL
> (which is allowed, but shouldn't be required IMO).  I suggest a
> compromise that we can make the policy:
>
> "All EPEL package dependencies (build-time or runtime) MUST ALWAYS be
> satisfiable within the Target Base or EPEL itself.  Weak package
> dependencies are allowed on packages from additional RHEL channels
> that are not part of the Target Base, such as the HighAvailability
> channel."
>
> --
> Carl George

We discussed this a bit further at today's EPEL Steering Committee.
One alternative that was suggested was to just add the HA and RS repos
to the target base list.  The initial impact of that would be that
several packages already in EPEL8 would become policy violations and
would have to be retired.

HA8 package
EPEL8 package

awscli-1.14.50-5.el8
awscli-1.18.156-1.el8

google-api-python-client-1.6.5-3.el8
google-api-python-client-1.6.7-10.el8

python-boto3-1.6.1-2.el8
python-boto3-1.15.15-1.el8

python-botocore-1.9.1-2.el8
python-botocore-1.18.15-1.el8

python-fasteners-0.14.1-14.el8
python-fasteners-0.14.1-20.el8

python-oauth2client-4.1.2-6.el8
python-oauth2client-4.1.3-9.el8

python-s3transfer-0.1.13-1.el8
python-s3transfer-0.3.4-1.el8

python-uritemplate-3.0.0-3.el8
python-uritemplate-3.0.0-10.el8

Two of these are the same version (but higher release) than what's in
HA/RS.  The other six are newer versions, so retiring them in favor of
the HA/RS package would result in version downgrades.  All eight will
require users do a distrosync instead of an upgrade to switch to a
maintained package.  Further complicating the matter, four of the
eight packages are x86_64 only.  Retiring them will create the need
for <package>-epel variants for the missing architectures.  RS is
mostly the same content as HA, with some additional packages that
currently do not overlap with EPEL, but do note that RS is not
available for aarch64, so EPEL packages would need to `ExcludeArch:
aarch64` if they need to depend on those additional packages (cmirror
and dlm in RS8; ctdb, dlm, and gfs2-utils in RS9).

Another aspect to consider is that allowing non-weak dependencies on
channels that are disabled by default results in a bad user
experience.  We already see this happen dependencies in CRB.  Adding
more channels this can happen with would make the user experience
worse.  In addition to being disabled by default, not all RHEL
subscriptions include access to these channels.

[Felix Schwarz]

Am 24.03.22 um 03:18 schrieb Carl George:
> python-boto3-1.6.1-2.el8
> python-boto3-1.15.15-1.el8
>
> python-botocore-1.9.1-2.el8
> python-botocore-1.18.15-1.el8
>
> python-fasteners-0.14.1-14.el8
> python-fasteners-0.14.1-20.el8
>
> python-oauth2client-4.1.2-6.el8
> python-oauth2client-4.1.3-9.el8
>
> python-s3transfer-0.1.13-1.el8
> python-s3transfer-0.3.4-1.el8

That might have implications for the "certbot" package - there is at least one 
plugin which requires that stack. *If* the RHEL packages have similar versions 
and provide Python 3 modules it might not be a problem (we had some bad 
experience with Red Hat's RPMs in EPEL 7/Python 2).

certbot + plugins provide a comprehensive test suite so as a test one could do a 
test build with the HA repos. If the test suites pass everything should be fine.

Unfortunately I'm pretty busy atm.

Felix

[Carl George]

On Thu, Mar 24, 2022 at 3:46 AM Felix Schwarz
 wrote:
> Am 24.03.22 um 03:18 schrieb Carl George:
> > python-boto3-1.6.1-2.el8
> > python-boto3-1.15.15-1.el8
> >
> > python-botocore-1.9.1-2.el8
> > python-botocore-1.18.15-1.el8
> >
> > python-fasteners-0.14.1-14.el8
> > python-fasteners-0.14.1-20.el8
> >
> > python-oauth2client-4.1.2-6.el8
> > python-oauth2client-4.1.3-9.el8
> >
> > python-s3transfer-0.1.13-1.el8
> > python-s3transfer-0.3.4-1.el8
>
> That might have implications for the "certbot" package - there is at least one
> plugin which requires that stack. *If* the RHEL packages have similar versions
> and provide Python 3 modules it might not be a problem (we had some bad
> experience with Red Hat's RPMs in EPEL 7/Python 2).
>
> certbot + plugins provide a comprehensive test suite so as a test one could do a
> test build with the HA repos. If the test suites pass everything should be fine.
>
> Unfortunately I'm pretty busy atm.
>
> Felix

Thanks for pointing these out Felix.  I found
python-certbot-dns-google and python-certbot-dns-route53 that require
some of these packages.  I did local mock builds using a custom one
off config comprised of RHEL8, RHEL8 HA, and EPEL8 with the above
packages excluded.  The %check sections of both certbot plugins passed
(26 and 17 tests respectively).  I don't know if that is sufficient to
declare that the dependency version downgrade will have no impact on
those plugins, but at least the test suites pass.

I'm happy to repeat this step with any other EPEL packages people are
aware of that require these packages.

[Felix Schwarz]

Am 24.03.22 um 19:52 schrieb Carl George:
> I found python-certbot-dns-google and python-certbot-dns-route53 that require
> some of these packages.  I did local mock builds using a custom one off
> config comprised of RHEL8, RHEL8 HA, and EPEL8 with the above packages 
> excluded.  The %check sections of both certbot plugins passed (26 and 17 
> tests respectively).  I don't know if that is sufficient to declare that the 
> dependency version downgrade will have no impact on those plugins, but at 
> least the test suites pass.

In general certbot is pretty leniant in terms of requirements so slightly older
versions should work fine.

Also these plugins don't have a lot of users so impact should be limited. Worst 
case upstream is usually pretty responsive to help us out when we need advice 
about relaxed version requirements.

Thank you for testing the plugins, Carl.

Felix

[Kevin Fenzi]

On Wed, Mar 23, 2022 at 09:18:08PM -0500, Carl George wrote:
> On Wed, Mar 23, 2022 at 2:54 PM Carl George  wrote:
> > Typically EPEL inherits policy from Fedora, diverging when necessary.
> > Here is the corresponding section of Fedora policy.
> >
> > "All package dependencies (build-time or runtime, regular, weak or
> > otherwise) MUST ALWAYS be satisfiable within the official Fedora
> > repositories."
> >
> > https://docs.fedoraproject.org/en-US/packaging-guidelines/#_package_dependen...
> >
> > We don't consider HA or RS part of the base RHEL distribution
> > (referred to in policy as the "Target Base").  However, I don't think

Well, for 8 and 9... for 7 we do. ;)
> > we should strictly forbid any dependency on HA or RS packages, because
> > that would require unnecessary duplication of HA/RS packages in EPEL
> > (which is allowed, but shouldn't be required IMO).  I suggest a
> > compromise that we can make the policy:
> >
> > "All EPEL package dependencies (build-time or runtime) MUST ALWAYS be
> > satisfiable within the Target Base or EPEL itself.  Weak package
> > dependencies are allowed on packages from additional RHEL channels
> > that are not part of the Target Base, such as the HighAvailability
> > channel."
> >
> > --
> > Carl George
>
> We discussed this a bit further at today's EPEL Steering Committee.
> One alternative that was suggested was to just add the HA and RS repos
> to the target base list.  The initial impact of that would be that
> several packages already in EPEL8 would become policy violations and
> would have to be retired.

Yeah, I guess thats pretty anoying in 8 since we didn't start with them.
;(

So, if we did allow weak deps to packages in other non our Base repos,
wouldn't that not actually work for the case that started this thread?

ie, say I have a foo-plugin package and foo is in a different non epel
base rhel channel and I add a Reccomends for it in epel. People who have
the channel enabled would be fine but if someone else installed
foo-plugin it would just... not work.

Also could we tell if deps changed? Say I have foo-plugin in epel
Reccommending foo, and RHEL drops foo. None of our 'will it install' or
broken deps type checks will know that it is now not working. ;(

If we don't add HA and RS to the base epel repos, I guess we could just
allow people to build those things they need in epel, but then of course
they get to maintain them. ;(

Perhaps instead of a strict rule we could just ask everything that has
this issue to get an exception?

Not an easy case.

kevin

[Carl George]

On Thu, Mar 24, 2022 at 5:50 PM Kevin Fenzi  wrote:
> On Wed, Mar 23, 2022 at 09:18:08PM -0500, Carl George wrote:
> > On Wed, Mar 23, 2022 at 2:54 PM Carl George  wrote:
> > > Typically EPEL inherits policy from Fedora, diverging when necessary.
> > > Here is the corresponding section of Fedora policy.
> > >
> > > "All package dependencies (build-time or runtime, regular, weak or
> > > otherwise) MUST ALWAYS be satisfiable within the official Fedora
> > > repositories."
> > >
> > > https://docs.fedoraproject.org/en-US/packaging-guidelines/#_package_dependen...
> > >
> > > We don't consider HA or RS part of the base RHEL distribution
> > > (referred to in policy as the "Target Base").  However, I don't think
>
> Well, for 8 and 9... for 7 we do. ;)
> > > we should strictly forbid any dependency on HA or RS packages, because
> > > that would require unnecessary duplication of HA/RS packages in EPEL
> > > (which is allowed, but shouldn't be required IMO).  I suggest a
> > > compromise that we can make the policy:
> > >
> > > "All EPEL package dependencies (build-time or runtime) MUST ALWAYS be
> > > satisfiable within the Target Base or EPEL itself.  Weak package
> > > dependencies are allowed on packages from additional RHEL channels
> > > that are not part of the Target Base, such as the HighAvailability
> > > channel."
> > >
> > > --
> > > Carl George
> >
> > We discussed this a bit further at today's EPEL Steering Committee.
> > One alternative that was suggested was to just add the HA and RS repos
> > to the target base list.  The initial impact of that would be that
> > several packages already in EPEL8 would become policy violations and
> > would have to be retired.
>
> Yeah, I guess thats pretty anoying in 8 since we didn't start with them.
> ;(
>
> So, if we did allow weak deps to packages in other non our Base repos,
> wouldn't that not actually work for the case that started this thread?
>
> ie, say I have a foo-plugin package and foo is in a different non epel
> base rhel channel and I add a Reccomends for it in epel. People who have
> the channel enabled would be fine but if someone else installed
> foo-plugin it would just... not work.

What I suggested as policy would be to allow weak dependencies on
non-base channels, not to incorrectly identify all hard dependencies
as weak if they're in these channels.  If an EPEL package has a hard
dependency, that dependency should be in the target base or EPEL
itself.  If the dependency actually is weak (optional), it would be
useful to allow those to be provided by non-target-base channels.

The plugin example is a bit of a grey area.  If the software is
designed as such that no one uses the plugin directly, but rather the
plugin extends the functionality of the main software which is used
directly, then I think users can figure out they need to install both
the main software and the plugin, regardless of the dependency
relationship.  This will vary case by case, and we could allowed these
by Steering Committee exception only.
> Also could we tell if deps changed? Say I have foo-plugin in epel
> Reccommending foo, and RHEL drops foo. None of our 'will it install' or
> broken deps type checks will know that it is now not working. ;(

As far as I know RHEL doesn't really drop packages, they stay on the
CDN for the life of distro.  Even if they do get dropped, this seems
like an edge case we shouldn't really need to worry too much about.
If it happens and it results in an EPEL package not working, we'll
know it should have been a Requires and not a Recommends all along,
which will lead to either the maintainer adding the necessary
dependency to EPEL, or retiring the package with the missing
dependency.
> If we don't add HA and RS to the base epel repos, I guess we could just
> allow people to build those things they need in epel, but then of course
> they get to maintain them. ;(

This is already allowed, which is why we have EPEL packages that would
need to be retired if HA/RS are added to the target base.
> Perhaps instead of a strict rule we could just ask everything that has
> this issue to get an exception?

As stated above I'd be fine with an exception workflow if a maintainer
feels it's justified to intentionally mislabel a hard dependency as
weak, but the general case of truly weak (optional) dependencies from
non-target-base channels shouldn't need exceptions IMO.
> Not an easy case.
>
> kevin
> _______________________________________________
> epel-devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]...
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[Stephen John Smoogen]

On Fri, 25 Mar 2022 at 00:25, Carl George  wrote:
> On Thu, Mar 24, 2022 at 5:50 PM Kevin Fenzi  wrote:
> > On Wed, Mar 23, 2022 at 09:18:08PM -0500, Carl George wrote:
> > > On Wed, Mar 23, 2022 at 2:54 PM Carl George  wrote:
> > Also could we tell if deps changed? Say I have foo-plugin in epel
> > Reccommending foo, and RHEL drops foo. None of our 'will it install' or
> > broken deps type checks will know that it is now not working. ;(
>
> As far as I know RHEL doesn't really drop packages, they stay on the
> CDN for the life of distro.  Even if they do get dropped, this seems
> like an edge case we shouldn't really need to worry too much about.
> If it happens and it results in an EPEL package not working, we'll
> know it should have been a Requires and not a Recommends all along,
> which will lead to either the maintainer adding the necessary
> dependency to EPEL, or retiring the package with the missing
> dependency.

RHEL doesn't drop any packages from the CDN without a major item. Various
packages which have already gone past their 'Appstream lifecycle' in 8 are
still there. The problem was one of two places.  One, the CentOS rebuild
which only kept the latest in their dot releases. Two, the scripts Fedora
used to reposync from the mirrors usually got only the latest from a dot
branch which would drop certain packages when they were 'removed'. The
second was fixed when we no longer synced from /X.Y/ but only with /X/,
this then keeps the older packages. [I had switched to syncing X.Y so we
would be better able to deal with dot releases breaking CentOS users but
that was seen as breaking koji in other ways so we went back to the X
method.]