Hi,
While looking into a package upgrade I came across pynose [1]. It appears to be a drop-in replacement for nose [2], which has been deprecated in Fedora 32 [3].
There are still quite a few packages depending on nose, that have not yet moved away from it. The official successor to nose is nose2 [4]. But that package does not support all of the behavior of nose.
Therefore, I'm thinking of introducing pynose as a drop in replacement of deprecated nose. Pynose uses the same namespace as nose, but provides python3dist(pynose). Thus adding Provides: for nose would make it a drop-in replacement for packages currently depending on nose.
As a preliminary smoke test I built all packages depending on nose in Copr against pynose [5]. Packages failing to build against pynose I also built against nose [6]. I haven't checked all packages in detail yet, but the amount packages failing appears manageable (17).
Going by [3] I suppose this would also need to go through the changes process. If so, I'd be willing to drive that. But I could use some help/support since this would be my first change proposal.
In a way this e-mail is kind of an informal general rehearsal. With that said, please shoot! š«
[1] https://github.com/mdmintz/pynose [2] https://github.com/nose-devs/nose [3] https://fedoraproject.org/wiki/Changes/DeprecateNose [4] https://github.com/nose-devs/nose2 [5] https://copr.fedorainfracloud.org/coprs/gui1ty/pynose [6] https://copr.fedorainfracloud.org/coprs/gui1ty/nose
Cheers,
Hi.
This sounds like an interesting idea for one more reason. We are currently working on the update of pytest to version 8 and that version no longer runs nose setup/teardown functions/methods which, as far as we know now, is a change that breaks a lot of packages we previously migrated from nose to pytest.
For some of those it might be easier to switch them to pynose than to try to fix their tests for pytest.
But we already saw some successors of nose and they weren't really successfull.
LumĆr
On 4/9/24 19:30, Sandro wrote:
Hi,
While looking into a package upgrade I came across pynose [1]. It appears to be a drop-in replacement for nose [2], which has been deprecated in Fedora 32 [3].
There are still quite a few packages depending on nose, that have not yet moved away from it. The official successor to nose is nose2 [4]. But that package does not support all of the behavior of nose.
Therefore, I'm thinking of introducing pynose as a drop in replacement of deprecated nose. Pynose uses the same namespace as nose, but provides python3dist(pynose). Thus adding Provides: for nose would make it a drop-in replacement for packages currently depending on nose.
As a preliminary smoke test I built all packages depending on nose in Copr against pynose [5]. Packages failing to build against pynose I also built against nose [6]. I haven't checked all packages in detail yet, but the amount packages failing appears manageable (17).
Going by [3] I suppose this would also need to go through the changes process. If so, I'd be willing to drive that. But I could use some help/support since this would be my first change proposal.
In a way this e-mail is kind of an informal general rehearsal. With that said, please shoot! š«
[1] https://github.com/mdmintz/pynose [2] https://github.com/nose-devs/nose [3] https://fedoraproject.org/wiki/Changes/DeprecateNose [4] https://github.com/nose-devs/nose2 [5] https://copr.fedorainfracloud.org/coprs/gui1ty/pynose [6] https://copr.fedorainfracloud.org/coprs/gui1ty/nose
Cheers,
On 10-04-2024 11:57, LumĆr Balhar wrote:
This sounds like an interesting idea for one more reason. We are currently working on the update of pytest to version 8 and that version no longer runs nose setup/teardown functions/methods which, as far as we know now, is a change that breaks a lot of packages we previously migrated from nose to pytest.
For some of those it might be easier to switch them to pynose than to try to fix their tests for pytest.
Sure. That could probably be done independently of the "greater" goal of making pynose a drop-in replacement. Of course, it still requires the package to be added to Fedora. It's kinda ready for review. But I'd like to polish it some more. Stay tuned.
But we already saw some successors of nose and they weren't really successfull.
Well, from my preliminary examination of the packages that failed with pynose, I found six could trivially be reverted to using nosetests. Another six failed for unrelated reasons. And four need some more investigation[1].
For the trivial packages it was usually replacing `%{__python3} setup.py test` with `nosetests -v`. Going by the project's description, "pynose is an updated version of nose". Not a successor that needs adaptation, but basically a continuation. Time will tell.
[1] Ik know that adds up to 16. One package sneaked its way onto the list, though it works fine with pynose.
-- Sandro
On 09. 04. 24 19:30, Sandro wrote:
Therefore, I'm thinking of introducing pynose as a drop in replacement of deprecated nose. Pynose uses the same namespace as nose, but provides python3dist(pynose). Thus adding Provides: for nose would make it a drop-in replacement for packages currently depending on nose.
FTR We MUST NOT add RPM Provides for python3dist(nose) unless we package nose dist-info metadata.
For example %pyproject_buildrequires queries Python for installed packages (hence it reads the packaged dist-info/egg-info metadata) and it will see nose is not installed.
It will then query dnf to install python3dist(nose). dnf will install pynose.
%pyproject_buildrequires will see nose is not installed. It will query dnf to install python3dist(nose) again. dnf will install nothing.
The %pyproject_buildrequires phase will end prematurely when dnf installs nothing.
On 10-04-2024 12:04, Miro HronÄok wrote:
On 09. 04. 24 19:30, Sandro wrote:
Therefore, I'm thinking of introducing pynose as a drop in replacement of deprecated nose. Pynose uses the same namespace as nose, but provides python3dist(pynose). Thus adding Provides: for nose would make it a drop-in replacement for packages currently depending on nose.
FTR We MUST NOT add RPM Provides for python3dist(nose) unless we package nose dist-info metadata.
Thanks for pointing that out. I was indeed experimenting with it.
Is there some documentation or example for packaging extra dist-info metadata?
I also played around pith setuptools' `provide` only to learn later on that pip ignores it entirely (as well as `obsoletes`, which is deprecated). So, the pyproject-rpm-macros are probably not honoring that either.
For example %pyproject_buildrequires queries Python for installed packages (hence it reads the packaged dist-info/egg-info metadata) and it will see nose is not installed.
It will then query dnf to install python3dist(nose). dnf will install pynose.
%pyproject_buildrequires will see nose is not installed. It will query dnf to install python3dist(nose) again. dnf will install nothing.
The %pyproject_buildrequires phase will end prematurely when dnf installs nothing.
Agreed. If we do add python3dist(nose) it needs to work not cause more issues. Most packages I've looked at recently have a BR for python3-nose. That's covered by adding "%py_provides python3-nose". But dependency resolution in %pyproject_buildrequires uses python3dist. If the package configuration has a dependency on nose declared, I would like that to be satisfiable, both in RPM and pip, by installing python3-pynose.
If that is too much hassle or simply (currently) not possible, a fallback to not providing nose at all, is also possible. In that case more packages might need to be patched and we need to educate people te replace dependencies on nose with pynose.
My preference at the moment is for the former.
-- Sandro
On 10. 04. 24 17:30, Sandro wrote:
On 10-04-2024 12:04, Miro HronÄok wrote:
On 09. 04. 24 19:30, Sandro wrote:
Therefore, I'm thinking of introducing pynose as a drop in replacement of deprecated nose. Pynose uses the same namespace as nose, but provides python3dist(pynose). Thus adding Provides: for nose would make it a drop-in replacement for packages currently depending on nose.
FTR We MUST NOT add RPM Provides for python3dist(nose) unless we package nose dist-info metadata.
Thanks for pointing that out. I was indeed experimenting with it.
Is there some documentation or example for packaging extra dist-info metadata?
Not really, because it is a hack that should not be done if it can be avoided.
You can see a working example in python-lark
https://src.fedoraproject.org/rpms/python-lark/c/c7a9aa2e7b0b1d9d621ac60f73c...
I also played around pith setuptools' `provide` only to learn later on that pip ignores it entirely (as well as `obsoletes`, which is deprecated). So, the pyproject-rpm-macros are probably not honoring that either.
For example %pyproject_buildrequires queries Python for installed packages (hence it reads the packaged dist-info/egg-info metadata) and it will see nose is not installed.
It will then query dnf to install python3dist(nose). dnf will install pynose.
%pyproject_buildrequires will see nose is not installed. It will query dnf to install python3dist(nose) again. dnf will install nothing.
The %pyproject_buildrequires phase will end prematurely when dnf installs nothing.
Agreed. If we do add python3dist(nose) it needs to work not cause more issues. Most packages I've looked at recently have a BR for python3-nose. That's covered by adding "%py_provides python3-nose". But dependency resolution in %pyproject_buildrequires uses python3dist. If the package configuration has a dependency on nose declared, I would like that to be satisfiable, both in RPM and pip, by installing python3-pynose.
If that is too much hassle or simply (currently) not possible, a fallback to not providing nose at all, is also possible. In that case more packages might need to be patched and we need to educate people te replace dependencies on nose with pynose.
My preference at the moment is for the former.
If we are to retire python-nose at the same time, I'd do:
- have python3-pynose %py_provides (and Obsolete) python3-nose - don't mess with dist metadata at all
That way:
- packages that use upstream requirements will need to be updated (preferably upstream first => good) - packages that manually BuildRequire python3-nose will likely keep working
If the pynose package has a "nose" importable module, providing python3-nose even follows https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_provides_...
On 10-04-2024 17:50, Miro HronÄok wrote:
On 10. 04. 24 17:30, Sandro wrote:
On 10-04-2024 12:04, Miro HronÄok wrote:
On 09. 04. 24 19:30, Sandro wrote:
Therefore, I'm thinking of introducing pynose as a drop in replacement of deprecated nose. Pynose uses the same namespace as nose, but provides python3dist(pynose). Thus adding Provides: for nose would make it a drop-in replacement for packages currently depending on nose.
FTR We MUST NOT add RPM Provides for python3dist(nose) unless we package nose dist-info metadata.
Thanks for pointing that out. I was indeed experimenting with it.
Is there some documentation or example for packaging extra dist-info metadata?
Not really, because it is a hack that should not be done if it can be avoided.
You can see a working example in python-lark
https://src.fedoraproject.org/rpms/python-lark/c/c7a9aa2e7b0b1d9d621ac60f73c...
<snip>
Agreed. If we do add python3dist(nose) it needs to work not cause more issues. Most packages I've looked at recently have a BR for python3-nose. That's covered by adding "%py_provides python3-nose". But dependency resolution in %pyproject_buildrequires uses python3dist. If the package configuration has a dependency on nose declared, I would like that to be satisfiable, both in RPM and pip, by installing python3-pynose.
If that is too much hassle or simply (currently) not possible, a fallback to not providing nose at all, is also possible. In that case more packages might need to be patched and we need to educate people te replace dependencies on nose with pynose.
My preference at the moment is for the former.
If we are to retire python-nose at the same time, I'd do:
- have python3-pynose %py_provides (and Obsolete) python3-nose Ā - don't mess with dist metadata at all
That way:
- packages that use upstream requirements will need to be updated Ā Ā (preferably upstream first => good) Ā - packages that manually BuildRequire python3-nose will likely keep working
If the pynose package has a "nose" importable module, providing python3-nose even follows https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_provides_...
Well, `pynose` provides `nose` as an importable module, which is installed into `%{python3_sitelib}/nose`. Since that conflicts with python3-nose, proper Provides and Obsoletes are required, indeed.
Thanks for the example, I used that to properly provide `nose`. Building against that package reduced the troublesome packages to one (two more are currently FTBFS in rawhide/F40) compared to 11+2[1] when using only `%py_provides python3-nose`.
I think that justifies carrying the "hack". Moreover, `nose` has been dead for a long time upstream, yet we still have packages depending on it. In a way the justification here doesn't differ very much from that of `lark`.
I agree that maintainers of packages still depending on `nose` should ask upstream to switch to `pynose` or some other testing framework. Yet, I wouldn't be surprised that exactly those packages are also un(der)maintained upstream.
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
[1] Some failures were unrelated to `nose` vs. `pynose` and have been fixed by having `pynose` require `setuptools`.
-- Sandro
On 11. 04. 24 11:55, Sandro wrote:
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
I'd start by:
Packaging pynose without hacks (only making it Conflict with nose, no compatibility Provides, Obsoletes or dist-infos).
That way, pro-active packagers can switch already.
And the change proposal can then describe what will be *added* to pynose, rather than describing the approach from scratch.
On 11-04-2024 13:54, Miro HronÄok wrote:
On 11. 04. 24 11:55, Sandro wrote:
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
I'd start by:
Packaging pynose without hacks (only making it Conflict with nose, no compatibility Provides, Obsoletes or dist-infos).
That way, pro-active packagers can switch already.
That makes sense. Review is up [1]. If enough packagers adapt, I may not need to go through the changes process.
And the change proposal can then describe what will be *added* to pynose, rather than describing the approach from scratch.
Since predicting the future is difficult, I'll start on writing up a proposal while the package is being introduced, anyway.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=2274514
-- Sandro
On 11. 04. 24 15:05, Sandro wrote:
On 11-04-2024 13:54, Miro HronÄok wrote:
On 11. 04. 24 11:55, Sandro wrote:
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
I'd start by:
Packaging pynose without hacks (only making it Conflict with nose, no compatibility Provides, Obsoletes or dist-infos).
That way, pro-active packagers can switch already.
That makes sense. Review is up [1]. If enough packagers adapt, I may not need to go through the changes process.
And the change proposal can then describe what will be *added* to pynose, rather than describing the approach from scratch.
Since predicting the future is difficult, I'll start on writing up a proposal while the package is being introduced, anyway.
I see "# Package doesn't provide any tests" in the %check section. That certainly feels a bit dodgy. This successor of a test framework decided to ditch all of the tests it used to have? That is certainly a red flag.
On 11-04-2024 15:17, Miro HronÄok wrote:
On 11. 04. 24 15:05, Sandro wrote:
On 11-04-2024 13:54, Miro HronÄok wrote:
On 11. 04. 24 11:55, Sandro wrote:
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
I'd start by:
Packaging pynose without hacks (only making it Conflict with nose, no compatibility Provides, Obsoletes or dist-infos).
That way, pro-active packagers can switch already.
That makes sense. Review is up [1]. If enough packagers adapt, I may not need to go through the changes process.
And the change proposal can then describe what will be *added* to pynose, rather than describing the approach from scratch.
Since predicting the future is difficult, I'll start on writing up a proposal while the package is being introduced, anyway.
I see "# Package doesn't provide any tests" in the %check section. That certainly feels a bit dodgy. This successor of a test framework decided to ditch all of the tests it used to have? That is certainly a red flag.
More like a chicken and egg story, maybe? If I were to provide a testing framework, I'd very much like to use that testing framework for testing.
<insert picture of a dog chasing its own tail>
Anyway, I'll contact upstream asking them about it. It's the least I can do. I'll also ask about the documentation link on PyPI, which points to the RTD page of ye olde š.
-- Sandro
On 11-04-2024 15:30, Sandro wrote:
I see "# Package doesn't provide any tests" in the %check section. That certainly feels a bit dodgy. This successor of a test framework decided to ditch all of the tests it used to have? That is certainly a red flag.
More like a chicken and egg story, maybe? If I were to provide a testing framework, I'd very much like to use that testing framework for testing.
<insert picture of a dog chasing its own tail>
Anyway, I'll contact upstream asking them about it. It's the least I can do. I'll also ask about the documentation link on PyPI, which points to the RTD page of ye olde š.
Upstream uses GitHub Actions for testing. I haven't looked into how those are set up. Apparently they also test with pynose on SeleniumBase (same maintainer, see elsewhere in this thread).
While this still not provides us with the ability to test downstream, all tests I've run while investigating the feasibility of my idea so far haven't brought up any issues.
If anyone wants to chime in, here is the ticket:
https://github.com/mdmintz/pynose/issues/25
Last but not least, the documentation link on PyPI was fixed right away and will be in the next release.
-- Sandro
For a proposed nose successor, pynose doesnāt seem to have gained much community traction so far: it has seven stars on GitHub[1] (compared to 770 for nose2, which itself was never that widely adopted and has fewer than ten dependent packages in Fedora); and the imperfect but fairly useful reverse dependency index at Wheelodex[2] shows only three packages that explicitly depend on it.
As far as I can tell, pynose exists because SeleniumBase[3] found it easier to fork nose than to port away from it. Iām *definitely* not saying we should only package things with a bunch of GitHub stars, but I do have some concerns about whether or not pynose is going to remain a generally-useful project in the medium-to-long term.
[1] https://github.com/mdmintz/pynose/stargazers
[2] https://www.wheelodex.org/projects/pynose/rdepends/
[3] https://github.com/seleniumbase/SeleniumBase
On 4/11/24 9:17 AM, Miro HronÄok wrote:
On 11. 04. 24 15:05, Sandro wrote:
On 11-04-2024 13:54, Miro HronÄok wrote:
On 11. 04. 24 11:55, Sandro wrote:
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
I'd start by:
Packaging pynose without hacks (only making it Conflict with nose, no compatibility Provides, Obsoletes or dist-infos).
That way, pro-active packagers can switch already.
That makes sense. Review is up [1]. If enough packagers adapt, I may not need to go through the changes process.
And the change proposal can then describe what will be *added* to pynose, rather than describing the approach from scratch.
Since predicting the future is difficult, I'll start on writing up a proposal while the package is being introduced, anyway.
I see "# Package doesn't provide any tests" in the %check section. That certainly feels a bit dodgy. This successor of a test framework decided to ditch all of the tests it used to have? That is certainly a red flag.
On 11-04-2024 15:49, Ben Beasley wrote:
For a proposed nose successor, pynose doesnāt seem to have gained much community traction so far: it has seven stars on GitHub[1] (compared to 770 for nose2, which itself was never that widely adopted and has fewer than ten dependent packages in Fedora); and the imperfect but fairly useful reverse dependency index at Wheelodex[2] shows only three packages that explicitly depend on it.
Well, Wheelodex doesn't show pysb. They have just switched from nose to pynose. When looking into updating the package to the latest release the noses were no longer aligned (pun intended). That's how I started looking into pynose.
As far as I can tell, pynose exists because SeleniumBase[3] found it easier to fork nose than to port away from it. Iām *definitely* not saying we should only package things with a bunch of GitHub stars, but I do have some concerns about whether or not pynose is going to remain a generally-useful project in the medium-to-long term.
I think you are right there. The current maintainer of pynose is also a/the maintainer of SeleniumBase.
Currently, pynose is seeing regular releases - seven since May 2023. When I just inquired regarding unit tests, I received a response within minutes. This might just be coincidence. But it shows the project is alive.
Regarding usefulness, almost all packages in Fedora still depending on nose worked out of the box when I tested with a build of pynose mimicking nose also in the Python meta data. Two packages I couldn't test since they are currently FTBFS. One package, pycdio, is using `%python3 setup.py nosetests`. I have not yet looked into why that works with nose but not with pynose. But that line could simply be changed back to `nosetests -v`.
I think the takeaway here is swapping an unmaintained package for a maintained package. How widely it will get adopted seems lees important to me. The future will tell.
[1] https://github.com/mdmintz/pynose/stargazers
-- Sandro
On 11-04-2024 13:54, Miro HronÄok wrote:
On 11. 04. 24 11:55, Sandro wrote:
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
I'd start by:
Packaging pynose without hacks (only making it Conflict with nose, no compatibility Provides, Obsoletes or dist-infos).
That way, pro-active packagers can switch already.
And the change proposal can then describe what will be *added* to pynose, rather than describing the approach from scratch.
I put together a first draft for a change proposal [1]. The pynose package has been reviewed and will be imported as soon as all fires have been put out.
Miro, I added you as a proposal owner as per the comment "owners of all affected packages need to be included here" since you are maintaining python-nose.
I would appreciate a review from people having more experience witch change proposals before I submit it.
LumĆr, if you have a list of packages that are affected by the removal of nose setup/teardown functions/methods, I can add them as a separate bullet point list to the proposal as well as to the testing repo in Copr and start working on those, porting them back to `nosetests`.
[1] https://fedoraproject.org/wiki/Changes/ReplaceNoseWithPynose
-- Sandro
On 25-04-2024 15:15, Sandro wrote:
On 11-04-2024 13:54, Miro HronÄok wrote:
On 11. 04. 24 11:55, Sandro wrote:
While I ponder those thoughts some more, moving forward in either direction, the next step would be writing a change proposal?
I'd start by:
Packaging pynose without hacks (only making it Conflict with nose, no compatibility Provides, Obsoletes or dist-infos).
That way, pro-active packagers can switch already.
And the change proposal can then describe what will be *added* to pynose, rather than describing the approach from scratch.
I put together a first draft for a change proposal [1]. The pynose package has been reviewed and will be imported as soon as all fires have been put out.
Miro, I added you as a proposal owner as per the comment "owners of all affected packages need to be included here" since you are maintaining python-nose.
I would appreciate a review from people having more experience witch change proposals before I submit it.
LumĆr, if you have a list of packages that are affected by the removal of nose setup/teardown functions/methods, I can add them as a separate bullet point list to the proposal as well as to the testing repo in Copr and start working on those, porting them back to `nosetests`.
[1] https://fedoraproject.org/wiki/Changes/ReplaceNoseWithPynose
I intent to go forward with the proposal and submit it as ready for wrangler by the end of Friday next week.
If anyone has any feedback or would like to join in on the proposal, this is the last call to action. Of course, there's still plenty of opportunity for feedback as part of the changes process itself.
LumĆr, should you have a list of packages that are impacted by the dropped functions in pytest 8.x and have previously been using nose, I'd be happy to add that information.
-- Sandro
python-devel@lists.fedoraproject.org
-
Ben Beasley -
LumĆr Balhar -
Miro HronÄok -
Sandro