F35 Change: Debuginfod By Default (Self-Contained Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/DebuginfodByDefault
== Summary ==
Fedora users / developers who need to debug/trace distro binaries can
make use of the recently activated elfutils-debuginfod servers to
automatically fetch debugging data and source code, instead of having
to use `# sudo dnf` commands.
== Owner ==
* Name: [[User:fche| Frank Ch. Eigler]]
* Email: fche(a)redhat.com
== Detailed Description ==
Numerous fedora debugging-type tools have built-in capabilities to use
the [https://sourceware.org/elfutils/Debuginfod.html debuginfod]
protocol to fetch debuginfo/source code automatically. We would like
to activate a setting so that Fedora's debuginfod servers are
automatically used, rather than requiring hand-editing individual
users' dot files.
== Feedback ==
There has existed
[https://www.spinics.net/lists/fedora-devel/msg279002.html broad
interest] in a Fedora debuginfod server since the project was proposed
/ announced in 2020, and several distros already operate public
servers of this nature. Some of the distros configure their
installations by default to talk to those servers, some do not.
Turning on this by default has some limited privacy implications.
Some Debian users have
[https://lists.debian.org/debian-devel/2021/02/msg00262.html expressed
concerns] that this facility "calls home" during debugging, so it may
expose a limited amount of information about what a user is debugging.
The information is limited to the build-id and source code file names
of programs being debugged, and is only sent to the servers if their
machine lacks locally installed debuginfo. Whether this should be
opt-in or opt-out and how has been resolved there via an install-time
query to the sysadmin. In contrast, on OpenSUSE Tumbleweed, it is
[https://build.opensuse.org/request/show/879926 simply defaulted-on],
and we have heard of no controversy.
We anticipate discussing this topic on the mailing list, and noting it
in the release notes either way.
== Benefit to Fedora ==
This will improve developers' experience.
It may reduce download server burden, as only individual
ELF/DWARF/source files are downloaded rather than entire `-debuginfo`
and `-debugsource` RPMs.
It would help Fedora catch up to other distros who put up `debuginfod`
servers already. :-)
== Scope ==
* Proposal owners:
The work could consist one extra parameter in the `elfutils.spec`
`%configure`. Its effect is to arrange for the
`elfutils-debuginfod-client` RPM
to install an `/etc/profile.d` file that sets the `DEBUGINFOD_URLS`
environment variable automatically to
`https://debuginfod.fedoraproject.org/`. (At the time of this
writing, the _staging_ server is getting ready for testing:
`https://debuginfod.stg.fedoraproject.org/`.)
* Other developers: None - relevant code has been previously upstreamed!
* Release engineering: None - our team is operating the
`debuginfod[.stg].fedoraproject.org` VMs.
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
None.
Note that these servers index all active Fedora releases (32-), all
architectures, so users of those versions can already set
`DEBUGINFOD_URLS` manually to take advantage.
== How To Test ==
* Install `elfutils-debuginfod-client`
* Open arbitrary fedora binary via gdb.
** Admire the immediate downloading of debuginfo and source code.
* Run `eu-stack -v -p $pid` for an arbitrary process.
** Admire the immediate downloading of debuginfo to give precise file:line data.
== User Experience ==
Primarily: users running debuggers, profilers, tracing tools on
internet-capable machines can work immediately, without switching to
privileged users and fragile manual `dnf` commands to install this
data.
== Dependencies ==
The `debuginfod` servers at `fedora-infra` need to be up.
== Contingency Plan ==
* Contingency mechanism: change the elfutils-debuginfod-client subrpm
to not set the default `DEBUGINFOD_URLS` environment variable for all
users. In the case of a server outage, the debugger tools revert to
debuginfo-less operation, prior to this feature.
* Contingency deadline: shortly before freeze
* Blocks release? No
== Documentation ==
There is upstream documentation in the debugging tools as well as
associated with the client code / cli tooling. What our Release Notes
would focus on however is the _automatic activation_ of this facility
via the environment variable.
== Release Notes ==
TBD.
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
3 days, 3 hours
some man pages have bugs, can't be grep'd
by Chris Murphy
I've been seeing this since clean installing Fedora 30. I don't recall
ever seeing it before, including on a Fedora 29 -> Fedora 30 upgraded
system (is now the clean installed system).
[chris@flap ~]$ man rpm | grep -C 10 rpmverbosity
<standard input>:176: warning [p 3, 0.8i]: cannot adjust line
[chris@flap mantest]$ man rpm >rpm.stdout 2>rpm.stderr
[chris@flap mantest]$ ll
-rw-rw-r--. 1 chris chris 62 Jul 16 14:24 rpm.stderr
-rw-rw-r--. 1 chris chris 28498 Jul 16 14:24 rpm.stdout
Is this a bug that should be reported against rpm or something else?
I'm certain I've seen it in other man pages, but offhand I can't find
another example.
--
Chris Murphy
2 months
Unretire ulogd (or another NFLOG logger?)
by Chris Adams
I'd like to use NFLOG to log firewall drops (so that the kernel message
log isn't spammed by them), but it doesn't appear there's anything
currently in Fedora that can read that other than "tcpdump -i nflog".
It looks like ulogd was retired a while back because it only had a SysV
init script and nobody stepped up to convert it to systemd (which should
be really simple, since the old init script is pretty much a textbook
template of a SysV init script).
Am I missing anything? Is that all it needs? Is there another NFLOG
logger daemon?
--
Chris Adams <linux(a)cmadams.net>
3 months
Co-maintainers for my ham packages
by Matt Domsch
Due to an impending move to NYC and related downsizing of my house into a
2-bedroom apartment, I'm selling all my ham radio gear. Therefore I won't
be able to test any of the Fedora packages I maintain with actual
hardware. Would anyone be interested in maintaining or co-maintaining
these?
- direwolf - Sound Card-based AX.25 TNC
- CubicSDR - Cross-Platform Software-Defined Radio Panadapter
- liquid-dsp - Digital Signal Processing Library for Software-Defined Radios
- sdrpp - SDRPlusPlus bloat-free SDR receiver software
- SoapySDR - A Vendor Neutral and Platform Independent SDR Support Library
- soapy-rtlsdr - SoapySDR module for RTL-SDR hardware
Thanks,
Matt N5MLD
6 months
F38 proposal: Rpmautospec by Default (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/Rpmautospec_by_Default
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Rpmautospec (`%autorelease` and `%autochangelog`) is recommended as
the default approach.
Packaging Guidelines and other documentation are adjusted to describe
this approach first.
Various tools that provide spec file templates are adjusted.
== Owner ==
* Name: [[User:Nphilipp| Nils Philippsen]], [[User:Zbyszek| Zbigniew
Jędrzejewski-Szmek]]
* Email: nphilipp - at - redhat.com, zbyszek - at - in.waw.pl
== Detailed Description ==
{{admon/note|Brief reminder about rpmautospec|
The spec file contains:
<pre>
Version: 1.2.3
Release: %autorelease
...
%changelog
%autochangelog
</pre>
Rpmautospec uses git history. Whenever the package is built
(`.src.rpm` is generated), rpmautospec tooling will replace the
`%autorelease` macro with the number of commits since the last commit
that changed the `Version` field, and the `%autochangelog` macro with
a text generated from `git log`.
For details see the
[https://docs.pagure.org/fedora-infra.rpmautospec/principle.html
docs].
}}
Rpmautospec has been deployed in Fedora since F35
([[Changes/rpmautospec]]), and 3423/23045 packages use it (15%).
But it is still a "second-class citizen": most documentation doesn't
mention it, and many packagers know that it exists but don't use it in
their packages. We think that it's reasonable to switch to
`%autorelease`+`%autochangelog` for almost all packages and that
Packaging Guidelines and various packaging howtos should recommend
that approach to packagers. The "traditional" approach of
manually-managed `Release` and `%changelog` will remain valid and will
be documented as a fallback.
This change is targeted at Fedora 38, but it will actually apply to
all releases. The goal is to update the Packaging Guidelines and other
prominent documentation and tools now, and other docs and tools
possibly at a later time. Changing packages is out of scope.
It is worth mentioning that `rust2rpm` uses
`%autorelease`+`%autochangelog` since a few releases, so most rust
packages have switched. (Generally, rust spec files are recreated
using the generator for each new version, so the switch would happen
whenever a new version is packaged unless the packager opts out.)
== Feedback ==
* Thread on fedora-devel in August 2022:
[https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
rpmautospec by default]
** open issues: a bunch have been fixed.
** maintenance: Nils will add some co-maintainers.
** compatibility with rpmdevtools, fedpkg/rpkg, fedora-review: see
Scope section.
== Benefit to Fedora ==
Various packaging workflows become smoother for packagers and contributors:
* packagers don't need to touch the `Release` field on updates
* packagers describe changes just once in the git commit message, the
`%changelog` entry is autogenerated
* patches to the spec file can be cherry-picked between branches
without trivial conflicts
* pull requests on src.fedoraproject.org can be merged without trivial conflicts
* in workflows that regenerate the spec file (rust2rpm, pip2rpm, …)
`%changelog` section doesn't need to be copied over
== Scope ==
* Proposal owners:
** provide pull requests to Packaging Guidelines and other docs to
make `%autorelease`+`%autochangelog` the default
** implement fixes for issues reported by packagers
** make semi-regular releases of rpmautospec
([https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
0.3.1 was released on November 17th, and should be deployed to
production in about two weeks])
* Other developers:
** provide pull requests to other docs as appropriate
** accept the changes to documentation
** update other spec file generators (pip2rpm, others?)
* Somebody (TBD):
** `fedora-review` — https://pagure.io/rpkg/issue/641
** `fedpkg import` — with https://pagure.io/rpkg/c/3087dd7, the
command will fail. A replacement workflow that instead restores
`%autorelease`+`%autochangelog` in the file committed to dist-git
needs to be implemented.
* Related work
** https://pagure.io/rpkg/c/3087dd7
** https://src.fedoraproject.org/rpms/fedora-packager/pull-request/4
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: a list of places to be updated
** https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs
** https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning
** https://docs.fedoraproject.org/en-US/package-maintainers/Packaging_Tutori...
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
Rpmautospec is already used by a decent number of packages, so any
issues are already being seen and need to be fixed anyway.
== How To Test ==
* Convert an existing package: `rpmautospec convert`. Ideally this
step is done right before a version bump so that the release numbers
restart at `-1`.
* Do local builds (`fedpkg local`, `fedpkg mockbuild`). Verify
correctness of version-release (`rpm -qpi`) and the changelog (`rpm
-qp --changelog`).
* Do builds in koji. Verify correctness of version-release and the changelog.
* For new packages, use `%autorelease`+`%autochangelog`. Repeat all
the tests listed above.
* Assume you are a newbie packager. Read the packaging docs and check
that the workflow is clear and the intructions are sufficient to use
rpmautospec tooling correctly.
== User Experience ==
No changes visible to end users.
== Dependencies ==
None.
== Contingency Plan ==
If it turns out that the rpmautospec workflows have unknown problems,
we can revert changes to documentation.
* Contingency mechanism: Revert changes to documentation by reverting
the appropriate commits. This can be done easily by FPC.
* Contingency deadline: Any time.
* Blocks release? No.
== Documentation ==
This page and any changes to Packaging Guidelines and other documents.
== Release Notes ==
Not needed.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
7 months
Thoughts welcome: interface between automated test gating and the
"critical path"
by Adam Williamson
Hi folks!
I have one of those definitional quandaries and I figured I'd throw it
at the lists for some input.
Right now, we kind of take advantage of the "critical path" concept for
automated update testing and gating via openQA.
openQA does not test all updates, only critical path updates plus
updates containing any package on a short allowlist.
Bodhi and Greenwave do not gate all updates on the openQA tests since
not all updates are tested; again we use the critpath definition. If an
update is critical path, it gets gated on the openQA tests. If it
isn't, it doesn't.
If you've been paying attention, that means there's a bit of a hole:
the packages on the 'allowlist'. These are tested, but not gated.
What's on the allowlist? Basically, FreeIPA-related packages. We have a
good set of FreeIPA tests in openQA, and both I and the FreeIPA team
wanted relevant updates to be tested with them. But those packages are
not on the critical path. So I set up this 'allowlist' mechanism.
However, the lack of gating kinda sucks. I would much prefer if we
could gate these updates as well as just testing them.
The obvious thing to do is add the FreeIPA-related packages to the
critical path, but that really is not supported by the critical path
definition:
https://fedoraproject.org/wiki/Critical_path_package
which defines it as packages required to "perform the most fundamental
actions on a system", with a list of specific areas, none of which is
"run a domain server".
So...what to do?
I can think of I guess four options:
1. Broaden the definition of the "critical path" somehow. We could just
write in that it includes FreeIPA functionality, I guess, though that
seems special purpose. We could broaden it to include any functionality
covered by the release criteria, which would be quite a big increase
but seems like a reasonable and concise definition. Or we could write
in that it includes any functionality that is exercised by the gating
openQA tests, though that seems a bit arbitrary - there's no
particularly great organizing principle to the openQA tests we choose
to run on updates, if I'm honest, it's a sort of grab bag I came up
with.
2. Keep the current "critical path" concept but define a broader group
of "gated packages" somewhere (could be comps or somewhere else). This
would require engineering work - we'd have to touch probably the openQA
scheduler, Bodhi, and greenwave configs. It's also another maintenance
burden.
3. Add gating config to each allowlisted package repo's gating.yml one
by one. I don't really like this option. It'd be a chunk of work to do
initially, and multiples the maintenance required when the list of
tests to gate on changes for some reason.
4. Do nothing, just keep only gating things that are "critical path" on
the current definition. In a utopian future, we'd manage to deploy
openQA in the cloud or get a giant pile of super fast servers so we
could test and gate *every* update, and this wouldn't be an issue any
more.
What do folks think? Any preferences? Thanks!
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net
9 months, 2 weeks
Packaging a cross-compilation environment (wasi-libc)
by Jan Staněk
Hi list,
in order to be able to compile WASM natively on Fedora,
I'm trying to package wasi-libc[1] to provide the Web Assembly System
Interface.
[1]: https://github.com/WebAssembly/wasi-libc
My trouble is that this is in essence a cross-compilation environment,
and I have zero experience in trying to package these.
Also, I did not have much luck in trying to find any related
documentation.
Some issues I have run into so far:
- This is a libc implementation, which would probably conflict with the
glibc package by default. Looking at musl, the solution seems to be to
install into `/usr/{target}` prefix (i.e. `/usr/wasm32-wasi/include`).
Not really sure how this works, any pointers appreciated.
- Clang seems to have issue with `-fstack-clash-protection` flag for the
`wasm32-wasi` target. What's the proper way to adjust these?
Any additional tips on cross-compilation support in Fedora would also be
appreciated :)
Thanks in advance for any help!
--
Jan Staněk
Software Engineer, Red Hat
jstanek(a)redhat.com irc: jstanek
9 months, 2 weeks
