F40 Change: Privacy-preserving Telemetry for Fedora Workstation (System-Wide)
by Aoife Moloney
Important process note: we are experimenting with using Fedora
Discussion as part of the Changes process. Change announcements (like
the one you are reading right now) will still be sent to the
devel-announce mailing list, but the conversation about each change
will take place on Fedora Discussion at
https://discussion.fedoraproject.org/t/f40-change-request-privacy-preserv...
This will follow the same process as before, just with discussion in a
different format
https://docs.fedoraproject.org/en-US/program_management/changes_policy/
You can subscribe to and interact with these conversations by email.
See https://discussion.fedoraproject.org/t/guide-to-interacting-with-this-sit...
for detailed instructions. To make sure you do not miss anything, make
sure that you have the Change Proposal category set to “Watching” —
or, if you just want to get notified about new changes but not every
reply in the conversation, to “Watching First Post”. (Click on the
little bell icon at the top right of the category page.)
The below document represents a proposed Change. As part of the
Changes process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
The Red Hat Display Systems Team (which develops the desktop) proposes
to enable limited data collection of anonymous Fedora Workstation
usage metrics.
Fedora is an open source community project, and nobody is interested
in violating user privacy. We do not want to collect data about
individual users. We want to collect only aggregate usage metrics that
are actually needed to achieve specific Fedora improvement objectives,
and no more. We understand that if we violate our users' trust, then
we won't have many users left, so if metrics collection is approved,
we will need to be very careful to roll this out in a way that
respects our users at all times. (For example, we should not collect
users' search queries, because that would be creepy.)
We believe an open source community can ethically collect limited
aggregate data on how its software is used without involving big data
companies or building creepy tracking profiles that are not in the
best interests of users. Users will have the option to disable data
upload before any data is sent for the first time. Our service will be
operated by Fedora on Fedora infrastructure, and will not depend on
Google Analytics or any other controversial third-party services. And
in contrast to proprietary software operating systems, you can
redirect the data collection to your own private metrics server
instead of Fedora's to see precisely what data is being collected from
you, because the server components are open source too.
Keep in mind this Fedora change proposal is just that: a proposal. It
must undergo community review and must be approved by the
community-elected Fedora Engineering Steering Committee (FESCo) before
it can be implemented, just like any other Fedora change proposal. We
welcome community participation and fully expect this proposal may
need to be modified significantly depending on Fedora community
feedback.
== Owner ==
* Name: [[User:catanzaro|Michael Catanzaro]]
* Email: <mcatanzaro(a)redhat.com>
== Detailed Description ==
We intend to deploy the Endless OS metrics system.
[https://blogs.gnome.org/wjjt/2023/07/05/endless-oss-privacy-preserving-me...
This blog post] contains a description of how the system works. We do
not plan to deploy the eos-phone-home component in Fedora.
=== How will data collection be approved? ===
The proposal owners feel it is essential to ensure the Fedora
community has ultimate oversight over metrics collection. Community
control is required to maintain user trust. If this change proposal is
approved, then we'll need new policies and procedures to ensure
community oversight over metrics collection and ensure Fedora users
can be confident that our metrics collection does not violate their
privacy.
We can say "we would never collect personally-identifiable data" and
write software that really doesn't collect any such data, but this
alone will never be enough to ensure user confidence. We will need a
metrics collection policy that describes what sort of data may be
collected by Fedora (anonymous, non-invasive), and what sort of data
may not be collected. Such a policy does not exist currently. We will
also want to ensure the Fedora community has ultimate control over
which particular metrics are collected. One option is that each metric
to be collected should be separately approved by FESCo. Collection of
particular metrics in a particular data format is ultimately an
engineering decision, and therefore FESCo seems like an appropriate
approval point. Because FESCo members are elected regularly by the
Fedora community, this also provides the community with ultimate
control over metrics collection via the election process. But other
oversight and approval structures would work too.
=== What data might we collect? ===
We are not proposing to collect any of these particular metrics just
yet, because a process for Fedora community approval of metrics to be
collected does not yet exist. That said, in the interests of maximum
transparency, we wish to give you an idea of what sorts of metrics we
might propose to collect in the future.
One of the main goals of metrics collection is to analyze whether Red
Hat is achieving its goal to make Fedora Workstation the premier
developer platform for cloud software development. Accordingly, we
want to know things like which IDEs are most popular among our users,
and which runtimes are used to create containers using Toolbx.
Metrics can also be used to inform user interface design decisions.
For example, we want to collect the clickthrough rate of the
recommended software banners in GNOME Software to assess which banners
are actually useful to users. We also want to know how frequently
panels in gnome-control-center are visited to determine which panels
could be consolidated or removed, because there are other settings we
want to add, but our usability research indicates that the current
high quantity of settings panels already makes it difficult for users
to find commonly-used settings.
Metrics can help us understand the hardware we should be optimizing
Fedora for. For example, our boot performance on hard drives dropped
drastically when systemd-readahead was removed. Ubuntu has maintained
its own readahead implementation, but Fedora does not because we
assume that not many users use Fedora on hard drives. It would be nice
to collect a metric that indicates whether primary storage is a solid
state drive or a hard disk, so we can see actual hard drive usage
instead of guessing. We would also want to collect hardware
information that would be useful for collaboration with hardware
vendors (such as Lenovo), such as laptop model ID.
Other Fedora teams may have other metrics they wish to collect. For
example, Fedora localization wishes to count users of particular
locales to evaluate which locales are in poorer shape relative to
their usage.
This is only a small sample of what we might want to know; no doubt
other community members can think of many more interesting data points
to collect. But note the purpose of all of the above metrics is to
inform specific design decisions, not to build tracking profiles. We
only need to collect data in aggregate, and have no need to associate
the data we collect with particular users.
=== Metrics transparency ===
Transparency is required to provide confidence that Fedora metrics
collection is not creepy or invasive. Since Fedora is open source, a
developer can review the source code to verify exactly what it is
doing and what data is being collected. But most Fedora users are not
software developers, and few software developers have time or
inclination to review the source code of the operating system to see
what it is doing. To retain user trust, we need an easy way for users
to understand exactly what data we are collecting. We propose to
maintain a documentation page showing the current metrics database
schema, so users can see exactly which fields are in the database and
what example data looks like.
Experienced users may gain additional confidence by building and
running their own metrics collection server; all of the components of
the server (discussed below) are open source, and we will provide
instructions for how to run a simple server yourself and view its
metrics database. You can redirect metrics from Fedora's server to
your own by changing a URL in a configuration file.
=== User control ===
A new metrics collection setting will be added to the privacy page in
gnome-initial-setup and also to the privacy page in
gnome-control-center. This setting will be a toggle that will enable
or disable metrics collection for the entire system. We want to ensure
that metrics are never submitted to Fedora without the user's
knowledge and consent, so the underlying setting will be off by
default in order to ensure metrics upload is not unexpectedly turned
on when upgrading from an older version of Fedora. However, we also
want to ensure that the data we collect is meaningful, so
gnome-initial-setup will default to displaying the toggle as enabled,
even though the underlying setting will initially be disabled. (The
underlying setting will not actually be enabled until the user
finishes the privacy page, to ensure users have the opportunity to
disable the setting before any data is uploaded.) This is to ensure
the system is opt-out, not opt-in. This is essential because we know
that opt-in metrics are not very useful. Few users would opt in, and
these users would not be representative of Fedora users as a whole. We
are not interested in opt-in metrics.
To make this a little more confusing, metrics collection is actually
separate from uploading. Collection is always initially enabled, while
uploading is always initially disabled. The graphical toggle enables
or disables both at the same time. That is, a newly-installed Fedora
system will always collect metrics locally at first, but the collected
metrics will be deleted and never submitted to Fedora if the user
disables the metrics collection toggle on the privacy page. If the
user leaves the toggle enabled, then the collected metrics may be
submitted only after finishing the privacy page.
Metrics uploading will be opt-in for users who upgrade from previous
versions of Fedora Workstation, because we don't yet have a mechanism
to ask the user to consent to data collection after a system upgrade
like we do for new installations, but metrics collection will be
opt-out. That is, your upgraded system will collect metrics locally
but will never submit them to Fedora. If you visit the privacy page in
gnome-control-center, then both collection and uploading will be
either enabled or disabled depending on the user's selection. Unlike
gnome-initial-setup, the switch in gnome-control-center will default
to off if the user has not seen the switch in gnome-initial-setup and
has not previously selected a value for the setting.
This might sound complicated, but it is consistent. If the user has
not yet made a decision whether to allow telemetry, we collect it
locally so that it's ready to submit if the user approves telemetry in
the future, but we never upload it. Once the user makes a decision,
then we either upload it or delete it and stop collecting.
=== GDPR ===
It is Fedora Legal's obligation to ensure our data collection complies
with legal requirements in the jurisdictions in which Red Hat
operates. This is not an obligation of the Fedora community, so there
is no need to discuss GDPR rules on our mailing lists. The proposal
owners will not respond to mailing list posts that discuss GDPR or
similar legal obligations during this change proposal discussion. In
short, let's keep discussion focused on what Fedora SHOULD or SHOULD
NOT do, rather than what we MUST or MUST NOT do.
That said, Fedora Legal has determined that if we collect any
personally-identifiable data, the entire metrics system must be
opt-in. Since we are only interested in opt-out metrics due to the low
value of opt-in metrics, we must accordingly never collect any
personally-identifiable data. We must also not collect any data that
could become personally-identifiable if combined with other data,
which notably means IP addresses must not be stored. We only want to
collect anonymous data anyway, but we need to be especially mindful of
the possibility that combining two "anonymous" data points could
result in the data no longer being anonymous.
=== Fedora data collection policy ===
Fedora Legal requires that we publish a Fedora data collection policy
separate from the existing
[https://fedoraproject.org/wiki/Legal:PrivacyPolicy Fedora Privacy
Policy], which is designed to address usage of Fedora websites. This
is currently a work in progress that we're not quite ready to share
yet. You can expect it to be very short and very generic.
=== Metrics server infrastructure ===
We propose to deploy Azafea, the open source metrics collection server
used by Endless OS. An Azafea deployment consists of five components:
an nginx proxy server,
[https://github.com/endlessm/azafea-metrics-proxy
azafea-metrics-proxy], redis, [https://github.com/endlessm/azafea
azafea itself], and a Postgres database. nginx proxies HTTP requests
to azafea-metrics-proxy, which is itself a simple HTTP server that
adds metrics into the redis database, where they will be fetched by
Azafea and stored into Postgres. We will provide instructions on how
to set up your own server and see for yourself what data gets
collected.
=== Metrics client infrastructure ===
The client side consists of [https://github.com/endlessm/eos-metrics
eos-metrics], [https://github.com/endlessm/eos-event-recorder-daemon
eos-event-recorder-daemon], and
[https://github.com/endlessm/eos-metrics-instrumentation
eos-metrics-instrumentation]. eos-metrics is a D-Bus interface that
applications and services may use to record events, plus a GObject
library that provides a simple API around the D-Bus interface.
eos-event-recorder-daemon is the service that actually implements this
interface: it collects incoming metrics, batches them together, and
sends them to the metrics server at predefined intervals.
eos-metrics-instrumentation is the component that actually collects
specific metrics. Originally, we had planned to not use this component
and instead write our own fedora-metrics-instrumentation that would
collect only a few particular metrics that are approved via Fedora
community process. However, currently we are planning to ship
eos-metrics-instrumentation and instead ensure that it is not
collecting more metrics than would be acceptable to the Fedora
community. A review process to decide which metrics to collect and
which metrics to disable will be required.
=== Data set considerations ===
Although we assume the metrics server administrator is not malicious
and will not actively attempt to deanonymize users, we will still take
reasonable precautions to make it difficult to correlate metrics to a
particular user, starting by not storing any IP address information in
the metrics database. Additionally, each metric that we collect will
be considered individual, non-correlatable data by default, unless
approved to be correlated with particular other metrics via future
Fedora community process. That is, if a user submits two data points,
we usually don't want the ability to know that these data points were
both submitted by the same user.
Each metric is stored in the database with a Unix timestamp indicating
when it was generated on the client. If abused, this timestamp could
allow correlation of data points that are collected at the same time
as each other, or at a fixed time offset to other events. For example,
if the system were designed to collect two metrics exactly 300 seconds
after the system were booted, then just looking at the timestamps
would be enough to determine that both metrics recorded at the same
time were submitted by the same user. Accordingly, we should consider
modifying the metrics server to reduce timestamp granularity at least
somewhat.
=== History ===
Currently Fedora's only form of metrics collection is
[https://fedoraproject.org/wiki/Changes/DNF_Better_Counting DNF Better
Counting], but this only counts Fedora installations. That is useful,
but we want to count more than just how many users we have.
Fedora's first metrics collection attempt was
[https://en.wikipedia.org/wiki/Smolt_(Linux) Smolt], a precursor to
hw-probe which collected data on user hardware. The current proposal
is different from Smolt because it will collect more than just
hardware data, and also because Smolt collected only opt-in data. The
current proposal would be opt-out, not opt-in.
This change proposal will likely be compared to the Ubuntu spyware
complaints from a decade ago, when Ubuntu desktop users' search
queries were sent to Amazon by default. Let's not do that.
== Feedback ==
We will endeavor to update this section of the change proposal to
include a summary of Fedora community discussion of this proposal.
== Benefit to Fedora ==
The main benefit to Fedora is that we will be able to use collected
metrics to inform design decisions. It is very common for developers
to wish to know something about how Fedora software is used, and we
will finally have a way to answer such questions.
Occasionally, Red Hat might need to collect specific metrics to
justify additional time spent on contributing to Fedora or additional
investment in Fedora.
== Scope ==
* Proposal owners:
This change requires substantial technical and nontechnical work from
the change owners. Most notably, we will need to package eos-metrics,
eos-event-recorder-daemon, and eos-metrics-instrumentation properly
for Fedora; they are currently packaged in a copr. We also still need
to modify eos-metrics-instrumentation so that it does not send events
not approved for use in Fedora, as we expect to collect less data than
Endless OS.
* Other developers:
This proposal will require substantial effort by Community Platform
Engineering (CPE) to host the metrics server infrastructure.
* Release engineering: [https://pagure.io/releng/issues/11514 #11514]
* Policies and guidelines: New processes and guidelines are proposed
above under the section "How will data collection be approved?"
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: This change does not align with any
current [https://docs.fedoraproject.org/en-US/project/initiatives/
Fedora Initiatives], which are very limited in scope. That said, one
of the main purposes of metrics collection is to determine whether we
are achieving other objectives not listed on the wiki page. For
example, we want Fedora Workstation to become the premier developer
workstation operating system. To that end, we want to know how many of
our users are using particular IDEs.
== Upgrade/compatibility impact ==
We would like to enable metrics upload for upgraded systems, but this
isn't trivial because we want to obtain user consent before enabling
metrics upload. This would require us to design a user interface that
would run on upgraded systems and present the setting to users. We
have not yet created such a user interface, so for now metrics upload
will need to default to disabled for systems upgraded from older
versions of Fedora. Since the underlying setting will be off by
default, we don't need to do anything special to achieve this.
== How To Test ==
The ultimate goal is to see metrics appear in the Postgres database of
a metrics server, but configuring and running the server is not
trivial. Accordingly, we propose to publish a separate document
detailing how to set up and configure a metrics server for testing
purposes, how to redirect metrics to the custom server, and how to
force the client to immediately submit metrics to ease testing.
Although we don't actually expect many community members to seriously
run their own metrics servers, we still want to document the steps
involved so that interested developers can see exactly how it works.
== User Experience ==
A new metrics collection setting will be added to the privacy page in
gnome-initial-setup and also to the privacy page in
gnome-control-center. This setting will be a simple toggle that will
enable or disable all metrics upload for the entire system. Users who
do not want any metrics upload should feel confident that uploading
can be disabled with a simple toggle.
Fedora users should be confident that Fedora metrics collection
respects their privacy and collects only limited, anonymous usage
data.
== Dependencies ==
Any package that wishes to collect a metric would need to depend on
eos-metrics. For example, if we were to collect statistics on which
system settings panels are used most frequently, then the
gnome-control-center package would need to depend on eos-metrics in
order to send a metric to eos-event-recorder-daemon.
== Contingency Plan ==
* Contingency mechanism: We would need to remove the eos-metrics,
eos-event-recorder-daemon, and eos-metrics-instrumentation packages
from the workstation-product comps group, and rebuild any packages
that gained a dependency on eos-metrics.
* Contingency deadline: Beta freeze
* Blocks release? Yes, if the change is incomplete, it will need to be
reverted before release.
== Documentation ==
This feature will depend on several different upstream projects with
varying amounts of documentation.
The client side consists of eos-metrics, eos-event-recorder-daemon,
and eos-metrics-instrumentation. The best documentation of eos-metrics
available online is its
[https://github.com/endlessm/eos-metrics/blob/master/data/com.endlessm.Met...
D-Bus interface XML]. eos-metrics also contains normal API
documentation that will be built and installed in a docs subpackage,
but this is not currently available online. The
eos-event-recorder-daemon and eos-metrics-instrumentation components
do not appear to have any online documentation.
On the server end, the metrics server consists of azafea-metrics-proxy
feeding metrics into redis, where they will be pulled by azafea and
then added to a Postgres database. Documentation for
[https://github.com/endlessm/azafea-metrics-proxy/tree/master/docs/source
azafea-metrics-proxy] and
[https://github.com/endlessm/azafea/tree/master/docs/source azafea]
can be reviewed online.
[https://azafea.readthedocs.io/en/latest/events.html Events recognized
by the server are documented here.] Note that this documentation is
currently focused on use by Endless OS rather than by Fedora, and
includes documentation of many events that are no longer sent by
Endless OS. This change proposal does not propose to enable sending
any particular events in Fedora.
== Release Notes ==
Release Notes are not required for initial proposal. We need to write
the release notes before change freeze.
--
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
_______________________________________________
devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org
To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora...
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
4 weeks, 1 day
some man pages have bugs, can't be grep'd
by Chris Murphy
I've been seeing this since clean installing Fedora 30. I don't recall
ever seeing it before, including on a Fedora 29 -> Fedora 30 upgraded
system (is now the clean installed system).
[chris@flap ~]$ man rpm | grep -C 10 rpmverbosity
<standard input>:176: warning [p 3, 0.8i]: cannot adjust line
[chris@flap mantest]$ man rpm >rpm.stdout 2>rpm.stderr
[chris@flap mantest]$ ll
-rw-rw-r--. 1 chris chris 62 Jul 16 14:24 rpm.stderr
-rw-rw-r--. 1 chris chris 28498 Jul 16 14:24 rpm.stdout
Is this a bug that should be reported against rpm or something else?
I'm certain I've seen it in other man pages, but offhand I can't find
another example.
--
Chris Murphy
1 month, 3 weeks
Unretire ulogd (or another NFLOG logger?)
by Chris Adams
I'd like to use NFLOG to log firewall drops (so that the kernel message
log isn't spammed by them), but it doesn't appear there's anything
currently in Fedora that can read that other than "tcpdump -i nflog".
It looks like ulogd was retired a while back because it only had a SysV
init script and nobody stepped up to convert it to systemd (which should
be really simple, since the old init script is pretty much a textbook
template of a SysV init script).
Am I missing anything? Is that all it needs? Is there another NFLOG
logger daemon?
--
Chris Adams <linux(a)cmadams.net>
2 months, 3 weeks
fedrq - new repoquerying tool
by Maxwell G
Hi Fedorians,
I've been working on a repoquerying tool called fedrq [1] that I'd like
to share with you. Here's the elevator pitch: fedrq provides a friendly
interface to query the Fedora repositories. It makes it really easy to
query across Fedora and EPEL branches. It uses the dnf Python bindings
(libdnf5 backend is almost done) and doesn't shell out to dnf repoquery.
Amongst other things, fedrq allows querying for reverse dependencies,
packages that contain a certain Provide or file, subpackages of an SRPM,
and general package metadata. My favorite features are the easy branch
switching, `fedrq subpkgs` (there's no real equivalent in dnf
repoquery), and the ability to dump package metadata as JSON. The many
threads about how to properly query for dependencies when doing SO name
bump rebuilds and my own frustrations with dnf repoquery inspired this
tool.
You can install fedrq from PyPI or from gotmax23/fedrq [2] on Copr. The
EXAMPLES section of fedrq(1) [3] provides a good intro and demonstration
of fedrq's functionality. The tool is very much a WIP and any feedback
is appreciated.
For reverse dependency rebuilds, you probably want the following
command:
```
$ fedrq whatrequires -X -F source $(fedrq subpkgs SRCNAME) # equivalent
$ fedrq subpkgs SRCNAME | fedrq whatrequires -X -i -F source # equivalent
```
This accounts for any package that depends on any subpackage/binary RPM
produced by SRCNAME at buildtime and/or runtime, and spits out the names
of the source packages that need to be rebuilt. For simple packages that
output only one RPM, `fedrq whatrequires -F source PKGNAME` is
sufficient.
[1] https://sr.ht/~gotmax23/fedrq/
[2] https://copr.fedorainfracloud.org/coprs/gotmax23/fedrq/
[3] https://gotmax23.srht.site/fedrq/fedrq.1.html#EXAMPLES
--
Thanks,
Maxwell G (@gotmax23)
Pronouns: He/They
3 months, 1 week
Co-maintainers for my ham packages
by Matt Domsch
Due to an impending move to NYC and related downsizing of my house into a
2-bedroom apartment, I'm selling all my ham radio gear. Therefore I won't
be able to test any of the Fedora packages I maintain with actual
hardware. Would anyone be interested in maintaining or co-maintaining
these?
- direwolf - Sound Card-based AX.25 TNC
- CubicSDR - Cross-Platform Software-Defined Radio Panadapter
- liquid-dsp - Digital Signal Processing Library for Software-Defined Radios
- sdrpp - SDRPlusPlus bloat-free SDR receiver software
- SoapySDR - A Vendor Neutral and Platform Independent SDR Support Library
- soapy-rtlsdr - SoapySDR module for RTL-SDR hardware
Thanks,
Matt N5MLD
5 months, 3 weeks
F38 proposal: Rpmautospec by Default (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/Rpmautospec_by_Default
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Rpmautospec (`%autorelease` and `%autochangelog`) is recommended as
the default approach.
Packaging Guidelines and other documentation are adjusted to describe
this approach first.
Various tools that provide spec file templates are adjusted.
== Owner ==
* Name: [[User:Nphilipp| Nils Philippsen]], [[User:Zbyszek| Zbigniew
Jędrzejewski-Szmek]]
* Email: nphilipp - at - redhat.com, zbyszek - at - in.waw.pl
== Detailed Description ==
{{admon/note|Brief reminder about rpmautospec|
The spec file contains:
<pre>
Version: 1.2.3
Release: %autorelease
...
%changelog
%autochangelog
</pre>
Rpmautospec uses git history. Whenever the package is built
(`.src.rpm` is generated), rpmautospec tooling will replace the
`%autorelease` macro with the number of commits since the last commit
that changed the `Version` field, and the `%autochangelog` macro with
a text generated from `git log`.
For details see the
[https://docs.pagure.org/fedora-infra.rpmautospec/principle.html
docs].
}}
Rpmautospec has been deployed in Fedora since F35
([[Changes/rpmautospec]]), and 3423/23045 packages use it (15%).
But it is still a "second-class citizen": most documentation doesn't
mention it, and many packagers know that it exists but don't use it in
their packages. We think that it's reasonable to switch to
`%autorelease`+`%autochangelog` for almost all packages and that
Packaging Guidelines and various packaging howtos should recommend
that approach to packagers. The "traditional" approach of
manually-managed `Release` and `%changelog` will remain valid and will
be documented as a fallback.
This change is targeted at Fedora 38, but it will actually apply to
all releases. The goal is to update the Packaging Guidelines and other
prominent documentation and tools now, and other docs and tools
possibly at a later time. Changing packages is out of scope.
It is worth mentioning that `rust2rpm` uses
`%autorelease`+`%autochangelog` since a few releases, so most rust
packages have switched. (Generally, rust spec files are recreated
using the generator for each new version, so the switch would happen
whenever a new version is packaged unless the packager opts out.)
== Feedback ==
* Thread on fedora-devel in August 2022:
[https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
rpmautospec by default]
** open issues: a bunch have been fixed.
** maintenance: Nils will add some co-maintainers.
** compatibility with rpmdevtools, fedpkg/rpkg, fedora-review: see
Scope section.
== Benefit to Fedora ==
Various packaging workflows become smoother for packagers and contributors:
* packagers don't need to touch the `Release` field on updates
* packagers describe changes just once in the git commit message, the
`%changelog` entry is autogenerated
* patches to the spec file can be cherry-picked between branches
without trivial conflicts
* pull requests on src.fedoraproject.org can be merged without trivial conflicts
* in workflows that regenerate the spec file (rust2rpm, pip2rpm, …)
`%changelog` section doesn't need to be copied over
== Scope ==
* Proposal owners:
** provide pull requests to Packaging Guidelines and other docs to
make `%autorelease`+`%autochangelog` the default
** implement fixes for issues reported by packagers
** make semi-regular releases of rpmautospec
([https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
0.3.1 was released on November 17th, and should be deployed to
production in about two weeks])
* Other developers:
** provide pull requests to other docs as appropriate
** accept the changes to documentation
** update other spec file generators (pip2rpm, others?)
* Somebody (TBD):
** `fedora-review` — https://pagure.io/rpkg/issue/641
** `fedpkg import` — with https://pagure.io/rpkg/c/3087dd7, the
command will fail. A replacement workflow that instead restores
`%autorelease`+`%autochangelog` in the file committed to dist-git
needs to be implemented.
* Related work
** https://pagure.io/rpkg/c/3087dd7
** https://src.fedoraproject.org/rpms/fedora-packager/pull-request/4
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: a list of places to be updated
** https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs
** https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning
** https://docs.fedoraproject.org/en-US/package-maintainers/Packaging_Tutori...
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
Rpmautospec is already used by a decent number of packages, so any
issues are already being seen and need to be fixed anyway.
== How To Test ==
* Convert an existing package: `rpmautospec convert`. Ideally this
step is done right before a version bump so that the release numbers
restart at `-1`.
* Do local builds (`fedpkg local`, `fedpkg mockbuild`). Verify
correctness of version-release (`rpm -qpi`) and the changelog (`rpm
-qp --changelog`).
* Do builds in koji. Verify correctness of version-release and the changelog.
* For new packages, use `%autorelease`+`%autochangelog`. Repeat all
the tests listed above.
* Assume you are a newbie packager. Read the packaging docs and check
that the workflow is clear and the intructions are sufficient to use
rpmautospec tooling correctly.
== User Experience ==
No changes visible to end users.
== Dependencies ==
None.
== Contingency Plan ==
If it turns out that the rpmautospec workflows have unknown problems,
we can revert changes to documentation.
* Contingency mechanism: Revert changes to documentation by reverting
the appropriate commits. This can be done easily by FPC.
* Contingency deadline: Any time.
* Blocks release? No.
== Documentation ==
This page and any changes to Packaging Guidelines and other documents.
== Release Notes ==
Not needed.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
6 months, 3 weeks
F39 Change Proposal: Allow Removal of tzdata (System-Wide)
by Aoife Moloney
https://fedoraproject.org/wiki/Changes/AllowRemovalOfTzdata
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Allow the removal of tzdata especially on containers in order to minimize size.
== Owner ==
* Name: Patsy Griffin (Franklin)
* Email: patsy(a)redhat.com
== Detailed Description ==
This change will allow the removal of tzdata. When tzdata is removed,
the system will default to UTC. In order to reduce overhead, many
container installations now remove the data associated with tzdata but
cannot fully remove the package due to dependencies by other packages.
This results in confusion regarding the expected timezone info.
In order for this to work, we need packages that use tzdata at run
time to switch from Require'ing tzdata to Recommend'ing tzdata. These
packages should also gracefully handle instances where tzdata has been
removed. For example, this would require recognizing that tzdata is
not present and providing an appropriate error, such as recommending
that the user install tzdata.
== Feedback ==
In June of 2021, we proposed creating a new tzdata sub-package that
would only provide the UTC timezone. As part of the discussion around
this proposal, it was recommended that we completely remove tzdata. We
appreciated this input and welcome additional feedback.
== Benefit to Fedora ==
This change will allow tzdata to be removed from containers without
leaving inconsistent package remnants.
== Scope ==
* Proposal owners: No changes are needed to tzdata.
* Other developers: Some packages need to change their spec files from
`Requires: tzdata` to `Recommends: tzdata`. It would be beneficial if
all packages switched in this way, but it is not required. Supporting
optional tzdata installation for as many workloads as possible allows
those workloads to minimize their container image size.
List of packages which need to be changed:
* glibc (glibc-common)
* gcc (libstdc++)
* python3.XX (3.9, 3.10, 3.11, 3.12)
List of packages which would be beneficial to be changed:
* python3-dateutil
* python3-pytz
* libical
Upon acceptance of the change request we will file bugs to fix each of
these packages for Fedora 39.
* Release engineering: No changes needed.
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Community Initiatives:
== Upgrade/compatibility impact ==
If tzdata is already installed, then it continues to be installed on the system.
Following the Fedora Weak Dependencies Policy dnf will treat the
Recommends on tzdata as if it were a Requires and tzdata will always
be installed in a default system
(https://docs.fedoraproject.org/en-US/packaging-guidelines/WeakDependencies/)
However, the recommends will allow tzdata to be correctly uninstalled
in a container build file rather than having to use ‘rm -rf’ to delete
the zone files to recover space.
== How To Test ==
Language runtimes were installed and A/B tests carried out with tzdata
present and tzdata removed. The intent of these tests was to ensure
that the system can use the language frameworks without tzdata present
and that when the data was required that meaningful errors were
presented to the user.
Packages tested were C (glibc), C++ (libstdc++), Python (Python 3.11),
using their time and date APIs. This testing led to the correction of
the libstdc++ implementation as noted here:
https://gcc.gnu.org/cgit/gcc/commit/?id=4abd5bc600193e821fbc41995a0b8d9ea...
Developers can test this by installing Rawhide and uninstalling tzdata
and verifying their package operates as expected.
If tzdata cannot be uninstalled then we recommend filling a bug
against the package that Requires: tzdata and having a discussion with
the maintainer to make tzdata optional e.g. Recommends: tzdata.
== User Experience ==
The user can remove tzdata to minimize the container size.
With tzdata removed the system is UTC only.
== Dependencies ==
Fixing glibc, gcc, and python3.xx at a minimum to make tzdata
Recommends instead of Requires.
== Contingency Plan ==
* Contingency mechanism: Back out the change. Don’t do it.
* Contingency deadline: Can be backed out at the last minute since we
are only dealing with conversions from Requires to Recommends.
* Blocks release? No
== Documentation ==
Document that the tzdata package may be removed if needed to reduce
space on containers. With tzdata removed, the system will default to
UTC.
== Release Notes ==
--
Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford
7 months, 3 weeks
Potential changes to systemd RPM macros
by Andrea Bolognani
Hi,
this requires a bit of background, so please bear with me :)
Earlier this year, the libvirt project changed its RPM package to be
more granular[1] as a step towards a future where the legacy
monolithic daemon (libvirtd) is completely gone. This change has
happened upstream, but the Fedora package uses the same spec file so
it was quickly reflected in Rawhide.
As part of this change virtproxyd, a service which has historically
been part of the libvirt-daemon package, has been moved to the
newly-introduced libvirt-daemon-proxy package.
More recently we have realized that, despite our care in trying to
ensure smooth upgrades, splitting the package this way has
unfortunately caused unintended consequences[2][3]. Specifically,
since the libvirt-daemon-proxy package is a completely new one as far
as RPM is concerned, all its systemd units will get presets applied
during %post. In at least two completely valid deployment scenarios,
this results in some amount of brokenness due to local configuration
changes made by the admin getting discarded.
In order to avoid this problem, I have implemented a new set of RPM
macros[4][5] that base the decision of whether presets should be
applied for a systemd unit not on whether the package that contains
them is being newly installed, but rather on whether the unit itself
existed on the system before package installation. This should
address not only the scenario of services being split off to separate
packages, but also the one where an existing package starts shipping
additional services.
Now, since this is clearly not a libvirt-specific issue, I believe
this approach should be adopted across Fedora by way of these macros
(or comparable ones) being added to systemd. Packages would have to
migrate from the old macros over time, and at some point it should be
possible to get rid of them entirely.
Note that openSUSE already has its own distro-specific RPM macros for
dealing with systemd units, and in fact the approach that I have
implemented is partially inspired by theirs. Ideally, we'd be able to
collaborate with them to create a single set of macros that lives in
systemd and satisfies the needs of both Fedora and openSUSE.
The problem is that Fedora 39 and RHEL 9.3 are fast approaching and,
if we don't do anything about this issue before then, a subset of
libvirt users will see their deployments broken after upgrade. In the
interest of avoiding that, I would prefer to get the libvirt-specific
version of the macros merged as soon as possible, and focus on
upstreaming the work all the way to systemd as a follow-up.
Does this plan sound reasonable to the Fedora community? Are there
any serious concerns regarding the approach taken for the macros that
would cause them to be considered a complete no-go? Any scenarios
that I might have missed while implementing them?
Thanks for your patience in reading this far :) and looking forward
to your feedback!
[1] https://listman.redhat.com/archives/libvir-list/2023-January/237059.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=2210058
[3] https://listman.redhat.com/archives/libvir-list/2023-June/240226.html
[4] https://listman.redhat.com/archives/libvir-list/2023-July/240723.html
[5] https://listman.redhat.com/archives/libvir-list/2023-July/240729.html
--
Andrea Bolognani / Red Hat / Virtualization
8 months, 1 week
