Hi,
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
On Jul 17 18:32, Ian Chapman wrote:
Hi,
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
Probably, or dhcpd needs to set the IP_FREEBIND socket option.
This has been discussed already a couple of days ago on this list. It's a generic problem with the way network.target and network-online.target are defined now. As a consequence, a tracker bug has been created in bugzilla, see https://bugzilla.redhat.com/show_bug.cgi?id=1119787
Several per-package bug reports have been created and connected to the tracker bug already. It would be very helpful if you could create another bug report for dhcpd, along the lines of the other bug reports like https://bugzilla.redhat.com/show_bug.cgi?id=1096081 or https://bugzilla.redhat.com/show_bug.cgi?id=1119814. The important thing here is that you fill in the "Blocks" field with the number of the tracker bug 1119787, so this can be followed through.
HTH, Corinna
Ian Chapman writes:
Hi,
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
Won't make a difference.
Welcome to the systemd fan club.
Install the following file as /etc/systemd/system/wait-for-network.service
[Unit] Description=Wait for network ports to be initialized Before=network.target network-online.target After=network.service Wants=network.target
[Service] Type=oneshot ExecStart=/usr/bin/true
[Install] WantedBy=multi-user.target
On 17/07/14 18:45, Corinna Vinschen wrote:
On Jul 17 18:32, Ian Chapman wrote:
Hi,
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
Probably, or dhcpd needs to set the IP_FREEBIND socket option. The important thing here is that you fill in the "Blocks" field with the number of the tracker bug 1119787, so this can be followed through.
Changing from After=network.target to After=network-online.target seems to work around the problem. I've filed a bug against dhcpd as you suggested. Unfortunately it looks like Bind might be affected too, whilst it starts on boot, it doesn't respond to clients resolution requests until I manually restart the service after boot. I'll look a bit deeper into that one too and consider a bug report.
On 17/07/14 18:57, Sam Varshavchik wrote:
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
Won't make a difference.
Welcome to the systemd fan club.
Install the following file as /etc/systemd/system/wait-for-network.service
Thanks Sam, actually it did seem to work fine for me. I changed it to network-online.target and dhcpd did start on boot. I'm using the traditional 'network' service rather than NetworkManager because at the time NetworkManager didn't support bridges, so I'm not sure if that differs from your setup. I did however have to jump through similar hoops with autofs though on my client, eventually making the display manager depend on NetworkManager-wait-online.service because it was possible to login before autofs even started. It seems that Bind might have issues along similar lines to dhcpd too, except that it does start but refused to serve DNS requests until I restart the service.
Hi Ian,
On Jul 17 19:23, Ian Chapman wrote:
On 17/07/14 18:45, Corinna Vinschen wrote:
On Jul 17 18:32, Ian Chapman wrote:
Hi,
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
Probably, or dhcpd needs to set the IP_FREEBIND socket option. The important thing here is that you fill in the "Blocks" field with the number of the tracker bug 1119787, so this can be followed through.
Changing from After=network.target to After=network-online.target seems to work around the problem. I've filed a bug against dhcpd as you suggested. Unfortunately it looks like Bind might be affected too, whilst it starts on boot, it doesn't respond to clients resolution requests until I manually restart the service after boot. I'll look a bit deeper into that one too and consider a bug report.
I already reported the bind problem yesterday:
https://bugzilla.redhat.com/show_bug.cgi?id=1119815
Have a look at the tracker bug https://bugzilla.redhat.com/show_bug.cgi?id=1119787
It contains a list of dependent bug reports. The current list contains bug reports against bind, dovecot, ejabberd, openssh, postfix, radicale, and even an old bug report of mine against the initscripts network service.
Corinna
On Jul 17 14:28, Corinna Vinschen wrote:
Hi Ian,
On Jul 17 19:23, Ian Chapman wrote:
On 17/07/14 18:45, Corinna Vinschen wrote:
On Jul 17 18:32, Ian Chapman wrote:
Hi,
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
Probably, or dhcpd needs to set the IP_FREEBIND socket option. The important thing here is that you fill in the "Blocks" field with the number of the tracker bug 1119787, so this can be followed through.
Changing from After=network.target to After=network-online.target seems to work around the problem. I've filed a bug against dhcpd as you suggested. Unfortunately it looks like Bind might be affected too, whilst it starts on boot, it doesn't respond to clients resolution requests until I manually restart the service after boot. I'll look a bit deeper into that one too and consider a bug report.
I already reported the bind problem yesterday:
https://bugzilla.redhat.com/show_bug.cgi?id=1119815
Have a look at the tracker bug https://bugzilla.redhat.com/show_bug.cgi?id=1119787
It contains a list of dependent bug reports. The current list contains bug reports against bind, dovecot, ejabberd, openssh, postfix, radicale, and even an old bug report of mine against the initscripts network service.
Oh, and now dhcp, too, of course :)
Corinna
Ian Chapman writes:
On 17/07/14 18:57, Sam Varshavchik wrote:
Dhcpd on my server has suddenly started taking to start before the IP address (statically assigned) has been configured on the network interface and consequently bombs out. The systemd service for dhcpd has
After=network.target
should this be network-online.target?
Won't make a difference.
Welcome to the systemd fan club.
Install the following file as /etc/systemd/system/wait-for-network.service
Thanks Sam, actually it did seem to work fine for me. I changed it to network-online.target and dhcpd did start on boot.
It'll work for a while. At some point, it's going to break again (the next systemd update, likely).
I'm using the traditional'network' service rather than NetworkManager because at the time NetworkManager didn't support bridges, so I'm not sure if that differs from your setup.
No, that's my setup.
The stub service is the only reliable, permanent fix to this systemd brain damage.
The only reason your change "worked" is because it altered, slightly, how systemd chooses, more or less, at random, the order in which to fork off services it thinks should be started at the same time; in combination with the time it takes each individual service to start.
Despite the outward appearance, systemd is completely ignoring all Before and After directives. Only now, the combination you have, is working solely due to chance. At some point, an update to some other unrelated package might change its startup profile sufficiently to collapse the entire house of cards, and make everything break again.
Been there, done that, have the souvenirs on my fireplace mantle.
I did however have to jump through similar hoops with autofsthough on my client, eventually making the display manager depend on NetworkManager-wait-online.service because it was possible to login before autofs even started. It seems that Bind might have issues along similar lines to dhcpd too, except that it does start but refused to serve DNS requests until I restart the service.
That's because at the time it starts, not all network interfaces are set up, so it only binds itself to localhost. You can determine it by sifting through the syslog, looking at what bind is doing at startup.
In my case, all network servers were frakked up. dhcp. bind. httpd. privoxy.
On Jul 17 08:34, Sam Varshavchik wrote:
Despite the outward appearance, systemd is completely ignoring all Before and After directives.
No, it's not. You never replied to my mail in the thread you started a couple of days ago, but before/after do *not* define dependencies. They define an order only. An A after B is satisfied if A started. If it's doing anything useful is totally irrelevant.
[...] That's because at the time it starts, not all network interfaces are set up, so it only binds itself to localhost. You can determine it by sifting through the syslog, looking at what bind is doing at startup.
In my case, all network servers were frakked up. dhcp. bind. httpd. privoxy.
I'm really not sure what you're doing, but my experience is that all my networks services binding to explicit interface addresses failed, and they were started correctly after I added the network-waiter target I posted over in the other thread. There are two problems at work here, one is that the network initscript is broken for a long time (see my bug report at https://bugzilla.redhat.com/show_bug.cgi?id=782042) and the other is that the network target is not defined to be only reached when all network interfaces are up. That's what network-online.target is supposed to accomplish.
Corinna
On 17/07/14 20:28, Corinna Vinschen wrote:
boot, it doesn't respond to clients resolution requests until I manually restart the service after boot. I'll look a bit deeper into that one too and consider a bug report.
I already reported the bind problem yesterday:
https://bugzilla.redhat.com/show_bug.cgi?id=1119815
Have a look at the tracker bug https://bugzilla.redhat.com/show_bug.cgi?id=1119787
It contains a list of dependent bug reports. The current list contains bug reports against bind, dovecot, ejabberd, openssh, postfix, radicale, and even an old bug report of mine against the initscripts network service.
No worries. I've applied the fix to named. Dovecot and OpenSSH are working fine for me at the moment but if that changes I'll fix those up too.
On 17/07/14 20:34, Sam Varshavchik wrote:
how systemd chooses, more or less, at random, the order in which to fork off services it thinks should be started at the same time; in combination with the time it takes each individual service to start.
That's the hardest thing I find to get my head around with systemd. Two very similar systems, with services configured similarly but due to timing one works (perhaps due to luck) and the other doesn't and tracking down the reason feels painful.
-
Corinna Vinschen -
Ian Chapman -
Sam Varshavchik