Latest FC kernel still have SMP bugs?

[Mitch Wiedemann]

Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP and 
hanging?
I'm due to install some MP servers soon and want to avoid problems 
(imagine that!).

If said kernel is still having trouble, what's the easiest fix?

Thanks in advance.

-- 
Mitch Wiedemann
mc^2 Computer Consulting
mc2(a)lightlink.com
http://www.lightlink.com/mc2

[Phil Schaffner]

On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
>  Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP
> and 
>  hanging?
>  I'm due to install some MP servers soon and want to avoid problems 
>  (imagine that!). 
Still having problems with sudden hard lockups that leave no apparent
clues, and show no easily discernible pattern, on 2 SMP machines running
2174 here.

>  
>  If said kernel is still having trouble, what's the easiest fix? 
I'm wondering also.  Considering trying FC2T2 on one as an experiment,
but obviously that's not a good choice for a server that needs to be
dependable.

Phil

[Nigel Wade]

Mitch Wiedemann wrote:
>  Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP
> and 
>  hanging?
>  I'm due to install some MP servers soon and want to avoid problems 
>  (imagine that!).
>  
>  If said kernel is still having trouble, what's the easiest fix?
>  
>  Thanks in advance.
>   
I'm currently investigating SMP here. When I tried stock FC1 and the latest 
2.4.22-1.2174.nptl I had severe problems with LDAP. I didn't get any hard 
lockup, but I would have delays of between 4 and 300 secs in LDAP response 
times. I have a strong suspicion that the delays are related to FUTEX, but 
that's hard to prove.

I found that disabling hyperthreading is the answer to my problems. When I 
disabled it in the BIOS LDAP started behaving itself.

I'm currently stress testing the LDAP server and it's managing ~40,000 
requests per minute with the peak search times of around 5ms. I read in some 
other threads that mounting/unmounting NFS and Samba filesystems lead to 
hard lockups so I'm mounting/listing/unmounting one of each every half hour. 
The test will be left running for the next 72 hours to see what happens...


-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[WipeOut]

Nigel Wade wrote:

>  Mitch Wiedemann wrote:
>
> > Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP 
> > and hanging?
> > I'm due to install some MP servers soon and want to avoid problems 
> > (imagine that!).
> >
> > If said kernel is still having trouble, what's the easiest fix?
> >
> > Thanks in advance.
> >
>
>  I'm currently investigating SMP here. When I tried stock FC1 and the 
>  latest 2.4.22-1.2174.nptl I had severe problems with LDAP. I didn't 
>  get any hard lockup, but I would have delays of between 4 and 300 secs 
>  in LDAP response times. I have a strong suspicion that the delays are 
>  related to FUTEX, but that's hard to prove.
>
>  I found that disabling hyperthreading is the answer to my problems. 
>  When I disabled it in the BIOS LDAP started behaving itself.
>
>  I'm currently stress testing the LDAP server and it's managing ~40,000 
>  requests per minute with the peak search times of around 5ms. I read 
>  in some other threads that mounting/unmounting NFS and Samba 
>  filesystems lead to hard lockups so I'm mounting/listing/unmounting 
>  one of each every half hour. The test will be left running for the 
>  next 72 hours to see what happens...
>
>  
Compile a SMP kernel from vanilla kernel.org source and it works like a 
dream.. Even with HT enabled.. :)

Later..

[Nigel Wade]

WipeOut wrote:
>  Nigel Wade wrote:
>  
> > Mitch Wiedemann wrote:
> >
> >> Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP 
> >> and hanging?
> >> I'm due to install some MP servers soon and want to avoid problems 
> >> (imagine that!).
> >>
> >> If said kernel is still having trouble, what's the easiest fix?
> >>
> >> Thanks in advance.
> >>
> >
> > I'm currently investigating SMP here. When I tried stock FC1 and the 
> > latest 2.4.22-1.2174.nptl I had severe problems with LDAP. I didn't 
> > get any hard lockup, but I would have delays of between 4 and 300 secs 
> > in LDAP response times. I have a strong suspicion that the delays are 
> > related to FUTEX, but that's hard to prove.
> >
> > I found that disabling hyperthreading is the answer to my problems. 
> > When I disabled it in the BIOS LDAP started behaving itself.
> >
> > I'm currently stress testing the LDAP server and it's managing ~40,000 
> > requests per minute with the peak search times of around 5ms. I read 
> > in some other threads that mounting/unmounting NFS and Samba 
> > filesystems lead to hard lockups so I'm mounting/listing/unmounting 
> > one of each every half hour. The test will be left running for the 
> > next 72 hours to see what happens...
> >
> >
>  Compile a SMP kernel from vanilla kernel.org source and it works like a 
>  dream.. Even with HT enabled.. :)
>  
>  Later..
>  
>  
>  
>   
My system crashed at 23:35. The cron job to mount/unmount filesystems was 
set to run at :10 and :35 every hour. Given what other people report in this 
and other threads I'd say either mount or unmount was the culprit.

I think I'll be building my own kernel now.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Nigel Wade]

Nigel Wade wrote:

>  
>  My system crashed at 23:35. The cron job to mount/unmount filesystems 
>  was set to run at :10 and :35 every hour. Given what other people report 
>  in this and other threads I'd say either mount or unmount was the culprit.
>  
>  I think I'll be building my own kernel now.
>   
I built the latest 2.4 kernel, but LDAP won't run with that kernel (more 
precicely, the BDB backend won't start and slapd seg. faults). So that's a 
non-starter.

I guess I'm back to square one. The system won't operate in SMP with the FC 
kernel and won't operate at all without a FC kernel. So it looks like Fedora 
is snafu'd.

Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, and when 
I tried an upgrade to openldap I started getting FUTEX locking problems.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[WipeOut]

Nigel Wade wrote:

>  Nigel Wade wrote:
>
> >
> > My system crashed at 23:35. The cron job to mount/unmount filesystems 
> > was set to run at :10 and :35 every hour. Given what other people 
> > report in this and other threads I'd say either mount or unmount was 
> > the culprit.
> >
> > I think I'll be building my own kernel now.
> >
>
>  I built the latest 2.4 kernel, but LDAP won't run with that kernel 
>  (more precicely, the BDB backend won't start and slapd seg. faults). 
>  So that's a non-starter.
>
>  I guess I'm back to square one. The system won't operate in SMP with 
>  the FC kernel and won't operate at all without a FC kernel. So it 
>  looks like Fedora is snafu'd.
>
>  Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, and 
>  when I tried an upgrade to openldap I started getting FUTEX locking 
>  problems.
>  
This may be a stupid question since it sounds like you know what you are 
doing but I thought I would ask anyway..

Did you use the FC .config when recompiling the kernel? don't know if it 
will make any difference but it may do..

Later..

[Nigel Wade]

WipeOut wrote:
>  Nigel Wade wrote:
>  
> > Nigel Wade wrote:
> >
> >>
> >> My system crashed at 23:35. The cron job to mount/unmount filesystems 
> >> was set to run at :10 and :35 every hour. Given what other people 
> >> report in this and other threads I'd say either mount or unmount was 
> >> the culprit.
> >>
> >> I think I'll be building my own kernel now.
> >>
> >
> > I built the latest 2.4 kernel, but LDAP won't run with that kernel 
> > (more precicely, the BDB backend won't start and slapd seg. faults). 
> > So that's a non-starter.
> >
> > I guess I'm back to square one. The system won't operate in SMP with 
> > the FC kernel and won't operate at all without a FC kernel. So it 
> > looks like Fedora is snafu'd.
> >
> > Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, and 
> > when I tried an upgrade to openldap I started getting FUTEX locking 
> > problems.
> >
>  This may be a stupid question since it sounds like you know what you are 
>  doing but I thought I would ask anyway..
>  
>  Did you use the FC .config when recompiling the kernel? don't know if it 
>  will make any difference but it may do..
>  
>  Later..
>  
>   
Yes, I used kernel-2.4.22-i686-smp.config. Running 'make oldconfig' asked a 
few questions about new features to which I answered N.

I presume that BDB is built using NPTL, and the stock kernel doesn't have 
NPTL support (AFAIK, that is backported into RH9/RHEL3/FC1 from the 2.6 
kernel). But this is conjecture, I really don't know enough about it.

Attempting to start slapd with LD_ASSUME_KERNEL=2.4.1 doesn't help.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Norman Gaywood]

On Fri, Apr 02, 2004 at 11:31:06AM +0100, Nigel Wade wrote:
>  I guess I'm back to square one. The system won't operate in
> SMP with the FC 
>  kernel and won't operate at all without a FC kernel. So it looks like 
>  Fedora is snafu'd.
>  
>  Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, and when 
>  I tried an upgrade to openldap I started getting FUTEX locking problems. 
Install the RHEL3 kernel on FC1. I installed:

kernel-smp-2.4.21-9.0.1.EL
kernel-smp-unsupported-2.4.21-9.0.1.EL

from mirror.physics.ncsu.edu:/pub/whitebox/3.0/en/updates/i686

This has been working well for me. You need to use the rpm switches:

rpm --oldpackage -i kernel*

-- 
Norman Gaywood, Systems Administrator
School of Mathematics, Statistics and Computer Science
University of New England, Armidale, NSW 2351, Australia

norm(a)turing.une.edu.au            Phone: [phone]
http://turing.une.edu.au/~norm    Fax:   [phone]

Please avoid sending me Word or PowerPoint attachments.
See http://www.fsf.org/philosophy/no-word-attachments.html

[Nigel Wade]

Norman Gaywood wrote:
>  On Fri, Apr 02, 2004 at 11:31:06AM +0100, Nigel Wade wrote:
>  
> >I guess I'm back to square one. The system won't operate in SMP with the FC 
> >kernel and won't operate at all without a FC kernel. So it looks like 
> >Fedora is snafu'd.
> >
> >Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, and when 
> >I tried an upgrade to openldap I started getting FUTEX locking problems.
>  
>  
>  Install the RHEL3 kernel on FC1. I installed:
>  
>  kernel-smp-2.4.21-9.0.1.EL
>  kernel-smp-unsupported-2.4.21-9.0.1.EL
>  
>  from mirror.physics.ncsu.edu:/pub/whitebox/3.0/en/updates/i686
>   
Thanks, that's an interesting suggestion. I really don't want to install 
RPMS from an unknown source, but I'll have a go at building my own.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Nigel Wade]

Nigel Wade wrote:
>  Norman Gaywood wrote:
>  
> > On Fri, Apr 02, 2004 at 11:31:06AM +0100, Nigel Wade wrote:
> >
> >> I guess I'm back to square one. The system won't operate in SMP with 
> >> the FC kernel and won't operate at all without a FC kernel. So it 
> >> looks like Fedora is snafu'd.
> >>
> >> Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, 
> >> and when I tried an upgrade to openldap I started getting FUTEX 
> >> locking problems.
> >
> >
> >
> > Install the RHEL3 kernel on FC1. I installed:
> >
> > kernel-smp-2.4.21-9.0.1.EL
> > kernel-smp-unsupported-2.4.21-9.0.1.EL
> >
> > from mirror.physics.ncsu.edu:/pub/whitebox/3.0/en/updates/i686
> >
>  
>  Thanks, that's an interesting suggestion. I really don't want to install 
>  RPMS from an unknown source, but I'll have a go at building my own.
>   
Ok, the bottom line is that this doesn't work either.

I've built the RHEL3 kernel on FC1 and run the system up using this. 
Ordinary ldap requests seem quite happy, but I'm still seeing the futex lock 
problem with nss_ldap. E.g. when I add 'hosts: files ldap dns' to 
/etc/nsswitch.conf I get:

# strace ping host
munmap(0xb75ff000, 4096)                = 0
uname({sys="Linux", node="hostname", ...}) = 0
futex(0x53e4ec, FUTEX_WAIT, 2, NULL

Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but isn't a 
solution for a system relying on LDAP for all authentication and NSS functions.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Norman Gaywood]

On Tue, Apr 06, 2004 at 03:38:21PM +0100, Nigel Wade wrote:
>  Nigel Wade wrote:
>  >Norman Gaywood wrote:
>  >>On Fri, Apr 02, 2004 at 11:31:06AM +0100, Nigel Wade wrote:
>  >>>Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, 
>  >>>and when I tried an upgrade to openldap I started getting FUTEX 
>  >>>locking problems.
>  >>Install the RHEL3 kernel on FC1. I installed:
>  >>
>  >>kernel-smp-2.4.21-9.0.1.EL
>  >>kernel-smp-unsupported-2.4.21-9.0.1.EL
>  
>  Ok, the bottom line is that this doesn't work either.
>  
>  I've built the RHEL3 kernel on FC1 and run the system up using this. 
>  Ordinary ldap requests seem quite happy, but I'm still seeing the futex 
>  lock problem with nss_ldap. E.g. when I add 'hosts: files ldap dns' to 
>  /etc/nsswitch.conf I get:
>  
>  # strace ping host
>  munmap(0xb75ff000, 4096)                = 0
>  uname({sys="Linux", node="hostname", ...}) = 0
>  futex(0x53e4ec, FUTEX_WAIT, 2, NULL 
Interesting. I thought the EL kernel had the fast futex code back-ported
but I have not checked this. So I thought the EL kernel would work like
RH9 and FC1 kernels.

Could I ask what what steps you took to build the EL kernel on FC1? I
didn't have much luck building with rpmbuild when I tried this.

>  Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but
> isn't a 
>  solution for a system relying on LDAP for all authentication and NSS 
>  functions. 
Another suggestion for you to try is on one of the bugzillas. That is,
to rebuild the FC1 kernel with low latency scheduling turned off. Maybe
that's your best option.

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962

Comments #42 and #44

-- 
Norman Gaywood, Systems Administrator
School of Mathematics, Statistics and Computer Science
University of New England, Armidale, NSW 2351, Australia

norm(a)turing.une.edu.au            Phone: [phone]
http://turing.une.edu.au/~norm    Fax:   [phone]

Please avoid sending me Word or PowerPoint attachments.
See http://www.fsf.org/philosophy/no-word-attachments.html

[Nigel Wade]

Norman Gaywood wrote:
>  On Tue, Apr 06, 2004 at 03:38:21PM +0100, Nigel Wade wrote:
>  
> >Nigel Wade wrote:
> >
> >>Norman Gaywood wrote:
> >>
> >>>On Fri, Apr 02, 2004 at 11:31:06AM +0100, Nigel Wade wrote:
> >>>
> >>>>Unfortunately, RH9/RHEL3 don't have the version of LDAP I require, 
> >>>>and when I tried an upgrade to openldap I started getting FUTEX 
> >>>>locking problems.
> >>>
> >>>Install the RHEL3 kernel on FC1. I installed:
> >>>
> >>>kernel-smp-2.4.21-9.0.1.EL
> >>>kernel-smp-unsupported-2.4.21-9.0.1.EL
> >
> >Ok, the bottom line is that this doesn't work either.
> >
> >I've built the RHEL3 kernel on FC1 and run the system up using this. 
> >Ordinary ldap requests seem quite happy, but I'm still seeing the futex 
> >lock problem with nss_ldap. E.g. when I add 'hosts: files ldap dns' to 
> >/etc/nsswitch.conf I get:
> >
> ># strace ping host
> >munmap(0xb75ff000, 4096)                = 0
> >uname({sys="Linux", node="hostname", ...}) = 0
> >futex(0x53e4ec, FUTEX_WAIT, 2, NULL
>  
>  
>  Interesting. I thought the EL kernel had the fast futex code back-ported
>  but I have not checked this. So I thought the EL kernel would work like
>  RH9 and FC1 kernels. 
I've since found a bug listed in bugzilla specifically related to ping and 
futex - https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=110563
The workaround there is to build nss_ldap from PADL rather than use the FC1 
version. That will be my next step.

>  
>  Could I ask what what steps you took to build the EL kernel on FC1? I
>  didn't have much luck building with rpmbuild when I tried this. 
I built the kernel RPMs from src.rpm on a RH9 system, I couldn't get them to 
build on FC1. I then installed the kernel-source RPM on the FC1 system. 
After the ususal 'make mproper; cp configs/whatever .config; make oldconfig; 
make xconfig' I edited the Makefile and set it to use gcc32 rather than gcc. 
Then 'make dep; make bzImage; make modules; make modules_install; make 
install'. The reboot with the new kernel.

>  
>  
> >Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but isn't a 
> >solution for a system relying on LDAP for all authentication and NSS 
> >functions.
>  
>  
>  Another suggestion for you to try is on one of the bugzillas. That is,
>  to rebuild the FC1 kernel with low latency scheduling turned off. Maybe
>  that's your best option.
>  
>  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962
>  
>  Comments #42 and #44
>   
Ok, that's new since I last looked. I'll give it a go and see what happens.


-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Nigel Wade]

Norman Gaywood wrote:
>  On Tue, Apr 06, 2004 at 03:38:21PM +0100, Nigel Wade wrote: 
>  
>  Another suggestion for you to try is on one of the bugzillas. That is,
>  to rebuild the FC1 kernel with low latency scheduling turned off. Maybe
>  that's your best option.
>  
>  https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962
>  
>  Comments #42 and #44
>   
This could be the cure for the SMP/umount problem.

I'm running the "hangemhi" script (below in this thread) which continually 
mount/umounts a file via the loop device - it's passed the 1/2 million mark 
and counting...

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Dave Jones]

On Wed, 2004-04-07 at 15:17, Nigel Wade wrote:

>  > Another suggestion for you to try is on one of the bugzillas.
> That is,
>  > to rebuild the FC1 kernel with low latency scheduling turned off. Maybe
>  > that's your best option.
>  > 
>  > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962
>  > 
>  > Comments #42 and #44
>  This could be the cure for the SMP/umount problem. 
Yes, it is looking that way.

>  I'm running the "hangemhi" script (below in this
> thread) which continually 
>  mount/umounts a file via the loop device - it's passed the 1/2 million mark 
>  and counting... 
*nod* I've been running it all afternoon, and it seems to be behaving
itself on my dual p4 with ht whereas it previously
hung after 60 or so iterations.

For now I've just turned off the lowlat patch completely in CVS.
Its been a while since there's been a FC1 kernel update, so there's
quite a bit of stuff pending, so I'll try and get out an update
in the next week or so.

	Dave

[Jason L Tibbitts III]

> >>>> "DJ" == Dave Jones <davej(a)redhat.com&gt;
> writes: 
DJ> For now I've just turned off the lowlat patch completely in
DJ> CVS. Its been a while since there's been a FC1 kernel update, so
DJ> there's quite a bit of stuff pending, so I'll try and get out an
DJ> update in the next week or so.

Thanks; I had assumed that you folks were working exclusively on 2.6
now but it's good to know that at least one more FC1 kernel update
will be forthcoming.

Folks who need this fixed now but who want to keep everything in RPM
are welcome to pull the packages I built from
http://www.math.uh.edu/~tibbs/kernel-packages or follow the
instructions posted in bug 109962 and build your own.  It's not
difficult.

 - J<

[Jay Daniels]

On Thu, Apr 08, 2004 at 10:21:46AM -0500, Jason L Tibbitts III wrote:
>  >>>>> "DJ" == Dave Jones
> <davej(a)redhat.com&gt; writes:
>  
>  DJ> For now I've just turned off the lowlat patch completely in
>  DJ> CVS. Its been a while since there's been a FC1 kernel update, so
>  DJ> there's quite a bit of stuff pending, so I'll try and get out an
>  DJ> update in the next week or so.
>  
>  Thanks; I had assumed that you folks were working exclusively on 2.6
>  now but it's good to know that at least one more FC1 kernel update
>  will be forthcoming.
>  
>  Folks who need this fixed now but who want to keep everything in RPM
>  are welcome to pull the packages I built from
>  http://www.math.uh.edu/~tibbs/kernel-packages or follow the
>  instructions posted in bug 109962 and build your own.  It's not
>  difficult.
>  
>   - J< 
Downloading, it's late - will try your kernel-smp in the morn.

thanks,

jay

[Jay Daniels]

On Fri, Apr 09, 2004 at 02:03:06AM -0400, Jay Daniels wrote:
>  On Thu, Apr 08, 2004 at 10:21:46AM -0500, Jason L Tibbitts III
> wrote:
>  > >>>>> "DJ" == Dave Jones <davej(a)redhat.com&gt; writes:
>  > 
>  > DJ> For now I've just turned off the lowlat patch completely in
>  > DJ> CVS. Its been a while since there's been a FC1 kernel update, so
>  > DJ> there's quite a bit of stuff pending, so I'll try and get out an
>  > DJ> update in the next week or so.
>  > 
>  > Thanks; I had assumed that you folks were working exclusively on 2.6
>  > now but it's good to know that at least one more FC1 kernel update
>  > will be forthcoming.
>  > 
>  > Folks who need this fixed now but who want to keep everything in RPM
>  > are welcome to pull the packages I built from
>  > http://www.math.uh.edu/~tibbs/kernel-packages or follow the
>  > instructions posted in bug 109962 and build your own.  It's not
>  > difficult.
>  > 
>  >  - J<
>  
>  Downloading, it's late - will try your kernel-smp in the morn.
>  
>  thanks,
>  
>  jay 

Didn't work, hanged while loading modules just like the stock fedora
core 1 smp kernel!

Where do I need to look to find out what the problem is with this dual
xeon processor box that won't run smp kernels?

Stock single processor kernel works great.  All smp kernels I've tried
on this box crashes just after checking /tmp and I believe it's
happening when loading modules.

It's a supermicro superserver board with dual xeon 2.66 ghz and 2GB of
ram.  The board model is a MBD-X5DEI-GG-0

Has anyone tested the latest Fedora Core 2 test smp kernels on such a
system?   If that those smp kernels work, I see no need to compile
from source.

Also note that the Redhat 9.0 smp kernel worked fine.


jay

[Jason L Tibbitts III]

> >>>> "JD" == Jay Daniels
> <drs(a)pointyhats.com&gt; writes: 
JD> Didn't work, hanged while loading modules just like the stock
JD> fedora core 1 smp kernel!

Hmmm, that wasn't the problem I built those kernels to fix so it seems
you have a separate issue.  They eliminated the hangs when unmounting
filesystems.

I know my systems will hang hard early in boot if I boot with acpi=on,
but I have little need for ACPI so it's not a big deal for me to leave
it off.  This kernel has no effect on that problem for me.  I haven't
tried to debug things further.

JD> Where do I need to look to find out what the problem is with this
JD> dual xeon processor box that won't run smp kernels?

Sorry, no idea.  Can you get anything out of the kernel via Alt-SysRQ?

JD> It's a supermicro superserver board with dual xeon 2.66 ghz and
JD> 2GB of ram.  The board model is a MBD-X5DEI-GG-0

Hmm, I'm running SuperMicro X5DPE-G2 boards, dual Xeon 2.80GHz with
4GB of RAM.  Perhaps we're seeing something similar.  The last kernel
message I see is a message about disk quotas.  I'll have to play with
one of them to see if I can make anything happen.

JD> Has anyone tested the latest Fedora Core 2 test smp kernels on
JD> such a system?

Sorry, I've got my hands full but I do have a full mirror so I could
try an upgrade.  It's pretty certain that with those kernels you'll
see an entirely different set of problems.  I doubt you can run the
kernels without upgrading the whole system, but I'll try to build
what's in rawhide now and see.

Later: Nope, I can't seem to build kernel-2.6.5-1.309; it dies trying
to do "make nonint_oldconfig" because there's no target in the
Makefile for it.  Hmmm.

 - J<

[Herbert Gasiorowski]

>  I've built the RHEL3 kernel on FC1 and run the system up using
> this. 
>  Ordinary ldap requests seem quite happy, but I'm still seeing the futex 
>  lock problem with nss_ldap. E.g. when I add 'hosts: files ldap dns' to 
>  /etc/nsswitch.conf I get:
>  
>  # strace ping host
>  munmap(0xb75ff000, 4096)                = 0
>  uname({sys="Linux", node="hostname", ...}) = 0
>  futex(0x53e4ec, FUTEX_WAIT, 2, NULL
>  
>  Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but isn't 
>  a solution for a system relying on LDAP for all authentication and NSS 
>  functions. 
The problem remains in FC2 (Test 3) with the 2.6 kernel!

LD_ASSUME_KERNEL=2.4.1 seems to cure at least "ping host"


-- 
mfg
Herbert Gasiorowski
http://www.mathematik.uni-marburg.de/~gasi/

[Nigel Wade]

Herbert Gasiorowski wrote:
> > I've built the RHEL3 kernel on FC1 and run the system up
> using this. 
> > Ordinary ldap requests seem quite happy, but I'm still seeing the 
> > futex lock problem with nss_ldap. E.g. when I add 'hosts: files ldap 
> > dns' to /etc/nsswitch.conf I get:
> >
> > # strace ping host
> > munmap(0xb75ff000, 4096)                = 0
> > uname({sys="Linux", node="hostname", ...}) = 0
> > futex(0x53e4ec, FUTEX_WAIT, 2, NULL
> >
> > Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but 
> > isn't a solution for a system relying on LDAP for all authentication 
> > and NSS functions.
>  
>  
>  The problem remains in FC2 (Test 3) with the 2.6 kernel!
>  
>  LD_ASSUME_KERNEL=2.4.1 seems to cure at least "ping host"
>  
>   
This is in some way related to nscd.

If nscd isn't running hosts don't resolve using LDAP. AFAIK, nscd should not 
be compulsory.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Herbert Gasiorowski]

Nigel Wade wrote:
>  Herbert Gasiorowski wrote:
>  
> >> I've built the RHEL3 kernel on FC1 and run the system up using this. 
> >> Ordinary ldap requests seem quite happy, but I'm still seeing the 
> >> futex lock problem with nss_ldap. E.g. when I add 'hosts: files ldap 
> >> dns' to /etc/nsswitch.conf I get:
> >>
> >> # strace ping host
> >> munmap(0xb75ff000, 4096)                = 0
> >> uname({sys="Linux", node="hostname", ...}) = 0
> >> futex(0x53e4ec, FUTEX_WAIT, 2, NULL
> >>
> >> Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but 
> >> isn't a solution for a system relying on LDAP for all authentication 
> >> and NSS functions.
> >
> >
> >
> > The problem remains in FC2 (Test 3) with the 2.6 kernel!
> >
> > LD_ASSUME_KERNEL=2.4.1 seems to cure at least "ping host"
> >
> >
>  
>  This is in some way related to nscd.
>  
>  If nscd isn't running hosts don't resolve using LDAP. AFAIK, nscd should 
>  not be compulsory.
>   
Thanks, this seems to solve my problem and finally I managed to install nscd!

-- 
mfg
Herbert Gasiorowski
http://www.mathematik.uni-marburg.de/~gasi/

[Nigel Wade]

Herbert Gasiorowski wrote:
>  Nigel Wade wrote:
>  
> > Herbert Gasiorowski wrote:
> >
> >>> I've built the RHEL3 kernel on FC1 and run the system up using this. 
> >>> Ordinary ldap requests seem quite happy, but I'm still seeing the 
> >>> futex lock problem with nss_ldap. E.g. when I add 'hosts: files ldap 
> >>> dns' to /etc/nsswitch.conf I get:
> >>>
> >>> # strace ping host
> >>> munmap(0xb75ff000, 4096)                = 0
> >>> uname({sys="Linux", node="hostname", ...}) = 0
> >>> futex(0x53e4ec, FUTEX_WAIT, 2, NULL
> >>>
> >>> Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but 
> >>> isn't a solution for a system relying on LDAP for all authentication 
> >>> and NSS functions.
> >>
> >>
> >>
> >>
> >> The problem remains in FC2 (Test 3) with the 2.6 kernel!
> >>
> >> LD_ASSUME_KERNEL=2.4.1 seems to cure at least "ping host"
> >>
> >>
> >
> > This is in some way related to nscd.
> >
> > If nscd isn't running hosts don't resolve using LDAP. AFAIK, nscd 
> > should not be compulsory.
> >
>  
>  Thanks, this seems to solve my problem and finally I managed to install 
>  nscd!
>   
I do this, but I have to say I don't like it. It's very precarious - if nscd 
hangs or crashes for some reason the m/c is unusable. You can't even login 
as root at the console to fix nscd if host resolution is hanging. Eventually 
pretty much everything hangs.


-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

ldap: ping does not return

[Herbert Gasiorowski]

 From "Re: Latest FC kernel still have SMP bugs?"

Nigel Wade wrote:
>  Herbert Gasiorowski wrote:
>  
> > Nigel Wade wrote:
> >
> >> Herbert Gasiorowski wrote:
> >>
> >>>> I've built the RHEL3 kernel on FC1 and run the system up using 
> >>>> this. Ordinary ldap requests seem quite happy, but I'm still seeing 
> >>>> the futex lock problem with nss_ldap. E.g. when I add 'hosts: files 
> >>>> ldap dns' to /etc/nsswitch.conf I get:
> >>>>
> >>>> # strace ping host
> >>>> munmap(0xb75ff000, 4096)                = 0
> >>>> uname({sys="Linux", node="hostname", ...}) = 0
> >>>> futex(0x53e4ec, FUTEX_WAIT, 2, NULL
> >>>>
> >>>> Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but 
> >>>> isn't a solution for a system relying on LDAP for all 
> >>>> authentication and NSS functions.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> The problem remains in FC2 (Test 3) with the 2.6 kernel!
> >>>
> >>> LD_ASSUME_KERNEL=2.4.1 seems to cure at least "ping host"
> >>>
> >>>
> >>
> >> This is in some way related to nscd.
> >>
> >> If nscd isn't running hosts don't resolve using LDAP. AFAIK, nscd 
> >> should not be compulsory.
> >>
> >
> > Thanks, this seems to solve my problem and finally I managed to 
> > install nscd!
> >
>  
>  I do this, but I have to say I don't like it. It's very precarious - if 
>  nscd hangs or crashes for some reason the m/c is unusable. You can't 
>  even login as root at the console to fix nscd if host resolution is 
>  hanging. Eventually pretty much everything hangs. 
Actually it does not solve my problem: I need to use "hosts: files ldap dns"
in nsswitch.conf, but then again ping never returns and nscd crashes.
(It only works with "hosts: files dns ldap" and nscd)

-- 
mfg
Herbert Gasiorowski
http://www.mathematik.uni-marburg.de/~gasi/

ldap: ping does not return

[Nigel Wade]

Herbert Gasiorowski wrote:
>   From "Re: Latest FC kernel still have SMP bugs?"
>  
>  Nigel Wade wrote:
>  
> > Herbert Gasiorowski wrote:
> >
> >> Nigel Wade wrote:
> >>
> >>> Herbert Gasiorowski wrote:
> >>>
> >>>>> I've built the RHEL3 kernel on FC1 and run the system up using 
> >>>>> this. Ordinary ldap requests seem quite happy, but I'm still 
> >>>>> seeing the futex lock problem with nss_ldap. E.g. when I add 
> >>>>> 'hosts: files ldap dns' to /etc/nsswitch.conf I get:
> >>>>>
> >>>>> # strace ping host
> >>>>> munmap(0xb75ff000, 4096)                = 0
> >>>>> uname({sys="Linux", node="hostname", ...}) = 0
> >>>>> futex(0x53e4ec, FUTEX_WAIT, 2, NULL
> >>>>>
> >>>>> Setting LD_ASSUME_KERNEL=2.4.1 cures this particular problem, but 
> >>>>> isn't a solution for a system relying on LDAP for all 
> >>>>> authentication and NSS functions.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> The problem remains in FC2 (Test 3) with the 2.6 kernel!
> >>>>
> >>>> LD_ASSUME_KERNEL=2.4.1 seems to cure at least "ping host"
> >>>>
> >>>>
> >>>
> >>> This is in some way related to nscd.
> >>>
> >>> If nscd isn't running hosts don't resolve using LDAP. AFAIK, nscd 
> >>> should not be compulsory.
> >>>
> >>
> >> Thanks, this seems to solve my problem and finally I managed to 
> >> install nscd!
> >>
> >
> > I do this, but I have to say I don't like it. It's very precarious - 
> > if nscd hangs or crashes for some reason the m/c is unusable. You 
> > can't even login as root at the console to fix nscd if host resolution 
> > is hanging. Eventually pretty much everything hangs.
>  
>  
>  Actually it does not solve my problem: I need to use "hosts: files ldap 
>  dns"
>  in nsswitch.conf, but then again ping never returns and nscd crashes.
>  (It only works with "hosts: files dns ldap" and nscd)
>   
I just added a reply in Bugzilla to your report of this.

Here, I can get nscd to work with "host: files ldap dns", but I had to 
restart it after I changed nsswitch.conf.


-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

ldap: ping does not return

[Herbert Gasiorowski]

>  
>  I just added a reply in Bugzilla to your report of this.
>  
>  Here, I can get nscd to work with "host: files ldap dns", but I had to 
>  restart it after I changed nsswitch.conf.
>  
>   
Ok: I finally got it (hopefully):

I missed that some libldap does a reverse lookup on the
ip number in ldap.conf (or the like).

So I need to add the ldap-server in /etc/hosts avoiding
the endless loop (and nscd so ping will not stop in futex).

-- 
mfg
Herbert Gasiorowski
http://www.mathematik.uni-marburg.de/~gasi/

ldap: ping does not return

[Nigel Wade]

Herbert Gasiorowski wrote:
> >
> > I just added a reply in Bugzilla to your report of this.
> >
> > Here, I can get nscd to work with "host: files ldap dns", but I had to 
> > restart it after I changed nsswitch.conf.
> >
> >
>  
>  Ok: I finally got it (hopefully):
>  
>  I missed that some libldap does a reverse lookup on the
>  ip number in ldap.conf (or the like).
>  
>  So I need to add the ldap-server in /etc/hosts avoiding
>  the endless loop (and nscd so ping will not stop in futex).
>   
If you are going to use LDAP for host resolution you should use an IP number 
rather than a hostname in authconfig. Or, to change it later, edit the entry 
in /etc/ldap.conf and /etc/openldap/ldap.conf and replace the hostname with 
its IP number.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Luc Bouchard]

On Fri, 2004-04-02 at 04:18, Nigel Wade wrote:

>  My system crashed at 23:35. The cron job to mount/unmount filesystems
> was 
>  set to run at :10 and :35 every hour. Given what other people report in this 
>  and other threads I'd say either mount or unmount was the culprit.
>  
>  I think I'll be building my own kernel now. 
Mount/unmount seems to be the problem with our SMP boxes as well.  The
machine runs fine until I try to umount something (i.e. the CDROM). 
This is reproducible on each box when running SMP kernels.  Next we are
trying to disable HT on the servers.  The hardware is HP Proliant DL360
w/ dual Xeon 2.4GHz CPUs.  This behaviour does not occur on our old
Proliant 1600 w/ Dual PIII 600.  Also the old box does not use X at
all.  The new boxes do.  YMMV

Luc Bouchard

[Matt Morgan]

On 04/01/2004 09:48 AM, Mitch Wiedemann wrote:

>  Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP 
>  and hanging? 
Seems to be true on my PC.

>  I'm due to install some MP servers soon and want to avoid
> problems 
>  (imagine that!).
>
>  If said kernel is still having trouble, what's the easiest fix?
>  
I have an Intel Hyperthreading 2.8Ghz Pentium 4 on some no-name box, 
with an Asus motherboard. I used to crash daily with the SMP kernel 
(vmlinuz-2.4.22-1.2174.nptlsmp), immediately after logon (basically I 
could logon once, after each reboot, before logging in became 
impossible). It also crashed randomly throughout the day. Turning off 
yum nightly updates helped a little; I think it fixed the logon problem 
but not the random-crashes problem (not sure though!). Additionally 
turning off acpi helped a lot, but I still crashed once every three or 
four days (did not leave it in this state very long, though).

Using the non-smp kernel, vmlinuz-2.4.22-1.2174.nptl, works really well 
(no crashing yet, in about 10 days), even with acpi and yum nightly 
updates on. So I just use the non-smp kernel. This is a pretty fast 
computer and I can't really tell any difference in speed--but I'm using 
it for a workstation, not a server, and it's a lot more powerful than I 
need for email, browsing, and office apps.

--Matt

[Jean-Rene Cormier]

On Thu, 2004-04-01 at 10:48, Mitch Wiedemann wrote:
>  Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP
> and 
>  hanging?
>  I'm due to install some MP servers soon and want to avoid problems 
>  (imagine that!). 
On my Dell Precision 530 with dual Xeon 1.5GHz I have no problem with
the SMP but I don't have HT on this computer. On a Dell PowerEdge 2650
with dual Xeon 2GHz, the computer locks at boot time with the SMP kernel
but with the non-smp one it works just fine. Right now I'm running it
with the non-smp kernel until a fix comes out.

Jean-Rene Cormier

[Chris Miller]

On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
>  Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP
> and 
>  hanging?
>  I'm due to install some MP servers soon and want to avoid problems 
>  (imagine that!). 
I run around 100 SMP servers with no problem.  

Hardware.
Dual Xeon servers with HT on supermicro server boards.
Dual AMD on Tyan 2469,2468,2462
Dual Opteron on Tyan 2880



And I do run the fedora kernel.  Anything I can do to help let me know. 
But I have never seen any of the problems anyone is talking about.




>  If said kernel is still having trouble, what's the easiest fix?
>  
>  Thanks in advance.
>  
>  -- 
>  Mitch Wiedemann
>  mc^2 Computer Consulting
>  mc2(a)lightlink.com
>  http://www.lightlink.com/mc2
>

[WipeOut]

Chris Miller wrote:

> On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
>   
>
> >Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP and 
> >hanging?
> >I'm due to install some MP servers soon and want to avoid problems 
> >(imagine that!).
> >    
> >
>
> I run around 100 SMP servers with no problem.  
>
> Hardware.
> Dual Xeon servers with HT on supermicro server boards.
> Dual AMD on Tyan 2469,2468,2462
> Dual Opteron on Tyan 2880
>
>
>
> And I do run the fedora kernel.  Anything I can do to help let me know. 
> But I have never seen any of the problems anyone is talking about.
>
>
>   
>  
I am guessing your systems were minimum installs withe the required 
components added after install or you did a server install of some sort 
without installing X..

Here is my experience, I have two systems, both are P4 2.4G HT systems..

System one has a minimum install with the required software added 
(apache, mysql, php, etc).. This system is perfectly stable with the 
standard FC SMP kernels..

System two has a workstation install and will freeze when cron or 
anacron run the respective routines... It will ocasionally hang at other 
times too..

My solution on System two was to build a kernel from the vanilla source 
on kernel.org and since then I have had a perfectly stable PC..

Later..

[Norman Gaywood]

On Thu, Apr 01, 2004 at 04:31:57PM -0500, Chris Miller wrote:
>  On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
>  > Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP and 
>  > hanging?
>  
>  I run around 100 SMP servers with no problem.  
>  
>  Hardware.
>  Dual Xeon servers with HT on supermicro server boards.
>  Dual AMD on Tyan 2469,2468,2462
>  Dual Opteron on Tyan 2880
>  
>  And I do run the fedora kernel.  Anything I can do to help let me know. 
>  But I have never seen any of the problems anyone is talking about. 
I'd be interested if you could run the script from:

http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962#c40

and see how you go. The script creates a small file in /tmp, makes a
filesystem image out of that file, and then continuously does a
loop-back mount/umount until you get sick of it and ^C out.

Any FC1 SMP kernel I have used on several types of systems, including
AMD single processor systems running the athlon SMP kernel, will lockup
in less than 500 mount/umounts. Usually much faster.

Other kernels can do mount/umounts for as long as you want. I've gone
upto 20,000 before I ^C out.

This is the script:

#!/bin/sh

NAME="hangemhi"
SIZE=6000

MNTPOINT=/tmp/${NAME}
IMAGE=/tmp/${NAME}.img

echo Creating something to mount
dd if=/dev/zero of=${IMAGE} bs=1k count=${SIZE}
mke2fs -F -q ${IMAGE} ${SIZE}

echo Creating the mount point
mkdir -p ${MNTPOINT}

echo Now doing mount/umount until the system hangs
COUNT=1
while true
do
  echo -en ${COUNT} "\r"
  mount -r -o loop -t ext2 ${IMAGE} ${MNTPOINT}
  umount ${MNTPOINT}
  COUNT=`expr ${COUNT} + 1`
done


-- 
Norman Gaywood, Systems Administrator
School of Mathematics, Statistics and Computer Science
University of New England, Armidale, NSW 2351, Australia

norm(a)turing.une.edu.au            Phone: [phone]
http://turing.une.edu.au/~norm    Fax:   [phone]

Please avoid sending me Word or PowerPoint attachments.
See http://www.fsf.org/philosophy/no-word-attachments.html

[Chris Miller]

37700 mount/unmount and still going.   I will let it run on my test box
here at my office.  And give a update to my findings.  But let me ask
you what file system do you use on your boxs?

On Thu, 2004-04-01 at 19:21, Norman Gaywood wrote:
>  On Thu, Apr 01, 2004 at 04:31:57PM -0500, Chris Miller wrote:
>  > On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
>  > > Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP and 
>  > > hanging?
>  > 
>  > I run around 100 SMP servers with no problem.  
>  > 
>  > Hardware.
>  > Dual Xeon servers with HT on supermicro server boards.
>  > Dual AMD on Tyan 2469,2468,2462
>  > Dual Opteron on Tyan 2880
>  > 
>  > And I do run the fedora kernel.  Anything I can do to help let me know. 
>  > But I have never seen any of the problems anyone is talking about.
>  
>  I'd be interested if you could run the script from:
>  
>  http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962#c40
>  
>  and see how you go. The script creates a small file in /tmp, makes a
>  filesystem image out of that file, and then continuously does a
>  loop-back mount/umount until you get sick of it and ^C out.
>  
>  Any FC1 SMP kernel I have used on several types of systems, including
>  AMD single processor systems running the athlon SMP kernel, will lockup
>  in less than 500 mount/umounts. Usually much faster.
>  
>  Other kernels can do mount/umounts for as long as you want. I've gone
>  upto 20,000 before I ^C out.
>  
>  This is the script:
>  
>  #!/bin/sh
>  
>  NAME="hangemhi"
>  SIZE=6000
>  
>  MNTPOINT=/tmp/${NAME}
>  IMAGE=/tmp/${NAME}.img
>  
>  echo Creating something to mount
>  dd if=/dev/zero of=${IMAGE} bs=1k count=${SIZE}
>  mke2fs -F -q ${IMAGE} ${SIZE}
>  
>  echo Creating the mount point
>  mkdir -p ${MNTPOINT}
>  
>  echo Now doing mount/umount until the system hangs
>  COUNT=1
>  while true
>  do
>    echo -en ${COUNT} "\r"
>    mount -r -o loop -t ext2 ${IMAGE} ${MNTPOINT}
>    umount ${MNTPOINT}
>    COUNT=`expr ${COUNT} + 1`
>  done
>  
>  
>  -- 
>  Norman Gaywood, Systems Administrator
>  School of Mathematics, Statistics and Computer Science
>  University of New England, Armidale, NSW 2351, Australia
>  
>  norm(a)turing.une.edu.au            Phone: [phone]
>  http://turing.une.edu.au/~norm    Fax:   [phone]
>  
>  Please avoid sending me Word or PowerPoint attachments.
>  See http://www.fsf.org/philosophy/no-word-attachments.html
>

[Chris Miller]

Made it to 85799 then the box went down.  With the fedora kernel and
Reiserfs.

On Thu, 2004-04-01 at 21:53, Chris Miller wrote:
>  37700 mount/unmount and still going.   I will let it run on my test
> box
>  here at my office.  And give a update to my findings.  But let me ask
>  you what file system do you use on your boxs?
>  
>  On Thu, 2004-04-01 at 19:21, Norman Gaywood wrote:
>  > On Thu, Apr 01, 2004 at 04:31:57PM -0500, Chris Miller wrote:
>  > > On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
>  > > > Is the FC1 2.4.22-1.2174.nptl kernel still having problems with SMP and 
>  > > > hanging?
>  > > 
>  > > I run around 100 SMP servers with no problem.  
>  > > 
>  > > Hardware.
>  > > Dual Xeon servers with HT on supermicro server boards.
>  > > Dual AMD on Tyan 2469,2468,2462
>  > > Dual Opteron on Tyan 2880
>  > > 
>  > > And I do run the fedora kernel.  Anything I can do to help let me know. 
>  > > But I have never seen any of the problems anyone is talking about.
>  > 
>  > I'd be interested if you could run the script from:
>  > 
>  > http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962#c40
>  > 
>  > and see how you go. The script creates a small file in /tmp, makes a
>  > filesystem image out of that file, and then continuously does a
>  > loop-back mount/umount until you get sick of it and ^C out.
>  > 
>  > Any FC1 SMP kernel I have used on several types of systems, including
>  > AMD single processor systems running the athlon SMP kernel, will lockup
>  > in less than 500 mount/umounts. Usually much faster.
>  > 
>  > Other kernels can do mount/umounts for as long as you want. I've gone
>  > upto 20,000 before I ^C out.
>  > 
>  > This is the script:
>  > 
>  > #!/bin/sh
>  > 
>  > NAME="hangemhi"
>  > SIZE=6000
>  > 
>  > MNTPOINT=/tmp/${NAME}
>  > IMAGE=/tmp/${NAME}.img
>  > 
>  > echo Creating something to mount
>  > dd if=/dev/zero of=${IMAGE} bs=1k count=${SIZE}
>  > mke2fs -F -q ${IMAGE} ${SIZE}
>  > 
>  > echo Creating the mount point
>  > mkdir -p ${MNTPOINT}
>  > 
>  > echo Now doing mount/umount until the system hangs
>  > COUNT=1
>  > while true
>  > do
>  >   echo -en ${COUNT} "\r"
>  >   mount -r -o loop -t ext2 ${IMAGE} ${MNTPOINT}
>  >   umount ${MNTPOINT}
>  >   COUNT=`expr ${COUNT} + 1`
>  > done
>  > 
>  > 
>  > -- 
>  > Norman Gaywood, Systems Administrator
>  > School of Mathematics, Statistics and Computer Science
>  > University of New England, Armidale, NSW 2351, Australia
>  > 
>  > norm(a)turing.une.edu.au            Phone: [phone]
>  > http://turing.une.edu.au/~norm    Fax:   [phone]
>  > 
>  > Please avoid sending me Word or PowerPoint attachments.
>  > See http://www.fsf.org/philosophy/no-word-attachments.html
>  > 
>

[Chris Rouch]

On Fri, 2 Apr 2004 10:21:23 +1000
Norman Gaywood <norm(a)turing.une.edu.au&gt; wrote:

>  On Thu, Apr 01, 2004 at 04:31:57PM -0500, Chris Miller wrote:
>  > On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
>  > > Is the FC1 2.4.22-1.2174.nptl kernel still having problems with
>  > > SMP and hanging?
>  > 
>  > I run around 100 SMP servers with no problem.  
>  > 
>  > Hardware.
>  > Dual Xeon servers with HT on supermicro server boards.
>  > Dual AMD on Tyan 2469,2468,2462
>  > Dual Opteron on Tyan 2880
>  > 
>  > And I do run the fedora kernel.  Anything I can do to help let me
>  > know. But I have never seen any of the problems anyone is talking
>  > about.
>  
>  I'd be interested if you could run the script from:
>  
>  http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962#c40
>  
>  and see how you go. The script creates a small file in /tmp, makes a
>  filesystem image out of that file, and then continuously does a
>  loop-back mount/umount until you get sick of it and ^C out.
>  
>  Any FC1 SMP kernel I have used on several types of systems, including
>  AMD single processor systems running the athlon SMP kernel, will
>  lockup in less than 500 mount/umounts. Usually much faster.
>   
Our servers have ~10 partitions mounted, all local ext3 and have no
problems with the SMP kernel. However the workstations  (with
hyperthreading switched on) automount some of their disks from the
servers. They tend to lock up after a few days. So one workaround
appears to be to use fixed nfs mounts at boot time.

FWIW, running the script, the workstations would lock up between 1000
and 1400 mounts.


Regards,

Chris

[WipeOut]

Chris Rouch wrote:

> On Fri, 2 Apr 2004 10:21:23 +1000
> Norman Gaywood <norm(a)turing.une.edu.au&gt; wrote:
>
>   
>
> >On Thu, Apr 01, 2004 at 04:31:57PM -0500, Chris Miller wrote:
> >    
> >
> >>On Thu, 2004-04-01 at 09:48, Mitch Wiedemann wrote:
> >>      
> >>
> >>>Is the FC1 2.4.22-1.2174.nptl kernel still having problems with
> >>>SMP and hanging?
> >>>        
> >>>
> >>I run around 100 SMP servers with no problem.  
> >>
> >>Hardware.
> >>Dual Xeon servers with HT on supermicro server boards.
> >>Dual AMD on Tyan 2469,2468,2462
> >>Dual Opteron on Tyan 2880
> >>
> >>And I do run the fedora kernel.  Anything I can do to help let me
> >>know. But I have never seen any of the problems anyone is talking
> >>about.
> >>      
> >>
> >I'd be interested if you could run the script from:
> >
> >http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=109962#c40
> >
> >and see how you go. The script creates a small file in /tmp, makes a
> >filesystem image out of that file, and then continuously does a
> >loop-back mount/umount until you get sick of it and ^C out.
> >
> >Any FC1 SMP kernel I have used on several types of systems, including
> >AMD single processor systems running the athlon SMP kernel, will
> >lockup in less than 500 mount/umounts. Usually much faster.
> >
> >    
> >
>
> Our servers have ~10 partitions mounted, all local ext3 and have no
> problems with the SMP kernel. However the workstations  (with
> hyperthreading switched on) automount some of their disks from the
> servers. They tend to lock up after a few days. So one workaround
> appears to be to use fixed nfs mounts at boot time.
>
> FWIW, running the script, the workstations would lock up between 1000
> and 1400 mounts.
>
>
> Regards,
>
> Chris
>
>
>
>   
>  
I personally don't think it has anything to do with NFS mounts, I don't 
have any but the still have the lockup's on my HT desktop.. No problem 
with a vanilla SMP kernel..

[Norman Gaywood]

On Fri, Apr 02, 2004 at 10:57:11AM +0100, WipeOut wrote:
>  I personally don't think it has anything to do with NFS mounts, I
> don't 
>  have any but the still have the lockup's on my HT desktop.. No problem 
>  with a vanilla SMP kernel.. 
Actually, the trigger for the bug is a umount not a mount. It also seems
to be be any type of umount. The test script mentioned earlier in the
thread does a loop-back mount/umount.

There may be other ways to trigger it other than umount.

-- 
Norman Gaywood, Systems Administrator
School of Mathematics, Statistics and Computer Science
University of New England, Armidale, NSW 2351, Australia

norm(a)turing.une.edu.au            Phone: [phone]
http://turing.une.edu.au/~norm    Fax:   [phone]

Please avoid sending me Word or PowerPoint attachments.
See http://www.fsf.org/philosophy/no-word-attachments.html