Re: svn commit: r237367 - head/sys/fs/nfsclient

On 01.07.12 23:43, Konstantin Belousov wrote:
> On Sun, Jul 01, 2012 at 08:33:09PM +0200, Andreas Tobler wrote:
>> On 01.07.12 19:05, Konstantin Belousov wrote:
>>> On Sun, Jul 01, 2012 at 03:56:52PM +0200, Andreas Tobler wrote:
>>>> On 01.07.12 15:41, Konstantin Belousov wrote:
>>>>> On Sun, Jul 01, 2012 at 03:37:18PM +0200, Andreas Tobler wrote:
>>>>>> On 01.07.12 14:04, Konstantin Belousov wrote:
>>>>>>> On Sun, Jul 01, 2012 at 01:23:02PM +0200, Andreas Tobler wrote:
>>>>>>>> On 21.06.12 11:26, Konstantin Belousov wrote:
>>>>>>>>> Author: kib
>>>>>>>>> Date: Thu Jun 21 09:26:06 2012
>>>>>>>>> New Revision: 237367
>>>>>>>>> URL: http://svn.freebsd.org/changeset/base/237367
>>>>>>>>>
>>>>>>>>> Log:
>>>>>>>>>    Enable deadlock avoidance code for NFS client.
>>>>>>>>
>>>>>>>>
>>>>>>>> Hm, since this commit I fail with my nfs installworld/kernel.
>>>>>>>>
>>>>>>>> I have a builder which installs world/kernel to a nfs mounted
>>>>>>>> directory.
>>>>>>>> Namely used for cross builds.
>>>>>>>>
>>>>>>>> Now since this commit I get the following when I install kernel to the
>>>>>>>> nfs directory:
>>>>>>>>
>>>>>>>> ..
>>>>>>>> install -o root -g wheel -m 555   zfs.ko.symbols
>>>>>>>> /netboot/sparc64/boot/kernel
>>>>>>>> install: /netboot/sparc64/boot/kernel/zfs.ko.symbols: No such file or
>>>>>>>> directory
>>>>>>>> *** [_kmodinstall] Error code 71
>>>>>>>> ..
>>>>>>>>
>>>>>>>> The file is there, a local install of the tree works without problems.
>>>>>>>> Reverting to r237366 also makes it work again.
>>>>>>>>
>>>>>>>> The server is a -CURRENT, r237880, The client, -CURRENT too.
>>>>>>>>
>>>>>>>> How can I help to track down the real issue?
>>>>>>>
>>>>>>> Is it always the same file in the install procedure which causes the
>>>>>>> failure ? Even more, is the failure pattern always the same ?
>>>>>>
>>>>>> I'd say so yes. When installing a kernel onto a nfs mounted fs then
>>>>>> always (in my cases) the zfs.ko.symbols was the failing pattern.
>>>>>> I tried ppc64 and sparc64 as target. With both it was the above file.
>>>>>>
>>>>>> When doing a installworld, it was, also in both cases, ppc64/sparc64,
>>>>>> the cc1 in libexec which failed.
>>>>>>
>>>>>>> Might be, start with ktrace-ing the whole make invocation, including
>>>>>>> the children processes.
>>>>>>
>>>>>> Some recipes how to start?
>>>>> ktrace -o <file on local fs> -i make installkernel
>>>>> Then kdump and cut the lines around relevant failure.
>>>>
>>>> ktrace -f, right?
>>> Right, but without -i it is useless.
>>
>> Ah, yes, seems clear now after reading the man page.
>>
>>>> I placed the whole kdump here:
>>>>
>>>> http://people.freebsd.org/~andreast/dumped_installkernel.log
>>>>
>>>> It is not clear to me where the failure starts :)
>>> Because logs do not contain tracepoints from the children.
>>> See above about -i.
>>>
>>> I asked about excerpt because I expect the proper log to have an order
>>> of magnitude bigger size.
>>
>> Ok. The dump is around 100MB, I hope I extracted as much as needed:
>>
>> http://people.freebsd.org/~andreast/dumped_installkernel-7.log
>>
>>>>>>> I used buildworld on the NFS-mounted obj/ as the test for the changes.
>>>>>>
>>>>>> Here the obj is local, only the src and the destination is on the
>>>>>> nfs/netboot server.
>>>>>
>>>>> I just finished build on NFS obj/ and did several rounds of installs
>>>>> for world and kernel into nfs-mounted destdir. It seems I cannot
>>>>> reproduce
>>>>> this locally.
>>>>
>>>> Ok. I try with an nfs obj too.
>>
>> So, I was not able to reproduce the failure with an nfs mounted obj dir.
>>
>> But I was able to reproduce the failure with three different machines
>> which all have the obj local and the destination mounted via nfs.
>>
>> Are you able to try with a local obj too?
> Below are two patches. Please follow my instructions literally to get
> most of your bug report.
>
> First, please apply the usr.bin/xinstall patch only, and retry installkernel
> (no need to use ktrace). It should show the proper error, short write, with
> zero-sized result, instead of garbage ENOENT from errno.

Done. No expected output. Iow, no message containing this:

short write to %s: %jd bytes written, %jd bytes asked to write

> Next, please apply the sys/fs/nfsclient patch, which should fix the core
> cause.

Still the same failure:
===> zfs (install)
install -o root -g wheel -m 555   zfs.ko /netboot/powerpc64/boot/kernel
install -o root -g wheel -m 555   zfs.ko.symbols 
/netboot/powerpc64/boot/kernel
install: /netboot/powerpc64/boot/kernel/zfs.ko.symbols: No such file or 
directory
*** [_kmodinstall] Error code 71

Does it matter what filesystem the local dir is on?

Here the output from mount:

/dev/ada0p4 on /export (ufs, NFS exported, local, journaled soft-updates)

[root@tcx58] /usr/obj/> df -k .
Filesystem  1024-blocks     Used    Avail Capacity  Mounted on
/dev/ada0p4   132098652 30672276 90858484    25%    /export

I'm off at work for the rest of the day. I'll try if I can reproduce the 
issue there as well.

Thanks a lot!
Andreas
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"