Re: svn commit: r237367 - head/sys/fs/nfsclient

On 02.07.12 06:40, Andreas Tobler wrote:
> On 01.07.12 23:43, Konstantin Belousov wrote:
>> On Sun, Jul 01, 2012 at 08:33:09PM +0200, Andreas Tobler wrote:
>>> On 01.07.12 19:05, Konstantin Belousov wrote:
>>>> On Sun, Jul 01, 2012 at 03:56:52PM +0200, Andreas Tobler wrote:
>>>>> On 01.07.12 15:41, Konstantin Belousov wrote:
>>>>>> On Sun, Jul 01, 2012 at 03:37:18PM +0200, Andreas Tobler wrote:
>>>>>>> On 01.07.12 14:04, Konstantin Belousov wrote:
>>>>>>>> On Sun, Jul 01, 2012 at 01:23:02PM +0200, Andreas Tobler wrote:
>>>>>>>>> On 21.06.12 11:26, Konstantin Belousov wrote:
>>>>>>>>>> Author: kib
>>>>>>>>>> Date: Thu Jun 21 09:26:06 2012
>>>>>>>>>> New Revision: 237367
>>>>>>>>>> URL: http://svn.freebsd.org/changeset/base/237367
>>>>>>>>>>
>>>>>>>>>> Log:
>>>>>>>>>>     Enable deadlock avoidance code for NFS client.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hm, since this commit I fail with my nfs installworld/kernel.
>>>>>>>>>
>>>>>>>>> I have a builder which installs world/kernel to a nfs mounted
>>>>>>>>> directory.
>>>>>>>>> Namely used for cross builds.
>>>>>>>>>
>>>>>>>>> Now since this commit I get the following when I install kernel to the
>>>>>>>>> nfs directory:
>>>>>>>>>
>>>>>>>>> ..
>>>>>>>>> install -o root -g wheel -m 555   zfs.ko.symbols
>>>>>>>>> /netboot/sparc64/boot/kernel
>>>>>>>>> install: /netboot/sparc64/boot/kernel/zfs.ko.symbols: No such file or
>>>>>>>>> directory
>>>>>>>>> *** [_kmodinstall] Error code 71
>>>>>>>>> ..
>>>>>>>>>
>>>>>>>>> The file is there, a local install of the tree works without problems.
>>>>>>>>> Reverting to r237366 also makes it work again.
>>>>>>>>>
>>>>>>>>> The server is a -CURRENT, r237880, The client, -CURRENT too.
>>>>>>>>>
>>>>>>>>> How can I help to track down the real issue?
>>>>>>>>
>>>>>>>> Is it always the same file in the install procedure which causes the
>>>>>>>> failure ? Even more, is the failure pattern always the same ?
>>>>>>>
>>>>>>> I'd say so yes. When installing a kernel onto a nfs mounted fs then
>>>>>>> always (in my cases) the zfs.ko.symbols was the failing pattern.
>>>>>>> I tried ppc64 and sparc64 as target. With both it was the above file.
>>>>>>>
>>>>>>> When doing a installworld, it was, also in both cases, ppc64/sparc64,
>>>>>>> the cc1 in libexec which failed.
>>>>>>>
>>>>>>>> Might be, start with ktrace-ing the whole make invocation, including
>>>>>>>> the children processes.
>>>>>>>
>>>>>>> Some recipes how to start?
>>>>>> ktrace -o <file on local fs> -i make installkernel
>>>>>> Then kdump and cut the lines around relevant failure.
>>>>>
>>>>> ktrace -f, right?
>>>> Right, but without -i it is useless.
>>>
>>> Ah, yes, seems clear now after reading the man page.
>>>
>>>>> I placed the whole kdump here:
>>>>>
>>>>> http://people.freebsd.org/~andreast/dumped_installkernel.log
>>>>>
>>>>> It is not clear to me where the failure starts :)
>>>> Because logs do not contain tracepoints from the children.
>>>> See above about -i.
>>>>
>>>> I asked about excerpt because I expect the proper log to have an order
>>>> of magnitude bigger size.
>>>
>>> Ok. The dump is around 100MB, I hope I extracted as much as needed:
>>>
>>> http://people.freebsd.org/~andreast/dumped_installkernel-7.log
>>>
>>>>>>>> I used buildworld on the NFS-mounted obj/ as the test for the changes.
>>>>>>>
>>>>>>> Here the obj is local, only the src and the destination is on the
>>>>>>> nfs/netboot server.
>>>>>>
>>>>>> I just finished build on NFS obj/ and did several rounds of installs
>>>>>> for world and kernel into nfs-mounted destdir. It seems I cannot
>>>>>> reproduce
>>>>>> this locally.
>>>>>
>>>>> Ok. I try with an nfs obj too.
>>>
>>> So, I was not able to reproduce the failure with an nfs mounted obj dir.
>>>
>>> But I was able to reproduce the failure with three different machines
>>> which all have the obj local and the destination mounted via nfs.
>>>
>>> Are you able to try with a local obj too?
>> Below are two patches. Please follow my instructions literally to get
>> most of your bug report.
>>
>> First, please apply the usr.bin/xinstall patch only, and retry installkernel
>> (no need to use ktrace). It should show the proper error, short write, with
>> zero-sized result, instead of garbage ENOENT from errno.
>
> Done. No expected output. Iow, no message containing this:
>
> short write to %s: %jd bytes written, %jd bytes asked to write

Sorry for the confusion, I learned that I have to patch install from the 
buildworld and not only the host one:

===> zfs (install)
install -o root -g wheel -m 555   zfs.ko /netboot/powerpc64/boot/kernel
install -o root -g wheel -m 555   zfs.ko.symbols 
/netboot/powerpc64/boot/kernel
install: short write to /netboot/powerpc64/boot/kernel/zfs.ko.symbols: 0 
bytes written, 6959816 bytes asked to write
*** [_kmodinstall] Error code 71

>> Next, please apply the sys/fs/nfsclient patch, which should fix the core
>> cause.

And it really helps if I install the fresh built kernel to the right 
place :(

===> zfs (install)
install -o root -g wheel -m 555   zfs.ko /netboot/powerpc64/boot/kernel
install -o root -g wheel -m 555   zfs.ko.symbols 
/netboot/powerpc64/boot/kernel
===> zlib (install)
install -o root -g wheel -m 555   zlib.ko /netboot/powerpc64/boot/kernel
install -o root -g wheel -m 555   zlib.ko.symbols 
/netboot/powerpc64/boot/kernel
kldxref /netboot/powerpc64/boot/kernel

Thank you very much for the patience and the solution!

Andreas

_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscribe@freebsd.org"