[packman] FYI: i586 dead, x86_64 TW lot of "failed build attempts"

[Stefan Seyfried]

Hi Stefan,

On 25.09.21 10:38, Stefan Botter wrote:
> Hi munix9,
> 
> Am Donnerstag, den 23.09.2021, 21:07 +0200 schrieb munix9:
>> for "i586 dead" see
>> https://pmbs.links2linux.de/monitor
> 
> This I could fix.
> 
>> for "x86_64 TW lot of failed build attempts" see eg.
>> https://pmbs.links2linux.de/package/live_build_log/Multimedia/aften/openSUSE_Tumbleweed/x86_64
> 
> Here I am lost. Suddenly all build attempts on all workers fail with
> "unable to mount root fs". I am not sure, if this is limited to
> tumbleweed, It has definitively nothing to do with the OS version or
> other setup of the build hosts ("I did not do anything, I swear!" ;).

I had a look and tried to reproduce locally with

   osc build --vm-type kvm openSUSE_Tumbleweed x86_64

and it just works here (on an up-to-date Tumbleweed laptop, will try on 
15.3 later)

I suspected differences in the qemu-kvm command line, but the only 
difference I could spot (apart from different filesystem paths) was 
(with '-' the bad one from pmbs, '+' being the good one locally):

-rng-random,filename=/dev/random,id=rng0
+rng-random,filename=/dev/hwrng,id=rng0
...
--mem-prealloc
--mem-path /dev/hugetlbfs
...
--smp 1
+-smp 4

I also retried with "-j1" and "-smp 1" also worked fine.

OK, I retried on a 15.3 machine and it fails the same way as on PMBS.

The fun thing is: the rootfs can be mounted but for some reason it is 
not deemed "usable_root" by dracut

debugged this and found, that "/sys" is missing.

And alas, the update of the "build" package from

* Fri Aug 20 2021 Adrian Schröter <adrian at suse.de>
Features:
...
- Also create the /sys dir when preinstalling (to satisfy dracut)

The commit in github is much older:

commit 207a4729849c055f12ff8592e927a3e3db236097
Author: Michael Schroeder <mls at suse.de>
Date:   Wed Feb 24 11:31:48 2021 +0100

     Also create the /sys dir when preinstalling

     The background for this change is that dracut has a usable_root()
     function that checks the existance of /proc, /sys, and /dev.

diff --git a/init_buildsystem b/init_buildsystem
index 80dcdb7..b5106d2 100755
--- a/init_buildsystem
+++ b/init_buildsystem
@@ -569,6 +569,7 @@ fi
  mkdir -p "$BUILD_ROOT" || cleanup_and_exit 4
  mkdir -p "$BUILD_ROOT/.build" || cleanup_and_exit 4
  mkdir -p "$BUILD_ROOT/proc" || cleanup_and_exit 4
+mkdir -p "$BUILD_ROOT/sys" || cleanup_and_exit 4
  mkdir -p "$BUILD_ROOT/dev/pts" || cleanup_and_exit 4
  touch $BUILD_IS_RUNNING


I did simply add the "mkdir -p $BUILD_ROOT/sys" to 
/usr/lib/build/init_buildsystem on my 15.3 system and now everything 
works locally.
Doing the same on your OBS server machine should do the trick as well 
(the workers fetch the build code from the OBS server IIRC).

Or just update to the latest version of the build package :-)

Hope this helps,

	Stefan
-- 
Stefan Seyfried

"For a successful technology, reality must take precedence over
  public relations, for nature cannot be fooled." -- Richard Feynman