lvm trips up grub

mauricev · January 31, 2018

I run a script regularly on my computer that prepares all the configuration. It also reinstalls grub. But recently, it occasionally has a problem with lvm and that causes the machine, which is virtual, to get stuck at the grub screen unable to find the kernel.

When I install grub

/usr/sbin/grub-install --target=i386-pc --no-floppy /dev/sda

And it happens to respond

Installing for i386-pc platform.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
Installation finished. No error reported.

Grub will get stuck and not be able to find the kernel.

Boot.conf looks like this

boot {
	generate grub
	default "Funtoo Linux genkernel" 
	timeout 3 
}


"Funtoo Linux genkernel" {
	kernel kernel-debian-sources-x86_64-4.8.15-1
	initrd initramfs-debian-sources-x86_64-4.8.15-1
	params += dolvm real_root=auto rootfstype=ext4 resume=swap:/dev/mapper/funtoo-swap quiet
}

mauricev · February 1, 2018

/dev/sda1: LABEL="boot" UUID="2149e6d9-3d79-492d-9d6f-db6b952852a5" TYPE="ext2" PARTUUID="4f090ca0-01"
/dev/sda2: UUID="awsCeO-ZbWl-OO0i-zxsF-E22P-9WZb-08aoG1" TYPE="LVM2_member" PARTUUID="4f090ca0-02"
/dev/mapper/funtoo-swap: LABEL="swap" UUID="1eca2d56-9c4b-4673-94a3-a9c76cca4967" TYPE="swap"
/dev/mapper/funtoo-log: LABEL="log" UUID="92c90ca6-216f-4b77-9338-07f17073377b" TYPE="ext4"
/dev/mapper/funtoo-root: LABEL="root" UUID="1fb6792e-fc44-49cb-aa57-9fcd42ee1979" TYPE="ext4

bcowan · February 2, 2018

This post still confuses me. /boot not being mounted has nothing to do with grub. Grub uses its own abstraction layer to read grub.cfg which points it to a initrd/kernel. This "abstraction layer" never "mounts" anything, grub has no knowledge of mounts. So only your grub.cfg could be wrong, or not written because you didn't manually mount /boot, but pretty sure boot-update automounts /boot so grub-mkconfig can write grub.cfg unless specified to not automount.

bcowan · February 7, 2018

Now that you have concisely written down the steps you were doing and what you expected vs. what was happening, I do not see this as a bug but expected behavior. Grub-install indeed needs a mounted /boot or passed another writeable directory with the —boot-directory argument. This is done one single time only during your initial install. ( it writes a few files, notably grub.env and device.map ) You should not have to run grub-install again unless you corrupted something or another OS etc overwrites it. You then can run boot-update. It seems you are doing these steps in the wrong order. The install page documents these steps in the correct order. Boot should not need to be mounted for a working startup. The only thing that should need ran again is boot-update when you change kernels etc, never a grub-install.

mauricev · January 31, 2018

/etc/fstab is configured to mount /dev/sda1 on /boot, but it's ignored. Why is it ignored?

Anyway, when /boot is not mounted, these errors occur and grub ends up being confused about where the kernel/initramfs are.

palica · February 1, 2018

cat /etc/fstab

mauricev · February 1, 2018

LABEL=root              /               ext4            noatime         0 1
LABEL=boot		/boot		ext2		noatime		1 2
LABEL=swap		none		swap		sw		0 0
LABEL=log		/var/log	ext4		noatime		0 2

Oleg Vinichenko · February 1, 2018

for the disklabel, you need to recompile the kernel again. the --disklabel argument added while back into the build process. real_root need set explicitly so that it will look

real_root=/dev/mapper/root (exact one on your box).

mauricev · February 1, 2018

I'm not sure what problem your answer is trying to address, 1) grub tripping up when it's installed and /boot isn't mounted, or 2) why /boot doesn't mount at boot time.

On other systems without LVM, there isn't either problem. You certainly don't need the --disklabel argument or an actual root passed.

Oleg Vinichenko · February 1, 2018

/boot is never mounted at boot time. Rest of problem is unclear for me out of description.

There will be some fixes in genkernel regarding LVM, which may (or may not) fix this particular one.

mauricev · February 1, 2018

I don't understand what you mean that /boot is never mounted at boot time. I have a bunch of funtoo machines and when they startup, /boot is mounted. Only on the machine with lvm does it not mount.

The original problem was that on the lvm machine, the /etc/fstab entry for mounting /boot is being ignored. Installing grub with /boot not mounted results in those LVM errors and grub being unable to load the kernel. Manually mounting /boot seems to fix this problem.

palica · February 1, 2018

paste output of

blkid

please

palica · February 1, 2018

Grub reports: Installation finished. No error reported.

paste screenshot of grub error upon boot.

mauricev · February 1, 2018

There is no error as you can see. It just seems that grub can't find the kernel on /boot.

palica · February 1, 2018

did you run boot-update?

it looks you don't have /boot/grub/grub.cfg generated, do you?

mauricev · February 1, 2018

It seems to make sense that in order to run boot-update, /boot must be mounted. How else would save the grub.cfg file? But does grub installation require access to the grub.cfg file at the precise time it's being installed? If there the answer is no, then there is a bug in the installation of grub. However, either way, it shouldn't be complaining about LVM.

If the answer is yes, then the only bug is that utilizing LVM blocks the mount of /boot in /etc/fstab at boot time.

Here is the contents of grub.cfg

set timeout=3

  insmod part_msdos
  insmod ext2
  set root=(hostdisk//dev/sda,msdos1)
  search --no-floppy --fs-uuid --set 2149e6d9-3d79-492d-9d6f-db6b952852a5
if loadfont /grub/unifont.pf2; then
   set gfxmode=text
   insmod all_video
   terminal_output gfxterm
fi

set menu_color_normal=cyan/blue
set menu_color_highlight=blue/cyan

menuentry "Funtoo Linux genkernel - kernel-debian-sources-x86_64-4.8.15-1" {
  insmod part_msdos
  insmod ext2
  set root=(hostdisk//dev/sda,msdos1)
  search --no-floppy --fs-uuid --set 2149e6d9-3d79-492d-9d6f-db6b952852a5
  linux /kernel-debian-sources-x86_64-4.8.15-1 dolvm rootfstype=ext4 resume=swap:/dev/mapper/funtoo-swap quiet real
_root=/dev/funtoo/root rootfstype=ext4
  initrd /initramfs-debian-sources-x86_64-4.8.15-1
	set gfxpayload=keep
}
set default=0

palica · February 2, 2018

enable RC_LOG in /etc/rc.conf reboot and paste /var/log/rc.log

mauricev · February 2, 2018

This log is from the boot where grub is working and /boot is not being mounted.

rc.log

mauricev · February 2, 2018

Here it is again with verbose logging turned on. I don't see any reference to attempting to mount /boot specifically.

rc.log

palica · February 2, 2018

so to sum up:

grub install ok
lvm is not the culprit (those warning can be ignored /run/lvm/lvmetad.socket ...)
/boot not being automounted from openrc is causing the problem

unable to reproduce in virtualbox, boot gets mounted. After reboot what do you get when running

mount |grep boot

try rebuilding openrc, when that doesn't help, then I don't know.

mauricev · February 2, 2018

Not exactly. There are two scenarios, running the grub installation and /boot mounted and one where it’s not mounted.

With /boot not mounted

1) grub will report bogus LVM warnings and no actual error.

2) At the subsequent startup, grub will not find the kernel and get stuck at the grub command prompt.

With /boot mounted (The rc logs represent this scenario.)

1) grub will install fine with no warnings, and the machine will boot up fine

2) /boot mount entry in /etc/fstab gets ignored and /boot is never mounted (no output from mount |grep boot)

To reproduce all this, did you create a disk formatted to follow what you see in blkid? I believe this exact configuration is critical to reproducing these bugs.

palica · February 2, 2018

image.png.77fb6e503a7596de34c7ba5fc0239c52.png

this is my setup, using labels. separate /boot, noauto is not set in fstab, boot gets mounted by openrc. still unable to reproduce.

palica · February 2, 2018

try putting quotes in fstab for your labels

mauricev · February 2, 2018

I actually changed it to device name

/dev/funtoo/root / ext4 noatime 0 1

/dev/sda1 /boot ext2 noauto,noatime 1 2

/dev/funtoo/swap none swap sw 0 0

/dev/funtoo/log /var/log ext4 noatime 0 1

/dev/funtoo/root        /               ext4            noatime         0 1
/dev/sda1		/boot		ext2		noauto,noatime	1 2
/dev/funtoo/swap	none		swap		sw		0 0
/dev/funtoo/log		/var/log	ext4		noatime		0 1

And that has no effect. /boot doesn't get mounted.

palica · February 2, 2018

4 minutes ago, mauricev said:

/dev/sda1 /boot ext2 noauto,noatime

noauto = doesn't mount it automatically

so if you have the whole time noauto there it just won't get mounted, because that is how it is designed.

mauricev · February 2, 2018

I don't know where that fstab came from, but somehow that fixed the other fstab and now it started mounting /boot. Very weird indeed.

lvm trips up grub

Question

Link to comment

Share on other sites

43 answers to this question

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites