Jump to content











Photo
- - - - -

Grub4DOS Disk Read Error Happens on Physical Hardware but not VM


  • Please log in to reply
61 replies to this topic

#1 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 17 June 2020 - 09:41 PM

Hello,
 
I'm creating a secure boot chain for my work. Currently the boot process is:
1. Boot into GRUB2
2. (From GRUB2) boot into a minimal Linux kernel (5.6.7, kernel that I compiled with necessary modules)
3. Linux performs integrity checks on the filesystem
4. If successful, kexec into Grub4DOS
5. (From Grub4DOS) boot into Windows
 
I'm having an annoying issue that I just cannot seem to solve. I have implemented the boot process on a physical drive that I attach to my VMs via USB-SATA adapter. Through this method, the boot process works perfectly - though I must note that it didn't start to work perfectly until I changed my VM's hard drive settings to IDE (from SATA and SCSI). 
 
I have production equipment that I plugged my disk into and I am stuck at Grub4DOS trying to boot into Windows. It gives Error 25: Disk Read Error, but I'm woefully unsure as to why. I checked my production equipment's BIOS and (although I am unable to change any setting in the BIOS) we format (is that the right word?) our SATA to IDE, matching what I had to do with my VM.
 
I opened an issue at Grub4DOS' GitHub but I'm unsure how active the developers are, so I am hoping to receive some advice here.

 

Thanks



#2 alacran

alacran

    Gold Member

  • .script developer
  • 1699 posts
  •  
    Mexico

Posted 18 June 2020 - 06:00 AM

You have another thread related to same subject, and have been getting support from two of our more reputated members, there is no reason to open a new thread.

 

Anyway I will make some comments trying to help solve your issues.

 

You never said on this or previous thread what Windows OS you are trying to boot.  After reading your issue report I know now it is Win10 and will assume it is x64 (as it is the ussual).

 

You got several comments on Grub4DOS' GitHub one of them was use the new grub4dos version 0.46a wich is until active development yet,

 

 

steve6375 commented yesterday

0.4.6a is the current build version of grub4dos. Development of 0.4.5c is discontinued,
Can you try 0.4.6a? Why re-compile when you can download the precompiled version + utilities?
http://grub4dos.chen...ries/downloads/

 

I'm also agree you don't need to compile it yourself, you may edit very easily internal menu located into grld file by means of BootIce if required. 

 

Also you mentioned on your fist post here:

 

 

I checked my production equipment's BIOS and (although I am unable to change any setting in the BIOS) we format (is that the right word?) our SATA to IDE, matching what I had to do with my VM.

 

HDDs do not have Sata format and IDE format, only setting related to this is located on the BIOS or firmware, and it is related to IDE & AHCI (Advanced Host Controller Interface [AHCI] is a technical standard defined by Intel that specifies the operation of Serial ATA [SATA] disk controllers)

Of course the respective drivers should be loaded on the OS during boot, selecting and activating them is done during OS install. Usually if an OS was installed on a MB that has AHCI setting active and we want to change it to IDE the recommended option is reinstall the OS. Nevertheless I have done it on a PC running Win7x64 with the help of a guy that comparted a lot of registry settings required.

 

But on the contrary it is very easy to change from IDE to AHCI (the exact opposite setting you want):

Following has been proven to work from Win7 to 8.1, (not tested on Win10)

Open the Registry editor.
    Navigate to the following key

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\storahci

    Change the Start DWORD value from 3 to 0.
    Reboot your PC and set the SATA mode to AHCI.

Hope this comments made this subject a little more clear to you.

 

Best Regards

 

alacran



#3 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 18 June 2020 - 06:19 AM

You have another thread related to same subject, and have been getting support from two of our more reputated members, there is no reason to open a new thread.
 



I apologize, I realize now they're really the same issue and should not have opened a new thread.

Anyway I will make some comments trying to help solve your issues.
 
You never said on this or previous thread what Windows OS you are trying to boot.  After reading your issue report I know now it is Win10 and will assume it is x64 (as it is the ussual).
 
You got several comments on Grub4DOS' GitHub one of them was use the new grub4dos version 0.46a wich is until active development yet,
 



I did start using 0.4.6a and am still having the same issue, unfortunately. I realize it's still in active development, but as Steve said, it's used in over 100,000 PCs, what makes this one so different?
 

I'm also agree you don't need to compile it yourself, you may edit very easily internal menu located into grld file by means of BootIce if required. 
 



This is very interesting, I'll try this, thank you. I'd also like to note that I have tried the compiled version as well with the same issue.

Also you mentioned on your fist post here:
 
HDDs do not have Sata format and IDE format, only setting related to this is located on the BIOS or firmware, and it is related to IDE & AHCI (Advanced Host Controller Interface [AHCI] is a technical standard defined by Intel that specifies the operation of Serial ATA [SATA] disk controllers)
Of course the respective drivers should be loaded on the OS during boot, selecting and activating them is done during OS install. Usually if an OS was installed on a MB that has AHCI setting active and we want to change it to IDE the recommended option is reinstall the OS. Nevertheless I have done it on a PC running Win7x64 with the help of a guy that comparted a lot of registry settings required.
 
Best Regards
 
alacran


This explanation is very informative! Thank you, I truly wasn't sure of the true nature of the setting (not that I could edit it in the BIOS to begin with). Since it's IDE, I have no intention on changing that setting.

#4 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 18 June 2020 - 06:54 AM

This is the actual set of commands (I presume you are using them assembled in your preset_menu.lst) that you are using and that create the Error 25, right?

 

default 0
timeout 5

title Windows
rootnoverify (hd0,0)
chainloader /bootmgr

 

How (exactly) are you invoking the grub4dos via kexec?

Usually when using indirect loading one would use a config file instead of modifying the embedded menu.lst.

 

Try using a pre-compiled version (latest) untouched.

 

In it, get to command line (press c if you enter into a menu.lst) and try issue the

geometry (hd0)

command.

 

Unless I am reading it wrongly, in the post on github, you have an initial:

root is (0x80,0)

 

It is possible (please read as probable) that the issue lies in the kexec chainloading or in the *whatever* your Linux kernel does before running kexec, and this could also be *something* that is only triggered on a specific motherboard/BIOS only (hence it works in the VM but doesn't on your real hardware, see:

 

http://reboot.pro/to...c-specifically/

 

Generally speaking (and not particularly useful to solve the actual problem, but JFYI) if you cannot 

root (hd0,0)

and have to use instead

rootnoverify (hd0,0)

it is a sign that *something* is not right in what grub4dos "sees" about the disk(s).

 

Since grub4dos can be booted by GRUB2 "directly", you can try using kexec to run GRUB2 and see if it works, though I believe you need to make a "custom" compile of GRUB2 as ELF:

https://lists.gnu.or...4/msg00367.html

(very old note. I don't think that anyone was interested in the issue)

But can you get to GRUB2 from the grub4dos (loaded by kexec)?

 

Also:

root (hdx,y)

chainloader /bootmgr

 

is the "normal" way to boot a Windows NT Vista+ bypassing the volume bootector.

 

If you have installed GRUB2, the volume bootsector should have remained the "normal" MS one invoking BOOTMGR, so you can try:

root (hdx,y)

chainloader +1

 

 

:duff:

Wonko



#5 alacran

alacran

    Gold Member

  • .script developer
  • 1699 posts
  •  
    Mexico

Posted 18 June 2020 - 07:18 AM

I have been booting USB sticks using grub4dos for many years, even before the SATA disk were developed, and exactly same USB stick is capable to boot and have full read/write access to HDDs on PCs using IDE or AHCI drivers, then the problem should not be on grub4dos, I'm agree with Wonko it seems more related to the way you call grub4dos

 

As I don't know how old are your PCs it is better to play the safe way, get an spare USB stick format it using RMPrepUSB (see attached picture No. 1), install grub4dos, boot from it, and just try to boot Win10 from it with following commands on your menu.lst (since you said the system is Bios booting not UEFI, remember grub4dos do not work on UEFI):

 

 

title Find and load Windows (Loads BOOTMGR)
find --set-root /bootmgr
chainloader /bootmgr

 

NOTE: It is better to have The Metro boot or GUI boot disabled, see attached picture No.2

 

Then if the PC boots fine (as it should be), you need to start checking for the issue on other part of your booting process.

 

alacran

Attached Thumbnails

  • 1.png
  • 2.png


#6 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 18 June 2020 - 05:17 PM

Found it (couldn't find it before) a thread where tinybit explains the probable reasons for this issue:

http://reboot.pro/to...-kexecgrub4dos/

 

And here is a possible (maybe :unsure:) workaround using an image in ramdisk:

http://reboot.pro/to...sector-aligned/

 

:duff:

Wonko



#7 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 18 June 2020 - 07:32 PM

This is the actual set of commands (I presume you are using them assembled in your preset_menu.lst) that you are using and that create the Error 25, right?
 
default 0
timeout 5

title Windows
rootnoverify (hd0,0)
chainloader /bootmgr
 
How (exactly) are you invoking the grub4dos via kexec?
Usually when using indirect loading one would use a config file instead of modifying the embedded menu.lst.

Yes, that is the set of commands I was using. I've been able to use root (hd0,0) as well as rootnoverify (hd0,0) - I just wanted to see whether it would fix my problem or not. Obviously, it did not lol.
 
A big problem that I ran into (and is a large reason as to why I chose to modify the embedded menu.lst) is that Grub4Dos would take far too long trying to find menu.lst.
 

Try using a pre-compiled version (latest) untouched.
 
In it, get to command line (press c if you enter into a menu.lst) and try issue the
geometry (hd0)
command.

I have a terrible problem that I think is unrelated (but may not be): after kexec-ing, my USBs stop being powered so I'm unable to use my keyboard to get to the command line. 
I don't have this problem on VMs, but I also don't have my initial problem on VMs. :|
 
I will add the command to my menu.lst and see what happens, however.
 

Unless I am reading it wrongly, in the post on github, you have an initial:
root is (0x80,0)
 
It is possible (please read as probable) that the issue lies in the kexec chainloading or in the *whatever* your Linux kernel does before running kexec, and this could also be *something* that is only triggered on a specific motherboard/BIOS only (hence it works in the VM but doesn't on your real hardware, see:
 
http://reboot.pro/to...c-specifically/
 
Generally speaking (and not particularly useful to solve the actual problem, but JFYI) if you cannot 
root (hd0,0)
and have to use instead
rootnoverify (hd0,0)
it is a sign that *something* is not right in what grub4dos "sees" about the disk(s).

The actual code that is run to kexec into grub4dos is from a Python init script:
subprocess.run(["/sbin/kexec", "-l", "/sbin/grub.exe"])
subprocess.run(["/sbin/kexec", "-e"])

Since grub4dos can be booted by GRUB2 "directly", you can try using kexec to run GRUB2 and see if it works, though I believe you need to make a "custom" compile of GRUB2 as ELF:
https://lists.gnu.or...4/msg00367.html
(very old note. I don't think that anyone was interested in the issue)
But can you get to GRUB2 from the grub4dos (loaded by kexec)?

At least from my own research, it seemed like the ability to kexec into Grub2 was... planned? https://bugs.launchp...ls/ bug/1297316
 
But even with your link, it feels like kexec-ing to grub2 is a dead-end. :/
 
 

Found it (couldn't find it before) a thread where tinybit explains the probable reasons for this issue:
http://reboot.pro/to...-kexecgrub4dos/
 
:duff:
Wonko


Ah, I see:
 

Kexec might work with newer hardware, but kexec + grub.exe cannot.
 
Why? because grub.exe requires a working real-mode BIOS environment. With newer hardware - Intel's AHCI - the BIOS is no longer functioning once the Linux protected-mode (AHCI) driver is installed (the driver is normally built-in to the kernel).
 
With old hardware, a protected-mode driver does not destroy the realmode environment, so after switching from protected-mode to real-mode, BIOS will continue to work fine.
 
That is why "kexec+grub.exe" works on virtual machines(qemu, etc.) but does not work on newish real machines.
 
Some part of the BIOS is functioning well, others not.
 
In your case, the video/display is OK. The keyboard input is not working, even the CTRL+ALT+DEL is not functioning.
 
The BIOS could simply hangup on a INT13 call. So even CTRL+ALT+DEL might not work. The intel AHCI specification prohibit the hardware to return to real-mode after pmode drivers gains control.


My keyboard not working is related, then. The thread just kind of trails off, so I'm really not sure if there even *is* an answer but: what should I do?
I am a little confused as to how the BIOS specification is IDE but AHCI still interferes with this? I must be misunderstanding something.

EDIT:

I have been booting USB sticks using grub4dos for many years, even before the SATA disk were developed, and exactly same USB stick is capable to boot and have full read/write access to HDDs on PCs using IDE or AHCI drivers, then the problem should not be on grub4dos, I'm agree with Wonko it seems more related to the way you call grub4dos
 
As I don't know how old are your PCs it is better to play the safe way, get an spare USB stick format it using RMPrepUSB (see attached picture No. 1), install grub4dos, boot from it, and just try to boot Win10 from it with following commands on your menu.lst (since you said the system is Bios booting not UEFI, remember grub4dos do not work on UEFI):
 
 
NOTE: It is better to have The Metro boot or GUI boot disabled, see attached picture No.2
 
Then if the PC boots fine (as it should be), you need to start checking for the issue on other part of your booting process.
 
alacran


I made a USB following these specifications and am able to boot into Windows just fine, so it must be something to do with my script/kexec.

Edited by DapperDeer, 18 June 2020 - 08:27 PM.


#8 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 18 June 2020 - 09:25 PM

For some reason I'm unable to edit my post to provide an update, otherwise I wouldn't double-post like this.
 

Kexec might work with newer hardware, but kexec + grub.exe cannot.
 
Why? because grub.exe requires a working real-mode BIOS environment. With newer hardware - Intel's AHCI - the BIOS is no longer functioning once the Linux protected-mode (AHCI) driver is installed (the driver is normally built-in to the kernel).

 
The bold part gave me a brainblast. I looked at my kernel's config and saw that AHCI support was enabled. I disabled it and Grub4DOS was not only much faster, but booted into Windows seamlessly.
Of course though, something else must go wrong: Windows is unable to complete its boot due to an "Inaccessible Boot Device" but I think that's a much easier problem to diagnose.

Thank you all so much for your patience and help!

#9 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 19 June 2020 - 08:01 AM

Good. :)

 

The inaccessible boot device usually is a SATA (or IDE) driver issue, is that a 0x0000007b?

 

It is possible that *something* in your *special* way to call grub.exe via kexec messes up with disks, and there may be the need to remap disks before chainloading the BOOTMGR.

 

 

subprocess.run(["/sbin/kexec", "-l", "/sbin/grub.exe"])

 

You can add there as a parameter the (simple) configfile, needing not to recompile the grub4dos, and needing not a menu.lst.

 

only as an example:

--command-line="--config-file=root(hd0,0);geometry (hd0); pause --wait=10;chainloader (hd0,0)+1; pause  --wait=5; boot"

 

Also - at least for experimenting - you can try the "Vista boot floppy" approach, loosely you can have a floppy image containing the BOOTMGR and \boot\BCD, like here:
http://www.multiboot....uk/floppy.html

and put it in a ramdisk:
http://reboot.pro/to...sector-aligned/

 

There are a few caveats in booting Windows 10 (and possibly also 8/8.1) this way (connected to sleep/hybernation and similar) but while testing it could be useful. 

 

About kexec-able GRUB2, I doubt that the good guys that develop it officially have any interest in adding the feature, as their concept of the U in the Universal in GRUB is seemingly "Undoubtedly Linux only", but recently we have a very good new fork of GRUB2 by a developer that is very responsive to requests and already has made this version of GRUB2 far better than the original, adding to it many of the features of grub4dos, so maybe  it could be worth to let him know of this (of course that will take time, even if he is interested in it):
https://github.com/a1ive/grub

 

You can see the added features/commands via google translate here:
https://translate.go...o/grub2_zh.html

 

 

 

:duff:

Wonko



#10 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 19 June 2020 - 05:42 PM

Wonko the Sane, on 19 Jun 2020 - 01:01 AM, said:
Good. :)

The inaccessible boot device usually is a SATA (or IDE) driver issue, is that a 0x0000007b?

It is possible that *something* in your *special* way to call grub.exe via kexec messes up with disks, and there may be the need to remap disks before chainloading the BOOTMGR.

There isn't an actual error code, it says "ERROR CODE: Inaccessible Boot Device"
Do you think there's something I need to change in the registry settings from AHCI to IDE?

Wonko the Sane, on 19 Jun 2020 - 01:01 AM, said:
You can add there as a parameter the (simple) configfile, needing not to recompile the grub4dos, and needing not a menu.lst.

only as an example:
--command-line="--config-file=root(hd0,0);geometry (hd0); pause --wait=10;chainloader (hd0,0)+1; pause --wait=5; boot"

Unfortunately using the --command-line parameter wasn't really working, it would try to find menu.lst and when it couldn't, it would drop to grub's cmdline.
So, my menu.lst looks like this:

default 0
timeout 5

title Windows
    root (hd0,0)
    geometry (hd0)
    chainloader (hd0,0)+1
I remember reading that Windows used to need to use some map commands but I don't know how useful that would be here since Windows is my first partition.


Wonko the Sane, on 19 Jun 2020 - 01:01 AM, said:
Also - at least for experimenting - you can try the "Vista boot floppy" approach, loosely you can have a floppy image containing the BOOTMGR and \boot\BCD, like here:
http://www.multiboot....uk/floppy.html
and put it in a ramdisk:
http://reboot.pro/to...sector-aligned/

There are a few caveats in booting Windows 10 (and possibly also 8/8.1) this way (connected to sleep/hybernation and similar) but while testing it could be useful.


:duff:
Wonko

I'll keep these in mind and do some reading/experimenting with them! Thank you, the forked Grub2 seems really cool.

#11 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 19 June 2020 - 07:21 PM

Maybe, just a shot in the dark, I don't know, there is a parsing error due to the double quotes.

 

Everyone using kexec reported the approach to be working (but they ran kexec on command line), the "whole string" in your case would be something like:

subprocess.run(["/sbin/kexec", "-l", "/sbin/grub.exe --command-line="--config-file=root(hd0,0);geometry (hd0); pause --wait=10;chainloader (hd0,0)+1; pause --wait=5; boot""])

 

At least on windows/command .exe the double quotes would probably create a mess/be escaped/whatever, cannot say in Linux.

 

Can you try - for the time of the experiment - to run kexec manually from Linux command line?

 

Or in any case, does now the keyboard work?

 

The inaccessible boot device issue might be connected to what grub4dos "sees"  - at least from what you posted earlier, when you still had  AHCI, the message about disk (0x80,0) is NOT normal.

 

One way or the other, now that you have it almost working you need to get on command line and be able to run test commands, i.e. before blindly chainloading you should be able to inspect the environemnt and hopefully  - if possible - fix it.

 

This is unfortunately VERY wrong (when testing):

 

title Windows
root (hd0,0)
geometry (hd0)
chainloader (hd0,0)+1

 

you will have no time to see the output of the root (if any) geometry or chainloader command as the (implied in menu.lst, needed explicitly on command line) will attempt booting instantly.

 

 

title Windows
root (hd0,0)
pause --wait=5

geometry (hd0)
pause --wait=5
chainloader (hd0,0)+1

pause --wait=10

 

will give you time to see what happens.

 

:duff:

Wonko



#12 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 19 June 2020 - 08:01 PM

Maybe, just a shot in the dark, I don't know, there is a parsing error due to the double quotes.
 
Everyone using kexec reported the approach to be working (but they ran kexec on command line), the "whole string" in your case would be something like:
subprocess.run(["/sbin/kexec", "-l", "/sbin/grub.exe --command-line="--config-file=root(hd0,0);geometry (hd0); pause --wait=10;chainloader (hd0,0)+1; pause --wait=5; boot""])

The command that I was using was:


subprocess.run(["/sbin/kexec", "-l", "/sbin/grub.exe", "--command-line=\"--config-file=root(hd0,0);geometry (hd0); pause --wait=10;chainloader (hd0,0)+1; pause --wait=5; boot\""])

as without the comma separator between /sbin/grub.exe and the commandline, it thinks that the file it's looking for is named /sbin/grub.exe --command-line="--config-file=root(hd0,0);geometry (hd0); pause --wait=10;chainloader (hd0,0)+1; pause --wait=5; boot.
So, I gave it the extra parameter of the command line but with no luck, unfortunately.
 

 

At least on windows/command .exe the double quotes would probably create a mess/be escaped/whatever, cannot say in Linux.
 
Can you try - for the time of the experiment - to run kexec manually from Linux command line?
 
Or in any case, does now the keyboard work?

Uhm, I don't think I actually have access to the Linux command line. We boot into the bzImage and the initramfs and it runs whatever is in the init script, which in this case is in Python.
Additionally, the keyboard is still non-functioning.
 

The inaccessible boot device issue might be connected to what grub4dos "sees"  - at least from what you posted earlier, when you still had  AHCI, the message about disk (0x80,0) is NOT normal.
 
One way or the other, now that you have it almost working you need to get on command line and be able to run test commands, i.e. before blindly chainloading you should be able to inspect the environemnt and hopefully  - if possible - fix it.
 
This is unfortunately VERY wrong (when testing):
 
title Windows
root (hd0,0)
geometry (hd0)
chainloader (hd0,0)+1
 
you will have no time to see the output of the root (if any) geometry or chainloader command as the (implied in menu.lst, needed explicitly on command line) will attempt booting instantly.
 
 
title Windows
root (hd0,0)
pause --wait=5
geometry (hd0)
pause --wait=5
chainloader (hd0,0)+1
pause --wait=10
 
will give you time to see what happens.
 
:duff:
Wonko

Is there anything specific I should be looking for in these outputs?

drive 0x80(LBA): C/H/S=10836/255/63 and then some sector information.
It also shows the drive and its partitions, each partitions filesystem type (ntfs, ext2) and their respective hex code 0x07 and 0x83.

The output looked exactly the same as on my VM, if that matters any.



#13 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 20 June 2020 - 07:22 AM

Yep, there must be another way to escape the double quotes or anyway have that line pass the right parameters, but if the menu.lst works (in the sense that is read correctly), that is good as well, later you will be able anyway to get rid of the menu.lst file and embed it in grub.exe.

Have you tried with single quotes?

something *like*:

subprocess.run(['/sbin/kexec', '-l', '/sbin/grub.exe', '--command-line=\"--config-file=root(hd0,0);geometry (hd0); pause --wait=10;chainloader (hd0,0)+1; pause --wait=5; boot\"'])

or similar

Or maybe turning the shell off:
https://www.jimmyg.o...-argument-lists

 

 

Not having the keyboard working is a pity, however (for testing) once in production it may even be an advantage as it will keep the boot chain integrity.

 

If in those lines you add a ls and possibly even a blocklist and change the chainloader command from the bootsector to the BOOTMGR you will have more information, but if the root is correctly established and the filesystem are seen correctly maybe the issue in booting is *somewhere* post grub4dos:

 

title Windows
root (hd0,0)

root
pause --wait=5
geometry (hd0)
ls /boot/BCD

blocklist /boot/BCD

ls /bootmgr
blocklist /bootmgr

pause --wait=5
chainloader /bootmgr

pause --wait=10

 

Another thing I would try (another shot in the dark) would be to see if loading the grub4dos USB stack changes something, even if it is not intended for the keyboard but for storage devices, maybe it "resets" ("side effect/collateral damage") *something*.

I.e. add after the first pause in the above a:

usb --init

 

And another thing I would try doing would be a disk re-map (again it shouldn't change anything but maybe re-mapping does *something*):

 

 

title Windows
map (hd0) (hd1)

map --hook

root (hd1,0)

root
pause --wait=5
geometry (hd1)
ls /boot/BCD

blocklist /boot/BCD

ls /bootmgr
blocklist /bootmgr

pause --wait=5
chainloader /bootmgr

pause --wait=10

 

Anyway, now that you are using this menu.lst, you can add in the GRUB2 configuration file an entry to chainload grub4dos directly and see if there are any differences and if the Windows boot directly.

I.e.

GRUB2 entry #1 ->linux(->grub4dos->windows)

vs:
GRUB2 entry #2->grub4dos->windows

 

:duff:

Wonko



#14 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 22 June 2020 - 10:21 PM

I added the Grub4DOS entry in my Grub2 and am able to boot to Windows perfectly fine, so something is happening between Linux and the kexec call that's messing up the boot to Windows.

 

Because of the "inaccessible" part, I checked to see if I was mounting the Windows filesystem somewhere in Linux. Turns out I had an old fstab that would mount /dev/sda1 (Windows partition) on / instead of /dev/sda4 (Linux partition). I corrected that but unfortunately am still unable to boot into Windows.

 

This is what my menu.lst looks like:

default 0
timeout 5
 
title Windows
root (hd0,0)
root
pause --wait=5
usb --init
geometry (hd0)
ls /boot/BCD
blocklist /boot/BCD
ls /bootmgr
blocklist /bootmgr
pause --wait=5
chainloader /bootmgr
pause --wait=10

and the result of that (before getting Windows' BSOD) is: https://imgur.com/n2VYFjs

 

I tried doing the `map (hd0) (hd1) map --hook` commands but Grub4DOS was having issues with that. 



#15 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 23 June 2020 - 07:10 AM

I added the Grub4DOS entry in my Grub2 and am able to boot to Windows perfectly fine, so something is happening between Linux and the kexec call that's messing up the boot to Windows.

Yep, this is definite proof that - notwithstanding what you disabled in your Linux - that the Linux still messes up with *something* at BIOS level.

 

The "inaccessible boot device" message is the (from the stupid BOOTMGR from recent Windows) way to show what was (and still is) "0x0000007b Inaccessible boot device" erroro message, this means either (essentially/most common) "you either attempted to boot from a non-existing device or the device you attempted to boot from is a device for which a valid protected mode driver is not found".

 

This happens basically when the BOOTMGR switches from Real mode to Protectd mode, the Windows BOOTMGR runs initially in protected mode, trusting whatever info is in the BIOS then, midway, switches to protected mode, re-scans hardware and loads drivers for booting the actual windows.

 

Since going "directly" BIOS->GRUB2->grub4dos->Windows works just fine, the driver is just fine, so it can be any of:
1) *something* changed the ID, checksum, or *whatever* of the boot device in such a way that it is not anymore valid when the switch happens
2) *something* changed in the BIOS environment in such a way that the BOOTMGR looks for another device.

 

I hoped that by re-mapping the disk grub4dos would have re-dected/corrected whatever was wrong before, and the fact that you are having issues with that mapping is a confirmation of the thesis that *something* in the BIOS is changed by your Linux boot.

 

Boot the grub4dos from grub2 directly and try the remapping from this "clean" environmentm add another entry to menu.lst:

 

title Windows remapped
map (hd0) (hd1)
map --hook
root (hd1,0)
root
pause --wait=5
geometry (hd1)
ls /boot/BCD
blocklist /boot/BCD
ls /bootmgr
blocklist /bootmgr
pause --wait=5
chainloader /bootmgr
pause --wait=10

 

It should work normally.

 

I don't know if it is even possible but maybe there is some way to make a sort of snapshot of the BIOS tables before your Linux boots and restore it when grub4dos takes control. :unsure:

 

:duff:

Wonko



#16 wimb

wimb

    Platinum Member

  • Developer
  • 3121 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 23 June 2020 - 08:03 AM

The bold part gave me a brainblast. I looked at my kernel's config and saw that AHCI support was enabled. I disabled it and Grub4DOS was not only much faster, but booted into Windows seamlessly.
Of course though, something else must go wrong: Windows is unable to complete its boot due to an "Inaccessible Boot Device" but I think that's a much easier problem to diagnose.
 

 

Could it be just this: AHCI is disabled in Linux and then the Windows 10 installed with AHCI on does not boot anymore

 

Fix it by:

-  Change Start value in Windows 10 registry setting for some driver services - difficult  :unsure: so that it corresponds to Windows 10 Installed for setting IDE instead of AHCI

   You want the reverse of what is described here - www.partitionwizard.com/partitionmagic/enable-ahci-after-win-10-installation.html

   Also have a look here and here and here and here

-  Or easier: Install Windows 10 with setting IDE on - Reboot with your desired way of booting through Grub2 > Linux > Grub4dos > Windows Boot Manager

- Or Simply Boot Windows 10 first in Safe Mode by pressing F8 and let Windows 10 adjust the settings in registry ....and Reboot Windows 10 in normal mode



#17 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 23 June 2020 - 08:29 PM

Yep, this is definite proof that - notwithstanding what you disabled in your Linux - that the Linux still messes up with *something* at BIOS level.
 
The "inaccessible boot device" message is the (from the stupid BOOTMGR from recent Windows) way to show what was (and still is) "0x0000007b Inaccessible boot device" erroro message, this means either (essentially/most common) "you either attempted to boot from a non-existing device or the device you attempted to boot from is a device for which a valid protected mode driver is not found".
 
This happens basically when the BOOTMGR switches from Real mode to Protectd mode, the Windows BOOTMGR runs initially in protected mode, trusting whatever info is in the BIOS then, midway, switches to protected mode, re-scans hardware and loads drivers for booting the actual windows.
 
Since going "directly" BIOS->GRUB2->grub4dos->Windows works just fine, the driver is just fine, so it can be any of:
1) *something* changed the ID, checksum, or *whatever* of the boot device in such a way that it is not anymore valid when the switch happens
2) *something* changed in the BIOS environment in such a way that the BOOTMGR looks for another device.
 
I hoped that by re-mapping the disk grub4dos would have re-dected/corrected whatever was wrong before, and the fact that you are having issues with that mapping is a confirmation of the thesis that *something* in the BIOS is changed by your Linux boot.
 
Boot the grub4dos from grub2 directly and try the remapping from this "clean" environmentm add another entry to menu.lst:
 
title Windows remapped
map (hd0) (hd1)
map --hook
root (hd1,0)
root
pause --wait=5
geometry (hd1)
ls /boot/BCD
blocklist /boot/BCD
ls /bootmgr
blocklist /bootmgr
pause --wait=5
chainloader /bootmgr
pause --wait=10
 
It should work normally.
 
I don't know if it is even possible but maybe there is some way to make a sort of snapshot of the BIOS tables before your Linux boots and restore it when grub4dos takes control. :unsure:
 
:duff:
Wonko

 
I've tried a few different kexec options because I had an inkling that maybe kexec is doing something with the drives, but unfortunately none of the options I've tried have really worked.
I'm not sure what that extra menu.lst is supposed to prove, but it did work.
 

Could it be just this: AHCI is disabled in Linux and then the Windows 10 installed with AHCI on does not boot anymore
 
Fix it by:
-  Change Start value in Windows 10 registry setting for some driver services - difficult  :unsure: so that it corresponds to Windows 10 Installed for setting IDE instead of AHCI
   You want the reverse of what is described here - www.partitionwizard.com/partitionmagic/enable-ahci-after-win-10-installation.html
   Also have a look here and here and here and here
-  Or easier: Install Windows 10 with setting IDE on - Reboot with your desired way of booting through Grub2 > Linux > Grub4dos > Windows Boot Manager
- Or Simply Boot Windows 10 first in Safe Mode by pressing F8 and let Windows 10 adjust the settings in registry ....and Reboot Windows 10 in normal mode


https://imgur.com/HALMAQTthis is my device manager after doing the safe reboots - it very much looks like it's in IDE mode but I'm still unable to boot.
That was my immediate thought, that AHCI was enabled on Windows but I'm still getting the INACCESSIBLE BOOT DEVICE error. :(



#18 wimb

wimb

    Platinum Member

  • Developer
  • 3121 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 24 June 2020 - 05:00 AM

 https://imgur.com/HALMAQTthis is my device manager after doing the safe reboots - it very much looks like it's in IDE mode but I'm still unable to boot.

That was my immediate thought, that AHCI was enabled on Windows but I'm still getting the INACCESSIBLE BOOT DEVICE error. :(

 

It sounds like you used the Safe Mode booting in the already working boot sequence Grub2 > Grub4dos > Windows

 

In Windows 10 you can use BOOTICE 1.3.3.2 to modify the BCD of the current system and use Professional mode to change the BootMenuPolicy into Legacy instead of Standard

Also add New WIM boot entry so that you have at least two entries in the Boot Manager Menu

 

 

When booting with Windows 10 you are now able to arrive at the BootManager Menu and use F8 to select Safe Mode 

 

The idea is to use Safe Mode booting  in the non-working boot sequence Grub2 > Linux > Grub4dos > Windows



#19 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 24 June 2020 - 07:58 AM

 
I'm not sure what that extra menu.lst is supposed to prove, but it did work.

It was just a test to see if - by any chance - the issue you reported:

 

I tried doing the `map (hd0) (hd1) map --hook` commands but Grub4DOS was having issues with that. 

was a grub4dos one, since the re-mapping works fine when booted more directly via GRUB2 it is another confirmation that the environment in which grub4dos runs is different and it is "good" when kexec is it not in the equation and become "bad" when you introduce the kexec. 

 

From what I understand (not necessarily 100% accurate) the BIOS upon booting sets a number of values *somewhere*,

https://wiki.osdev.o...emory_Map_(x86)

 

in a normal booting these values are read by the real mode of BOOTMGR, and passed to itself when the protected mode kicks in, then hardware is re-scanned to load the drivers.

GRUB2 does not modify these values.

grub4dos does not modify these values.

 

Most probably not kexec, but rather the Linux kernel you boot midway actually instead changes something and the kexec passes to grub4dos some of these wrong values, that are passed unchanged to the BOOTMGR by grubdos and this *whatever* change in parameters is what makes BOOTMGR fail.

 

The chainloader command (try help chainloader in grubdos) does have a number of parameters, that can change some of these values but I have no idea which parameter may be the culprit nor whether it will be among the parameters that existing chainloader options can change, though - at least in theory - anything can be changed by write or dd in grub4dos.

 

:duff:

Wonko



#20 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 24 June 2020 - 10:19 PM

It sounds like you used the Safe Mode booting in the already working boot sequence Grub2 > Grub4dos > Windows
 
In Windows 10 you can use BOOTICE 1.3.3.2 to modify the BCD of the current system and use Professional mode to change the BootMenuPolicy into Legacy instead of Standard
Also add New WIM boot entry so that you have at least two entries in the Boot Manager Menu
 
attachicon.gifBOOTICE-2020-06-24_084435.jpg
 
When booting with Windows 10 you are now able to arrive at the BootManager Menu and use F8 to select Safe Mode 
 
The idea is to use Safe Mode booting  in the non-working boot sequence Grub2 > Linux > Grub4dos > Windows

 
I was having a terrible time with my BCD, constant "Access denied" and "BCD Store does not exist" type errors, so I reinstalled Windows on my IDE hardware and ran into the same exact problem. 
 
 

From what I understand (not necessarily 100% accurate) the BIOS upon booting sets a number of values *somewhere*,
https://wiki.osdev.o...emory_Map_(x86)
 
in a normal booting these values are read by the real mode of BOOTMGR, and passed to itself when the protected mode kicks in, then hardware is re-scanned to load the drivers.
GRUB2 does not modify these values.
grub4dos does not modify these values.
 
Most probably not kexec, but rather the Linux kernel you boot midway actually instead changes something and the kexec passes to grub4dos some of these wrong values, that are passed unchanged to the BOOTMGR by grubdos and this *whatever* change in parameters is what makes BOOTMGR fail.
 
The chainloader command (try help chainloader in grubdos) does have a number of parameters, that can change some of these values but I have no idea which parameter may be the culprit nor whether it will be among the parameters that existing chainloader options can change, though - at least in theory - anything can be changed by write or dd in grub4dos.
 
:duff:
Wonko


So I found that kexec has a "debug mode" where it details a lot more information during its process. Lots of hex code and assembly, but I think most importantly is the entry location.
https://imgur.com/MMwMYh3

I truthfully have no idea what any of it means, but from the Wiki you linked it doesn't necessarily look correct?



#21 wimb

wimb

    Platinum Member

  • Developer
  • 3121 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 25 June 2020 - 04:42 AM

 
I was having a terrible time with my BCD, constant "Access denied" and "BCD Store does not exist" type errors, so I reinstalled Windows on my IDE hardware and ran into the same exact problem. 
 

 

Can you give screenshot of disk management showing the partitioning of your Windows disk.

Can you give screenshot of Windows Explorer showing details and where Boot folder and bootmgr file and Windows folder are located



#22 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 25 June 2020 - 06:43 AM

Can you give screenshot of disk management showing the partitioning of your Windows disk.

Can you give screenshot of Windows Explorer showing details and where Boot folder and bootmgr file and Windows folder are located

 

Partitions: https://imgur.com/B47DkXA

Windows Explorer, bootmgr and Boot dir: https://imgur.com/uVe7sEq



#23 wimb

wimb

    Platinum Member

  • Developer
  • 3121 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 25 June 2020 - 07:33 AM

 
I was having a terrible time with my BCD, constant "Access denied" and "BCD Store does not exist" type errors, so I reinstalled Windows on my IDE hardware and ran into the same exact problem. 
 

 

Thanks for the Screenshots. At least we have a better insight of your configuration.

 

I understand that your BIOS Setting is IDE and that Windows is booting in MBR BIOS mode with bootmgr and Boot\BCD located on same partition as where Windows is installed.

It is not a complete fresh install where the Windows partition was first formatted with NTFS. 

The free space is limited so that an extra VHD next to Windows is not an option ....(in case of more space you could easily add a really fresh Windows 10 Installed in VHD).

 

It remains strange that internal HDD booting with

Grub2 > Grub4dos > Windows 10  is booting OK

Grub2 > Linux > kexec Grub4dos > Internal HDD Windows 10 Fails and also Fails when F8 Safe Mode booting is applied in booting Windows 10

and

VM USB SATA IDE adapter connected to HDD > Grub2 > Linux > kexec Grub4dos > Windows 10 is booting OK

 

Can you confirm that the conclusions made on my side are correct.

 

I think you need to test with a complete new fresh installed Windows 10 on internal harddisk partition formatted with NTFS and without using VM ware.

In that case you get rid of all the drivers for the VM ware that now might cause trouble ....

 

 
USB_FORMAT combines MBR BIOS mode booting USB > Windows BootManager > Grub2 > Grub4dos > Windows 10
 
== 


#24 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15548 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 June 2020 - 09:40 AM

Side idea.

 

 

3. Linux performs integrity checks on the filesystem

 

Which kind of integrity checks?

 

Can these integrity checks be made in either GRUB2 or grubdos? (or in another real mode OS, such as FreeDOS?)

 

:duff:

Wonko



#25 DapperDeer

DapperDeer

    Newbie

  • Members
  • 28 posts
  •  
    United States

Posted 25 June 2020 - 05:56 PM

Side idea.

 

 

Which kind of integrity checks?

 

Can these integrity checks be made in either GRUB2 or grubdos? (or in another real mode OS, such as FreeDOS?)

 

:duff:

Wonko

 

RSA Key hashing and validating as well as entire partition overwriting using `dd`.

 

We tried these integrity checks in Grub2 but it was INCREDIBLY slow, I mean 30-40 minute boot times. 

 

 

Thanks for the Screenshots. At least we have a better insight of your configuration.

 

I understand that your BIOS Setting is IDE and that Windows is booting in MBR BIOS mode with bootmgr and Boot\BCD located on same partition as where Windows is installed.

It is not a complete fresh install where the Windows partition was first formatted with NTFS. 

The free space is limited so that an extra VHD next to Windows is not an option ....(in case of more space you could easily add a really fresh Windows 10 Installed in VHD).

 

It remains strange that:

USB > Grub2 > Grub4dos > Internal HDD Windows 10  is booting OK

USB > Grub2 > Linux > kexec Grub4dos > Internal HDD Windows 10 Fails and also Fails when F8 Safe Mode booting is applied in booting Windows 10

USB > Grub2 > Linux > kexec Grub4dos > VM USB SATA-IDE adapter HDD Windows 10 is booting OK

 

Can you confirm that the conclusions made on my side are correct.

 

I think you need to test with a complete new fresh installed Windows 10 on internal harddisk partition formatted with NTFS and without using VM ware.

In that case you get rid of all the drivers for the VM ware that now might cause trouble ....

 

 
 
USB_FORMAT combines MBR BIOS mode booting USB > Windows BootManager > Grub2 > Grub4dos > Windows 10
 

 

 

I am currently in the process of installing Windows 10 fresh. I will need to use VMWare for my Linux distros so I can install grub2/grub4dos and my Linux Kernel, but I don't think any drivers will be installed through that. I'll update here when I have more progress.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users