Jump to content











Photo
* * * * * 1 votes

Hack Bootmgr to boot Windows in BIOS to GPT

bios gpt bootmgr winload

  • Please log in to reply
374 replies to this topic

#51 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 May 2014 - 04:35 PM

cdob, on 26 May 2014 - 5:39 PM, said:
Post #37 contains a floppy disk example http://reboot.pro/to...e-2#entry182138
Yes, a floppy disk image works at "mixed mode" setup.

Good. :)

cdob, on 26 May 2014 - 5:39 PM, said:
bcdboot likes to copy unicode language files too.
This won't fit at a 1.44 MB floppy disk. Copy \bootmgr and \boot\bcd only.
bcdboot may work at a bigger floppy disk.

But, unless I am mistaken, one could make a temporary image to run bcdboot (which would also automagically create the "right" BCD) and later copy just the few needed files to the "normally sized" floppy image.
Probably this would be easier than running BCDEDIT manually to create the BCD. :unsure:
It would be however interesting (IMHO) to have a set of BCDEDIT commands to create the BCD on the floppy image.

cdob, on 26 May 2014 - 5:39 PM, said:
VHD is the OS default image format nowadays.

Yes, but one has to make always sure it is the "static" or "fixed" type for this use.

@Sascha Weaver
It is possible that diskpart have some "strange" limitations, like using "whole" Cylinder.
That would make the minimal image (not necessarily NTFS) 1x255x63=16.065 sectors (possibly comprising either the 63 "old school" or 2048 "new school" hidden sectors).
If you want I can prepare for you the smaller (empty) image.
How much is the space needed for a "full" bcdboot output? (i.e. the BOOTMGR+\boot\BCD+ all the fonts/MUI/whatever)

:duff:
Wonko

#52 Sascha Weaver

Sascha Weaver
  • Members
  • 5 posts
  •  
    China

Posted 27 May 2014 - 05:23 AM

@Sascha Weaver
It is possible that diskpart have some "strange" limitations, like using "whole" Cylinder.
That would make the minimal image (not necessarily NTFS) 1x255x63=16.065 sectors (possibly comprising either the 63 "old school" or 2048 "new school" hidden sectors).
If you want I can prepare for you the smaller (empty) image.
How much is the space needed for a "full" bcdboot output? (i.e. the BOOTMGR+\boot\BCD+ all the fonts/MUI/whatever)

:duff:
Wonko

 

Wow, so it's diskpart's fault, not letting me to create NTFS partition smaller than 8 MiB? I see.

 

The full size of the files that bcdboot copies is about 19 MiB (Windows 8.1). Smaller enough.

 

I tried with 150 MiB image and 10 MiB image, the load time is barely the same. I cannot feel a difference. (I am using Samsung 840 Pro SSD) So shrinking the image is not very necessary. 32 MiB is not a bad idea. :-)



#53 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 27 May 2014 - 10:25 AM

Wow, so it's diskpart's fault, not letting me to create NTFS partition smaller than 8 MiB? I see.

Well, no, it is a possibility/guess (that would need some experiments to be confirmed)
 

The full size of the files that bcdboot copies is about 19 MiB (Windows 8.1). Smaller enough.

Yes/No.
I mean, that is what the "plain" BCDBOOT command does, adding ALL languages/settings, but the idea (mine at least) was to use it with the /l <locale> switch:
http://technet.micro...7(v=ws.10).aspx
in order to see what was the "minimal" size with:
a. plain "English" (or other "western" language) <- this should need not any particular added font/unicode/whatever :unsure:
b. your own "larger" locale because of East Asian Fonts (i.e. "your" full bcdboot output)
 

I tried with 150 MiB image and 10 MiB image, the load time is barely the same. I cannot feel a difference. (I am using Samsung 840 Pro SSD) So shrinking the image is not very necessary. 32 MiB is not a bad idea. :-)

If you use (instead of memdisk) grub4dos "direct" mapping (i.e. without the --mem switch), time needed will be minimal and de facto independent from size of image even on very slow devices.

 

:duff:

Wonko



#54 Sascha Weaver

Sascha Weaver
  • Members
  • 5 posts
  •  
    China

Posted 27 May 2014 - 10:42 AM

I mean, that is what the "plain" BCDBOOT command does, adding ALL languages/settings, but the idea (mine at least) was to use it with the /l <locale> switch:

http://technet.micro...7(v=ws.10).aspx
in order to see what was the "minimal" size with:
a. plain "English" (or other "western" language) <- this should need not any particular added font/unicode/whatever :unsure:
b. your own "larger" locale because of East Asian Fonts (i.e. "your" full bcdboot output)

 

I think the minimal size of files to sucessfully boot Windows is about 419 KiB. That's a 295 KiB "bootmgr" and a 24 KiB "BCD" file.

 

In my previous experiments, I use bcdboot to get a default output (without /l option, my locale is English), but these files still include East Asian fonts and MUI files. Then I delete all the files except "bootmgr" and "BCD" and the folder "Boot". It can still boot Windows successfully.

 

 

Maybe doing some patch / hack on these two files can result a even smaller size. But I do not see the meaning of doing so. Windows is only a gaming platform for me while my main OS is Arch Linux. :-) Booting into Windows with a 10 MiB file or a 100 MiB file does not mean a lot of difference to me :-)

 

 

Oh, BTW, in my early experiments, I try to use a full dump (dd_rescue /dev/sdb sdb.img) of my MBR USB flash disk to boot Windows, it failed with "cannot find memory target" error. The capacity of the flash disk is 4 GB so the full dump of it is 4 GB in size, which, I think, is too large for memdisk to load.  :loleverybody:


Edited by Sascha Weaver, 27 May 2014 - 10:45 AM.


#55 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 05 August 2014 - 06:31 AM

HI guys, back after a long time.

 

@cdob and @SaschaWeaver : Did you both use floppy image? or HDD Image? If HDD, then was anything required to be written to the PBR of the partition in the HDD image ?



#56 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 05 August 2014 - 01:00 PM

Never mind... I was able to do the same as Wzyboy with floppy image. Now got to try it with an MBR harddisk...

<excitement>

 THIS IS SO COOL!

</excitement>


#57 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 05 August 2014 - 01:42 PM

 

Never mind... I was able to do the same as Wzyboy with floppy image. 

 

 

... but in the meantime we got (a bit :unsure:) ahead, JFYI,  ;):

http://reboot.pro/to...im-wim9mgr-wim/

cdob found that it is possible to have just the \boot\BCD on the floppy (if the case) see starting from here:

http://reboot.pro/to...r-wim/?p=186211

and devised yet another nice approach, an "only" BOOTMGR and a floppy image built on-the-fly (or premade as in the example attached to my last post):

http://reboot.pro/to...r-wim/?p=186287

to which you dd just the \boot\BCD

 

:duff:

Wonko



#58 cdob

cdob

    Gold Member

  • Expert
  • 1469 posts

Posted 05 August 2014 - 02:27 PM

Did you both use floppy image?

I was able to do the same as Wzyboy with floppy image.

I used a floppy image with a bootmgr PBR.

Now got to try it with an MBR harddisk...

I haven't used a MBR hard disk at this condition.
 

... but in the meantime we got (a bit :unsure:)

This uses grub4dos. How to load grub4dos from a GPT disk?
http://reboot.pro/to...in-a-gpt-setup/
Maybe Syslinux MBR, syslinux and grub.exe. Does grub.exe finds menu.lst at a GPT disk?

#59 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 05 August 2014 - 04:05 PM

This uses grub4dos. How to load grub4dos from a GPT disk?
http://reboot.pro/to...in-a-gpt-setup/
Maybe Syslinux MBR, syslinux and grub.exe. Does grub.exe finds menu.lst at a GPT disk?

Why not, as in the thread you mentioned, have a hybrid MBR/GPT and a small partition dedicated to grub4dos and accompanying files?

 

I mean the GPT issue can be worked around with a hybrid MBR IMHO, the problem is with EFI/UEFI, I believe. 

 

The EFI Syslinux version (besides seeming at first sight "experimental") misses (still) memdisk:

https://wiki.archlin...f_UEFI_Syslinux

 

We could make a "special" grldr (or - possibly a grub.exe) with the "embedded" menu.lst modified to suit, but the issue woudl still be the lack of BIOS interface. :dubbio: but BIOS+GPT should be entirely possible.

 

:duff:

Wonko



#60 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 06 August 2014 - 03:20 PM

Wow nice. The spoof an empty image and then write bootmgr and bcd onto it is interesting. Sha0 had talked about how wimboot can do something similar, it does feel like a cleaner approach, since the image is never actually fully formed. Or is your method also similar in this respect?

 

@Wonko the Sane : Using a hybrid MBR defeats the whole purpose of this hack/project. Hybrid MBRs are quite dangerous and ugly, and require both MBR and GPT partition tables to be synchronised. And one could run windows directly using a hybrid MBR, no grub4dos or other boot manager needed.

 

Yes technically you perhaps maybe make an MBR disk that contained 1 or 2 or 3 GPT partitions and one "hidden" partition for grub4dos. That would however be the most flagrantly spec-violating arrangement in existence. Much easier to update grub4dos.

 

You could however store MBR on a GPT partition. But then, there are few advantages. bootmgr is perhaps the only modern software not to support GPT, and we're all working to solve it.


Edited by milindsmart, 06 August 2014 - 03:44 PM.


#61 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 06 August 2014 - 04:52 PM

@milindsmart

Maybe you could make a post re-stating the actual objectives/goals (and limititations) of the whole exercise, it is very possible that I lost focus from what the original point was, and it is likely that following Sascha Weaver experiments I am going "off topic".

 

If I got it right, he used GRUB2 (the thing that the good Linux guys call GRUB) as "main" bootloader on a GPT disk (but in BIOS mode).

From GRUB2 you can - as he did - chainload memdisk and have it load an image (be it of a floppy or of a small HD like device).

I don't see why (on BIOS) this image cannot contain the actual grub4dos and one or more floppy images.

 

Or is there an issue with this approach? :unsure:

 

:duff:

Wonko



#62 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 06 August 2014 - 06:15 PM

Sure. The objective :

Boot Windows on a GPT disk on BIOS machines without lying to the OS about the firmware (UEFI emulation), disk partitioning (Hybrid MBR) or any other major aspect of the hardware. 

 

Sascha Weaver did exactly that, which you've understood perfectly. I also have been able to replicate his setup.

 

Grub4dos would have no issues being bootstrapped by GRUB on GPT. My only question would be, why Grub4dos? What's so exclusive to it?



#63 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 06 August 2014 - 07:24 PM

OK, now I don't get it.

The "plain" floppy image as Sascha Weaver reported and as you replicated works with GRUB2 + memdisk to map a "monolithic" floppy (or hard disk) image.

 

The "exclusive" part of grub4dos maybe the feature of being far more flexible in mapping and creating modifying images, i.e. allow more "options" (whethr actually *needed* or only *fancied*).

 

In the meantime I did one of my half-@§§ed tests that seemingly work to boot directly to grub4dos. :unsure:

 

I am far from being able to provide a repeatable/detailed set of instructions, and have not the means/way to test it thoroughly, but basically, taking advantage from the fact that there is a "hole" in grldr.mbr (originally intended for the backup of the pre-grub4dos-install MBR), I tried the following (on XP 32 bit, i.e. with no "native" GPT support at all):

1) created a GPT disk with GDISK http://www.rodsbooks...k/download.html + added a single partition to it

2) left the GPT header on sector 0x01 http://thestarman.na...asm/mbr/GPT.htm

3) moved the actual partition entry from sector 0x02 to sector 0x3E (changing also the "FirstUsableLBA" from 0x22 to 0x5E and rebuiilding the CRC32 checksum

4) deleted the backup at the end of the volume

5) used GDISK to repair the GPT backup

6) edited the protective partition table entry to start from 2048 (instead of 1)

7) changed the partition type from 0XEE to 0x0B

8) backed up sector 0X01 (GPT header)

9) used bootice to install to the MBR the grldr.mbr

10) restored the backup (GPT header) to sector 0X01 (overwriting the backup of the MBR that the bootice wrote to it)

11) mounted and formatted the  volume as FAT32

12) copied to it grldr

13) changed back partition type to 0xEE

14) made the partition active

15) ran GDISK to "repair" the whole thing until everything was ok with it with exception of:

 

Warning: The 0xEE protective partition in the MBR is marked as active. This is
technically a violation of the GPT specification, and can cause some EFIs to
ignore the disk, but it is required to boot from a GPT disk on some BIOS-based
computers. You can clear this flag by creating a fresh protective MBR using
the 'n' option on the experts' menu.

 

 

and:

Warning: 0xEE partition doesn't start on sector 1. This can cause problems
in some OSes.

 

16) tested the image in Qemu

17) booted to grub4dos fine

 

( I may have done some more hex editing and Gdisk repairs when experimenting, but the ones I listed should be the ones needed, YMMV :ph34r:

 

Cannot really say how "compatible" this setup can be, but it seems to me - if confirmed to work :dubbio: - a kind of "semi-hybrid" approach, with the least "deviations" from the stupid GPT "pseudo-standard".

 

:duff:

Wonko



#64 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 06 August 2014 - 08:21 PM

Can Grub4Dos do things that syslinux cannot?

It worked? Am a bit surprised. But you haven't created any GPT partitions... Only it's support infrastructure. Also have a look at this insightful email archive of a (lib)parted dev.
 
BTW, did you intend to reply to http://reboot.pro/to...setup/?p=186317 ?
 
Shall I reply there or here?

Edited by milindsmart, 06 August 2014 - 08:47 PM.


#65 cdob

cdob

    Gold Member

  • Expert
  • 1469 posts

Posted 06 August 2014 - 09:35 PM

Can Grub4Dos do things that syslinux cannot?


Given a half generic bcd file, read partition data and edit \boot\bcd on the fly: fix bcd to match partition data.
http://reboot.pro/to...e-2#entry145675

Change partition layout, use the same floppy image and it's bootable still.

This approach works at a MBR disk, no idea if this can be ported to a GPT disk.

#66 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 07 August 2014 - 06:28 AM

@cdob : Whoa. You're not even using partitions... just scanning the disk and reading+manipulating bytes. Doing this, you can use ANY partitioning system on planet. GPT cannot stop you if you're ignoring its presence completely.

But why? Why not use some partition module? Any particular reason...? Also, are there no boot-time modules to read+parse the BCD format properly and edit it? Some open-source library somewhere....?

But the final result is extremely flexible... edit BCD on the fly to work with whatever is available on the disk.

Edited by milindsmart, 07 August 2014 - 06:31 AM.


#67 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 07 August 2014 - 09:03 AM



It worked? Am a bit surprised.

Which is good. :)
 

But you haven't created any GPT partitions...

Haven't I? :unsure:

 

C:\batches\GPTTESTS>gdisk32.exe 6:
GPT fdisk (gdisk) version 0.8.10

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): v

Caution: More than one 0xEE MBR partition found. This can cause problems
in some OSes.

No problems found. 1954 free sectors (977.0 KiB) available in 1
segments, the largest of which is 1954 (977.0 KiB) in size.

Command (? for help): p
Disk 6:: 4096000 sectors, 2.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 9AD70591-A062-4C71-86BB-FBAC77E38102
Partition table holds up to 128 entries
First usable sector is 94, last usable sector is 4095966
Partitions will be aligned on 2048-sector boundaries
Total free space is 1954 sectors (977.0 KiB)

Number Start (sector) End (sector) Size Code Name
1 2048 4095966 2.0 GiB 0700 Microsoft basic data

Command (? for help):

 

 

:duff:

Wonko



#68 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 07 August 2014 - 12:23 PM

[...]taking advantage from the fact that there is a "hole" in grldr.mbr (originally intended for the backup of the pre-grub4dos-install MBR), [...]

 
Aha... That's pure genius. I didn't understand this at first. Very ingenious overall, I must say. But its useful only for GRUB and Grub4Dos, because every other boot manager can use PBR/VBR of a given active volume instead to store their raw bootcode. Coming to think of it, even GRUB can. Using the BIOS Boot partition and Legacy Bootable GPT flag one can do it completely spec compliant. No matter which "feature" of MBR booting you take, GPT has a better way of doing it.

Cannot really say how "compatible" this setup can be, but it seems to me - if confirmed to work :dubbio: - a kind of "semi-hybrid" approach, with the least "deviations" from the stupid GPT "pseudo-standard".

Yeah you can term it semi-hybrid, where the only spec-violation is the position of 0xEE protective MBR partition. But fully compliant > semi-hybrid surely? And why all the hate against GPT? What does it do wrong, according to you?

#69 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 07 August 2014 - 01:58 PM

No hate at all :), simple statement about the GPT:
1) being NOT a standard (as when noone respects something, it is not a standard)
2) being it stupid (and I mean it).

Read attentively the given page:
http://thestarman.na...asm/mbr/GPT.htm
(besides the set of related gdisk pages: http://www.rodsbooks.com/gdisk/ ) to understand how each and every developer/programmer have found ways to NOT respect the actual "standard" (and this is NOT a fault of the good guys that wrote the standard) and to understand how stupid the standard is (and this is their fault).

A few examples:
  • Field "Revision": a 32 bit field to host a numerical value, currently 0x01, allowing up to 0xFFFFFFFF, i.e. 4,294,967,295, let's say that they are expecting that the standard will be "often" revised
  • Field "HeaderSize": a 32 bit value x that should always be 92<x<sector size ad that is instead always 92 (and I would presume that since the most disrupting late change is from a sector size "standard" from 512 bytes to 4096 bytes, it will take some time before a device with a sector size of 4,294,967,295 bytes will be in use
  • Field "HeaderCRC32": OK, though a CRC32 makes no real sense, a simpler checksum would have done fine, and typically a CRC32 is appended to the "transmitted" data (not inserted in the middle of it)
  • Field "Reserved": OK. (though since the length of the header is variable it would have been more senceful, instead of leaving "a hole" to add optional fields enlarging the header depending on "version")
  • Field "myLBA": the GPT header (which is actually coded as "EFI PART", which obviously makes no sense, since the GPT partitioning scheme is NOT exclusively linked to EFI), uses a 64 bit field which is always 0x01 (as seemingly the GPT header must reside on LBA1) :dubbio:
  • ...

BTW, there is no need for the "fake" protective partition to be set as active with flag 0x80 as the grldr.mbr will find the grldr allright with it set to 0x00.
So the only "deviation" remaining would be that the protective partition doesn't start on LBA 1.

:duff:
Wonko

#70 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 08 August 2014 - 02:55 PM

OK. :smiling9:

Just tested the "trick" of leaving the "EFI PART" GPT header on LBA1 and moving the GPT partition entry.

 

It works fine with wee :thumbsup: and a "RAW" following grldr (but actually this can be placed *anywhere* on disk. :yahoo:

 

Since partitions are expected to be starting on 2048 sectors or multiples, I moved the GPT partition table to 0x0C00 (ending on 0x0C10), i.e. 3072, so the first partition entry available is on 0x1000 or 4096.

 

This leaves us with free sectors from 63 to 3071. one could also move the GPT partition table further on, until it is bordering the 4096 sector or start of first partition.

 

One could also try using, see:

http://reboot.pro/to...t-a-filesystem/

the 127 sectors version of Wee that allows direct floppy mapping and dd a floppy (or superfloppy image) in this space, this would allow much more freedom to have, besides grldr whatever menu.lst or grub4dos batches to translate/map the existing GPT partitions (of course within the addressable roughly 2 Tb space) blocklists to mapped devices accessible by grub4dos.

 

Once booted to grub4dos, all your (accessible) sectors are belong to us. ;)

 

This way the "protective" MBR is left untouched, with a single non-active EE partition spanning from LBA1 to the end of disk.

Gdisk is perfectly fine with this:

C:\batches\GPTTESTS>gdisk32.exe 6:
GPT fdisk (gdisk) version 0.8.10

Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): v

No problems found. 1024 free sectors (512.0 KiB) available in 1
segments, the largest of which is 1024 (512.0 KiB) in size.

Command (? for help): p
Disk 6:: 1228800 sectors, 600.0 MiB
Logical sector size: 512 bytes
Disk identifier (GUID): 79134ECB-BF4F-46AF-AE4E-76FC9404A732
Partition table holds up to 128 entries
First usable sector is 3072, last usable sector is 1228766
Partitions will be aligned on 2048-sector boundaries
Total free space is 1024 sectors (512.0 KiB)

Number Start (sector) End (sector) Size Code Name
1 4096 1228766 598.0 MiB 0700 Microsoft basic data

Command (? for help): q

And also a quick test from a WIndows 7 in a VM shows how it is recognized as GPT disk.

 

:duff:

Wonko



#71 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 09 August 2014 - 09:25 AM

No hate at all :), simple statement about the GPT:
1) being NOT a standard (as when noone respects something, it is not a standard)

Agree... A standard is a real standard only when people follow it. But still, it is a proposed standard.

2) being it stupid (and I mean it).
[...] and to understand how stupid the standard is (and this is their fault).

It's not all that stupid, just being ABUNDANTLY cautious and generous. Remember ,there's JUST ONE GPT header and Partition entry table. Wasting BYTES is NOT an issue.
 

  • Field "Revision": a 32 bit field to host a numerical value, currently 0x01, allowing up to 0xFFFFFFFF, i.e. 4,294,967,295, let's say that they are expecting that the standard will be "often" revised

Lol. Yeah perhaps, considering these points you just raised, they DO intend to revise, though not as heavily as this :D

 

  • Field "HeaderCRC32": OK, though a CRC32 makes no real sense, a simpler checksum would have done fine, and typically a CRC32 is appended to the "transmitted" data (not inserted in the middle of it)
  • Field "Reserved": OK. (though since the length of the header is variable it would have been more senceful, instead of leaving "a hole" to add optional fields enlarging the header depending on "version")

 

Never thought about this, I completely agree with you... These 2 seem bonehead decisions.
 

 

  • Field "myLBA": the GPT header (which is actually coded as "EFI PART", which obviously makes no sense, since the GPT partitioning scheme is NOT exclusively linked to EFI), uses a 64 bit field which is always 0x01 (as seemingly the GPT header must reside on LBA1) :dubbio:

 

GPT is a PART of the UEFI specification. It is encouraged to be used along with UEFI firmware. UEFI + MBR and BIOS + GPT are incidentally supported. The idea is that MBR and BIOS, both antiquated de facto standards should be replaced by real planned standards
 

BTW, there is no need for the "fake" protective partition to be set as active with flag 0x80 as the grldr.mbr will find the grldr allright with it set to 0x00.
So the only "deviation" remaining would be that the protective partition doesn't start on LBA 1.

 

OK. :smiling9:
Just tested the "trick" of leaving the "EFI PART" GPT header on LBA1 and moving the GPT partition entry.
  
This way the "protective" MBR is left untouched, with a single non-active EE partition spanning from LBA1 to the end of disk.
[...]
And also a quick test from a Windows 7 in a VM shows how it is recognized as GPT disk.


In fact, I was thinking this exact same thing prompted by your previous post : GPT does NOT restrict us from using raw code directly from sectors... And the gaps that are possible make it EASY to go even more crazy than possible in MBR disks in putting all sorts of code in all sorts of corners and gaps, while remaining COMPLETELY compliant.

BUT : There is a slightly better way! Use a BIOS Boot Partition and put whatever raw code you want there.... no interference of file systems, but ALSO sure that no random installer is going to assume your bootloader/ISO as free space :) Why WOULD anyone not want to use partitions instead of random sectors is beyond me. Just assign the right attributes and no OS will even show it.
 



#72 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 09 August 2014 - 09:48 AM

@cdob and Sha0  I managed to boot using wimboot with NO pre-existing image!

 

My setup : 

gpt1 : NTFS

/bootmgr.exe

/Boot/BCD

 

gpt2: BIOS Boot Partition with raw Grub2 core.img code

 

gpt3 : FAT

/wimboot

/grub2/grub.cfg

/grub2 [other files+folders]

 

/gpt4 : NTFS

/Windows

 

My grub.cfg entry :

menuentry 'Windows GPT bootmgr wimboot' {
    linux16 /wimboot
    initrd16 \
        newc:bootmgr.exe:(hd0,gpt1)/bootmgr.exe \
        newc:bcd:(hd0,gpt1)/Boot/BCD  
}

Windows boots fine. Works with bootmgr & bootmgr.exe equally well.



#73 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 09 August 2014 - 03:58 PM

In fact, I was thinking this exact same thing prompted by your previous post : GPT does NOT restrict us from using raw code directly from sectors... And the gaps that are possible make it EASY to go even more crazy than possible in MBR disks in putting all sorts of code in all sorts of corners and gaps, while remaining COMPLETELY compliant.

BUT : There is a slightly better way! Use a BIOS Boot Partition and put whatever raw code you want there.... no interference of file systems, but ALSO sure that no random installer is going to assume your bootloader/ISO as free space :) Why WOULD anyone not want to use partitions instead of random sectors is beyond me. Just assign the right attributes and no OS will even show it.
 

 

I may be wrong, but you sound like believing that the "Bios Boot Partition" is *somehow* part of *a* "standard".

This is not exact. :(

 

The "Bios Boot Partition" is a "hack" done by the good GNU/GRUB2 guys, as a matter of fact it is a "proprietary" approach, that though (obviously) well devised and coded :), is far form being properly documented:

https://www.gnu.org/...stallation.html

i.e. it is unsuitable to be even called a "proposed extension" to the "standard", they simply put together "a way out" for the GPT disks on BIOS and implemented it in their tool.

BTW, by design the "BIOS Boot Partition" does not even contain a filesystem, and as such it is not anything different from a "RAW" blocklist.

 

So, we have a de facto non-standard and the good Linux guys of the GNU/GRUB2 project put together a non-standard way to modify or "hack" it (for use with their own bootloader).

http://forums.fedora...ad.php?t=270586

http://forums.fedora...86&postcount=12

 

 

 

http://www.rodsbooks...sk/booting.html

BIOS-based computers, whether they use MBR or GPT, rely on a boot loader in the first sector of the disk to help get the computer booted. In fact, the first 440 bytes of the MBR data structure are devoted to this boot loader. DOS and Windows place a very simplistic boot loader in this space. Other OSes and third-party utilities enable placing more sophisticated boot loaders in the MBR, although these boot loaders usually rely on multiple stages—the boot loader code loads a secondary boot loader that's located elsewhere, and that boot loader may even load a third stage. In principle, these boot loaders can work just fine when the MBR is in fact a GPT protective MBR. In practice, the boot loader needs to be GPT-aware in order to work. The GRUB 2 boot loader, when used on a GPT disk, works best when you to have a BIOS Boot Partition (GPT fdisk code EF02) on the disk. Most boot loaders, including the patched versions of GRUB Legacy (version 0.97) that include GPT support, don't require a BIOS Boot Partition. If you do need it, the BIOS Boot Partition can be quite small—sometimes as small as 32 KiB, although I've seen reports that some configurations require more space than this—sometimes over 1 MiB. If you align your partitions to 1 MiB boundaries, 1 or 2 MiB is the logical size.

 

 

I feel free to invent my own non-standard ways to hack this non-standard ;).

 

Reality check :w00t:

  1. BIOS reads the MBR (first absolute sector of the boot device)
  2. A "normal" MBR  code does more or less the following:
  3. parses the partition table (that the defacto standard sets at offset 446 in the same sector, with 4 entries)
  4. checks that one (and one only) of the partition entries is Active (and primary)
  5. loads the corresponding partition bootsector
  6. do whatever the bootsector is supposed to do 

more or less, the only "intelligence" the MBR code has is to recognize the (BTW simple) way partitions (please read as disk extensions) are mapped in the partition table and load the "right" address on disk.

 

What the "special" MBR for a "BIOS boot partition" does is instead:

  1. BIOS reads the MBR (first absolute sector of the boot device)
  2. the "special" MBR code does more or less the following:
  3. parses the partition table (that the defacto standard sets at offset 446 in the same sector, with 4 entries) to find the "EE protective" partition entry to find the start address of the "EFI PART" header or (OUTside of *any* standard) assumes that it is on LBA1, or (still OUTside of *any* standard) only parses the first of the four entries
  4. parses the "EFI PART" header to find the start address of the GPT partition table or (OUT of *any* standard) assumes that it is on LBA2
  5. finds the ID of "21686148-6449-6E6F-744E-656564454649" or "Hah!IdontNeedEFI" (which is ALSO OUTside of *any* standard) addresses (coded in complex way)
  6. loads the corresponding partition bootsector
  7. do whatever the bootsector is supposed to do <- the *whatever* is ONLY a specific loader (GRUB modified or GRUB2) that "knows" natively how to deal with the GPT partitioning scheme

To sum up, BIOS/MBR is simple (but limited in accessible size), EFI/GPT is not limited by size but is complex (and this extreme complexity led to a "standard" that is more than 2000 pages, that noone - seriously - can expect to actually fully comply with) and since the Intel geniuses that wrote the "standard" did not even think of a possibility to give BIOS accessibility, the good GRUB2 guys invented their own way.

 

More than a "proposed" standard, it is something that being overly (and needlessly) complex and having been "forced down the throat" of programmers/users led to a number of absolutely non-standard implementations, thus nullifying the whole  validity of a "standard".

 

It is a situation not entirely different from the good ol' ISO9660 (or ECMA 119) for YEARS Bios and OS programmers have implemented non-standards around it, and some aspects (actually contained in the standard if read properly) have been misinterpreted till recently (though related a far less "important" matter), JFYI:

http://www.msfn.org/...ppy-emulation/ 

 

 

grub4dos - currently - knows nothing about GPT partitioning schemes, BUT is (IMHO) more flexible or, in any case, is a tool with which I am familiar and that allows me far more "freedom" than anything else.

 

Before or later the good grub4dos guys will have time (and will) to start experimenting with GPT but until then, we need to find other ways (and it is entirely possible that they will come out with another non-standard way to boot/load an OS on a GPT disk).

 

Though very improbable :dubbio:, it is also possible that the good GRUB2 guys (the ones that senselessly renamed GRUB to "GRUB legacy" and GRUB2 to "GRUB", thus creating among the biggest set of misunderstandings in computing history) will come to their senses and realize that grub4dos has tens of features in addition to their creature and they will start porting these features into GRUB2, but until then the U in GRand Unified Bootloader should more properly be used for GRand Unix Bootloader or GRand Unflexible Bootloader ;).

 

I am just putting together a different way to boot grub4dos from a GPT disk, it will take some time to be assembled, but I'll post it here as soon as I have a somewhat working prototype. 

 

:duff:

Wonko



#74 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 10 August 2014 - 08:48 AM

Wonko, with all due respect, I think you have a warped view of the standard. The basic facts are all correct, but your interpretation is... unsupported.
 
You are right, the BIOS Boot Partition is not part of the standard. The GRUB guys have picked one particular GUID for the purpose of holding raw boot code. You must remember that OSes and utilities are NOT supposed to mess with a partition if they don't recognize the GUID. There are too many GUIDs available to squabble over, so if you want to do something special, just pick your own GUID and use it as the partition type. Calling it proprietary is misleading.
 

So, we have a de facto non-standard and the good Linux guys of the GNU/GRUB2 project put together a non-standard way to modify or "hack" it (for use with their own bootloader).


GPT is not a non-standard. The standard ONLY says, please put your code in SOME partition. Rest is upto you.

So it's entirely within the standard to do what you want with a partition : putting a filesystem or not is completely a choice. It is most certainly NOT a hack.

It is not a de facto standard. It is a formal standard that went top-down... Standards committee to OS developers to firmware designers to consumers. A de facto standard is generally bottom-up : OS devs come up with something, and the firmware guys come up with something else, and by market forces one of them "wins".
 

I feel free to invent my own non-standard ways to hack this non-standard ;).


Please do, just like we are all doing. But be rest assured that it won't be a hack. Just a creative use of a standard, which is the whole point of making a standard : allow creativity without worrying about compatibility.
 
  • ...
  • do whatever the bootsector is supposed to do <- the *whatever* is ONLY a specific loader (GRUB modified or GRUB2) that "knows" natively how to deal with the GPT partitioning scheme

The *whatever* is ANY boot loader which knows how to deal with GPT Partitioning Scheme. It is NOT specific. Any boot loader can be written to work with GPT. It's an open standard. I downloaded the damn spec at least 5 different times.
 

To sum up, BIOS/MBR is simple (but limited in accessible size), EFI/GPT is not limited by size but is complex (and this extreme complexity led to a "standard" that is more than 2000 pages, that noone - seriously - can expect to actually fully comply with) and since the Intel geniuses that wrote the "standard" did not even think of a possibility to give BIOS accessibility, the good GRUB2 guys invented their own way.


What do you want them to have done? They allowed the use of the MBR! Once you have raw code that you can use, Anyone can do anything... The GPT definition is specified in rigorous detail, thereby allowing the boot loader to find any partition it wants. After that it's up to the boot loader. What could Intel have done to improve accessibility?
 

More than a "proposed" standard, ...


Proposed and published and followed and implemented and used =/= proposed.
 

It is a situation not entirely different from the good ol' ISO9660 (or ECMA 119) for YEARS Bios and OS programmers have implemented non-standards around it, and some aspects (actually contained in the standard if read properly) have been misinterpreted till recently (though related a far less "important" matter), JFYI:
http://www.msfn.org/...ppy-emulation/ 


El Torito is a hack. That they didn't do away with it in UEFI is a pity... 
 
 

grub4dos - currently - knows nothing about GPT partitioning schemes, BUT is (IMHO) more flexible or, in any case, is a tool with which I am familiar and that allows me far more "freedom" than anything else.

You're right that Grub4Dos does seem to have some pretty awesome exclusive features... But rewriting something to use GPT instead of MBR is not something that is so difficult as to not be done in the past 3-4 years of the ascendancy of UEFI.
 

Before or later the good grub4dos guys will have time (and will) to start experimenting with GPT but until then, we need to find other ways (and it is entirely possible that they will come out with another non-standard way to boot/load an OS on a GPT disk).

 
I'll believe you when they rename it to something other than Grub4DOS  :angry: . BTW what other ways need to be found? 
 

Though very improbable :dubbio:, it is also possible that the good GRUB2 guys (the ones that senselessly renamed GRUB to "GRUB legacy" and GRUB2 to "GRUB", thus creating among the biggest set of misunderstandings in computing history) will come to their senses and realize that grub4dos has tens of features in addition to their creature and they will start porting these features into GRUB2, but until then the U in GRand Unified Bootloader should more properly be used for GRand Unix Bootloader or GRand Unflexible Bootloader ;).

Of course GRUB is the one at fault here : Even with GPT it still relies on hardcoded sector numbers to find a partition, instead of just scanning the damn partition table. Simply shrinking or expanding a partition results in the grub rescue prompt... If the syslinux gptmgr can scan for and jump to an active partition in 512 bytes, what prevents grub from doing it in 1MB?

#75 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 10 August 2014 - 11:11 AM

El-Torito is NOT a "hack" it is part of ISO9660/ECMA117 (which ARE actually a "standard").

 

The fact that this standard has been poorly implemented in several tools and firmwares, and that it took several years to have (some) largely compliant software and firmware is only a confirmation on how tough it is to make things "fully compliant", and this happened with a relatively simple (when compared to UEFI/GPT) set of specifications.

 

EFI/GPT is right now - at the most - an "industry standard".

 

The fact that noone respects it fully is a sign that - at the moment - it is not "standard", though it is possible that ISO or ECMA (or both) will take it into a "proper" standard.

 

Right now it is a set of "specifications".

 

The good GNU/GRUB guys solved a problem with their own bootloader/bootmager, i.e. what is called "BIOS boot partition" should be called "GRUB2 BIOS boot partition".

 

Any programmer can pick a GUID (there are so many of them ;)) and use the same approach :).

 

But what about non-programmers?

Should they be limited to use the Microsoft bootloader or GRUB2?

Or maybe they could use a few "tricks" and use what is already available?

 

Of course these would be highly experimental/dangerous :ph34r: and more than anything else half-@§§ed "hacks"

Test for the day :whistling:

Make a test GPT disk image, add to it whatever partitions you see fit, using any tool you have available (like MS diskpart and/or GPT gdisk), only make sure (this would be the default for any tool) that the partition(s) are aligned to 2048 sectors.

Then dd the GPT_MBLDR.mbr to the MBR of the image (LBA0) and dd the GPT_GRLDR.ima starting on sector LBA 63.

Try booting from the GPT image.

What happens? :unsure: (It boots in qemu fine).

 

:duff: Wonko

Attached Files







Also tagged with one or more of these keywords: bios, gpt, bootmgr, winload

4 user(s) are reading this topic

0 members, 4 guests, 0 anonymous users