Multiboot UEFI boot-capable thumb drive for Win 7/8 & Linux installation

Hi,

Ok, I am trying to create a bootable thumb drive that will have the following features:

• Boots in UEFI mode
  
  • I can get it to do this with an installation of GRUB (not GRUB-legacy)
• Boots in standard BIOS mode (via the compatibility module in EFI)
  
  • This I can get GRUB to do
• Can install multiple Linux distributions
  
  • Again, GRUB will do this, though I might have some problems with Fedora
• Can install all versions of Windows 7 and 8
  
  • This is where I am running into problems
  • Grub4Dos will not work for this, it runs in real mode, which I can't get into when in EFI mode
  • I'm not sure I can even get into Syslinux from GRUB, and I think it needs real mode as well
  • Windows won't boot from a loop device, so I can't use loopback
• Boot to a DOS prompt
  
  • I don't think this is possible, since a DOS prompt needs real mode

My current thought is that I can get this working if I can find a way to mount the ISO's in a way that Windows can understand. This is what the thumb drive I currently use does, but it won't boot to UEFI since I am using Syslinux, which doesn't boot to UEFI (actually, I'm technically using grub4dos and WinFira, with Syslinux being the bootloader).

I know that I can boot to a Windows installation if it's extracted to the root, but the problem with this is that only one Windows OS can be set up this way at a time, since it needs to be in the root. If GRUB had file manipulation capabilities, then I might be able to make this would work, but I don't think it does. Obviously, I could do this with a script in an OS, but I would rather not. This installation will be used by multiple people so I need to keep it as simple as I can.

Have you looked more at the RMPrepUSB site? He is also active here, but there are a lot of good tutorials on his site!!!

See especially #43 http://www.rmprepusb...ials/firawiniso While more oriented to grub4dos, you may be able to extrapolate and use the techniques with grub


Scott

May I dare to ask WHY is actual UEFI booting needed?

As a workaround (maybe useful, maybe not ):
http://xpcboot.weebly.com/
http://pete.akeo.ie/...-usb-stick.html
or use a hybrid GPT/MBR disk, like:
http://www.rodsbooks.com/gdisk/
http://www.rodsbooks...isk/hybrid.html


Wonko

Have you looked more at the RMPrepUSB site? He is also active here, but there are a lot of good tutorials on his site!!!

See especially #43 http://www.rmprepusb...ials/firawiniso While more oriented to grub4dos, you may be able to extrapolate and use the techniques with grub


Scott

I actually have a thumb drive set up using those instructions, it's just not UEFI capable. The problem that I am running into is mounting a Windows ISO in GRUB. The tools for doing that all seem to require being in real mode, which is not available from GRUB in UEFI mode.

May I dare to ask WHY is actual UEFI booting needed?

I work in a hardware validation lab at Intel. Specifically, we do platform compatibility validation. This means we are often installing OS's onto our systems. Because UEFI is becoming more prevalent (especially with the changes introduced in Windows 8), some of those installs MUST be done with UEFI enabled to ensure that everything works right. Putting all of the images onto one thumb drive that I can boot from would be really useful. I can do that right now, using the thumb drive I created with instructions from RMPrepUSB. Except that that thumb drive won't boot to UEFI, which means that I have to have separate media for UEFI, which kinda makes the thumb drive redundant since installation disks can be used for both standard installs and UEFI installs.

As a workaround (maybe useful, maybe not ):
http://xpcboot.weebly.com/
http://pete.akeo.ie/...-usb-stick.html
or use a hybrid GPT/MBR disk, like:
http://www.rodsbooks.com/gdisk/
http://www.rodsbooks...isk/hybrid.html


Wonko

I'll take a look at those websites.

Edited by cyphersmith, 13 September 2012 - 05:50 PM.

Sounds more like you might want to look at ISOStick (or the Zalman USB drive container), where they allow you to have a LOT of ISO's on the storage, and select one that then is emulated out on the USB as if it was a physical DVD/CD...

Seems like Intel could afford something like either of these...


Scott

Sounds more like you might want to look at ISOStick(or the ZalmanUSB drive container), where they allow you to have a LOT of ISO's on the storage, and select one that then is emulated out on the USB as if it was a physical DVD/CD...

Seems like Intel could afford something like either of these...


Scott

I have to say that I was unaware of the ISOStick, or the Zalman case. I would prefer the ISOStick, personally, but I haven't finished that thread, so I don't know if it's generally available yet. I'm looking into it.
EDIT: There is one other reason that I'm looking into methods using GRUB. We just bought 64GB USB 3.0 thumb drives for the whole team. I would like to use those so that we don't have to purchase more equipment.

Edited by cyphersmith, 13 September 2012 - 09:33 PM.

Good point, and it's always good to know if there are other ways to do the job...I'm no "grub" expert, but will be interested to see what you came up with.

Very good.

The given answer translates nicely to "I'm doing this for fun and to learn new things".
(reasons that Wonko fully approves of )

I mean, the burden of having two different USB sticks is unbearable, not only if you have to carry them with you, but all the space they occupy in the lab....




Another thing, maybe useful or maybe not, that you could probably make use of is the grub24dos:
http://reboot.pro/13282/
https://sourceforge.net/projects/grub24dos

now renamed/evolved to grub2win:
http://reboot.pro/17454/
http://sourceforge.n...jects/grub2win/

As it may allow better "interchange" between the various pieces of the puzzle.



Wonko

Very good.

The given answer translates nicely to "I'm doing this for fun and to learn new things".
(reasons that Wonko fully approves of )

I mean, the burden of having two different USB sticks is unbearable, not only if you have to carry them with you, but all the space they occupy in the lab....




Another thing, maybe useful or maybe not, that you could probably make use of is the grub24dos:
http://reboot.pro/13282/
https://sourceforge....jects/grub24dos

now renamed/evolved to grub2win:
http://reboot.pro/17454/
http://sourceforge.n...jects/grub2win/

As it may allow better "interchange" between the various pieces of the puzzle.



Wonko

Well, I took a look at grub4win, and I don't think I can get it to work. I think it works from the Windows bootloader, but I don't think that the Windows bootloader, but I'm not sure that you can get the Windows bootloader on a thumb drive to actually have a proper BCD menu from a thumb drive that doesn't have its RMB flipped so it's not seen as a removable drive . And I don't know if you actually can flip the RMB bit on the drive I have (it's a Lexar Triton USB 3.0 drive). The only thread I saw about doing that with this drive wasn't very encouraging (the drive was made unuseable).

I work in a hardware validation lab at Intel.

Ask your coworkers to integrate ISO file mount at BIOS/UEFI.
Boot a Intel motherboard and select a ISO file at a USB stick.

Well´next step would be a agreement at hardware and software manufacturer.
That's a suggestion to all manufacturers.
Assume a config file /efi/boot/mount
The UEFI writes ISO file name to config file /efi/boot/mount and OS read the config file.

but I'm not sure that you can get the Windows bootloader on a thumb drive to actually have a proper BCD menu from a thumb drive that doesn't have its RMB flipped so it's not seen as a removable drive .

The BCD menu works fine at a removable drive, at least at BIOS.

Should work at UEFI too Walkthrough: Boot Windows PE from a UEFI-based Hard Disk
Try it. Does BCD works at UEFI at a removable drive?

Ask your coworkers to integrate ISO file mount at BIOS/UEFI.
Boot a Intel motherboard and select a ISO file at a USB stick.

Well´next step would be a agreement at hardware and software manufacturer.
That's a suggestion to all manufacturers.
Assume a config file /efi/boot/mount
The UEFI writes ISO file name to config file /efi/boot/mount and OS read the config file.


The BCD menu works fine at a removable drive, at least at BIOS.

Should work at UEFI too Walkthrough: Boot Windows PE from a UEFI-based Hard Disk
Try it. Does BCD works at UEFI at a removable drive?

It's going to take me a bit to go through the information at that link. it looks like good information.
Mounting the ISO from EFI would be nice. That should actually be doable. An EFI program should be able to be written which will do exactly that. In fact, a lot can be done in EFI before really even hitting the bootloarder. I just don't have the knowledge myself to write it. Shoot, it can probably be set up to let the user choose the ISO to mount, mount it, then hand things over to whichever disk is mounted. Windows would likely still require a driver such as FiraDisk to see the drive, but that can be integrated into the ISO. Though I wouldn't be able to do it for Win8 x64 (or probably Server 2012 x64) because of the change that MS did to the signing method with it.

Mounting the ISO from EFI would be nice.

Is this a requirement? Do you require a ISO file?
Can you use another approach?

... but I don't think that the Windows bootloader, but I'm not sure that you can get the Windows bootloader on a thumb drive to actually have a proper BCD menu from a thumb drive that doesn't have its RMB flipped so it's not seen as a removable drive .

Well, it works, besides USB sticks (Removable or FIxed) even from floppies:
http://www.multiboot....uk/floppy.html

Just for the record, for MBR devices only Windows, and only NT based Windows AFAIK care about the Removable or Fixed status of a USB stick, and they ONLY do in the "protected mode" part, and the only issue is that only first partition (actually the partition which address is in first partition entry in the partition table) is mounted/mountale/accessed (unless you use a filter driver).
What happens with GPT is what you should actually test and know.


Wonko

Is this a requirement? Do you require a ISO file?
Can you use another approach?

Certainly, but one of the problems with Windows is that the install must be in the root directory. This is not the case for an ISO.

Well, it works, besides USB sticks (Removable or FIxed) even from floppies:
http://www.multiboot....uk/floppy.html

Just for the record, for MBR devices only Windows, and only NT based Windows AFAIK care about the Removable or Fixed status of a USB stick, and they ONLY do in the "protected mode" part, and the only issue is that only first partition (actually the partition which address is in first partition entry in the partition table) is mounted/mountale/accessed (unless you use a filter driver).
What happens with GPT is what you should actually test and know.


Wonko

I think you're right here. I hadn't thought of doing this using a GPT rather than an MBR. Since I don't think that an OS that can't recognize a GPT will be required, I could set up a GPT on the thumb drive and chainload each Windows OS install from its own partition. Assuming that Windows won't complain about that. A similar thing could be done for the various Linux distros, though they wouldn't need to have their own partitions. Of course, those can be ISOs (though some distros make that difficult).

Well, for the record, we have by now several ways to boot (and install) Windows NT based systems from a .iso or from "non-iso", as well we have ways to have a multi-partitioned MBR working allright (by simply changing the parition order or anyway writing the partition table on-the-fly with grub4dos or similar tools) that is using BIOS.
Besides the several methods listed here:
http://www.msfn.org/...ndows-from-usb/
Steve6375 has several tutorials on his site http://www.rmprepusb.com/

How these method are "portable" to UEFI (and since I cannot say how grub4dos behaves with either UEFI or GPT) and to have them working with grub2 (or similar) is actually the object of this topic, what I can provide (at the most) is info on how to do these things on "normal" BIOS/MBR, but all the rest is up to you, only trying to help you follow Mikhail Kalashnikov's advice :
http://reboot.pro/15298/#entry136216


Wonko

but one of the problems with Windows is that the install must be in the root directory.

Root directory from which part?
Use several partitions and flip MBR data on the fly. No idea at EFI mode.

VHD images are supported out of the box.
You may wrap installation files inside a VHD image.
As for USB stick: there are several VHD files.
As for windows install: root directory is used still.
A windows 7 example http://reboot.pro/14755/#entry131133

However native VHD mounting requires a NTFS partition.
Does EFI read NTFS partition? Most likely a addional FAT partiton is required.
Back to another multi partition approach. Should be possible at removable drive still.

I hadn't thought of doing this using a GPT rather than an MBR. Since I don't think that an OS that can't recognize a GPT will be required, I could set up a GPT on the thumb drive and chainload each Windows OS install from its own partition.

Yes, GPT at USB would be another approach.

Sample: Configure BIOS/MBR-Based Hard Disk Partitions by Using Windows PE and DiskPart
http://technet.micro...y/hh825677.aspx

Sample: Configure UEFI/GPT-Based Hard Drive Partitions by Using Windows PE and DiskPart
http://technet.micro...y/hh825686.aspx

Idea:
Add Windows and *nix to EFI System Partition
Let EFI bootmanger select between both. There is no need to chainload a bootmager.

However native VHD mounting requires a NTFS partition.
Does EFI read NTFS partition? Most likely a addional FAT partiton is required.
Back to another multi partition approach. Should be possible at removable drive still.

Windows and EFI share first partition limitation.

Given a removable USB stick at MBR mode: http://en.wikipedia....ter_boot_record
FAT16 first partiton - MBR offset +446 - bootmgr and \boot\
NTFS second partiton - MBR offset +462 - VHD image

Windows mount first partiton only at a removable stick. http://reboot.pro/16274/
Exception: a previous partition types 0x00 is ignored. The next partiton entry is mounted.
Bootmgr dosn't detect \boot\bcd at 0x00 partition type.

UEFI searches first partition to auto-detect a OS loader. http://en.wikipedia....mware_Interface
UEFI finds a file /EFI/BOOT/BOOTx64.EFI at a FAT 0x06 partiton type.
Contrary a partition type 0x00 is ignored: file /EFI/BOOT/BOOTx64.EFI is not found



Given a removable USB stick at MBR mode:
NTFS first partiton - MBR offset +446
FAT16 second partiton - MBR offset +462

FAT16 partiton type set to 0xEF
http://en.wikipedia....ystem_partition
At EFI mode file /EFI/BOOT/BOOTx64.EFI is not found here. EFI boot is missing
EFI partiton type 0xEF is ignored.


UEFI MBR mode and native VHD volumes are a riddle still.


Or goto GPT at USB?

Windows and EFI share first partition limitation.

Given a removable USB stick at MBR mode: http://en.wikipedia....ter_boot_record
FAT16 first partiton - MBR offset +446 - bootmgr and boot
NTFS second partiton - MBR offset +462 - VHD image

Windows mount first partiton only at a removable stick. http://reboot.pro/16274/
Exception: a previous partition types 0x00 is ignored. The next partiton entry is mounted.
Bootmgr dosn't detect bootbcd at 0x00 partition type.

UEFI searches first partition to auto-detect a OS loader. http://en.wikipedia....mware_Interface
UEFI finds a file /EFI/BOOT/BOOTx64.EFI at a FAT 0x06 partiton type.
Contrary a partition type 0x00 is ignored: file /EFI/BOOT/BOOTx64.EFI is not found



Given a removable USB stick at MBR mode:
NTFS first partiton - MBR offset +446
FAT16 second partiton - MBR offset +462

FAT16 partiton type set to 0xEF
http://en.wikipedia....ystem_partition
At EFI mode file /EFI/BOOT/BOOTx64.EFI is not found here. EFI boot is missing
EFI partiton type 0xEF is ignored.


UEFI MBR mode and native VHD volumes are a riddle still.


Or goto GPT at USB?

So I ran into a problem with using GPT on the thumb drive.Windows cannot boot to a GPT drive if you're using BIOS instead of EFI. I think, however, that I can get around this using Hybrid MBR's. Though I think that I might need to change partition order on the fly (since a removable disk is only supposed to have one partition).

I haven't done this yet, gonna try today, but here's what I'm thinking will work. Boot to grub. In the menu, there should be the following choices:

• EFI Windows 7 x64
• EFI Windows 8 x64
• Non-EFI Windows 7 x64
• Non-EFI Windows 8 x64
• Non-EFI Windows 7 x86
• Non-EFI Windows 8 x86
• Various linux distros

To do this, I will use GPT with hybrid MBR's. One for each of the 64-bit Windows installations, and one for the rest of the Windows installations. The Linux installations, I think, don't need to be in a hybrid MBR, though they can. The thing I am trying to figure out right now is how to set up the grub.cfg.

So I ran into a problem with using GPT on the thumb drive.Windows cannot boot to a GPT drive if you're using BIOS instead of EFI. I think, however, that I can get around this using Hybrid MBR's. Though I think that I might need to change partition order on the fly (since a removable disk is only supposed to have one partition).

I haven't done this yet, gonna try today, but here's what I'm thinking will work. Boot to grub. In the menu, there should be the following choices:

• EFI Windows 7 x64
• EFI Windows 8 x64
• Non-EFI Windows 7 x64
• Non-EFI Windows 8 x64
• Non-EFI Windows 7 x86
• Non-EFI Windows 8 x86
• Various linux distros

To do this, I will use GPT with hybrid MBR's. One for each of the 64-bit Windows installations, and one for the rest of the Windows installations. The Linux installations, I think, don't need to be in a hybrid MBR, though they can. The thing I am trying to figure out right now is how to set up the grub.cfg.

This didn't work for me.  I couldn't get Windows to see the partitions.  I am running out of ideas. 

 

If I can modify the directory structure while I was in GRUB or possible before then, I could probably do it.  Then I could have each Windows OS in a sub-directory, delete whatever is currently in the root and copy the version that is needed into the root.  I could do this manually before doing the install, it's just less than elegant.  Anyone know if this can be done by GRUB, or something else?  I suspect it could be done within the EFI shell as a shell script, so it may be time to learn EFI shell scripting.

This didn't work for me.  I couldn't get Windows to see the partitions.  I am running out of ideas. 

 

If I can modify the directory structure while I was in GRUB or possible before then, I could probably do it.  Then I could have each Windows OS in a sub-directory, delete whatever is currently in the root and copy the version that is needed into the root.  I could do this manually before doing the install, it's just less than elegant.  Anyone know if this can be done by GRUB, or something else?  I suspect it could be done within the EFI shell as a shell script, so it may be time to learn EFI shell scripting.

By default, Windows will not provide the partitions for your USB flash storage device.  You will need a filter driver, such as diskmod.  Hybrid MBR/GPT has some problems with older (U)EFI, so I'd abandon that approach.  For widest compatibility with (U)EFI and BIOS, I'd suggest a plain MBR, with a FAT partition whose type code in the partition table is 0xEF.

 

If you cannot use a disk filter driver, I'd say you have to drop one of:

• USB flash storage as the USB storage device
• Multiple partitions on the USB storage device

UEFI Multiboot USB - Windows 8 and Linux Parted Magic
 

http://reboot.pro/to...e-2#entry165182

 

I used portable USB-harddisk with Standard MBR and FAT32 Boot + NTFS System partition

 

After booting with Windows 8 from USB you can run WinNTSetup2

and use R-mouse to mount Setup ISO and prepare drive for Install of Windows 7 or 8 or XP

 

http://www.msfn.org/...-winntsetup-v2/

http://www.911cd.net...showtopic=24424

 

[attachment=14093:W8U-SetupW7.png]

By default, Windows will not provide the partitions for your USB flash storage device.  You will need a filter driver, such as diskmod.  Hybrid MBR/GPT has some problems with older (U)EFI, so I'd abandon that approach.  For widest compatibility with (U)EFI and BIOS, I'd suggest a plain MBR, with a FAT partition whose type code in the partition table is 0xEF.

 

If you cannot use a disk filter driver, I'd say you have to drop one of:

• USB flash storage as the USB storage device
• Multiple partitions on the USB storage device

 

I'm not committed to multiple partitions, I'm committed to multiple versions of Windows.  If I can't boot them from ISO, which seems to be the case currently with UEFI, I have to find some other way to boot them.  Multiple partitions was an idea because if the Windows ISO is extracted to a thumb drive it requires that it be in the root of the partition.  Can the filter driver be loaded during install?

 

UEFI Multiboot USB - Windows 8 and Linux Parted Magic
 

http://reboot.pro/to...e-2#entry165182

 

I used portable USB-harddisk with Standard MBR and FAT32 Boot + NTFS System partition

 

After booting with Windows 8 from USB you can run WinNTSetup2 and use mounted ISO

to prepare drive for Install of Windows 7 or 8 or XP

 

http://www.msfn.org/...-winntsetup-v2/

http://www.911cd.net...showtopic=24424

 

W8U-SetupW7.png

 

The reason that I want multiple installers on my USB drive is because I am often installing different OS's on my systems.  This is because I do hardware testing in the OS, doing compatibility validation.  I might have several different systems set up, and might change, or reinstall, the OS several times on a bad day.  Since our testing is done with Ubuntu, openSuSE, Red Hat, and three or more versions of Windows, in both 32bit and 64bit versions of all of the above, being able to install them all from one thumb drive I can carry with me is useful. 

 

I think my idea of using EFI shell scripts is the best way for now, until there is a way to "mount" an ISO from UEFI.  You can do this with GRUB2 via loopback, but loopback doesn't work for Windows and some flavors of Linux.

The reason that I want multiple installers on my USB drive is because I am often installing different OS's on my systems. 

 

That is exactly what you can do with the proposed strategy.

Boot from USB and use WinNTSetup to Install all Windows versions using mounted ISO files.

That is exactly what you can do with the proposed strategy.

Boot from USB and use WinNTSetup to Install all Windows versions using mounted ISO files.

 

I apologize for being unclear.  I never have more than one OS installed at a time unless I am testing virtualization, it doesn't help with the testing that I do and it can complicate things.

The proposed strategy Installs one version of Windows at a time on drive of local harddisk and virtualization is not used.