Jump to content











Photo
- - - - -

Some doubts about SDI, WIM, and RAMdisk

wim sdi ramdisk winpe

  • Please log in to reply
8 replies to this topic

#1 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 28 August 2014 - 11:44 AM

I have a bunch of questions about SDIs, WIMs, and RAMdisks :
  • A RAMdisk is basically a disk image loaded into RAM. Right?
  • There exist some options about ramdisks in BCD for Bootmgr, mainly for PE.
    WHY do they boot PE with ramdisk?
  • Apparently boot.wim is a "bootable" WIM. In what way is it bootable?
  • Why can't install.wim also be booted from? Why cannot it be made a "bootable" WIM? Discount the W8.1u1 WIMBoot possibility... it still cannot be done from a DVD or network this way.
  • Why is Windows PE booting from a bootable WIM? TechNet says it can be booted without a RAM disk as well. Why is this not done normally?
  • What's the point of an SDI? If the WIM itself is bootable, then why need anything else?
Sorry for the challenging tone, but I felt that's the best way to get pointed answers, for my benefit as well as future readers'. I have also referred http://technet.micro...5(v=WS.10).aspx , but it only makes me beg the question... WHY all this??

Edited by milindsmart, 28 August 2014 - 11:45 AM.


#2 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 August 2014 - 12:56 PM

Well, you have many questions, some of which are fine (and you have possibly already a valid answer for them :thumbup: ) while some seemingly need to be de-composed and re-composed. :w00t:

There is a bit of confusion between RAMdisk (like Imdisk that loads actually RAM volumes) and RAMdisk (like Winvblock, Firadisk or Gavotte's RAMdisks)  that mount actually "disks" (whole disks).

But, yes, a RAMdisk is an image of a mass storage device (whole or partial) loaded into RAM.

 

What actually "boots" is not the "boot.wim" :w00t: but the "boot.sdi", to which the wim contents are "mounted".

Sure you can have a "flat" PE actually this was used a lot of time ago for early VistaPE's, example:

http://reboot.pro/to...-on-usb-device/

 

The .wim format is NOT a volume image, it is a compression format (used for the contents of  volume), much more similar to (say) .zip or .7z, it misses some (needed) parts for the mounting as a volume (typically the PBR and filesystem structure).

Most probably the WOF driver in Windows 8/8.1u1 is nothing but a "normal" "filedisk" (not necessarily a RAMdisk) that recreates on-the-fly the missing structures and exposes at the end a "normal" volume. :unsure:

 

About the "traditional" use of .wim and of the wimmount drivers see (only seemingly unrelated):

http://nativeex.boot...x/WimCaptEx.htm

http://reboot.pro/to...wimtoolbatches/

http://reboot.pro/to...ndling-wo-waik/

 

The .sdi format on the other hand is a "plain" volume image with a prepended "header", see, JFYI:

http://www.msfn.org/...ize-of-bootsdi/

 

 

The use of .sdi dates back to Server 2003 SP1, and Windows Embedded, though at the time it was used as a plain "volume image" and not as a "mounting container" for external files (still JFYI and to give you some info on the "history"):

http://www.911cd.net...showtopic=10482

http://www.911cd.net...topic=19333&hl=

 

And there has always been the possibility, using the above Server 2003 SP1 approach, to use compressed images, example:

http://www.911cd.net...showtopic=19737

 

The idea of having a "small core" and one or more "mounted" external images is also not really new, Sanbarrow at the time did an excellent work with MOA/LODR-packs:

http://reboot.pro/to...uff-lodr-packs/

 

 

:duff:

Wonko



#3 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 29 August 2014 - 01:49 PM

What actually "boots" is not the "boot.wim" :w00t: but the "boot.sdi", to which the wim contents are "mounted". [...]
The .wim format is NOT a volume image, it is a compression format (used for the contents of  volume), much more similar to (say) .zip or .7z, it misses some (needed) parts for the mounting as a volume (typically the PBR and filesystem structure).


Which ties in beautifully to what Sha0 said in Bootmgr hacking thread :

My understanding is that in Microsoft Windows NT descendants, there are two sorts of modes for using a RAM-disk:

  • The disk image for the virtual disk contains a regular filesystem
  • The disk image is a "dummy" and a special filesystem driver uses an archive to pretend that filesystem is on that virtual disk
With the former, the disk image is just an image of a regular disk. It can have any regular disk filesystem on it. With the latter, the disk image is just a dummy to pretend "there is a disk" and the files are provided by a special filesystem driver, such as the one that knows how to mount .WIM files.

[...]
[wimboot(from ipxe) is] actually doing a similar strategy to .WIMs themselves, only back one step in the boot-process. You hand it a set of files and it emulates an INT 0x13 disk with a FAT filesystem on it containing those files. The magic is that it doesn't use any disk image; sector-read requests have dynamically-generated content without having built an entire disk image in memory.


So here, boot.sdi is this "dummy" disk image, that contains only what really MUST be on an ACTUAL disk image, with everything else on the boot.wim ....
 

Most probably the WOF driver in Windows 8/8.1u1 is nothing but a "normal" "filedisk" (not necessarily a RAMdisk) that recreates on-the-fly the missing structures and exposes at the end a "normal" volume. :unsure:


As he says, the SDI strategy is subtly different from the concept of mounting the WIM on the NTFS filesystem contained in the boot.sdi : The read requests are redirected from a much higher level to the WIM files, just like by wof.sys. wimboot OTOH lets the request go all the way down to the sector read, which then is hooked and spoofed. Another difference is that the SDI strategy operates in 32-bit mode, while wimboot operates at a 16-bit level.

This is all so beautiful...

Edited by milindsmart, 29 August 2014 - 01:55 PM.


#4 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 29 August 2014 - 02:47 PM

So here, boot.sdi is this "dummy" disk image, that contains only what really MUST be on an ACTUAL disk image, with everything else on the boot.wim ....
 

 

 

WHERE is "here"?

 

I wouldn't be so sure about your description.

 

Try mounting an existing boot.sdi image with imdisk, providing an offset of 8192 bytes.

Or try removing the first 8192 bytes and open the rest in 7-zip.

The boot.sdi is completely empty.

 

Then, try making a boot.sdi formatted as FAT. :dubbio:

 

Just in case (additional info):

http://technet.micro...5(v=ws.10).aspx

 

:duff:

Wonko



#5 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 29 August 2014 - 03:37 PM

here - > in context of Sha0's explanation of "a dummy to pretend there is a disk and the files are provided by a special filesystem driver".

 

Checking...



#6 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 29 August 2014 - 03:57 PM

here - > in context of Sha0's explanation of "a dummy to pretend there is a disk and the files are provided by a special filesystem driver".

 

Checking...

Yep :)

which still means NOT that:

 

 

... boot.sdi is this "dummy" disk image, that contains only what really MUST be on an ACTUAL disk image, with everything else on the boot.wim ....
 

 

as EVERYTHING (not "everything else") is in the boot.wim and NOTHING (not "only what really MUST be on an ACTUAL disk image") is in the boot.sdi (which is completely EMPTY).

This is the difference between what Sha0 wrote and the description you proposed, which is OK :) as long as you specify how the "only what really MUST be on an ACTUAL disk image" is NOT any "content" but the actual filesystem of the volume (a boot.sdi contains a volume image, not a disk one, to be picky, i.e. the first sector after the 8192 bytes header is a bootsector or PBR).

 

:duff:

Wonko



#7 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 29 August 2014 - 09:01 PM

You're right, it's completely empty, it just contains a blank filesystem with the $ files...

Wonko the Sane, on 29 Aug 2014 - 9:27 PM, said:
(a boot.sdi contains a volume image, not a disk one, to be picky, i.e. the first sector after the 8192 bytes header is a bootsector or PBR).

Agreed... It's a volume image.

So what does this 8192 byte SDI header contain?

#8 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 30 August 2014 - 08:29 PM

A bigger question.... The boot chain for SDI RAM disk based PE is :
El Torito -> bootmgr + BCD -> (boot.sdi) -> (boot.wim) -> windows/system32/winload.(exe|efi) ...

Why such an excessive chain? After all the WIM gets mounted in the volume image inside the SDI... Why not just have sent a VHD with the entire contents and be done with it, just like VHD native boot...

El Torito -> bootmgr + BCD -> (boot.vhd) -> windows/system32/winload.(exe|efi) ...

You might say "The entire contents of Boot.wim will be somewhat big to load in RAM"... In which case, we have to recall that the current SDI/WIM arrangement is ALSO residing on disk anyway... So why not instead keep the contents of boot.wim in flat format in UDF/ISO format?

El Torito -> bootmgr + BCD -> windows/system32/winload.(exe|efi) ...

You might say that it would be slow since it's accessing from disc for everything.. Which is the case with the SDI/WIM arrangement too!

#9 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 31 August 2014 - 11:25 AM

So what does this 8192 byte SDI header contain?

You will find possibly some details on it looking for topics revolving around Windows XP embedded (that is what the format was originally used):

http://reboot.pro/to...yslinux-sdic32/

http://msdn.microsof...y/ms838543.aspx

 

About how they chose to use it in Vista :ph34r: PE 2.x booting, I believe it is just the usual MS approach at re-using existing code.

 

On the other hand the use of .wim makes a lot of sense as it is a highly compressed format (i.e. it is the key factor that allows the Vista :ph34r:, 7 , etc. BLOAT to fit into a DVD), and as said the amount of data that is actually trasferred to RAM is very little.

Remember that at the time there was not (yet) "native" .vhd booting and even support for .vhd's outside the MS (please read as Conectix) Virtual PC was scarce and experimental, we are talking 2006 or so, still JFY:
http://reboot.pro/to...storage-driver/

 

Just for your interest, if you look hard enough you may be able to find a XP based PE made by a good Chinese guy that used two or three years ago - if I recall correctly - a version of native.exe to mount (in the "native" or BootExecute" envirinment) a .wim image (making use of the wim related drivers like the mentioned http://reboot.pro/to...wimtoolbatches/ ).

The build used the Server 2003 SP1 ramdisk to (quickly) load a very small image with just the "native" environment and in it a sort of autoexec.bat automatically ran a mount command in native.exe that mounted to it the "rest of the files".

The native.exe used is a modified version of the code by amdf:

http://hex.pp.ua/nt-...tions-shell.php

http://hex.pp.ua/nt-...s-shell-eng.php

http://hex.pp.ua/nat...difications.php

http://translate.goo...SyGdPkYWUVCz-uw

http://code.google.com/p/nativeshell/

 

:duff:

Wonko







Also tagged with one or more of these keywords: wim, sdi, ramdisk, winpe

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users