Jump to content











Photo
- - - - -

VHD_WIMBOOT - Apply and Capture of WIM Files for OS in VHD

grub4dos ramdisk svbus wimlib windows 10 ssd usb wim wimboot vhd

  • Please log in to reply
87 replies to this topic

#76 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14707 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted A week ago

How does the VHD file know to what WIM file it is connected and what occurs when using Upd WimBoot button ?

 

The relation between VHD and WIM is not in the registry and not in the BCD entry, but it is inside the VHD in file \System Volume Information\WimOverlay.dat

 

You can open the VHD in 7-zip and then Copy the file WimOverlay.dat to a place where you can watch the binary content of this file by using TinyHexer.

 

It shows the WIM File path and the MountedDevices registry entry with the Disk Signature of the Drive where the WIM file is located. 

 

attachicon.gifWimOverlay-7z-2019-04-09_152644.png

Very interesting. :)

 

Can you post a sample WimOverlay.dat?

 

It should be trivial to make a Structure Viewer for Tiny Hexer or a direct parser for that file.

 

:duff:

Wonko


  • wimb likes this

#77 wimb

wimb

    Gold Member

  • Developer
  • 2482 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted A week ago

Very interesting. :)

 

Can you post a sample WimOverlay.dat?

 

It should be trivial to make a Structure Viewer for Tiny Hexer or a direct parser for that file.

 

 

Sure, it is certainly very interesting. :)

 

I was looking for how actually VHD WIMBOOT works and then I found this file WimOverlay.dat

 

Attached File  WimOverlay.zip   21.08KB   8 downloads == WimOverlay-TinyHexer-2019-04-10_153600.png



#78 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14707 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted A week ago

The good news are that seemingly only you and me were unaware of this mechanism :blush: :w00t:.

The bad news are that Wimlib (that actually does support it BOTH through Windows calls AND through driect editing) has this to say on the matter:

https://wimlib.net/man1/wimapply.html

 

 

In addition, this option works best when running on Windows 8.1 Update 1 or later, since that is the first version of Windows that contains the Windows Overlay Filesystem filter driver ("WOF"). If the WOF driver is detected, wimlib will create the WIMBoot "pointer files" using documented ioctls provided by WOF.

Otherwise, if the WOF driver is not detected, wimlib will create the reparse points and edit the file "\System Volume Information\WimOverlay.dat" on the target volume manually. This is potentially subject to problems, since although the code works in certain tested cases, neither of these data formats is actually documented by Microsoft. Before overwriting this file, wimlib will save the previous version in "\System Volume Information\WimOverlay.wimlib_backup", which you potentially could restore if you needed to.

 

The format is not documented. :(

 

Extracting the filename is still possible, of course but I was fantasizing about the possibility to use a "same" .vhd with different .wim files ...

 

The Wimlib Author did attempt to document the format, however:

https://github.com/C...de/wimlib/wof.h

 

I'll see what I can do with this latter info. :unsure:

 

:duff:

Wonko



#79 wimb

wimb

    Gold Member

  • Developer
  • 2482 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted A week ago

 

Extracting the filename is still possible, of course but I was fantasizing about the possibility to use a "same" .vhd with different .wim files ...

 

The Wimlib Author did attempt to document the format, however:

https://github.com/C...de/wimlib/wof.h

 

 

Thanks for the Info  :)

 

WimOverlay.dat uniquely identifies the WIM File.

 

You can have different VHD's combined with the "same" WIM file, but not the other way around .....



#80 alacran

alacran

    Silver Member

  • .script developer
  • 875 posts
  •  
    Mexico

Posted A week ago

Thanks for the Info  :)

 

WimOverlay.dat uniquely identifies the WIM File.

 

You can have different VHD's combined with the "same" WIM file, but not the other way around .....

 

Yes, in fact I have tested it. 

 

But also you can have multiple indexes (wimboot capables) on same wim file (as usually in any other wim file). And one or more VHDs linked to each index, See: http://reboot.pro/to...e-5#entry210406

 

EDIT: But if I attach the VHDs and run in Power Shell: Get-WIMBootEntry -Path "J:\"  Only got the wim file path, not any mention to the respective index.
 

alacran


  • antonino61 likes this

#81 wimb

wimb

    Gold Member

  • Developer
  • 2482 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted A week ago

 

EDIT: But if I attach the VHDs and run in Power Shell: Get-WIMBootEntry -Path "J:\"  Only got the wim file path, not any mention to the respective index.
 

 

And what do you get in admin command window with

fsutil wim enumwims J:


#82 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14707 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted A week ago

I was thinking the other way round. :unsure:

 

Maybe we could edit the  \System Volume Information\WimOverlay.dat on-the-fly or possibly have several pre-made ones by the "normal" procedure, rnamed to *something else*:

\System Volume Information\WimOverlay.dbt

\System Volume Information\WimOverlay.dct

\System Volume Information\WimOverlay.ddt :w00t: :ph34r:

...

\System Volume Information\WimOverlay.dyt

and rename the "current" \System Volume Information\WimOverlay.dat to \System Volume Information\WimOverlay.dzt and any of the other ones back to \System Volume Information\WimOverlay.dat.

 

This would probably be possible through a batch grub4dos script running at boot time.

 

:duff:

Wonko



#83 alacran

alacran

    Silver Member

  • .script developer
  • 875 posts
  •  
    Mexico

Posted A week ago

 

And what do you get in admin command window with

fsutil wim enumwims J:

 

Then I got: I:\Wimboot\10-WB.wim:2   the full path and the image index, 2 in this case, attached images of Power Shell and Command prompt.

 

EDIT: The following command is valid for both VHDs 10x86 and 10x64, created respectively from 10-WB.wim image 1 and image 2, if I want to relocate the coupled Wim-VHDs on primary partition of external device:

 

Run as admin directly on each attached VHD drive:

 

DISM.exe /Update-WIMBootEntry /Path:J:\ /DataSourceID:0 /ImageFile:K:\Wimboot\10-WB.wim

 

Where:

 

J:\    Is the attached VHD located on internal HD, before copy it to USB, latter if you want it can be copied to K:\ root,

 

I made this before copy the VHD because if I want to only Ramboot and then make a LZ4 compressed VHD (only can be loaded in Ram, and only work if made from a fixed size VHD) it is faster to do it on the HD, and latter copy to USB only the 10x64-WB.vhd.lz4 (130 MB) and 10x86-WB.vhd.lz4 (99.2 MB) as required (and not the full 1.5 GB for each fixed size VHD).

Or if for any reason I want to be able to also boot from bootmanager and not only Ramboot the OSs (very improbable for me since Ramboot is IMHO the best and fastest option) then I need to make and/or copy 480 MB for the expandable 10x64.vhd and 360 MB for the expandable 10x86.vhd.

 

K:\Wimboot\10-WB.wim    is the wim file already copied to single primary (NTFS) partition of my external USB device.

 

All BCDs and grub4dos menus were made manually.

 

So just applied this command to 10x86.vhd and also the same to 10x64.vhd to relocate the files.

 

By the way I have a new addition to moded WimBootCompress-2019-03-31.ini (attached).

 

[ExclusionList]
\MSOCache\*  >>> This is the hidden cache folder after installing Office (saving to compress in your wim file from about 300 MB to almost 1 GB, depending on version), sometimes it may be created on other partition if available.

 

 

alacran

Attached Thumbnails

  • Wim location PS.png
  • Wim location Command Promt.png

Attached Files



#84 wimb

wimb

    Gold Member

  • Developer
  • 2482 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted A week ago

 

DISM.exe /Update-WIMBootEntry /Path:J:\ /DataSourceID:0 /ImageFile:K:\Wimboot\10-WB.wim

 

 

VHD_WIMBOOT will attach the selected VHD file and then exactly use the same code to Update WimBoot the VHD for the selected WIM file.

 

Thanks for the new WimBootCompress.ini file that also excludes MSOCache folder.



#85 antonino61

antonino61

    Frequent Member

  • Advanced user
  • 294 posts
  •  
    Italy

Posted 6 days ago

Dear Wimb,

I had a hard time trying to make version 2.0 work as well as version 1.8; started thinking there was something wrong with my vhd+wim baking or my actual files, as 2.0 would not complete bcd tasks and gave a lot of registry copying errors. I resorted to 1.8 back again, and everything went back to normal. no issues at all. do not know if I am the only one to have noticed that, but I can assure u it was a bit of a headache. 

nino



#86 wimb

wimb

    Gold Member

  • Developer
  • 2482 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 6 days ago

 

I had a hard time trying to make version 2.0 work as well as version 1.8; started thinking there was something wrong with my vhd+wim baking or my actual files, as 2.0 would not complete bcd tasks and gave a lot of registry copying errors. I resorted to 1.8 back again, and everything went back to normal. no issues at all. do not know if I am the only one to have noticed that, but I can assure u it was a bit of a headache. 

 

 

The core of the program is the same for all versions from 1.5 until 2.0 and there are no registry operations involved for the VHD + WIM



#87 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14707 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 13 hours ago

The Wimlib Author did attempt to document the format, however:

https://github.com/C...de/wimlib/wof.h

 

I'll see what I can do with this latter info. :unsure:

 

The attached is a tentative-only and more-than-half[1]-@§§ed attempt to a Structure VIewer for TinyHexer

 

There is something "wrong" (or I am completely failing to understand it in the wimlib notes) in the format, I tried to do my best to adapt the Structure Viewer to the sample provided, but it needs some serious checking against a number of different  Wimoverlay.dat files, particularly to check if both the "MBR" and the "GPT" data are parsed correctly.

 

I found a 40 bytes section that is seemingly missing from the wimlib document and - maybe - I found how to determine the offset and size of the path. :unsure:

 

 

 

:duff:

Wonko

 

[1] I would say 3/4 to 4/5, it is 10 years or so I don't touch Tiny Hexer Structure Viewer langauage (and I have never been actually familiar with it, and it has a few quirks for the beginners)

Attached Files



#88 wimb

wimb

    Gold Member

  • Developer
  • 2482 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 9 minutes ago

Thanks for making the WimOverlay.mps script for TinyHexer.

 

The mpth_small.exe version of TinyHexer that I normally use, does not have the Structure Viewer.

 

Download mpth_18.exe from https://www.softpedi...iny-hexer.shtml has Structure Viewer

 

More Info on using Tiny Hexer Scripts was given by you in http://reboot.pro/to...-hexer-scripts/

 

After mounting VHD as drive P: then use fsutil in admin command window

fsutil wim enumwims P:

The 40 bytes section in WimOverlay.dat being undocumented contains at offset 0x30 a 16 bytes guid that identifies the WIM file.

This guid is also shown in the fsutil output

 

TinyHexer-WimOverlay-2019-04-19_075410.png







Also tagged with one or more of these keywords: grub4dos, ramdisk, svbus, wimlib, windows 10, ssd, usb, wim, wimboot, vhd

2 user(s) are reading this topic

1 members, 1 guests, 0 anonymous users


    wimb