Jump to content











Photo
- - - - -

VHD_WIMBOOT - Apply and Capture of WIM Files for OS in VHD

ramdisk grub4dos svbus wimlib windows 10 ssd usb vhd wim wimboot

  • Please log in to reply
269 replies to this topic

#251 Camiel

Camiel

    Member

  • Members
  • 41 posts

Posted 10 August 2020 - 04:39 AM

Not sure if this is the right thread to post this but wimlib, or this procedure in general, creates a lot of those unmovable $Files like "$Secure:$SII"  "$Reparse:$R" "$Secure:$SDH" along with the $MFT ones, in the final wimboot vhd, and towards the end of the drive, that cannot get defraged by normal means, and so they keep a dynamic vhd in its full size at disk at all times, thereby delay grub4d load in to ram too.

 

If there's a solution/workaround I would like to know.



#252 alacran

alacran

    Gold Member

  • .script developer
  • 1484 posts
  •  
    Mexico

Posted 10 August 2020 - 05:22 AM

@ Camiel

 

Better use a fixed size VHD and latter once tested it boots fine from your USB, copy it to your HD, fully defragment it (the best possible) and  compress it with lz4_compressor for Ramboot, you may also use 7-zip to compress as gzip but lz4_compressor is faster, final size is very close and grub4dos 0.46a loads it faster to RAM.

 

It is good practice to always run CCleaner and fully defragment the VHD before capture it.

 

NOTE: Expandable VHDs if compressed do not boot when loaded to Ram by grub4dos.

 

alacran



#253 Camiel

Camiel

    Member

  • Members
  • 41 posts

Posted 11 August 2020 - 08:52 AM

Thank you alacran

The lz4 compressor is of course a wonderful tool, I always use it on fixed vhds, obviously.

But anyhow for dynamic disks I just found they can get "compacted" through diskpart.

Just leaving it here in case anyone needs it.



#254 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15464 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 11 August 2020 - 09:03 AM

But anyhow for dynamic disks I just found they can get "compacted" through diskpart.

Just leaving it here in case anyone needs it.

Well, you are leaving here "nothing" or only a "vague hint"..

 

HOW (exactly) do you "compact" dynamic vhd's (which are NOT dynamic disks) through diskpart?

 

Like this?

https://social.techn...h-diskpart.aspx

 

And now, maybe useful, maybe not, how to create a more compact NTFS filesystem:
http://reboot.pro/to...disk-emulation/

 

 

:duff:

Wonko



#255 Camiel

Camiel

    Member

  • Members
  • 41 posts

Posted 11 August 2020 - 01:10 PM

@Wonko the Sane

 

Yes. Like this.

I thought, you know, is just a simple looking it up, nothing complex. But you're right, feels hanging. Thanks.



#256 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15464 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 11 August 2020 - 01:44 PM

@Wonko the Sane

 

Yes. Like this.

I thought, you know, is just a simple looking it up, nothing complex. But you're right, feels hanging. Thanks.

Yep :), I had the impression you had a wonderful demonstration but that it wouldn't fit in the margin ;)

 

:duff:

Wonko



#257 wimb

wimb

    Platinum Member

  • Developer
  • 2966 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 3 days ago

Update VHD_WIMBOOT Version 3.0
 
- Update Manual
 
Download: VHD_WIMBOOT-30  Or from GitHub VHD_WIMBOOT-30  
 

Manual:  Attached File  VHD_WIMBOOT.pdf   1MB   0 downloads

 

VHD_WIMBOOT_2020-07-26_130511.jpg
 

 


  • alacran likes this

#258 alacran

alacran

    Gold Member

  • .script developer
  • 1484 posts
  •  
    Mexico

Posted 3 days ago

Thanks, good work my friend, but since the new default Compact mode for Win10 is now 8K, in accordance with following two quoted posts, I suggest to add 8K Compression option too. Or maybe better have all options: 4K, 8K, 16K and LZX, on a future version 3.1

 

From this post: http://reboot.pro/to...e-2#entry215932

 

I ran Compact /compactos:always  in a elevateted command prompt from a running 10x64 (standard install, not compact) on a VHD,  In my test got a compression of 1.8 to 1, I assume they were 4K compacted (standard Compact value on 10), see attached picture.

EDIT: Using new WOF_Compress-38, I was able to verify it was 8K compressed, then 8K seems to be the new stardard for Win10 Compact on this 10 19H1.

 

And your post : http://reboot.pro/to...e-6#entry215980

 

I can confirm that indeed compact /CompactOS:always as applied to online recent Win10x64 results in XPRESS8K Compression of OS files

 

 

alacran



#259 wimb

wimb

    Platinum Member

  • Developer
  • 2966 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 3 days ago

Thanks, good work my friend, but since the new default Compact mode for Win10 is now 8K, in accordance with following two quoted posts, I suggest to add 8K Compression option too. Or maybe better have all options: 4K, 8K, 16K and LZX, on a future version 3.1

 

 

Sure I will Add Compact:8K and Compact:16K mode for Capture and Apply in next version of VHD_WIMBOOT to make it even more complete.


  • alacran likes this

#260 wimb

wimb

    Platinum Member

  • Developer
  • 2966 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted A day ago

Update VHD_WIMBOOT Version 3.1

 

- Extra Mode Options:  APPLY      WIM File in Compact Mode with XPRESS8K and XPRESS16K Compression
- Extra Mode Options:  CAPTURE WIM File in WimBoot Mode with XPRESS8K and XPRESS16K Compression 
- Update Manual

- Update WimBootCompress.ini files - Added wofadk.sys to [PrepopulateList] section

 

Download: from GitHub VHD_WIMBOOT-31  

 

Manual:  Attached File  VHD_WIMBOOT.pdf   1MB   2 downloads

 

:cheers:


  • alacran likes this

#261 alacran

alacran

    Gold Member

  • .script developer
  • 1484 posts
  •  
    Mexico

Posted A day ago

Good work my friend, the program is more complete now.

 

JFYI:

 

Talking about WimBootCompress.ini, and only trying to unify criteria, I decided to take a closer look to yours, JFX's and mine:

 

Since last time I edited WimBootCompress.ini, I don't have this three lines on the [PrepopulateList] list, and all has being working fine so far:

 

wof.sys
wofadk.sys

\Windows\System32\Config\SYSTEM

 

I consider the two first are contained under: \Windows\System32\drivers\*.sys and already selected to be copied uncompressed. As this in fact is done.

 

About \Windows\System32\Config\SYSTEM  >>> I consider this line is contained under: \Windows\System32\config\* and already selected to be copied uncompressed. As this in fact is done.

 

Nevertheless there is no harm having those 3 lines, just they are redundant.

 

You can take a look to last version of  WinNTSetup\Tools\WimBootCompress.ini [ExclusionList] and none of them are listed there. (Remember new version now verify all in that list is real size files, and expands them if required, to avoid Dism issue).

 

Also I just noticed JFX added this two lines to the end of his list (we already have them):

 

\Windows\System32\pwdrvio.sys
\Windows\System32\pwdspio.sys

 

But his list do not have this two lines we have on our lists:

 

\Windows\System32\mcupdate_AuthenticAMD.dll  >>>  File is present from 7 to 10 OSs, thinking in Ramboot we better keep the line until further testing.
\Windows\System32\mcupdate_GenuineIntel.dll  >>>  File is present from 7 to 10 OSs, thinking in Ramboot we better keep the line until further testing.

 

Another difference is JFX list has (IMHO when dealing with VHDs no edition to our list required):

 

\BOOTNXT  >>>  We don't have this line.  >>>  It is proved it isn't required to boot from VHD, it may be required only for booting directly from real HD.
\Boot\*  >>>  We have \Boot\BCD  >>>  It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.
\EFI\*  >>>  We have \EFI\Microsoft\Boot\BCD  >>>  It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.

 

alacran



#262 gbrao

gbrao

    Frequent Member

  • Advanced user
  • 397 posts
  •  
    India

Posted 22 hours ago

\Windows\System32\pwdrvio.sys
\Windows\System32\pwdspio.sys

 

 That's from Minitool Partition Wizard.

 

That's one of the reasons I prefer Eassos Disk Genius.

 

MPW installs these :

"$SYSDIR\pwdrvio.sys"
"$SYSDIR\pwdspio.sys"
"$SYSDIR\pwNative.exe"

 



#263 alacran

alacran

    Gold Member

  • .script developer
  • 1484 posts
  •  
    Mexico

Posted 20 hours ago

 

 That's from Minitool Partition Wizard.

 

That's one of the reasons I prefer Eassos Disk Genius.

 

MPW installs these :

"$SYSDIR\pwdrvio.sys"
"$SYSDIR\pwdspio.sys"
"$SYSDIR\pwNative.exe"

 

Yes, but \Windows\System32\pwNative.exe do not create problems if it is compressed in case of Compact install or just a Pointer File in case of Wimboot Install, as I have already tested: http://reboot.pro/to...10/#entry210870

 

And also have reports from antonino61 the new version doesn't have this issue with any of them, but to play safe it is better to keep the other two mentioned on the list, just in case, as you never know what version the user has, and anyway this two files are very small: 18.7 KB and 12.2 KB respectively, (in the version I have).

 

alacran



#264 wimb

wimb

    Platinum Member

  • Developer
  • 2966 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 20 hours ago

Good work my friend, the program is more complete now.

 

Talking about WimBootCompress.ini, and only trying to unify criteria, I decided to take a closer look to yours, JFX's and mine:

 

Since last time I edited WimBootCompress.ini, I don't have this three lines on the [ExclusionList] list, and all has being working fine so far:

 

wof.sys
wofadk.sys

\Windows\System32\Config\SYSTEM

 

I consider the two first are contained under: \Windows\System32\drivers\*.sys and already selected to be copied uncompressed. As this in fact is done.

 

About \Windows\System32\Config\SYSTEM  >>> I consider this line is contained under: \Windows\System32\config\* and already selected to be copied uncompressed. As this in fact is done.

 

Nevertheless there is no harm having those 3 lines, just they are redundant.

 

 

The three lines in the [PrepopulateList] section are redundant as long as \Windows\System32\drivers\*.sys and \Windows\System32\config\* exist.

But these lines belong to the required minimum and it is good to have them as to survive on editing e.g. as done by antonino61

It is not the [ExclusionList] section we are talking about as you will know  ;)

 

I don't have extra info on the two lines given below, but indeed as you say it is better safe to keep these lines

\Windows\System32\mcupdate_AuthenticAMD.dll
\Windows\System32\mcupdate_GenuineIntel.dll

It is good to make a critical comparison between the different versions of WimBootCompress.ini 

For the moment I think the version of VHD_WIMBOOT is still OK  ;)

 

BTW It is pwNative.exe (program) and not pwNative.sys (driver)



#265 wimb

wimb

    Platinum Member

  • Developer
  • 2966 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 19 hours ago

JFX has \Windows\System32\mcupdate_*.dll which takes care of the two lines that we use.

 

JFX scans the driver files in folder drivers to keep installed drivers uncompressed.

But the Minitool Partition Wizard drivers do occur in System32 instead of System32\drivers

That is why it is needed to add the extra two lines

\Windows\System32\pwdrvio.sys
\Windows\System32\pwdspio.sys


#266 alacran

alacran

    Gold Member

  • .script developer
  • 1484 posts
  •  
    Mexico

Posted 19 hours ago

The three lines in the [PrepopulateList] are redundant as long as \Windows\System32\drivers\*.sys and \Windows\System32\config\* exist.

But these lines belong to the required minimum and it is good to have them as to survive on editing e.g. as done by antonino61

It is not the [ExclusionList] section we are talking about as you will know  ;)

 

I don't have extra info on the two lines given below, but better safe to keep these lines

\Windows\System32\mcupdate_AuthenticAMD.dll
\Windows\System32\mcupdate_GenuineIntel.dll

It is good to make a critical comparison between the different versions of WimBootCompress.ini 

For the moment I think the version of VHD_WIMBOOT is still OK  ;)

 

BTW It is pwNative.exe (program) and not pwNative.sys (driver)

OK, I understand now they are there for safety reasons, as I said there is no harm having those lines, and thanks for making me notice my two stupid mistakes.

Both are edited now: [ExclusionList] and pwNative.exe

 

But basically excluding some very minor differences the lists are almost the same. And I think this is a good result from the comparison with new JFX list.

 

alacran



#267 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15464 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 18 hours ago

But basically excluding some very minor differences the lists are almost the same. And I think this is a good result from the comparison with new JFX list.

Which is good :), two additional pairs of eyes are useful to spot possible issues, I might add that by doing these comparisons you created an useful reference or.(unconventional) documentation about the settings.

 

About the 

 

\Boot\*  >>>  We have \Boot\BCD  >>>  It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.
\EFI\*  >>>  We have \EFI\Microsoft\Boot\BCD  >>>  It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.

 

the "boot" folders contain a ton of local language related files, maybe that is the reason why the whole stuff (as opposed to only the BCD file) is kept uncompressed.

 

:duff:

Wonko



#268 alacran

alacran

    Gold Member

  • .script developer
  • 1484 posts
  •  
    Mexico

Posted 18 hours ago

 

JFX has \Windows\System32\mcupdate_*.dll which takes care of the two lines that we use.

 

Good eyes, I didn't see it, then only remaining differences are in:

 

 

\BOOTNXT  >>>  We don't have this line.  >>>  It is proved it isn't required to boot from VHD, it may be required only for booting directly from real HD.
\Boot\*  >>>  We have \Boot\BCD  >>>  It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.
\EFI\*  >>>  We have \EFI\Microsoft\Boot\BCD  >>>  It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.

 

Wich makes again WinNTSetUp fully capable to Wimboot and Compact installs from 7 to 10 the way we both want/prefer.

 

Now only advantages of wimlib-imagex are: it is capable to make a Compact or Wimboot install on a smaller space (VHD or HD) as it compresses the files on Ram before copy them, and this way also helps to avoid fragmentation.

Not as WinNTSetup and Dism do, first copying the file and then compressing it, which also creates a lot of free space fragmentation.

 

alacran


  • wimb likes this

#269 wimb

wimb

    Platinum Member

  • Developer
  • 2966 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 18 hours ago

Now only advantages of wimlib-imagex are: it is capable to make a Compact or Wimboot install on a smaller space (VHD or HD) as it compresses the files on Ram before copy them, and this way also helps to avoid fragmentation.

Not as WinNTSetup and Dism do, first copying the file and then compressing it, which also creates a lot of free space fragmentation.

 

 

 

The smaller Used Space in VHD and avoiding fragmentation are very important properties of wimlib

which is best guaranteed when using VHD_WIMBOOT for Capture and Apply of WIM files.

 

The Manual has been adjusted in favor of using for FILEDISK Compact LZX mode and for RAMDISK WimBoot mode 



#270 alacran

alacran

    Gold Member

  • .script developer
  • 1484 posts
  •  
    Mexico

Posted 17 hours ago

Which is good :), two additional pairs of eyes are useful to spot possible issues, I might add that by doing these comparisons you created an useful reference or.(unconventional) documentation about the settings.

About the

\Boot\* >>> We have \Boot\BCD >>> It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.
\EFI\* >>> We have \EFI\Microsoft\Boot\BCD >>> It is proved our line is minimum required to boot from VHD, it may be required only for booting directly from real HD.

the "boot" folders contain a ton of local language related files, maybe that is the reason why the whole stuff (as opposed to only the BCD file) is kept uncompressed.

:duff:
Wonko

The difference on those two lines between WinNTSetup and VHD_WIMBOOT is:

If there is a Wimboot or Compact install directly on the root of a partition/drive of a HD/USB device, all the files into Boot and EFI folder are copied by WinNTSetup uncompressed on the Boot partition, to let the machine boot. Which is fine in this case.

If the install is made into a VHD, WinNTSetup does same as in previous paragraph, and also copies all same files uncompressed into the VHD. Which is a waste of space since from all those files only both BCD uncompressed are required into a VHD to boot it.

But VHD_WIMBOOT (if selecting a Wimboot or Compact mode) from all the files contained into Boot and EFI folders, only uncompresses both BCD files into the VHD, consuming less space into the VHD, and all the booting files, are made uncompressed on the Boot partition/drive when formating using USB_FORMAT (if used) and latter edited (if they exists), or created (if they don't exists), by VHD_WIMBOOT.

 

The smaller Used Space in VHD and avoiding fragmentation are very important properties of wimlib
which is best guaranteed when using VHD_WIMBOOT for Capture and Apply of WIM files.

The Manual has been adjusted in favor of using for FILEDISK Compact LZX mode and for RAMDISK WimBoot mode


I'm totally agree with you.

alacran







Also tagged with one or more of these keywords: ramdisk, grub4dos, svbus, wimlib, windows 10, ssd, usb, vhd, wim, wimboot

2 user(s) are reading this topic

0 members, 2 guests, 0 anonymous users