Jump to content











Photo
- - - - -

Compact mode installs

compact mode

  • Please log in to reply
80 replies to this topic

#1 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 25 July 2020 - 03:17 PM

Info on this thread applies to VHDs and also real drives, mainly thinking on save space and portability, of course if you want to make a Compact install for every day use I recommend use Compact 8K (new standard Compact mode on Win10), and in this case after final install you can re-enabe a small pagefile if needed. It may depend of your RAM and installed programs as some of them (as Acrobat programs) require a pagefile.

 

I decided to start all this test using 10x64 Edu as many teachers and studends may have the possibility to activate it on the KMS service of the high school or college, then once legaly activating the first install, it is totally portable. But same applies to Enterprise if the user is working in a company having KMS service, and he is allow to do it

 

An option to legaly activate the Home or Pro versions could be using your MS account, but I haven't tested this option.

 

First I will start quoting some posts made on other threads, just to keep together all the previous comments about this subject.

 

Summary of my procedure.

 

Spoiler

 

Why wimlib is a better option than WinNTSetup when dealing with Compact installs.

 

Spoiler

 

Some info usefull to select your Compact mode compression.

 

Spoiler

 

WinSxS_reduce commands by cdob, I only use base_winsxs.cmd because winsxs_hardlink_system.cmd has created some issues in my OSs and the gain using it is very low.

 

Spoiler

 

Get an idea of how much space you can get back using base_winsxs.cmd to help reduce the size required for your install.

 

Spoiler

 

Wimb plans to add Compact install option to his fantastic tool VHD_WIMBOOT

 

Spoiler

 

Useful info for WinPEs, Wimboot and Compact installs: http://reboot.pro/to...mpact-installs/

 

NOTE: For 7, 8.0 and 8.1 see: http://reboot.pro/to...e-2#entry215570

 

well for a first post it is long enought. I will add future posts latter.

 

alacran


Edited by alacran, 20 September 2020 - 07:11 AM.

  • sebus and wimb like this

#2 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 25 July 2020 - 04:09 PM

Attached two pictures to ilustrate the right way to verify the real used space on your mass storage device when looking to a mounted VHD from a 10 OS.

 

Picture No. 1 shows the info when selecting properties of all files and folders contained into the VHD (or a drive)

 

Picture No. 2 shows the info when selecting properties of the mounted VHD drive.

 

Both from same 6 GB fixed size VHD Compact mode 4K (WinSxS_reduce commands by cdob, were not used in this case).

 

As you can see the info is very different:

 

On picture No.1 there are very hight numbers, because the size info related to the files includes the size if all files were uncompressed + all the size that all soft and hard links represents, event if they are only pointers. Basically this info of used size from a 6 GB VHD is not the right info we need, then we have the size on disk, this is the size used by uncompresed + compressed files + all the size that all soft and hard links represents.

 

On picture No. 2, numbres are more close to reality, because the used size info related to the drive is the size used by uncompresed + compressed files and do not include all the size that all soft and hard links represents. It is basically the size of used clusters.

 

Then as you allready noticed the right way to check used space on a drive or virtual drive is as on picture No. 2

 

Also this is the cause why we get very high numbers when we try to measure the used size of WinSxS folder using Explorer, the info we get includes the size if all files were uncompressed (if any file is compressed) + all the size that all soft and hard links represents, event if they are only pointers, also we have the size on disk, this is the size used by uncompresed + compressed files (if any) + all the size that all soft and hard links represents.

 

New versions of Dism (since 8.1) have a command to get real numbers, and let you decide if a cleanning procedure  of obsolete files on WinSxS is required, I don't remember it right now but I will talk about it latter.

 

Edit: Mentioned command is on Post No. 4.

 

alacran

Attached Files

  • Attached File  1.png   329.38KB   1 downloads
  • Attached File  2.png   298KB   1 downloads


#3 wimb

wimb

    Platinum Member

  • Developer
  • 3756 posts
  • Interests:Boot and Install from USB
  •  
    Netherlands

Posted 25 July 2020 - 04:31 PM

Nice Topic. :)

 

Good that you collect here the info on Compact mode Installs and that you start the discussion on the measurement of Used Size of the disk drive.


  • alacran likes this

#4 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 25 July 2020 - 04:58 PM

A new option has been added to the DISM tool for Windows 8.1 to help determine how much disk space the WinSxS folder really uses.

 

 

A new option has been added to the DISM tool for Windows 8.1 to help determine how much disk space the WinSxS folder really uses.

Analyze the size of the component store (WinSxS folder)

  • Open an elevated command window and type:

    Dism.exe /Online /Cleanup-Image /AnalyzeComponentStore

 

Note

The /AnalyzeComponentStore option isn’t recognized on Windows 8 and earlier.

 

The information returned is:

 
Windows Explorer Reported Size of Component Store This value the size of the WinSxS folder if computed by Windows Explorer. This value doesn’t factor in the use of hard links within the WinSxS folder.
 
Actual Size of Component Store This value factors in hard links within the WinSxS folder. It doesn’t exclude files that are shared with Windows by using hard links.
 
Shared with Windows This value provides the size of files that are hard linked so that they appear both in the component store and in other locations (for the normal operation of Windows). This is included in the actual size, but shouldn’t be considered part of the component store overhead.
 
Backups and Disabled Features This is the size of the components that are being kept to respond to failures in newer components or to provide the option of enabling more functionality. It also includes the size of component store metadata and side-by-side components. This is included in the actual size and is part of the component store overhead.
 
Cache and Temporary Data This is the size of files that are used internally by the component store to make component servicing operations faster. This is included in the actual size and is part of the component store overhead.
 
Date of Last Cleanup This is the date of the most recently completed component store cleanup.
 
Number of Reclaimable Packages This is the number of superseded packages on the system that component cleanup can remove.
 
Component Store Cleanup Recommended This is a component store cleanup recommendation. Cleanup is recommended when performing a cleanup process may reduce the size of the component store overhead.

Based on this analysis you can determine the overhead of the WinSxS folder by taking the sum of the backups and disabled features size with the cache and temporary data size.

Source: https://docs.microso...e-winsxs-folder

 

NOTE: Bolds and data layout are mine.

 

alacran



#5 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 July 2020 - 05:02 PM

Hmmm, once it was easier, the pie doesn't lie:

https://docs.microso...did-my-space-go

what's that now, the doughnut doesn't distort? :w00t:

 

How does DU behave in such a setup?

https://docs.microso...ls/downloads/du

 

 

:duff:

Wonko



#6 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 25 July 2020 - 09:51 PM

Following is relevant info taken from: https://docs.microso...ktop/compact-os

NOTE:In some cases I have added some additional commets to clarify the info.

 

 

Acording to MS info when using the Compact mode on 10 OS, (remember all commands on this section apply to Compact 4K, the standard option) :

 

You can query whether the operating system is running Compact OS, and change it at any time, using the Compact.exe command.

 

EDIT: New info from my post No. 46: http://reboot.pro/to...e-2#entry215932

 

I ran Compact /compactos:always  in a elevateted command prompt from a running 10x64 (standard install, not compact) on a VHD,  In my test got a compression of 1.8 to 1, I assume they were 4K compacted (standard Compact value on 10), see attached picture.

EDIT: Using new WOF_Compress-38, I was able to verify it was 8K compressed, then 8K seems to be the new stardard for Win10 Compact on this 10 19H1.

 

From Windows PE, determine if the OS is compacted:

Compact.exe /CompactOS:Query /WinDir:E:\Windows

Where E:\Windows is the folder where Windows has been installed.

 

 

From an online installation, change from non-compacted to compacted OS:

Compact.exe /CompactOS:always

 

 

Size comparisons

The table below shows the additional space saved by using compact OS, Single instancing, and reducing or turning Off Hiberfile on 2GB (x86 processor architecture) and 4GB (x64 processor architecture), on Windows 10, version 1607:

 

The table do not keep the correct aligments when pasted here and it was better to make an image and attach it.

 

This is another usefull info from: https://winaero.com/...thm-windows-10/

 

Starting with Windows 10, you can use LZX compression. This is the strongest algorithm available for NTFS. Its compression ratio is about 50% for certain files, which is much more than the default NTFS compression.

Before you proceed, you need to know that files compressed with LZX in Windows 10 cannot be opened in Windows 7, Windows 8.1 or in any previous Windows version.

 

To compress files on NTFS using the LZX algorithm in Windows 10, do the following.

    Open an elevated command prompt.
    Type or paste the following command:

    compact /c /s /a /i /exe:lzx "C:\data\*"

Substitute the C:\data portion with the actual folder or drive path you want to compress.

This will apply LZX compression optimized for executable files which are read frequently and not modified.

Others algorithms are:

    XPRESS4K (fastest) (default)
    XPRESS8K
    XPRESS16K

It will compress other files as well.

The switches are as follows:

/c - Compresses the specified files. Directories will be marked so that files added afterward will be compressed unless /EXE is specified.

/s - Performs the specified operation on files in the given directory and all subdirectories. Default "dir" is the current directory.

/a - Displays files with hidden or system attributes. These files are omitted by default.

/i - Continues performing the specified operation even after errors have occurred. By default, COMPACT stops when an error has encountered.

 

 

To unpack the compressed files (restore defaults), use the following command:

compact /u /a /s /exe "C:\data\*"

 

This info is good, but we need to also remember wimb made a GUI program to make this task: WOF_Compress

 

 

Anyway as said on first post under Summary of my procedure:

 

If you want a Compact mode LZX, doing your final VHD and latter apply Compact LZX to it is foolish, if you can install it directly as Compact mode LZX, using wimlib.

 

Also under Why wimlib is a better option than WinNTSetup when dealing with Compact installs:

 

WinNTSetup first installs all, and latter Compacts it (standard Dism procedure), the advantage of using wimlib for Compact mode installs is it first compacts the files on RAM and then write them to the VHD, this lets you use an smaller size for the target VHD, that WinNTSetup can't use, I have had this trouble with WinNTSetup during my tests.

 

 

alacran

Attached Files


Edited by alacran, 15 September 2020 - 11:59 PM.


#7 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 25 July 2020 - 10:04 PM

how do I use du to check hardlinked stuff?



#8 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 25 July 2020 - 10:26 PM

how do I use du to check hardlinked stuff?

 

I have no idea how to use it, since I never have used it before. But on its page you can find command and arguments.

 

But it seems to me it is only for getting files/folders sizes.

 

alacran



#9 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 25 July 2020 - 11:16 PM

ok, here is what I found from wimlib for wimb:

 

 
wimoptimize d:\wimboot\install.wim --compress=LZX:999 - this will strip a few kbs from the existing wim, but test results are significantly better, and initial used space in vhd, well, let me check

 

 

wimoptimize d:\wimboot\install.wim --solid unfortunately exceeds wof.sys requirements, so it will not make pointers on the applying stage; it will only transfer full files to the vhd.

 

ps: initial used space in vhd.vhd stays the same, whether u run wimoptimize d:\wimboot\install.wim --compress=LZX:999 or not.



#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 July 2020 - 08:21 AM

how do I use du to check hardlinked stuff?

Try simply 

du C:

or

du C:\

(or whatever drive letter is the mounted image)

if I get it right :unsure:, it should list all files and directories including hardlinks (once) and provide the same result as the pie/doughnut.

 

Then try with the -u switch.

 

:duff:

Wonko



#11 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 09:13 AM

Hmmm, once it was easier, the pie doesn't lie:

https://docs.microso...did-my-space-go

what's that now, the doughnut doesn't distort? :w00t:

 

How does DU behave in such a setup?

https://docs.microso...ls/downloads/du

 

 

:duff:

Wonko

 

My good friend, have some patience please, I also need to sleep some hours, and I was running Win7 when I saw this post from you, I'm on 10 now and I have the info from the running OS, not the VHD:

After running:

du C:\Windows\WinSxS

I got:

size: 6446555474 bytes = 6.0038 GB (GB calculated by me)

size on disk: 6675951616 bytes = 6,2174 GB (GB calculated by me)


After running:

Dism.exe /Online /Cleanup-Image /AnalyzeComponentStore

I got:

Component Store(WinSxS) information:

Explorer reported size: 6.50 GB

Actual size: 6.44 GB

    Shared with Windows: 5.99 GB
    Backups and disabled features: 451.44 MB
    Cache and temporary data: 0 Bytes

Date of last cleanup: 2017-12-13

Number of reclaimable packages: 0
Component Store cleanup recomended: Yes

Attached Files

  • Attached File  Du.png   343.74KB   0 downloads
  • Attached File  Dism.png   371.39KB   0 downloads


#12 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 July 2020 - 09:40 AM

here it yields file and folder counts, not lists.



#13 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 09:48 AM

Attached pictures of info related to same 6 GB VHD Compacted 4K, to get Dism info it was necessary to boot from it, that's why it took me some time to get Dism VHD picture, as shown on it a clean up is recommended, I will rum it and make a new picture latter to compare, but from previous pictures and this new pictures the du command is not giving same info as Dism, also Dism info is more complete.

 

alacran

Attached Files



#14 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 10:54 AM

Attached pictures of same VHD, on pictures left is before and right is after running the command:

 

Dism.exe /online /Cleanup-Image /StartComponentCleanup /ResetBase

 

As we can see the saved space is not a big gain.

 

alacran

Attached Files


  • antonino61 likes this

#15 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 July 2020 - 11:44 AM

@alacran

The comparison (in bytes) needs to be between the output of DU on the WHOLE volume and the "volume properties".

If you prefer, IF DU does what I suppose it should do, it would represent a command line way to have the same data as the GUI "right click on volume -> properties".

 

 

@antonino61

Yep, my bad it lists and counts and outputs the total counts.

 

:duff:

Wonko


  • antonino61 likes this

#16 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 02:20 PM

I formated the 6 GB VHD, and reinstalled on it same OS but this time as Compact LZX, just to compare:

 

This a summary:

 

Compact 4K install on 6 GB VHD:

 

Total used space on the VHD before reduction: 4.92 GB

Total used space on the VHD after WinSxS reduction: 3.97 GB (Sorry I didn't keep this photo)

 

 

Compact LZX install on 6 GB VHD:

 

Total used space on the VHD before reduction: 3.95 GB

Total used space on the VHD after WinSxS reduction: 3.10 GB

 

I attached 2 pictures the one with 2 images at left is 4K and at right is LZX, the other with a single image is LZX after running  base_winsxs.cmd from cdob.

 

 

Now I'm going to make a little analisys of the info:

 

Comparing 4K and 4K Reduced there is a saving of almost 1 GB

 

Comparing 4K and LZX there is a saving of almost 1 GB

 

Comparing 4K Reduced and LZX we can say they are almos same size.

 

Comparing 4K Reduced and LZX Reduced there is a saving of 0.87 GB, lets say almost 1 GB

 

Comparing 4K and LZX Reduced there is a saving of 1.82 GB, lets say almost 2 GB

 

Comparing LZX and LZX Reduced there is a saving of 0.85 GB, lets say almost 1 GB

 

All of them boot fine, so the criteria to select the appropiate compression for its intended use shoul be:

 

4K: This is the biggest in size, it is good for low resources equipment and can work on any system, the best option for everyday work, could be used same as a WinPE but with all OS features.

 

4K Reduced: Medium size, it is good for low resources equipment and can work on any system, it is possible you can't run fine some programs, good as a portable OS, could be used same as a WinPE but with all OS features.

 

LZX: Medium size, requires high RAM and a medium to high resources processor, good as a portable OS, if not dealing with low resourses PCs, could be used same as a WinPE but with all OS features.

 

LZX Reduced: This is the smallest in size, requires high RAM and a medium to high resources processor, it is possible you can't run fine some programs, good as a portable OS, if not dealing with low resourses PCs, could be used same as a WinPE but with all OS features.

 

NOTE: All suggestions/comments to this thread are wellcome, please don't ask me for more test, if you have some dubts make your own test to compare, also DO NOT compare your test results with mine, as your install will never be same as mine because of different programs installed, drivers, etc, as an example I have on this installs on Documents folder 134 MB of portable programs, mayority of them are uncompressables, so this is also afecting the size of this installs.

 

alacran

Attached Files



#17 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 02:38 PM

@alacran

The comparison (in bytes) needs to be between the output of DU on the WHOLE volume and the "volume properties".

If you prefer, IF DU does what I suppose it should do, it would represent a command line way to have the same data as the GUI "right click on volume -> properties".

 

:duff:

Wonko

 

Sorry my friend, I allready formated the VHD and made a new install Compact LZX, and by the way that program is slow and do not give as many info compared with Dism.exe /Online /Cleanup-Image /AnalyzeComponentStore cmmand as I found when testing both on WinSxS folder, then getting all drive size will be a long procedure especially on a drive with this very high compression, but maybe I have time to do it latter, other option can be create a new small VHD with some  info on it and then it can be done faster, if you only want to compare with Explorer, please let me know it this will be a good option for you.

 

alacran



#18 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 03:02 PM

Of course having enought RAM on your system Compact installs VHDs cold be Rambooted, and starting from 4K Reduced the VHD size can be smaller.

But during my previous test on Rambooting Wimboot VHDs on 4 GB Ram machines, the biggest size capable to run from a 4 GB RAM system with only the integrated Graphic Processor was a 1.5 GB VHD, so aproximately the RAM required is at least 2.66X the size of the VHD as a minimum.   But of course it depends on how much RAM is shared with the integrated Graphic Processor, and then available RAM can vary.

alacran



#19 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 July 2020 - 03:09 PM

Sorry my friend, I allready formated the VHD and made a new install Compact LZX, and by the way that program is slow and do not give as many info compared with Dism.exe /Online /Cleanup-Image /AnalyzeComponentStore cmmand as I found when testing both on WinSxS folder, then getting all drive size will be a long procedure especially on a drive with this very high compression, but maybe I have time to do it latter, other option can be create a new small VHD with some  info on it and the it can be done faste, if you only want to compare with Explorer, please let me know it this will be a good option for you.

 

alacran

 

 

Well, you started it with the "how to calculate size, wrong and right", I was only trying to provide a command line way (the obvious command line way for the "wrong" way is a DIR /S of the root of the volume) for the "right" way.

The DISM command only takes into account the WinSXS folder, not the whole volume so it is not directly comparable to "right click on volume->Properties".

Having a command line way may be useful for logging/automating tasks/etc.

 

:duff:

Wonko



#20 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 03:38 PM

Attached a photo of info requested, to my surprice when checking the all drive was very much faster than when checking WinSXS folder.

 

alacran

Attached Files



#21 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 03:49 PM

@ Wonko

 

It seems to me numbers don't match:

 

Info from Du for size on disk (used space) on GB is: 3.066852569580078125 GB

 

Info of used espace on Explorer is: 3.10 GB

 

alacran



#22 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 July 2020 - 03:50 PM

Cpu capabilities want considering as well, I guess - a high-end cpu can handle strongest compression levels in favor of lesser data-seeking scope.

#23 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 26 July 2020 - 03:56 PM

I used the word processor wich is same thing.

 

alacran



#24 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 July 2020 - 05:23 PM

Very nice pun, we call it paronomasia or paranomasia, as I like it.

#25 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 July 2020 - 06:36 PM

@ Wonko

 

It seems to me numbers don't match:

:

Maybe there is after all some issues with hidden or filesystem files and what not.

 

Another (easier) way try:

fsutil volume diskfree I:

 

:duff:

Wonko




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users