323 replies to this topic

[antonino61]

btw, I have found out that the smallest winsxs u have got as a result of cdob's procedure works instead of any winsxs in any other system (then of course u might as well apply wimb's procedure, as you might have in any other case, of course). let me see if u can cdob it again. see you in a bit.

 

Edit: still wants confirming, pls wait.

[wimb]

Most files in WinSxS folder have hardlink to files in other folders.

What you want is to save space, but in any case keep in WinSxS folder the files that have no hardlink such as gdiplus.dll in

\windows\winsxs\*_microsoft.windows.gdiplus_*\*.* 

base_winsxs.cmd is doing that but it throws away too much.

 

An example, let us investigate the files in:

\windows\winsxs\*_microsoft-windows-servicingstack*\*.*
\Windows\WinSxS\*_microsoft-windows-twinui*\*.* 

file cleanupai.dll and iissetupai.dll in servicingstack are occurring only in WinSxS folder and are thrown away by the present base_winsxs.cmd

 

 

On the other hand we can use wimlib that takes hardlink into account and each distinct file content is stored only once in the WIM archive:

 

 

wimlib fully supports the file contents deduplication or "Single Instance Storage" which is built into the WIM format. Each distinct file contents is identified by its SHA-1 message digest and is stored only once in a given WIM archive. This is true even if the contents are referenced from multiple WIM images in the same archive, from multiple identical files or hard links, or from multiple file streams such as named data streams. When creating or updating a WIM archive, wimlib automatically performs the needed deduplication. This capability means that the WIM format and wimlib are useful for incremental backups and for distributing multiple similar directory trees such as operating system images (for example).

 

[Wonko the Sane]

Most files in WinSxS folder have hardlink to files in other folders.

 

Only to reiterate that in NTFS files are NOT really-really in a folder or in another one, even the first (single) reference to a file is essentially a hardlink.

 

There are tools to obtain deduplication (via hardlinks) on the "underlying" NTFS.

 

A nice list:
https://tinyapps.org...hard_links.html

 

 

 

Wonko

[wimb]

Only to reiterate that in NTFS files are NOT really-really in a folder or in another one, even the first (single) reference to a file is essentially a hardlink.

 

There are tools to obtain deduplication (via hardlinks) on the "underlying" NTFS.

 

A nice list:
https://tinyapps.org...hard_links.html

 

 

 

Thanks for the interesting link.

 

It would be interesting to make a list of files in WinSxS folder for data that have only one hardlink and should not be removed from WinSxS folder such as 

file cleanupai.dll and iissetupai.dll in servicingstack

[antonino61]

thanx wonko, I have tried clonespy. it allowed me to remove other files, most of them old ones related to software which is no longer there. then I ran registry1staidportable which did the rest of the job. I am still after trying to export the "smallest" winsxs to other builds and possibly see if the batches do any good. in a bit.

[Wonko the Sane]

Thanks for the interesting link.
 
It would be interesting to make a list of files in WinSxS folder for data that have only one hardlink and should not be removed from WinSxS folder such as 
file cleanupai.dll and iissetupai.dll in servicingstack

Check finddupe:
https://tinyapps.org...hard_links.html
https://www.sentex.c...andel/finddupe/
 
With the -listlink option.

Please note how useful it can be the -bat option.


Wonko

[antonino61]

now trying finddupe

[antonino61]

a big thanx to wonko for his light-shedding inputs. finddupe is just one of the examples enabling me to look closer and closer to "the real thing". now I think I can understand what wimb implied earlier when he said something like "u can do it all with wimboot, after all that is what wimboot does". fine, but I think it can do it all the better on a full vhd shrunk in advance. maybe I am wrong, lets see anyway. on trying finddupe, I think we would need a parameter that will enable us to exclude some directories. I did an all-out try - total disaster. recopied the vhd and tried again with some handmade restrictions like not create hardlinks on hhctrl.ocx and the like, to at least enable registry1staid to recompose the linking structure. wonderful job but would not boot. I think we need to exclude c:\boot. one more thing - finddupe tends to hardlink some of the system32 content to syswow-etc. when I think both are needed, for booting at least. now I have tried to run finddupe -bat work.bat -hardlink -j -sigs c:\programdata\** and it has worked. what is the real name of \program files? finddupe will not accept spaces. pls help.

 

ps.: also finddupe -bat work.bat -hardlink -j -sigs c:\users\** has worked

[Wonko the Sane]

 finddupe will not accept spaces. pls help.

No program will accept spaces, enclose the path in double quotes (like in all the other 99.99999% programs when the path contains spaces).

 

And/or use dir /AD /x to list the "short" (and spaceless) names of the directories.

 

 

 one more thing - finddupe tends to hardlink some of the system32 content to syswow-etc. when I think both are needed, for booting at least.

This is the LAST time I say this, there are NO "contents" in System32 (nor in any other directory) there are ONLY links to extents that are addressed in the System32 folder or in another folder.

 

 

Wonko

[antonino61]

believe me, wonko, I thought this referred only to winsxs. that the system32 has no file it is the first time I have heard it. sorry. btw, PROGRA~1\** works, now trying PROGRA~2\**

 

ps: is that the reason why u cannot decide whether x hardlinx y or y hardlinx x, as it all depends on which the real file is located in?

 

ps: apparently not, as ur predicament applies to the whole fs, not just to the windows dir. anyway I have made a work.bat that works, here it is, it was obtained by launching 

 

finddupe -bat work.bat -hardlink -j -sigs -rdonly -z c:\windows\**

 

and then by deleting all system32--syswoa64 instances. these dont work apparently because all is system32 is hardlinked to syswoa64 and not the other way round. if one told me how to direct the hardlinking I would try and see if I am right. for the time being, this work.bat works.

Attached Files

•  work.bat   727.39KB   3 downloads

[wimb]

There are many, many files (in fact too many) in WinSxS folder that refer to data having only 1 hardlink.

I think it is not so good idea to remove most of the WinSxS folder.

 

Better is do Capture and Apply with wimlib as used in VHD_WIMBOOT so that each distinct file content is stored only once.

wimlib normal mode will give already a nice reduction in used size of the Win10x64 Installation.

 

Far better is using Capture and Apply in WimBoot mode so that almost all filedata is located in the WIM file , and only referred to by pointers in the VHD file.

In that case the used size of the full Win10x64 Installation in VHD is about only 500 MB for a Portable Win10x64 System booting from VHD loaded into RAMDISK

The VHD + WIM filecombo can be located on Portable SSD connected via USB to various hardware.

 

Comparison of WinSxS filelist (Left) with Content having 1 Unique HardLink (Right)

 

 Unique_HardLink-2020-07-17_182802.jpg   225.14KB   4 downloads

[antonino61]

ok wimb, for portability and for system with a lot of software installed, not for those of us whose systems have next to nothing installed. i can tell u that the size winsxs is shrunk to will stay the same for months, even years, with nothing else installed, in which cases it is well worth the trouble of shrinking it, and then, why not, wimboot, if need be.

[antonino61]

*quote

if one told me how to direct the hardlinking I would try and see if I am right.

*quote

 

no dice. I did it manually inverting the hardlinks by cutting and pasting the addresses. no way, if u have 32bit apps (more apps rely on 32bit software than most of us will ever know). for fully 64bit apps that would be ok, but there is no telling if it is the case for most of us.

[Wonko the Sane]

@antonino

It seems like you cannot reach the necessary abstraction level to understand the matter in detail.

 

 

 

 

believe me, wonko, I thought this referred only to winsxs. that the system32 has no file it is the first time I have heard it. sorry. btw, PROGRA~1\** works, now trying PROGRA~2\**

 

ps: is that the reason why u cannot decide whether x hardlinx y or y hardlinx x, as it all depends on which the real file is located in?

 

 

Well, at least I tried:
http://reboot.pro/to...f-out/?p=215304

 

Wonko

[antonino61]

yes  wonko, this is why I corrected myself. no filename that comes to us is, IT-wise, the real name. but we ordinary mortals cannot get there. and I posted what I have and have not achieved. I also tried the comparison u advocated, but to no practical avail - it is like trying to find a needle in a haystack, so I gave up the last command suggested on the site u posted. the work.bat has to be done on a per-machine basis and I said how that had been done. all hardlinks work fine, except for the system32--syswoa64 ones, never mind syswoa64--system32 if u have 32bit-related software (wonder who does not). btw, I had previously done the hard linking of |program files and |program files (x86) by typing \progra~1\** and \progra~2\**. aint it correct? not so much stuff, though.

[antonino61]

well, I have now discovered that if u run tweaking.com windows repair in advance, u will be able to make 

finddupe -bat work.bat -hardlink -j -sigs -rdonly -z c:\windows\** work, even for system32 and syswoa64

 

here is the new file. the rest of the finddupe instances posted above are ok.

 

Attached Files

•  work.bat   983.98KB   5 downloads

[wimb]

I have determined in Win10x64 folder WinSxS the number of files referring to data on harddisk having 1 hardlink only.

Such files should not be removed since then the corresponding data will be lost ....

Excluded in the search were the folders Backup, Catalogs and FileMaps.

 

In my case from the total of 46.655 files in 13.410 WinSxS root folders there are 24.648 files (about 53%) that refer to data having 1 hardlink

It means it is not good to remove most of the WinSxS folder .....

 

== 

[Wonko the Sane]

wimb, on 19 Jul 2020 - 2:04 PM, said:

I have determined in Win10x64 folder WinSxS the number of files referring to data on harddisk having 1 hardlink only.
Such files should not be removed since then the corresponding data will be lost ....
Excluded in the search were the folders Backup, Catalogs and FileMaps.

In my case from the total of 46.655 files in 13.410 folders there are 24.648 files (about 53%) that refer to data having 1 hardlink
It means it is not good to remove most of the WinSxS folder .....

WinSxS-HL-2020-07-19_135041.jpg

It depends in which context.

If the context is "keep ALL the functionalities of a standard install" you are obviously correct.

If the context is "reduce as much as possible source" having 24.648 files (of which - say - 18.398 are NEVER used by any program in the reduced environment) is meaningless, what is needed is finding which files are not used in the reduced environment and remove them or (alternative) remove nonetheless the whole 24.648 files and re-add only the 6.250 that are actually needed (or more if they are needed).

Heck!

This is "Windows Extreme" it is too easy (and actually out of place) to reiterate "better leave things as they are in order to risk less", the whole section is about crazy, risky, (and in some case even pointless) dangerous hacks (largely attempted just for the sake of it).


Wonko

[wimb]

The context is that you want stable operating system that not refuses at unexpected moments.

 

The point is that you can keep "all the fluff" in the installation almost without using space 

by putting most of the data in the WIM archive and only referring by pointers to that data (that includes a lot of fluff).

That WIMBOOT approach is however not officially supported anymore by Microsoft, allthough it is still working quite well ....

 

It would be nice if Microsoft would reduce Windows 10x64 to a core minimum of say 10% of the installation 

and that people can add the fluff at will .....instead of now that we are trying to remove the unwanted parts of the operating system.

 

Removal of the unneeded part of the WinSxS folder does not save that much space - it is only about 825 MB ......

We can and want to remove at least 10 GB .... and would like to keep the size small 

 

The core funtionality of Windows 10x64 is the best OS ever made by Microsoft.

Booting and Install from USB and fast install and automatic install of all drivers for most hardware ....

 

At the same time Windows 10 is almost the worst OS ever made by Microsoft

It consumes a lot of disk space and is all the time growing .....

Microsoft easily can do better and improve and slim down Windows 10

[antonino61]

wimboot can be used anyway, so it is no variable, as it can be used even after the fluff-out, thus making the wim even smaller. this initiative of mine, which would not go any further than itself without the help of most of u, is in no way whatsoever an alternative to wimboot; it is rather a further enhancement (of the shrinking process).

[antonino61]

btw, getting back to what we were saying earlier, I think it is better to hardlink \windows\** without the -rdonly and -z parameters, to be on a safer side.

[wimb]

wimboot can be used anyway, so it is no variable, as it can be used even after the fluff-out, thus making the wim even smaller. this initiative of mine, which would not go any further than itself without the help of most of u, is in no way whatsoever an alternative to wimboot; it is rather a further enhancement (of the shrinking process).

 

Question:

In case of using VHD_WIMBOOT for Capture and Apply in WimBoot mode of Win10x64

then how much used space in the VHD is saved by previous removal of the unneeded part of the WinSxS folder by using the cdob batches ?

Is it more then 10 MB ?

[antonino61]

wimb, I have 2gb, 2.5gb 3gb wims.

as for the size of win10, with all the fluff in, including store, xbox etc. u got a point. but, who wants all that fluff? I do not. I am still a desktop user, not a tile swiper. with all the fluff out, I can tell u that win7 paradoxically uses more space and is nimbler. it is the worst system with the fluff. without the fluff it is the slimmest and nimblest microsoft could ever have made. I tried to do on win7 the same as I do on win10, well, I simply could not. so it depends of what win10 environment u r talking about.

wonko calls it "reduced environment; I call it "normal environment", as a desktop had always been before the smartphone era.

[antonino61]

btw, be cagey and copy the vhd someplace else before running the \windows\** hardlinker. some software may not work. the rest of the hardlinks (programdata\**, program files, program files (x86) and users still work ok.

 

ps: the \windows\** kinda thing seems to apply both to 14xxx and 19*** builds. so do it and check. if anything vital to u does not work, u can resort to your backup copy.

[wimb]

Download: Dir_List.zip contains program Hard_Link_List_x64.exe

 

- Dir_List Folder with Programs Dir_List_x64.exe and Hard_Link_List_x64.exe

 

- Dir_List_x64.exe allows to select a folder so that Folder_List and File_List of that Path are created

 

- Hard_Link_List_x64.exe allows to select a folder so that first Folder_List and File_List are created

  Then Hard_Link_List_x64.exe auto uses h_list.cmd and File_List to create Single_Hard_Link_List and Multi_Hard_Link_List

 

- Program h_list.cmd can be used separately in Admin command window with File_List (can be modified) as parameter 

  e.g. File_List_P_WinSxS.txt as parameter to search in P:\Windows\WinSxS folder for Single Or Multi Hard_Link data

 

- Program h_sxs.cmd can be used in Admin command window with Windows Path as parameter for search in WinSxS folder 

  e.g. P:\Windows as parameter to search in P:\Windows\WinSxS folder for Single Or Multi Hard_Link data

  In the search in WinSxS folder then the folders Backup, Catalogs and FileMaps are excluded in case of using h_sxs.cmd

 

 

- Program h_path.cmd can be used in Admin command window with any Path as parameter for search of type of Hard_Link data in that Path

  e.g. P:\Windows\System32 as parameter to search in P:\Windows\System32 folder for Single Or Multi Hard_Link data

 

- Output in all cases is Single_Hard_Link_List_YYYY-MM-DD.txt and Multi_Hard_Link_List_YYYY-MM-DD.txt files

 

Disable Windows Defender to increase the speed ....

 

You might use Dir_List to make File_List and use notepad++  with  Plugins > Compare option to Compare File_Lists

and can conclude what is the difference in the File Lists

 

More Info on Hard Links 

https://docs.microso...component-store

https://en.wikipedia.../wiki/Hard_link

https://web.archive....at-are-they-for

 

- WinSxS_reduce by cdob - http://reboot.pro/to...f-out/?p=215317