Jump to content











Photo

Windows Image File Boot (WIMBoot)

wimboot

  • Please log in to reply
163 replies to this topic

#151 Biatu

Biatu

    Member

  • Members
  • 76 posts
  •  
    United Kingdom

Posted 20 April 2017 - 05:14 PM

@Wonko, I have a theory, on earilier version of WinpE everything just uses boot.sdi, which has a small NTFS fs on it. The wim is loaded into ram, and then the wim is overlain over the small sdi in memory, no? If im not mistaken M$ just modified this paradigm for a full system.



#152 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 20 April 2017 - 06:51 PM

@Wonko, I have a theory, on earilier version of WinpE everything just uses boot.sdi, which has a small NTFS fs on it. The wim is loaded into ram, and then the wim is overlain over the small sdi in memory, no? If im not mistaken M$ just modified this paradigm for a full system.

 

Yes and no (if I get what you mean).

NTFS inside the boot.sdi is used (this feature is available in NTFS at least since 2K)  i.e. it is a feature (alternative to assigning a DOS drive letter)  as a mount point of the contents of the boot.wim expanded inside memory (a ramdisk).

https://technet.micr...ibrary/cc771845

 

 

The WOF driver directly creates a volume (or however proper pointers to files) directly from the compressed contents of the wim, at least this is my understanding:
http://reboot.pro/to...mboot/?p=187014

http://reboot.pro/to...mboot/?p=187100

The issue (and most probably the reason why wimboot has seemingly been abandoned after 8.1) is that a Wimboot is not serviceable easily as each and every update would go "outside of it" creating a situation with "duplicates" not entirely unlike that of the stupid WinSxS and thus - little by little - hogging disk space (which was the reason for Wimboot in first instance, being able to deploy the stupid Windows 8.1 to stupidly underpowered - diskwise - devices like tablets and similar), and was replaced by compactOS:
https://msdn.microso...ktop/compact-os

 

:duff:

Wonko



#153 Biatu

Biatu

    Member

  • Members
  • 76 posts
  •  
    United Kingdom

Posted 20 April 2017 - 07:22 PM

ok, then what about mounting a wim on disk? that creates pointer files no?

WinSXS is supposed to be hardlinked anyway, so not dups, unless in the case of WinPE where builders actually copy the files and not linking them (but that don't really matter as wim compression dedups the data)

 

Personally CompactOS is useful, but I think a union-like fs in windows would be extremely powerful...esp in WinPE environs


Edited by Biatu, 20 April 2017 - 07:25 PM.


#154 Biatu

Biatu

    Member

  • Members
  • 76 posts
  •  
    United Kingdom

Posted 20 April 2017 - 07:28 PM

For example, using some g4d tricks, you can set the boot image in a WIM on the fly, enabling multiple versions of WinPE in a single WIM, programs drivers etc, mount on the fly only as needed.

Having a layout like this:

Wim/Win7.32

Wim/Win7.64

Wim/Programs

Wim/Programs.32

Wim/Programs.64

Wim/Drivers

Wim/Drivers.32

Wim/Drivers.64

 
Idea for x64:

Link Wim/Win7.32/Windows/System32 > X:/WIndows/Syswow64


Edited by Biatu, 20 April 2017 - 07:29 PM.


#155 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 20 April 2017 - 08:03 PM

Definitely I am not following you :(, particularly when you mention (unspecified) grub4dos tricks. :unsure:

 

WinSxs concept is "right" for the hardlinking, it's the issue behind it (DLL HELL) that creates the bloat (as you have n versions of a same dll).

The Windows 8.1 wimboot update would create the same in the sense that latest dll would be outside the wim and inside it there would be an older version, then, little by little each and every file would be updated and thus a large part of the system wuld end up residing outside the wim.

 

:duff:

Wonko



#156 Biatu

Biatu

    Member

  • Members
  • 76 posts
  •  
    United Kingdom

Posted 20 April 2017 - 08:10 PM

sorry for the confusion, what I was saying is modify the byte that specifies which image to boot in the wim header. 

Ah, i guess ur right about the versioning. Off the top of my head, a solution would be to version the functions instead of the entire lib, and maybe use a JIT like system when something is called so that in memory it would function like the needed version

 

About the reparse points, when a WIM is mounted on a disk (for editing) with DISM, are the same type of links created in that process?


Edited by Biatu, 20 April 2017 - 08:12 PM.


#157 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 21 April 2017 - 08:34 AM

Ah, i guess ur right about the versioning. Off the top of my head, a solution would be to version the functions instead of the entire lib, and maybe use a JIT like system when something is called so that in memory it would function like the needed version

Yes, that would have been a nice solution, had it been implemented in a timely manner (around NT 4.00 times).

 

As I see it, a DLL is a library of functions (actually it is a library of functions ;)) not entirely different from a real  library containing books.

 

In a real library, a book is indexed (besides by name) by the area, cabinet,  shelf, and position on the shelf and - of course - before these detailed location data, by the street address of the library itself.

 

Now imagine that every time you replace or add a single book with a new version in the library you build a new building right besides the original building on the south side, put in it first a copy  of ALL the books that were in the original library and only then you add or replace the particular book.

 

It doesn't make much sense, put this way, does it? :dubbio:

 

Additionally, you also make a copy of the street number of the original building and place it on the south side of it, while you put a new one on the north side, so that people coming from the south side will mistake the new building for the old one, whilst the people coming from the north side will see the proper street address ... :frusty:

 

... before or later you will occupy with new buildings all the available space, up to Vista it was possible (with a trick or two) to install on a non-NTFS filesystem, with 7 barely so (probably removing some functionalities), JFYI:

http://reboot.pro/to...rdlinked-files/

 

About the reparse points, when a WIM is mounted on a disk (for editing) with DISM, are the same type of links created in that process?

Yes, the Wim (intended as a file format) is nothing but a form of filesystem representation (compressed), the mounting via DISM is very similar to the mounting of a (plain) volume to a mountpoint (or reparse point).

 

:duff:

Wonko



#158 Biatu

Biatu

    Member

  • Members
  • 76 posts
  •  
    United Kingdom

Posted 21 April 2017 - 09:44 AM

@wonko, lol neat analogy. I meant literally versioning the exports, or at least implementing a reliable dedup on NTFS.
 

 

... before or later you will occupy with new buildings all the available space, up to Vista it was possible (with a trick or two) to install on a non-NTFS filesystem, with 7 barely so (probably removing some functionalities), JFYI:

http://reboot.pro/to...rdlinked-files/

Like CDFS?

 

 

Yes, the Wim (intended as a file format) is nothing but a form of filesystem representation (compressed), the mounting via DISM is very similar to the mounting of a (plain) volume to a mountpoint (or reparse point).

So, in theory, can we hook into the mounting function, or implement one that will essentially put those reparse files anywhere? Or...mount a wim, then hardlink/symlink needed files elsewhere?



#159 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 21 April 2017 - 11:30 AM

Like CDFS?

No, like FAT32:
http://reboot.pro/to...files/?p=182890
 
 
 

So, in theory, can we hook into the mounting function, or implement one that will essentially put those reparse files anywhere? Or...mount a wim, then hardlink/symlink needed files elsewhere?

 
The issue is - I believe - that hardlinks that work just fine at the lower possible level (since they are actually duplicated $MFT entries for a same file) are rigorously "same volume/filesystem only" and softlinks (and mountpoints/reparse points) *somehow* (and the actual implementation in different OS  may well vary) are at a "higher level" and they are simpy "not seen" or "not parsed properly" by *everything*, an example (old I know) :
 
http://web.archive.o...showtopic=23626
 
Maybe, just like symlinks need a "special" driver on XP:
http://schinagl.priv...nksforwindowsxp
 but are "included" in later OS, the situation has changed and support for those is "at all levels". :unsure:

There is this thingy here, but hard to say if it works/is maintained, etc.:
http://write-code.bl...h/label/fsredir
http://reboot.pro/to...9-fsredir-v101/
and seemingly it is only x86 :dubbio:
 
:duff:
Wonko

#160 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 12 July 2017 - 02:26 AM

Relevant : One can, if they so wish, apply for a reparse tag from Microsoft by sending some basic details like Name, product, driver GUID, and a couple of other technical fields like "high-latency". The link is https://docs.microso...int-tag-request

 

@Wonko: Correct me if I'm wrong, but a less-bombastic, less-heavily marketed version of "DoubleSpace" from our former Reboot.pro WIMBoot enthusiast Simon is basically the solution for the problem of files going "outside" the WIM and slowly killing disk space, and in a broader sense provides the ability to service a WIMBoot volume that Microsoft should have provided in the first place. Or have I missed something?



#161 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 12 July 2017 - 08:09 AM

No ideas.

wimboot (the MS feature) was nothing but a meteor passing into the sky.

It has since been deprecated and I believe (but cannot really say) completely removed, replaced by system compression:

https://blogs.window...pact-footprint/

 

:duff:

Wonko 



#162 milindsmart

milindsmart

    Frequent Member

  • Advanced user
  • 201 posts
  • Location:Bangalore
  •  
    India

Posted 12 July 2017 - 03:25 PM

Well the compression ratio by CompactOS is not nearly as good as that obtained by Wimboot. If the merits are present, then I hardly think we will be stopped by its mere lack of official support :-) after all Windows 8.1u1 still supports it.

The question is : is the idea technically sound, and can it be made practical with some additional tooling?

#163 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 12 July 2017 - 03:45 PM

Well the compression ratio by CompactOS is not nearly as good as that obtained by Wimboot. If the merits are present, then I hardly think we will be stopped by its mere lack of official support :-) after all Windows 8.1u1 still supports it.

The question is : is the idea technically sound, and can it be made practical with some additional tooling?

No, the question is rather:

 Does it make sense to supercompress something bound to be in a continuous state of flux and thus needing a continuous re-compressing/updating to be "tamed" (or if you prefer to keep the "outside" files under control)?

NO (generally speaking).

It makes (made) some sense IF and ONLY IF there are objective hardware limits, such as it was the case for the original Wimboot on low-low spec tablets and mini-tablets (or similar) with a too small amount of storage to effectively host *any* Windows NT 6+[1], or in the very rare cases where - due to extreme slowness of the storage subsystem - it may be convenient to use a lot of processor resources to speed up things[2].

 

In any case both imply a basically "static" structure, which Windows 10 doesn't guarantee anymore.

It may still be useful for a "OS snapshot" or for a PE, but in this case you don't actually need to update it.

 

 

:duff:

Wonko

 

Correct answers, Wimboot being only a "partial" and "inconvenient" solution:

[1] A lighter, smaller OS

[2] A faster storage subsystem and a lighter, smaller OS



#164 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 13 July 2017 - 06:04 PM

Well the compression ratio by CompactOS is not nearly as good as that obtained by Wimboot. If the merits are present, then I hardly think we will be stopped by its mere lack of official support :-) after all Windows 8.1u1 still supports it.

The question is : is the idea technically sound, and can it be made practical with some additional tooling?

 

First let me tell you I have experimented a lot with Wimboot and CompactOS and I am totally agree with Wonko.

 

Wimboot: looks very nice when OS is just installed, but once you start using it, it grows up (depending of use and/or updates).

 

Compact: every time you add someting to your OS it is compacted, so it does not grow up as much.

 

Using WinNTSetup you can deploy the OS even more compacted than using Windows guides.  But then you need to remember you have to pay a price for compression every time you read (not very notorious with a good CPU) or write someting (in this case bottleneck is your CPU + disk speed).

 

Compact mode is a usefull alternative if you want to have a full system install on a *.VHD located on your internal disk or on an external USB 3.0 drive as a second boot option (maybe 10 GB - without pagefile,  depending of OS and progams), It is always better than booting from a PE.

 

So you need to do some test to find the higher compression that fits your needs without a high penalty in speed.

 

alacran

 

EDIT: Forgot to say WinNTSetup allows you to make Wimboot or Compact installs even of OS's that do not support it normally as Win7.


Edited by alacran, 13 July 2017 - 09:02 PM.






Also tagged with one or more of these keywords: wimboot

3 user(s) are reading this topic

0 members, 3 guests, 0 anonymous users