163 replies to this topic

[Wonko the Sane]

 

I thought that only ntoskrnl could load a driver?

I was conforted in that idea by the fact that the system registry hive is also a plain file as indeed this hive contains the list of drivers to be loaded by ntoskrnl.

So I would bet on between 4 and 5 in your previous post 

Yep , but the at least in my previous post may mean that *something* needs to be passed (as a parameter or *whatever* ) to ntoskrnl (or to the earlier loaded winload.exe) to "tell it" to load the WOF.SYS driver first.

 

If you prefer, IF there is the need of a "special request" to load the WOF.SYS driver, then it is to be seen if the request is originated (and later passed on) :

 

a. between 2 and 3,
or
b. between 3 and 4
or
c. between 4 and 5

 

As said, doing the BOOTMGR replacement experiment may give some further hints about what is happening....

 

If we can exclude a "particular" BOOTMGR version from the equation (and hopefully exclude also the winload.exe in a later similar experiment) supporting earlier OS version should be more likely to happen... 

 

Wonko

[erwan.l]

I am applying a wimboot file as we speak (from a Winpe8.1 PRE U1 with wimlib-imagex).

I'll boot once to make sure my wimbooted system is fine.

 

Then, I will replace the bootmgr from my system (i.e 8.1 u1) with the one from my Winpe (i.e 8.1 pre u1).

[milindsmart]

Is ntoskrnl a flat file? Or is it merely not compressed, but still only within the WIM, with only a pointer file on boot volume?

I suspect the latter... So I guess it would be between 3 and 4. But of course, let's not underestimate winload... It could easily be able to read ntoskrnl all on its own, without WoF...

But it would be needless complexity... Ntoskrnl.exe is best kept flat...

Edited by milindsmart, 04 September 2014 - 07:17 PM.

[Wonko the Sane]

Is ntoskrnl a flat file? Or is it merely not compressed, but still only within the WIM, with only a pointer file on boot volume?

I suspect the latter... So I guess it would be between 3 and 4. But of course, let's not underestimate winload... It could easily be able to read ntoskrnl all on its own, without WoF...

But it would be needless complexity... Ntoskrnl.exe is best kept flat...

 

As a side note (REAL, not rhetorical) questions:

1. Is there a way (booting a Linux, access the filesystem through NTFS4DOS, or grub4dos or any similar non-native/non-MS NTFS filesystem driver) to see the difference between a "real", resident file and a "link to a file inside .wim" one?
2. And/or (though similar it is a different question) is there a way in the booted 8.1u1 (DIR or similar tools, possibly FSUTIL) to distinguish them?
3. Or a third party tool to the same effect?

 

Wonko

[erwan.l]

Under windows 8.1u1, you can make a difference by cheching size vs size on disk.

A pointer will have a very small size on disk when its size will be its "real" size.

[Wonko the Sane]

Under windows 8.1u1, you can make a difference by cheching size vs size on disk.

A pointer will have a very small size on disk when its size will be its "real" size.

You mean by right clicking on each file and reading its "properties"?  

 

I was thinking more of something capable of producing the equivalent of a DIR C:\ /S>D:\mydir.txt kind of file list.

 

Wonko

[erwan.l]

You mean by right clicking on each file and reading its "properties"?  

 

I was thinking more of something capable of producing the equivalent of a DIR C:\ /S>D:\mydir.txt kind of file list.

 

Wonko

 

I remember that Synchronicity had released such a tool (somewhere on wimlib-imagex thread).

 

EDIT : here.

Not sure if the tool is still available thus or updated.

[misty]

From http://reboot.pro/to...ation/?p=183900

...Just out of curiosity I then tried replacing bootmgr with an older version (from Windows 7 (SP1)) - Windows 8.1 then failed to reboot. Looks like the WOF driver isn't the only dependency...

This indicates that bootmgr is also a dependency. A retest (by another party) would be good - preferable using all other bootmgr versions (from Windows Vista/Vista SP1/7/7 SP1/8/8.1 and maybe server versions too - if they differ) to confirm if the Windows 8.1u1 bootmgr is required.

I did some tests a week ago using WinPE 5.0 updated using simonkings tool to include the WoF driver. Following the proceedure I documented here the updated WinPE failed to Wimboot - I vaguely recall also testing it with the WinPE 8.1u1 bootmgr - it still failed to boot. Again, a retest by another party would be useful.
 

....MIsty attempted *something* loosely similar but failed:
http://reboot.pro/to...mboot/?p=183968
(using the same Registry entries alacran posted and that were confirmed to be "right" by the use simonking made of them)

Since it was a half-hearted (besides possibly half-@§§ed ) attempt (no offence whatever intended to the nice try Misty made, of course) I would consider the negative result as a non-definitive one (still seeing the glass half full ).

Hahaha - no offence taken - half-@§§ed sums my efforts up perfectly. I need to relook at my little test, but I think that simonking might have included some additional registry settings and files. The test was done on a different PC and I'm busy with work and personal commitments at the moment. Fingers crossed someone will have figured out how to add wimboot support to non 8.1u1 sources ready for when I next have time to play. Good luck and keep up the good work.

Regards,

Misty

[simonking]

Microsoft assigns altitudes to file system filter drivers. This altitude determines which filter driver gets loaded first. For example, you want to load your WoF driver before you load your anti-virus driver that does dynamic scanning, etc.

 

You can also tell if a file is stored inside a WIM using some IOCTL calls. DriveSpace 5.1 implements this to skip frivolous recompression of files stored inside a WIM using NTFS, which results in a net loss of space. Eric's tool also does the same thing on the command line as noted here.

 

My tool doesn't do much else than Misty's. I suppose the WoF driver may have dependencies on other system components, kernel dlls, etc. to run properly. Or it might even have a hard-coded version check.

 

What makes the most sense for our approach is to build a new filter driver. This would be no trivial task, however: Providing fast random read-only access to files inside an archive, and also transparent extraction of the pointers upon write request.

 

What'd blow everything Microsoft's done out the water would be if you could seamlessly recompress and reabsorb modified files back into the archive, re-pointer-izing files that are added after WIMBoot'ing or modification. But that's really hard, as Microsoft's simpler approach shows.

[milindsmart]

That reminds me ... Is there some thread listing all the different bootmgr versions? That would be useful for my other experiments too.

Perhaps it's better we list only bootmgr.exe versions, since that's what matters usually.

[Wonko the Sane]

@Misty

Yep , but there are some (unrelated to the matter at hand) changes between 7 and 8 BOOTMGR (and also in the \boot\BCD), hence I believe that going in one step only from 8.1u1 to 7 is too big a leap, and the idea was to try again with:

1. 8.1u1 (winbooted)->8.1 BOOTMGR
2. 8.1u1 (winbooted)-> 8 BOOTMGR

@simonking

As said, we do have a (IMHO) very similar third party filter driver, open source:

http://schinagl.priv...nksforwindowsxp

which should tell us that it is perfectly possible to write this kind of drivers

 

@milindsmart

I am not sure to understand the note about BOOTMGR vs. BOOTMGR.EXE (after all they are essentially the *same* file).

 

What I can tell you is that having a proper list, will likely take a lot of time/effort, at least judging from the difficulties we had to §@ç#ing extort (through torture or very, very loud shouting ) the similar kind of information related to msv1_0.dll in the PassPass thread or the amount of work I had to put together to make a "proper" list of bootsect.exe versions here :

http://www.msfn.org/...sions-compared/

 

Wonko

[simonking]

jaclaz, did you link the correct page? The Link Shell Extension tool is a glorified UI, its not a driver.

[Wonko the Sane]

jaclaz, did you link the correct page? The Link Shell Extension tool is a glorified UI, its not a driver.

The link should bring you around two thirds down the page where the Symbolic Links Filter Driver for XP is talked about.

 

If your browser doesn't get you there directly , just scroll down, reading until you find:

Technically it seems that NTFS5 even shipped with WindowsXP has always supported symbolic links, but the functionality was not available for user mode applications, and even not for the upper layers of ntfs.sys

 

 

Wonko

[milindsmart]

Yeah the first thing we must check is, given a properly wimbooted system, replacing the bootmgr with a pre-u1 file, does it still boot. One step at a time...

 

Bootmgr.exe is contained within bootmgr appended to a 16-bit header that decompresses it... that decompression stuff could surely have changed, we don't know till we verify. Perhaps you could say, we need 2 lists, one of bootmgr.exe and other of bootmgr.header

 

Let me see if I can find the time to start a list like that, because it'll become very useful soon.

[simonking]

Aha. I like where he says "And, sensation (!)". Sounds like Tintin.

 

Still I am not sure if a symlink driver is the same thing as a compression driver though.

 

Eric, does any of this ring a bell?

[Wonko the Sane]

Still I am not sure if a symlink driver is the same thing as a compression driver though.

Which is good, since - seemingly - the WOF.sys (like the mentioned XP driver) is mainly a File System Filter Driver (which seems like being one among the most complex kind of drivers from what I have seen, or at least one of those less "talked about"), though there are some resources available, like:

http://www.codeproje...Driver-Tutorial

and even the MS documentation is not that bad:

http://msdn.microsof...2(v=vs.85).aspx

and - provided that they can put aside some time on this - we may have Sha0 and Karyonix on our side:

http://reboot.pro/to...em-redirection/

 

 

I am pretty sure that - in due time - Sinchronicity (or someone else) will be able to study/solve the "compression" part and I believe that the wimlib already contain the needed related parts, after all it can access the contents of a .wim created with the /wimboot switch, can't it?

 

I still believe the glass to be half-full, but we need to go on step-by-step, as said...

 

Wonko

[misty]

I did a quick retest this morning with a standard (unmodified) Winre.wim taken from a Windows 8.1 Update 1 (Enterprise Evaluation) installation.

I did the following -

• Applied Winre.wim to a clean partition (D: drive)
• 
  

  wimlib-imagex.exe apply Winre.wim 1 D:\

• Added bootmgr (from Windows 8.1u1 source)
• Added a BCD store (manually created and hardcoded to boot D:)
• Captured the D: drive using the wimboot flag
• 
  

  wimlib-imagex.exe capture D:\ Wimboot.wim "WoF Test" --config=customwimbootcompress.ini --wimboot

• Reformatted D: and applied the contents of wimboot.wim to it
• 
  

  wimlib-imagex.exe apply Wimboot.wim D:\ --wimboot

This booted successfully - no surprises there.

I replaced the Windows 8.1u1 bootmgr with bootmgr from Windows 8.1 - it still booted.

I replaced the Windows 8.1 bootmgr with bootmgr from Windows 7 (SP1) - it still booted.

Could be my earlier tests using a wimbooted MistyPE (based on boot.wim from a Windows 8.1u1 iso) were not done correctly - I'll retest this after the weekend. Could alternatively be due to the different sources used (Winre.wim as apposed to boot.wim) or other factors.

Can someone please verify the bootmgr tests - if possible using a full (wimbooted) Windows 8.1 Update 1 installation. I'm not in a position to do this myself until next week.

Regards,

Misty

[Wonko the Sane]

I did a quick retest this morning with a standard (unmodified) Winre.wim taken from a Windows 8.1 Update 1 (Enterprise Evaluation) installation.
....

I replaced the Windows 8.1u1 bootmgr with bootmgr from Windows 8.1 - it still booted.

I replaced the Windows 8.1 bootmgr with bootmgr from Windows 7 (SP1) - it still booted.

Could be my earlier tests using a wimbooted MistyPE (based on boot.wim from a Windows 8.1u1 iso) were not done correctly - I'll retest this after the weekend. Could alternatively be due to the different sources used (Winre.wim as apposed to boot.wim) or other factors.
...

 

Very good news

 

It is possible that *somehow* the MistyPE was missing *some* component that is needed (let's say for the sake of it - and as a guess - the FLTMGR).

 

While you have this "test install" handy, can you go a couple steps down the ladder?

I mean test Windows 7 (no SP1) BOOTMGR and also a Vista one (possibly a SP0 one, jumping over the Vista SP's)?

 

Can you also test with an earlier Winload.exe?

 

 

Wonko

[misty]

While you have this "test install" handy, can you go a couple steps down the ladder?
I mean test Windows 7 (no SP1) BOOTMGR and also a Vista one (possibly a SP0 one, jumping over the Vista SP's)?
 
Can you also test with an earlier Winload.exe?

Erm - slight problem - my test install has already been overwritten when reinstalling the OS I'm currently writing on - oops!!!

Will only take approximately 5 minutes to reinstall the test system - and a while longer to locate the relevant bootmgr and winload files - only issue is my one year old has woken up and it's very difficult to get her to stop slapping my keyboard!!!

If the little blighter agrees to another nap later then I'll give it a go - time is very limited as I'm away for the weekend to celebrate my wedding anniversary and need to pack, etc. If I don't get the time before I go then I'll have a play when I get back.

[Wonko the Sane]

No deadline/time limit , and even if there was one, it would be expressed in Latin Mediterranean time with the new MTM or JAGT correction factor and the binary friendly 32 days/month convention :

http://www.msfn.org/...itex/?p=1085137

 

Wonko

[misty]

...It is possible that *somehow* the MistyPE was missing *some* component that is needed (let's say for the sake of it - and as a guess - the FLTMGR).

I doubt it was due to FLTMGR as it initially wimbooted when using the Windows 8.1u1 bootmgr - it was only when I changed bootmgr that it failed to boot. This was some time ago, using a different version of wimlib and a different core - with other variables (programs and settings, etc) also introduced by MistyPE - or perhaps I was just tired and made a mistake.

[misty]

@Wonko
Just tested bootmgr from Windows 7 (gold) and Windows Vista (gold) sources - the OS booted successfully with both versions - remember this is a WIMBoot of a Winre 5.1 (obtained from a Windows 8.1 Update 1 Enterprise Evaluation install).

I also tried replacing the Windows 8.1u1 winload.exe with a Windows 8.1 version - it failed to boot. I didn't see any error messages displayed - no boot splash screen - just a black screen - no hard disk activity according to the pretty little LED that usually flashes when the HDD is being accessed.

As this appeared identical to my results when attempting to wimboot WinPE 5.0 (with WoF driver added via simonking's tool and using bootmgr from Windows 8.1u1) I thought I'd have another go with wimbooting this same WinPE 5.0 - this time using winload.exe from update 1 sources. I didn't boot, but it did get further than before - this time the boot splash screen was displayed (the one with the blue Windows logo and a circle of dots) before an error message was displayed...

Your PC ran into a problem & needs to restart....

...PROCESS1_INITIALIZATION_FAILED

Any thoughts?

Regards,

Misty

[simonking]

Excellent research, Misty - is that to say the boot manager of OS's all the way down to NT 6.0 successfully boot a Windows 8.1 based WinRE image? Still a long ways to go to actually booting those older OS's with WIMBoot, but one variable seems to have been eliminated. Great job!

 

BTW I've seen that error many times during my testing. Generally it means something really bad happened, I guess, to state the obvious. In my case I most frequently encountered it when the drive I was trying to boot off of became Bitlocked somehow. I believe this error in general means that the core OS process could not start; typically due to disk read errors such as a locked drive.

[Wonko the Sane]

@Misty

Not much to say.

 

From the results of the experiments on the WinRE.wim seemingly the *whatever* "key component" (or setting/parameter/etc.) is NOT to be inserted in point a. (between 2 and 3), which is "good news" , but rather in point b. (between 3 and 4) which is "bad news" or at least "much worse news" than if it was to be inserted in point c. ).

 

I would still be not-so-negative about the experiment with the WinPE 5.0, as it is still possible that *something else* (running or started/non started drivers or *whatever* Registry settings) are simply a-missing (or configured differently) or that some kind (it would not be the first time, particularly with Windows 7 and 8 based setups) of timing issues happened.

 

It is also entirely possible that (say) Winload.exe checks version of ntoskrnl.exe and throws a fit finding that it is not the expected version.

 

When doing this kind of "Frankestein builds" a lot of patience and perseverance is traditionally needed....

 

Is the "PROCESS1_INITIALIZATION_FAILED" all you have as error (or do you have an error code or STOP ERROR)?

I mean:

http://msdn.microsof...2(v=vs.85).aspx

 

 

Wonko

[misty]

I don't recall any stop error being displayed. Will test again after the weekend as I'm out of time today.

One last comment - just wimbooted a full Windows 8.1 Update 1 using Windows 7 bootmgr. My first attempt failed - changed the BCD store and it worked! The BCD store was the same one I used for Winre and was created using the following batch -

@echo off
setlocal

:: Do not use spaces in paths and do 
:: NOT wrap in quotes

set BCDEDIT=%SYSTEMROOT%\system32\bcdedit.exe
set BCDSTORE=%~dp0BCD

ECHO Creating new BCD Store...
ECHO.
%BCDEDIT% /createstore %BCDSTORE% 

ECHO Creating {bootmgr} entry...
ECHO.
%BCDEDIT% /store %BCDSTORE% /create {bootmgr}
%BCDEDIT% /store %BCDSTORE% /set {bootmgr} description "Boot Manager"
%BCDEDIT% /store %BCDSTORE% /set {bootmgr} device boot
%BCDEDIT% /store %BCDSTORE% /set {bootmgr} timeout 20

ECHO Creating WinPE entry...
ECHO.
for /f "tokens=2 delims={}" %%g in ('%BCDEDIT% /store %BCDSTORE% /create /application osloader') do set GUID={%%g}

%BCDEDIT% /store %BCDSTORE% /set %GUID% path \Windows\system32\winload.exe
%BCDEDIT% /store %BCDSTORE% /set %GUID% systemroot \Windows
%BCDEDIT% /store %BCDSTORE% /set %GUID% detecthal Yes
%BCDEDIT% /store %BCDSTORE% /set %GUID% winpe Yes
%BCDEDIT% /store %BCDSTORE% /set %GUID% osdevice partition=D:
%BCDEDIT% /store %BCDSTORE% /set %GUID% device partition=D:
%BCDEDIT% /store %BCDSTORE% /set %GUID% description "WoF Test"
%BCDEDIT% /store %BCDSTORE% /displayorder %GUID% /addlast

echo.
echo.
echo.
pause
:_end
endlocal