Jump to content











Photo

Updated WinRE Extractor

winre winpe windows 8 wimboot

  • Please log in to reply
60 replies to this topic

#1 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 10 February 2015 - 04:33 PM

Hey Folks,

 

Here's an updated version of my free WinRE extractor:

 

http://www.zipmagic....magicwinre.7zip

 

I have included both 32 bit and 64 bit versions of the tool in this iteration.

 

Enjoy folks, and I hope everyone's had a swell start to 2015 at reboot.pro!

 

- Simon King.



#2 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 10 February 2015 - 08:04 PM

There's a new command line parameter in this new build:

 

ZIPmagic WinRE Update Utility (www.zipmagic.co)
Copyright© 2014-2015 Simon King. All rights reserved.

     Usage: zipmagicwinre.exe <filename> [-x] [-u [-d]]
<filename>: Path to WIM file, deleted if found
         x: Find and extract WIM from WinRE partition
         u: Update WIM (image index 1) with WIMGAPI libraries and WoF driver
         d: Delete winpeshl.ini from WIM (image index 1), requires -u

 

The -d option does something seemingly pointless; it removes the file \windows\system32\winpeshl.ini from the extracted WIM. Why? Its kind of a long story:

 

There have recently been some reports that DoubleSpace actually causes a net loss of storage on some 16 GB WIMBoot tablets; these reports are easily understood considering the WIMBoot recovery partition on these systems are separate, and DoubleSpace does not remove these partitions after compressing the disk. As such, I am adding support to DoubleSpace to recognize and remove such partitions after processing is complete; including extending the main partition to reclaim that space.

 

Now, to test, I got my hands on such a tablet. I use Macrium Reflect (Free Edition) to image and restore the tablets I work on to their default factory state, which is vital given the kind of work I do. [Unfortunately the recovery media created on pre-WIMBoot'ed tablets does not work accurately to restore them to their base state; Microsoft's recovery software assumes (mistakenly) that the WIMBoot partition has been left untouched during a recovery operation. See the paragraph above for an explanation of why this recovery software wouldn't work in my case.]

 

Regretfully, the micro sized, 16 GB, WIMBoot'ing tablet I am using does not come with a keyboard dock (unlike the, for example, Asus T100 users who reported this issue). After booting into Macrium Reflect Free Edition from my USB thumb drive using an OTG cable, I found out that touch input does not work either. Also, connecting a USB keyboard or mouse through an OTG cable did not work as well - the devices were simply not working. This resulted in the nasty situation of being in an WinPE desktop without any opportunity to interact with the device!

 

I knew from testing DoubleSpace on the tablet that it was responsive to user input. I also noticed that Macrium Reflect may consume a third-party WIM when creating a recovery USB drive. Unfortunately, their build processes always failed when I tried to use the WIM extracted using my tool in this post. In analyzing their logs, I noticed the process was choking apparently because the recovery WIM file already had \windows\system32\winpeshl.ini inside it (tested with their most recent build at the time of this writing).

 

We've now arrived at the happy-ending part of the story. The new "-d" switch simply removes this file, allowing Macrium Reflect to successfully build the recovery WIM. After which, booting into Macrium Reflect Free Edition from the USB thumb drive results in working touch input. Yaaay!

 

Now I can get back to improving DoubleSpace for this significant scenario. All this work just to delete a partition when users could just as well delete it themselves, you ask? Well, DoubleSpace is meant to be a single-click tool, so yes, it has been time well spent :)

 

And I hope any users in my position, unable to control their tablets after booting into a custom WIM, would also benefit from this enhancement - saving the hassle of manually having to configure their tablet drivers for their custom WIMs.

 

Enjoy, rebooters!



#3 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 11:34 AM

Hi Simon,

 

Just wanted to report the below error :

ONtumDe.png .

 

Seems your proggie is trying to access both my network drives and internal sd reader (which shows 4 empty physical drives).

 

At the end of the process, not winre could be found (whereas I do have one).

 

Regards,

Erwan



#4 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 11 February 2015 - 02:04 PM

Thank you very much for the bug report, Erwan!

 

What is the exact path to your WinRE location?

 

This will help me identify and fix the bug.



#5 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 02:33 PM

My winre.wim is in c:\Recovery\07dd7d82-eba6-11e0-a7a2-aaf583753449 .

I use this path (c:\recovery) from within my QuickPE batch (in my signature).

 

Winre Extractor should skip network drives and empty drives (my card reader shows 4 empty physical drives).



#6 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 11 February 2015 - 03:17 PM

Did you manually create this path? It would appear standard Windows installations do not follow this path.

 

Currently the extractor follows a heuristic of standard OEM paths, as you can see it simply pings each path in succession.

 

If there is a programmatic approach, I would love to hear about it.

 

Please note that I do not call considering reagentc a programmatic approach, as I find command line parsing to be unreliable.

 

One would hope there is a more direct access to the reagentc queried location(s) in this regard.



#7 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 03:42 PM

No clue where this path comes from and i cannot remember how I found it, but it was there :)

Google probably led me there.



#8 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 11 February 2015 - 04:15 PM

I guess, the better question for me to ask is - how exactly did you install Windows?



#9 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 11 February 2015 - 04:24 PM

Would you also mind posting the contents of this file:

 

C:\Windows\System32\Recovery\ReAgent.xml

 

? It appears that this file may be a fairly reliable indicator of the WinRE location.

 

To x-ref the information, we already have the GUIDs of your partitions available from your prior posting; so that is great :)



#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 11 February 2015 - 04:47 PM

For what is worth, card readers are known trouble makers, solution/workaround (for batch) here:

http://www.msfn.org/...-gui/?p=1005829

 

 

:duff:

Wonko



#11 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 05:34 PM

I guess, the better question for me to ask is - how exactly did you install Windows?

 

windows 7 iso applied to a usb stick with rufus



#12 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 05:35 PM

Would you also mind posting the contents of this file:

 

C:\Windows\System32\Recovery\ReAgent.xml

 

? It appears that this file may be a fairly reliable indicator of the WinRE location.

 

To x-ref the information, we already have the GUIDs of your partitions available from your prior posting; so that is great :)

<?xml version='1.0' encoding='utf-8'?>
<WindowsRE version="1.0">
    <WinreBCD id="{f06d08d1-3582-11e1-a7ce-00247e0d105f}"/>
    <WinreLocation path="\Recovery\f06d08d1-3582-11e1-a7ce-00247e0d105f" id="3611991224" offset="32256" guid="{00000000-0000-0000-0000-000000000000}"/>
    <ImageLocation path="" id="0" offset="0" guid="{00000000-0000-0000-0000-000000000000}"/>
    <OsInstallLocation path="" id="0" offset="0" guid="{00000000-0000-0000-0000-000000000000}"/>
    <InstallState state="1"/>
    <OsInstallAvailable state="0"/>
    <WinREStaged state="0"/>
    <OperationParam path=""/>
    <OsBuildVersion path="7600.16385.x86fre.win7_rtm.090713-1255"/>
    <OemTool state="0"/>
    <BootKey state="0"/>
    <IsServer state="0"/>
    <ScheduledOperation state="4"/>
</WindowsRE>

my recovery folder :

 

 Répertoire de c:\Recovery
 
11/11/2012  20:09    <REP>          .
11/11/2012  20:09    <REP>          ..
01/10/2011  06:19    <REP>          07dd7d82-eba6-11e0-a7a2-aaf583753449
11/11/2012  20:09    <REP>          f06d08d1-3582-11e1-a7ce-00247e0d105f
 
both folders contain a winre.wim


#13 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 11 February 2015 - 05:39 PM

windows 7 iso applied to a usb stick with rufus

 

Ah that might explain it, this tool is intended for Windows 8.x, primarily Windows 8.1 and higher (for WIMBoot).

 

I don't know if rufus changes standard paths either, the GUID ID for your recovery partition is all 0's, which doesn't make a lot of sense as it points to an invalid ID.

 

That might be a nail in the coffin for the idea of using reagentc to locate WinRE.



#14 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 05:57 PM

Ah that might explain it, this tool is intended for Windows 8.x, primarily Windows 8.1 and higher (for WIMBoot).

 

I don't know if rufus changes standard paths either, the GUID ID for your recovery partition is all 0's, which doesn't make a lot of sense as it points to an invalid ID.

 

That might be a nail in the coffin for the idea of using reagentc to locate WinRE.

 

My bad, apologies for the waste of time.

I missed the fact that Winre Extractor was targeting Win8.x.

 

However, second point : I would still exclude network drives and empty drives (size=0) from your scan thus avoiding error like in the 1st post.



#15 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 11 February 2015 - 06:03 PM

Does it work normally if you just hit continue?

 

The code in this tool is primarily consumed by DoubleSpace at boot time within a WinRE instance, so network drives and empty drives would be automatically exempt.

 

It also looks like DoubleSpace does call SetErrorMode to avoid being tripped by errors like these, but this command line variant does not - that would also completely avoid the issue. I can add it in if there's a want :)



#16 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 06:17 PM

Does it work normally if you just hit continue?

 

The code in this tool is primarily consumed by DoubleSpace at boot time within a WinRE instance, so network drives and empty drives would be automatically exempt.

 

It also looks like DoubleSpace does call SetErrorMode to avoid being tripped by errors like these, but this command line variant does not - that would also completely avoid the issue. I can add it in if there's a want :)

 

The tool goes to completion, but will displayed as manay dialogbox (errors) as i have network and empty drives.

If you have a set of hardcoded path, you would add c:\recovery?



#17 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 11 February 2015 - 06:21 PM

Understood. Would be definitively cured with a process wide SetErrorMode call.

 

c:\recovery is already in the list. However, on Windows 8.x, the subfolders are not GUIDs; instead the path seems fixed as:

 

c:\recovery\windowsre

 

Which is why in your case the detection failed.



#18 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 February 2015 - 06:25 PM

Understood. Would be definitively cured with a process wide SetErrorMode call.

 

c:\recovery is already in the list. However, on Windows 8.x, the subfolders are not GUIDs; instead the path seems fixed as:

 

c:\recovery\windowsre

 

Which is why in your case the detection failed.

 

Ok got it.

You may want to discard this windows 7 specific then and maybe warn the user that the tool is to be run on win8.x only (although it would be a nifty tool on win7 as well)

 

Thinking out loud, you  might be able to take advantage of the windows to go (win 8.x) folder which (to be checked) may also contain a winre.



#19 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 13 February 2015 - 12:05 AM

Here's an interesting problem that has come up next:

 

It is not possible to run diskpart.exe from this extracted WinRE. diskpart.exe itself crashes with a nasty error, trying to read a null address.

 

I had coded DoubleSpace to use WMI initially; this didn't work because WMI is unavailable on WinRE. I needed to use WMI because only it seems to export APIs for deleting and extending partitions (which is the main point of the exercise on the 16 GB tablet).

 

I later re-coded it to use diskpart.exe, only to have it fail - possibly it too uses the same APIs I was trying to consume.

 

Using DISM to add WMI and storage packages to this WinRE will bloat the 'smaller than 20 MB' download of ZIPmagic back into the neighborhood of 500 MB - 3 GB (the size of the Windows Kit containing the packages). A very prohibitive size indeed!

 

The only other solution seems to be to reboot after processing inside WinRE and then use the RunOnce registry key under HKLM to re-start DoubleSpace to reclaim the now-unneeded legacy WIM boot image and partition. But I don't like this solution - if the user gets confused, for example, as to why an app is requesting elevation before (s)he has even logged in - and declines elevation - the space is lost.

 

Anyone got any clever ideas for a better solution?



#20 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 February 2015 - 12:39 AM

Which api are you looking for?

Rather than using an intermediate party ( diskpart or wmi), why not use win32 low level api?

#21 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 13 February 2015 - 10:06 AM

I am not aware of any documented API other than the WMI classes to delete or extend a disk volume. If you are aware of any IOCtl calls, for example, with documentation so they can actually be used; please help :) DoubleSpace's users will really appreciate it.



#22 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 February 2015 - 11:10 AM

I am not aware of any documented API other than the WMI classes to delete or extend a disk volume. If you are aware of any IOCtl calls, for example, with documentation so they can actually be used; please help :) DoubleSpace's users will really appreciate it.


I use low level api in clonedisk to perform these tasks.
I will document in a next post.

#23 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 February 2015 - 11:32 AM

I use the following IOCTL's, FSCTL_EXTEND_VOLUME and FSCTL_SHRINK_VOLUME, to extend or shring a volume.

 

I can share (delphi) code if needed.



#24 simonking

simonking

    Frequent Member

  • Advanced user
  • 236 posts
  •  
    Australia

Posted 13 February 2015 - 12:34 PM

That's exactly what I am looking for! And yes, Delphi code would be perfect, since DoubleSpace is all Delphi. Thank you Erwan!



#25 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 February 2015 - 04:53 PM

That's exactly what I am looking for! And yes, Delphi code would be perfect, since DoubleSpace is all Delphi. Thank you Erwan!

 

A dirty copy/paste from CloneDisk. If you need a working/complete example (binary+source), i'll put up something later.

function _EXTEND_VOLUME(drive:string;new_size:int64):boolean;
var
buf:pointer;
hDevice:thandle;
dwBytesReturned:dword;
ret:boolean;
begin
result:=false;

 hDevice := CreateFile( pchar(drive),GENERIC_WRITE or GENERIC_READ,
 FILE_SHARE_READ or FILE_SHARE_WRITE,nil, OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL, 0);

ret:=DeviceIoControl( hDevice,FSCTL_EXTEND_VOLUME,@new_size,sizeof(new_size),nil, 0,dwBytesReturned,nil );
result:=ret;
if ret=false
  then showmessage('FSCTL_EXTEND_VOLUME failed:'+SysErrorMessage(GetLastError))
  else
  begin
  dwBytesReturned := 0;
  ret:=DeviceIoControl(hDevice,FSCTL_DISMOUNT_VOLUME,nil,0,nil,0,dwBytesReturned,nil);

  dwBytesReturned := 0;
  getmem(buf,sizeof(_disk_geometry));
  ret:=DeviceIoControl(hDevice,IOCTL_DISK_UPDATE_DRIVE_SIZE,nil,0,buf,sizeof(_disk_geometry),dwBytesReturned,nil);
  if ret=false then showmessage('IOCTL_DISK_UPDATE_DRIVE_SIZE failed+'+SysErrorMessage(GetLastError));
  freemem(buf);

  end;

closehandle(hdevice);
end;

  • simonking likes this





Also tagged with one or more of these keywords: winre, winpe, windows 8, wimboot

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users