Jump to content











Photo
* * * * * 4 votes

wimlib, with ImageX implementation

wim imagex winpe boot

  • Please log in to reply
365 replies to this topic

#76 coder

coder

    Member

  • Members
  • 63 posts
  •  
    United States

Posted 14 August 2013 - 10:56 PM

@wonko,

you might get paid for posting and drifting here but I do not...

 

Do you think I have the time to properly criticize when someone i.e.

 

1) recommends n hard/soft links (like busybox) for a Windows command line application??? :buehehe:
2) recommends the creation of n sub-apps in order to save n sub-commands??? :crazyrocker:
3) says is coding a GUI frontend in Delphi today (8/2013)??? :drunk:

Sorry man I just do not have that time....

 

 

Syncronicity’s work is good, hopefully he’s not going to pay much attention to the last posts


Edited by coder, 14 August 2013 - 11:04 PM.


#77 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15108 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 14 August 2013 - 11:52 PM

@wonko,
you might get paid for posting and drifting here but I do not...

Sure, evidently my contract with the board owner has better economical return than yours :smiling9:, though I may dare to say that we are payed proportionally to the quality of our contributions  :whistling:
 

Do you think I have the time to properly criticize when someone i.e.

That's mainly your issue, if you now state how you don't have the time to properly criticize, you are simply confirming that you quickly (hence gratuitiously and/our lousily) criticized just for the sake of it, i.e. without providing any "productive" contribution to the board.

When quickly reviewing your past posts history, I cannot but notice that, set apart similar "gratuitious" and "quick" comments/critiques, and once excluded the postings about how good a specific software tool - which BTW I seemingly suggested to you at the time - very little remains in terms of "productive" contributions.

erwan.l, has instead a tradition of various, often VERY relevant contributions to the board :worship:, with nice tools like clonedisk and the offline registry tool, various pxe and ipxe related tools, and lately the Quick PE 4 thingy.

In order to gain the kind of respect needed to be actually listened to when you attempt to denigrate in a few one-liners the ideas of a highly respected member of the community (and here specifically the use of Delphi or other not-latest-programming language) you might want to show us/produce/provide some of your own work, so that - when you have not time to provide reasons/background to your critics - we could have an actual idea of your capabilities as programmer and consequently see your comments under this new light . (i.e. maybe coming from an highly respectable or authoritative source :dubbio:

 

:cheers:
Wonko



#78 synchronicity

synchronicity

    Frequent Member

  • Advanced user
  • 165 posts
  •  
    United States

Posted 21 August 2013 - 04:28 AM

Hi,

 

I'm looking to release wimlib 1.5.0 and have posted a beta version (x86 Windows binaries are at http://sourceforge.n...in.zip/download).  I'll not replicate all the additions to the NEWS file here, but this update adds support for pipable and delta WIMs and has various other improvements.  If anyone wants to test the beta version and make suggestions it would be great; but either way I'll do some more testing and reviewing when I have a chance and then make the official 1.5.0 release.



#79 coder

coder

    Member

  • Members
  • 63 posts
  •  
    United States

Posted 22 August 2013 - 05:13 PM

@wonko,

well, if you get paid by line everything is clear now :cheerleader:



#80 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 2794 posts
  • Location:Nantes - France
  •  
    France

Posted 22 August 2013 - 05:30 PM

@coder,

for my understanding and modest experience, which part of 'gratuitious' did you not get?  :dubbio:



#81 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 2794 posts
  • Location:Nantes - France
  •  
    France

Posted 22 August 2013 - 05:43 PM

Hi,

 

I'm looking to release wimlib 1.5.0 and have posted a beta version (x86 Windows binaries are at http://sourceforge.n...in.zip/download).  I'll not replicate all the additions to the NEWS file here, but this update adds support for pipable and delta WIMs and has various other improvements.  If anyone wants to test the beta version and make suggestions it would be great; but either way I'll do some more testing and reviewing when I have a chance and then make the official 1.5.0 release.

 

Hi Synchronicity,

 

I am currently testing 1.5 : i like the batch files already :)

 

Thanks !

Erwan



#82 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15108 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 August 2013 - 12:58 PM

If I may, cmd files (batches) should have .cmd (and not .bat) file extension.

Still IMHO, the batches - as they are - are of little use, in the sense that they allow the user to avoid ONLY the trivial part of the command, and don't do *anything* about the parameters.

I mean, right now the generic syntax of the tool is:
wimlib-imagex <put here any among 12 intuitive commands> SOURCE WIMFILE [IMAGE_NAME [IMAGE_DESCRIPTION]] [<put here any among a zillion of non-trivial options>]

 

the batches only take care of the part in green (which is not IMHO a problem), and do nothing about the red part.

 

I'll see if I can put together an example of what I had in my mind :) :unsure:.

:cheers:
Wonko

 

P.S.: Attached a couple "tentative" ideas, please have a look at them and let me know if you think that *something* like those could be of use or not

Attached Files



#83 synchronicity

synchronicity

    Frequent Member

  • Advanced user
  • 165 posts
  •  
    United States

Posted 25 August 2013 - 08:02 PM

Hi, thanks for testing the new version!

 

wlihelper.cmd seemed to be redundant with wimlib-imagex itself, seeing as the intention appears to be to wrap around every command.  With regards to the messages about files not existing or not being the proper type: well, wimlib-imagex already is supposed to do this, and any places where the messages are not sufficient can be improved.

 

wmkboot.cmd makes more sense seeing as manipulating the bootable index with `wimlib-imagex info' never made 100% sense in the first place, although this is what Microsoft's ImageX does.  Perhaps there could be an alternate command such as `wimlib-imagex mkboot' that makes this easier.  However, there are other properties of a WIM file that can be modified besides the boot index, such as the presence of an integrity table and the read-only flag, and I feel there shouldn't be a separate command for each.  (Edit:  also `wimlib-imagex info' can still be improved, for example making it list the available images in cases where an image was not specified but one was needed.)


Edited by synchronicity, 25 August 2013 - 08:07 PM.


#84 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15108 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 August 2013 - 08:47 AM

Yes, the idea behind the first is/was to help people making an action (proper command line) interactively (of course it is just a sketch).

If you prefer is - or is intended to become - a way to prevent people from using incorrect syntax and attempt to "guide" them to make a proper command. 

 

The only "good" thing it adds is the piping to "more" so that the help screen doesn't scroll away (this could be a "native" addition to the actual .exe), I know from experience how most "common" users will have difficulties with this, which is typically what the Author :worship: and "power users" know how to deal with, but rest assured, rarely the "average user".

 

The idea behind the second is something else.

I use a known Linux example to explain this.

Say 95% of people using Linux never used tar if not to UNtar something.

I.e. the command line tar -xvf <something> is used in a ratio of approximately 12747:1 :w00t:

It makes a lot of sense to make an alias "untar" to "tar -xvf".

The common action a normal user may want to do to a .wIm file regarding "info" could be to make one image bootable and know the names/description of the images inside it (most of the other info is useful, but not "commonly needed").

Another thing that may have some use (though right now I have no idea of what it could be) is the GUID.

IMHO noone normally would want to know:

  • Path
  • Compression type
  • Part Number
  • Size 
  • Integrity info
  • Relative path junction
  • Pipable
  • Directory count
  • File count
  • Total Bytes
  • Hard Link bytes
  • Creation time
  • Last modification times

:cheers:

Wonko



#85 Vortex

Vortex

    Frequent Member

  • Advanced user
  • 239 posts

Posted 29 August 2013 - 06:51 AM

Hi synchronicity,

 

Thanks for version 1.5 beta. My preliminary tests on Windows PE 1.x are OK. wimlib-imagex captures successfully my XP installation and restores the backup without problem. The NTFS permissions are restored correctly.



#86 synchronicity

synchronicity

    Frequent Member

  • Advanced user
  • 165 posts
  •  
    United States

Posted 01 September 2013 - 10:49 PM

Hi,

 

I've made the official 1.5.0 release of wimlib.  As usual it can be downloaded from https://sourceforge....ts/wimlib/files.

 

With regards to Wonko's post above:  I opted not to add auto-paging support for this release as it's difficult to get it to work consistently and in a configurable way on all supported platforms, mainly due to Microsoft's broken implementation of popen().  Also, the "wimlib-imagex" front-end isn't really intended to be used interactively by people without a basic knowledge of using the command line.  In terms of the information displayed by "wimlib-imagex info", I see the use in a stripped-down listing showing only image indices and names, but I feel that different users would find different information useful, and it's best to display all the information by default, unless it were to get the point where the amount of information displayed is overwhelming.



#87 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15108 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 05 September 2013 - 03:09 PM

Very early report (and possibly a false alarm :w00t: :ph34r:), but it seems like the Windows 8.1 format has been changed :frusty: (possibly in minor things :unsure:):

 http://www.msfn.org/...ions/?p=1048758

and that the "normal" 7 SP1 DISM is not compatible/working on it.

 

If anyone is testing the stupid Windows 8.1 and could check if wimlib works with those images or not and report, I am sure it would be useful.

 

:cheers:

Wonko 



#88 Tripredacus

Tripredacus

    Frequent Member

  • Expert
  • 234 posts
  • Interests:K-Mart-ian Legend
  •  
    United States

Posted 05 September 2013 - 03:56 PM

I no longer have the error in my console, but using DISM to inject a driver to Windows 8.1 Pro x64 image generated a 0xC0000135 message. It was not right away, it would happen maybe 10 minutes after running the /ADD-DRIVER command. I've been using this version of DISM with Windows 8.0 with no problems. It is able to capture and deploy Windows 8.1 without a problem, just does not let me add drivers. I haven't tried adding packages since there are no updates available for Windows 8.1 yet. :)



#89 synchronicity

synchronicity

    Frequent Member

  • Advanced user
  • 165 posts
  •  
    United States

Posted 05 September 2013 - 05:04 PM

I happened to download the Windows 8.1 preview a few days ago and the WIM format had no apparent changes.  In theory, Microsoft would increment the file format version number if they updated it.  However, from experience, even without a version number change they are prone to introduce bugs that require workarounds.  Definitely post back here if any specific incompatibilities are found.



#90 Tripredacus

Tripredacus

    Frequent Member

  • Expert
  • 234 posts
  • Interests:K-Mart-ian Legend
  •  
    United States

Posted 05 September 2013 - 05:57 PM

I never did anything with Preview versions myself and be aware that I do not use your tool personally.



#91 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 2794 posts
  • Location:Nantes - France
  •  
    France

Posted 05 September 2013 - 07:15 PM

Tested wimlib 1.5.0 with QuickPE and a windows 8.1 preview boot.wim and it worked fine.

 

 

Tested wimlib from within windows 8.1 and it worked fine.

 

Regards,

Erwan



#92 Tripredacus

Tripredacus

    Frequent Member

  • Expert
  • 234 posts
  • Interests:K-Mart-ian Legend
  •  
    United States

Posted 05 September 2013 - 08:36 PM

I am using Windows 8.1 RTM not Preview edition. :ph34r:



#93 misty

misty

    Gold Member

  • Developer
  • 1033 posts
  •  
    United Kingdom

Posted 13 September 2013 - 01:37 PM

@synchronicity
This is an incredible tool. Thank you!

The only issue I have had so far has been reading information from some older (unmodified) WinPE's - versions 2.0 and 2.1. These were either built or extracted from the respective WAIK's.

I suspect this is due more to the source files than wimlib. Interestingly wimlib had no problems obtaining image information from the boot.wim files in Windows Vista\Vista SP1 source DVD's - which share the same codebase as the boot.wim files that wouldn't work.

Regards,

Misty

#94 synchronicity

synchronicity

    Frequent Member

  • Advanced user
  • 165 posts
  •  
    United States

Posted 13 September 2013 - 03:22 PM

Hi,

 

I got ahold of the WAIK v2.1 and examined the WinPE WIMs.  They had a quirk that needed to be handled, which I've now patched.  But, new Windows binaries won't be released until v1.5.1 (whenever that is) unless someone really needs this fixed.

 

The 1.5.0 version will fail to open these WIMs with wimlib-imagex printing:

 

ERROR: Exiting with error code 20:
       An entry in the WIM's lookup table is invalid.

If there's in fact another problem as well let me know.



#95 misty

misty

    Gold Member

  • Developer
  • 1033 posts
  •  
    United Kingdom

Posted 13 September 2013 - 03:30 PM

@synchronicity

Excellent news. And incredibly fast work - it would have taken me longer just to download the WAIK than you took to resolve the problem and respond!

 

Personally I'm not in any rush for this fix and will await the release of the next version to see what other improvements you can make on what is already an excellent tool.

 

Thanks.

 

Regards,

 

Misty



#96 synchronicity

synchronicity

    Frequent Member

  • Advanced user
  • 165 posts
  •  
    United States

Posted 11 October 2013 - 05:10 PM

I've released wimlib v1.5.1 including the above-mentioned fix for the WAIK v2.1 WIMs.  Otherwise only a few minor changes.


  • misty likes this

#97 misty

misty

    Gold Member

  • Developer
  • 1033 posts
  •  
    United Kingdom

Posted 11 October 2013 - 08:17 PM

@synchronicity
Thanks. I'll check out the new release later. I'm currently working on a new project - wimlib-imagex is fundamental to the build process (as it is in MistyPE) so thank you very much for this excellent program.
 

:cheers:

 

Best wishes

 

Misty



#98 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 2794 posts
  • Location:Nantes - France
  •  
    France

Posted 11 October 2013 - 08:56 PM

@synchronicity

Tested in my project QuickPE : works like a charm :)

 

Thanks for this gem !

 

/Erwan



#99 synchronicity

synchronicity

    Frequent Member

  • Advanced user
  • 165 posts
  •  
    United States

Posted 18 November 2013 - 12:57 AM

wimlib v1.5.2 has been released.  It contains one experimental feature as well as several bug-fixes:

  1. Added a new experimental LZX compressor which can be enabled by passing '--compress-slow' to `wimlib-imagex capture' or `wimlib-imagex optimize'. (The latter is only applicable if the WIM is already LZX-compressed and the '--recompress' option is also given.)  The experimental compressor is much slower but compresses the data slightly more --- currently usually to within a fraction of a percent of the results from WIMGAPI/ImageX.
  2. A workaround has been added for compatibility with versions of WinPE that interpret alternate data stream entries in the boot WIM incorrectly.
  3. An alignment bug that caused a crash in the LZX decompressor on some builds was fixed.
  4. wimlib now attempts to clear the WIM_HDR_FLAG_WRITE_IN_PROGRESS flag in the WIM header when restoring the previous state of a WIM it failed to successfully update.
  5. Added a workaround to avoid an access denied error on Windows when replacing a WIM file that another process has opened.
In hindsight, the problem worked around by (2) was likely the same problem noted in post #61 of this thread.
 
I tried again to improve wimlib's LZX compressor, which is fairly fast, but has generally been compressing several percent worse than Microsoft's implementation.  Using two algorithms derived from 7-Zip's DEFLATE encoder I was able to get the compression ratio almost the same as Microsoft's implementation, typically a fraction of a percent better or worse, depending on the data.  However, it's fairly slow so I didn't enable it by default.  But if you have LZX-compressed WIMs compressed entirely by wimlib and are not in a hurry, they can be recompressed with `wimoptimize WIMFILE --recompress --compress-slow' to potentially create a 1-5% smaller WIM file.  (Perhaps make a backup just in case --- this is still an experimental feature!)

Edited by synchronicity, 18 November 2013 - 12:57 AM.


#100 ilko

ilko

    Silver Member

  • Advanced user
  • 500 posts
  •  
    Bulgaria

Posted 18 November 2013 - 06:58 AM

Thanks for the update.

 

A few test on Win 7 x64 SP1- Copy cloud sharing agent enabled for the working folder- no more problems with renaming the temp files, thanks. No issues with the added files with ADS either.

 

On few years old laptop, 4GB RAM, Core2Duo P8800 "--recompress --compress-slow" took 6:26 minutes with 100% CPU usage, one might have to add the "--threads" option if needs multitasking while optimizing.

2013/11/17 22:31:22- Optimizing boot.wim
2013/11/17 22:37:48- "D:\DropBox\Copy\Win Setup\TempWST\sources\boot.wim" original size: 173489 KiB
Writing LZX-compressed data using 2 threads
0 MiB of 467 MiB (uncompressed) written (0% done)
...
239 MiB of 467 MiB (uncompressed) written (51% done)
...
467 MiB of 467 MiB (uncompressed) written (100% done)
"D:\DropBox\Copy\Win Setup\TempWST\sources\boot.wim" optimized size: 171238 KiB
Space saved: 2251 KiB
2013/11/17 22:37:48-





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users