365 replies to this topic

[ilko]

Great job synchronicity, thanks.

 

I've tested optimize and update options on a couple of wim files, both worked fine. Add option is quite quick, actually instant with the tested small files added, which is quite nice.

 

A little suggestion if I may- is it possible update option to not need redirection from CMDFILE, omitting the need of CMDFILE for single commands?
Something like:

 

wimlib-imagex update boot.wim 2 --verbose add test.txt /Windows/test.txt

Or did I miss already implemented similar syntax?

[synchronicity]

A little suggestion if I may- is it possible add option to not need redirection from CMDFILE, omitting the need of CMDFILE for single commands?

 

I actually was considering doing this in exactly the way you suggested, but it would run into some problems with parsing the command.  In particular this would create confusion between the global options to `wimlib-imagex update' and the local options that may be passed to individual update commands.  Currently they are compatible, but they may not be in the future.  Also, since I don't require specifying the image number or name if the WIM only has 1 image, certain command lines would become ambiguous if an update command could be directly appended to the command line.

 

However, an alternative would be to have a special option to `wimlib-imagex update' like --command=STRING that reads a command directly from an argument to an option.   Then you would be able to do something like:

 

wimlib-imagex update boot.wim 2 --verbose --command="add test.txt /Windows/test.txt"

[ilko]

However, an alternative would be to have a special option to `wimlib-imagex update' like --command=STRING that reads a command directly from an argument to an option.   Then you would be able to do something like:

 

wimlib-imagex update boot.wim 2 --verbose --command="add test.txt /Windows/test.txt"

Perfect.

Might have to figure out how paths containing spaces would be handled, i.e.

wimlib-imagex update boot.wim 2 --verbose --command="add c:\documents and settings\test.txt \Windows\my test.txt"

[synchronicity]

Might have to figure out how paths containing spaces would be handled, i.e.

 

With the syntax I suggested above, you would have to include the quotes in the command string somehow, either by escaping them or by using a different kind of quote (the update commands currently accept either double or single quotes.  And I really hope that people aren't going to be wanting to add files that themselves have quotes in their names...).

[synchronicity]

Hi,

 

I am looking to make a 1.4.1 release and would appreciate any testing on what I've done so far:

 

https://downloads.so...86-BETA-bin.zip

 

This has the --command option for `wimlib-imagex update', case-insensitive WIM paths on Windows, progress message units depending on the total size, and a few other changes.

Edited by synchronicity, 21 May 2013 - 06:38 PM.

[ilko]

Great, thanks.

 

These commands run with no errors under XP, NTFS partition:

wimlib-imagex update boot.wim --verbose --rebuild --command="add 'C:\tmp\test space\test.txt' '\windows\my test.txt'"
wimlib-imagex update boot.wim --verbose --command="delete '\wIndows\my test.txt'"
wimlib-imagex update boot1.wim 2 --verbose --command="add 'C:\tmp\test space\test.txt' '\windows\my test.txt'"
wimlib-imagex update boot1.wim 2 --verbose --command="delete '\windows\my test.txt'"
wimlib-imagex.exe optimize boot1.wim
wimlib-imagex.exe optimize boot.wim
wimlib-imagex update boot.wim --verbose --rebuild --command="add 'C:\tmp\test space\tESt.txt' '\wiNDows\my test2.txt'"

 

This one fails, outer single quotes:

C:\tmp\test space>wimlib-imagex update boot1.wim 2 --verbose --command='add "C:\tmp\test space\test.txt" "\Windows\my test.txt"'
Usage:
wimlib-imagex update WIMFILE [IMAGE_NUM | IMAGE_NAME] [--check] [--rebuild]
                       [--threads=NUM_THREADS] [DEFAULT_ADD_OPTIONS]
                       [DEFAULT_DELETE_OPTIONS] [--command=STRING] [< CMDFILE]
See wimlib-imagex-update.pdf in the doc directory for more details.
C:\tmp\test space>

 

Added:

Escaping the double quotes also seems to work fine:

wimlib-imagex update boot.wim --verbose --rebuild --command="add \"C:\tmp\test space\test.txt\" \"\windows\my test3.txt\""

 

Outer single quotes fail as above. Does command option expect the string in double quotes or I am missing something?:

wimlib-imagex update boot.wim --verbose --rebuild --command='add \"C:\tmp\test space\test.txt\" \"\windows\my test4.txt\"'

[synchronicity]

Sounds good.  But I can't do anything about the outer single quotes because cmd.exe only recognizes double quotes.

[ilko]

It's perfectly fine now since there are couple of ways to use paths with spaces in them. 

 

Thanks again for this project, long-awaited.

[synchronicity]

I've now made the official version 1.4.1 release.  The NEWS is quite long so I won't post it here; mostly various bugfixes and minor improvements to wimlib-imagex and the library.

 

I posted 64-bit binaries for Windows as well as the 32-bit ones I've been posting, since it took like 30 extra seconds to do so and maybe someone wants them.

[erwan.l]

a quickie to report that wimlib is extremely handy to customise a winpe :

 

1-use MS WAIK to make my winpe :

 

copype amd64 C:\temp\winpe\amd64

 

2-then customize my wim file :

 

wimlib-imagex update winpe.wim --verbose --command="delete '\windows\system32\winpeshl.ini'"

 

wimlib-imagex.exe update winpe.wim --verbose --command="add 'c:\temp\winpe\winpeshl.ini' '\windows\system32\winpeshl.ini'"

 

wimlib-imagex update winpe.wim --verbose --command="delete '\windows\explorer++.exe'"

 

wimlib-imagex.exe update winpe.wim --verbose --command="add 'c:\temp\winpe\amd64\explorer++.exe' '\windows\explorer++.exe'"

 

3-content of my winpeshl.ini :

 

[LaunchApps]

%SYSTEMDRIVE%\Windows\system32\startnet.cmd

%SYSTEMDRIVE%\windows\explorer++.exe

 

and you got yourself a winpe starting explorer++

 

note : one could replace explorer++ by some batch launch a windows setup (from a pxe installation for example) or by bsexplorer...

 

/Erwan

[erwan.l]

Hi Synchronicity,

 

Reporting a possible bug (or is it a mistake on my side?)

 

When using the following command : wimlib-imagex.exe update boot.wim --verbose --command="add 'g:\quickpe4\pe_extra' '\tools'"

 

with the pe_extra folder containing 3 exe's, the last exe gets corrupted (not sure this is the right word) with the following content (26 bytes) :

[ZoneTransfer]ZoneId=3 .

 

Other 2 exe's are fine.

 

Any idea?

 

Regards,

Erwan

[Wonko the Sane]

 

 

Any idea?

 

That's an NTFS ADS (Alternate Data Stream):

http://code.google.c...outNTFS_Streams

 

Wonko

[synchronicity]

Hi,

 

It sounds like the contents of the file somehow got replaced with an alternate data stream.  I'm not quite sure how that could have happened, though, since I had tested that alternate data streams were working properly.  This was just on NTFS right?  Any chance you could send the resulting WIM file, or the output of `wimlib-imagex info --metadata WIMFILE'?

 

Coincidentally, I've actually rewritten a lot of the extraction code for the upcoming 1.5.0 release anyway, so it's possible that the bug no longer exists in the development version, but I'll make sure to do extra tests regarding alternate data streams on Windows.

 

Also, as long as I'm posting here, a semi-relevant question:  As was discussed earlier in this thread, a while back I ended up changing the name of the program from "imagex" to "wimlib-imagex", but there was some concern that the latter name is a little long and clunky, especially since you always have to type in an additional command anyway.  Furthermore, "ImageX" doesn't really mean anything anyway; it's just a name originally made up by Microsoft.  What I'm thinking is that the wimlib-imagex subcommands should actually be accessible simply as programs called "wimapply", "wimcapture", "wimappend", "wimupdate", ..., etc.  Does this make any sense, or is just silly to change the name a second time?  (Another advantage of separate commands is that it's a good setup for packaging a "wimtools" package for Linux, separate from the "wimlib" library.)

[erwan.l]

Hi Wonko, Synchronicity.

 

Thanks Wonko for the quick answer : nice catch, indeed, it leads to an ADS file.

Strange as I copied 3 files from 3 different locations into my folder and one of them end up being an ADS.

How can I spot the difference (the file being an ADS) from windows explorer?

 

Synchronicity :

 

Here attached the content of wimlib-imagex info --metadata boot.wim.

 

Boot.wim is there : http://erwan.labalec.fr/boot.wim (give 30mns to complete the upload).

 

About changing the name : i dont have much of an opinion there.

I know naming a tool can be challenging

Do I understand you would like to come up with several binaries? I am less enthusiasts there...

 

Regards,

Erwan

 

edit : solved my ADS "issue" with streams.exe from sysinternals Microsoft.

edit : once i deleted the streams, my wim file is all fine containing the "real" file.

Attached Files

•  meta.zip   692.57KB   1482 downloads

[Icecube]

Also, as long as I'm posting here, a semi-relevant question:  As was discussed earlier in this thread, a while back I ended up changing the name of the program from "imagex" to "wimlib-imagex", but there was some concern that the latter name is a little long and clunky, especially since you always have to type in an additional command anyway.  Furthermore, "ImageX" doesn't really mean anything anyway; it's just a name originally made up by Microsoft.  What I'm thinking is that the wimlib-imagex subcommands should actually be accessible simply as programs called "wimapply", "wimcapture", "wimappend", "wimupdate", ..., etc.  Does this make any sense, or is just silly to change the name a second time?  (Another advantage of separate commands is that it's a good setup for packaging a "wimtools" package for Linux, separate from the "wimlib" library.)

 

You can use the same approach as busybox.

Busybox makes symlinks to itself:

ln -s busybox ls

So running ls will start busybox and run the ls related code

# So:
$ ls
# Will do the same as:
$ busybox ls

So based on the basename of the program it determines which subfunction to run.

 

On linux you can make symlinks to wimlib-imagex:

ln -s wimlib-imagex wimapply
ln -s wimlib-imagex wimcapture
ln -s wimlib-imagex wimappend
ln -s wimlib-imagex wimupdate
...

On Windows you can make soft links with mklink (restrictions http://en.wikipedia....a_symbolic_link):

mklink wimapply.exe wimlib-imagex.exe
mklink wimcapture.exe wimlib-imagex.exe
mklink wimappend.exe wimlib-imagex.exe
mklink wimupdate.exe wimlib-imagex.exe
...

Using batch files might be more universal (doesn't need admin rights to create soft links, works also of FAT12/16/32 and works on Windows XP).

 

wimapply.cmd batch script:

%~dp0\wimlib-imagex.exe apply %*

For linux you can also work with shell scripts similar to the Windows batch file (although creating symlinks on Linux shouldn't be a problem, and the symlinks start wimlib-imagex directly instead of first invoking a shell):

 

wimapply shell script:

#!/bin/sh
${0%/*}/wimlib-imagex apply ${@}

[synchronicity]

Erwan:

 

I extracted the "tools" directory from the WIM you uploaded, and the files appeared to be extracted correctly, including QtWeb.exe (which had a Zone.Identifier named data stream).  I'm not sure what you mean by copying an ADS into a folder, since I don't think it's even possible to (directly) "rename" an ADS and turn it into a file.

 

'dir /R' can also display alternate data streams, although I highly doubt there's any way to show them in Explorer.  Microsoft's software is known not to support their own filesystem very well.

 

About the naming:  Actually, now I'm thinking more along the lines of having just 1 binary (wimlib-imagex), but it will support being invoked as, for example, "wimapply" as a shortcut for "wimlib-imagex apply".  This technique is often used on UNIX-like systems to allow a single binary to implement more than 1 program, and it requires hard links or symbolic links to be set up.  And I see that as I was typing this, IceCube recommended this same way, and also pointed out this could be done on Windows using batch scripts (necessary if a ZIP file is used as the method of distribution).  So, "wimlib-imagex" would stay as a single binary and the documentation would stay essentially the same, but there would be "shortcuts" for the commands.

[erwan.l]

Erwan:

 

I extracted the "tools" directory from the WIM you uploaded, and the files appeared to be extracted correctly, including QtWeb.exe (which had a Zone.Identifier named data stream).  I'm not sure what you mean by copying an ADS into a folder, since I don't think it's even possible to (directly) "rename" an ADS and turn it into a file.

 

'dir /R' can also display alternate data streams, although I highly doubt there's any way to show them in Explorer.  Microsoft's software is known not to support their own filesystem very well.

 

About the naming:  Actually, now I'm thinking more along the lines of having just 1 binary (wimlib-imagex), but it will support being invoked as, for example, "wimapply" as a shortcut for "wimlib-imagex apply".  This technique is often used on UNIX-like systems to allow a single binary to implement more than 1 program, and it requires hard links or symbolic links to be set up.  And I see that as I was typing this, IceCube recommended this same way, and also pointed out this could be done on Windows using batch scripts (necessary if a ZIP file is used as the method of distribution).  So, "wimlib-imagex" would stay as a single binary and the documentation would stay essentially the same, but there would be "shortcuts" for the commands.

 

 

About qtweb.exe in the tools folder, it should be a 6.5 meg executable.

But it appears my chrome download folder is stuffed with ADS... So issue is really on my system not with wimlib.

Once I delete the ADS, the wim file contains what I want it to contain.

Thanks thus for having looked at it.

 

About the hard/soft links to wimlib-imagex, it is actually a good idea. What about supplying a bunch of cmd batches for windows users in the zip file?

 

Regards,

Erwan

[coder]

About the hard/soft links to wimlib-imagex, it is actually a good idea. What about supplying a bunch of cmd batches for windows users in the zip file?

 

Regards,

Erwan

 

sometimes your design decision/opinions are really scary...

[Wonko the Sane]

About qtweb.exe in the tools folder, it should be a 6.5 meg executable.

But it appears my chrome download folder is stuffed with ADS... So issue is really on my system not with wimlib.

Once I delete the ADS, the wim file contains what I want it to contain.

 

The idea of using Opera didn't even cross your mind, didn't it ?

JFYI, Opera doesn't attach senseless Zone Indentifiers to downloaded files.

 

@Icecube

The idea of links seems to me a good one on Linux, where this approach is common.

Not so much on Windows, where a number of tools (particularly copy, backup and restore tools) are AFAICR not particularly "happy" about links.

 

@synchronicity

As said earlier, the name "as is" sucks big, and I personally welcome the idea of changing the name to something more "distinctive".

 

The idea of providing simple .cmd as helpers to use the tools, is IMHO a very nice one , as a lot of people have some sort of allergy to complex command lines, especially when to do the most common tasks you need to remember (and type) a number of switches/whatever.

 

Of course I am "partial" to simple .cmd files, because is more or less the only thing i understand/can manage , but they do - though less elegant than links - pass correctly an Occam's Razor test AND they will work *everywhere* AND NOT be limited to NTFS filesystems .

 

Casually, that is the approach that I have often taken (in a completely UNauthorized way ) to make otherwise excellent tools actually usable by "common users", here are two examples (JFYI - maybe they can provide you some form of inspiration):

http://reboot.pro/to...wimtoolbatches/

http://reboot.pro/to...elease-unwbzip/

If you want/need any assistance on this, just say so .

 

@Coder

Sometimes your contributions to the board appear gratuitious (meaning #3 ):

http://www.thefreedi....com/gratuitous

 

Wonko

[coder]

@Coder

Sometimes your contributions to the board appear gratuitious (meaning #3 ):

http://www.thefreedi....com/gratuitous

 

Wonko

 

your technical comments are OK;
the rest... well...

[erwan.l]

@coder,

Your comments do not bring any added value to this thread.

I suggest you stop it there for the sake of Synchronicity and his wimlib-imagex tool?

 

As for my  <quote>"sometimes your design decision/opinions are really scary..."</quote>, sorry to make you feel this way...   

And for sure, this is not to be debated there.

 

Regards,

Erwan

[coder]

@coder,

And for sure, this is not to be debated there.

 

Regards,

Erwan

 

Well…I consider this is the right place for my comment criticizing your suggested design criteria.

I think Synchronicity can really benefit from it.

[synchronicity]

Hi,

 

It's absolutely possible (and intended) to use wimlib-imagex in scripts, and in fact on UNIX-like systems it's actually distributed with a bash script called "mkwinpeimg" that simplifies the process of creating a customized Windows PE image (including automatically grabbing the needed files from the Windows DVD or WAIK).  I would certainly be open to including other useful scripts as well.  But the batch files like "wimapply" I was talking about wouldn't really be "scripts" per se, just 1 or 2 lines of batch as a shortcut to the corresponding wimlib-imagex command as a replacement for the hard links that will be installed on UNIX-like systems.

 

Also note that I've been trying to simplify the command-line syntax of wimlib-imagex a bit; for example, "wimlib-imagex capture" (or now "wimcapture") will default to LZX ("maximum") compression, since that's what's usually desired, and it also will treat integrity tables as persistent without having to constantly specify --check.  Furthermore it never has required images to be explicitly named, unlike Microsoft's ImageX.  So for example, to create a LZX compressed WIM, you can run simply:

wimcapture winpe winpe.wim

As opposed to Microsoft's ImageX where the same would be:

imagex /capture winpe winpe.wim "winpe" /compress maximum

Edit:  It also might be worthwhile to note that wimlib itself (the library) contains nearly 10 times as much code as wimlib-imagex, and wimlib-imagex is really just a command-line front-end to the library.  It would be possible for someone to implement their own front-end (perhaps even a graphical one) if desired.

 

Edit again:  The released version of wimlib-imagex still defaults to XPRESS compression; I meant that the next release will default to LZX.

Edited by synchronicity, 14 August 2013 - 03:38 PM.

[Wonko the Sane]

Well…I consider this is the right place for my comment criticizing your suggested design criteria.
I think Synchronicity can really benefit from it.

A quick peek on wikipedia may be needed to fully understand the difference between "good" and "lousy" criticism:
http://en.wikipedia....iticism#Quality
 

The main effect of lousy criticism is usually that, rather than clarifying things, it becomes disorienting or confusing to people. Therefore, lousy criticism is usually regarded as unhelpful, or as an unwanted distraction getting in the way of things. The only thing a lousy criticism achieves is to make it clear that somebody has an objection (although the objection is not well-taken).

 

 

Criticisms are usually well accepted if they are "good", whilst "lousy" ones, like merely commenting vaguely on an idea (and provide not any argument to support the critique and/or propose possible alternatives to it) are more suited to "polls" or whenever "popularity" is seeked as opposed to "brainstorming", "being propositive", etc.

 

Your apodictical and unjustified lack of appreciation (which represents a good example of lousy criticism) of erwan.l's idea has been duly recorded, you can move on alright.

 

Wonko

[erwan.l]

...

Edit:  It also might be worthwhile to note that wimlib itself (the library) contains nearly 10 times as much code as wimlib-imagex, and wimlib-imagex is really just a command-line front-end to the library.  It would be possible for someone to implement their own front-end (perhaps even a graphical one) if desired.

 

 

 

Hi Synchronicity,

 

I have started to develop a GUI frontend in delphi for your wimlib library.

Helas, the complexity (casting c types to delphi is a PITA) versus time made me put that project on hold.

 

For now, I really enjoy wimlib-imagex and have made it part of one my projects named QuickPE .

Wimlib then allows me to easily modify my boot.wim "on the fly" rather than mounting it, etc.

 

For sure, excellent and impressive work !

 

Cheers,

Erwan