Jump to content











Photo
- - - - -

reWIM

waik raw create extract wim

  • Please log in to reply
63 replies to this topic

#26 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 22 May 2012 - 03:22 PM

http://freecode.com/projects/jwim :frusty:

Ready to change the WimLib / jWim name in multiPE on a daily base :suda:

Peter

#27 0scar

0scar

    Member

  • Members
  • 90 posts

Posted 22 May 2012 - 03:58 PM

UltraWim seems to be a vacant mark.

#28 Nuno Brito

Nuno Brito

    Platinum Member

  • .script developer
  • 10545 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 22 May 2012 - 04:06 PM

Why the "freelancer" could not use the available 7-zip source as base?

It is coded in C and wasn't needed.

What was the reason to start (yet-again) from scratch?

For some developers it is faster/simpler to start from scratch than using a project made by someone else.


Why was java chosen as "engine"?

The compiled bytecode will run across diverse operative systems and machines. For example, when the new ARM version of Windows 8 comes out we will still be able to use the same old binaries. If we want to run this tool on android, a cloud OS or whatever they invent within the next 50 years, no problem.

Many other reasons that I fully subscribe were written at http://stackoverflow...age-over-others


http://freecode.com/projects/jwim :frusty:

No problem. Let's call this project "reWIM" and get the naming issue solved.
:)

#29 homes32

homes32

    Gold Member

  • .script developer
  • 1030 posts
  • Location:Minnesota
  •  
    United States

Posted 22 May 2012 - 04:13 PM

http://freecode.com/projects/jwim :frusty:

Ready to change the WimLib / jWim name in multiPE on a daily base :suda:

Peter

I saw that, but as it is a windowmanager implemented in Javascript and has nothing at all to do with images of any sort it should not be an issue especially if you stylize as jWIM

#30 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 22 May 2012 - 04:15 PM

No problem. Let's call this project "reWIM" and get the naming issue solved.

Sim, Boss :)

The change is done within minutes, and the upload does not need my assistance.

Peter

#31 patpat

patpat

    Member

  • Banned
  • 48 posts
  •  
    United States

Posted 22 May 2012 - 05:23 PM

start from scratch / java chosen as "engine" / etc,etc..

Your reasoning would be correct if you would be talking about a regular app but coding a WIM utility is far from the average case.

1) You cannot use a regular Java coder because you cannot give to him a written specification; there’s no one.
(that’s why your current implementation have several WIM format inconsistencies). The coder has to be able to generate the WIM specification what involves long hours parsing imageX disassembled listings and live code. Today it’s not really easy (cheap) finding a good Java coder with good assembler skills.

2) You cannot think of Java as a good language for this project considering all the low level byte oriented processes needed when handling a file format like WIM that mimics a whole file-system on a single file.

3) Java running on any platform? And what are you going to do with your Delphi coded Winbuilder? Are you going to get Delphi for ARM? If I were you I’d rather be trying to run away from Delphi than embracing challenges like coding a WIM library from scratch in Java.

4) The day that Windows for ARM will be released Microsoft will provide the corresponding compilers like they previously did with AMD64 and Itanium..

I think you should “rethink” your architectural decisions, if not probably some winbuilder “clone” is going to be the one that will really survive…

PS: BTW when are you going to release winbuilder 64Bits?

#32 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 22 May 2012 - 06:41 PM

@patpat

Let me disagree using my personal experience.
Since 1974 I programmed (whatever).
Since 1981 until I retired I worked for a worldwide known American company, first as employee, since 1987 as free lancer.
I can be proud that my work has been successful, and I have been called for critical implementations worldwide, e.g. 5 month in Rio.
When I started, the language has been DEC assembler.

And in assembler time we have had one and only one rule to fix a bug.
  • Collect ALL known issues.
  • Try to find the reason in the source code.
  • If you are not sure, do not try to change the code experimentally, write a function without that bug, starting from scratch.
I think it is also valid for adding something and also valid for currently used "higher" languages.

To use known libraries / snippets etc. is a personal decision of the developer. And nobody has the right to critizise the developer's "how"
BTW: Developing from scratch does not bind you to any known "Common License" and gives you the ability to use your own / favoured license model.

Peter

#33 MedEvil

MedEvil

    Platinum Member

  • .script developer
  • 7771 posts

Posted 22 May 2012 - 10:26 PM

BTW: Developing from scratch does not bind you to any known "Common License" and gives you the ability to use your own / favoured license model.

We had a rule in our company too. Don't reinventbuild the wheel, if you can just licence it.
Especialy if the two licences don't exclude eachother.

:cheers:

#34 paraglider

paraglider

    Gold Member

  • .script developer
  • 1729 posts
  • Location:NC,USA
  •  
    United States

Posted 23 May 2012 - 12:40 AM

I did a quick test of booting a win7 pe boot.wim created by imagex with max compression and recreated by this utility from a USB stick:

1) Original 404mb wim booted to shell loader in 62 seconds.
2) Boot.wim recreated by extracting the original wim with 7-zip and rebuliding using this utility produced a 1007mb boot.wim that reached the end of the progress bar stage after about 120 seconds then rebooted without ever showing the shell loader.

I conclude from this that the boot time is totally dependent on the size of the wim so wim compression is useful. I also conclude that there was something wrong in the wim file created by this utility.

#35 homes32

homes32

    Gold Member

  • .script developer
  • 1030 posts
  • Location:Minnesota
  •  
    United States

Posted 23 May 2012 - 02:19 AM

3) Java running on any platform? And what are you going to do with your Delphi coded Winbuilder? Are you going to get Delphi for ARM? If I were you I’d rather be trying to run away from Delphi than embracing challenges like coding a WIM library from scratch in Java.

PS: BTW when are you going to release winbuilder 64Bits?


Nuno has talked quite about about porting winbuilder to java. Personally I would rather see it rewritten in C++ or even :ph34r: C# but staying on topic I agree with move away from Delphi, as in my opinion it is becoming a bit of a dying language, and the compiler/IDE is expensive, and a number of issues and bugs are blamed on the current implimentaion of delphi 7 as well as their being no x64 compiler available (without upgrading).

#36 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14920 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 23 May 2012 - 12:57 PM

Maybe the tool was released a bit too early, as is it does not provide really *any* advantage over existing methods, once all the missing features will be added and possible bugs fixed it may prove to be a valid alternative, though I still remain perplexed by the java choice for it (only due to the implicit slowness of an interpreted language when going "low-level").
IMHO - just like for the Registry - a filesystem driver would be a better approach, but if the use is within a builder like Winbuilder possibly a handful of seconds more won't make a sensible difference, and sure it will be much more portable, with no people whining for the 64 bit version and for signed drivers.

:cheers:
Wonko

#37 ChrisR

ChrisR

    Silver Member

  • .script developer
  • 784 posts
  •  
    France

Posted 23 May 2012 - 02:48 PM

LZX and XPRESS compression algorithms can be added later. On my tests, the image without compression are faster to load and start.

I did some tests on the compression and it seems that:
Less compression (Without - /xpress - /lzx) gives a faster building. Compensated then by the creation of the Iso or the copy to USB (size).
but inversely and contrary to what I might think, more compression (/lzx - /xpress - Without) gives a faster starting.

I did a test with WIMlib (as delivered), mount boot.wim already made with WIMUtil /xpress (355 Mb),
copy all files in WIMlib\\WIMsource
rebuild with original run.bat (wimlib.jar), after creating the folder "Sources". Otherwise we have the message "Capture image failed." without error code or a message more explicit.
after that my building went well (814 Mb) and My Iso seems to start "windows is booting..." before an error.

:cheers:
  • Nuno Brito likes this

#38 Nuno Brito

Nuno Brito

    Platinum Member

  • .script developer
  • 10545 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 23 May 2012 - 02:53 PM

rebuild with original run.bat (wimlib.jar), after creating the folder "Sources". Otherwise we have the message "Capture image failed." without error code or a message more explicit.
after that my building went well (814 Mb) and My Iso seems to start "windows is booting..." before an error.

Hi Chris (and Paraglider that reported similar issue),

I noted from my tests that increasing the RAM available to the virtual machine might help on this case. Since it is not compressed, the whole image is loaded onto RAM before the system start. with a 1Gb WIM archive, I needed to use 1200Mb on my Virtual box instance, was unable to use QEmu with so much memory allocated.

Would be nice if we could just run the image straight from disk without loading into RAM to overcome this issue. For the moment, optimization and bug fixes are being applied for the next version, however compression is not yet at our reach. Sorry.

#39 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14920 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 23 May 2012 - 03:06 PM

Chris, JFYI, what you found by experiment has also a theoretical explanation.
Basically there is a ratio between processor speed, available RAM and transfer speed.
If you have an extremely "fast" processor and "low" transfer speed *anything* compressed will be faster.
If you have an extremely "slow" processor but a very "high" transfer speed (and enough RAM) *anything* UNcompressed will be faster.
Additionally most compression/decompression algorithms currently in use tend to be fairly "symmetrical", i.e. the amount of resources and time needed to compress something is more or less equal to those needed to UNcompress.
I suspect that the /xpress compression is less symmetrical, i.e. it is slower in compressing and faster in UNcompressing.
This is the "ideal" kind of compressor for PE builds, taken it to a limit it wouldn't be a problem if to compress the image it would take a few hours, IF the actual UNcompressing would be blazingly fast.

:cheers:
Wonko

#40 MedEvil

MedEvil

    Platinum Member

  • .script developer
  • 7771 posts

Posted 23 May 2012 - 03:26 PM

but if the use is within a builder like Winbuilder possibly a handful of seconds more won't make a sensible difference,

Unless your idea, how many seconds fit into a hand, differs greatly from mine, it won't be. I think the description 'a truckload of seconds' would be a better description. ;)

After the initial Java craze around the millenium change, companies soon realized, that even big corporations don't have soo much money to burn, to buy computers, which are fast enough to run Java as fast as a C and C++ program on conventional hardware.

Just check any of the sites, which have to deal with high traffic like ebay, facebook, ..., none use Java and that for a reason!
Only students, fresh off the universities, always come with the allegedly superiority of this shitty Java.

btw. Imho, there's no need to move straight to a driver. The use of any efficiently compiling programming language would do.

:cheers:

#41 ChrisR

ChrisR

    Silver Member

  • .script developer
  • 784 posts
  •  
    France

Posted 23 May 2012 - 04:18 PM

I noted from my tests that increasing the RAM available to the virtual machine might help on this case.


Yes, by increasing the memory to 1.5 GB the PE has started.

I suspect that the /xpress compression is less symmetrical, i.e. it is slower in compressing and faster in UNcompressing.
This is the "ideal" kind of compressor for PE builds, taken it to a limit it wouldn't be a problem if to compress the image it would take a few hours, IF the actual UNcompressing would be blazingly fast.


According to my tests /lzx, seems best to start a PE (faster in UNcompressing) but /xpress seems, in fact, best suited between the building and the starting

Otherwise, agree with Medevil, I also saw (by experiment ;)) some companies using Java, it *was* the fashion, back to C++ or C# (which believes in popularity, I'm not really a fan).
Despite a lot of young skill Java, freshly out of school.

:cheers:

#42 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14920 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 23 May 2012 - 05:23 PM

Unless your idea, how many seconds fit into a hand, differs greatly from mine, it won't be. I think the description 'a truckload of seconds' would be a better description. ;)

I see how you are not much familiar with the concept of "understatement" :whistling:
:cheers:
Wonko

#43 MedEvil

MedEvil

    Platinum Member

  • .script developer
  • 7771 posts

Posted 23 May 2012 - 06:39 PM

:dubbio: Can't find any reference, that anyone ever accused an Italian of 'understatement'! :poke: :lol:

:cheers:

#44 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 23 May 2012 - 06:50 PM

I know that one claim of this forum is to have fun.

But is it necessary to discuss OFFTOPIC stuff to have fun?

Peter :dubbio:

#45 MedEvil

MedEvil

    Platinum Member

  • .script developer
  • 7771 posts

Posted 23 May 2012 - 07:08 PM

:dubbio: Nor can i find any, that a German was ever accused of being easy going! :poke: :lol:

:cheers:

#46 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 23 May 2012 - 07:12 PM

I want to hold this topic readable, and you both are systimatically scramble the logic.

Peter

#47 MedEvil

MedEvil

    Platinum Member

  • .script developer
  • 7771 posts

Posted 23 May 2012 - 08:48 PM

I know, you're ready to jump on the jWimLib train, any second now. Unfortunately for you, the train you wanna jump on, doesn't exist yet, it's just a phantom. When you jump on it now, you land on your butt, as a few people have pointed out.

jWimLib, can't do hardlinks and can't do compression, which already makes it, at this point, perfectly useless, imo.
But in addition, it obviously can't even do wim correctly at all, as all attempts, to boot from a jWimLib created wim, have failed.

:cheers:

#48 Nuno Brito

Nuno Brito

    Platinum Member

  • .script developer
  • 10545 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 23 May 2012 - 09:21 PM

It is creating images just fine.

Look on psc's project for a pratical implementation inside wb or just use the batch script that is provided on the zip file.

Duplicate files are now avoided. Compression is not a priority for the moment. First we handle the reported issues before moving to more features.

#49 paraglider

paraglider

    Gold Member

  • .script developer
  • 1729 posts
  • Location:NC,USA
  •  
    United States

Posted 24 May 2012 - 01:30 AM

My test was using my physical machine not a VM. It has 6GB of ram.

#50 patpat

patpat

    Member

  • Banned
  • 48 posts
  •  
    United States

Posted 24 May 2012 - 11:29 PM

Compression is not a priority for the moment.

you planning WIM compression in Java ;-) you kidding righ?

First we handle the reported issues before moving to more features.


reported issues?? more features???
Nuno, your pre-design version (not even alpha) is very far from even having the minimum quality for being published…
You are using people’s time for testing code that you are going to keep closed, I think it’s a matter
of respect on this cases to wait until the software quality justifies people’s testing effort.

Why are you in such a hurry, publishing this kind of things???





Also tagged with one or more of these keywords: waik, raw, create, extract, wim

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users