77 replies to this topic

[Irongeek]

Hi all,
I plan to demo Winbuilder some more in a class/presentation, sort of like I did here:
http://www.irongeek....ices-usb-cd-dvd

but I want to go deeper. Currently it takes too long on my laptop to build the system. Any tips you can give to speed up the Win7pe build process? Been playing a little with RAM disks under the notion that hard drive speed may be the issue, what with all the files that need to be fetched.

[pscEx]

Hi all,
I plan to demo Winbuilder some more in a class/presentation, sort of like I did here:
http://www.irongeek....ices-usb-cd-dvd

but I want to go deeper. Currently it takes too long on my laptop to build the system. Any tips you can give to speed up the Win7pe build process? Been playing a little with RAM disks under the notion that hard drive speed may be the issue, what with all the files that need to be fetched.

Feel yourself as victim of Billy the Door: More (unnecessary) content, more (unwanted) user restrictions, more ...

Maybe you already saw or tried my project nativeEx_multiPE.
This project builds -except the individually transferred PE1, PE2, PE3 core- exactly the same final PE: Identical apps etc.

Times on my system (roughly from my remembering)
XP: 2 minutes
Vista: 9 minutes
Win7: 14 minutes

When you now try to find the advantages:
Vista: I do not know one
Win7: Many drivers make life easier also on 'exotic' systems.
Logically it would be the best, to use an XP source for your presentation.
But, unfortunatelly, today XP -even if it works great- is Dinosaur time, and is used only by 200% conservative geeks.

Especially to your question: To make things faster, you have only one possibility:
IMO there are a lot of WinSxS and manifest directories / files with many megabyte size copied to the target.

You can try to omit one after the other and check, whether this impacts the final PE.

Sorry, I have no "universal" suggestion which makes things faster.
The only suggestion I have, I mentioned above: Use XP source.

Peter

[Brito]

I plan to demo Winbuilder some more in a class/presentation, sort of like I did here:

Very good collection. You should add it on your signature space here in the forum.

I will be mentioning your website at our next weekly newsletter. When you have a video ready, don't forget to let us know as well.

You can also have a try at al_jo's Win7PE distro: http://al-jo.zxq.net/

[al_jo]

You can also have a try at al_jo's Win7PE distro: http://al-jo.zxq.net/

Hey, it’s even tinier now and more trimmed downed.
Under 10MB and on my machine, 5 minutes to build.
Just download, extract, run winbuilder.exe, point to your Win7 source and click “Play”
http://al-jo.99k.org/Win7PE_SEnano.7z

[Wonko the Sane]

If I may, a half-@§§ed idea.

Basically a winbuilder project does three things (very, very simplified and AFAIK/AFAICU):

• gets files from source and copies them in an "own" folder/folder structure
• assembles/writes/creates some registry entries, shortcuts, configuration files and extract from .scripts "embedded" files/apps
• put the whole set of things above together into (say) a .iso

It seems like with newish BLOATed sources, the time is dependant mainly on item #1.

What if we get some idea from a completely different field?

In restaurants rice and short pasta is normally "half cooked".
The actual chef starts cooking from these "pre-treated" base ingredients.

Can Winbuilder build be somehow separated between phase #1 and Phase #2?

Irongeek could have a "pre-assembled" base with most of the files already copied from source to the right folder/structure and run (I presume in a very short time) the other steps.

I know it's a kind of cheating, but it could also be useful, besides demonstrations/public meetings, also to testers and .script developers.

I don't know if it's possible , but if there would be a parser capable of reading the whole tree of .scripts and only execute the "CopyOrExpand" (or whatever is/are the actual copy from source command or commands) and later have in the actual run of Winbuilder an option to "skip" these same commands in each and every .script, the same loong time caused by the BLOAT could be divided into an "offstage" part and a quick "onstage" one.


Wonko

[RoyM]

nativeEx_Win7 had basebuild which I would like to see implemented again, I miss the good ole days.

[Sevilho]

If I may, a half-@§§ed idea.

Basically a winbuilder project does three things (very, very simplified and AFAIK/AFAICU):

• gets files from source and copies them in an "own" folder/folder structure
• assembles/writes/creates some registry entries, shortcuts, configuration files and extract from .scripts "embedded" files/apps
• put the whole set of things above together into (say) a .iso

It seems like with newish BLOATed sources, the time is dependant mainly on item #1.

What if we get some idea from a completely different field?

In restaurants rice and short pasta is normally "half cooked".
The actual chef starts cooking from these "pre-treated" base ingredients.

Can Winbuilder build be somehow separated between phase #1 and Phase #2?



Wonko

What about: If target file exist (and equal the source) then do not copy source file.

[pscEx]

If I may, a half-@§§ed idea.

Basically a winbuilder project does three things (very, very simplified and AFAIK/AFAICU):

• gets files from source and copies them in an "own" folder/folder structure
• assembles/writes/creates some registry entries, shortcuts, configuration files and extract from .scripts "embedded" files/apps
• put the whole set of things above together into (say) a .iso

It seems like with newish BLOATed sources, the time is dependant mainly on item #1.

What if we get some idea from a completely different field?

In restaurants rice and short pasta is normally "half cooked".
The actual chef starts cooking from these "pre-treated" base ingredients.

Can Winbuilder build be somehow separated between phase #1 and Phase #2?

Irongeek could have a "pre-assembled" base with most of the files already copied from source to the right folder/structure and run (I presume in a very short time) the other steps.

I know it's a kind of cheating, but it could also be useful, besides demonstrations/public meetings, also to testers and .script developers.

I don't know if it's possible , but if there would be a parser capable of reading the whole tree of .scripts and only execute the "CopyOrExpand" (or whatever is/are the actual copy from source command or commands) and later have in the actual run of Winbuilder an option to "skip" these same commands in each and every .script, the same loong time caused by the BLOAT could be divided into an "offstage" part and a quick "onstage" one.


Wonko

I disagree with your method to tell people "How easy you can ..."

You are an expert in DOS batch.
Please give a working example how to perform your idea with DOS batch.
Two days later you'll see the same with WinBuilder.exe.

Peter

[Wonko the Sane]

I disagree with your method to tell people "How easy you can ..."

I am absolutely NOT doing that.

You know, like:

If I may, a half-@§§ed idea.
....
I don't know if it's possible , but if there would be ....
....

However no problem , I wil gladly write a fake simplified Winbuilder .script in batch, and then a pre-processor for it, so that you can better get what the idea is/was.


Wonko

[pscEx]

However no problem , I wil gladly write a fake simplified Winbuilder .script in batch, and then a pre-processor for it, so that you can better get what the idea is/was.

See you tomorrow (?) with the result.
BTW: I'm very happy, that following your opinion the issue can be solved with a WinBuilder .Script.
Therefore it is not an issue of WinBuilder.exe. It is an issue of the individual project.

Peter

[MedEvil]

In restaurants rice and short pasta is normally "half cooked".
The actual chef starts cooking from these "pre-treated" base ingredients.

Can Winbuilder build be somehow separated between phase #1 and Phase #2?

Irongeek could have a "pre-assembled" base with most of the files already copied from source to the right folder/structure and run (I presume in a very short time) the other steps.

I don't quite understand. Why would copying the files from a precreated cache be faster, than copying them from the source wims?

[sbaeder]

In restaurants rice and short pasta is normally "half cooked".
The actual chef starts cooking from these "pre-treated" base ingredients.

I think part of this is the whole issue of SW is intellectual property and is licensed...Rice and pasta is tangible and purchased...Part of the reason for doing all the unpacking and such is to comply with the licenses of Microsoft...No one else can re-distribute their content in a different form...Otherwise, we could just build the whole PE!...

I find that with the caching I am already doing (using the recently added win7PE_SE ability to copy the files from an expanded location instead of a mounted WIM) that it only takes about 5 minutes or so to do a smallish build (one or two apps)...Of course that is with the COMP80 set "off" to speed things up!

MedEvil, this is faster, due to the way it has to mount and read the WIM file structure...more processing overhead...On a Win7 SP1 source, just setting up the mounted structure can take several minutes. After that, the speed isn't too different - but it is measurable)

Now, granted, I do have a fast system and SSD boot drive, etc. so your mileage may vary...Now, if we could just avoid the pesky legal aspects...

[RoyM]

Hi MedEvil
If your no familiar with (Joshua's Project) nativeEX_Win7
This is basically how it worked.
Z-BaseBuildEx.script would take the first few main copy scripts;

1-files
2-Config
3-Basic-files

it would run those files and let them do their job
then it would cache the results/files in a .7z file

Once that's done, it would then disable those scripts
so that on every build it would just use the cached
.7z file, Resulting in a much faster build from the
mere fact that Winbuilder had nothing to think about.
as in it didn't have to find files/copy files etc.

Please excuse the simplified explanation.

[pscEx]

Hi MedEvil
If your no familiar with (Joshua's Project) nativeEX_Win7
This is basically how it worked.
Z-BaseBuildEx.script would take the first few main copy scripts;

1-files
2-Config
3-Basic-files

it would run those files and let them do their job
then it would cache the results/files in a .7z file

Once that's done, it would then disable those scripts
so that on every build it would just use the cached
.7z file, Resulting in a much faster build from the
mere fact that Winbuilder had nothing to think about.
as in it didn't have to find files/copy files etc.

Please excuse the simplified explanation.

Thanks, RoyM, to mention my old idea.

In nativeEx_multiPE I intentionally did not use it again.
Unzipping a zipped directory tree also takes too much time. I'm currently working with using a real physical cache. But it can need up to several gb per source CD. When the user does not have that space on his HDD, ???

Give me some (Latin) days, and nativeEx_multiPE will give a solution which is suitable for everybody, taking care of different conditions of HDD space etc.

Peter

[RoyM]

Your Welcome
I Will be looking forward to any changes to speed things up

I run Winbuilder from raid0 SSD's also have 1Tb raid0 Serial HD's
So bring it on.
When developing every minute counts.

[Irongeek]

Nuno, If I make a video I will post. Thanks for discussing it guys, hope to see some more speed optimizations as time goes by.

[al_jo]

Ok, this one:
http://al-jo.99k.org...7PE_SEmicro2.7z
Builds in 3:30 minutes, on my machine!
Has q-dir, bsexplorer and runscanner.
13 scripts altogether.
Does it really need to be faster?


Ps. The final ISO will be 130MB

[MedEvil]

Once that's done, it would then disable those scripts
so that on every build it would just use the cached
.7z file, Resulting in a much faster build from the
mere fact that Winbuilder had nothing to think about.
as in it didn't have to find files/copy files etc.

Thanks for the explaination.
But it sounds to me, as if the speed up has nothing to do with the caching, but the fact that the Winbuilder copy routine gets replaced by the faster Windows one.
Same reason PEBuilder used to be much faster than WinBuilder.

[RoyM]

I would have to agree with that and less overhead as mentioned earlier

[ChrisR]

Nuno, If I make a video I will post. Thanks for discussing it guys, hope to see some more speed optimizations as time goes by.

Chose the project that suits you for your video but I'd be happy if you chose Win7PE SE (3500 downloads in 18 days for the latest version), base who has made smaller and faster children.

Thank you for the presentation
 Irongeek.jpg   86.08KB   22 downloads

[Wonko the Sane]

I don't quite understand. Why would copying the files from a precreated cache be faster, than copying them from the source wims?

It won't be faster (as a total), probably it would be overall slower, but it would simply "move" some time "away" from the actual build time (because that time has already been spent before).

As said is a form of "cheating" but could help in the case of Irongeek (realtime building on stage) or for developers (need to re-build several times to troubleshoot).


Wonko

[al_jo]

It seems that nobody is interested, but thought I should provide this info anyway:
This tweaked win7pe_micro is now building in less than 3 minutes on my machine:
http://al-jo.99k.org...7PE_SEmicro2.7z (9MB)
Final ISO will about 135MB.
Start-up time from UFD is less than one minute!
Have only Bsexporer & Filecommander on desktop and altogether 12 scripts checked.
Also now have 17 downloadable (needs Internet of course) system app scripts that is not checked. (Those scripts is made by Skybeam and updated by me)

Tested on this system:
GA-MA78LM-S2H RS780-SB710-7A66AG0NC-00 (Socket 754)
AMD Athlon II X3 425 Processor 2700MHz
Ram: 2.50 Gb (2 618 860 Kb) A0: 2048 Mb(EDO,155 ns)
ATI Radeon 3000 (700.0 Mb): 1920 x 1080
HDD: WDC WD3200AAKS-00V1A0

Attached Files

•  desktop.jpg   78.78KB   14 downloads

[MedEvil]

Yes and i'm sure we can build it even faster, if we make it still smaller and even less useful.

Why do we have this topic every couple of months?

A short absolute building time, is a completely pointless goal to achive!!!

A high ratio of, usabilty of the project against build time, that is a worthy goal.

Haveing the same functionality than project X, but building it in half the time. That's something i'd like to see.


Well, to be honest, for all i care, thea project could take 3 times as long as a similar one to buld, if the PE afterwards just boots 3 times faster.

[Budda]

I don't know, here is my input.

I have had in the past some build times of about 1.5 hours. This to me was tolerable because I've always told myself that once it's built all never have to touch it again!
Well, then better and more comprehensive builds came along and I found myself leaning towards modifications.

I've also found myself with long boot times for the PE as well and have desired for something decent. I like Al_Jo's idea of a quick and dirty build that is meant to accomplish a specific small task and deployment. I can also appreciate MedEvil's perspective of usability over speed. Regardless, I am always humbled by everyone's desire and ability to produce these free projects for the community to utilize in their every day life.

[al_jo]

Yes and i'm sure we can build it even faster, if we make it still smaller and even less useful.

Why do we have this topic every couple of months?

A short absolute building time, is a completely pointless goal to achive!!!

A high ratio of, usabilty of the project against build time, that is a worthy goal.

Haveing the same functionality than project X, but building it in half the time. That's something i'd like to see.


Well, to be honest, for all i care, thea project could take 3 times as long as a similar one to buld, if the PE afterwards just boots 3 times faster.

Can you please specify what is useless with this project?
I can run 100+ app scripts with it!
Exactly what is it you can not use in this project???


Edit:
@MedEvil
No need to reply: Project deleted!