66 replies to this topic

[windrv]

Given WB is quite mature now and that WinPE or VistaPE have inherent PE limitations on running, so:

Is there any WB plan to make universal mini 2k/xp/2k3/vista system images, just like that for win9x in the project of 'winimize', instead of their pe derivatives?

[Oleg_II]

I think it is perfectly possible to create folders structure and place needed files inside.

The problem is in creating system regestry on the fly. As far as I know Bart spent a lot of time for creating needed hives for BartPE.

It is much easier to use nLite for one ideal installation and then make something like XP in RAM (MobileOS project).


By the way, is it possible with WB to create HIVE files from INF files (for example converting HIVESFT.INF to SOFTWARE hive)?

[Brito]

Peter (PSC) went much further than Bart ever did and is the right the person for this "heavy" task.

If you look on nativeEx you'll notice that all these INF files (including shell32.dll) are scanned for retrieving informations and this is probably 2/3 of the registry work done with fully localized text (not just english as default).


Fun you mention this idea because I was also wondering the same thing - PE environments are by nature quite limited - we already know perfectly well how to mimic and add most features so why not remove the need to use the Windows Installation completely.

This would make things ideal for tweaking and customizing - no more strange workarounds - just apply changes on the fly and install directly from anywhere you wish.


Looking on unattended installs - I noticed that things can get complicated because there isn't a real alternative to the default windows installation procedure which takes too long to be completed.


This would make things much faster and give us a perfectly working OS which only contained the features we require.

I'm up for this challenge..

[MedEvil]

If all you wanna have is a XP that can be installed very fast on any computer, all you have to do is:
- install XP
- install your apps
- set everything up to your liking
- include driverpacks
- remove all hardware informations from the registry and set the default drivers for things like hdd and VGA.
- burn Image of the partition to DVD
Done.

If it has to really work on every computer, you will need a few versions with different hals, maybe even different kernels. Or fix that manualy before first start.

[Brito]

As you mention - I have the impression that we can always use a generic HAL and then implement one that is adequated for this task.

http://support.microsoft.com/kb/299340
http://support.microsoft.com/kb/309283
http://support.micro...b;EN-US;q237556


On boot.ini we can specify our generic HAL

http://www.praecelsu...cles/200002.htm
http://www.vmware.co...s32_disks8.html
http://msdn2.microso...y/ms792430.aspx

Never tried it myself, but sounds very possible to happen.


The interest here is that we could do all these steps in 3~5 minutes just like nativeEx..

[pscEx]

By the way, is it possible with WB to create HIVE files from INF files (for example converting HIVESFT.INF to SOFTWARE hive)?

Using this control code HoJoPE creates the software hive:

[HIVESFT.INF]AddRegHKLM,""@-2HKLM,"SOFTWARE\Microsoft\Command Processor","AutoRun"@-1HKLM,"SOFTWARE\Microsoft\Command Processor","CompletionChar"@-1HKLM,"SOFTWARE\Microsoft\Command Processor","DefaultColor"@-1HKLM,"SOFTWARE\Microsoft\Command Processor","EnableExtensions"@-1HKLM,"SOFTWARE\Microsoft\Command Processor","PathCompletionChar"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion","CurrentBuild"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion","InstallDate"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion","ProductName"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion","RegDone"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion","RegisteredOrganization"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion","RegisteredOwner"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion","SoftwareType"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\FontMapper"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\GRE_Initialize","FIXEDFON.FON"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\GRE_Initialize","FONTS.FON"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\GRE_Initialize","OEMFONT.FON"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\IniFileMapping"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Ports"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows","AppInit_DLLs"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows","DeviceNotSelectedTimeout"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows","GDIProcessHandleQuota"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows","Spooler"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows","swapdisk"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows","TransmissionRetryTimeout"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows","USERProcessHandleQuota"@-1HKLM,"SOFTWARE\Microsoft\Windows NT\CurrentVersion\WOW"@-1[HIVECLS.INF]AddRegHKCR,""@-2[Remove]HKLM,"Software\Microsoft\Active Setup\Installed Components\{4b218e3e-bc98-4770-93d3-2731b9329278}"

Peter

[MedEvil]

The interest here is that we could do all these steps in 3~5 minutes just like nativeEx..

You mean the setting of the right Hal and Kernel?
Very possible, if the user knows what is needed, we would just need a simple bat file where he could choose the appropriate ones to copy over.

Do you mean the whole process as described by me?
No way! I usually need about a week after a fresh install, till everything works again exactly the way i want it.

[Brito]

I mean the whole process as we already do with most projects - click and create..

You can add unattended installs of your favourite software apps and we can also define tweaks with our scripts as it has already been done before - not much difference - but we'll be avoiding the windows install completely which gives a huge boost on the overall project build.

NativeEx with registry creation and file copy takes around 1 minute to be complete on my machine - driverpacks uses around 2 minutes so we still have plenty of time left and already done the most of the hard work here..

If you test the new project I've posted, it will copy the whole contents of the CD (or source folder) to a temporary location where it can later be modified (on this case we're slipstreaming all driverpacks inside) and creating the image after testing in qemu. (>1Gb)

This is a very lengthy process as you can imagine (CD is usually 700Mb from retails sources) and this is completed in around 15 minutes. (including the time to create a liveXP project with a good ammount of applications included)

Efficient and unattended builds can become automated, try it..

[Oleg_II]

Peter (PSC) went much further than Bart ever did and is the right the person for this "heavy" task.

Peter
That is a bit more difficult that I can understand and use in practice right now but at least I can be sure now it is possible too

MedEvil
Creating an image is a good idea but imaging all different hardware it can be used on - it is not as universal as BartPE or nativeEx.

Nuno Brito
BartPE or nativeEx can run on different computers and use different hals so it can be made for an OS too.

Most time during OS installation is spent on two processes:
1. First of all on copying (and you can save this by preparing folder structure and placing files before creating regestry like in BartPE or nativeEx).

2. The second time consuming process - detecting and installing hardware on real hardware - this one takes time that hardly can be saved and, I think, is the most difficult to reproduce without real installation. This one is should be explored with the most attention in order to create such project.


As for customized installation - something like modular aproach in Linux, you're talking about something like vise versa nLite That will be even more difficult because it's easier to desassemble then assemble things...
I think it would be easier to take nLite for starters (create needed system configuration - all installation files will be edited by nLite) and then use WB to create a "universal" system from nlited source.

PS I'm mentioning nLite as a temporary tool to make installation smaller and faster. Nlited source is just a smaller source of full XP (main installation files are still the same). So it is not important at the moment will it be nlited or not or how it will be nlited. The main idea - pre-created system (nlited or not) should be able to run on ANY hardware.

[Brito]

...
Nuno Brito
BartPE or nativeEx can run on different computers and use different hals so it can be made for an OS too.

Most time during OS installation is spent on two processes:
1. First of all on copying (and you can save this by preparing folder structure and placing files before creating regestry like in BartPE or nativeEx).

2. The second time consuming process - detecting and installing hardware on real hardware - this one takes time that hardly can be saved and, I think, is the most difficult to reproduce without real installation. This one is should be explored with the most attention in order to create such project.
As for customized installation - something like modular aproach in Linux, you're talking about something like vise versa nLite That will be even more difficult because it's easier to desassemble then assemble things...
I think it would be easier to take nLite for starters (create needed system configuration - all installation files will be edited by nLite) and then use WB to create a "universal" system from nlited source.

No need for installation or worry about detecting/installing hardware - we already add driverpacks and PE boot disks boot just fine with minimal hardware support - XP Install CD's are growing outdated with little support to newer mass storage/LAN/WLAN drivers by default so we can also improve this section significantly.

An nlited install will mean that we're using the windows setup install and follows MS rules about this matter, this project goes far away from this concept and winnt32 can be ignored because it won't be needed at all - we just need to copy the files that we need from the source to a new folder and create an hive structure as it is already done with nativeEx - nothing more..

Meaning that we start with a very slim barebone and add up features/drivers as required - it's not a MS install or anything where people can even thing about using nlite or .NET. We only need to use one step to install XP where we need and that's it - or maybe install XP several times to different partitions without rebooting even once without crazy folders on temporary locations or text mode setups..

Cleaning up any bloatware and creating efficient work machines again.. (as jaclaz's signature)

Sorry for the confusion explaining things - but it can likely be made as simple as it is already done with other projects.

[Oleg_II]

No need for installation or worry about detecting/installing hardware - we already add driverpacks and PE boot disks boot just fine with minimal hardware support - XP Install CD's are growing outdated with little support to newer mass storage/LAN/WLAN drivers by default so we can also improve this section significantly.

It's not about drivers, it's about detecting and installing hardware on the fly (supposing you already have needed drivers). This process take some time that can't be saved anyway (only if drivers for much hardware are pre-installed - not a very good idea as it could lead to hardware conflicts).

An nlited install will mean that we're using the windows setup install and follows MS rules about this matter, this project goes far away from this concept and winnt32 can be ignored because it won't be needed at all - we just need to copy the files that we need from the source to a new folder and create an hive structure as it is already done with nativeEx - nothing more..

As I said, using nLite could be a temporary trick to make the installation source smaller and faster for the starting point of this project.
If using original source (HIVE) INF files for creating system HIVE files there will be a LOT of regestry keys that are not needed for bare-bone Windows. Well, if somebody (and I mean maybe Peter ) can create HIVE files already "cleaned" that would be great

Meaning that we start with a very slim barebone and add up features/drivers as required

Agree

[Oleg_II]

I'll try to explain what I mean under "clean" HIVE files:

This should be the system regestry in HIVE files form with installed components (only bare-bone, maybe with initialized networking as a most important component) but without specific hardware part - the last one should be created depending on real hardware during first boot (much like in PE environment).

[Oleg_II]

Meaning that we start with a very slim barebone and add up features/drivers as required

I already can add Russian and Chinese Simplified Pin Yin locales to Windows installation where all supporting files and regestry settings were deleted by nLite

And I'm going to add speach support to the installation where speach support including files and settings was deleted by nLite

There are also some simple components that can be made modular too: calculator, iexpress, dvdplay, some command line tools, etc. (they consist from files only and don't have or have minimum regestry settings).

[MedEvil]

I mean the whole process as we already do with most projects - click and create..

You can add unattended installs of your favourite software apps and we can also define tweaks with our scripts as it has already been done before - not much difference - but we'll be avoiding the windows install completely which gives a huge boost on the overall project build.

Well Nuno i know you're the scripting king but it would take me way longer to write all the needed scripts for my system then setting it up and imaging it.
After all we're not talking this time about a minimal system, but a full blown multi GB system.

MedEvil
Creating an image is a good idea but imaging all different hardware it can be used on - it is not as universal as BartPE or nativeEx.

A properly prepared image can do anything a PE can do, except run fom a ro medium and autodetect and set hal or kernel.
But on the plus side, you get a real XP with all the bells and whistles not some blown up PE.

And thinking about it, i can see no reason, why not someone should write a program that does the detecting and switching of the HAl and kernel in the installed image.

[Brito]

I was still talking about a minimal system - at least one capable of replacing the current PE environment limitations.

No need for scripts unless you wish some custom tweaks - use addons instead..

http://www.msfn.org/...p?showforum=132
http://www.msfn.org/...showtopic=85566


We won't need nlite for this task, just modify a bit the way how things are currently done with nativeEx and get all the positive changes as you've already mentioned - a "true" xp off the ground instead of one in PE mode. (it's really Peter's fault that the build differences between both are so little at this moment)

Would also remove limitations about installing programs inside - the image could be mounted with qemu and we could edit it whenever needed.

Did you know that qemu can simulate a virtual hardisk directly from a given folder path - not just physical partitions?


This can be simple if we join our heads to make this possible - not complicated..

[MedEvil]

I was still talking about a minimal system - at least one capable of replacing the current PE environment limitations.

Now you have me baffled, what limitations are you talking about?
I always thought PE was doing ok. It's not a true XP, but then any XP scraped down to that size also wouldn't be able to do much more.

[Brito]

Yes, it would - a full xp contains a much wider set of Win32 API already available to use, one side effect (for example) would be the possibility to enable themes by default and use a system which could expand itself in a much easier fashion that we do right now..

[Oleg_II]

Nuno Brito
There is really very few limitation with installing programs in PE environment - look at plugin section on 911cd.net Nearly ANY program can be installed under BartPE or nativeEx, it's just a matter of complicacy for creating needed plugin.
Except for...

MedEvil

Now you have me baffled, what limitations are you talking about?

Tell me about installing and using scanner under PE It is possible to install printers but it's a way TOO complicated. There are still some difficulties with networking - it is not easy to create WiFi and even DialUp connections under PE (comparing with orinal XP for example). And there is still not very easy to support multimedia functions of an OS in PE evironment (it is possible but as a result we get a system with a size of full OS but not as flexible as it (talking of installing software and hardware on the fly).


And I think we all are talking of a real OS created with WB here You just mean that an image is easier at the moment and Nuno the same full/modified image of a system can be created with WB (and can be even more universal as it is not installed yet on a specific hardware but can be run on all kind of it because of deleting some limitations during WB build process), right?

[was_jaclaz]

If I may, it appears to me that this topic is slowly but steadily turning into a "philosophic" matter about which "environment" is better, more extensible or more portable, i.e. ultimately in a Pe vs. Full OS discussion, resembling the many similar ones of the type 98 vs. XP, 2K vs. XP, Linux vs. XP, VISTA vs. Rest of the World, King Kong vs. Godzilla :
http://en.wikipedia....ng_vs._Godzilla
and so on.

So, I suggest to pragmatically put some basis to build upon:
1) It is useless (losing time that could be dedicated to actually create something) to discuss whether an "environment" is better than another, each has it's own ups and downs, let's assume that we actually like the idea of a "universal mini 2k/xp/Vista", regardless of possible existing or even "better" solutions.
2) It is unneeded (losing time that could be dedicated to actually create something) interrogating ourselves about the reasons that lead us to this new project, whatever it will turn to be: let us assume that we do this for either or both these two reasons:
a. for the sheer fun of it
b. because someone told us that it was [impossible/unuseful/unneeded] (choose one) and/or nobody did it before

This way we have both a target and a reason to get to it , and we can start working.

Back to topic, the idea of a minimal portable install is not really new, I remember having NT 4.00 reduced to something less than 50 Mb on a Zip disk, and even made (manually, with my "caveman" methods) a reduced win2k about 80 Mbytes (NOT portable) that I use as Emergency Recover Sytem on all my systems.
If anyone is interested, a list of the files is here:
http://www.msfn.org/...showtopic=41208

As I see it the big obstacle about all this project, whether done with my "caveman" methods or with more evoluted ones (I am thinking about all the great work done by Nuno and Peter/psc) is the lack of an easily manageable "dependencies" database.
(for an example of what it would be useful for, just check this thread here, where it took Peter several attempts to find out the problem was just a "duplicated" installed dependency:
http://www.boot-land...?...ic=2675&hl=

I already hinted the lack of such a tool a long time ago:
http://z3.invisionfr...hp?showtopic=80

I really don't know what or where to search for it, not even sure whether such a tool does exist, any idea, anyone?

jaclaz

[MedEvil]

@Nuno and Oleg
I don't want to talk you out of your project, it's neat idea!
I just fear that you might be disappointed with the result, as i think the reasons, you're doing it for, are not that solid.
IMO, you could get all the things running you'd like with a PE as well.

@Nuno
I see what you mean with Addons, didn't knew there was something like that. Thanks for the tip!
But the way i understand it, those addons just install an application, you don't get a chance to fully configure it, right?
In my case the installing of the software is usually quite fast, compared to the time the proper setting up takes.
So i was always more interested in not having to set up my software after a reinstall, than not having to install my software.
Maybe that explains the difference in our approaches.

@jaclaz
There exist a database, like the one you're looking for.
It's made by M$ itself and is called XP embedded.

[Brito]

Always good to read your comments!

There is a long list of ways to get the needed dependencies for most programs.

I've posted a very simple one here: http://www.boot-land...ript-t2665.html

With dependency walker it's also possible to profile an application - it will launch the program and start listing each new file (dependency) that is used. Not fullproof, but very handy.

Regmon and Filemon are also good set of tools to profile how a program behaves.

I think H7se has also implemented a very similar set of actions inside his H7pluginbuilder which automate this process.

The script coding itself is also becoming more flexible and easier to manage needed dependencies, adding files whenever not found inside the System folder of the windows install.

require_file,myLibrary.dll

Still some other details to solve, but some work is being done already..

[thunn]

Currenlt, I'm also working on a 100 MB. XP inram I long ago wondered if Peters skills could be appapted to this task of generating hives for a 'full' mini NT 5.1 / 5.2 OS. Wonderfull to see this discussion finally taking place.

[was_jaclaz]

@Nuno
@Medevil

Maybe I did not express myself properly, what I was asking for was not how to find dependencies or whether such a databse exists (though I fail to see what XP embedded has to do with this, expecially when we are talking, besides sheer "system" about applications as well).

What I would ideally want can be described (more or less, actually "more less" ) as follows:
1) A simple spreadsheet, like an Excel or OpenOffice.org calc:
First sheet called "Base" with 5 columns:
First column: Category of program or subset of OS function
Second column: Name of program or subset of OS function
Third Column: Version of program
Fourth Column: Version of OS
Fifth Column: List of all DLL's, OCX's, or whatever files dependencies, runtimes, etc.
Second sheet called "Registry":
Columns 1 to 4 identical to them on first sheet
Fifth column: List of all Registry keys needed
Sixth column: Values for each Registry key in fifth column
Third sheet called "Services":
Columns 1 to 4 identical to them on first sheet
Fifth column: List of all Services needed
Sixth column: Values for Start of each Service in fifth column
Fourth sheet called "Subsets":
Columns 1 to 4 identical to them on first sheet
Fifth Column: Link to ither spreadsheets (if available) that contain data for a certain subset of OS function
2) A method that can convert the result of field experiments by members (through the use of dependency tools) into the above "format"
3) A centralised repository for all the spreadsheets uploaded by members
4) A tool able to create, from a list of subsets and programs one wants in a certain build, a sort of database query from the data available in the various spreadsheets in the repository, a list of all files needed for the given configuration. This tool should create a sort of "multi-dimensional mesh" file that could be explored, for example selecting a single file, to see a list of all programs and subsets that need it, or selecting two or more programs or subsets, which files are "in common" between them, etc.. This "mesh" should also be capable of being manually edited (to remove or add a certain thing and also ALL it's dependencies) or to just a remove or add a single item for experimenting.
5) A tool able to parse this "mesh" and create a Winbuilder script capable of creating a system with NO duplicates, NO unneeded files or libraries.

I hope the above describes my idea better .

jaclaz

[MedEvil]

(though I fail to see what XP embedded has to do with this,

Xp Embedded is contrary to what most people believe not the name of an OS, but of an building environment.
It's a click your OS together program with an extensive database of dependecies of dlls, services and registry entries.
But you're right it's just for M$.

[Oleg_II]

Nuno Brito
One question: are you talking about a customized installation on one computer or about a customized installation that can be used on different computers (booting from CD or USB)?