Jump to content











Photo
* * * * * 2 votes

Separation of system-core and user-stuff: LODR-packs


  • Please log in to reply
520 replies to this topic

#1 sanbarrow

sanbarrow

    Silver Member

  • Developer
  • 788 posts
  • Location:Germany - Sauerland

Posted 03 January 2009 - 03:32 AM

Little suggestion for winbuilder if you don't mind ...

Please watch this one-minute video - it shows my prefered way of adding apps to a PE.
Every thing needs its name and so I called this way of adding apps LODR-packs.
LODR-pack means a procedure of packing an app in such a way that it can be Loaded On Demand.
In MOA I do everything like that ... dotnet2 ... java ... vmware workstation ... everything.
I also load about 2/3 of a typical LiveXP programs collection that way ... same with UBCD4win stuff.
This procedures makes it very easy to keep tons of apps ready at hand without need to rebuild the core over and over again.

A LODR-pack is a directory + a batch or a compiled script ready for use.
It can be automated by adding a single line to either prenetwork-batch.cmd or late-batch.cmd.
It does not need any registry stuff or files added at build-time.
It can be loaded from any path.

The batch creates a junction to the correct path and then speed-installs the app if necessary - like for example with VirtualBox.

If we could agree on some conventions you could use some of my stuff and I could maybe use some of yours ...

regards Ulli

By the way - do you know Damn Small Linux and its way of having a repository of LODR-packs ? - they call that .dsl packages.
We could have the same ...

#2 amalux

amalux

    Platinum Member

  • Tutorial Writer
  • 2813 posts
  •  
    United States

Posted 03 January 2009 - 04:11 AM

Forgive my ignorance but what are the advantages? Assuming it could be done, say an Acronis TrueImage LODR-pack for LiveXP; would it still be portable, bootable from any machine? Would it reduce image size/loading time? Could you still choose any read/write media to boot from? Could it still be RAM loadable for fast performance in PE?

#3 sanbarrow

sanbarrow

    Silver Member

  • Developer
  • 788 posts
  • Location:Germany - Sauerland

Posted 03 January 2009 - 04:22 AM

Would it still be portable, bootable from any machine?

- sure - your core system does the booting - not the app

Would it reduce image size/loading time?

- of course - LODR-packs can be where ever - you do not even need to add them to the build

Could you still choose any read/write media to boot from?

- same as #1 - you can boot from where ever your core-system boots from.

Could it still be RAM loadable for fast performance in PE?

- yeah - sure - in MOA I always ramload for performance

This is not about changing your LiveXP-core - its just about packing the apps slightly different.
You now use app-scripts - that means you must run a downloaded app-script through winbuilder once - if you pack as LODR-packs you can directly use it.
And it will not influence your core-system in any way - download a LODR-pack from a running LiveXP - test it - if its nice keep it and store it where ever - if its crap delete it again.

In MOA the complete instructions for adding VirtualBox are:
download from SUN
install once
keep the files
next time you need it - start it with the lodr-pack batch from here ....

Adding apps can't be any easier

#4 amalux

amalux

    Platinum Member

  • Tutorial Writer
  • 2813 posts
  •  
    United States

Posted 03 January 2009 - 04:36 AM

And it will not influence your core-system in any way - download a LODR-pack from a running LiveXP - test it - if its nice keep it and store it where ever - if its crap delete it again.

This sounds great but just to be sure I'm understanding correctly, you're saying this could work in a real hardware booted LiveXP or only in VM-environment like MOA?

#5 sanbarrow

sanbarrow

    Silver Member

  • Developer
  • 788 posts
  • Location:Germany - Sauerland

Posted 03 January 2009 - 04:39 AM

This sounds great but just to be sure I'm understanding correctly, you're saying this could work in a real hardware booted LiveXP or only in VM-environment like MOA?


Nothing of this requires any VM-environment - in fact Workstation is just another LODR-pack in MOA - I only load it on demand and it is not required

#6 amalux

amalux

    Platinum Member

  • Tutorial Writer
  • 2813 posts
  •  
    United States

Posted 03 January 2009 - 04:44 AM

Nothing of this requires any VM-environment - in fact Workstation is just another LODR-pack in MOA - I only load it on demand and it is not required

I'll be taking a close look at MOA (again) to if I can figure out how these LODR-packs are setup and how they can be ported to LiveXP. I'm sure to have questions along the way, thanks for your forward thinking here and any help you can provide this MOA noob :cheers:

#7 sanbarrow

sanbarrow

    Silver Member

  • Developer
  • 788 posts
  • Location:Germany - Sauerland

Posted 03 January 2009 - 04:48 AM

Look at the batch I made for VirtualBOX - here http://www.boot-land...?showtopic=6588

As all speed-install instructions are plain batch you can easily adopt it to your environment.

The dotnet2 or workstation packages I have are much more advanced but basically they work the same.
Both of those will not translate to winbuilder easily as we use such different layouts - but we could talk about reasonable standards ...

#8 Lancelot

Lancelot

    Frequent Member

  • .script developer
  • 5013 posts
  • Location:Turkiye/Izmir
  • Interests:*Mechanical stuff and Physics,
    *LiveXP, BartPE, SherpyaXPE,
    *Basketball and Looong Walking,
    *Buying outwear for my girlf (Reason: Girls are stupid about buying bad stuff to make themselves uglier :))
    *Girls (Lyric: Girl,...., You will be a womann, Soon)
    *Answering questions for "Meaning of life",
    *Helping people,

    Kung with LiveXP, Fu with Peter :)
  •  
    Turkey

Posted 03 January 2009 - 11:04 AM

sanbarrow

can you put a movie player for your movies so we can pause, forward backward when watching ;)

and yes, LODR-pack method you introduced (repeatedly :cheers: ) may be used with livexp, I just have a great idea for "Midway" :cheers: , but need some days :cheers:.

#9 Galapo

Galapo

    Platinum Member

  • .script developer
  • 3841 posts
  •  
    Australia

Posted 03 January 2009 - 11:09 AM

Good, I'm interested too.

One standard I'd like to suggest: no hard-coding of drive letters. Only use existing variables, or we could agree on a standard variable which a given project may define as a hard-coded path.

Regards,
Galapo.

#10 Lancelot

Lancelot

    Frequent Member

  • .script developer
  • 5013 posts
  • Location:Turkiye/Izmir
  • Interests:*Mechanical stuff and Physics,
    *LiveXP, BartPE, SherpyaXPE,
    *Basketball and Looong Walking,
    *Buying outwear for my girlf (Reason: Girls are stupid about buying bad stuff to make themselves uglier :))
    *Girls (Lyric: Girl,...., You will be a womann, Soon)
    *Answering questions for "Meaning of life",
    *Helping people,

    Kung with LiveXP, Fu with Peter :)
  •  
    Turkey

Posted 03 January 2009 - 11:13 AM

Galapo

;) i have a great idea, give me 2 days :cheers:

#11 Brito

Brito

    Platinum Member

  • .script developer
  • 10616 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 03 January 2009 - 11:24 AM

Hi Sanbarrow.

I agree with Lancelot, would you please upload to youtube and post the link here?

Makes it easier for everyone to see the video.

Thanks.

:cheers:

#12 sanbarrow

sanbarrow

    Silver Member

  • Developer
  • 788 posts
  • Location:Germany - Sauerland

Posted 03 January 2009 - 12:44 PM

One standard I'd like to suggest: no hard-coding of drive letters


Whats the problem with hard-coded driveletters ?

Look for example at dotnet.
The way I prepare this into a LODR-pack I do a regshot while installing it on the fly running PE and I directly use that reg-patch without any manual editing required for the lodr-pack.
If you introduce variables you need to manually edit the patch first - then you apply it later on and have to do something like reg-expand again.
I don't need to do all that.

Have a look at Linux - they put apps in /usr/bin or /usr/local/bin - which are hardcoded paths too.
What the Linux folks do with redirecting usr/bin to /dev/sda3 or whatever in fstab - I do with a single junction command at run-time.
That is so much easier than translating twice while using un-necessary variables.

#13 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 03 January 2009 - 02:08 PM

Congrats, Ulli ;)

The Load-On-Demand is a great idea!

Currently:
On every boot you have 'thousands' of RunOnces doing a lot of never used reg entries etc.
On every boot you start 'thousands' of services you never need.
On every boot you start 'thousands' of drivers you never need.
On every boot ...

With Load-On-Demand:
If you need in a certain recovery a certain app / feature / driver / service etc., just install it and you have it!

Great!

Please try to make it 'WB - Usable' by providing a script to generate a LODR package
(BTW: I would accept a cmd batch only if it is controlled by a WB script :cheers: )
@All: In addition to the well known 'App Scripts' I can imagine a class 'LODR Scripts'

Peter

EDIT:

I thought about it and will try to publish a nativeEx_LODR project next weeks.

Based on nativeEx_barebone_075 it will contain an app which inside the PE will offer a download list to download, install and execute the LODR app / driver / service.

I'm rather sure that such a project with minimum build time, minimum ISO size, minimum boot time and maximum features (depending on the collection of LODR packages) will have a great future.

@sanbarrow, can you PM me for tests one package, maybe the VMWare or VirtualBox?

#14 sanbarrow

sanbarrow

    Silver Member

  • Developer
  • 788 posts
  • Location:Germany - Sauerland

Posted 03 January 2009 - 03:20 PM

Peter - thanks a lot for your open minded attitude :cheers:

Based on nativeEx_barebone_075 it will contain an app which inside the PE will offer a download list to download, install and execute the LODR app / driver / service.


You got it - thats exactly what this is about.
Separate the core-system as cleanly as possible from the stuff that can be loaded on demand.
We sure would have to introduce a classification for the different core-systems we use.
We can't expect that a dotnet2-LODR-pack created for MOA or LiveXP runs on a 31 MB naked core-system - but that should be no problem.
Maybe we need core-classes like
"naked" - "native-XP" - "LiveXP" - "MOA-like" - "VistaPE" - "2k8-PE" - ... what ever the future may bring ...
So what - that doesn't change the beauty of this approach.

Sure - we may find out that some apps can't be LODR-packed ... maybe some Acronis stuff doesn't work that way.
But that doesn't matter - things that can't be packed this way can be added the usual way with app-scripts or plugins
- or if they are important enough - they may also be added to the core-system itself.

The big advantage of this approach is that we "advanced PE-hackers" can then create solid core-systems that can be easily reproduced by unexperienced users.
We can then leave adding the additional bells and whistles to the users.

Ulli

#15 MedEvil

MedEvil

    Platinum Member

  • .script developer
  • 7771 posts

Posted 03 January 2009 - 04:55 PM

Looks like deviding the projects into Vista based and XP based is outdated.
Maybe better groups would be:
- PE for old fashioned people and/or computers
- If you have enough RAM to run a full install from RAM, you mighty also have enough for this PE! ;)

Guys seriously, if i work only on machines that have so much RAM, that i can first install all applications to ramdisk and then even run them, then i need no stinking PE with tons of features missing. I simply go for an automated install.
We even already have a project like this!


:cheers:

#16 Brito

Brito

    Platinum Member

  • .script developer
  • 10616 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 03 January 2009 - 05:02 PM

Whats the problem with hard-coded driveletters ?


It's not just the driveletters, system folders are also hardcoded and this would mean a pack for each specific project which is what we worked so far to avoid in the first place.. :cheers:

You're still thinking on terms of a I386 from XP PE but on Windows PE 2.x it is used the \Windows or anything else so this would mean for a start to double the work to make the same pack available for two projects.

If in the future comes a 7PE that places all system files in \kernel then we'd have to redo all again.. ;)

Also, how would the Documents and Settings folder be handled? On XPE from sherpya you'd see it in english but what if you're copying the registry entries for the german OS, wouldn't the paths differ?

There are plenty of reasons to keep things as flexible as possible.

--------

Otherwise it would mean a return to static paths everywhere and would impose developers to create several packs for each project.

Look on bartPE with the INF format, it would never work under Windows PE 2.x if there weren't app scripts available to translate .inf into a generic script language that is shared between projects.

The same thing for nLite as all the addons were coded with XP in mind but now that Vista changed things a little, this automatically meant that all previously made addons would not be recycled.

It doesn't mean that bartPE plugins or nLite addons don't work with newer OS's, it's just that they are not flexible and now will mean repacking them again.

We've learnt this lesson some years ago while each wb project used it's own way to write scripts for adding programs.

The current method helps to build and maintain a large library of available apps that can be used in future projects for any sort of objective.

-------

Please note that I'm not against the idea of LODR but I'm an ecologist and would really be sad to witness such waste of bytes that wouldn't be recyclable by other projects.

We need to think about preserving the environment and any LODR-pack should be made as project-agnostic-friendly as much as possible to ensure it has a chance to survive in this crazy world of ever changing projects.. :cheers:

#17 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 03 January 2009 - 05:03 PM

Looks like deviding the projects into Vista based and XP based is outdated.
Maybe better groups would be:
- PE for old fashioned people and/or computers
- If you have enough RAM to run a full install from RAM, you mighty also have enough for this PE! ;)

Guys seriously, if i work only on machines that have so much RAM, that i can first install all applications to ramdisk and then even run them, then i need no stinking PE with tons of features missing. I simply go for an automated install.
We even already have a project like this!


:cheers:

Nobody demands that you use it.

I remember ICE age when I came up with the nativeEx idea.

There was also nobody who needed it. But I just did ...

Therefore, if we do not hurt or hinder somebody, please allow sanbarrow and me to waste our time ...

Peter

#18 Brito

Brito

    Platinum Member

  • .script developer
  • 10616 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 03 January 2009 - 05:11 PM

I remember ICE age when I came up with the nativeEx idea.
........
So, please allow sanbarrow and and me our personal fun.


Will do, sanbarrow is in good hands with no doubts.

I was just worried about seeing our work moving back to the dark ages.

Hope both of you have fun exploring further this method!

:cheers:

#19 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 03 January 2009 - 05:13 PM

I was just worried about seeing our work moving back to the dark ages.

I have the vision that just the opposite will happen!

Peter

#20 Brito

Brito

    Platinum Member

  • .script developer
  • 10616 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 03 January 2009 - 05:19 PM

You're wicked.. :cheers:

#21 MedEvil

MedEvil

    Platinum Member

  • .script developer
  • 7771 posts

Posted 03 January 2009 - 05:19 PM

please allow sanbarrow and me to waste our time ...

By all means, please waste away! :cheers:
It's not like i could stop you anyhow. ;)

:cheers:

#22 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 03 January 2009 - 05:27 PM

You're wicked.. :cheers:

It's my lack of English. I do not understand whether 'wicked' is a compliment or an agression.

But nevertheless: My vision is not new.

When some years ago Billy the Door invented dot net, he told us, that a small (seen from now: huge) core is sufficient. Everything else can be downloaded on demand.

And I hope, that our realization will become a bit better than the 'dot net' realization and can survive with a really small core.

(What is helpful: here we do not have to satisfy billions of users, we only have to satisfy some thousands of users ;) )

Peter

#23 Lancelot

Lancelot

    Frequent Member

  • .script developer
  • 5013 posts
  • Location:Turkiye/Izmir
  • Interests:*Mechanical stuff and Physics,
    *LiveXP, BartPE, SherpyaXPE,
    *Basketball and Looong Walking,
    *Buying outwear for my girlf (Reason: Girls are stupid about buying bad stuff to make themselves uglier :))
    *Girls (Lyric: Girl,...., You will be a womann, Soon)
    *Answering questions for "Meaning of life",
    *Helping people,

    Kung with LiveXP, Fu with Peter :)
  •  
    Turkey

Posted 03 January 2009 - 05:34 PM

Well

probably sanborrow and peter will work hard on this, and what i will do will be rubbish, so better to summerize my idea for "Midway":

Idea:
i thought of a different name, "wim-Lpack" (wim Load pack) and "LOD" (Load on Demand) as a suboption:

wim-Lpack
|
|---->LOD

when a user enable checkbox for wim-Lpack
during build
the build will check if FastStoneCapture_Files_v1.wim exists at %GlobalTemplates%\wimlpack_Files\ ,
--> if exists will copy to %Target_Prog%
--->if not exits, the folder will be wimmed (like FastStoneCapture_v1.wim ) and will be copied to %GlobalTemplates%\wimlpack_Files\ and %Target_Prog%

Than if LOD checkbox is selected, than it will copy a .cmd file and related shortcuts
--> if LOD checkbox not selected than it will add to mount .wim at startup

on livexp the wim file will be mounted (either at startup when LOD disabled or as descibed in sanbarrow's way) to
%RAMDriveLetter%\%ProgramsFolder%\%ProgramFolder%\


Well this idea have lots of advantages,
1) first of all it is easy to update the script so the relevant wim will be updated too, (+it is also easier to update .wim)
2) Also besides FastStoneCapture_v1_Files.wim , there can be FastStoneCapture_v1_Reg.reg, FastStoneCapture_v1_Shortcut_Desktop.7z with same idea
which will result the build be faster
3) the iso created with "create iso" will be smaller compared with disableing wim-Lpack unselected
4) there can be a verify to check if bootsdi and fbwf is selected for some scripts that needs to write %systemdrive%


Well these are my ideas, i hope you like (or understand), i will wait to see what peter and sanborrow do, good luck.

#24 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 03 January 2009 - 05:42 PM

Thanks, Lancelot!

I'm glad to see that there is #3 of the 'on demand' idea.

Currently I do not understand your post completelly (due to 'Midway'), but I'm going to try.

Peter :cheers:

EDIT: My be now I understood: What you are describing, logically describes my vision nearly completelly!

#25 sanbarrow

sanbarrow

    Silver Member

  • Developer
  • 788 posts
  • Location:Germany - Sauerland

Posted 03 January 2009 - 05:52 PM

@ medEvil

Guys seriously, if i work only on machines that have so much RAM, that i can first install all applications to ramdisk and then even run them, then i need no stinking PE with tons of features missing.


You miss the point - only the LODR-pack developer has to do this once. Then he creates a batch/script so that other users can use it with minimal requirements.
For example the combined dotnet2 and VMware-apps package I maintain called "esx-tools" needs about 250 Mb of free space in X: during creation.
Thats not practical for daily use of course but you only need to do that once.
Once that is LODR-packed you can then use it on a system with 15 Mb of free space in X:

I think the VirtualBox packages may work with 1 Mb of free space in X:
And by the way - I rarely use a ramdisk this days as in MOA I don't need one most of the times.

I was just worried about seeing our work moving back to the dark ages.


;) - hey this post gets really funny :cheers:

Hey - the LODR-pack I use for Workstation installs any recent version you throw at it at run-time in a matter of seconds with minimal RAM requirements.
If you call that ice-age style then - yeah I love the ice-age :cheers:




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users