App scripts hurting PE

I can now confirm that we actually have application scripts which hurt PE.
Namely they slow PE down to a crawl.

Up until now i always thought, LiveXps more sluggish behaviour came from the newer basic or core scripts.
But since NaughtyPE shows now the same behaviour, maybe even worst. It clearly comes from the app scripts.

Can't tell yet which script(s) are responsible.

Did anyone else encounter this and already managed to catch the guilty script(s) or at least narrow it down?

I can now confirm that we actually have application scripts which hurt PE.
Namely they slow PE down to a crawl.

Up until now i always thought, LiveXps more sluggish behaviour came from the newer basic or core scripts.
But since NaughtyPE shows now the same behaviour, maybe even worst. It clearly comes from the app scripts.

Can't tell yet which script(s) are responsible.

Did anyone else encounter this and already managed to catch the guilty script(s) or at least narrow it down?

That's the price for having 'universal' scripts.

Compare log of a shortcut calling BuildModel directly:

Run - Processing section: [Add-Shortcut] From file: [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script]        [Parameter] with parameters: [DE],[#$pSystemRoot#$p\system32\notepad.exe],[Notepad],[-SW:#$pSystemRoot#$p\system32\]        [Success] AddVariables - Added section [Variables] from file [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script] into the local variable list        [Success] IF - File does not exist: [%parse%] evaluated string: [If,NotExistFile,"%parse%",ExtractFile,"%BuildModelScript%","Folder","shortcutParse.exe",%ProjectTemp%]        [Success] ExtractFile - [shortcutParse.exe] to: [%BaseDir%\Temp\nativeEx_barebone_075]        [Ignore] IF - Directory exists: [%BaseDir%\Temp\nativeEx_barebone_075\Shortcuts] evaluated string: [If,NotExistDir,%ProjectTemp%\Shortcuts,DirMake,%ProjectTemp%\Shortcuts]        [Success] AddVariables - Added section [Variables] from file [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script] into the local variable list        [Ignore] IF - File does not exist: [%BaseDir%\Temp\nativeEx_barebone_075\buildModel_err.log] evaluated string: [If,ExistFile,"%ProjectTemp%\buildModel_err.log",FileDelete,"%ProjectTemp%\buildModel_err.log"]        [Ignore] IF - [DE] is not equal to: [1] evaluated string: [If,#1,Equal,1,ShellExecute,Hide,"%parse%","#$q/C:%ProjectInfo%#$q #$q/O:%shcTemp%#$q #$q/A:#1#$c#2#$q"]        [Success] IF - [DE] is not equal to: [1] evaluated string: [If,#1,NotEqual,1,ShellExecute,Hide,"%parse%","#$q/C:%ProjectInfo%#$q #$q/O:%shcTemp%#$q #$q/ST:#1#$q #$q/SE:#2#$q #$q/SL:#3#$q #$q#4#$q"]        [Success] ShellExecute: [Hide] using: [%BaseDir%\Temp\nativeEx_barebone_075\shortcutParse.exe] with parameters: ["/C:%BaseDir%\Workbench\nativeEx_barebone_075\UserData\ProjectInfo.ini" "/O:%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini" "/ST:DE" "/SE:%SystemRoot%\system32\notepad.exe" "/SL:Notepad" "-SW:%SystemRoot%\system32\"]        [Success] IniRead - [%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini] Section [Created] Key [Date] to variable: [%tim%] with value [2009-Jan-06 165838406]        [Success] IniRead - [%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini] Section [Shortcut] Key [shortCutName] to variable: [%nam%] with value [Notepad]        [Success] FileCopy - Copied [%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini] to: [%BaseDir%\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06 165838406.shc]        Run - Processing section: [Add-Shortcut] From file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]              [Parameter] with parameters: [V:\WB\wb#$sdebug\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06#$s165838406.shc]              [Success] AddVariables - Added section [Variables] from file [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script] into the local variable list              Run - Processing section: [shc-model] From file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]                    [Parameter] with parameters: [V:\WB\wb#$sdebug\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06#$s165838406.shc]                    [Ignore] IF - File exists: [%BaseDir%\Target\nativeEx_barebone_075\$ModelRAM\filled] evaluated string: [If,NotExistFile,"%TargetDir%\$ModelRAM\filled",FileCreateBlank,"%TargetDir%\$ModelRAM\filled"]                    [Ignore] IF - File exists: [%BaseDir%\Temp\nativeEx_barebone_075\shortcutWinExp.exe] evaluated string: [If,NotExistFile,"%ProjectTemp%\shortcutWinExp.exe",ExtractFile,"%ActiveShell%","Shortcuts",shortcutWinExp.exe,%ProjectTemp%]                    [Success] ShellExecute: [Hide] using: [%BaseDir%\Temp\nativeEx_barebone_075\shortcutWinExp.exe] with parameters: ["/C:%BaseDir%\Workbench\nativeEx_barebone_075\UserData\ProjectInfo.ini" "/I:%BaseDir%\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06 165838406.shc"]              Run - Processed section [shc-model] in file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]        Run - Processed section [Add-Shortcut] in file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]        [Ignore] IF - File does not exist: [%BaseDir%\Temp\nativeEx_barebone_075\buildModel_err.log] evaluated string: [If,ExistFile,"%ProjectTemp%\buildModel_err.log",ShellExecute,Open,"%ProjectTemp%\buildModel_err.log",""]  Run - Processed section [Add-Shortcut] in file: [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script]

And here using API:

Run - Processing section: [AddShortcut] From file: [%BaseDir%\Projects\nativeEx_barebone\Basic\Build\api.script]         [Parameter] with parameters: [Desktop],[],[#$pSystemRoot#$p\system32\notepad.exe],[Notepad]         [Info] Create ShortCut Desktop         [Success] StrFormat - converted: [] to variable: [%OptionShortcut%] with result: []         [Ignore] IF - [Desktop] is not equal to: [EditWith] evaluated string: [If,#1,Equal,"EditWith",If,#2,Equal,,Set,#2,"Edit with"]         [Success] ELSE - executing command: [If,#2,Equal,,StrFormat,filename,%Scriptdir%,#2]         [Success] IF - [] is equal to: [] evaluated string: [If,#2,Equal,,StrFormat,filename,%Scriptdir%,#2]         [Success] StrFormat - converted: [%Scriptdir%] to variable: [#2] with result: [Productivity]         [Ignore] IF - [%SystemRoot%\system32\notepad.exe] is not equal to: [] evaluated string: [If,#3,Equal,,Set,#3,"%PE_Programs%\%programFolder%\%programExe%"]         [Success] StrFormat - converted: [#3] to variable: [%CPath%] with result: [%SystemRoot%\system32\]         [Success] StrFormat - converted: [#3] to variable: [%CFilename%] with result: [notepad.exe]         [Ignore] IF - [%SystemRoot%\system32\] is not equal to: [] evaluated string: [If,%CPath%,Equal,,Set,#3,%PE_Programs%\%programFolder%\%CFilename%]         [Success] IF - [] is equal to: [] evaluated string: [If,#5,Equal,,StrFormat,path,#3,%My_Shortcut_FOLDER%]         [Success] StrFormat - converted: [#3] to variable: [%My_Shortcut_FOLDER%] with result: [%SystemRoot%\system32\]         [Ignore] IF - [] is equal to: [] evaluated string: [If,#5,NotEqual,,StrFormat,path,#5\,%My_Shortcut_FOLDER%]         [Ignore] IF - [Notepad] is not equal to: [] evaluated string: [If,#4,Equal,,Set,#4,%ProgramTitle%]         [Ignore] IF - [..] is equal to: [..] evaluated string: [If,".#6#7#8#9.",NotEqual,"..",Run,%API%,ShortcutOptions,#1,#2,#3,#4,#5,#6,#7,#8,#9]         [Success] IF - File exists: [%BaseDir%\Projects\nativeEx_barebone_075\Apps\Translate.ini] evaluated string: [If,ExistFile,"%ProjectDir%\Apps\Translate.ini",Begin]               [Success] IniRead - [%BaseDir%\Projects\nativeEx_barebone_075\Apps\Translate.ini] Section [Config] Key [Mode] to variable: [%Translate%] with value [0]               [Ignore] IF - [0] is not bigger than [0] evaluated string: [If,%Translate%,Bigger,0,Run,%API%,_Translate_%Translate%,#1,#2,#3,#4,#5,#6,#7,#8,#9]               [Success] ELSE - executing command: [Run,%API%,AddShortcut_#1,#1,#2,#3,#4]               Run - Processing section: [AddShortcut_Desktop] From file: [%BaseDir%\Projects\nativeEx_barebone\Basic\Build\api.script]                     [Parameter] with parameters: [Desktop],[Productivity],[#$pSystemRoot#$p\system32\notepad.exe],[Notepad]                     Run - Processing section: [Add-Shortcut] From file: [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script]                           [Parameter] with parameters: [DE],[#$pSystemRoot#$p\system32\notepad.exe],[Notepad],[-SW:#$pSystemRoot#$p\system32\]                           [Success] AddVariables - Added section [Variables] from file [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script] into the local variable list                           [Success] IF - File does not exist: [%parse%] evaluated string: [If,NotExistFile,"%parse%",ExtractFile,"%BuildModelScript%","Folder","shortcutParse.exe",%ProjectTemp%]                           [Success] ExtractFile - [shortcutParse.exe] to: [%BaseDir%\Temp\nativeEx_barebone_075]                           [Ignore] IF - Directory exists: [%BaseDir%\Temp\nativeEx_barebone_075\Shortcuts] evaluated string: [If,NotExistDir,%ProjectTemp%\Shortcuts,DirMake,%ProjectTemp%\Shortcuts]                           [Success] AddVariables - Added section [Variables] from file [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script] into the local variable list                           [Ignore] IF - File does not exist: [%BaseDir%\Temp\nativeEx_barebone_075\buildModel_err.log] evaluated string: [If,ExistFile,"%ProjectTemp%\buildModel_err.log",FileDelete,"%ProjectTemp%\buildModel_err.log"]                           [Ignore] IF - [DE] is not equal to: [1] evaluated string: [If,#1,Equal,1,ShellExecute,Hide,"%parse%","#$q/C:%ProjectInfo%#$q #$q/O:%shcTemp%#$q #$q/A:#1#$c#2#$q"]                           [Success] IF - [DE] is not equal to: [1] evaluated string: [If,#1,NotEqual,1,ShellExecute,Hide,"%parse%","#$q/C:%ProjectInfo%#$q #$q/O:%shcTemp%#$q #$q/ST:#1#$q #$q/SE:#2#$q #$q/SL:#3#$q #$q#4#$q"]                           [Success] ShellExecute: [Hide] using: [%BaseDir%\Temp\nativeEx_barebone_075\shortcutParse.exe] with parameters: ["/C:%BaseDir%\Workbench\nativeEx_barebone_075\UserData\ProjectInfo.ini" "/O:%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini" "/ST:DE" "/SE:%SystemRoot%\system32\notepad.exe" "/SL:Notepad" "-SW:%SystemRoot%\system32\"]                           [Success] IniRead - [%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini] Section [Created] Key [Date] to variable: [%tim%] with value [2009-Jan-06 165838406]                           [Success] IniRead - [%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini] Section [Shortcut] Key [shortCutName] to variable: [%nam%] with value [Notepad]                           [Success] FileCopy - Copied [%BaseDir%\Temp\nativeEx_barebone_075\shcTemp.ini] to: [%BaseDir%\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06 165838406.shc]                           Run - Processing section: [Add-Shortcut] From file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]                                 [Parameter] with parameters: [V:\WB\wb#$sdebug\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06#$s165838406.shc]                                 [Success] AddVariables - Added section [Variables] from file [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script] into the local variable list                                 Run - Processing section: [shc-model] From file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]                                       [Parameter] with parameters: [V:\WB\wb#$sdebug\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06#$s165838406.shc]                                       [Ignore] IF - File exists: [%BaseDir%\Target\nativeEx_barebone_075\$ModelRAM\filled] evaluated string: [If,NotExistFile,"%TargetDir%\$ModelRAM\filled",FileCreateBlank,"%TargetDir%\$ModelRAM\filled"]                                       [Ignore] IF - File exists: [%BaseDir%\Temp\nativeEx_barebone_075\shortcutWinExp.exe] evaluated string: [If,NotExistFile,"%ProjectTemp%\shortcutWinExp.exe",ExtractFile,"%ActiveShell%","Shortcuts",shortcutWinExp.exe,%ProjectTemp%]                                       [Success] ShellExecute: [Hide] using: [%BaseDir%\Temp\nativeEx_barebone_075\shortcutWinExp.exe] with parameters: ["/C:%BaseDir%\Workbench\nativeEx_barebone_075\UserData\ProjectInfo.ini" "/I:%BaseDir%\Temp\nativeEx_barebone_075\Shortcuts\Notepad_2009-Jan-06 165838406.shc"]                                 Run - Processed section [shc-model] in file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]                           Run - Processed section [Add-Shortcut] in file: [%BaseDir%\Projects\nativeEx_Core\Basic\Shells\Explorer.Script]                           [Ignore] IF - File does not exist: [%BaseDir%\Temp\nativeEx_barebone_075\buildModel_err.log] evaluated string: [If,ExistFile,"%ProjectTemp%\buildModel_err.log",ShellExecute,Open,"%ProjectTemp%\buildModel_err.log",""]                     Run - Processed section [Add-Shortcut] in file: [%BaseDir%\Projects\nativeEx_core\Basic\Build\4 - buildModel.Script]               Run - Processed section [AddShortcut_Desktop] in file: [%BaseDir%\Projects\nativeEx_barebone\Basic\Build\api.script]         [Success] [End]         [Ignore] ELSE - [Run,%API%,AddShortcut_#1,#1,#2,#3,#4]   Run - Processed section [AddShortcut] in file: [%BaseDir%\Projects\nativeEx_barebone\Basic\Build\api.script]

Peter

On my opinion, I don't think that the fact of using universal scripts is the main factor of slowdown.

What needs improvement in terms of speed optimizations are the functions that are called.

Often used functions like unpack, create shortcut and file associate could be made quite faster if they are cached or written to a text file/cache folder instead of applied directly to the target OS.

Also, if developers keep app scripts simple, things should speed up.


btw: Peter, would it be possible to break some of the lines on your post to make things readable?

Thank you!

btw: Peter, would it be possible to break some of the lines on your post to make things readable?

Maybe that is a 'feature' of the new server. I put the code into a codebox like a always did with 'much and large' code..

And in former times that caused a window with horizontal and vertical scrollbars.

Now there is only a vertical scrollbar in the codebox window.
And the horizontal one is on the page

Peter

Little misunderstanding.
I was not talking about the build time, though it was impressive with way over an hour!
No i'm talking about the running PE.
Even though no new services or application autorun, memory consumption has about doubled, HDD access happens only after a thinking pause, and the CPU is already 15%-30% stressed when only the desktop is shown!

My best guess is that some script royaly screwed up the registry!

Now there is only a vertical scrollbar in the codebox window.

I see both in safe skin. Everything looks fine.

I see both in safe skin. Everything looks fine.

With which browser?

jaclaz

Little misunderstanding.
I was not talking about the build time, though it was impressive with way over an hour!
No i'm talking about the running PE.
Even though no new services or application autorun, memory consumption has about doubled, HDD access happens only after a thinking pause, and the CPU is already 15%-30% stressed when only the desktop is shown!

My best guess is that some script royaly screwed up the registry!

Sorry for my misunderstanding.

This is a REALLY INTERESTING question!

BTW: My first idea has been: ALL the 'minimize space' fans, using excessively UPX and *.sy_ drivers etc.
But it does not matter whether I use 200 or 600 MB of a 650 MB CD.
And If my USB stick has only 256 MB, I buy for 5 € or 5 $ a new one with 1 GB.

Peter

BTW: My first idea has been: ALL the 'minimize space' fans, using excessively UPX and *.sy_ drivers etc.

Yes i love it too, when i see a 1MB exe in PE and discover upon click that it eats up 43MB of RAM!

I can now confirm that we actually have application scripts which hurt PE.
Namely they slow PE down to a crawl.

... interesting ... another argument for LODR-packs

Medevil,

Don't know what's gone wrong for you. I haven't noticed the slowdown or cpu usage you mention. Hopefully you'll find what's the cause.

Regards,
Galapo.

That points to: In Medevil's 'Test PE' are different functionalities than in Galapo's 'Test PE'
(And as far as I understood Medevil, that are not WB74 <> WB75 beta ?. It looks like some new app scripts or app script functionality)
BTW: Why not look at driver packages too? Some HWPNP in 'RunOnce'? ...


To find the differences could help a lot ...


Peter

I did a download in december of LiveXP and though it was nowhere near as catastrophic as my test-build, it reacted still noticable more sluggish than NaughtyPE. Especialy the thinking pause before accessing any drive was there.

I used a complete download, since i wanted to check for new app scripts available for possible NaughtyPE use.

If there is nothing like that in Galapos version, it eighter may have been fixed already (Lancelot is a busy bee! ) or results from my xpsp1 host system, but i thought we were long past this.



PS: 136MB 'windows' folder and 526MB 'Program files' folder in the Test build. That's all the apps scripts i could find.
Half of them are undelete programs! :cheers:

It looks like some new app scripts or app script functionality)
BTW: Why not look at driver packages too? Some HWPNP in 'RunOnce'? ...

Only app scripts were added to a default NaughtyPE. All driver and basics scripts were unchanged.
All errors in the log from new scripts were fixed by replacing api commands with non-api ones.

... interesting ... another argument for LODR-packs

Will LODR-packs prevent Registry writes, or how should they help?

PS: 136MB 'windows' folder and 526MB 'Program files' folder in the Test build. That's all the apps scripts i could find.
Half of them are undelete programs!

As already written at a different place:

Strange!

That reminds me a bit on the ideas I mentioned in my topic about 'LiveXP factory settings' ...

Here it looks like the old Roman 'sed ut utrimque fiat, major part meliorem vicit'

[English]As usually, the bigger party won against the better one [/English]

Peter

Peter

Will LODR-packs prevent Registry writes, or how should they help?

No, there is no
But as long as you do not need the package, it does not exist inside your PE.

And therefore: No registry 'overload' possible.

Peter

The registry database will work fast regardless it has one megabyte or 10 megabytes worth of data.

There must surely be other reason that can be pinpointed as the respective app script gets identified but I don't know what MedEvil is using so it's hard to guess.

Would you please a post a log?

-------

The ISO size might interfer if it slows the read access times and forces the CD to read files that are placed in very distant locations of the CD and the front and back movement might add some extra delay miliseconds if the OS is not optimized.

However, if you're running from an emulator this wouldn't surely apply so I'd point a finger to the hardware recognition software and drivers or services.

On later versions of VistaPE this caused a huge delay on the time it takes to start inside the emulator but working quick on real hardware.

As already written at a different place:

Refers this to the lot of undelete scripts or to the part that i threw all scripts into the same build for checking them out?

The registry database will work fast regardless it has one megabyte or 10 megabytes worth of data.

That's correct.
(On the other hand I remember that I usually reinstall my system once a year. After reinstall the previously 'endless' boot time is going down do some seconds.
And then, from week to week, it is increasing again ...)

But I think that you agree that there may be a difference between one and twenty 'RunOnce', ore between ten or two hundred started services, or ...

Therefore sanbarrow's idea to avoid ANY unnnecessary activity, will never increase boot time. It only can decrease it. Whether time is clearly measurable or not, is a different question, but it it will DECREASE!

Peter

There must surely be other reason that can be pinpointed as the respective app script gets identified but I don't know what MedEvil is using so it's hard to guess.

Would you please a post a log?

Since the build took almost two hours, i'm a bit reluctant to do it over just for a log.
But on the next build i will eighter post a log, if the problem persists or the names of the scripts that were sorted out, if it does not.

Refers this to the lot of undelete scripts or to the part that i threw all scripts into the same build for checking them out?

Maybe to both. It is necessary, on EVERY change, addition, update, ... of a project to think again about factory settings.
And what yesterday has been ok, can become today, just after one new script obsolete.

Peter

That's correct.
(On the other hand I remember that I usually reinstall my system once a year. After reinstall the previously 'endless' boot time is going down do some seconds.
And then, from week to week, it is increasing again ...)

How do you do that? The last time i had to reinstall XP was because my HDD said bye bye and that too quickly!

But I think that you agree that there may be a difference between one and twenty 'RunOnce', ore between ten or two hundred started services, or ...

In theory right, but why would anyone start a service or make a runonce entry, if it wasn't needed for the program?




PS: I noticed, i did change one other script too. I replaced the old autoCompress.script with a new one.
Please someone tell me that the new one does not require WB075.

In theory right, but why would anyone start a service or make a runonce entry, if it wasn't needed for the program?

Just because it is common practice.
Programmers think that users are to dumb to start a service manually - so they let Windows load it at startup.
Or they think user would not tolerate it if their app starts with a little delay because it starts its services on demand only.

Sorry sanbarrow, but the programers you're talking about here, are we, the script developers!
So i think your analysis does not quite fit.