40 replies to this topic

[pscEx]

The current ( C )API contains a lot of comments.

That is usually good for users to understand something about "What's going on?".

But it also slows down the build process:

Every line in a Run or Exec section is processed.

When the line is a comment, nothing happens due to the target, but:

• The line is read and parsed
• Maybe an entry into the log is made

When this is inside an often used API function, theoretically a lot of time is wasted.

Suggestion:

• When releasing a new API version, remove all comments which make test lines inactive.
• Remove syntax explanations / samples at the beginning of the function.

Of course, syntax explanations / samples are very helpful and should not disappear.

Here a way to preserve them.

Sample:

[My-Function]

//Line to explain something

//Line to explain something

//Line to explain something

//Line to explain something

FileCopy, ...

If, ...

Change to

[Comment_1]

//Line to explain something

//Line to explain something

//Line to explain something

//Line to explain something

[My-Function]

FileCopy, ...

If, ...

So the [Comment_1] lines are never processed by the WinBuilder script engine.
Multiple (unused) [Comment_?] sections are no problem.

Peter

[sbaeder]

So the [Comment_1] lines are never processed by the WinBuilder script engine.
Multiple (unused) [Comment_?] sections are no problem.

Will give this a try using the recent version used in win7PE_SE and do timing comparison!

[sbaeder]

I'm not claiming this is definitive, but I did grep and pivot table on the log file from my own personal set of tools inside a Win7PE_SE project to see which "routines" inside the Common_API script file were executed the most. Was kind of surprising...

498		Expand_file2

498		Expand_file2_BasicA

408		Expand_file2_BasicB_Action

128		Expand_FileQ2x86x86

112		AddShortcut2

96		Expand_FileQ2System32_Mui

90		reg_hiveload

90		reg_hiveunload

68		Expand_FileQ2System32

50		Run_FromRam2

42		Try-Delete

36		Depack2

28		Read_ENV2

28		Depack_2_0

28		Read_ENV2_ProjectInfo

28		Delete_File

28		Add_

26		Add_Url2

26		Add_URL_All

24		reg_hiveload_HKU

24		Depack_Archive_2

24		reg_hiveload_HKU_2

22		reg_hiveload_HKLM_2

18		reg_hiveload_HKCU

18		reg_hiveload_HKCU_2

14		CapiCommand

12		reg_hiveload_System

12		Copy_Program

12		CapiCommand_FileDelete

12		CaseSelect_

12		File_Delete_Fast_CMD

12		CaseSelect

12		Add_Pin_Process2

10		reg_hiveload_HKLM

10		reg_hiveload_Default

10		reg_hiveload_Default_2

10		_ScriptInterface_Write

10		Arch_check

10		Arch_Set

9		_ScriptInterface

8		reg_hiveload_Machine_2

8		Add_Auto_Run2

8		reg_hiveload_Machine

8		Add_Pin2_Pin_Read_Items

6		Arch_MultiArch

6		reg_hiveload_Software_2

6		reg_hiveload_Software

4		Depack_2_14

4		RegAdd_boot_NMcfg

4		Depack_2_35

4		Run_FromCD2

4		Arch_SingleArch

4		Add_Pin2_Pin_FindPlace

4		Extract_All_2

2		Add_To_PEShell2

2		reg_hiveload_User_2

2		AddDesktopini_P

2		Add_DesktopIni2

2		AddDesktopini_PNew

2		AddDesktopini_StartMenu2

2		RunFrom_2

2		ConvertVariableToMacro_Post80

2		Association2

2		reg_hiveload_User

2		CapiCommand_VariableToMacro

2		do_associate2

Before we make any changes to the API file, I'd like to get more clarity on how the engine processes it...For example, does it keep an index pointer to which line inside the file is the start of any routine it has "seen" so that if can just "fast forward"? Is it loaded into memory???

Peter (or Nuno?) feel free to PM me with more details if you think this is worth pursuing...

Scott

[sbaeder]

Would be even more interesting to see what type of scripts are using the CAPI for your results. For example, what is using the "expand" function so much?

You would have to parse the log file a bit to see what is calling what...May look at that...For now, just seeing if moving the top few routines higher up in the file makes a difference...these are just the run of the mill application scripts as well as the core scripts that make up Win7PE_SE...

BTW: Is there any chance of posting a copy of the current Common API as a single file, here or somewhere else? I'm having trouble finding it.

I will PM you a copy...Right now, there isn't anything centrally available (AFAIK)...

Right now, from just one "test", it looks like that for things that still are using the official, production release of WB (i.e. 80r3), that it has much less of an impact than it does when using the latest "pre-release" version. On the other hand, I want to get Peter to look at my results as well...

Scott

[al_jo]

For anyone interested, here is one of the the current Common API files, dated 2011.03.14:

I’m also interested in how, why (the reason) and who is updating PedroLe’s Capi script every week?
The attached version is v25rev59
The version from here:
http://galapo.net/ge...mmon_Api.script
is v25rev79

[paraglider]

What is worse is that they are actually deleting commands in the latest version. That should never happen in a common API.

[Wonko the Sane]

What is worse is that they are actually deleting commands in the latest version. That should never happen in a common API.

Maybe because, for the known reasons, the unwritable four letter word (but still IMHO good ) guys are creating a FORK , so there is no sense in the "Common" anymore.
Or if you prefer there will be an "authorized" CAPI on boot-land reboot.pro that will work on all projects coming from here and ANOTHER CAPI on <four letter word> site that will work with "their" projects.

About the utter foolishness of FORKs:
http://reboot.pro/4972/page__st__26


Wonko

[Wonko the Sane]

In my view, the CAPI needs to be re-written. It is not supposed to be some proxy for the winbuilder.exe programme. It is only a script AFAIK, and it should be lean, mean and efficient.

AND actually "common" (this time in the sense of "unique" ) AND §@ç#ing DOCUMENTED!

I mean what is the point of paraglider's efforts in documenting things, if every other day someone changes it or adds or removes something from it?

The CAPI is from what I can understand a variation of the Windows .DLL mechanism, which has ALREADY failed (raise your hand whoever never suffered form versioning nightmare?) even if it was "governed" by a central power (Microsoft).

To translate the concept in my dinosaur's technical language, instead of invoking a command, you invoke *something* that contains the command and that "forwards" the call to that command.
This has the advantage that you don't need a myriad of little files, and everything is "kept in order" inside a big "container" (and at each release this container grows in size)

But if NOONE keeps everything inside the container "in order and tidy", there is a problem.
And if more different people independently add, remove or modify the contents, we have a bigger problem. (CHAOS )

At this point it would be better to "split" each command inside the CAPI into a single .script file and give to the actual project/single .script Author the responsibility of adding it inside his/her project/.script, sure there will be some duplications, but at least things should work...

If you prefer, I find the source of the issues being not revolving around whether the CAPI needs to be rewritten or bettered (which is needed anyway ) , but rather around the way the CAPI (or anything "common") should be managed (please read as "Definitely not how currently it is NOT managed").


Wonko

[MedEvil]

I expect hints and tips for improvement like that, from newbies, who have no clue about the size of our community, but not from seaoned members.
Why are there so many CAPI verrsions floating around? Because since Pedro left, the CAPI is orphaned, and has noone to protect her anymore.
And what are the chances to find a couple of dozend members to take care of scripts, if we can not even find a single one, to take care of the CAPI? Slim to none, i would say.

Stop talking, like we would have dozends and more of willing developers at our disposal.
We have about 10 and those mostly hate to have to share toys or god forbid a plan!

[pscEx]

I'm not claiming this is definitive, but I did grep and pivot table on the log file from my own personal set of tools inside a Win7PE_SE project to see which "routines" inside the Common_API script file were executed the most. Was kind of surprising...

498		Expand_file2

498		Expand_file2_BasicA

...

The Expand_File is called by the macro require_file.

Let's do a test and run in codebox:

[process]

Echo,Require_File

require_file,msimg32.dll

Echo,CopyOrExpand

CopyOrExpand,%Source_sys%\msimg32.dll,%Target_sys%

Echo,Done

In a PE1 project: This generates a log with 25 log lines between the "Require_File" and "CopyOrExpand" echoes.
There is 1 log line between "CopyOrExpand" and "Done"

Now try in a win7 project (with mounted wims). This generates a log with 64 log lines between the "Require_File" and "CopyOrExpand" echoes. 33 of them are comment lines (See my initial post)
There is 1 log line between "CopyOrExpand" and "Done"

IMO: When a script developer writes a new script, and an app needs a certain file, the script developer should either know, from where and how to bring the needed file into the correct target place, or he should think about stopping script developing.

Peter

Attached Files

•  logW7.html   28.81KB   736 downloads
•  log_XP.html   20.08KB   687 downloads

[Wonko the Sane]

Scares me when you shout Wonko.

Wonko

[al_jo]

T

IMO: When a script developer writes a new script, and an app needs a certain file, the script developer should either know, from where and how to bring the needed file into the correct target place, or he should think about stopping script dveloping.

Peter

nice!

Or maybee not!

[pscEx]

The Expand_File is called by the macro require_file.

Just for the record: My concerns about require_file are not new:
http://reboot.pro/27...dpost__p__70002

Peter

[sbaeder]

Just to close the loop here on my limited testing...

Position inside the file makes a small difference - probably not worth all the trouble to try to figure out which routines should be where!

BUT, what does make a big difference is the way WB treats each line it parses (i.e. use the SYSTEM,COMP80,OFF to get the newer parsing routines that aren't 100% downward compatible). THIS made a HUGE difference in the time taken on my typical (??) project (i.e. 45% faster).

As Peter mentions in the the follow up to this on "turbo capi", reducing the number of lines that WB has to "parse" is what counts, not the number of lines it can skip over without any parsing...Based on my limited testing I would agree

Scott

[homes32]

Just for the record: My concerns about require_file are not new:
http://reboot.pro/27...dpost__p__70002

Peter

yes it is slower, but it also does the important job of including wow64 versions of the files if you are building from a x64 source, without any extra coding on the app script developer's part.

[pscEx]

yes it is slower, but it also does the important job of including wow64 versions of the files if you are building from a x64 source, without any extra coding on the app script developer's part.

Here we are!

My projects intentionally work with x86 source CD only.

But the "Common" API gives me all functionalities of x64 development, with the "small" inadvantage of time aspects, and I have no chance to switch this functionality off.

The API is common, and everybody MUST use it

Peter

[pscEx]

BUT, what does make a big difference is the way WB treats each line it parses (i.e. use the SYSTEM,COMP80,OFF to get the newer parsing routines that aren't 100% downward compatible). THIS made a HUGE difference in the time taken on my typical (??) project (i.e. 45% faster).

To avoid misunderstandings:
Build time of the project is about 45%.


Peter

[larioteo]

yes it is slower, but it also does the important job of including wow64 versions of the files if you are building from a x64 source, without any extra coding on the app script developer's part.

Sorry for being late @homes32, if you find the magic capi so usefull than please write here officialy what you guys whant to do with such commands.
- CapiC and its all subcommands
- ReadEnv
- IniVariables
- ...

Maybe you can explain us why it is needet in every script, there are more faster ways to make a project "easy to use". And to forcing everyone to using your stuff is the wrong way in my eyes, winbuilder projects without capi doesnt exist anymore, all appscripts are using senseless commands like as i seen in last time CapiC commands in appscripts, yes very compatible to other not capic using projects.

Its also no secret why you are forcing every project to use capic, after you guys have your winbuilder replacement we all need to use your slow stuff if we are continue using the capi. Thats not the goal in an open development it is rather killing the good hard work of others before you.

HistoryEntries like IniVariables support added isnt good for other project that don't want this or don't have this as example, but... enough from now.

***Removed***

Edited by Darijo, 24 November 2011 - 08:14 AM.

[Brito]

The API for app scripts has a good reason to exist. The commands were made available with the purpose of abstracting app scripts from projects. The idea is that you pick an app script from one project to another and it works without changes. In the past, one app script from one project would certainly not work in another.

So we moved from raw winbuilder commands inside app scripts to API commands and it was a good decision. Several projects have come and been gone, while app scripts written since 2007 remain working on newer projects today.

This allows building up a huge library of app scripts that can be reused in different projects. Going back to raw commands is faster, however forces app scripts to be rewritten when sources changes, when projects changes or whatever else changes. It is easier to modify a project than hundreds of app scripts. Or nor should we need to rewrite tons of app scripts that will just run on a single project, this would just be a waste of time and effort.

-------------

What we saw going wrong was the growing complexity and slug on app scripts. What changed since May to this part is the existence of an officially documented API reference for app scripts: http://code.google.c...iki/wbcommonapi

Right now, Scott, Homes, Peter and other volunteers have been working quite hard in keeping things simple and clear for us all.

There is still a lot of work ahead, however if you look on the current list. You will see that commands whose usefulness was questionable are not listed at all. Only commands proven as simple and necessary were included, following the principle of keeping things as simple as possible.


Hope this helps to understand what changed on this discussion since May, a lot of improvements were made to keep the standards open and simple.

[larioteo]

I rather would see it so, if i provide an Appscript than it is my job to make it working on all projects, if i say it works only on explorer builds than there arent so much left as example.

See it as real windows Setup, the setup.exe should work on all windows versions. and i decide how to manage them not the builder, an appscript should be a standalone script and not bindet to an capi api or something else.

Is it so hard to apply an application that works on every pe version? The main goal should be PE1 2 3 and not Capi v1 2 3

maybe you so understand me better, in my eyes an appscript should be isolated by the main build, the Project has its job provide me as example the perfect base to add my app, and not otherwize.

CAPI isnt so compatible as you all thing, alone the CapiC commands are so sensless that they only have one job bind appscripts on it.

If there would be some people who take it serious for money it could be handled as an extension. But so it is a never finished beta work in my eyes. Many hate that capi stuff i am not alone with my thinking. It has more than an half of so useless commands that you wanna cry when you read its syntax. And the most worsest thing at all it is making the buildtime 3 to 4 times slower, and this is for sure not the goal. If you want to add something nice to a project it takes days to test it, cause the build time is nearly catastrophic currently.

In my thinking there is no place for bindings in a project that is open, everything should be on one site rather a plugin than an one file operation. To deliver Winbuilder and than to say you need the capi to get it to work is also a little bit funny for newcomers. It sounds like

"You all must do what we want and what we think is better for you!"

As an developer you know that everybody thinks different so everybody should have the possibility to choose his way, cause you given the opportunity with winbuilder everybody, but on the other site you are taking it with the capi.

So what is your goal? More possibilities for others, or less, if less than i must say it is time to find something different without such restrictions on the personal mind.

[MedEvil]

The whole idea of the API is in theory good, but in reality nonsense.
Theoretical it let's app scripts run on all projects, but in reality, it has just helped that app scripts are today way worst, then what we used to have before the ©API.

One can't lower the bar these days and expect people to still jump high.
You ask for less, you get less!

[Brito]

Hello, there is some confusion going on.

When you write an app script, it is intended to be abstract from the project or API implementation.

About a year ago you would see some developers pushing an API that was not abstract and much of the code is nowadays unbearable to understand or manage by others. That is not what we want for the future. There is a lot of bloat and unnecessary code that needs to be cleaned away from modern app scripts.

We're working to bring back some simplicity and common good sense, this is a work in progress and more years will pass.

I just wish that more people could see how important it is to keep script code short and simple instead of understanding my comments as personal attacks.

What we saw going wrong was an API being developed to fit projects developed by a given group, and then we also see some other developers proposing to abandon any API and write app scripts that just work in projects by another group.

Neither of these options are good for end users. We write app scripts that are not stuck to any group. It is just stupid to limit app scripts to a single project and force other people rewriting them over and over again. This only works to advantage of a given project, at the cost of making life worse to all users and other projects.

[paraglider]

@Darijo,

I agree with you that CapiC should be removed. Unfortunately many scripts now use it. So you see this sort of thing:

StrFormat,REPLACE,%ScrollBox_RunFromWhere%," ","",%runfrom%
If,Not,%runfrom%,Equal,Rundefault,CapiC,VariableToMacro,%runfrom%,True

which is at best obscure as to what it is doing. Even worse you get in the log:

Improved (sarcasm) ex-builderX can not do this - Can you believe that

Two if statements would achieve the same thing and would be much more obvious what they are trying to achieve.

[MedEvil]

The problem is simply, that we eighter start having a reboot-API and a oven-API and App scripts will not be compatible between those two or we will have one huge slow API encompassing both, like we have now.

I hope, we're all keeping tabs on which projects are still reboot based and which are not anymore.
So that we can discuss reality here and not fantasy!

[homes32]

I'll try and answer your questions best I can...

Sorry for being late @homes32, if you find the magic capi so usefull than please write here officialy what you guys whant to do with such commands.
- CapiC and its all subcommands
- ReadEnv
- IniVariables
- ...

ReadEnv is left over from Nightman VistaPE API. Back in the time where CAPI started In order not to have 2 api's for vistaPE it was merged into CAPI and nightman's API was decommissioned.
I know few scripts/projects that still use it and it is being phased out.

as for CapiC and IniVariable, I don't use them nor endorse them and would be happy to see them gone.

Maybe you can explain us why it is needet in every script, there are more faster ways to make a project "easy to use". And to forcing everyone to using your stuff is the wrong way in my eyes, winbuilder projects without capi doesnt exist anymore,

who's forcing anyone to use CAPI???? I must have missed that memo because I write plenty of scripts with minimal use of CAPI and some without it at all!!! Just because it exists dose not mean you have to use it. feel free to ignore it completely. If you have a better way to do something please share it. otherwise stop complaining.

all appscripts are using senseless commands like as i seen in last time CapiC commands in appscripts, yes very compatible to other not capic using projects.

All???
I see very few app scripts using CapiC. I personaly don't and won't use it in its current form.

Its also no secret why you are forcing every project to use capic, after you guys have your winbuilder replacement we all need to use your slow stuff if we are continue using the capi. Thats not the goal in an open development it is rather killing the good hard work of others before you.

Again I missed the part where anyone is being forced to use the CAPI or any particular command it contains. If you don't like how a particular command is implemented then post a replacement that is better and I will be the first one to start using it.

what winbuilder replacement? did I miss another Memo???