Jump to content











Photo
- - - - -

Commas vs non-commas


  • Please log in to reply
121 replies to this topic

#101 Lancelot

Lancelot

    Frequent Member


  • .script developer
  • 5013 posts
  • Location:Turkiye/Izmir
  • Interests:*Mechanical stuff and Physics,
    *LiveXP, BartPE, SherpyaXPE,
    *Basketball and Looong Walking,
    *Buying outwear for my girlf (Reason: Girls are stupid about buying bad stuff to make themselves uglier :))
    *Girls (Lyric: Girl,...., You will be a womann, Soon)
    *Answering questions for "Meaning of life",
    *Helping people,

    Kung with LiveXP, Fu with Peter :)
  •  
    Turkey

Posted 16 March 2010 - 12:38 AM

Hi Galapo,

no need to write same things again.

Nuno approved cryptic syntax.

End of discussion.

@Nuno

Sure. Let's talk on examples of things that should be changed.


rule 3 is there, approved by you. no quote support and Crpytic syntax is official now.
http://winbuilder.ne...tax.html#syntax

(I hope short enough)

#102 Galapo

Galapo

    Platinum Member


  • .script developer
  • 3841 posts
  •  
    Australia

Posted 16 March 2010 - 01:35 AM

Ok, so I guess the decision then passes to project developers: do they want to rewrite scripts and develop scripts using such syntax, or choose not to follow the WB update train down this path and so just pick a WB version and stick with it into the future?

Regards,
Galapo.

#103 sbaeder

sbaeder

    Gold Member


  • .script developer
  • 1338 posts
  • Location:usa - massachusettes
  •  
    United States

Posted 16 March 2010 - 01:45 AM

Sorry that I have to disappoint you:
in alpha 81.0.0.28 to have the escape rather the comma, does not speed up processing.
Reason: Because the majority of 2 members out of 35000 members demanded, that is not implemented.


Any real data on how much of a change it is? Does it go 100% faster, and is that from 10 minutes to 5 minutes (or from 10 seconds to 5 seconds)...i.e. some data on the benefits of something like this is needed..

#104 Lancelot

Lancelot

    Frequent Member


  • .script developer
  • 5013 posts
  • Location:Turkiye/Izmir
  • Interests:*Mechanical stuff and Physics,
    *LiveXP, BartPE, SherpyaXPE,
    *Basketball and Looong Walking,
    *Buying outwear for my girlf (Reason: Girls are stupid about buying bad stuff to make themselves uglier :))
    *Girls (Lyric: Girl,...., You will be a womann, Soon)
    *Answering questions for "Meaning of life",
    *Helping people,

    Kung with LiveXP, Fu with Peter :)
  •  
    Turkey

Posted 16 March 2010 - 01:49 AM

Nope galapo,

This is one of the prepared ambush you jump in.
If one leaves the project, after a while project is unusable and requires a rescue for the new wb versions.
So far I see only JonF rescue VistaPE project to VistaPE-Capi (and enhanced) (which is originally created by famous Nightman)
and since we are active, we constantly rescue LiveXP (and enhance)

Rest is dust to dust, ashes to ashes until someone comes to rescue (rescue for a while :thumbup:).

new cryptic syntax rules now approved by Nuno
(which can be answered first day we ask with honesty, instead post games, vote games etc. run around that creates only sadness , besides with these experience now I guess we learn more),
besides above is a well known winbuilder development policy.

With the Nuno's approved cryptic rules, we now clearly know projects of today will not work in some time tomorrow again (if not updated constantly), wb will get cryptic for new users in some time.....


Really nothing to write more, All are more clear to me now, nothing to ask, nothing to discuss .

Enjoy your day, Now it is ended :thumbup:

#105 Brito

Brito

    Platinum Member


  • .script developer
  • 10616 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 16 March 2010 - 04:09 AM

new cryptic syntax rules now approved by Nuno

Sure.. :thumbup:

Really nothing to write more, All are more clear to me now, nothing to ask, nothing to discuss .

Somehow I sense that the urge to reply will be stronger than you.. :thumbup:

Now it is ended

Ok, excellent.

Hope we can now keep the tone of this discussion to a more professional level to analyze the facts.

---------------------------------------------------------------------


Context reminder from the first topic:

Let's be professionals and express good valid arguments why each method is better in certain situations than others and possibly identify the weak points as well.




Why is cryptic syntax used?

Commas and "" are some of the reserved parameters that are suggested to be encoded so that the engine can easily distinguish them.

There are wb commands where commas can be inserted without causing troubles because we know that after the first ," the rest of text will belong to the same parameter until the next " is found.

Example: echo,"Hello, I'm Nuno"

However, if for some reason we use echo,"Hello, I'm "Nuno""

Then we will only get as output: "Hello, I'm "

Since there are rare cases where we want to output "", this brought us the need for the #$x: echo,"Hello, I'm #$xNuno#$x" that will output correctly Hello, I'm "Nuno"

It's not a pretty solution but if you really want to output a "" then it's a way that doesn't add too much overhead on the engine to process, otherwise we'd be left trying to guess which "" belonged to where.

---------------------------------------------------------

Cryptic Syntax on the SET command

There are specific situations that drove the need for cryptic syntax. The first one appointed was the SET command because it could contain another command inside the command parameters and this was a terribly confusing situation to deal.

Using cryptic chars allows to distinguish these two parts. However, I have the opinion that the SET command should only be used to set the value of a variable. There are scripting alternatives that completely bypass the need for cryptic chars or complex notations that I will demonstrate in the last section of this topics.




How this affects the SET command

To better understand this impact, I will clarify why the cryptic syntax notation became necessary for the syntax requested by Galapo:
Set,MacroTest,"Exec,%scriptfile%,test",PERMANENT

There are a few problematic details in this code.

The first is the fact that a variable is being designed with the value of another variable which in turn might have it's own set of "" wrapped inside the value.

When the interpreter evaluates the value of "Exec,%scriptfile%,test", if for example %scriptfile% had the value of "c:\wb 080\Projects\LiveXP\test.script" then we would have a problem because the space on the path of c:\wb 080 will forcefully require "" under windows to keep it together.

The interpreter would ultimately understand the parameter from this case as: "Exec," because the rest of the parameter got cut by the second ".

The second problem are commas. As mentioned on the "Hello" example for the echo command. It's ok if we use a comma when we know what is the length of the parameter since it will either be enclosed between "" or it's the last parameter and we'll just copy until the end of line.

However, for Galapo's example:
Set,MacroTest,"Exec,%scriptfile%,test",PERMANENT
We don't know what is the last parameter. Since there is the risk that %scriptfile% breaks the enclosed "" and since there cases where not even "" are used to define when parameter ends and begins, we rely on commas to break the command onto it's distinct parameters. If one of the parameters contains several commas, how will we know when to stop?

Since there is the possibility to use switches in the last position, there is a huge overhead in trying to guess what is what inside the command.

Meaning that encoded chars are used once again to provide some sense on where each parameter begins and ends with a comma or possibly "", resulting in:
Set,MacroTest,Exec#$c%scriptfile%#$ctest,PERMANENT

Galapo and Lancelot publicly expressed their dislike for the above solution and I don't think that anyone enjoys it as well.

But I hope to have explained how the above notation came to appear as necessary since it's an overly complex combination.



Why are cryptic chars really needed in SET?

If we try to adopt this:
Set,MacroTest,"Exec,%scriptfile%,test",PERMANENT

It will not work well and be vulnerable to bugs and processing overhead which I think that most people will now understand how they occur when an unknown number of "" and commas appear inside a command parameter.

This is the reason that drives the need for encoding them in the first place.

I don't like it, but it's necessary on this case.

However, keep in mind that we are talking about the usage of SET on a complex case for a purpose to which it was not originally coded for.

--------------------------------------------



But how about an alternative?

Instead of fighting between a syntax prune to bugs or being forced to apply an overly complex syntax, I would propose an equivalent code that could be placed inside the script.project file of the project:
[variables]

MacroTest=Exec,%scriptfile%,test

This example had been mentioned before, and although it's not a perfect solution, it does solve the issues associated with the processing of an overly complex syntax and is compatible with older wb versions as demonstrated on the API commands since 2007.

So, sometimes we already had simpler ways of getting the same result that require neither SET nor cryptic syntax.

-----------------------------------



Reflection about the SET command

Please let me hear your thoughts about the SET command. I think I've covered most of the reasons why and when the cryptic syntax is needed to get some sense on things.

Let's not get extremists about the use of cryptic chars and claim that a more strict syntax will look something like Echo#$cHello#$c#$sI'm#$shere! and please don't add any more posts saying that "a hundred years ago it was written on the magical stone that cryptic chars shall not be adopted" or something like that.

That's not our goal, we also code scripts by hand just like humans do but there are times when they're needed and there are times when you just need to use your head to simplify the code by yourself without requiring rambo syntax on wb commands.. :thumbup:

---------------------

If there are any other wb commands that are troubling your mind, let's hear them but please keep the tone of professionalism and provide valuable facts to work and find solutions.

Thank you for reading.

:thumbup:

#106 Lancelot

Lancelot

    Frequent Member


  • .script developer
  • 5013 posts
  • Location:Turkiye/Izmir
  • Interests:*Mechanical stuff and Physics,
    *LiveXP, BartPE, SherpyaXPE,
    *Basketball and Looong Walking,
    *Buying outwear for my girlf (Reason: Girls are stupid about buying bad stuff to make themselves uglier :))
    *Girls (Lyric: Girl,...., You will be a womann, Soon)
    *Answering questions for "Meaning of life",
    *Helping people,

    Kung with LiveXP, Fu with Peter :)
  •  
    Turkey

Posted 16 March 2010 - 04:30 AM

Nuno,

I know the "story" of changing wb handling after 074 all these 2 years. (even without being a programmer, check here, post 30 "clever")

Keeping short (I do not want to reply line by line or write details on below things):

The Story WAS, when psc decided to change wb processing from quote " detection to the method you introduce above, We already knew there will be lots of troubles reason well explained by you, that is the reason we warned him and did not like the idea seeing one day cryptpic writing will be mandatory. At that times psc wrote (PUBLICKLY) it will not..... Dust to dust.

And we came today, what "all already did not want" and "confirmed it will not happen" came true: "Cryptic syntax" usages inserted to rules after these years. Bravo to psc. We clearly know what will be coming in next year.

Yes we get "post games" in professional level which is very far away from our "honest communication" in professional level. Thanks a lot for that, now everything is much more clear for us.

All I want is I do not want to reply FALSE accusations and/or post games anymore from neither you nor psc. I hope clear.

Like I wrote before to Galapo,
Enjoy your day, Now it is ended :thumbup:

#107 Galapo

Galapo

    Platinum Member


  • .script developer
  • 3841 posts
  •  
    Australia

Posted 16 March 2010 - 05:13 AM

Nope galapo,

This is one of the prepared ambush you jump in.
If one leaves the project, after a while project is unusable and requires a rescue for the new wb versions.
So far I see only JonF rescue VistaPE project to VistaPE-Capi (and enhanced) (which is originally created by famous Nightman)
and since we are active, we constantly rescue LiveXP (and enhance)

Rest is dust to dust, ashes to ashes until someone comes to rescue (rescue for a while :thumbup:).

Sadly, I guess you're right.

I have no desire to keep updating LiveXP (or any other project for that matter) to conform more and more to cryptic syntax. If that's the future for WB syntax, then it's not one I wish to participate in.

Any other projects like LiveXP which don't wish to maintain their project in this manner will need to open their project up to developers who do if it is to be maintained.

If anyone wants to continue maintaining LiveXP to keep it up-to-date with WB and bringing it into conformity with the cryptic syntax rules, I'd love to hear from you. Contact me or Lancelot by PM and we can supply you with the ftp details for maintaining the project in line with the syntax progress WB is moving towards.

Thanks,
Galapo.

#108 wendy

wendy

    Frequent Member


  • Lady
  • 290 posts
  • Location:one mile from the QR main line
  • Interests:Operating systems, Weights and Measures, Geometry
  •  
    Australia

Posted 16 March 2010 - 09:20 AM

You could add field separators that are evaluated at each line.

The corresponding rexx command is

PARSE VAR var 1 sc 2 field1 (sc) field2 (sc) ...

The lines like ',My data,001,' and #My data#001# give the same results, since in the first case, sc=',' and field1 = 'My data', field2 = '001'. In the second case, all is the same, except sc = '#'.

This allows for one to avoid -any- escape character inside data. I use this sort of setup, along with doing things like "variables start with y, z", to use in my command writing script 'weave.rex'.

#109 paraglider

paraglider

    Gold Member


  • .script developer
  • 1743 posts
  • Location:NC,USA
  •  
    United States

Posted 16 March 2010 - 12:35 PM

@Nuno,

I think all you are proving with your examples are that the scripting engine is buggy in its current incarnation. No one is suggesting that it is an easy job to parse the script lines when they contain " but it should be possible.

Of course its possible to code round bugs in the engine. But I don' think that is the point.

#110 TheK

TheK

    Frequent Member


  • Advanced user
  • 141 posts
  • Location:Germany (BW)
  •  
    Germany

Posted 16 March 2010 - 03:48 PM

How this affects the SET command

To better understand this impact, I will clarify why the cryptic syntax notation became necessary for the syntax requested by Galapo:

Set,MacroTest,"Exec,%scriptfile%,test",PERMANENT

There are a few problematic details in this code.

The first is the fact that a variable is being designed with the value of another variable which in turn might have it's own set of "" wrapped inside the value.

When the interpreter evaluates the value of "Exec,%scriptfile%,test", if for example %scriptfile% had the value of "c:\wb 080\Projects\LiveXP\test.script" then we would have a problem because the space on the path of c:\wb 080 will forcefully require "" under windows to keep it together.

The interpreter would ultimately understand the parameter from this case as: "Exec," because the rest of the parameter got cut by the second ".

The second problem are commas. As mentioned on the "Hello" example for the echo command. It's ok if we use a comma when we know what is the length of the parameter since it will either be enclosed between "" or it's the last parameter and we'll just copy until the end of line.

However, for Galapo's example:
Set,MacroTest,"Exec,%scriptfile%,test",PERMANENT
We don't know what is the last parameter. Since there is the risk that %scriptfile% breaks the enclosed "" and since there cases where not even "" are used to define when parameter ends and begins, we rely on commas to break the command onto it's distinct parameters. If one of the parameters contains several commas, how will we know when to stop?

Since there is the possibility to use switches in the last position, there is a huge overhead in trying to guess what is what inside the command.

Meaning that encoded chars are used once again to provide some sense on where each parameter begins and ends with a comma or possibly "", resulting in:
Set,MacroTest,Exec#$c%scriptfile%#$ctest,PERMANENT

Galapo and Lancelot publicly expressed their dislike for the above solution and I don't think that anyone enjoys it as well.

But I hope to have explained how the above notation came to appear as necessary since it's an overly complex combination.



Why don't you simply use a new command like this:

SetMacro,MacroTest,PERMANENT,"Exec,%scriptfile%,test"

Then you know exactly where the last parameter begins and where it ends.

Another way would be to split the line into the parameters BEFORE evaluating the variables.

Split:
Set

MacroTest

Exec,%scriptfile%,test  <- now you can remove the enclosing quotes

PERMANENT

Evaluate:
Set

MacroTest

Exec,&#34;c&#58;\wb 080\Projects\LiveXP\test.script&#34;,test

PERMANENT


#111 Brito

Brito

    Platinum Member


  • .script developer
  • 10616 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 16 March 2010 - 06:24 PM

PARSE VAR var 1 sc 2 field1 (sc) field2 (sc) ...

This is an interesting approach. But then the same problem would appear when people attempt to insert another command as the value for a variable.

Or I might be misunderstanding correctly what you've explained, if this is the case would you please point to a site that you consider comprehensive for newb in rexx like me? :thumbup:

Why don't you simply use a new command like this:

SetMacro,MacroTest,PERMANENT,&#34;Exec,%scriptfile%,test&#34;
This is a good suggestion and I like the name that you choose for the wb command. I had also thought about a new command but this would not provide direct compatibility with older versions.

This was the reason why I proposed a code that would do the same without adding more commands. This way there is no need to learn new syntax or hog the script interpreter with another possible command.

Another way would be to split the line into the parameters BEFORE evaluating the variables.

We also thought about that, but inside the single parameter how would know where ends if the parameter itself contains several commas?

You see, the comma is a reserved char. If after the Exec we find a comma then it will be understand as the end of parameter. We can use "" but in the cases as mentioned before there is also an huge array of possibilities to make things go wrong.

It's simply simpler if people don't use the SET command for that type of usage.



Of course its possible to code round bugs in the engine. But I don' think that is the point.

I didn't code round any bugs in the engine, this is the normal functioning of the syntax logic.

I just provided a simpler way of getting things done without using encoded chars.

No one is suggesting that it is an easy job to parse the script lines when they contain " but it should be possible.

On this case it's not feasible to attempt guessing what is inside the parameters, we should draw a line regarding the functionality pushed from SET to keep it simple. At least this is my opinion.

On PHP the use of a reserved char is solved with echo "Hello, I'm 'Nuno'..";

This subtle nuance between " and ' allows to overcome this type of issue. But in PHP (or most other languages I know), they don't have a set-like command with this type of scripting power, now I kind of figure why.. :thumbup:

#112 pscEx

pscEx

    Platinum Member




  • Amount:

    Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 16 March 2010 - 07:34 PM

@TheK, Paraglider, Wendy, Nuno:

You discuss about some theory which is not an issue in the current WinBuilder world.

Have a look at this post.

With the current alpha version ALL "Cryptic" or "Lancelot syntax" inputs seem to be posssible.

If there is still a trouble, a simple post (Please not with the title "inpredictable" nor with "Looooooooooong" nor with "Many Many Many Many Many Many Many Times", that could lead to ignorance) possibly can help.

Peter

#113 Galapo

Galapo

    Platinum Member


  • .script developer
  • 3841 posts
  •  
    Australia

Posted 16 March 2010 - 11:06 PM

SetMacro,MacroTest,PERMANENT,&#34;Exec,%scriptfile%,test&#34;
This is a good suggestion and I like the name that you choose for the wb command. I had also thought about a new command but this would not provide direct compatibility with older versions.

Well, it's nothing new -- I made the same suggestion back at the beginning of March: http://www.boot-land...?...ost&p=90523.

There are a few problematic details in this code.

The first is the fact that a variable is being designed with the value of another variable which in turn might have it's own set of "" wrapped inside the value.

When the interpreter evaluates the value of "Exec,%scriptfile%,test", if for example %scriptfile% had the value of "c:\wb 080\Projects\LiveXP\test.script" then we would have a problem because the space on the path of c:\wb 080 will forcefully require "" under windows to keep it together.

The interpreter would ultimately understand the parameter from this case as: "Exec," because the rest of the parameter got cut by the second ".

The second problem are commas. As mentioned on the "Hello" example for the echo command. It's ok if we use a comma when we know what is the length of the parameter since it will either be enclosed between "" or it's the last parameter and we'll just copy until the end of line.

However, for Galapo's example:

Set,MacroTest,&#34;Exec,%scriptfile%,test&#34;,PERMANENT
We don't know what is the last parameter. Since there is the risk that %scriptfile% breaks the enclosed "" and since there cases where not even "" are used to define when parameter ends and begins, we rely on commas to break the command onto it's distinct parameters. If one of the parameters contains several commas, how will we know when to stop?

Since there is the possibility to use switches in the last position, there is a huge overhead in trying to guess what is what inside the command.

Meaning that encoded chars are used once again to provide some sense on where each parameter begins and ends with a comma or possibly "", resulting in:
Set,MacroTest,Exec#$c%scriptfile%#$ctest,PERMANENT

Galapo and Lancelot publicly expressed their dislike for the above solution and I don't think that anyone enjoys it as well.

But I hope to have explained how the above notation came to appear as necessary since it's an overly complex combination.


Well, the syntax I posted I could happily modify to overcome your path containing spaces issue and the fact that I posted %scriptfile% without enclosed quotes is really only reflective that in some sense we've gotten used to some scripting without quotes. To overcome your issue we could simply write the line of code as:
Set,MacroTest,&#34;Exec,&#34;%scriptfile%&#34;,test&#34;,PERMANENT

I could tolerate this:
Set,MacroTest,&#34;Exec,#$q%scriptfile%#$q,test&#34;,PERMANENT

But I guess it's all irrelavant now that WB development has expressed their desire to remove quote syntax in line with the edited syntax page (and I guess the rejection of the agreement on script syntax that had been reached) and mandate the use of escapes.

Regards,
Galapo.

#114 sbaeder

sbaeder

    Gold Member


  • .script developer
  • 1338 posts
  • Location:usa - massachusettes
  •  
    United States

Posted 17 March 2010 - 01:26 AM

Hope we can now keep the tone of this discussion to a more professional level to analyze the facts.

A M E N ! ! !

I think this was a very good posting describing the issue! But, I'm not sure I agree 100% with all the conclusions...

It's not a pretty solution but if you really want to output a "" then it's a way that doesn't add too much overhead on the engine to process, otherwise we'd be left trying to guess which "" belonged to where.

OK, so far so good...IF a quote is a special character that can be used to "mask" the comma so that it is NOT a command delimiter, then YES, we need to "escape" the quote so that we know it is a "literal, and not something that is creating a single string. Same thing for a SPACE when it is inside a path...it has to have quotes so we know the space is not a delimiter.

There are specific situations that drove the need for cryptic syntax. The first one appointed was the SET command because it could contain another command inside the command parameters and this was a terribly confusing situation to deal.

Here is where "maybe" I begin to diverge...YES, it can be "confusing" if there is a bug in the specification of the command...BUT, we should be able to parse it *IF* constructed correctly.

Using cryptic chars allows to distinguish these two parts. However, I have the opinion that the SET command should only be used to set the value of a variable. There are scripting alternatives that completely bypass the need for cryptic chars or complex notations that I will demonstrate in the last section of this topics.

Yes, there is the example you gave. And I think that is certainly a viable option...But

When the interpreter evaluates the value of "Exec,%scriptfile%,test", if for example %scriptfile% had the value of "c:\wb 080\Projects\LiveXP\test.script" then we would have a problem because the space on the path of c:\wb 080 will forcefully require "" under windows to keep it together.

So, I would have said (in this case) that the specification of the variable %scriptfile% had an incorrect value, since the quotes inside it are being used inside something that was in fact already quoted...So, shouldn't the value of the variable be enclosed in the #$x ?

But eventually there is the issue of "recursion" here. How many times can you embed something into a variable and have it be "expanded"...what if the command set into the variable also had another variable, etc...somewhere this has to stop, or you get into all sorts of LISP like processing issues...

By the way, any insight into how that sequence was chosen...I sort of can understand #c for a comma (and then maybe even ## for a single #, but why 3 characters?

The second problem are commas. As mentioned on the "Hello" example for the echo command. It's ok if we use a comma when we know what is the length of the parameter since it will either be enclosed between "" or it's the last parameter and we'll just copy until the end of line.

However, for Galapo's example:

Set,MacroTest,&#34;Exec,%scriptfile%,test&#34;,PERMANENT
We don't know what is the last parameter. Since there is the risk that %scriptfile% breaks the enclosed "" and since there cases where not even "" are used to define when parameter ends and begins, we rely on commas to break the command onto it's distinct parameters. If one of the parameters contains several commas, how will we know when to stop?

Since there is the possibility to use switches in the last position, there is a huge overhead in trying to guess what is what inside the command.

I thinik this pertains to the recursion question above. To me, the answer is more that you parse the command into parameters FIRST. So, in the example, we get
P1 = Set  - i.e. the command

P2 = MacroTest - i.e. the variable to set

P3 = Exec,%scriptfile%,test - the content of the variable

P4 = PERMANENT
This way, there is no expansion of the variable scriptfile here, since it is INSIDE the quotes. OR if you want to allow it (and that is a choice that can be made either way), you expand it as you are saving the string assigned to the variable. Choice depends on if you want this to be interpreted later on with a change-able content of the variable (delayed binding)...

So I would be OK with either of the choices...Make variables be a section, and that is sort of a static binding of the content of an embedded variable is used or in place as a dynamic binding. But, I think that with some sanity on the parsing, we can avoid any ambiguity, etc...

I tend to agree that keeping the syntax simple, even if a bit restrictive, is much better for the "average" person. Afterall, not all of us want to program in APL, or even LISP...this should be a SIMPLE scripting language with some simple commands to allow building GUI interfaces, writing files to disk, and the other tasks related to creating working PE environments...do we really need complex variable substitution?

If it means keeping the syntax simple, then I am all for NOT adding in the cases where a variable is embedded inside another expression...

I now return you to your previous ranting :thumbup: (i.e. that's supposed to be a JOKE!)

Scott

#115 Galapo

Galapo

    Platinum Member


  • .script developer
  • 3841 posts
  •  
    Australia

Posted 17 March 2010 - 01:46 AM

I thinik this pertains to the recursion question above. To me, the answer is more that you parse the command into parameters FIRST. So, in the example, we get

P1 = Set  - i.e. the command

P2 = MacroTest - i.e. the variable to set

P3 = Exec,%scriptfile%,test - the content of the variable

P4 = PERMANENT
This way, there is no expansion of the variable scriptfile here, since it is INSIDE the quotes. OR if you want to allow it (and that is a choice that can be made either way), you expand it as you are saving the string assigned to the variable. Choice depends on if you want this to be interpreted later on with a change-able content of the variable (delayed binding)...


Hi Scott,

To fill you in on the current state, there are currently two possibilities to the general issue you raise above.

First is an example like this:
Set,MacroTest,&#34;Exec,%scriptfile%,test&#34;,PERMANENT
This expands the variable %scriptfile% into the macro MacroTest and writes the expanded path to script.project. When MacroTest macro is executed, it attempts to execute the section [test] in the script from which the MacroTest macro was initially set. So in script.project you'll get something like this under [variables]
MacroTest=Exec,c&#58;\WB\Projects\LiveXP\Apps\testscript.script,test

A second example is like this:
Set,MacroTest,&#34;Exec,#$pscriptfile#$p,test&#34;,PERMANENT
This does not expand a variable into the macro MacroTest but instead writes the string %scriptfile% to script.project. Thus when MacroTest macro is executed, it attempts to execute the section [test] not from the script from which the MacroTest macro was initially set but instead attempts to execute the section [test] from script.project. So in script.project this time under [variables] you will find something like this:
MacroTest=Exec,%scriptfile%,test
In the case the #$p escapes are understandable and perform the job well.

Regards,
Galapo.

#116 sbaeder

sbaeder

    Gold Member


  • .script developer
  • 1338 posts
  • Location:usa - massachusettes
  •  
    United States

Posted 18 March 2010 - 03:05 PM

Thanks - again, this helps in really understanding the "syntax"

Set,MacroTest,&#34;Exec,%scriptfile%,test&#34;,PERMANENT
This expands the variable %scriptfile% into the macro MacroTest and writes the expanded path to script.project. When MacroTest macro is executed, it attempts to execute the section [test] in the script from which the MacroTest macro was initially set. So in script.project you'll get something like this under [variables]
MacroTest=Exec,c&#58;\WB\Projects\LiveXP\Apps\testscript.script,test

So, the variable substitutions happen even inside the quotes string of a single parameter...BUT, again, the question I thought Nuno was asking was how did it parse the parameters when things like this happened...So, my question still is if this is a parameter parsing issue or not. If the original parameters are split up FIRST, then it shouldn't be an issue with parameter splitting...

Set,MacroTest,&#34;Exec,#$pscriptfile#$p,test&#34;,PERMANENT
This does not expand a variable into the macro MacroTest but instead writes the string %scriptfile% to script.project. Thus when MacroTest macro is executed, it attempts to execute the section [test] not from the script from which the MacroTest macro was initially set but instead attempts to execute the section [test] from script.project. So in script.project this time under [variables] you will find something like this:
MacroTest=Exec,%scriptfile%,test
In the case the #$p escapes are understandable and perform the job well.

Got it, so the parsing has some ordering to the processing (since it is a program after all), and we have a clear precedence that should be clearly documented - i.e. the the processing and expansion of a variable happens before we go in and transform the escaped characters.

Again, this makes sense, and if there is a "bug" due to someone using either the % or the " instead of the escaped characters, then that is just a bug in the logic...

So, maybe I still don't "get it", so putting aside all the nonsense and finger pointing, is there someone who can really point out the difficulty in the above situation. We need to document the flow of the expansions and when things get turned from a special character
, - parameter delimiter

% - indicates a variable

&#34; - indicates a string that is not split up or parsed in the first pass
???did I miss any???

#117 pscEx

pscEx

    Platinum Member




  • Amount:

    Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 18 March 2010 - 04:14 PM

, - parameter delimiter

% - indicates a variable

&#34; - indicates a string that is not split up or parsed in the first pass
is correct, but not complete.
The "first pass" splits arguments by commas, which are outside a quote pair.
Additionally: Then it replaces in every found argument comma and space by #$c and #$s for further processing (still commas and spaces disturbe in the later processing!)
Additionally: Then it removes all surrounding quotes around the found arguments (they disturbe later processing).

Now the historical problem:

To write "%var%" (with quotes) in order to tell WinBuilder, that inside are commas / spaces wich demand a special processing, does not help, because the quotes are removed in the very first pass (appearently '%var%' has no spaces or commas, and the content is unknown at this time).

Of course it is possible to change this, but that would demand a complete redesign of the script engine, and would lower the backwards compatibility remarkably.

BTW: The %ScriptFile% sample Galapo gave, would work "on your system as author of the script" in both cases, but if you want to publish your script, you should use the #$pScriptFile#$p version to allow your script to run on every system, which usually has directory structures different from your system.

So, the variable substitutions happen even inside the quotes string of a single parameter...BUT, again, the question I thought Nuno was asking was how did it parse the parameters when things like this happened...So, my question still is if this is a parameter parsing issue or not. If the original parameters are split up FIRST, then it shouldn't be an issue with parameter splitting...

You are right, Nuno did see this wrong.
The additional flags like PERMANENT, NOWARN, PRESERVE, etc. are written into an internal variable with
TCommandFlag = &#40;

	flgNone,

	flgNoWarn,

	flgPreserve,

	flgPermanent,

	flgGlobal,

	flgWarn,

	flgNoRec,

	flgShow,

	flgFast,

	flgAppend,

	flgPrepend,

	flgDelete,

	flgPlace,

	flgBefore,

	flgBehind,

	flgIndex

	&#41;;
and then deleted.
So they never would disturbe the parsing.

Peter

Edited some typos and formulations

#118 sbaeder

sbaeder

    Gold Member


  • .script developer
  • 1338 posts
  • Location:usa - massachusettes
  •  
    United States

Posted 18 March 2010 - 10:59 PM

First, thanks for explaining all this, hopefully we shed more light on the issues...and it can all be eventually added to the docs so that the syntax parsing is better understood...

, - parameter delimiter

% - indicates a variable

&#34; - indicates a string that is not split up or parsed in the first pass
is correct, but not complete.

which is why I asked :thumbup: "Seek first to understand" :)

The "first pass" splits arguments by commas, which are outside a quote pair.
Additionally: Then it replaces in every found argument comma and space by #$c and #$s for further processing (still commas and spaces disturbe in the later processing!)

Does this include inside a quoted arguement and is it ONLY comma and space?

Additionally: Then it removes all surrounding quotes around the found arguments (they disturbe later processing).

So, things like single quote (apostrophe) and all the other special characters aren't touched at this point, only the double quote is removed - right?

Now the historical problem:

To write "%var%" (with quotes) in order to tell WinBuilder, that inside are commas / spaces which demand a special processing, does not help, because the quotes are removed in the very first pass (apparently '%var%' has no spaces or commas, and the content is unknown at this time).

No, I was thinking more that inside the variable would be the #$q so that they aren't removed in later processing, but still signify that there is a string that might have spaces in it, etc.

Of course it is possible to change this, but that would demand a complete redesign of the script engine, and would lower the backwards compatibility remarkably.

OK, I can understand that, and will take it as a given, but sometimes to move forward, we have to make larger changes...Not saying this is one of them, but maybe???

BTW: The %ScriptFile% sample Galapo gave, would work "on your system as author of the script" in both cases, but if you want to publish your script, you should use the #$pScriptFile#$p version to allow your script to run on every system, which usually has directory structures different from your system.

Can you elaborate more here? How does a different directory structure affect the processing of the variable syntax? Is it because the % character might be used differently?

You are right, Nuno did see this wrong.
The additional flags like PERMANENT, NOWARN, PRESERVE, etc. are written into an internal variable with

TCommandFlag = &#40;

	flgNone,

	flgNoWarn,

	flgPreserve,

	flgPermanent,

	flgGlobal,

	flgWarn,

	flgNoRec,

	flgShow,

	flgFast,

	flgAppend,

	flgPrepend,

	flgDelete,

	flgPlace,

	flgBefore,

	flgBehind,

	flgIndex

	&#41;;
and then deleted.
So they never would disturb the parsing.

So, if it isn't about the parsing of the command parameters, how do the spaces and commas affect things and why do they need to be transformed into the escaped sequences above?

Again, thanks for the technical background on all of this...

Scott

#119 niche99

niche99

    Frequent Member


  • Advanced user
  • 235 posts
  • Location:Aberdeen, Scotland
  •  
    Scotland

Posted 29 November 2010 - 08:11 PM

A laymen's viewpoint.

There are 2 issues here.
1. Making the WinBuilder engine faster.
2. Allowing easy use of the engine by it's users.

Peter is arguing for syntax changes that produce for 1.
Lancelot, Galapo and I would say myself too are arguing for syntax that produces 2.

Is Peter (the engineer) designing and implementing a system that makes his life, and others like him, easier by producing faster testing runs of the system?

If the syntax parsing engine cannot be re-done then surely the point of an API is to make the underlying complexity of syntax invisible to the script writer. Does that not imply pre-processing anyway? Isn't WinBuilder just an interpreter and aren't they always slower anyway? I don't mind a build taking 20 minutes or more if it builds reliably every time.

I can tell you that if you want the community to engage with WinBuilder, it has to be a simple matter to write scripts with minimal debugging. Obfusticating the language used to write the scripts will just put potential script developers off and the comunity will eventually die.

The question Nuno and Peter want to ask themselves is how big and successful do you want WinBuilder to become? If you want WinBuilder to grow make it simpler to use.

Regards,
niche99

#120 pscEx

pscEx

    Platinum Member




  • Amount:

    Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 29 November 2010 - 08:33 PM

A laymen's viewpoint.

There are 2 issues here.
1. Making the WinBuilder engine faster.
2. Allowing easy use of the engine by it's users.

Peter is arguing for syntax changes that produce for 1.
Lancelot, Galapo and I would say myself too are arguing for syntax that produces 2.

Is Peter (the engineer) designing and implementing a system that makes his life, and others like him, easier by producing faster testing runs of the system?

If the syntax parsing engine cannot be re-done then surely the point of an API is to make the underlying complexity of syntax invisible to the script writer. Does that not imply pre-processing anyway? Isn't WinBuilder just an interpreter and aren't they always slower anyway? I don't mind a build taking 20 minutes or more if it builds reliably every time.

I can tell you that if you want the community to engage with WinBuilder, it has to be a simple matter to write scripts with minimal debugging. Obfusticating the language used to write the scripts will just put potential script developers off and the comunity will eventually die.

The question Nuno and Peter want to ask themselves is how big and successful do you want WinBuilder to become? If you want WinBuilder to grow make it simpler to use.

Regards,
niche99

The post is logically OFFTOPIC, because I stopped development of WinBuilder two month ago, and currently there is nobody who could do something for #1 or #2. (Because of the server crash, the SVN content is several month older than my personal WinBuilder source folder)

Peter

#121 niche99

niche99

    Frequent Member


  • Advanced user
  • 235 posts
  • Location:Aberdeen, Scotland
  •  
    Scotland

Posted 29 November 2010 - 08:57 PM

The post is logically OFFTOPIC, because I stopped development of WinBuilder two month ago, and currently there is nobody who could do something for #1 or #2. (Because of the server crash, the SVN content is several month older than my personal WinBuilder source folder)

Peter


Been very busy these past 6 months myself and haven't had time to plumb the depths of the Forum.
Sorry to hear you have stopped developing WinBuilder. Are you still developing the NativeEx projects?

Regards,
niche99

#122 pscEx

pscEx

    Platinum Member




  • Amount:

    Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 29 November 2010 - 09:12 PM

Been very busy these past 6 months myself and haven't had time to plumb the depths of the Forum.
Sorry to hear you have stopped developing WinBuilder. Are you still developing the NativeEx projects?

Regards,
niche99

nativeEx_Core, which is the base for e.g. NaughtyPE, BartPECore, LiveXP is also stopped.

I'm currently working on a new project EasyXP as a subproject of nativeEx_easyPE, which can process also source CDs with non-ansi-languages.

But that is an independent 'PE producing' project, not an 'experimental' project which can be used as base for other projects, like nativeEx_Core has been.

Peter




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users