Jump to content











Photo
- - - - -

Writing to the Grub4dos' keyboard buffer possible?

grub4dos

  • Please log in to reply
110 replies to this topic

#76 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 17 July 2021 - 12:19 AM

The board is set for giving limited capabilities to posters will less than 50 posts (this is because of spammers editing previous posts to add links to crappy sites), you are near to reach that number of posts, from then on you will have no more time limits on editing previous posts.

Thanks a lot, now this is clear to me.

 

The %* is ALL parameters excluded the %0 one.

 

In batch you have the shift command to manipulate parameter numbers/order, but the grub4dos implementation does not accept shift -1 if I recall correctly (not that it is usually needed).

Good to know about %*.

 

What I do not like so much about using shift in the main routine is losing arguments (with regard to in this 'function'-project). And if there are many arguments, it is a waste of variables, needed to hold the values of the arguments.

 

In one of my projects I was because of this *thing* condemned to endless debugging with varsleft.g4b (I believe it's from steve6375 - no author 'inside'. It's a very handy debugging-tool).

 

I experimented with call :%1 %* - seems to give no problems, I only have to test if shift inside a sub-routine does the same, without losing anything in the main routine.

 

I have been thinking about the dll-'idea' I should 'not attribute' to you.

To give this sort of call's a more unified structure, should their not be error-handling too? Although I have only vague notions of, for instance, the windows API, I know many different error-codes are needed.

 

To exemplify I made a function I really need: counting number of labels. My latest count (105 labels) took me a very long time, and I was not even sure about the final number. So I made the function 'cntlabel'. Still not very high-level, just meant to give an idea about a possible general structure.

!BAT
#FUNCTION.G4B by deomsh; thanks to Wonko the Sane
#DLL for Grub4Dos
#Arguments: function, mandatory arguments, optional arguments; errormsg: 0=No function given, 1=Function not supported
#Implemented: cntlabel (20210717)
debug off
setlocal && set *
if "%~1"=="" && set /a errormsg=0 > nul
if not exist errormsg && call :dbfunc %~1
if not exist errormsg && call :%~1 %*
endlocal && set result=%result% && set errormsg=%errormsg%
echo %1: result=%result% errormsg=%errormsg%
goto :eof

:cntlabel #Function: count number of labels; Argument: FILE; Optional: dos or mac; errormsg: 2=FILE not given, 3=FILE does not exist
setlocal && set *
if "%~2"=="" && endlocal && set /a errormsg=2 > nul && goto :eof
if not exist %~2 && endlocal && set /a errormsg=3 > nul && goto :eof
if /i "%~3"=="dos" && set EOL=\x0D\x0A ! set EOL=\x0A
if /i "%~3"=="mac" && set EOL=\x0D
echo EOL=%EOL%
cat --locate=%EOL%\x3A %~2 > nul &; set /a result=%@retval% > nul
endlocal && set result=%result%
goto :eof

:dbfunc #database of available functions
setlocal && set *
if not /i %~1==cntlabel && endlocal && set /a errormsg=1 > nul && goto :eof
endlocal
goto :eof

This is the 'print-screen' of the output (the optional argument gives also more possible uses of cntlabel):

grub> (hd0,0)/function.g4b
: result: errormsg=0 
grub> (hd0,00/function.g4b cntlabe 
cntlabe: result= errormsg=1 
grub> (hd0,0)/function.g4b cntlabel (hd0,0)/fattext.g4d
cntlabel: result= errormsg=3 
grub> (hd0,0)/function.g4b cntlabel (hd0,0)/fattext.g4b 
EOL=\x0A 
cntlabel: result=102 errormsg= 
grub> (hd0,0)/function.g4b cntlabel (hd0,0)/fattext.g4b dos 
EOL=\x0D0A 
cntlabel: result=102 errormsg= 
grub> (hd0,0)/function.g4b cntlabel (hd0,0)/fattext.g4b mac 
EOL=\X0D 
cntlabel: result= errormsg= 
grub> (hd0,0)/function.g4b cntlabel (hd0,0)/fattext.g4b ms-dos 
EOL=\0Ax 
cntlabel: result=102 errormsg= 
grub>

BTW I tried to make textual error messages too, but today I was plagued again by Error 16 Inconsistent filesystem structure. Has something to do with editing files in Windows in combination with Imdisk to have access to VBox in combination with 'saving' grub4dos-sessions in VBox. After 'refreshing everything' Error 16 disappeared.

 

What do you think about error-handling?
 



#77 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 17 July 2021 - 02:20 AM

Edit 'print-screen':

OCR was bad this time, my eyes too. The echo of the three last EOL-codes should have been the following ones:

EOL=\x0D\x0A

EOL=\x0D

EOL=\x0A

Edited by deomsh, 17 July 2021 - 02:21 AM.


#78 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 18 July 2021 - 09:37 AM

Every time you use call or invoke a file with parameters, it is a different stack of % parameters.

Using shift in "main" will shift the "main" ones, if you use shift in a subroutine it will shift the "call" parameters.

No problem there.

 

About error checking/handling, it could be a nice feature, but (IMHO) not needed and actually overkill in the specific case of "dividing" a large batch in a "main program"+"functions library".

 

I mean - unlke the mathfunc.g4b (lousy) example, the idea of a .lll file is that it is "never" called from command line, but only by programs, so it is the program Author that is responsible for calling the library (rectius the function in it) with the correct parameters.

 

If the idea is that of using these functions from command line, it would still make sense to have an intermediate level (not entirely unlike the rundll32.exe) that takes care of validating the user input/parameters and echo result(s) in a suitable format.

 

:duff:

Wonko


  • deomsh likes this

#79 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 27 July 2021 - 11:03 PM

@Wonko
I played a bit with your (?) idea of 'Loosely Linked Libraries' and I 'found' following structure.

Because with call :%1 %* it is possible to link to other .LLL-files, I think it could be useful, from a developmental perspective, to split the FUNCTION-file you envisioned in 'Implemented' and 'Linked' functions. The 'Linked' functions can be still in development, while the 'Implemented' functions are (more or less) 'ready'. Also the .LLL-files have the same structure, so can be tested from the command-line too (Grub4Dos doesn't care, as long !BAT is on the first line). After finishing a 'Linked' function, all that has to be done is copying the call-part to the main FUNCTION.G4B. For example I found my function 'CntLabel' counted '::' at the beginning of a line too, so already a (small) revision was needed (I always use # before a comment).

Below the structure of the general file FUNCTION.G4B and as an example MAKEUUID.LLL (echos should be commented out later).

!BAT
#FUNCTION.G4B v0.1.5 by deomsh, based on idea of Wonko the Sane
#DLL for Grub4Dos (version: from 2019)
#Arguments: Function, Mandatory Arguments, Optional Arguments
#Implemented: CntLabel; CntLine;
#Linked to .lll-file: CopyFile; IsMaxLen; ParseArg; ParseCmd; MakeUuid
debug status > nul ;; set debugsts=%@retval% ;; debug 0
echo %*
call :%~1 %*
echo Function: %1 result=%result%
debug %debugsts%
set debugsts=
goto :eof

:CopyFile
call CopyFile.lll %*
goto :eof

:IsMaxLen
call IsMaxLen.lll %*
goto :eof

:ParseArg
call ParseArg.lll %*
goto :eof

:ParseCmd
call ParseCmd.lll %*
goto :eof

:MakeUuid
call MakeUuid.lll %*
goto :eof

:CntLabel
#Function: count number of labels; Argument: FILE; Optional: dos or mac
setlocal && set *
set file=%~2 &; if "%file:~0,4%"=="(md)" && set FILE=%~dnx2 ! set FILE=%~dpnx2
if /i "%~3"=="dos" && set EOL=\x0D\x0A ! set EOL=\x0A
if /i "%~3"=="mac" && set EOL=\x0D
cat --locate=%EOL%\x3A %FILE% > nul &; set /a count1=%@retval% > nul
cat --locate=%EOL%\x3A\x3A %FILE% > nul &; set /a count2=%@retval% > nul
set /a result=%count1% > nul && if exist count2 && set /a result=%count1%-%count2% > nul
endlocal && set result=%result%
goto :eof

:CntLine
#Count lines in textfile; Argument: FILE; Optional: dos|unix|mac; Default: dos
setlocal && set *
set file=%~2 &; if "%file:~0,4%"=="(md)" && set FILE=%~dnx2 ! set FILE=%~dpnx2
if /i not "%~3"=="" if /i not "%~3"=="dos" || set EOL=\x0D\x0A &; cat --locate=%EOL% %FILE% > nul &; set /a result=%@retval% > nul
if /i "%~3"=="dos" && set EOL=\x0D\x0A &; cat --locate=%EOL% %FILE% > nul &; set /a result=%@retval% > nul
if /i "%~3"=="unix" && set EOL=\x0A &; cat --locate=\x0D %FILE% > nul |; cat --locate=%EOL% %FILE% > nul &; set /a result=%@retval% > nul
if /i "%~3"=="mac" && set EOL=\x0D &; cat --locate=\x0A %FILE% > nul |; cat --locate=%EOL% %FILE% > nul &; set /a result=%@retval% > nul
if not exist result && set /a result=0 > nul
endlocal && set result=%result%
goto :eof

:CleanMd
#cleans md-device with '00'
#Arguments: MdBase, Sectors; Optional: Skip
setlocal && set *
set /A basemd=%~2 > nul
set /A memrange=%~3 > nul
set /A skipmem=%~4 > nul
set /A "basemd=%basemd% * 0x200 + %skipmem%" > nul
set /A "memrange=%memrange% * 0x200 - %skipmem%" > nul
call Fn.24 %basemd% 0x00 %memrange% > nul
set /A result=%@retval% > nul
if %result%>=1 && set /a result=1 > nul ! set /a result=0 > nul
endlocal && set result=%result%
goto :eof

!BAT
#MAKEUUID.LLL v0.1 (20210727), by deomsh. Based on unpublished VOLUUID.G4B (20200615)
#Generate MS-DOS Volume Serial Number
#Mandarory Argument: MakeUuid
echo %*
call :%~1
echo Function: %~1 result=%result%
goto :eof

:MakeUuid
setlocal && set *
#debug 0
set date=%@date%
set /a year=%date:~0,4% > nul
set /a month=%date:~5,2% > nul
set /a day=%date:~8,2% > nul
#echo date: %day%-%month%-%year% 
set time=%@time%
set /a hour=%time:~0,2% > nul
set /a minute=%time:~3,2% > nul
set /a second=%time:~6,2% > nul
set /a minute=%time:~3,2% > nul
#set /a smallsec= > nul
#echo time: %hour%:%minute%:%second%:%smallsec%
if not exist smallsec && set /a random=%@random% > nul ! set /a random=%smallsec% > nul
#echo random: %random%
if %random%>=100 && set /A smallsec=%random% / 10 > nul
if %random%>=1000 && set /A smallsec=%random% / 100 > nul
if %random%>=10000 && set /A smallsec=%random% / 1000 > nul
set /A month=%date:~5,2% * 0x100  > nul
set /A second=%time:~6,2% * 0x100 > nul
#echo month: %month% day: %day% second: %second% smallsec: %smallsec%
set /A uuidlow=%month% + %day% + %second% + %smallsec% > nul
set /a llow=%@retval% > nul
#echo llow: %llow%
#echo uuidlow: %uuidlow%
set /A hour=%time:~0,2% * 0x100 > nul
#echo hour: %hour% minute: %minute% year: %year%
set /A uuidhigh=%hour% + %minute% + %year% > nul
set /a lhigh=%@retval% > nul
#echo lhigh: %lhigh%
#echo uuidhigh: %uuidhigh%
echo Volume Serial Number: %uuidlow:~2,4%-%uuidhigh:~2,4% 
if %lhigh%==6 && if %llow%==6 && set volumeid=x%uuidhigh:~4,2%\x%uuidhigh:~2,2%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul
if %lhigh%==6 && if %llow%==5 && set volumeid=x%uuidhigh:~4,2%\x%uuidhigh:~2,2%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul
if %lhigh%==5 && if %llow%==6 && set volumeid=x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul
if %lhigh%==5 && if %llow%==5 && set volumeid=x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul
#echo Volume Serial Number little-endian: \%volumeid% (write (md)-ready)
#echo Volume Serial Number big-endian: %uuidlow:~2,4%%%uuidhigh:~2,4%
#endlocal && set volumeid=%volumeid%
endlocal && set result=%volumeid%
goto :eof

BTW Use of the .G4B-extension for FUNCTION.G4B seems more 'universal' to me (in case of use of command --set-ext=.g4b in MENU.LST).

 

BTW2 Central use of setlocal/ endlocal doesn't seem flexible enough to me if more than one variable, or an unknown number of variables is returned.

BTW3 Calling such batch-files by accident (without %1 = Function) seems 'safe' to me, nothing has been changed (in case of FUNCTION.G4B unless variable debugsts is used before - may be better not centralized too?).

What bothers me is the question of 'Memory Management'.

If a function needs working memory, I wonder if it would be better to use the main routine allocating memory to a function (with an optional argument), or use (a set of) fixed memory-addresses for all functions (which the main routine shouldn't use)?


Edited by deomsh, 27 July 2021 - 11:22 PM.


#80 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 July 2021 - 09:16 AM

It seems nice to me. :)

 

Though (if the idea is to save memory/variables) you don't really-really need to assign all variables, unless you need in the formula to parse the values.

I.e.:

:CleanMd
#cleans md-device with '00'
#Arguments: MdBase, Sectors; Optional: Skip
setlocal && set *
set /A basemd=%~2 > nul
set /A memrange=%~3 > nul
set /A skipmem=%~4 > nul
set /A "basemd=%basemd% * 0x200 + %skipmem%" > nul
set /A "memrange=%memrange% * 0x200 - %skipmem%" > nul
call Fn.24 %basemd% 0x00 %memrange% > nul
set /A result=%@retval% > nul
if %result%>=1 && set /a result=1 > nul ! set /a result=0 > nul
endlocal && set result=%result%
goto :eof

could well be:

:CleanMd
#cleans md-device with '00'
#Arguments: MdBase, Sectors; Optional: Skip
setlocal && set *
set /A "basemd=%~2 * 0x200 + %~4" > nul
set /A "memrange=%~3 * 0x200 - %~4" > nul
call Fn.24 %basemd% 0x00 %memrange% > nul
set /A result=%@retval% > nul
if %result%>=1 && set /a result=1 > nul ! set /a result=0 > nul
endlocal && set result=%result%
goto :eof

Three variable assignments removed, not noticeable at all, particularly because the function will be run once or very rarely, still, as a general rule using as much as possible the call parameters reduces usage of variables ...

 

As a side-note/OT question, the UUID generator uses the officially unofficial DOS algorithms, doesn't it? :unsure:

 

https://msfn.org/boa...&comment=980297

 

https://msfn.org/boa...#comment-987748

 

A couple small nitpicks (at first sight)  in:

 

#echo date: %day%-%month%-%year%
set time=%@time%
set /a hour=%time:~0,2% > nul
set /a minute=%time:~3,2% > nul
set /a second=%time:~6,2% > nul
set /a minute=%time:~3,2% > nul
#set /a smallsec= > nul
#echo time: %hour%:%minute%:%second%:%smallsec%
if not exist smallsec && set /a random=%@random% > nul ! set /a random=%smallsec% > nul
#echo random: %random%
if %random%>=100 && set /A smallsec=%random% / 10 > nul
if %random%>=1000 && set /A smallsec=%random% / 100 > nul
if %random%>=10000 && set /A smallsec=%random% / 1000 > nul

 

 

#1 You have duplicated the minute assignment, probably a copy/paste effect :unsure:

#2 I am not too convinced of the formula for reducing the random value into a 0<=x<=99 range  :dubbio: I have to think about it ...

 

:duff:

Wonko

 

P.S.: No, I confirm, dividing by 10/100/1000 doesn't give "random enough" results, you'd better get two digits, i.e. *something like*:

set smallsec=%@random::~-2%

or (to avoid possible leading 0, and get an actual numeric value)

set /a smallsec=1%@random::~-2%-100

or

set /a smallsec=1%@random::~-1%1%@random::~-1%-100



#81 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 28 July 2021 - 11:59 PM

It seems nice to me. :)

 

Though (if the idea is to save memory/variables) you don't really-really need to assign all variables, unless you need in the formula to parse the values.

I.e.:

:CleanMd
#cleans md-device with '00'
#Arguments: MdBase, Sectors; Optional: Skip
setlocal && set *
set /A basemd=%~2 > nul
set /A memrange=%~3 > nul
set /A skipmem=%~4 > nul
set /A "basemd=%basemd% * 0x200 + %skipmem%" > nul
set /A "memrange=%memrange% * 0x200 - %skipmem%" > nul
call Fn.24 %basemd% 0x00 %memrange% > nul
set /A result=%@retval% > nul
if %result%>=1 && set /a result=1 > nul ! set /a result=0 > nul
endlocal && set result=%result%
goto :eof

could well be:

:CleanMd
#cleans md-device with '00'
#Arguments: MdBase, Sectors; Optional: Skip
setlocal && set *
set /A "basemd=%~2 * 0x200 + %~4" > nul
set /A "memrange=%~3 * 0x200 - %~4" > nul
call Fn.24 %basemd% 0x00 %memrange% > nul
set /A result=%@retval% > nul
if %result%>=1 && set /a result=1 > nul ! set /a result=0 > nul
endlocal && set result=%result%
goto :eof

Three variable assignments removed, not noticeable at all, particularly because the function will be run once or very rarely, still, as a general rule using as much as possible the call parameters reduces usage of variables ...

Thanks, good idea in case of %~2 and %~3. I agree 'things' should be easiest as possible (at least 'in the end').

But NOT good in case of %~4, because %~4 is an optional argument (to clean only from a given memory address, located above or below a 512 byte sector-border). The calculation is not working if %~4 is not defined, so set /A skipmem=%~4 > nul is still needed. I found out that set /a skipmem= will always set the variable to zero. Which will be the case if optional argument %~4 is not in use.

 

As a side-note/OT question, the UUID generator uses the officially unofficial DOS algorithms, doesn't it? :unsure:

 

https://msfn.org/boa...&comment=980297

 

https://msfn.org/boa...#comment-987748

 

A couple small nitpicks (at first sight)  in:

 

#1 You have duplicated the minute assignment, probably a copy/paste effect :unsure:

#2 I am not too convinced of the formula for reducing the random value into a 0<=x<=99 range  :dubbio: I have to think about it ...

 

:duff:

Wonko

 

P.S.: No, I confirm, dividing by 10/100/1000 doesn't give "random enough" results, you'd better get two digits, i.e. *something like*:

set smallsec=%@random::~-2%

or (to avoid possible leading 0, and get an actual numeric value)

set /a smallsec=1%@random::~-2%-100

or

set /a smallsec=1%@random::~-1%1%@random::~-1%-100

Yes, I red all that stuff, and other 'things' I could found. A while ago already. A MSFN-member, named Jaclaz, helped me a lot too. :worship:

I tried to use the DOS-stuff, but without the DOS hundreds-of-seconds, they are not available in Grub4Dos as far as I know. Using random-numbers instead will be still an aproxximation....

 

About:

#1 True - Copy/ paste should be abandoned...

#2 True. My arithmetical approach gives the first two digits (or in very seldom cases if %@random% is below 10 one digit). In the distribution there are too many (decimal) numbers between 12 and 25 (maximum is 32 767 according to steve6375's book). Although in practice the 'hundreds' can even be set to zero (by removing '#' on line #set /a smallsec= > nul - probably > nul is not needed in this case if debug is on, but will do no harm ).

 

Which Grub4Dos user will format two partitions in less than one second? Only if creating a bunch of memdrives with map --mem, or creating images, maybe.

 

It seems %@retval% needs about 20 milli-seconds to generate a new value (in 64-bits VBox, on my AMD Bulldozer-system), while MakeUuid takes about 0,3-0,5 ms (calculated from a 10 000-looptest). So probably a delay should be added too.

 

I tried your suggestion of smallsec=%@random::~-2% seems theoretical better (and more practical too: one line instead of three). A sample of 100 of these gave following distribution: 22 duplicates and 5 triplets (but 6 duplicates in the range of 90-98). I am not an expert, seems not bad to me.

 

Also a (dirty) 'print-screen' of all three possibilities with echo of \%result%: above based on seconds only, in the middle my %@random%-approach and below Wonko's (all Litte-Endian, ready for write FILE \%result%).

grub> makeuuid makeuuid 
2021-07-29 00:59:16
 \x20\x08\x1D\x17 \x20\x08\x1D\x17 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1
D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x
08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \
x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D
\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08\x1D\x18 \x20\x08 
\x1D\x18 \x20\x08\x1D\x18 
grub> makeuuid makeuuid 
2021-07-29 00:59:36
 \x20\x08\x35\x2B \x20\x08\x39\x2B \x20\x08\x2D\x2B \x20\x08\x46\x2B \x20\x08\x3
1\x2B \x20\x08\x34\x2B \x20\x08\x31\x20 \x20\x08\x32\x2B \x20\x08\x34\x2B \x20\x 
08\x2B\x2B \x20\x08\x32\x2B \x20\x08\x35\x2B \x20\x08\x48\x2B \x20\x08\x28\x2C \
x20\x08\x37\x2C \x20\x08\x2F\x2C \x20\x08\x3C\2C \x20\x08\x39\x2C \x20\x08\x79\
x2C \x20\x08\x3D\x2C \x20\x08\x55\x2C \x20\x08\x51\x2C \x20\x08\x2B\x2C \x20\x08
\x27\x2C \x20\x08\x3B\x2C 
grub> makeuuid makeuuid 
2021-07-29 01:00:38
 \xE5\x08\x7A\x2E \xE5\x08\x54\x2E \xE5\x08\x2E\x2E \xE5\x08\x50\x2E \xE5\x08\x1 
F\x2E \xE5\x08\x5E\x2E \xE5\x08\x72\x2E \xE5\x08\x4E\x2E \xE5\x08\x37\x2E \xE5\x
08\x40\x2E \xE5\x08\x45\x2E \xE5\x08\x53\x2E \xE5\x08\x49\x2E \xE5\x08\x40\x2E \
xE5\x08\x1F\x2E \xE5\x08\x72\x2E \xE5\x08\x53\x2E \xE5\x08\x41\x2E \xE5\x08\x38
\x2E \xE5\x08\x45\x2E \xE5\x08\x3B\x2E \xE5\x08\x41\x2E \xE5\x08\x27\x2E \xE5\x08
\x26\x2E \xE5\x08\x45\x2E 
grub>

BTW for easy-going experimentation I changed to a .G4D-file, with loop and beep 0 20


Edited by deomsh, 29 July 2021 - 12:27 AM.


#82 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 29 July 2021 - 08:06 AM

You can still ADD a 0 in the place of the "optional" (not anymore optional) parameter.

I.e. (in my perverted mind) something *like*:

 

:CleanMd

call AllFunctions.lll %0 %* 0

goto :eof

 

could do nicely, if the (optional) parameter is specified on original call, it will be %4 (or %5, unless you use shift to have parameters as in the original call), if it is not specified the trailing 0 will "shift" and take its place.

 

About randomness, (actually pseudo-randomness) it is one of those topics that can be debated at length (not that it actually matters in this particular use).

 

For what it matters you could also use the canonical xkcd algorithm ;) :

https://xkcd.com/221/

 

 

The:

set /a smallsec=1%@random::~-2%-100

on second thought is not good as the value of @random may well be <=9 (or a single digit if you prefer) :dubbio: this possibility should be taken into account.

But if we take the digits "from the left", it can work, though,  as there will be no leading 00's and we can ignore the string length, i.e.:
set /a smallsec=%@random::~0,2%

 

The:

set /a smallsec=1%@random::~-1%1%@random::~-1%-100

is correct[1], as the @random can never have a <0 value, and it is still a single line, though it will be slightly slower so it is probably overkill.

 

 

:duff:

Wonko

 

 

[1] and essentially amounts to throwing two 10 side dices:

https://en.wikipedia...l_trapezohedron



#83 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 31 July 2021 - 12:25 AM


For what it matters you could also use the canonical xkcd algorithm ;) :

https://xkcd.com/221/

Very nice, I should make a MakeRandom-function for xkcd :lol:

 


The:

set /a smallsec=1%@random::~-2%-100

on second thought is not good as the value of @random may well be <=9 (or a single digit if you prefer) :dubbio: this possibility should be taken into account.

This is not a problem in MAKEUUID.LLL, everything is counted and transformed with the textual approach. Values in the whole 0-99 range are processed, I hope in the right way.

 

I made a new version, with arguments this time, and I changed the name of the Function to MakeUuidFat. Output-format can be chosen, date/time/hundreth can be added if a certain MS-DOS UUID is wanted.

!BAT
#MAKEUUID.LLL v0.1.3 (20210730), by deomsh
#Generate MS-DOS Volume Serial Number
#Use: MAKEUUID.LLL MakeUuidFat [uuid|write] [year-month-day] [hour:minute:second] [hundreth]
#Mandarory Argument: MakeUuid
#Optional arguments: [uuid|write] [yyyy-mm-dd] [hh:mm:ss] [hundreth] (of seconds - default=0)
#Remarks: Function-name is not case-sensitive. [uuid|write] not case-sensitive, can be left out. If used: date time hundreth only last argument can be left out, their order is NOT free
call :%~1 %*
#echo %~1: result=%result%
goto :eof

:MakeUuidFat
setlocal && set *
if not /i "%~2"=="write" && if not /i "%~2"=="uuid" && set mode=uuid
if /i "%~2"=="write" && set mode=%~2 && shift
if /i "%~2"=="uuid" && set mode=%~2 && shift
set date=%~2
set time=%~3
set smallsec=%~4
if not exist date && set date=%@date%
set /a year=%date:~0,4% > nul
set /a month=%date:~5,2% > nul
set /a day=%date:~8,2% > nul
if not exist time && set time=%@time%
set /a hour=%time:~0,2% > nul
set /a minute=%time:~3,2% > nul
set /a second=%time:~6,2% > nul
if exist smallsec && set /a smallsec=%smallsec% > nul && goto :norandom
beep 0 20 ;; set /a smallsec=%@random:~-2% > nul
:norandom
#echo Date: %day%-%month%-%year% Time: %hour%:%minute%:%second%:%smallsec%
set /A month=%date:~5,2% * 0x100  > nul
set /A second=%time:~6,2% * 0x100 > nul
set /A uuidlow=%month% + %day% + %second% + %smallsec% > nul
set /a llow=%@retval% > nul
set /A hour=%time:~0,2% * 0x100 > nul
set /A uuidhigh=%hour% + %minute% + %year% > nul
set /a lhigh=%@retval% > nul
if %lhigh%==6 && if %llow%==6 && set volumeid=\x%uuidhigh:~4,2%\x%uuidhigh:~2,2%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=%uuidlow:~2,2%%%uuidlow:~4,2%%-%uuidhigh:~2,2%%%uuidhigh:~4,2%
if %lhigh%==6 && if %llow%==5 && set volumeid=\x%uuidhigh:~4,2%\x%uuidhigh:~2,2%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-%uuidhigh:~2,2%%%uuidhigh:~4,2%
if %lhigh%==5 && if %llow%==6 && set volumeid=\x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=%uuidlow:~2,2%%%uuidlow:~4,2%%-0%uuidhigh:~2,1%%%uuidhigh:~3,2%
if %lhigh%==5 && if %llow%==5 && set volumeid=\x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-0%uuidhigh:~2,1%%%uuidhigh:~3,2%
if %lhigh%==4 && if %llow%==6 && set volumeid=\x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=0%uuidlow:~1,2%%%uuidlow:~4,2%%-00%uuidhigh:~3,2%
if %lhigh%==4 && if %llow%==5 && set volumeid=\x%uuidhigh:~2,2%\x00\x0%uuidlow:~4,1%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-00%uuidhigh:~2,2%
if %lhigh%==3 && if %llow%==6 && set volumeid=\x00\x0%uuidhigh:~2,1%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=%uuidlow:~2,2%%%uuidlow:~4,2%%-000%uuidhigh:~2,1%
if %lhigh%==3 && if %llow%==5 && set volumeid=\x00\x0%uuidhigh:~2,1%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-000%uuidhigh:~2,1%
#echo Volume Serial Number: %volserial%
#echo Volume Serial Number little-endian: %volumeid% (for write FILE %^result%)
if %mode%==uuid && set result=%volserial% ! set result=%volumeid%
endlocal && set result=%result%
goto :eof

BTW: this version should be clean now (only a few echos left, can be activated if desired). I added lines to cope with years closer to 1 AD

 

After restudying Craig Wilson's 'Volume Serial Numbers' I added his '2003-10-19 22:33:27:1' example, together with use of other arguments in following 'print-screen':

grub> makeuuid.lll makeuuidFAT
date: 30-7-2021 time: 23:51:48:22
Volume Serial Number: 3734-1F18
Volume Serial Number little-endian: \x18\x1F\x34\x37 (for write FILE %result%)
makeuuidFAT: result=3734-1F18
grub> 
grub> makeuuid.lll makeuuidFAT write 
date: 30-7-2021 time: 23:51:51:74 
Volume Serial Number: 3A68-1F18 
Volume Serial Number little-endian: \x18\x1F\x68\x3A (for write FILE %result%) 
makeuuidFAT: result=\x18\x1F\x68\x3A
grub> 
grub> makeuuid.lll makeuuidFAT 2003-10-19 22:33:27 1
date: 19-10-2003 time: 22:33:27:1 
Volume Serial Number: 2514-1DF4 
Volume Serial Number little-endian: \xF4\x1D\x14\x25 (for write FILE %result%)
makeuuidFAT: result=2514-1DF4 
grub> 
grub> makeuuid.lll makeuuidFAT write 0001-01-01 00:00:00 0 
date: 1-1-1 time: 0:0:0:0 
Volume Serial Number: 0101-0001
Volume Serial Number little-endian: \x00\01\x01\x01 (for write FILE %result5)
makeuuidFAT: result=\x00\x01\x01\x01 
grub> _

BTW the only strange thing in Craig Wilson's paper is that the MS-DOS UUID isn't written in Litte-Endian in the picture on page 4. I checked again and again, but with uuid --write (hd2,0) 2514-1DF4 I get with cat --hex --skip=0x27 --length=4 (hd2,0)+1 always F4 1D 14 25 ??

 

I played a bit with uuid's before 1987. After attaching the VHD I used in Vbox with Disk Management, Windows 10 happily found no errors and displayed with DIR in cmd 0101-0001 :blink:

 

You can still ADD a 0 in the place of the "optional" (not anymore optional) parameter.

I.e. (in my perverted mind) something *like*:

 

:CleanMd

call AllFunctions.lll %0 %* 0

goto :eof

 

could do nicely, if the (optional) parameter is specified on original call, it will be %4 (or %5, unless you use shift to have parameters as in the original call), if it is not specified the trailing 0 will "shift" and take its place.

 

Although not needed for CleanMd, the idea from your 'perverted mind' 'saved my day' last thursday. I was trying to set several default values in a function to count variables, so without adding variables before all variables are safely in memory. So THANK YOU VERY MUCH. Even your call starting with %0 inspired my to a new argument (starting %0 is not needed in my approach, the function is always %1 before shifting and can be linked endlessly with :%1 %*).

!BAT
#CNTVARS.LLL v0.1 (20210729), by deomsh.
#Function: Count number of variables 
#Mandatory Arguments: CntVars
#Optional Arguments: [mdbase sectors %0]
#Remarks: (if used ALL needed: base memory in 512 byte sectors -default=0x3000; needed sectors for maximum number of variables of max length, max 511 sectors - default=60; %0 must be a label or preceding colon must be added)
#debug 0
#echo %*
##call :SubRoutine %*
call :%~1 %* 0x3000 60 
#echo Function: %~1 result=%result%
goto :eof

:SubRoutine
#echo %~0 %*
#call :%~1 %* 0x3000 60
call :%~1 %* 0x3000 60 %0
goto :eof

:CntVars
#echo %~0 %*
echo -n > (md)%~2+%~3
set > (md)%~2+%~3
setlocal && set *
echo -n && call Fn.4 ;; set /A r=%@retval% > nul ;; set /a y=%r%>>8 > nul ;; set /a x=%r%&0xFF > nul
cat --locate=\x0A (md)%~2+%~3 > nul ;; set /a result=%@retval% > nul
echo -n > (md)%~2+%~3
:LoopCntVars
set ARG=%~4 &; if not "%ARG%"=="" &; if %ARG:~0,1%==: && set ARG4=%~4 ! shift && goto :LoopCntVars
if exist ARG4 && set "message=Number of variables=%result% in %ARG4%" ! set "message=Number of variables=%result%"
set "message=%message%" ;; set msglen=%@retval% > nul ;; set /a h=79-%msglen% > nul
call Fn.5 %h% 24 || echo -n $[0x0F]%message%
call Fn.5 %x% %y%
#set
endlocal
#endlocal && set result=%result%
goto :eof

BTW, best used with: CNTVARS.LLL CntVars optional arguments && pause. For instance I tested in my editline-project call cntvars.lll CntVars 0x3900 60 %0 && pause to get on the last line shifted to the right: Number of variables=40 in :editline (:editline is the central submenu sub-routine).

 

BTW2 Of course 60 is overkill, one variable including name and overhead will in the average not come above 20-25 chars, so 3 sectors will do in most cases.

 

BTW3 Not meant as stand-alone utility. Can be used, but after line 23 only with && pause

 

Question: I don't see any possibility to make more than one default value (in fact I used one optional argument, consisting of two or three values). Will that be somehow do-able?

 

 

 

 

 

 


Edited by deomsh, 31 July 2021 - 12:37 AM.


#84 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 31 July 2021 - 09:44 AM

 

Question: I don't see any possibility to make more than one default value (in fact I used one optional argument, consisting of two or three values). Will that be somehow do-able?

I am not sure to understand the question (actually I am pretty sure I don't understand it), can you explain/make an example?

 

The Big vs. Little Endian question is always the same one, it is easy to make confusion.

 

in Craig Wilson's paper page 4 he shows a hex/view where the sequence of bytes is

25 14 1D F4

then he proceeds to state that the volume label is 2514-1DF4 :w00t: :

 

Here is an example from a formatted FAT12 floppy diskette.

With reference to Figure 1 below, at offset 39 (0x27), you can see the hex values 25 14 1D F4 which is the Volume Serial Number.

With FAT32 volumes, the Volume Serial Number is stored in the Boot Sector at offset 67 (0x43).

When formatted, this floppy diskette returned the volume serial 2514-1DF4.

BUT on page 5, there is figure 3 where it is evident:

Volume Serial Number is F41D-1425

 

that the VOL utility (like - say - WMI) interprets the bytes sequence as Big Endian.

 

Using my little date_from_volser sheet (which has limits to the values you can attribute to the bytes) and that uses the hex dump sequence, it is evident how the serial stated on page 4 is actually the hex dump separated in two words by a hyphen :w00t: and NOT the actual serial as returned by MS tools.

 

:duff:

Wonko



#85 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 31 July 2021 - 07:58 PM

First about the MS-DOS UUID only, I will later give a more lengthy version of my question regarding %*.

In Craig Wilson's paper, figure 1 on page 4 and figure 3 on page 5 seems to me fully consistent. I used the formula of figure 2 on page 5, interpreted in the Big-Endian order. Did you mean there is anything wrong with the order in the formula?

If 2514-1DF4 is written with Grub4Dos uuid to a FAT-partition, MS-DOS' VOL and DIR returns 2514-1DF4, and on disk is written in the PBR F4 1D 14 25. So Little-Endian order...

#86 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 01 August 2021 - 09:44 AM

The confusion, as often happens, is how you interpret the data and how you call it, by definition (or actually derived definition), a volume serial (or uuid) is *whatever* is returned by the VOL command (or the uuid one), consisting in two bytes, a hyphen and other two bytes, coming from a 4 bytes sequence in the bootsector that is written as a Big Endian 32 bit value..

 

The actual data in the paper (4 hex bytes sequence written to the volume serial location) is 25 14 1D F4.

The volume serial (as returned by the VOL command) is F41D-1425. <- this is the ONLY thing that you can properly call "Volume Serial" or "UUID"

 

It is clear that the VOL command does two operations:
1) interprets the 4 bytes as a Big Endian hex number
2) divides it in two two bytes words and inserts a hyphen between them

 

The uuid command in grub4dos does exactly the same as the VOL command (evidently to be consistent with it).

 

Quick example from a floppy:

uuid (fd0)

(fd0): UUID is "0AC4-AF63"

Filesystem type is FAT12, using all disk

 

cat --hex --skip=39 --length=4 (fd0)+1

00000026: 63 AF C4 0A

 

Your example implementation is seemingly "wrong" (i.e. reversed):

 

grub> makeuuid.lll makeuuidFAT 2003-10-19 22:33:27 1
date: 19-10-2003 time: 22:33:27:1
Volume Serial Number: 2514-1DF4
Volume Serial Number little-endian: \xF4\x1D\x14\x25 (for write FILE %result%)
makeuuidFAT: result=2514-1DF4

 

as what is actually written to disk (in the paper) is the sequence "25 14 1D F4" that is read by *any* VOL/UUID command as "F41D-1425".

 

There is a known (similar but fun) issue (JFYI) with Disk Signature on NT systems (JFYI).

 

The 4 bytes sequence at offset 440 in the MBR is (example):

59 AE 59 AE

 

If you go to HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices and look for the drive letter(s) assigned to the volume(s) on the disk, you will find something *like*:

\DosDevices\D: with value 59 AE 59 AE 00 7E 00 00 00 00 00 00

which basically means "the volume to which drive letter D: is assigned is on the disk with byte sequence at the disk signature offset of "59 AE 59 AE" and begins at offset 0x7E00 (32256 bytes)" OR "the volume to which drive letter D: is assigned is on the disk with Id 0xAE59AE59 and begins at offset 0x7E00 (32256 bytes)"

 

You can use Diskpart to retrieve this info, *like*:

diskpart

select disk 1

detail disk

and you will obtain:

Disk ID: AE59AE59

 

So, the "Disk ID" is seemingly a Big Endian number (though in a number of places, including some of my old posts, it is likely that you will find the bytes sequence "as is" called "disk signature").

 

Here is the fun part, we can get it also via WMI/wmic:

wmic DiskDrive get DeviceId, Signature
DeviceID Signature
\\.\PHYSICALDRIVE0 -1740662721
\\.\PHYSICALDRIVE1 -1369854375
...

Converting -1369854375 decimal to hex we get: FFFFFFFFAE59AE59

 

So, generally speaking the Disk Signature being a largely undocumented field, can be interpreted "as is" or as a Big Endian number by various different tools (and remember that Diskpart is a relatively recent tool, before it there wasn't AFAICT a MS original way to get that field).

 

And still two different tools by the same good MS guys call the same field "Disk Id" or "Signature" and return different values.

 

:duff:

Wonko



#87 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 02 August 2021 - 01:26 AM

Your example implementation is seemingly "wrong" (i.e. reversed):

 

grub> makeuuid.lll makeuuidFAT 2003-10-19 22:33:27 1
date: 19-10-2003 time: 22:33:27:1
Volume Serial Number: 2514-1DF4
Volume Serial Number little-endian: \xF4\x1D\x14\x25 (for write FILE %result%)
makeuuidFAT: result=2514-1DF4

 

as what is actually written to disk (in the paper) is the sequence "25 14 1D F4" that is read by *any* VOL/UUID command as "F41D-1425".

 


 

So, the "Disk ID" is seemingly a Big Endian number (though in a number of places, including some of my old posts, it is likely that you will find the bytes sequence "as is" called "disk signature").

 

What bothers me is the formula and order of the MS-DOS Volume Serial Number.

 

If Craig Wilson's Figure 1, 2 AND 3 are consistent, this will mean that the example calculation in Figure 2 calculates and arranges the Volume Serial ID in exactly the same ORDER as it is found on disk. So only the hyphen between the two parts looks a bit strange, in relation to how VOL reads the bytes on disk.

 

In that case I am wrong and I have to rewrite the order in my script.

 

But the problem remains... I RE-tested the results of my script in comparison with MS-DOS 7.1 FORMAT (like I did about a year ago while developing my script).

 

I made following MS-DOS batch (sadly FORMAT took too much lines, so time before is 'lost'):

ECHO= | DATE
ECHO= | TIME
FORMAT E: /V:DISK1GDOS71
ECHO= | DATE
ECHO= | TIME

This are my results using the batch in Vbox:

(.... = output of FORMAT)

Volume Serial Number is 4310-08EF


A:\ECHO= | DATE
De datum is nu ingesteld op ma 02-08-2021.
Typ nieuwe datum (dd-mm-jjjj):

A:\ECHO= | TIME
De tijd is nu ingesteld op   1:10:59,36
Typ nieuwe tijd:

A:\

A:\

If I use these date and time in Grub4Dos with clean MAKEUUID.LLL I get:

grub> makeuuid.lll MakeUuidFAT uuid 2021-08-02 01:10:59 36
MakeUuidFAT: result=4326-08EF
grub> makeuuid.lll MakeUuidFAT write 2021-08-02 01:10:59 36
MakeUuidFAT: result=\xEF\x08\x26\x43

So looks very close (there is a small writing delay) and consistent too with disk order written by FORMAT.

 

Am I still wrong about the MS-DOS Volume Serial Number ?

 

Also I found an interesting source on http://www.faqs.org/.../general/part3/

I copied the whole entrance on this subject (I do not understand the last part 28.3):

Subject: 28. Volume Serial Numbers
28.1  VOLUME SERIAL NUMBER FORMAT

The volume serial number was introduced with DOS 4.0 as part of an
extended boot record and is created through you either FORMAT a disk or
use DISKCOPY to create another disk.  The serial number is a function of
the time/date of the formatting or the diskcopying.  Note that DISCOPY
generates a new volume serial number so a DISKCOPY is not an exact image
of the source diskette.

28.2  CALCULATING THE VOLUME SERIAL NUMBER

For example, say a disk was formatted on 26 Dec 95 at 9:55 PM and 41.94
seconds.  DOS takes the date and time just before it writes it to the
disk.

Low order word is calculated:               Volume Serial Number is:
    Month & Day         12/26   0c1ah
    Sec & Hundrenths    41:94   295eh               3578:1d02
                                -----
                                3578h

High order word is calculated:
    Hours & Minutes     21:55   1537h
    Year                1995    07cbh
                                -----
                                1d02h

Note that DOS interrupt 21h Functions 2ah, Get DOS Date, and 2ch, Get
DOS Time, are particularly suited to getting the date and time for
calculating the Volume Serial Number.

28.3 READING AND SETTING THE VOLUME SERIAL NUMBER

To read the Volume Serial Number, use the IOCTL call, Int 21h function
440dh Minor Code 66h, Get Media ID.  To write the Volume Serial Number,
use the IOCTL call, Int 21h function 440dh Minor Code 46h, Set Media ID.

WARNING!  These IOCTL calls use a structure that also contains the
volume label and file system type.  So that you do not create errors
with these values, I recommend that you always Minor Code 66h to
initialize the structure before setting the Volume Serial Number to a
new value and writing it back to the disk.

Contributor:  Raymond Moon, raymoon@moonware.dgsys.com
Last changed: 17 Feb 96

The calculation and order are exactly like Colin Wilson's Figure 2 (only date&time are different of course).

 

If I knew what means in regard to the order how: 'IOCTL call, Int 21h function 440dh Minor Code 46h, Set Media ID' will write to disk, I would be more sure about my experimental results.

 

BTW: I do not say Colin Wilson is wrong, he is a Forensic Examiner, I am only some nitwit B)

Also it's possible different MS-FORMAT versions are writing in a different way ??


Edited by deomsh, 02 August 2021 - 02:21 AM.


#88 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 02 August 2021 - 01:13 PM

I don't really know. :w00t:

 

On my own volser.xls:

1) with 2003-10-19 22:33:27 1 I get 25141DF4
2) with 1995-12-26 21:55:41 94 I get 35781D02

3) with 2021-08-02 01:10:59 36 I get 432608EF (expected 43100BEF, actuallly corresponding to 2021-08-02 01:10:59 14)

 

Since your difference is in the "lower" word, 4326-4310, and the day cannot have changed, it is all due to 0x16=22 100th of second delay (quite possible).

 

This tells us that the "lower" world is definitely the leftmost one in the output of VOL (or of the FORMAT) command.

 

So after all it seems like Colin Craig Wright *somehow* inverted the values.

 

And no, i don't think that the formula changed in different DOS versions. (though I believe that NT family of OS use a completely different generating algorithm, more "random", possibly using the internal random generator instead of date/time).

 

I don't think you actually want to know about DOS int21h, (whooosh, that was the sound of it passing way over my head) but if you really want to know check:

http://www.ctyme.com/intr/rb-2896.htm

 

:duff:

Wonko

 

P.S.: Corrected name, sorry Craig, I was thinking (cannot say why) about Colin Ramsden (of WinFE fame) and mixed up the name.



#89 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 03 August 2021 - 12:14 AM

Edit: although I started my previous post right with 'Craig Wilson', later on he became 'Colin'... Don't no why: sorry Craig!

@Wonko: Thanks for all the help!

I run some more tests on MS-DOS 5.00/ 6.00/ 7.00a. If I started my dos-batch with 'mode 80,50' I can see time before and after formatting on the screen (in Dos 5 and 6 date/ time must be passed by pressing enter). Actual Low order Word was always between calculated ones. So As you said: same type of Volume Serial.

I did some quick tests with the same batch in Windows 10 CMD: indeed seems random.

So the change of function name to 'MakeUuidFat' is not justified anymore. I'm going back to 'MakeUuid' (much shorter too).

I have been reading about Int21, some other sources too. Lucky me, I don't need to get into this stuff...

#90 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 03 August 2021 - 07:54 AM

All in all it seems to me like the DOS only way is nothing but a quick workaround due to the lack of a pseudo-random generator.

 

Since NT creates pseudo-random volume serials (and has a random device built-in) I cannot see why in grub4dos we cannot use a similar approach using the @random in grub4dos. 

 

I.e. something loosely *like*:

 

 

:rand_32
#no parameters
set /A byte0=100+%@random:~0,3%
set /A byte1=100+%@random:~0,3%
set /A byte2=100+%@random:~0,3%
set /A byte3=100+%@random:~0,3%
set /A result=0x%byte0:~-2%%byte1:~-2%%byte2:~-2%%byte3:~-2%

 

As I showed with the fictional swiss watchmaker backup routine :w00t: after all the DOS way isn't that much random:

 

https://msfn.org/boa...&comment=987748

 

:duff:

Wonko



#91 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 03 August 2021 - 11:36 PM

All in all it seems to me like the DOS only way is nothing but a quick workaround due to the lack of a pseudo-random generator.

 

Since NT creates pseudo-random volume serials (and has a random device built-in) I cannot see why in grub4dos we cannot use a similar approach using the @random in grub4dos. 

 

I.e. something loosely *like*:

:rand_32
#no parameters
set /A byte0=100+%@random:~0,3%
set /A byte1=100+%@random:~0,3%
set /A byte2=100+%@random:~0,3%
set /A byte3=100+%@random:~0,3%
set /A result=0x%byte0:~-2%%byte1:~-2%%byte2:~-2%%byte3:~-2%

Quote in quote is a bit too difficult for me to qoute...

 

But: 'your wish is my command'

!BAT
#MAKEUUID.LLL v0.1.5 (20210803), by deomsh. Random parts based on random-ideas of Wonko the Sane
#Generate MS-DOS Volume Serial Number
#Use: MAKEUUID.LLL MakeUuidFat [uuid|write] [year-month-day] [hour:minute:second] [hundreth]
#Mandarory Argument: MakeUuid
#Optional arguments: [uuid|write] [dos|random] [yyyy-mm-dd] [hh:mm:ss] [hundreth] (of seconds - default=0)
#Remarks: Function-name is not case-sensitive. [uuid|write] not case-sensitive, can be left out; default is uuid. [dos|random] not case-sensitive, can be left out; default is dos. Order of first two optional arguments is NOT free. If used (with dos only): date time hundreth, only last argument can be left out, their order is NOT free
call :%~1 %*
echo %~1: result=%result%
goto :eof

:MakeUuid
setlocal && set *
if not /i "%~2"=="write" && if not /i "%~2"=="uuid" && set mode=uuid
if /i "%~2"=="write" && set mode=%~2 && shift
if /i "%~2"=="uuid" && set mode=%~2 && shift
if not /i "%~2"=="dos" && if not /i "%~2"=="random" && set generate=dos
if /i "%~2"=="dos" && set generate=%~2 && shift
if /i "%~2"=="random" && set generate=%~2 && shift
if %generate%==dos call :DosUuid DosUuid %mode% %2 %3 %4 
if %generate%==random && call :Random32Uuid Random32Uuid %mode%
endlocal && set result=%result%
goto :eof

:DosUuid
setlocal && set *
set mode=%~2
set date=%~3
set time=%~4
set smallsec=%~5
if not exist date && set date=%@date%
set /a year=%date:~0,4% > nul
set /a month=%date:~5,2% > nul
set /a day=%date:~8,2% > nul
if not exist time && set time=%@time%
set /a hour=%time:~0,2% > nul
set /a minute=%time:~3,2% > nul
set /a second=%time:~6,2% > nul
if exist smallsec && set /a smallsec=%smallsec% > nul && goto :norandom
beep 0 20 ;; set /a smallsec=%@random:~-2% > nul
:norandom
#echo Date: %day%-%month%-%year% Time: %hour%:%minute%:%second%:%smallsec%
set /A month=%date:~5,2% * 0x100  > nul
set /A second=%time:~6,2% * 0x100 > nul
set /A uuidlow=%month% + %day% + %second% + %smallsec% > nul
set /a llow=%@retval% > nul
set /A hour=%time:~0,2% * 0x100 > nul
set /A uuidhigh=%hour% + %minute% + %year% > nul
set /a lhigh=%@retval% > nul
if %lhigh%==6 && if %llow%==6 && set volumeid=\x%uuidhigh:~4,2%\x%uuidhigh:~2,2%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=%uuidlow:~2,2%%%uuidlow:~4,2%%-%uuidhigh:~2,2%%%uuidhigh:~4,2%
if %lhigh%==6 && if %llow%==5 && set volumeid=\x%uuidhigh:~4,2%\x%uuidhigh:~2,2%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-%uuidhigh:~2,2%%%uuidhigh:~4,2%
if %lhigh%==5 && if %llow%==6 && set volumeid=\x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=%uuidlow:~2,2%%%uuidlow:~4,2%%-0%uuidhigh:~2,1%%%uuidhigh:~3,2%
if %lhigh%==5 && if %llow%==5 && set volumeid=\x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-0%uuidhigh:~2,1%%%uuidhigh:~3,2%
if %lhigh%==4 && if %llow%==6 && set volumeid=\x%uuidhigh:~3,2%\x0%uuidhigh:~2,1%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=0%uuidlow:~1,2%%%uuidlow:~4,2%%-00%uuidhigh:~3,2%
if %lhigh%==4 && if %llow%==5 && set volumeid=\x%uuidhigh:~2,2%\x00\x0%uuidlow:~4,1%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-00%uuidhigh:~2,2%
if %lhigh%==3 && if %llow%==6 && set volumeid=\x00\x0%uuidhigh:~2,1%\x%uuidlow:~4,2%\x%uuidlow:~2,2% > nul && set volserial=%uuidlow:~2,2%%%uuidlow:~4,2%%-000%uuidhigh:~2,1%
if %lhigh%==3 && if %llow%==5 && set volumeid=\x00\x0%uuidhigh:~2,1%\x%uuidlow:~3,2%\x0%uuidlow:~2,1% > nul && set volserial=0%uuidlow:~2,1%%%uuidlow:~3,2%%-000%uuidhigh:~2,1%
#echo Volume Serial Number: %volserial%
#echo Volume Serial Number little-endian: %volumeid% (for write FILE %^result%)
if %mode%==uuid && set result=%volserial% ! set result=%volumeid%
endlocal && set result=%result%
goto :eof

:Random32Uuid
setlocal && set *
set mode=%~2
#set /a random=%@random% > nul
set c=0 > nul
:LoopRandom32Byte
#call Cmp@Rnd.lll CmpRandom %random%
#set /a random=%result% > nul
set /A Byte%c%=%@random:~-3,3% > nul
#set /A Byte%c%=100+%random:~0,3% > nul
call if %^Byte%c%%%<=15 && call set Byte%c%=0%^Byte%c%:~2,1% ! call set Byte%c%=%^Byte%c%:~2,2%
if %c%<=2 && set /a c=%c%+1 > nul && goto :LoopRandom32Byte
if %mode%==uuid && set result=%Byte0%%%Byte1%-%Byte2%%%Byte3%
if %mode%==write && set result=\x%Byte3%\x%Byte2%\x%Byte1%\x%Byte0% 
#set
endlocal && set result=%result%
goto :eof

This are the possibilities of v0.1.5:

grub> makeuuid.lll MakeUuid
MakeUuid: result=1266-1EED
grub> 
grub> makeuuid.lll MakeUuid write
MakeUuid: result=\xED\x1E\x26\x15
grub> 
grub> makeuuid.lll MakeUuid random
MakeUuid: result=3C3C-3710
grub> 
grub> makeuuid.lll MakeUuid write random
MakeUuid: result=\x29\x16\x38\x61
grub> 
grub> makeuuid.lll MakeUuid 2021-08-03 23:08:00 0
MakeUuid: result=0803-1EED
grub> 
grub> makeuuid.lll MakeUuid write 2021-08-03 23:08:00 0
MakeUuid: result=\xED\x1E\x03\x08
grub> 

BTW *somehow* a 20 ms delay between the %@retval%'s was this time not needed. I did looptests up to 10 000 random uuid-generations, but saw no repeats. So a direct comparison of random-numbers seems not to be needed (as I expected first, making Cmp@Rnd.lll).

 

 

As I showed with the fictional swiss watchmaker backup routine :w00t: after all the DOS way isn't that much random:

I agree.

Base of High order word is one higher each year, and repeats every day. Same principle with Low order word, base is one higher each day, and repeats every minute. So many collisions are possible.

Only sad creation date/time cannot be decoded.

 

I think about a third uuid, optional argument 'private' or 'home' in such a way I can decode creation date/time afterwards of any FAT-partition with this type of uuid. Also while unique in time. But I am afraid I will loose the hundrenths because of 32 bits uuid.... Which on the other hand is fully compatible with Grub4Dos time-format, delay of pause --wait=1 would be enough for LOCAL unique uuid's (first second is just the length of time until the running second is done). What do you think?
 



#92 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 04 August 2021 - 01:36 PM

Yes, but if the algorithm (not random) is unique date/time based, there is actually no real need of the 100th of seconds, I mean, there are 24*60*60=86,400 seconds in a same day, using the birthday collision :w00t: formula :

https://en.wikipedia...irthday_problem

https://www.dcode.fr/birthday-problem

you will have a probability of a collision of roughly 6% with 100 concurrent volumes (and of 0,4% with a "more normal" max of 26)

and while it is not like you are normally formatting a new volume in less than a second, it is even less probable that you get such a high number of volumes "coming from outside.

 

Instead of the wait=1, I would personally rather make an "initial, dummy" serial, storing it as "last serial", then generate a new one , compare the latter with the former, and if it is the same loop again until it is different, this will be (slightly) slower for the first run, but should be faster on subsequent runs :unsure:.

 

About the alternative algorithm, why not Unix Epoch (it will last until 2038 in 32 bit version)?

 Naaah, the calculation is way too complex  :ph34r: due to leap years and what not.

 

The Windows time seems much easier, since after all it is only a binary encoding of the date (and the only offset value is years since 1980 that are easy to calculate):

https://docs.microso...s?view=msvc-160

https://docs.microso...s-date-and-time

BUT it has 2 seconds approximation :dubbio:

 

Just in case, I am attaching a few of my half-@§§ed batches (intended for NT cmd.exe but easy to "port" to grub4dos batch) that are related/could be of inspiration.

 

As always they are very far from being foolproof, no or very little input validation/error checks, they only represent a quick example.

 

:duff:

Wonko

 

Attached Files



#93 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 04 August 2021 - 09:41 PM

Yes, but if the algorithm (not random) is unique date/time based, there is actually no real need of the 100th of seconds, I mean, there are 24*60*60=86,400 seconds in a same day, using the birthday collision :w00t: formula :

https://en.wikipedia...irthday_problem

https://www.dcode.fr/birthday-problem

you will have a probability of a collision of roughly 6% with 100 concurrent volumes (and of 0,4% with a "more normal" max of 26)

and while it is not like you are normally formatting a new volume in less than a second, it is even less probable that you get such a high number of volumes "coming from outside.

@Wonko, thanks for the info. Good news, although my idea was meant for 'private' or 'home', so on one or a few systems. I think the probability of collisions will be (almost?) 0%

 

Instead of the wait=1, I would personally rather make an "initial, dummy" serial, storing it as "last serial", then generate a new one , compare the latter with the former, and if it is the same loop again until it is different, this will be (slightly) slower for the first run, but should be faster on subsequent runs :unsure:.

I dropped the pause --wait=1 already. But 'storing' a previous value in a .lll-function... :unsure:

I think it's better to make another function, like 'CmpUuid' thats retrieves all 32-bits UUID's from all available drives and compares with a given UUID.

 

About the alternative algorithm, why not Unix Epoch (it will last until 2038 in 32 bit version)?

 Naaah, the calculation is way too complex  :ph34r: due to leap years and what not.

I vaguely remember Unix TimeStamp. Is that related? I will search more info and see if I can add a fourth UUID-species. Optional argument [dos|random|unix|second] would be nice. :tabletalk:

 

The Windows time seems much easier, since after all it is only a binary encoding of the date (and the only offset value is years since 1980 that are easy to calculate):

https://docs.microso...s?view=msvc-160

https://docs.microso...s-date-and-time

BUT it has 2 seconds approximation :dubbio:

 

Just in case, I am attaching a few of my half-@§§ed batches (intended for NT cmd.exe but easy to "port" to grub4dos batch) that are related/could be of inspiration.

 

As always they are very far from being foolproof, no or very little input validation/error checks, they only represent a quick example.

 

:duff:

Wonko

Oeps, MAKESECU.LLL is already finished. But thanks a lot, I will study your batches precisely, they are always very instructive. :thumbsup:

 

The leap years were a bit difficult, because I had to figure out how the modulo works in calc. NOWHERE any example or explanation, but I succeeded by trial and error (in Limbo X86 on my smartphone). If I counted right this UUID will last up to 2116-02-07 06:28:15 according to my script. :yawn:

!BAT
#MAKESECU.LLL v0.1.0 (20210804), by deomsh
#Generate Volume Serial Number in seconds from 1980-01-01 00:00:00 (year 1 AM)
#Use: MAKESECU.LLL MakeSecUuid [uuid|write][year-month-day] [hour:minute:second]
#Mandarory Argument: MakeSecUuid
#Optional arguments: [uuid|write] [yyyy-mm-dd] [hh:mm:ss]
#Remarks: Function-name is not case-sensitive. [uuid|write] not case-sensitive, can be left out; default is uuid. Order of optional arguments is NOT free. If used: date time, only last argument can be left out, their order is NOT free
call :%~1 %*
echo %~1: result=%result%
goto :eof

:MakeSecUuid
setlocal && set *
if not /i "%~2"=="write" && if not /i "%~2"=="uuid" && set mode=uuid
if /i "%~2"=="write" && set mode=%~2 && shift
if /i "%~2"=="uuid" && set mode=%~2 && shift
set date=%~2
set time=%~3
if not exist date && set date=%@date%
set /a year=%date:~0,4% > nul
set /a month=%date:~5,2% > nul
set /a day=%date:~8,2% > nul
#if not exist time && pause --wait=1
if not exist time && set time=%@time%
set /a hour=%time:~0,2% > nul
set /a minute=%time:~3,2% > nul
set /a second=%time:~6,2% > nul
#echo Date: %day%-%month%-%year% Time: %hour%:%minute%:%second%
call :DaysInYears DaysInYears %year% &; set yearsdays=%result% &; set result=
#echo Number of Days in past years: %yearsdays%
call :DaysInMonths DaysInMonths %year% %month% &; set /a monthdays=%result% > nul &; set result=
#echo Number of Days in past months: %monthdays%
set /a numdays=%yearsdays% + %monthdays% + %day% - 1 > nul
#echo Number of Days past: %numdays%
set /a secondsC=%hour% * 60 + %minute% * 60 + %second% > nul
#echo Number of Seconds in current day: %secondsC%
set /a seconds=%numdays% * 24 * 60 * 60 > nul
#echo Number of Seconds of past days: %seconds%
set /a seconds=%seconds% + %secondsC% > nul
#echo Number of past Seconds since start of 1980: %seconds%
set /A seconds=%seconds% > nul &; set /a hexlen=%@retval% > nul
#echo Number of past Seconds since start of 1980 in hex: %seconds% hexlen=%hexlen%
if %hexlen%==10 && set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x%seconds:~-6,2%\x%seconds:~-8,2% && set volserial=%seconds:~-8,2%%%seconds:~-6,2%-%seconds:~-4,2%%%seconds:~-2,2%
if %hexlen%==9 && set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x%seconds:~-6,2%\x0%seconds:~-7,1% && set volserial=0%seconds:~-7,1%%%seconds:~-6,2%-%seconds:~-4,2%%%seconds:~-2,2%
if %hexlen%==8 && set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x%seconds:~-6,2%\x00 && set volserial=00%seconds:~-6,2%-%seconds:~-4,2%%%seconds:~-2,2%
if %hexlen%==7 && set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x0%seconds:~-5,1%\x00 && set volserial=000%seconds:~-5,1%-%seconds:~-4,2%%%seconds:~-2,2%
if %hexlen%==6 && set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x00\x00 && set volserial=0000-%seconds:~-4,2%%%seconds:~-2,2%
if %hexlen%==5 && set volumeid=\x%seconds:~-2,2%\x0%seconds:~-3,1%\x00\x00 && set volserial=0000-0%seconds:~-3,1%%%seconds:~-2,2%
if %hexlen%==4 && set volumeid=\x%seconds:~-2,2%\x00\x00\x00 && set volserial=0000-00%seconds:~-2,2%
if %hexlen%==3 && set volumeid=\x0%seconds:~-1,1%\x00\x00\x00 && set volserial=0000-000%seconds:~-1,1%
#echo Volume Serial Number: %volserial%
#echo Volume Serial Number little-endian: %volumeid% (for write FILE %^result%)
if %mode%==uuid && set result=%volserial% ! set result=%volumeid%
endlocal && set result=%result%
goto :eof

:DaysInYears
setlocal && set *
set /a year=%~2-1 > nul
set /a numdays=%~3 > nul
if %year%<=1979 && endlocal && set /a result=%numdays% > nul && goto :eof
:LoopDaysInYears
call :GetLeapYear GetLeapYear %year% &; set leap=%result% && set result=
set /a numdays=%numdays% + 365 + %leap% > nul
if %year%>=1981 && set /a year=%year% - 1 > nul && set leap= && goto :LoopDaysInYears
endlocal && set result=%numdays%
goto :eof

:DaysInMonths
setlocal && set *
call :GetLeapYear GetLeapYear %~2 &; set leap=%result%
set /a month=%~3 > nul &; set /a month=%month% - 1 > nul
set /a numdays=%~4 > nul
if %month%==0 && endlocal && set /a result=%numdays% > nul && goto :eof
:LoopDaysInMonths
checkrange 1,3,5,7,8,10,12 calc %month% > nul && set /a numdays=%numdays% + 31 > nul
checkrange 4,6,9,11 calc %month% > nul && set /a numdays=%numdays% + 30 > nul
if %month%==2 && set /a numdays=%numdays% + 28 + %leap% > nul
if %month%>=2 && set /a month=%month% - 1 > nul && goto :LoopDaysInMonths
endlocal && set result=%numdays%
goto :eof

:GetLeapYear
setlocal && set *
set /a year=%~2 > nul
checkrange 0 calc %year% % 4 > nul && set /a leap=1 > nul ! set /a leap=0 > nul
if %year:~-2,2%==00 && if not %year:~-3,3%==000 && set /a leap=0 > nul
endlocal && set result=%leap%
goto :eof

BTW all sub-routines are in the same .LLL-format, so can be easily substituted in other functions, or used as stand-alone function.

 

I checked a few whole years, comparing with of https://www.epoch101...seconds-in-2021 . Also seconds in the current year seems good (epoch101 uses GMT!). See the 'print screen':

grub> (f41) MAKESECU.LLL MakeSecluid 2021-01-01 00:00:00 
Date: 1-1-2021 Time: 0:0:0 
Number of Days in past years: 14976 
Number of Days in past months: 0 
Number of Days past: 14976 Number of past Seconds since start of 1980: 1293926400 
MakeSecluid: result=4D1F-C000 
grub> set a uuid2020=0x%result:~0,4%%result:~5%
1293926400 (HEX: 0x4D1FC000) 
grub> (fd1)/MAKESECU.LLL MakeSecuu id 
Date: 4-8-2021 Time: 23:7:41 
Number of Days in past years: 14976 
Number of Days in past months: 212 
Number of Days past: 15191 
Number of past Seconds since start of 1980: 1312585661 
MakeSecluid: result=4E3C-77BD 
grub> set a sec2021=-%uuid2020%+0x%result:~0,4%%result:~5%
18659261 (HEX:0x11CB7BD) 
grub> calc 212+3*24+23*60+7*60+41 ;; echo %@retval%
18659261 (HEX:0x11CB7BD) 18659261 
grub>

Brackets in calc, would be very handy. But I saw you already delivered that in a new batch....

 


Edited by deomsh, 04 August 2021 - 10:28 PM.


#94 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 05 August 2021 - 08:39 AM

Nice. :)

 

I am not too convinced of the:

 

 

#echo Number of past Seconds since start of 1980: %seconds%
set /A seconds=%seconds% > nul &; set /a hexlen=%@retval% > nul
#echo Number of past Seconds since start of 1980 in hex: %seconds% hexlen=%hexlen%
if %hexlen%==10 && set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x%seconds:~-6,2%\x%seconds:~-8,2% && set volserial=%seconds:~-8,2%%%seconds:~-6,2%-%seconds:~-4,2%%%seconds:~-2,2%
...

 part.

 

I mean, can't you pad with 0's up to length 10? :unsure:

 

Something loosely *like*:

 

set /A seconds=%seconds%

 

set stringsecs=000000000%seconds:~2%

set seconds=%stringsecs:~-10%

set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x%seconds:~-6,2%\x%seconds:~-8,2% && set volserial=%seconds:~-8,2%%%seconds:~-6,2%-%seconds:~-4,2%%%seconds:~-2,2%

 

(i personally hate IF's and conditional execution in general ;))

 

As another example (not necessarily a good one):

 

 

:LoopDaysInMonths
checkrange 1,3,5,7,8,10,12 calc %month% > nul && set /a numdays=%numdays% + 31 > nul
checkrange 4,6,9,11 calc %month% > nul && set /a numdays=%numdays% + 30 > nul
if %month%==2 && set /a numdays=%numdays% + 28 + %leap% > nul
if %month%>=2 && set /a month=%month% - 1 > nul && goto :LoopDaysInMonths
endlocal && set result=%numdays%
goto :eof

 

 

could be (just saying):

 

 

:LoopDaysInMonths
set /a numdays=%numdays% + 31 > nul
checkrange 4,6,9,11 calc %month% > nul && set /a numdays=%numdays% - 1 > nul
if %month%==2 && set /a numdays=%numdays% -3 + %leap% > nul
if %month%>=2 && set /a month=%month% - 1 > nul && goto :LoopDaysInMonths
endlocal && set result=%numdays%
goto :eof

 

Another way could be a call with parameters and shift :w00t:

 

:LoopDaysInMonths

call :to_days %month% 31 28 31 30 31 30 31 31 30 31 30 31

...

....

 

:to_days

shift %1

set /a numdays=%numdays% + %1 > nul

goto :eof

 

No idea if something like this could be actually "better" or "faster", only first idea that crossed my mind, and a general way to get values from an indexed array, possibly useful in some other cases.

 

:duff:

Wonko



#95 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 05 August 2021 - 09:01 AM

Making a separate post to introduce the (very marginal in this case) issue with Unix Epoch (or any other date/time expressed as seconds passed since a start date/time), leap seconds :ph34r::

 

https://stackoverflo...nd-leap-seconds

https://www.johndcoo...5/leap-seconds/

 

I.e. directly encoding a date/time coming from the system clock (even with +/- 1 second approximation) may be more accurate (in the sense of reflecting more accurately UTC) than timestamps that ignore leap seconds :w00t:.

 

After all the good MS guys that used the (poor) encoding I was talking about in my earlier post were not as bad as they have been depicted in all these years ...

 

For our internal use we could invent :w00t: a custom encoding similar to the referenced one:

 

https://docs.microso...s?view=msvc-160

 

starting from 2020 with 6 bits for the year (i.e. good until 2083 or starting from 2000 and good until 2063) and re-use the excess bit for the seconds field, using for them 6 bits instead of 5, that would give us second precision.

 

:duff:

Wonko



#96 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 06 August 2021 - 10:24 PM

Nice. :)

 

I am not too convinced of the:

 part.

 

I mean, can't you pad with 0's up to length 10? :unsure:

 

Something loosely *like*:

 

set /A seconds=%seconds%

 

set stringsecs=000000000%seconds:~2%

set seconds=%stringsecs:~-10%

set volumeid=\x%seconds:~-2,2%\x%seconds:~-4,2%\x%seconds:~-6,2%\x%seconds:~-8,2% && set volserial=%seconds:~-8,2%%%seconds:~-6,2%-%seconds:~-4,2%%%seconds:~-2,2%

 

(i personally hate IF's and conditional execution in general ;))

 

As another example (not necessarily a good one):

 

 

could be (just saying):

 

Another way could be a call with parameters and shift :w00t:

 

:LoopDaysInMonths

call :to_days %month% 31 28 31 30 31 30 31 31 30 31 30 31

...

....

 

:to_days

shift %1

set /a numdays=%numdays% + %1 > nul

goto :eof

@Wonko

Thanks, the first one is really easygoing. I made a screenshot. Not that I hate IF-statements, but to make 8 lines for all cases is a precise job... :(

 

Also a database as arguments for a call is an interesting possibility. While working at my setp/ commandline 'thing' I decided to write ALL keyboard codes with corresponding 'key' to memory. I used write, which gave very longlines (all \x's included, and between 0A's).



#97 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 06 August 2021 - 11:11 PM

Making a separate post to introduce the (very marginal in this case) issue with Unix Epoch (or any other date/time expressed as seconds passed since a start date/time), leap seconds :ph34r::

 

https://stackoverflo...nd-leap-seconds

https://www.johndcoo...5/leap-seconds/

 

I.e. directly encoding a date/time coming from the system clock (even with +/- 1 second approximation) may be more accurate (in the sense of reflecting more accurately UTC) than timestamps that ignore leap seconds.

Leap seconds... :merc:

I studied this subject a bit, and I see no problems to add leap seconds to my 'Seconds-from-1980-UUID'. I made one function, to provide leap years too (leap years function recycled from MAKESECU.LLL)

!BAT
#GETLEAP.LLL v0.1 (20210806), by deomsh.
#Function: Get leap years or years with leap seconds
#Use: GetLeap year|second yyyy [mm]
#Mandatory Arguments: GetLeap year|second yyyy Argument year|second can be left out, default is year. 
#Optional Arguments: [mm] Month: only with leap-seconds. Order is NOT free.
#Remarks: Leap-seconds: in database known until juli 2021. 
#debug 0
#echo %*
call :%~1 %*
echo Function: %~1 result=%result%
goto :eof

:GetLeap
setlocal && set *
if not /i "%~2"=="year" && if not /i "%~2"=="second" && set get=year
if /i "%~2"=="year" && set get=%~2 && shift
if /i "%~2"=="second" && set get=%~2 && shift
if %get%==year call :GetLeapYear GetLeapYear %~2
if %get%==second && call :GetLeapSecond GetLeapSecond %~2 %~3
endlocal && set result=%result%
goto :eof

:GetLeapYear
setlocal && set *
set /a year=%~2 > nul
checkrange 0 calc %year% % 4 > nul && set /a leap=1 > nul ! set /a leap=0 > nul
if %year:~-2,2%==00 && if not %year:~-3,3%==000 && set /a leap=0 > nul
endlocal && set result=%leap%
goto :eof

:GetLeapSecond
setlocal && set *
set /a leapsec=0 > nul
checkrange 1981,1982,1983,1985,1992,1993,1994,1997,2012,2015 calc %~2 > nul && set /a leapsec=1 > nul &; if %~3>=1 && if %~3<=6 && set /a leapsec=0 > nul 
checkrange 1973,1974,1975,1976,1977,1978,1979,1987,1989,1990,1995,1998,2005,2008,2016 calc %~2 > nul && set /a leapsec=1 > nul &; if %~3>=7 && if %~3<=12 && set /a leapsec=0 > nul 
if %~2==1972 && if "%~3"=="" && set /a leapsec=2 > nul
if %~2==1972 && if %~3>=7 && set /a leapsec=1 > nul
endlocal && set result=%leapsec%
goto :eof

BTW the optional month-argument is meant for the current year.

 

 

After all the good MS guys that used the (poor) encoding I was talking about in my earlier post were not as bad as they have been depicted in all these years ...

 

For our internal use we could invent :w00t: a custom encoding similar to the referenced one:

 

https://docs.microso...s?view=msvc-160

 

starting from 2020 with 6 bits for the year (i.e. good until 2083 or starting from 2000 and good until 2063) and re-use the excess bit for the seconds field, using for them 6 bits instead of 5, that would give us second precision.

 

I do not really see the use of these timestamp-UUID. What's the advantage?

 

With seconds there is no waste. By trial and error using MAKESECU.LLL I found 2016-02-07 06:28:15 as upper limit (leap seconds not used). But now there is pcalc.g4b to check:

grub> pcalc ((((((((2116-1980)*365)+((2116-1980)/4-1)+(31+6))*24)+6)*60)+28)*6> 
string=((((((((2116-1980)*365)+((2116-1980)/4-1)+(31+6))*24)+6)*60)+28)*60)+15 
string=(((((((136*365)+((2116-1980)/4-1)+(31+6))*24)+6)*60)+28)*60)+15 
string=((((((49640+((2116-1980)/4-1)+(31+6))*24)+6)*60)+28)*60)+15 
string=((((((49640+(136/4-1)+(31+6))*24)+6)*60)+28)*60)+15 
string=((((((49640+33+(31+6))*24)+6)*60)+28)*60)+15 
string=((((((49640+33+37)*24)+6)*60)+28)*60)+15 
string=(((((49710*24)+6)*60)+28)*60)+15 
string=((((1193040+6)*60)+28)*60)+15 
string=(((1193046*60)+28)*60)+15 
string=((71582760+28)*60)+15 
string=(71582788*60)+15 
string=4294967280+15 
 4294967295 (HEX:0xFFFFFFFF)
grub> _

BTW if any spaces are left, don't panic. I tried to clean the 'picture'. Many spaces were introduced by the 'not so bad' online-converter website I still have to use (no button found, also still can't edit old posts). Makes such posting rather unconvenient for me, and much too time-consuming.
 


Edited by deomsh, 06 August 2021 - 11:28 PM.


#98 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 15873 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 07 August 2021 - 07:30 AM

The issue is only a theoretical one (not any real consequence in real life).

 

An "encoded" timestamp (whichever way it is encoded) is actually a "snapshot" of what the real time clock of the PC showed, a "calculated as offset" timestamp is prone to errors in calculation by different calculation algorithms (i.e. "plain" Epoch vs. one which counts leap seconds).

 

As a matter of fact, I recommend you to NOT introduce/add leap seconds, as that means updating the routine/calculation once per year or so to be "current".

 

And, if we are going to be really picky, there is leap second smearing :w00t: :ph34r: which not *all* NTP servers do or did in  a same way (again only FYI):

https://developers.g....com/time/smear

 

:duff:

Wonko



#99 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 20 August 2021 - 09:43 AM

As a matter of fact, I recommend you to NOT introduce/add leap seconds, as that means updating the routine/calculation once per year or so to be "current".



The past is certain, the future not - especially in case of leap seconds (according to the wikipedia article). Total of leap seconds is now 27, since 1980 still 18. Updates can be on 30 june/ 31 december, so TWICE a year. Not updating the database will give only a difference of 1-2 seconds, same as setting time in BIOS manually (although UEFI seems to have possibilities to synchronize with UTC). Further the last leap second was in 2016 and it's not sure of leap seconds will be continued.

So your Swiss watchmakers should not format drives during leap second smearing, nor format portable drives in a different Time Zone.

I made a new version of GETLEAP.LLL which uses your idea of read-out a database from arguments of a call. To gain speed I added cumulative values. Also a base-year can be set.

Script: see next post - problems while posting with a smartphone

BTW I introduced your way of setting debug messages and saving the sending-to-nul bytes.
BTW2 I added a Help-function to my .lll-format, to easily read-out details of a function.

I tested speed differences, using the cumulative values is much faster (should be). But also debug msg=0 seems a bit faster (in the average). As soon I have access to real hardware I will repeat these tests. See the 'print-screen' below. (GETLEAPA.LLL is the previous version, GETLEAPB.LLL the new version still with > nul and GETLEAPC.LLL is the version above => BELOW now):

Output: see next post - problems while posting with a smartphone



#100 deomsh

deomsh

    Member

  • Advanced user
  • 99 posts
  •  
    Netherlands

Posted 20 August 2021 - 09:57 AM

!BAT
#GETLEAP.LLL v0.3 (20210819), by deomsh.
call :%~1 %*
#echo Function: %~1 result=%result%
goto :eof

:GetLeap
setlocal && set *
debug msg=0
if not /i "%~2"=="year" && if not /i "%~2"=="second" && set get=year
if /i "%~2"=="year" && set get=%~2 && shift
if /i "%~2"=="second" && set get=%~2 && shift
if %get%==year call :GetLeapYear GetLeapYear %~2
if %get%==second && call :GetLeapSecond GetLeapSecond %~2 %~3 %~4
endlocal && set result=%result%
debug msg=3
goto :eof

:GetLeapYear
setlocal && set *
set /a year=%~2
checkrange 0 calc %year% % 4 && set /a leap=1 ! set /a leap=0
if %year:~-2,2%==00 && if not %year:~-3,3%==000 && set /a leap=0
endlocal && set result=%leap%
goto :eof

:GetLeapSecond
setlocal && set *
set /a year=%~2
if /i %year%<=1971 && endlocal && set /a leapsec=0 && set /a leapsecC=0 && goto :eof
if /i "%~3"=="cum" && set mode=cum
if not "%~4"=="" && set /a baseyear=%~4
set /a leapsecC=0
if %mode%==cum && if exist baseyear && call :GetLeapSecondCum GetLeapSecondCum %baseyear% 2016=27 2015=26 2012=25 2008=24 2005=23 1998=22 1997=21 1995=20 1994=19 1993=18 1992=17 1990=16 1989=15 1987=14 1985=13 1983=12 1982=11 1981=10 1979=9 1978=8 1977=7 1976=6 1975=5 1974=4 1973=3 1972=2 &; set /a leapsecC=%result% && set result=
if %mode%==cum && call :GetLeapSecondCum GetLeapSecondCum %year% 2016=27 2015=26 2012=25 2008=24 2005=23 1998=22 1997=21 1995=20 1994=19 1993=18 1992=17 1990=16 1989=15 1987=14 1985=13 1983=12 1982=11 1981=10 1979=9 1978=8 1977=7 1976=6 1975=5 1974=4 1973=3 1972=2 &; set /a result=%result%-%leapsecC% &; endlocal && set result=%result% && goto :eof
set /a month=%~3
set /a leapsec=0 #&& set /a leapsecC=0
checkrange 1981,1982,1983,1985,1992,1993,1994,1997,2012,2015 calc %year% && set /a leapsec=1 &; if %month%>=1 && if %month%<=6 && set /a leapsec=0 
checkrange 1973,1974,1975,1976,1977,1978,1979,1987,1989,1990,1995,1998,2005,2008,2016 calc %year% && set /a leapsec=1 &; if %month%>=7 && if %month%<=12 && set /a leapsec=0 
if %year%==1972 && if "%month%"=="0" && set /a leapsec=2
if %year%==1972 && if %month%>=7 && set /a leapsec=1
endlocal && set result=%leapsec%
goto :eof

:GetLeapSecondCum
setlocal && set *
set /a year=%~2-1 && shift
set /a leapsecC=0
:LoopGetLeapSecondCum
if %~2<=%year% && endlocal && set /a result=%~3 && goto :eof
if not "%~2"=="" && shift && shift && goto :LoopGetLeapSecondCum
endlocal && set result=%leapsecC%
goto :eof

:Help
echo Function: Get leap years or years with leap seconds
echo Use: GetLeap year|second yyyy [cum [baseyear]]|[mm]
echo Mandatory Arguments: GetLeap year|second yyyy Argument year|second can be left out, default is year. 
echo Optional Arguments: cum,baseyear|mm Month: only with leap-seconds, without cum. Argument cum, baseyear can left out
echo Remarks: Leap-seconds: in database known until juli 2021. 
set result=
goto :eof

Limbo x86 PC Emulator
grub> debug 1 ;; looptest num:1000 insmod round:1 VARS:dummy;result POST.G4B
Testtime: 0.0 seconds / 1000 tests  <-:-> Bias: 6800.8 ms
grub> looptest GETLEAPA.LLL GetLeap second 2021 cum ; set result
Testtime: 330.9 seconds / 1000 tests
result=27
grub> looptest GETLEAPB.LLL GetLeap second 2021 cum ; set result
Testtime: 14.5 seconds / 1000 tests
result=27
grub> looptest GETLEAPC.LLL GetLeap second 2021 cum ;; set result
Testtime: 15.2 seconds / 1000 tests
result=27
grub> looptest runs:5 GETLEAPB.LLL GetLeap second 2021 cum
Testtime: 16.6 seconds / 1000 tests
Testtime: 18.3 seconds / 1000 tests
Testtime: 20.6 seconds / 1000 tests
Testtime: 19.3 seconds / 1000 tests
Testtime: 18.7 seconds / 1000 tests
grub> looptest runs:5 GETLEAPC.LLL GetLeap second 2021 cum ; ; debug status
Testtime: 15.0 seconds / 1000 tests
Testtime: 17.2 seconds / 1000 tests
Testtime: 18.1 seconds / 1000 tests
Testtime: 17.6 seconds / 1000 tests
Testtime: 18.1 seconds / 1000 tests
Debug is now 1
grub>







Also tagged with one or more of these keywords: grub4dos

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users