4 replies to this topic

[Wonko the Sane]

User Deomsh on msfn.org (he is also member here) found a peculiar issue with grub4dos (which I presume i connected with LZMA decompression).

 

Basically some (possibly all) window .lnk files ( shortcuts)  that have a header of 4C 00 00 00:

https://helgeklein.c...ing-a-shortcut/

are considered by grub4dos compressed files with obvious (bad) consequences with a number of commands. 

 

See starting from here:

https://msfn.org/boa...comment=1186398

 

More tests need to be made to ascertain more exactly the behaviour with various commands, targets and sources, but it affects also the internal dd, read and write commands besides cat, so it is advised to use etensively the raw command/prefix if you want to be sure that grub4dos doesn't automagically do some decompression on "false poitives".

 

Wonko

 

 

 

 

 

[tinybit]

The LZMA header is 13 bytes long:

 

1 byte      Properties

4 bytes    Dictionary Size

8 bytes    Uncompressed Size

 

grub4dos recognizes an LZMA file only if its Dictionary Size is a power of two, i.e., 2**n, and the Uncompressed Size is not 0xFFFFFFFFFFFFFFFF which is for "unknown".

 

Sure, the Properties byte must be less than or equal to 0xE8.

So the .lnk files are treated as LZMA files.

 

Grub4dos never resolve .lnk files. Users should avoid opening .lnk files from within grub4dos.

[Wonko the Sane]

Grub4dos never resolve .lnk files. Users should avoid opening .lnk files from within grub4dos.

Sure it does not, noone asks that, but it should treat the file "normally" when cat --hex read, write and dd, etc.

 

The issue was found when it was attempted to copy .lnk files.

 

If you prefer one need to use "raw" on *any* command, something that normally isn't used, IMHO (though I understand how at this time it is probably not possible anymore to correct the behaviour) the autodetection feature should be limited to only (say) the map command and an added "expand" parameter added to the other command if one really-really need to expand the lzma and gzip files.

 

As I see it, out of a multitude of generic commands that may be used on a poor, innocent file, only a few of them actually *need* expansion of a compressed file, and in those case the user actually:
1) knows that the file is compressed
2) actually wants to decompress it

 

This way with auto-recognition, a said particularly with LZMA that - unlike gzip - has not a "fixed" header, the risk of mis-recognition has moved from extremely rare/impossible (gzip) to rather possible and as we have seen there is an used (even if not very common) format that is mis-recognized as LZMA. 

 

Most probably the "best" option for the future would be to remove suppor for LZMA_Alone (leaving the .gzip one) and replace it with .xz format:

http://fileformats.a...wiki/LZMA_Alone

 

which has a "proper" header:

 

 

Header Magic Bytes

The first six (6) bytes of the Stream are so called Header
Magic Bytes. They can be used to identify the file type.

Using a C array and ASCII:
const uint8_t HEADER_MAGIC[6]
= { 0xFD, '7', 'z', 'X', 'Z', 0x00 };

In plain hexadecimal:
FD 37 7A 58 5A 00

Notes:
- The first byte (0xFD) was chosen so that the files cannot
be erroneously detected as being in .lzma format, in which
the first byte is in the range [0x00, 0xE0].
- The sixth byte (0x00) was chosen to prevent applications
from misdetecting the file as a text file.

If the Header Magic Bytes don't match, the decoder MUST
indicate an error.

 

Wonko

[karyonix]

The LZMA header is 13 bytes long:

 

1 byte      Properties

4 bytes    Dictionary Size

8 bytes    Uncompressed Size

 

grub4dos recognizes an LZMA file only if its Dictionary Size is a power of two, i.e., 2**n, and the Uncompressed Size is not 0xFFFFFFFFFFFFFFFF which is for "unknown".

Just this header is non-specific. It should not automatically run LZMA decoder based on this condition alone.
If you want auto-decompression, it should be limited to files that end with ".lzma".
For other files, LZMA decoder should be disabled by default and optionally be enabled by a preceding command.

[Wonko the Sane]

Just this header is non-specific. It should not automatically run LZMA decoder based on this condition alone.
If you want auto-decompression, it should be limited to files that end with ".lzma".
For other files, LZMA decoder should be disabled by default and optionally be enabled by a preceding command.

Yes, exactly and auto-recognition (also for gzip files) should (could) be limited to commands where decompressing a .gzip file makes sense (in practice limited IMHO to map --mem).

 

Or - this might be another valid approach, still IMHO - add an "environment variable" so that when writing a batch one could set it, *like*

set autouncompress=0

and be sure that files are treated as RAW, and can unset it, *like*:

set autouncompress=

if the author wants the "special feature of on-the-fly uncompression". 

 

Wonko