Jump to content











Photo
- - - - -

lz4_compressor

vhd grub4dos lz4

  • Please log in to reply
30 replies to this topic

#1 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 13 June 2019 - 07:29 PM

Posted Image

File Name: lz4_compressor
File Submitter: alacran
File Submitted: 13 Jun 2019
File Updated: 24 Feb 2021
File Category: Tools

lz4_compressor was made using Autoit, it is a Compressor and Decompressor GUI for LZ4.

It was made to facilitate the LZ4 compression of fixed size Wimboot VHDs (Not applicable to expandable VHDs, since they don't boot when compressed), in order to greatly reduce the size of the file that will be loaded to Ram by means of grub4dos for MBR or for UEFI + SVBus driver, and this way improve the time to load it in Ram. Even if it was made for VHDs you can select any other files as *.iso, *.img or *.ima, etc., to compress them if you want.

The download has all required files/folders to work on x86 and x64 OSs. It is an executable that detects OS architecture and automatically selects the LZ4 x86 or x64 version as required.

Settings/Parameters:
-12 --content-size --favor-decspeed is preselected on this tool because result is very close to the smallest achievable size but having a faster decompression.
-12 --content-size is available on this tool too, as this is top level for compression.
--decompress --no-sparse are the actual parameters used since this version to decompress, to disable sparse mode (default) decompression that was creating an issue when decompressing VHD.lz4 files.
Additionally you can type your prefered parameters too.

LZ4 version included on Download is v1.9.1, see Note-2

LZ4 page on GitHub: https://github.com/lz4/lz4/releases

Thanks to my good friend Retokener (wimlib-clc author) for all the info, time and patience during the development of this new tool. Without his help this little program would not be working as expected.

Password of 7z file is my user name on this forum: alacran

This is only to avoid AV or browser to cancel download just in case a false positive virus warning.

The compression achieved highly depends on free space on the source file and compressibility of content.

NOTE: On this new version I fixed decompression of VHDs.

NOTE-2: I tested this program using the last 1.93 version of lz4 compressor and it do not give any meaningful advantage or reduction in time consumed or size of final compressed file, so didn't see the need to update it. But if the user wants to run his/her own tests just download the new lz4 version and extract it on the respective x64 or x86 folders of the lz4_compressor program, replacing previous files.

If you want to comment something related to lz4_compressor go to this thread: http://reboot.pro/in...showtopic=22062

NOTE: All links on this page to reboot.pro Topic(s) and Post(s) were updated using standard URLs, avoiding the use of Friendly URLs.


alacran

Click here to download this file
  • wimb and Tokener like this

#2 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 June 2019 - 08:21 PM

Hi Alacran,

 

Good job.

If I may, rather than calling the exe, you can also interface the dll/api directly.

 

All it is really is 3 functions to call : 

function LZ4_compress1(source:pointer; dest:pointer; srcSize:integer;dstCapacity:integer):integer;cdecl;
function LZ4_uncompress1(source:pointer;dest:pointer;compressedSize:integer;dstCapacity:integer):integer;cdecl;
function LZ4_compressBound1(isize:integer):integer;cdecl;
 
I decided to integrate this in latest version of Clonedisk as LZ4 is a great compression method delivering better performance compare to 1:1 in some cases (less cost on IO, more cost on CPU) and on modern CPU's.
 
Also, take in consideration that there is a legazy lz4 format (a simple few bytes header before each block) and modern lz4 format (a more sophisticated header).
Not sure which (or both) format support Grub4dos.
 
Regards,
Erwan


#3 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 13 June 2019 - 10:33 PM

thank u ever so much dear Alacrán. tried it the way it was set by default (-12 etc.). compress bound, I do not know, I did not even try it, could I have? and if I did, what would it do? make it any smaller? any bigger? any quicker?

btw, I think we have sound reason to stop calling it the WIMBoot era and start calling it the WIMBALACRAN Era!!!!

nino



#4 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 13 June 2019 - 11:55 PM

@ erwan.l

Thanks, maybe sometime in the future, I'm just starting with AutoIt and I still have a lot to learn, as said this tool was made with the help and guidance of Retokener.

About performance: I have compared LZ4 and gzip (from 7-Zip v19.0.0) and I can say LZ4 is a lot faster and give you a very much smaller file (more than 44 % smaller using LZ4 v1.9.1), AFAIK only this two compressors are valid on grub4dos 0.46a.

 

@ antonino61

Pre-configured parameters:

-12 --content-size --favor-decSpeed = Default Compression parameters, the best option to reduce loading to Ram time.

-12 --content-size = Optional Compression parameters to get the smaller *.lz4 file but not the faster to load on Ram.

-d = This is the parameter that allow us to decompress a *.lz4 file.

 

alacran

 

Edit: For decompression it was necesary to use:  --decompress --no-sparse to disable sparse mode (default) decompression that was creating an issue on VHD.lz4 files decompression.


  • antonino61 likes this

#5 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 14 June 2019 - 06:26 AM

thank you very much for the compression and decompression parameters, alacrán. what I was alluding to was erwan's last observation

function LZ4_compressBound1(isize:integer):integer;cdecl;

or, for that matter, all 3 of them. anything applicable now?

nino



#6 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 14 June 2019 - 09:54 AM

About LZ4_compressBound1 :
 
The below function
function LZ4_compressBound1(isize:integer):integer;cdecl;
is only needed to figure out the dstCapacity:integer
in the below function
function LZ4_compress1(source:pointer; dest:pointer; srcSize:integer;dstCapacity:integer):integer;cdecl;
 
So from a non dev p.o.v, this function is of no interest.


#7 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 14 June 2019 - 11:23 AM

Probably stating the obvious but note that performance will greatly vary depending on the content of your source.

 

Example below, backuping a empty (virtual) drive.

Now, the more datas will contain my drive, the more blocks will have to be processed (cpu), the more blocks will have to be written to disk (io), the slower it will get.

 

In my example below, clearly LZ4 outperforms by far the raw 1:1 method since my source is filled with zero and since only 4 MB of (compressed) datas will ultimately be generated and written to disk.

 

Also, ratio wise, LZ4 is rather on the low end compared to other compressors but clearly on the high end when it comes to speed.

This, unless you go for LZ4-HC but you then lose the benefit of speed on the compression end.

On that matter, LZ4-HC is still a good candidate in the grub4dos scenario : compress once with high ration, decompressed many times with great speed.

You then get the best of both world : one of the highest ratios, the best decompression speed ever.

 

There is a nice benchmark grid here : https://github.com/lz4/lz4.

 

Backup with LZ4 against an empty drive

 

Note that in reality my (mechanical) drive cannot even achieve such high speeds (64MB/s) but still, effectively, 1GB was backuped in 16 secs hence reporting 64MB/s.

 

n2bsXYc.png

 

Backup with RAW against an empty drive

 

DCZOrpx.png



#8 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 14 June 2019 - 01:41 PM

Probably stating the obvious but note that performance will greatly vary depending on the content of your source.

 

Also, ratio wise, LZ4 is rather on the low end compared to other compressors but clearly on the high end when it comes to speed.

This, unless you go for LZ4-HC but you then lose the benefit of speed on the compression end.

On that matter, LZ4-HC is still a good candidate in the grub4dos scenario : compress once with high ration, decompressed many times with great speed.

You then get the best of both world : one of the highest ratios, the best decompression speed ever.

 

Yes, performance (Speed) is inversely proportional to source file content (used space). And compressed file size is directly proportional to source file content (used space),  But this is usual behaviour of all compressors.

 

And just as you said the following makes LZ4 the best candidate when using grub4dos 0.46a, it is necesary to clarify older grub4dos versions AFAIK only support gzip compression:

On the grub4dos scenario : compress once with high ratio, decompressed many times with great speed. You then get the best of both worlds: one of the highest ratios, the best decompression speed ever.

 

Another thing to take in consideration is Wimboot fixed size VHDs (1.5 GB is my prefered size) usually have about 1 to 1.2 GB of unused space, wich make them a very good candidate for compression, as an example my 10x64-WB.vhd.lz4 file is only 130 MB when the source VHD is 1.5 GB, this is:

 

1536 - 130 = 1406 MB unwasted or 91.536 % space saved.

 

And comparing with an expandable VHD of same 10x64 (480 MB) just to see the diference:

 

NOTE: If an expandable VHD is LZ4 compressed, it DO NOT boot.:

 

480 - 130 = 350 MB unwasted or 72.916 % space saved.

 

I have to clarify new LZ4 version 1.9.1 has a little better compresion ratio than versions before v1.9.0 (wich has a bug fixed on v1.9.1).

 

alacran



#9 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 14 June 2019 - 02:05 PM

sorry, as I gather alacrán can do his math alright, I would like to have him check my ratio:  vhd.vhd 2,105mb compressed to 1,158mb vhd.vhd.lz4. the compression process ends with percentage counter at 55% ca. what I see at deployed c:\ is 2,000mb full size, 600mb used speed. is everything alright or can it be bettered?

nino



#10 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 14 June 2019 - 04:05 PM

@ antonino61

 

You are giving me two diferent sizes for the source VHD: vhd.vhd 2,105mb compressed to 1,158mb vhd.vhd.lz4 and also c:\ is 2,000mb full size, 600mb used.

 

But since the numbers are not too far from each other, I can say this:

 

All depends of used space and full size, a 600 MB used space from 2,000 MB full size, sounds fine to me, you still have 1,400 MB free, wich also sounds good to me, of course as allways used space depends of installed software and drivers and that is user preference.

 

If I had 32 GB of Ram as you do, I should not worry too much for the full size and I may have increased it to at least 4 GB, giving me more free space on (C:\) Rambooted OS, and virtually do not afecting loading to Ram time, since the *.lz4 file will be almost same size.

 

About numbers you said:

 

vhd.vhd 2,105mb compressed to 1,158mb vhd.vhd.lz4

 

It is 2105 - 1158 = 947 MB unwasted or (947*100)/2105 = 44.98 % space saved, (increasing the full size will increase saved space too).

 

About:

 

process ends with percentage counter at 55%

 

It is correct 55 % used + 44.98 % saved = 99.98 % (wich I take as 100 %)

 

alacran



#11 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 14 June 2019 - 04:32 PM

sorry alacrán, most of ur info is correct. mine is not so correct as urs - i must get confused when dealing with numbers - what I meant was 600 free space, I do not know how I could possibly have written used speed, which means nothing. sorry again. 600mb is the free space out of 2,000mb which is the full size. the rest of my info is ok. now u say that if I had, say, a 4gb full size, with the same useed space as I have (1400mb), it would not affect preramloading and booting times or that it would be even faster?

would the 4gb full size *.vhd that the *.vhd.lz4 comes from be any faster or slower or equal when booting as filedisk with normal bootmgr than the 2gb one that I have now?

what good does increased saved space do to the system?

ps.: I have 64gb ram



#12 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 14 June 2019 - 05:07 PM

1,400 mb of used space looks too big for a Wimboot install to me but as you know it depends of all runing software and drivers on your system, and that is only in acordance with your preferences.

 

600 mb free space sounds too small for my taste.

 

I allways try to have a minimum of 1 to 1.2 GB of free space on the VHD.

 

 

u say that if I had a 4gb full size, with the same useed space as I have (1400mb) it would not affect preramloading and booting times?

 

Yes, if it is compressed.

 

 

would it not affect booting times when booting as filedisk with normal bootmgr either?

 


A compressed file as *.lz4 can not be loaded as filedisk by bootmanager or grub4dos 0.46a, grub4dos loads it on Ram, and only on Ram.

 

 

what good does increased saved space do to the system?

 

 

None, it was only a comment to explain to you where saved space comes from.

 

ps.: I have 64gb ram

 

 

Good for you, congratulations.

 

alacran



#13 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 14 June 2019 - 06:02 PM

sorry alacrán, u were too fast to answer my questions, so the second last one is inaccurate according to my correction. pls check the ultimate version of my post and answer accordingly. as for the 600mb being too small, would u pls tell me what risks there are in keeping it so small? so far it has hardly ever changed. as for 1400mb being too much used space for wimboot, 35 items are installed, most of which are visual c++, directx and drivers (nvidia being the bulkiest).

nino



#14 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 14 June 2019 - 10:45 PM

well, I have tested the 4gb fixed vhd as filedisk with bootmgr. only half a second faster --> not worthwhile enlarging, if u also consider the great unlikelihood of its shrunk version (1.5gb) to bulge to red as a fixed vhd (260mb steady, across hours of work; the bulge is much more likely to occur with dynamic vhds); spacewise, the gain is even more noticeable. as for the lz4 version, it came down to 870 megs, which will preramload in 6 secs, to be added to the usual booting time (half a second slower than the 4gb version, which comes down to 1125mb as lz4). so the greater the free space the quicker the loading and booting? not necessarily, definitely not here. now what is the reason for the discrepancy in behavior between what I experience and what u experience? what am I doing that does not yield what u have envisaged? that's also not to mention that, despite the shrunk lz4 size is only 1125mb in the case of the 4gb vhd, if I check the resmon.exe results I see the 4gb occupying the first portion of memory instead of the newly obtained 1.5gb (I remember telling u this already a month ago or so, whether fixed or dynamic did it behave the same then, if I am not mistaken). if I check the disk size in the explorer window, I rightly see the uncompressed values (total size (4gb) and free space (2.something gigs), as opposed to 1.5gb total size and 260mb now). so the only effect of compression should be seen in the preramloading stage (ur main concern if I remember correctly, because it is the only thing u can control, i remember u saying): well in both cases I have recorded a 6sec delay relative to the uncompressed versions if loaded as filedisks with conventional bootmgr. so should one leave so much unused space to what avail?

nino



#15 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 15 June 2019 - 03:46 AM

There is not any fixed rule to follow here yet, you can use the sizes you prefer after all it is only a matter of preference, I only gave you some guidances according to my preferences that also have demostrated to me to work fine.  I suggest you to open a new post and run your own experiments (as I did) and publish your findings there, also concentrate only on time to load in Ram since it is the only thing you can more or less cotrol by means of file size to be loaded, the total booting time is very much more influenced with each program to be loaded during boot and can vary a lot on each user.

 

Try several VHD sizes allways using same Wimboot source *.wim, same PC and same Drive, and make note of each Full size, used size and time to load in Ram, latter when you have several samples you can put them in a table and you will find the tendence, try to annalize it by yourself, not only ask other people, also think by yourself.

Make sure all data you report is correct, verify it twice before posting since you are very prone to change one number for another.

 

alacran



#16 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 15 June 2019 - 04:54 AM

Well it is sad to me to inform there is someting wrong with LZ4 compressed VHDs decompression, I have allready proven it is caused by LZ4 program (not this GUI) since I got same results when doing it with this lz4_compressor GUI or directly running lz4.exe on command line (tested using LZ4 x86 and x64).

 

Also I have ran some test compressing and decompressing WinPE.iso(s) (using this GUI) too and all is working fine, even rebooted from decompressed WinPE.iso. The issue seems to be present only on VHDs.

 

But if I decompress a LZ4 compressed Wimboot VHD even if I get a *.vhd file with exactly the same SHA1 Hash it can't be mounted on Windows.

 

I tested several SHA1 before publish this GUI and all seemed to be fine, but never tryed to mout a decompressed VHD until now.

 

So far my findings are that if I check properties of the source VHD and the new decompresed VHD the size on disk is not the same, see attached pictures.

 

On following pictures 10x64-WB-.vhd is the source file and 10x64-WB.vhd is the decompressed file.

 

It seems during decompression the VHD is not enterely rebuilded and unused space is not taken in consideration, another theory may be it remains in some kind of compressed state, then not using the full size on disk.

 

But even if Windows is not able to mount the decompressed VHD it can be opened with 7-Zip as any other VHD file.

 

When the *.lz4 is loaded by grub4dos 0.46a it is decompressed and loaded to Ram and then Windows bootmanager is called to boot it, and all this process do not have any issue.

 

Any ideas?

 

Maybe karyonix (who was involved in implementing LZ4 capability on grub4dos0.46a) could give us some info.

 

EDIT: I just wrote an issue report on LZ4 page on GitHub: https://github.com/lz4/lz4/issues/732

 

alacran

Attached Thumbnails

  • VHD SHA1.png
  • VHD Properties.png


#17 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 15 June 2019 - 05:27 AM

@alacrán,

thanx for ur prompt reply. One more caveat on what u said: there is a 4second difference between booting off a dynamic vhd and booting off a fixed vhd, the latter being faster; so this is another thing that could influence booting times besides preramloading, which in my case, I repeat, did not yield so different free-space-bound results. as for numbers and tables, I might as well publish some, but that would not make the results any different from what I have already communicated. I will anyway.



#18 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 15 June 2019 - 08:57 AM

Well it is sad to me to inform there is someting wrong with LZ4 compressed VHDs decompression, I have allready proven it is caused by LZ4 program (not this GUI) since I got same results when doing it with this lz4_compressor GUI or directly running lz4.exe on command line (tested using LZ4 x86 and x64).
 
Also I have ran some test compressing and decompressing WinPE.iso(s) (using this GUI) too and all is working fine, even rebooted from decompressed WinPE.iso. The issue seems to be present only on VHDs.
 
But if I decompress a LZ4 compressed Wimboot VHD even if I get a *.vhd file with exactly the same SHA1 Hash it can't be mounted on Windows.
 
I tested several SHA1 before publish this GUI and all seemed to be fine, but never tryed to mout a decompressed VHD until now.
 
So far my findings are that if I check properties of the source VHD and the new decompresed VHD the size on disk is not the same, see attached pictures.
 
On following pictures 10x64-WB-.vhd is the source file and 10x64-WB.vhd is the decompressed file.
 
It seems during decompression the VHD is not enterely rebuilded and unused space is not taken in consideration, another theory may be it remains in some kind of compressed state, then not using the full size on disk.
 
But even if Windows is not able to mount the decompressed VHD it can be opened with 7-Zip as any other VHD file.
 
When the *.lz4 is loaded by grub4dos 0.46a it is decompressed and loaded to Ram and then Windows bootmanager is called to boot it, and all this process do not have any issue.
 
Any ideas?
 
Maybe karyonix (who was involved in implementing LZ4 capability on grub4dos0.46a) could give us some info.
 
EDIT: I just wrote an issue report on LZ4 page on GitHub: https://github.com/lz4/lz4/issues/732
 
alacran


I have not experienced that using the dll/api.
I will test this specific matter later today.

#19 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 15 June 2019 - 01:29 PM

@ erwan.l

 

I have not experienced that using the dll/api.
I will test this specific matter later today.

 

I will appreciate a lot your test, thanks in advance.

 

Are you using the very last v1.9.1?

 

I suggest you to use a fixed size VHD with some data on it, not an empty VHD to test this.

 

I'm waiting for an answer on LZ4 page on GitHub, maybe there is a bug on lz4.exe v1.9.1 and they can fix it soon.

 

alacran



#20 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 15 June 2019 - 03:32 PM

I did test the following :
 
-create a fixed vhd with some contents in there
-compress it
-uncompress it
-compare before/after : no different - exact same match
 
Give it a try with Clonedisk (32bits)?
 
I am using the 1.9.1 lz4 library.
 
ztY33ic.png


#21 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 15 June 2019 - 05:54 PM

@ erwan.l

 

When using lz4_compressor GUI my 10x64-WB.vhd.lz4 file is 130 MB

 

I tried Clonedisk 2.3.8 (32 bits) on 7x86 and also on 10x64 and in both cases it fail. AV disabled Admin acount, and UAC permanently disabled on both OSs.

 

On 7x86:

First compressed 10x64-WB.vhd (1536 MB) got  10x64-WB.vhd.lz4 (160 MB)

Then when decompressing got this message:

readfile error @ 168350876: An invalid access was made to the memory location, Continue and ignore next error?

 

On 10x64

First compressed 10x64-WB.vhd (1536 MB) got  10x64-WB.vhd.lz4 (160 MB)

Then when decompressing got this message:

readfile error @ 168350473: An invalid access was made to the memory location, Continue and ignore next error?

 

In both cases the re-created 10x64-WB.vhd is only 211 MB (should be 1536 MB) and can't be mounted on the OS.

 

Please see attached pictures.

 

I think you should try with a Wimboot VHD if you want to reproduce the issue. See Edit 2

 

EDIT: Tried to Ramboot with the 160 MB 10x64-WB.vhd.lz4 made with Clonedisk and it do not boot, there is a message about corrupted partition.

 

EDIT 2: Just to check on a non Wimboot VHD, I created Prueba.vhd 1536 MB fixed size (using Windows disk manager, initialized as MBR, and formated it as NTFS with Partition Guru, only primary active partition), copied into it an *.iso 252 MB file, detached the VHD and using Clonedisk 2.3.8 (32 bits) compressed it and latter decompressed it, and it fail, please see attaced pictures, starting from 1 Clonedisk from 7x86.png .... to 8 Bug-2.png,

 

alacran

Attached Thumbnails

  • 7 Bug-1.png
  • 8 Bug-2.png
  • 1 Clonedisk from 7x86.png
  • 2 Clonedisk from 10x64.png
  • 3 Recreated VHD.png
  • 4 Prueba-1.png
  • 5 Prueba-2.png
  • 6 Prueba-3.png

Edited by alacran, 16 June 2019 - 06:03 AM.
More info


#22 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 16 June 2019 - 09:37 AM

@Alacran: i will try to follow your steps and reproduce it on my side.
I guess the iso part is irrelevant right since we compress/uncompress a vhd ?

#23 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 16 June 2019 - 10:13 AM

@Alacran: i will try to follow your steps and reproduce it on my side.
I guess the iso part is irrelevant right since we compress/uncompress a vhd ?

 

Yes it was only a file of an good size copied to the VHD to test.



#24 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 16 June 2019 - 05:16 PM

Yes it was only a file of an good size copied to the VHD to test.

 

Ok I believe I managed to reproduce the issue on my side :

-created a 1.5GB vhd

-stuffed some data in there (an iso of about 250MB)

-compressed it : OK

-uncompressed : got an error... leaving me with a truncated file.

 

I use chunks of 512KB.

I have wrongly assumed that compressing 512KB of datas will always give a compressed chunk of 512KB (or less) and therefore while uncompressing while allocating a memory buffer of 512KB only (which normally was enough to cover my uncompressed data).

Actually in some rare situations, compressing 512KB of datas can end up in a compressed chunk slightlly bigger.

 

Long shorty short, this is now properly handled (LZ4 has an api to tell what to expect).

 

And data before / after is identical again.

 

Download latest Clonedisk 32 bits and try again to see if you are encountering issues in your experiments.

 

Thank you for spotting this bug !



#25 alacran

alacran

    Platinum Member

  • .script developer
  • 2710 posts
  •  
    Mexico

Posted 16 June 2019 - 09:49 PM


Download latest Clonedisk 32 bits and try again to see if you are encountering issues in your experiments.

 

 

Very good, The last version can compress and decompress Prueba.vhd and also 10x64-WB.vhd fine.

 

Only comments I have are:

 

1 - 10x64-WB.vhd.lz4 made with Clonedisk is 160 MB

 

2 - 10x64-WB.vhd.lz4 made with lz4 command line or the GUI is 130 MB (using this parameters: -12 --content-size --favor-decSpeed)

 

3 - Clonedisk do not recognize 10x64-WB.vhd.lz4 made with lz4 command line or the GUI (made using previous parameters) as a valid lz4 file if I try to decompress it.

 

EDIT: 4 - I don't know what parameters/settings (if any) you are using on your tool, but it only recognize an *.lz4 as a valid file when it was made with it, just tried several settings and no one was accepted for decompression.

 

I very well know more compression requires more time to get it, but when dealing with a file to be used for Ramboot the smaller the better, compression time is made only once and time is in no way critical.

 

This parameters " -12 --content-size --favor-decSpeed " is good when user wants to get the best compression but with a very good decompression speed, it seems specificaly made for Ramboot scenarios, so an option to select this setting would be good.

 

This parameter " -12 --content-size " is good when user wants to get the best compression no matter decompression time, usefull for compress info that will not be decompressed frecuently. I know this is not so important/usefull as previous since for this case we can use 7-Zip that is even a more known compressor and in some cases can give you more compression, also WofCompress from JFX and or latter WOF_Compress wimb GUI with the code you developed for it, can be a very good option in this case.

 

Thanks for all the time and effort you have inverted on this subject.

 

Clonedisk has allways been one of my favorite tools but sometimes it is a little confuse to use it with so many optios it has, it could be great if our good friend misty make one of his fantastic guides/tutorials for Clonedisk, I have the Hope he reads this comment, maybe he bites the bullet.

 

EDIT-2: Bad news my friend, the 10x64-WB.vhd.lz4 made with Clonedisk DO NOT Ramboot, info of heats/sectors per track is not found and then grub4dos selects the tipical 255/63 (wich in fact the source VHD was before compressing it, and allready re-checked with BootIce) and  there is a message about partition table invalid or corrupt.

 

Your friend

 

alacran







Also tagged with one or more of these keywords: vhd, grub4dos, lz4

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users