165 replies to this topic

[antonino61]

well, taking a closer look at it, I can give some credit to alacran for a 1sec difference 29 vs 30 secs on global booting (from start of preloading to arriving at the interface) with lz4 vs fixed vhd, which, in my view, is still not worth the trouble of compressing, speedwise. spacewise, it is. let me now check if it is possible to boot the lz4 dynamic vhd, but I pretty much doubt it.

in a bit

nino

[antonino61]

well, as I was saying, impossible to compress dynamic vhd any further and make it boot. so for transportation purposes, u might as well compress it, but  in order to ramboot it u have to decompress it back again. dynamic vhd rambooting versus lz4 fixed vhd --> practically a 1 second difference in favor of lz4. space-saving-wise, 768mb (used space on dynamic vhd) vs 184mb (lz4-compressed fixed vhd). u know urselves what is best for each of u.

[Wonko the Sane]

 

All test ran on same I3 3225, 3.3 GHZ, 8 GB Ram at 1333 MHZ

1- 1536 MB : 12s -> 128 MB/s

2- 480 MB: 7s -> 69 MB/s

3- 233 MB: 4s -> 58 MB/s

4- 135 MB: 3.67s -> 37 MB/s

 

#3 and #4, since include decompression are influenced by the algorithm and by the processor power, so on faster machines should be able to improve times.

3 on 4: Size ratio: 233/135=1.72 Transfer speed ratio: 58/37=1.57

 

 

1- 1536 MB : 12s -> 128 MB/s
2- 233 MB: 7s -> 33 MB/s
3- 480 MB: 4s -> 120 MB/s
4- 135 MB: 3.67s -> 37 MB/s

 

 

#2 and #4, since include decompression are influenced by the algorithm and by the processor power, so on faster machines should be able to improve times.

2 on 4: Size ratio: 233/135=1.72 Transfer speed ratio: 33/37=0.9

The speed ratio is near to 1, whilst size ratio is almost double that, on machines with slow media and fast processor lz4 will win, on machines with fast media and slow processor gz may result faster.

 

 

#1 is seemingly near to the max transfer rate.

 

#2 is the one that seems like not as fast as it could .

 

For fast machines, I would say that nothing can beat compressed images.

 

An interesting test would be if there is any noticeable difference between:

 

lz4.exe -9 --content-size e:\win.vhd e:\win.vhd.lz4

and:

 

lz4.exe -12 --content-size --favor-decSpeed e:\win.vhd e:\win.vhd.lz4

 

The #2 remains - at face value - slower than expected, it should mean that the processor is involved anyway

 

Another possible way, static truncated .vhd (RAMboot ONLY):

http://reboot.pro/to...n-many-ramdisk/

 

The same image of 1536 MB could be truncated to 500-550 MB in size but experience the "full" transfer speed, almost independent from processor speed.

 

Wonko

[alacran]

1- 1536 MB : 12s -> 128 MB/s

2- 480 MB: 7s -> 69 MB/s

3- 233 MB: 4s -> 58 MB/s

4- 135 MB: 3.67s -> 37 MB/s

 

#3 and #4, since include decompression are influenced by the algorithm and by the processor power, so on faster machines should be able to improve times.

3 on 4: Size ratio: 233/135=1.72 Transfer speed ratio: 58/37=1.57

The ratios are very near, on machines with slow media and fast processor lz4 will win, on machines with fast media and slow processor gz may result faster.

 

 

#1 is seemingly near to the max transfer rate.

 

#2 is the one that seems like not as fast as it could .

 

Wonko

 

@ Wonko

Sorry to tell you, but you are wrong, you missplaced 2 and 3, since used the order on my post, but it was in accordance with when it was tested and also coincidentally the speed, Reordering in the right way (for facilitate the calculations) we have this:

VHD								Time to load on RAM

1 - 10x64-WB.vhd Ramboot using g4d  grub4dos-0.4.6a-2018-12-23		12 seconds

2 - 10x64-WB.vhd.gz Ramboot using g4d  grub4dos-0.4.6a-2018-12-23	7 seconds

3 - 10x64-WB-EXP.vhd Ramboot using g4d  0.4.5c Modd by kyrionix		4 seconds

4 - 10x64-WB.vhd.lz4 Ramboot using g4d  grub4dos-0.4.6a-2019-03-25	3.67 seconds


VHD					Used	Saved espace

1 - 10x64-WB.vhd			1536 MB	0 % saved espace

2 - 10x64-WB.vhd.gz			233 MB	84.83 % saved espace

3 - 10x64-WB.vhd Expandable		480 MB	68.75 % saved espace

4 - 10x64-WB.vhd.lz4			135 MB	91.21 % saved espace


Then:

1- 1536 MB / 12s >>>> 128 MB/s

2- 233 MB / 7s >>>>>>> 33 MB/s

3- 480 MB / 4s >>>>>> 120 MB/s

4- 135 MB / 3.67s >>>> 37 MB/s

Then all makes more sence 1 and 3 are not compressed then they load at a higer rate in MB/s, 2 and 4 are Compressed so they load at a slower rate in MB/s, because of the CPU time involved to decompress them, but since 4 has the smaller size (just 135 MB), and also loading slightly faster than 2 (which loads at 33 MB/s) at 37 MB/s (on my PC) it is the faster.

 

Of course all this numbers may vary from one PC to another. Depending of CPU, Ram and Mass Storage device speeds, but this can give you an idea to let you select the option you prefer in accordance with your own case.

 

alacran

[Wonko the Sane]

Good , then it all makes sense, deleting/correcting incorrect data in my previous post.

 

The test with different lz4 parameters won't likely make much of a difference on your hardware, that appears as being relatively fastish in data transfer and slowish as CPU, maybe it is worth it for someone in the opposite case fastish CPU and slowish transfer rate (example CF card via USB 2.0 adapter).

 

Wonko

[alacran]

The command: lz4.exe -9 --content-size e:\win.vhd e:\win.vhd.lz4

 

Makes a file about 2 MB smaller than the other command: lz4.exe -12 --content-size --favor-decSpeed e:\win.vhd e:\win.vhd.lz4

 

Which supposed to be for improving decoding speed. I don't expect any significative difference between both.

 

EDIT: It was a typo, in reduced size.

 

alacran

Edited by alacran, 27 March 2019 - 08:35 PM.

[Wonko the Sane]

The command: lz4.exe -9 --content-size e:\win.vhd e:\win.vhd.lz4

 

Makes a file about 2 gb smaller than the other command: lz4.exe -12 --content-size --favor-decSpeed e:\win.vhd e:\win.vhd.lz4

 

Which supposed to be for improving decoding speed. I don't expect any significative difference between both.

 

alacran

2 GB smaller?

 

Wonko

[alacran]

Impossible, I mean 2 MB, just corrected previous post, thanks.

[Wonko the Sane]

Impossible, I mean 2 MB, just corrected previous post, thanks.

 

Wheeew, everything is back to normality .

 

The  -9 or -12 are levels of compression, it is IMHO normal that there is little difference in size at high levels of compression, what I presume may (or may not) make a difference is the "--favor-decSpeed" switch.

 

Wonko

[antonino61]

dear wonko and alacrán,

the best in terms of speed, space and operation together comes to be the old alacran's dynamic vhd. alternately, one might prefer lz4 if it is going to be used all the time, without variations (this is why I factored operation in). Cpu-wise, there is not much of a difference on an i9+nvme+m.2+ssd architecture like mine. In my view, u fail to consider the importance of post loading in ram (the rest of the booting), which is more taxing than the first stage u seem to focus so much on. Of course, u must have your own reasons for doing so. I have an overall hardware and software architecture, so I think I have to consider the whole process, not just some details. I do not think the battle is so much on the second or fraction of second as it is in a more discursive approach which will surely sound unscientific, but it is still the only way to see and tell the wood from the trees.

[alacran]

@ Wonko

 

lz4.exe -12  --content-size --favor-decSpeed i:\W81x64_ES_2.vhd i:\W81x64_ES_21.vhd.lz4

Size: 126 MB    Loading to Ram time: 3.88 seconds >>>>> still the fastest


lz4.exe -9 --content-size i:\W81x64_ES_2.vhd i:\W81x64_ES_22.vhd.lz4

Size: 124 MB    Loading to Ram time: 4.85 seconds

 

EDIT: Take this as only approximate, since my eyes - finger reaction time is not so fast as required.

 

 

@ antonino61

 

It is so simple as this:

 

I concentrate so much in time to load in Ram, because it is the only time I can influenciate/alter/change by means of the file loaded to Ram, been it: uncompressed fixed size, uncompressed expandable size, compressed gz, compressed LZ4 (2 different options).

 

Once the file is loaded on Ram there is nothing I can do to change booting time, well the source wimboot wim file compression may affect this time (especially on very slow machines, but it was already mentioned since post No. 1) and I can live with that since size reduction of this file is something we are looking for.

 

Then since Once the file is loaded on Ram there is nothing I can do to change booting time, then that is something I don't care anymore.

 

alacran

[antonino61]

are u sure there is nothing that can be done once the file is loaded on ram?

[alacran]

are u sure there is nothing that can be done once the file is loaded on ram?

 

If you have an idea let me know.

[antonino61]

pls tell me what exactly happens once the file is loaded on ram and tons of ideas will shower on ur head. I am only trying to be optimistic; for a start, my mind is going towards what gets loaded which is not applicable to our hardware and software architecture, but not only that. I am thinking of dedicated routines on a per-machine basis.

[alacran]

The booting process goes on and you can't interupt or alter it.

 

Disable all not required to boot as AV if any, programs that load during boot at every boot, prefetch, superfetch, unnecessary services as Windows Search, etc.

 

In your case you have commented you have a lot of things running on your OS, disable them and load them only if and when you will use them.   But all you can do is BEFORE loading on Ram.

 

That's why I said there is nothing I can do to improve boot speed once loaded on Ram.

 

alacran

[antonino61]

so all the *.sys files are needed regardless of the hardware one has? do we need 100 printers, 59 monitors 64 mice and stuff? 

just for the sake of comparison, have u timed the whole booting process on ur machine?

do we know why conventional booting with bootmgr is 4 seconds quicker than filedisk booting with grub4dos (in the case of fixed vhds, of course)?

[alacran]

I'm not interested on an OS that only can boot in a single set of hardware, so I don't care. I need an OS that can boot on any PC.

 

Your second question has already been answered by Wonko, different drivers give different speed, MS driver is faster than SVBus Driver.

 

alacran

[antonino61]

I think a quicker driver will be written one day.

[alacran]

JFYI. It is also possible to make wimboot source wim files with multiple indexes (as any other wim file), in fact I have just made a 10-WB.wim LZX compressed, containing as index 1 Win10x86 and as index 2 Win10x64, so from each respective index I deployed this 2 wimboot VHDs: 10x86-WB.vhd and also a separate 10x64-WB.vhd

 

This is very usefull to save espace if we want to have x86 and x64 versions, saved espace on wimboot source wim file:

Wimboot Wim file	Size

10x64-WB.wim		3.92 GB

10x86-WB.wim		2.21 GB

Total espace		6.13 GB


10-WB.wim		3.73 GB

6.13 - 3.73 = 2.4 GB = 39.15 % saved espace

 

 

alacran

[antonino61]

A great dubbio/duda/doubt, not to say perplexity.

To whomever may answer this question - I am calling upon all men of good intellect and understanding, such as wonko, alacrán, wimb and/or anybody else around.

Would it be unmaidenly to hypothesize some sort of inverse-proportional correlation between the level of compression of the wim file and the "bulging" of the vhd? 

I mean it is almost the 10th time that I have enlarged (1.5gb-3.0gb) my lzx-compressed dynamic vhd (lxz-compressed 6gb) as a result of successive bulging. I have edited some text with wordpad and with wpsoffice for a couple of hours. I have not installed anything new. Now the vhd is going 250mb free space. can it be too little at 3gb?

nino 

[blackbalfur]

In fact you are slowing down windows when you do not give it enough room to operate.

 

https://www.howtogee...our-windows-pc/

 

You need to figure out how to use windows without making it grow that large or you just give it enough room so it can run freely.

 

If you would use portable alternatives outside the vhd it would not grow that much.

 

Instead of wpsoffice you could try portable LibreOffice and run it outside the vhd file:

 

https://portableapps...office_portable

 

I think you should be more open to alternatives if you want to use such cramped vhd sizes to be fully functional and fast.

[antonino61]

Dear Blackbalfur,

I appreciate ur advice, I already have portable apps, as I had already downloaded the software upon ur suggestion a few days ago. As regards LibreOffice, I already have it portable, it is part of the same package, which can update all software included. I am already applying ur advice on the vhd, which is lzx-compressed, as well as the wim file, though. What I still am not sure of is whether the slowing down of everything is only affected by the amount of free space on the vhd or is also affected by vhd and/or wim compression levels, namely, would it be any better if they both were xexpress-compressed? The free space thing, I think I have solved already in the meantime, should I resort to smaller compression levels or is the trade-off anyplace else? Wonko has already said that high-end pcs hardly ever suffer the effects of compression on speed and can also let u enjoy smaller loading times. What do u say to that?

nino

ps.: the funny thing is that nothing happens to free space if I edit photos or watch films or videoclips. it mostly happens with text processing!

[blackbalfur]

The amount of free space (the lack of it) will allways result in your os slowing down.

 

My opinion is that the compression of the vhd files has no influence of the working speed in the os when totally booted.

 

Because it is loaded in ram and the compressed copied to ram vhd has no meaning then anymore.

 

Another story is the compressing on the wimboot file, the more compression the more it has to "work" to get things working inside the vhd.

 

alacran has stated this in this topic.

 

wonko has a good point that high-end computers suffer less on the effects of the compression.

 

Just because they have more power to get things going.

[antonino61]

My Dear Blackbalfur, 

following ur statements, I have just tried all compression options (even no compression at all), both for the wim and for the vhd, by enlarging both files according to the various alternatives provided by gimagex, vhd_boot, etc ... well, just a negligible difference in speed (in the order of fractions of seconds, and I am not even sure whether less or more) on this pc, and a considerable difference in size. So, now that I have solved the lack-of- space problem on my original vhd, I'd rather stay as I am - the best of all possible worlds, again, at least on this pc.

what still makes me rack my brains is that free space will not shrink an inch even if I play high graphics games, or as I was saying, edit pictures, etc. only when I process text with wpsoffice does it shrink. can it be that the software specifically expects to find pagefile.sys which is totally inexistent here so it tries to make up for it somehow, e.g. by trying to cache something?

nino

[Wonko the Sane]

... or as I was saying, edit pictures, etc. only when I process text with wpsoffice does it shrink. can it be that the software specifically expects to find pagefile.sys which is totally inexistent here so it tries to make up for it somehow, e.g. by trying to cache something?

Well, there is nothing preventing you to create a (small) pagefile on the .vhd, if that is the case.

 

But it is not that the "bulging" cannot be tracked.

I mean, create a booting system (.wim + .vhd).

Make a copy of BOTH.

Boot the first set of .wim+.vhd.

Stop/shut down the system after a simple boot, without making any activity on it.

Boot the second set, run in it the *whatever* wpsoffice program doing the things that normally triggr the "bulging".

Once the .vhd has become "tight" in space, stop/shut down the system.

Now, booted from a "normal" system, attach the first .vhd and make :

DIR /S>C:\Mytempdir\1stsetvhd.txt

 then attach the second .vhd and make a:

DIR /S>C:\Mytempdir\2ndsetvhd.txt

 

Compare the two files (I suggest you to use a proper comparison tool, *like* - example - ExamDiff http://www.prestosof...dp_examdiff.asp).

 

Which differences do you find?

 

Wonko