Jump to content











Photo
- - - - -

SVBus Virtual SCSI Host Adapter for GRUB4DOS

svbus virtual scsi host adapter grub4dos

  • Please log in to reply
137 replies to this topic

#26 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 26 November 2018 - 12:23 PM

@antonino61
For Grub4dos usage and the basics (including drive mapping), have a look at http://diddy.boot-la...os/Grub4dos.htm. It's an old guide, but is IMNSHO fairly well written and still relevant - at least for the basics. No disrespect, but based upon your posts you would be well advised to read up on the basics.

A more advanced guide is available at https://www.rmprepus...orials/grub4dos
 

...I have always had my vhd with mbr as C: on what appears as D: but it is in fact the physical c: (first hdd)...on the host drive (the physical c: which is OS-wise d:)...

Oh dear, references to mount points (e.g. C:) being described as physical drives were sure to get a response from Wonko :jaclaz:

Wonko appears to have been gentle in his responses (for a change). antonino61, be careful how you respond as Wonko's patience varies.

Windows has it's own logic for assigning drive letters. For example, in one of my multiboot systems -
(hd0,0) boot partition with Grub4dos AND bootmgr
(hd0,1) contains Windows 7
(hd0,2) contains Windows 8
When (hd0,0) is set as active and Windows 7 is booted from (hd0,1), (hd0,1) is mounted as C:
When (hd0,0) is set as active and Windows 8 is booted from (hd0,2), (hd0,2) is mounted as C:
When (hd0,1) is set as active and Windows 7 is booted from it, it is mounted as C:
When (hd0,2) is set as active and Windows 8 is booted from it, it is mounted as C:
When I boot WinPE from external media, whichever partition on (hd0) that's marked as active is mounted as C:

When referring to physical disks/partitions, I prefer to use the Grub4dos device syntax (see here).
 

The whole point (that I usually fail to convince people of) is that one should NEVER use a pre-made menu.lst when experimenting, BUT rather use the command line to send commands one by one (this way one can see which error - if any - or other feedback grub4dos produces)....


You convinced me years ago Wonko. Despite convincing me, I still used the pre-made menu.lst from the SVBus instructions - out of pure laziness. I remember enough about Grub4Dos to know what the commands in the sample do, and would have used the commandline had there been any issues with my setup. Honest!


Misty

#27 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 26 November 2018 - 12:26 PM

Now on to a Grub4dos related question.

When I first experimented with Firadisk and file backed disk mapping, the file backed disk needed to be contiguous or it could not be mapped.

Is this still an issue in Grub4dos?

And what about on SSD storage?

:cheers:

Misty

#28 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 November 2018 - 01:29 PM

When I first experimented with Firadisk and file backed disk mapping, the file backed disk needed to be contiguous or it could not be mapped.

Is this still an issue in Grub4dos?


Well, it is a feature, not an issue.

Q&A:
How do you suppose that grub4dos can map a non-contiguous image?
Well, it could well have a set of multiple blocklists/fragments.

And how can the loading driver (firadisk/Winvblock/SVbus) "hook" this hypothetical multi-blocklist?
It could as well parse the list of blocklists/fragments.

Would there be a limit to the amount of blocklists/fragments?
I am glad you asked this question :smiling9: , of course there would be a limit, most probably 1, 2, 9, 15, 255 or 32.255, at the moment this limit is set to 1 (one)  :jaclaz:
Now, if you think about a "PhysicalDrive" as a contiguous set of blocks (as a real physical device is) you can well understand how the virtual representation of it in grub4dos cannot but be contiguous, and that if the limit of one blocklist would  be changed *somehow* to - say - 9 the next post would be by someone that has a file in 10 fragments asking for 15, and if 15 was made the new limit the next user would be asking for 255 ... 
 

Wonko appears to have been gentle in his responses (for a change). antonino61, be careful how you respond as Wonko's patience varies.

... usually for *some reasons* ...
 
 
:duff:
Wonko

#29 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 02:16 PM

Sorry about the inaccuracy. What I meant by "physical C:" was actually \\.\PhysicalDrive0. What I meant by my perplexity about your (hd0,1) was that I could not see the applicability of it to my situation. Anyway, now I will try (hd0,1) and simply (hd0) on the filedisk entry and tell you what has happened. 



#30 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 November 2018 - 02:23 PM

Anyway, now I will try (hd0,1) and simply (hd0) on the filedisk entry and tell you what has happened.

 
You see?  :frusty:  
  

The whole point (that I usually fail to convince people of) is that one should NEVER use a pre-made menu.lst when experimenting, BUT rather use the command line to send commands one by one (this way one can see which error - if any - or other feedback grub4dos produces).

 
:duff:
Wonko

#31 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 02:49 PM

i did try the commands one by one as wonko suggested we do.

find --set-root --ignore floppies /vhd.vhd

yielded 

(hd0,0)

map /vhd.vhd (hd0)

yielded

total sectors ... etc. (which I perceived as something good)

map --hook

yielded nothing (which I perceived as something good)

rootnoverify (hd0)

yielded

cannot mount the selected partition, so I tried

rootnoverify (hd0,1)

which yielded something worse, like no such partition or something, as far as I can remember, so I tried

rootnoverify (hd0,0)

which yielded nothing (I perceived this as something good too), so I went

chainloader /bootmgr

which yielded

will booot ... etc, so I typed

boot

and here I am

does this all mean that I am booting from my vhd?



#32 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 26 November 2018 - 03:04 PM

rootnoverify (hd0)
Device (hd0) is the whole disk - not a partition.

rootnoverify (hd0,1)
Device (hd0,1) is Disk 0, Partition 2 (using MS numbering - disks are numbered from zero and paritions from one). Assuming this is a VHD with only one parition, (hd0,1) does not exist.

rootnoverify (hd0,0)
Set the root device as Disk 0, Partition 1. Try using the command root (hd0,0) instead and compare the output.

And yes, you probably are booted from the VHD. Use diskpart to check. E.g.
Select disk 0
Detail Disk

#33 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 03:04 PM

and then, now that I have looked at my \\.\physical drive 0 (I mean 1st hdd), I only see the grldr called bootmgr on its root (no boot dir nor real bootmgr, which are present only in the vhd). so, I do not see how my system can possibly boot from an inexistent bcd on the host drive. the only possibility that i see is that my system boots thanks to the bcd in the vhd.



#34 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 03:06 PM

I had already told wonko that root (hd0,0) does work. I will try it again and tell you if it makes a difference.



#35 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 03:12 PM

dear misty, your assumption is correct. your suggestion that I should use root ... instead of rootnoverify proved successful, plus the booting is kinda quicker. so I will make that permanent, if there are no more caveats from you guys.


Edited by antonino61, 26 November 2018 - 03:12 PM.


#36 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 26 November 2018 - 03:52 PM

Use diskpart List Disk command to identify your VHD file based on it's size. E.g. -
Microsoft DiskPart version 6.3.9600

Copyright (C) 1999-2013 Microsoft Corporation.
On computer: W530

DISKPART> list disk

  Disk ###  Status         Size     Free     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  Disk 0    Online          223 GB      0 B
  Disk 1    Online           20 GB      0 B
On my system, the VHD file is 20GB in size and is easy to identify from the two available disk entries.

Use diskpart Select disk # command to select the disk (in my case Select disk 1)

Now run the detail disk command and check the output.

SVBus disk -
Spoiler



Native boot VHD -
Spoiler


#37 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 26 November 2018 - 04:08 PM

When I first experimented with Firadisk and file backed disk mapping, the file backed disk needed to be contiguous or it could not be mapped.

Is this still an issue in Grub4dos?

Well, it is a feature, not an issue.

Q&A:
How do you suppose that grub4dos can map a non-contiguous image?
Well, it could well have a set of multiple blocklists/fragments....


:white_flag: :white_flag: :white_flag:

I'm out of touch with grub4dos and haven't kept up with developments, hence my question.

Also, I know very little about Solid State Drives - my test system has a new Crucial BX300 drive. Some of the information that I've read implies that files get scattered around and potentially fragmented to help with wear levelling. Maybe I've been lucky on my new system as every file I have tried to boot has appeared contiguous - maybe this is due to how the SSD disk controller works?

I've also read articles that strongly advise against defragmenting files on SSD storage to prevent wear. And some articles imply that the controller does its own thing anyway.

Does fragmentation even occur on SSD storage?

:white_flag:

Misty

#38 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 04:27 PM

I did try diskpart and it reports practically the same as you have shown in your case.

just out of curiosity, how long does it take to preload 20gigs into ram? are u sure you are ok with your current deployment?



#39 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 November 2018 - 05:52 PM

@Misty

What you might be missing is that (since some 20 years or more) hard disks are ALREADY (just like SSD's) some kind of blackboxes.

I.e. when you write a file on a normal hard disk there is *something* (the filesystem and its driver) that interrogates *something else* (the hard disk controller) about the contents of sectors.

The hard disk controller interrogates *yet something else* (the internal mappings and G-List and/or P-List) to make a correspondence between the filesystem/driver request and the actual mapping on the disk platters.

Flash media introduced wear leveling, but that doesn't change the complete "abstraction" between the actual device and what the filesytem/driver *sees*.

Imagine that you have a very small device:

 

 

JFYI, the generic idea (simplified) on how wear leveling works on a flash device (be it a USB stick or a SSD or a CF card) is the following.
1) the actual device has a controller
2) the controller holds addresses (Real addresses, in the following "R") and addresses (Translated addresses, in the following "T"), through a table of correspondences, including the info about how many times the address has been written to.
3) the T-addresses are less than the R-addresses (there are some "spare" sectors, in the following "S") and are the only address that the OS can actually access

Imagine you have a brand new (very small) device that has in total 5 sectors (of which the OS will see only 3), when it is brand new the table will (loosely) look something *like*:

R0=T0;0
R1=T1;0
R2=T2;0
R3=S0;0
R4=S1;0

After some time of use the situation might become (for some reasons the operating system writes many more times to T1 than on T0, while T2 is written an "average" of times):
R0=T0;314
R1=T1;2120
R2=T2;874
R3=S0;0
R4=S1;0

the wear leveling algorithm may decide that the sector R1 (that the OS accesses as and uses as T1) has been written way too many times and then intervenes, doing the following:

1) copy the contents of sector R0 to S0
2) copy the contents of sector R1 to S1
3) copy the contents of S0 to R1 AND remap the R0 to T1
4) copy the contents of S1 to R0 AND remap the R1 to T0

The situation will become:

R0=T1;314
R1=T0;2120
R2=T2;874
R3=S0;1
R4=S1;1

and after some time of similar usage, will eventually become something *like* (in brackets previous writes+new writes):

R0=T1;2314 (314+2000)
R1=T0;2410 (2120+290)
R2=T2;1640 (874+766)
R3=S0;1
R4=S1;1

:duff:

Wonko
 



#40 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 26 November 2018 - 05:54 PM

@antonino61

I did try diskpart and it reports practically the same as you have shown in your case....

:thumbsup:


...just out of curiosity, how long does it take to preload 20gigs into ram?...

I hadn't timed it. But seeing as you asked. Just tested and it took 1 minute 32 seconds. This is reading the file from a Crucial BX300 SSD.
 

...are u sure you are ok with your current deployment?...

Not sure what you mean by my current deployment? I'm experimenting and having fun - so I'm more than happy.

@Wonko
I will read through your last post slowly to try and digest the information. Cheers.

@everyone
I have now tested 64-bit versions of Windows XP, Windows 7 (Home Basic and Ultimate), Windows 8.1 Update (Enterprise) and Windows 10.0.14393 (LTSB).

All working without any obvious issues. XP was more difficult to setup, but only because I'm more used to .wim style deployments. The difficulty with XP was due to the need to intergrate AHCI drivers into the F8 floppy containing SVBus - and merging Txtsetup.oem files.

Thanks for all of your hard work schtrom :thumbsup:

:cheers:

Misty

#41 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 06:22 PM

@misty

I guess you could have even more fun if you moved (and later junctioned) all out of the vhd to some other drive except  the following: the boot dir, bootmgr, windows\appreadiness, windows\boot, windows\cbstemp, windows\en-us, windows\fonts, windows\inf, windows\logs, windows\performance, windows\prefetch, windows\system32\catroot, windows\system32\codeintegrity, windows\system32\config, windows\system32\drivers and the root files in c:\, c:\windows and c:\windows\system32.

all the rest can reside on another drive (junctioned to c:\, of course), so as to leave you with a 3850-or-something-gig vhd and guarantee the persistence one requires to use the system also on an everyday basis, even from ram. the small size of the vhd thus obtained reduces preloading times and alllows the system to reach the windows interface.

Whether the files residing on another drive will load more slowly on account of not being in ram from the very beginning anymore is still an open question, at leas for me. so, I am willing to submit it to wonko and/or any other expert in this forum.



#42 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 26 November 2018 - 06:44 PM

@antonino61
I was thinking of trying wimboot (see here) and/or CompactOS - just haven't gotten around to it yet.

wimboot from a .wim file on the internal drive should be possible and will significantly reduce the required .vhd file size. Alternatively having the .wim on the vhd (loaded into RAM) should save some space and be more responsive.

Now I just need the time to test it all.

Misty

#43 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 26 November 2018 - 10:15 PM

just finished reading the thread. not worth the trouble. all in all, as wonko once said, do compress only if u have to, otherwise just don't. and I totally and thoroughly agree with him, as the rationale here is: what you gain in space you lose in time, and the other way round. so, all in all, my solution still proves the best, for speed and persistence, unless you have other no-compress alternatives.


Edited by antonino61, 26 November 2018 - 10:15 PM.


#44 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 28 November 2018 - 07:54 PM

getting back to the grub4dos loading thing, just in case something goes wrong, is there any way to have it load windows in safe mode or in any other way whatever than normal mode?



#45 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 28 November 2018 - 11:13 PM

...is there any way to have it load windows in safe mode or in any other way whatever than normal mode?


Assuming you are referring to Windows 7/8/8.1/10, you could modify the BCD store in your .vhd file. E.g.
bcdedit /store c:\boot\bcd /set {default} advancedoptions yes
This will always display the F8 boot menu.

Misty

#46 antonino61

antonino61

    Gold Member

  • Advanced user
  • 1525 posts
  •  
    Italy

Posted 29 November 2018 - 08:32 AM

thank you ever so much, misty.



#47 Dietmar

Dietmar

    Frequent Member

  • Advanced user
  • 243 posts
  •  
    Afghanistan

Posted 31 January 2019 - 06:31 PM

https://www.usenix.o...n/halderman.pdf

 

https://faui1-files....es_coldboot.pdf

 



#48 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 31 January 2019 - 06:39 PM

https://isitchristmas.com/

or, if you prefer:

http://hasthelargeha...heworldyet.com/

 

:duff:

Wonko



#49 Gerolf

Gerolf

    Member

  • Members
  • 75 posts
  •  
    Germany

Posted 12 April 2019 - 03:29 AM

When I had problems installing Windows 2000 to VHD using WinVBlock, Wonko advised me to try SVBus -- and it worked as desired. While the SVBus tutorial shows an elegant way to use the Windows 10 Setup media to create a VHD on the clean internal hard disk of a real computer, I decided to use a virtual machine. I created three VHDs of 1, 2 and 16 GiB with Windows Disk Management and also attached the large one using that program, not ImDisk, because the VHD is accessible with Rufus then, so that I could install Grub4DOS to it.

 

Then I copied the small VHDs onto the large one. I also had to create a floppy disk image, to copy the contents of the SVBus bin folder to it, and to copy this SVBus floppy image to the large VHD. Also a Windows 2000 ISO was required. The MENU.LST could be copied from the tutorial, just the file names had to be adapted. I created a new virtual machine in VBox and attached the outer 16 GiB VHD to the IDE controller. There were no problems installing Windows 2000 to the inner 1 GiB VHD and to boot the system afterwards. All this was easily done, compared with my former efforts.

 

The tutorial does not mention Windows XP, however, and when I tried to repeat the procedure with that system, I got an error message during the text mode part of the Setup: The files svbus.inf and svbusx86.sys could not be copied to the inner 2 GiB VHD. So the second part of the Setup did not start unless I had copied both files manually to \Windows\inf and \Windows\System32\drivers, respectively. Afterwards, also Windows XP could be installed and run successfully in VBox. Fine!

 

Gerolf

 



#50 Guest_AnonVendetta_*

Guest_AnonVendetta_*
  • Guests

Posted 17 June 2019 - 06:10 AM

I have an Win10 LTSC x64 install.wim, customized with MSMG Toolkit to remove all unwanted bloat from the image. Now I want to add SVBus to the WIM so that it becomes a part of the OS even on a clean install. I see on the first page of this thread that instructions have been provided for injecting the SVBus drivers into a WIM. But what about the Registry entries? Are they even necessary? That seems partly covered too, but the instructions are for a 32 bit/x86 WinPE. I'm trying to use this with an installed Windows, not a PE.

 

Is it required to run Windows 10 x64 in Test Signing mode if SVBus is to be used? If so then it is unsuitable for my purposes. There seems to be an attachment for a signed driver posted on the first page, but Windows refuses to install it when right clicking on the inf. Error:

 

"Install error: The hash for the file is not present in the specified catalog file. The file is likely corrupt or the victim of tampering."

 

When trying to use the official installer, but replacing some of the files with the signed files, install succeeds, but I get an Unknown Device in Device Management.







Also tagged with one or more of these keywords: svbus, virtual scsi host adapter, grub4dos

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users