Jump to content











Photo
- - - - -

Announcing open source virtual SCSI miniport driver


  • Please log in to reply
147 replies to this topic

#126 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3042 posts
  • Location:Nantes - France
  •  
    France

Posted 07 October 2013 - 06:45 PM

version 1.0

 

-support sharedmemory proxy thru devio

 

devio shm:disk \\.\PhysicalDriveX 0 0 or devio shm:disk c:\disk.img 0 0

Then, indicate disk as the SHM 'device' in ImgMount.

 

Devio.exe (latest version) is in the zip file.

 

@Olof : what is the difference between using the offset parameter of devio or using the offset parameter in the Arsenal driver? Same result right?

For now, I hardcoded 0 as offset parameter to the driver thus at some point I was planning on making it visible in the GUI.

 

EDIT : removed the attachment and added ImgMount to the download section. See here.

 

/Erwan

Attached Thumbnails

  • mount2.png

  • Olof Lagerkvist likes this

#127 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 07 October 2013 - 06:54 PM

@Erwan

 

Perfect. You are doing a really great job with this. Really appreciated! :) :good:

 

Specifying offset to devio compared to driver should not mean any practical differences.

 

Because devio.exe natively supports some basic things with dynamic size vhd files, this means that dynamic vhd files could be used in this way. This does not include fancy stuff like differencing disk files or similar, but basic usage of vhd files is supported.

 

I would also think that it now would be easy to add support for the plugin system in devio. There have been modules around for some time linking to for example libewf.dll and similar. I also once made a module for using DiscUtils.dll in this way, so that users did not need to have .NET 4.0 installed. With this method, Framework 3.5 (.NET 2.0 core) is sufficient, which could be interesting in many cases.

 

Again, great piece of work!

:cheers:



#128 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3042 posts
  • Location:Nantes - France
  •  
    France

Posted 07 October 2013 - 07:17 PM

@Erwan

 

Perfect. You are doing a really great job with this. Really appreciated! :) :good:

 

Specifying offset to devio compared to driver should not mean any practical differences.

 

Because devio.exe natively supports some basic things with dynamic size vhd files, this means that dynamic vhd files could be used in this way. This does not include fancy stuff like differencing disk files or similar, but basic usage of vhd files is supported.

 

I would also think that it now would be easy to add support for the plugin system in devio. There have been modules around for some time linking to for example libewf.dll and similar. I also once made a module for using DiscUtils.dll in this way, so that users did not need to have .NET 4.0 installed. With this method, Framework 3.5 (.NET 2.0 core) is sufficient, which could be interesting in many cases.

 

Again, great piece of work!

:cheers:

 

Thanks Olof, You are welcome :)

After a long day of work, it just relaxes me to play with such a fine piece of code (your driver).

 

The Arsenal driver is fine with monolithic VHD file : it is after all only a RAW file with a 512 bytes footer.

And there are probably other raw files out there wich are simply appended with a fix chunk which I could then list in ImgMount.

 

For the other types of VHD (differencing, etc), I already have a delphi class to deal with the MS VirtualDisk native api (in windows 7 and up) so I could easily add support if there is a real need later.

Actually I even use this API in windows 8 to mount iso in CloneDisk

 

About devio modules, would you have some examples, or links to modules so that I can play with?

 

While we are at it, I feel I have made a full tour of the driver but are there any cool features I would have missed in current version?

Waiting for next version now ;)

 

There are still a few features I will add to ImgMount in shortly coming new versions like dumping a memory image to disk.

 

Thanks !

 

/Erwan



#129 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 07 October 2013 - 07:18 PM

The 0 parameter, for number of blocks, makes devio check the size of the underlying device automatically and subtract the offset from that. So, you will automatically use "the remaining" data, after your offset. If you want to use just a piece of that, for example one of many volumes, you need to specify that manually later, like you say.

 

If you are looking for using just one single volume, I would say that the partition number syntax I mentioned earlier is easier. It reads partition table and gets offset and size automatically from there. I personally use that syntax more with ImDisk.

Yep :), the original idea was to "generalize as much as possible" (i.e. without being "confined" to a partition entry).

 

As an example (in "real mode", with grub4dos) I can:

map --mem (hd0)12+217 (hd3)
map --hook

what is in memory and I can access later as (hd3) is just the amount of sectors I asked for (217 in the random example above).

 

Now, you may want to ask which practical use this could have, and to be frank I have not one right now :ph34r:, if not something connected (more or less loosely :ph34r:) to the never finalized (nor actually having much uses) OFS:

http://reboot.pro/to...ub4dos/?p=19803

http://reboot.pro/to...ub4dos/?p=24320

(which was abandoned, though promising, since the actual problem/issue that inspired it was solved differently)

 

@erwan.l

Very good. :smiling9:

 

Go, erwan go!  :cheerleader:

 

:cheers:

Wonko



#130 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 07 October 2013 - 07:40 PM

Thanks Olof, You are welcome :)

After a long day of work, it just relaxes me to play with such a fine piece of code (your driver).

 

The Arsenal driver is fine with monolithic VHD file : it is after all only a RAW file with a 512 bytes footer.

And there are probably other raw files out there wich are simply appended with a fix chunk which I could then list in ImgMount.

 

For the other types of VHD (differencing, etc), I already have a delphi class to deal with the MS VirtualDisk native api (in windows 7 and up) so I could easily add support if there is a real need later.

Actually I even use this API in windows 8 to mount iso in CloneDisk

 

About devio modules, would you have some examples, or links to modules so that I can play with?

 

 

I can see what I can come up with. Some sample code and devio syntax for it was described by v77 here:

http://reboot.pro/to...-proxy-example/

It has also been discussed now and then in this forum, so it could be worth searching around a bit.

 

But dynamic sized vhd files are automatically supported natively in devio.exe. But the implementation is extremely "basic", so it just support the most simple usages. Nothing like differencing disks and similar. For other cases I always use DiscUtils library instead. But that one requires .NET, so it might not always be possible.

 

 

Yep :), the original idea was to "generalize as much as possible" (i.e. without being "confined" to a partition entry).

 

As an example (in "real mode", with grub4dos) I can:

map --mem (hd0)12+217 (hd3)
map --hook

what is in memory and I can access later as (hd3) is just the amount of sectors I asked for (217 in the random example above).

 

Now, you may want to ask which practical use this could have, and to be frank I have not one right now :ph34r:,

 

I think I have only used this for one very specific case. The ugly disk layout that some people call "cascaded extended partition table", or something like that (don't remember exact). It is the case where a partition record in MBR links to extended partition table somewhere, just like it is supposed to be, but where this extended partition table in turn has further links to extended partition tables within itself. (With variants.) Generally, Windows can read and understand such partition tables quite nicely. But not safely edit them. In some scenarios you would simply lose some or all of the partitions if you remove one single partition from Disk Management. Therefore, it could be useful to mount the extended partition as a separate disk and let Windows read and edit that partition table like if it were the primary partition table.



#131 Blackcrack

Blackcrack

    Frequent Member

  • Advanced user
  • 458 posts
  •  
    Germany

Posted 08 October 2013 - 07:01 AM

an idea/suggestion,

it is possible per sub-menu from mouse if klick on the Drive in explorer ? (as extra option):

"Move/Copy virtual Disk to USB-Stick and make it bootable" / "Move to USBDrive and make Bootable"

if it is Hang in for have an Manager who exporting the vhd-file to an usbdrive/stick and add directly an grub-boot.

The idea behind, if anyone hang in the vhd-file and has work on it and want try out directly or want directly work with it, 

for have an possible to create directly or copy/move the file and let create an bootable usb-stick..

with an own small manager who can copy file/s and configuring the grub with the files who be included in the stick..

maybe different ones, vhd and/or iso  and other possible booting files

 

if it's possible .. ..

 

best regards

blacky



#132 TheHive

TheHive

    Platinum Member

  • .script developer
  • 4203 posts

Posted 08 October 2013 - 07:24 AM

Needs instructions or steps on getting started.

Run the gui and then guess that Install driver is next but then it shows a window to point to 'a driver'.

There are some zip files awealloc.zip,Drivers.zip, and devio.exe.

There is no mention of where to get devcon.exe, which is referenced  on the posts.

 

Thanks



#133 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3042 posts
  • Location:Nantes - France
  •  
    France

Posted 08 October 2013 - 07:55 AM

Needs instructions or steps on getting started.

Run the gui and then guess that Install driver is next but then it shows a window to point to 'a driver'.

There are some zip files awealloc.zip,Drivers.zip, and devio.exe.

There is no mention of where to get devcon.exe, which is referenced  on the posts.

 

Thanks

 

I have re done the zip file: zip files in a zip file was not a good idea. (re download).

 

Extract it to a folder named ImgMount.

In that folder you will find the main executable (imgmount) and 2 folders : one named awealloc, another one named arsenal.

Each of these contain drivers.

 

In ImgMount, when clicking install, it will offer to look for the inf file : go into the arsenal folder, your os version (winxp or win7), then select the inf file : phdskmnt.inf .

See that post from Olof for details around different driver versions.

 

For the awealloc driver, for now, this is a manual installation : right click, install.

You need this only if you decide to use physical memory (vs virtual memory) when creating a memory virtual disk.

 

For devcon, ImgMount should tell you if it cannot find it in the current folder where to download it. (here).

 

Hope this clarifies a bit the "how to".

 

Regards,

Erwan



#134 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 08 October 2013 - 09:59 AM

I have re done the zip file: zip files in a zip file was not a good idea. (re download).
 
Extract it to a folder named ImgMount.
In that folder you will find the main executable (imgmount) and 2 folders : one named awealloc, another one named arsenal.
Each of these contain drivers.

In ImgMount, when clicking install, it will offer to look for the inf file : go into the arsenal folder, your os version (winxp or win7), then select the inf file : phdskmnt.inf .
See that post from Olof for details around different driver versions.


How about some auto-selection of .inf file by Windows kernel major/minor versions, like the code I have in the setup GUI?
 
        If ver.Major >= 6 AndAlso ver.Minor >= 1 Then
            kernel = "Win7"
            hasStorPort = True
        ElseIf ver.Major >= 6 AndAlso ver.Minor >= 0 Then
            kernel = "WinLH"
            hasStorPort = True
        ElseIf ver.Major >= 5 AndAlso ver.Minor >= 2 Then
            kernel = "WinNET"
            hasStorPort = True
        ElseIf ver.Major >= 5 AndAlso ver.Minor >= 1 Then
            kernel = "WinXP"
            hasStorPort = False
        ElseIf ver.Major >= 5 AndAlso ver.Minor >= 0 Then
            kernel = "Win2K"
            hasStorPort = False
        Else
            kernel = "Not supported"
            hasStorPort = False
        End If
I have used this piece of code for driver version selection in various projects for quite some time. Ref: MainForm.vb in setup GUI project or MainModule.vb in setup CLI project.
 

For the awealloc driver, for now, this is a manual installation : right click, install.


...or install (latest) ImDisk.

#135 bilou_gateux

bilou_gateux

    Frequent Member

  • Expert
  • 230 posts
  •  
    France

Posted 19 October 2013 - 05:30 PM

Version 0.9

 

Based on comments from Winko

 

-added an uninstall option (no reboot needed for win7 and up)

-have updated XP driver with inf file from Bilou_Gateux

-install and uninstall now both use devcon.exe from MS

 

About rebooting under xp, it is actually not needed for the driver to work but only for the device manager to display correct informations.

 

Thats probably the best I can do around drivers installation as I dont plan for now to study the setuapi windows functions.

 

/Erwan

 

 

@Olof

I'll see if I can understand the modified .inf by Bilou_gateaux and find a way (I seem to remember that it is possible) to modify it in such a way to have also the CtlUnit thingy "embedded", so that we have a "monolithic" .inf.


:cheers:

Wonko

 

I have improved my batch script to install it from command line:

no reboot

no popup Windows

no Unknown Device in device manager

mount image ok from erwan.l GUI

 

@Windows install time:

REG ADD HKLM\software\Policies\Microsoft\Windows\DriverSearching /v DontSearchWindowsUpdate /t REG_DWORD /d 1 /f
REG ADD HKLM\software\Policies\Microsoft\Windows\DriverSearching /v DontPromptForWindowsUpdate /t REG_DWORD /d 1 /f
REG ADD HKLM\software\Policies\Microsoft\Windows\DriverSearching /v DontSearchCD /t REG_DWORD /d 1 /f
REG ADD HKLM\software\Policies\Microsoft\Windows\DriverSearching /v DontSearchFloppies /t REG_DWORD /d 1 /f

I use method described here.

REG ADD HKLM\Software\Policies\Microsoft\Windows\DeviceInstall\Settings /v SuppressNewHWUI /T REG_DWORD /d 1 /f

and rebuild INFCACHE.1 after 'integrating' ctlunit driver

DriverSigning-Off.exe
copy CtlUnit\ctlunit.inf %SystemRoot%\inf
copy CtlUnit\ctlunit.cat %SystemRoot%\i386
chktrust.exe -win2k -r -acl %SystemRoot%\i386\ctlunit.cat
InfCacheBuild.exe -k -f -i -p %SystemRoot%\inf
devcon32.exe install WinXP\phdskmnt.inf Root\PhDskMnt

InfCachebuild by Sherpya available here


  • Olof Lagerkvist likes this

#136 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 19 October 2013 - 06:07 PM

@bilou_gatueux
Nice work :).
I had no time to dedicate to any further testing :(, but I was thinking about the good ol' MS VSS service driver:
http://reboot.pro/to...storage-driver/
(side tool/batch maybe useful in conjunction with Arsenal Image Mounter too, here: http://reboot.pro/to...st-and-ddlistw/ )

Unless I am mistaken, it has the same "low level" hooking as the Arsenal thingy and the good MS guys made it installable/uninstallable without particular issues :unsure:

Maybe "tracing" what vstorcontrol does (and/or checking the source) it is possible to understand the mechanism used. :dubbio:

:cheers:
Wonko

#137 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 19 October 2013 - 06:54 PM

@bilou_gatueux
Nice work :).
I had no time to dedicate to any further testing :(, but I was thinking about the good ol' MS VSS service driver:
http://reboot.pro/to...storage-driver/
(side tool/batch maybe useful in conjunction with Arsenal Image Mounter too, here: http://reboot.pro/to...st-and-ddlistw/ )

Unless I am mistaken, it has the same "low level" hooking as the Arsenal thingy and the good MS guys made it installable/uninstallable without particular issues :unsure:

Maybe "tracing" what vstorcontrol does (and/or checking the source) it is possible to understand the mechanism used. :dubbio:


Yes, but like I mentioned before, this Arsenal Image Mounter, on XP, is a non-plug-and-play driver that allocates non-plug-and-play ISA bus resources just to satisfy scsiport.sys physical hardware resource requirements, without going all the way around making a separate virtual ISA bus driver just to emulate hardware resources for that part to work. This is a "minimalistic" workaround that does not require an additional driver, nor does it require any implementing of SCSI port or disk driver logic that built-in drivers already provide. This has, unfortunately, the drawback that since this driver becomes a non-plug-and-play driver with allocated physical hardware resources, it cannot be removed without reboot.

There are all sorts of ways to avoid that and make it possible to remove the driver without reboot if anyone wants. One could write a plug-and-play virtual ISA bus driver for scsiport.sys to be happy about plug-and-play resources like some projects have done. One could write a full SCSI port driver and avoid scsiport.sys altogether that some projects have done. To the storage stack and things like disk.sys and Disk Management, this will look pretty much the same. In my opinion, any of these alternatives would mean a few weeks of work of limited use because it is only needed on XP for the specific possibility to remove the driver without reboot. It will not provide any other features. On Server 2003 and later, there is storport.sys driver that accepts completely virtual plug-and-play miniports directly. But of course, if anyone wants to go for solving it somehow, I will do my best to support that project.

#138 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3042 posts
  • Location:Nantes - France
  •  
    France

Posted 19 October 2013 - 07:10 PM

Hi Guys,

 

As far as installation of drivers goes, I am rather ok with that part (even if indeed this part can be improved) :)

I have been using the Arsenal driver on several platforms these last days and had no problem installing / deinstalling and this without rebooting.

 

What I would like to see if new features added to the driver :

iscsi,

aoe (may be not need as other free/open source alternative exists),

cdrom emulation,

plugin support (to support other formats).

 

For sure, easy to ask from me  :)

 

Regards,

Erwan



#139 Rasqual

Rasqual
  • Members
  • 7 posts
  •  
    France

Posted 20 October 2013 - 02:58 AM

Hello Olof,

Thanks for the great tool :)

Maybe there was some option I missed out, but doesn't ArsenalImageMounterMountTool 1.0.9.0 (GUI) always mount images as readonly? I do not encounter this issue when adding from the CLI tool.

Windows 7 SP 1 x64

#140 Blackcrack

Blackcrack

    Frequent Member

  • Advanced user
  • 458 posts
  •  
    Germany

Posted 20 October 2013 - 06:24 AM

"aoe (may be not need as other free/open source alternative exists),"

 

but i bet with you, not many peoples want install 1001 programms for 1001 things ;)

therefor it is impotant to be give an program who support all possibles ..

and not  like Winzip, Winrar .. at start, only his own format therefore i take  bandizip ,

if you see what i mean :) therefor.. if you make an programm who be multiused for diffrent things/formats,

then make it right and maybe add some other possibles also

(and think not if it's need, because then you can think about the programme as self .... no, don't think about it.., imo it is an fail)

So, add all what it is possible and make the maximum of usable of it ;)

 

best regards

Blacky

 

Hi Guys,

 

As far as installation of drivers goes, I am rather ok with that part (even if indeed this part can be improved) :)

I have been using the Arsenal driver on several platforms these last days and had no problem installing / deinstalling and this without rebooting.

 

What I would like to see if new features added to the driver :

iscsi,

aoe (may be not need as other free/open source alternative exists),

cdrom emulation,

plugin support (to support other formats).

 

For sure, easy to ask from me  :)

 

Regards,

Erwan



#141 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 20 October 2013 - 10:19 AM

Hi Guys,
 
As far as installation of drivers goes, I am rather ok with that part (even if indeed this part can be improved) :)
I have been using the Arsenal driver on several platforms these last days and had no problem installing / deinstalling and this without rebooting.

I agree. This part works okay the way it is right now. But you cannot remove the driver without reboot on XP. But in my opinion, that is not worth too much work to solve.

What I would like to see if new features added to the driver :
iscsi,
aoe (may be not need as other free/open source alternative exists),
cdrom emulation,

I agree it would be a lot more useful to give priority to these things.

plugin support (to support other formats).

There is already a plugin system in the driver. It is compatible with shared memory option in ImDsk/devio. It is currently used when you mount virtual machine formats (vhd, vdi, vmdk etc) or encase files from the MountTool.

Hello Olof,Thanks for the great tool :)Maybe there was some option I missed out, but doesn't ArsenalImageMounterMountTool 1.0.9.0 (GUI) always mount images as readonly? I do not encounter this issue when adding from the CLI tool.Windows 7 SP 1 x64

When you open an image file in the MountTool, you can untick the "open read only" box in the open file dialog to mount in read/write mode.

#142 Rasqual

Rasqual
  • Members
  • 7 posts
  •  
    France

Posted 20 October 2013 - 11:09 AM

When you open an image file in the MountTool, you can untick the "open read only" box in the open file dialog to mount in read/write mode.


Okay, I'm so used to programs ignoring this box that I wouldn't care if it were ticked or not. Good to know.

Another issue I noticed is, even after you unmount an image, the file remains locked in kernel mode (you may delete the file in explorer, but it comes back when you hit refresh).
Closing the application automatically purges the file.

 

Edit: @erwan.l I checked out your tool, are there use cases where you'd run in as a standard user, or does it have to be run as admin under all circumstances?

Wouldn't it be better to add a manifest? (see http://stackoverflow...-in-manifest-in )


Edited by Rasqual, 20 October 2013 - 11:57 AM.


#143 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 20 October 2013 - 01:57 PM

Okay, I'm so used to programs ignoring this box that I wouldn't care if it were ticked or not. Good to know.

Another issue I noticed is, even after you unmount an image, the file remains locked in kernel mode (you may delete the file in explorer, but it comes back when you hit refresh).
Closing the application automatically purges the file.


I just tested and you are right. There is a handle leak somewhere. It is not in kernel mode though, it is in the MountTool somewhere. I will try to fix it. Thanks for this info!

#144 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 20 October 2013 - 11:43 PM

Found and fixed the above mentioned handle leak in MountTool. The problem was that when an image files was opened through DiscUtils.dll, a handle to the image file was left open when virtual disk had been removed.

 

Latest ArsenalImageMounterMountTool:

https://github.com/A...aster/MountTool



#145 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 22 October 2013 - 12:27 PM

UNrelated stuff :w00t: that noone here will care about :(, that I am posting just to have a handy bookmark:

http://support.micro...om/?kbid=298503

http://openvpn.net/a...1/msg00341.html

http://code.google.c...ec=svn279&r=279

 

Devcon (and it's source code) is now MS-LPL:

http://code.msdn.mic...Sample-4e95d71c

 

:cheers:

Wonko



#146 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 22 October 2013 - 12:43 PM

UNrelated stuff :w00t: that noone here will care about :(, that I am posting just to have a handy bookmark:
http://support.micro...om/?kbid=298503
http://openvpn.net/a...1/msg00341.html


Thanks! This is useful information, but yes, it is unrelated to matters discussed in this thread. The articles are about how to disable warnings etc about unsigned or incorrectly signed plug-and-play drivers or cat files. The discussion in this thread relate to correctly signed drivers and cat files. In one case even a non-plug-and-play thingy, that is not affected by the registry settings covered by the articles at all.

But anyway, it is useful information in somewhat related cases, where automated driver installs cause headaches for one reason or another. ;)

#147 kievigreen

kievigreen

    Newbie

  • Members
  • 18 posts
  •  
    Ukraine

Posted 05 March 2016 - 11:32 PM

HI all

And it is possible to start Windows in VHD as WinVBlock?


Edited by kievigreen, 05 March 2016 - 11:34 PM.


#148 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 06 March 2016 - 10:00 AM

HI all

And it is possible to start Windows in VHD as WinVBlock?

No, as it has no mechanism to hook grub4dos or memdisk mappings.

 

:duff:

Wonko






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users