Announcing open source virtual SCSI miniport driver

I am happy to announce that a commercial virtual SCSI miniport project I have been working on about a year ago, has now been released as open source under A-GPL license.

 

Arsenal Image Mounter, at GitHub:

https://github.com/A...l-Image-Mounter

 

It is essentially a virtual storport miniport driver, working under Windows Server 2003 (with KB932755) up to Windows 8. It is also possible to use it under Windows XP, it will then act as a non-pnp ISA scsiport miniport driver, but essentially with the same features.

 

Unlike ImDisk, this thingy emulates a SCSI adapter with SCSI disks. This makes it automatically integrated with things like Disk Management, Volume Shadow Copy services, partitioning and so on. Then, just like ImDisk, it provides features for redirecting disk I/O requests to a proxy using shared memory communication. This uses the same protocol as devio with ImDisk, so same devio.exe could be used in this case.

 

Directory structure:

 

DriverSetup
Setup tools and signed driver packages that can be used to install the driver components. Run the GUI setup tool in this directory to install the driver. No reboot should be necessary, except in some scenarios on Windows XP.

 

MountTool
Compiled and ready-to-run simple GUI mount tool and a command line mount/management utility. Primarily intended to show what this project can be used for.

 

UserMode
Visual Studio 2010/2012 solution with all user-mode components. This includes Arsenal.ImageMounter.dll that can be referenced from other applications to use the public API. It also includes driver setup applications and some debug tools.

 

KernelMode
This directory contains a WDK 7 kernel mode driver project. This builds the kernel mode component used to create a virtual SCSI adapter.

 

Comments are of course welcome. Please remember though, that the code is not owned by me. It is free to use in open-source, A-GPL compatible, projects. There are alternative licensing options available for commercial use.

 

I am happy to announce ...

 

And I am happy about the announce!

 

I see also (and with the greatest pleasure ) that a 2K version exists!

 

Wonko

Hi Olof,

 

Very nice.

 

I installed the driver (win7) : went all fine.

Then used the Image Mounter to mount a raw image : i now have a "physical" disk appearing in my disk management console

 

It has bustype = fiber ? no big deal but may be worth noticing.

 

Also, a GUI to install the driver (like imdisk) would be nice?

 

I also like the applet concept within imdisk. Could this be applied to this storport driver as well?

Right now, when i close the mounter, my E drive vanishes and one could be concerned about opened handles, etc.

EDIT : i can use ArsenalImageMounterControl.exe /add /filename=x:\_images\xp.img, and my disk will persist.

 

As a whole, great work for sure.

I am sure this will soon become a must have tool like imdisk.

 

Regards,

Erwan

Hi Olof,
 
Very nice.

 
Thanks!
 

I installed the driver (win7) : went all fine.
Then used the Image Mounter to mount a raw image : i now have a "physical" disk appearing in my disk management console

It has bustype = fiber ? no big deal but may be worth noticing.
 
Also, a GUI to install the driver (like imdisk) would be nice?

 
Just download DriverSetup and run ArsenalImageMounterGUISetup.exe. Click Install. That's it, pretty much.
 

I also like the applet with imdisk.
Right now, when i close the mounter, my E drive vanishes and one could be concerned about opened handles, etc.
EDIT : i can use ArsenalImageMounterControl.exe /add /filename=x:\_images\xp.img, and my disk will persist.

That is correct. That command line utility is also good to have when you want to use a devio like tool to mount some special image file format. You can specify /proxy=shm and an object name for the /filename parameter.
 

But as a whole, great work for sure.
I am sure this will soon become a must have tool like imdisk.

The GUI mount tool dismounts everything when you close it. This is pretty much just to be safe. If you for example mount a vdi file using the "Through DiscUtils" button, the image file access is done within this application. If you were allowed to exit the application without dismounting, it would just hang on disk I/O. This is not really a problem for raw devices, but to make things simple they follow the same logic.
 
But anyway, I have seen that there are better GUI coming up. There seem to be some good guys working on it.

it is not an suggestion, if you integrate it directly in you ImDisk ?
for have maybe some possibility's more ?
best regards
Blacky

it is not an suggestion, if you integrate it directly in you ImDisk ?
for have maybe some possibility's more ?
best regards
Blacky

 

 

It is quite a different thing. Technically they are entirely different. The driver parts have almost nothing in common. However, I have tried to use as much ImDisk-compatible code as possible when it comes to things like "devio" communication protocol, public API and similar things. But the basic technical differences make it even difficult to keep the public API entirely the same. So, I would think they would need to live their lives in separate places.

 

Also, licensing is different. It would, however, be possible to fork off a A-GPL compatible ImDisk project and include this new image mounter there. But, then, why? It would still mean several different code bases to manage.

oh, oky thank's Olof for the fast reply
best regards
Blacky

 
Thanks!
 

 
Just download DriverSetup and run ArsenalImageMounterGUISetup.exe. Click Install. That's it, pretty much.
 

That is correct. That command line utility is also good to have when you want to use a devio like tool to mount some special image file format. You can specify /proxy=shm and an object name for the /filename parameter.
 

The GUI mount tool dismounts everything when you close it. This is pretty much just to be safe. If you for example mount a vdi file using the "Through DiscUtils" button, the image file access is done within this application. If you were allowed to exit the application without dismounting, it would just hang on disk I/O. This is not really a problem for raw devices, but to make things simple they follow the same logic.
 
But anyway, I have seen that there are better GUI coming up. There seem to be some good guys working on it.

 

I've obsiously been a bit too enthousiatic with regards to this project.

 

ArsenalImageMounterGUISetup.exe popup message

 

To run this application, you first must install one of the following versions of the .NET Framework:

v4.0.30319

 

 

 

Windows 7 supporting .NET 3.5 out of the box.

 

I'd stick to .NET 2.0.  It's a very small install if it's not present, It was present 95% of the time on Windows XP.

 

I'm running Windows XP embedded (extra low footprint with complete functionality fits 1GB system partition) 

 

can't install bloated newer versions of .NET.

 

Could we have in near future imdisk like installer and command line tool running out of the box without forcing us to use latest Microsoft technologies.

I am afraid not. From start, this project was designed to support Windows XP or later. That made it possible to use .NET 4.0, which saved a quite large amount of development time compared to if we would have used for example .NET 3.5. So right now, this piece of software uses .NET 4.0 specific things in far too many ways to make translation to .NET 2.0/3.5 compatible core and easy task. Or put it this way, it is quite possible that this project had not been developed at all if we would have needed to use only .NET 3.5 compatible features.

 

You could of course, however, install the driver without any .NET requirements. Just use the install legacy hardware wizard in Windows Vista or 7 Device Manager, Install Hardware wizard in Control Panel in Server 2003 and select the correct .inf file among the setup files. On XP, you would instead use "sc create" command and manually copy the driver file to system32\drivers. All of this should be fairly easy to automate in some way without .NET 4.0 requirements, if anyone would like to try it.

 

Then, someone could, probably fairly easy, build a version of ArsenalImageMounterControl.exe command line control tool that would use a framework on the .NET 2.0 core, for example .NET Framework 3.5. It would be possible with this to mount raw disk images, make ram disks or redirecting I/O to native tools like devio.exe. But all integration with external image file libraries such as DiscUtils or libewf would be impossible to use.

 

It should also be said that this is not a problem on Windows 8, where .NET 4.0 is supported out-of-the-box.

 

It should also be said that this is not a problem on Windows 8, where .NET 4.0 is supported out-of-the-box.

Windows 8 has much more serious problems than just .NET 4.0, IMHO , for all the rest of actually working MS Operating Systems the show stopper is of course .NET .

 

Mind you I perfectly understand the reasons you are using or were compelled to use .NET 4.0 , but still the final result is unfortunately of no practical use on most "experimental" or "low resource" builds (where actually an alternative driver would have been useful).

 

Back to Total Mounter, at least for dinosaurs' use.

 

Wonko

Windows 8 has much more serious problems than just .NET 4.0, IMHO , for all the rest of actually working MS Operating Systems the show stopper is of course .NET .

 

Mind you I perfectly understand the reasons you are using or were compelled to use .NET 4.0 , but still the final result is unfortunately of no practical use on most "experimental" or "low resource" builds (where actually an alternative driver would have been useful).

 

Back to Total Mounter, at least for dinosaurs' use.

 

 

Of course I understand this. But, the driver itself of course does not need .NET. To make use of at least the most basic tasks around it, I really hope that someone could make some simple control application that would not require .NET, at least not 4.0.

 

However, I have a strong feeling that contributions will rather be graphical user interfaces than minimalistic control programs. But, that is just a feeling based on somewhere around 999 out of 1000 forum/twitter/e-mail/etc posts or so...

Hi Gents,

 

if you need basic control interface to handle the driver, be it from the command line, or thru a GUI, not requiring dotnet, I'll be happy to help with my modest delphi skills.

 

Delphi is (maybe) old stuff but at least it is "native" i.e using windows api's only.

 

Regards,

Erwan

Hi Gents,

 

if you need basic control interface to handle the driver, be it from the command line, or thru a GUI, not requiring dotnet, I'll be happy to help with my modest delphi skills.

 

Delphi is (maybe) old stuff but at least it is "native" i.e using windows api's only.

 

 

That would be most appreciated.

 

Essentially what you need to do, is to open the \\.\ScsiN: device and send IOCTL_SCSI_MINIPORT control code down this device, with the various SMP_IMSCSI_xxx controls and data structures defined in KernelMode/inc/common.h. However, this common.h file, most MSDN help and other references sources are in C/C++, so you would need some translation to Delphi. In C/C++ you would simply #include this file and fill the structures with data.

Sounds good.
Will give it a shot this week focusing first on mounting / unmouting a drive.

Regards,
Erwan

Hi,
an suggestion ..

it is maybe not an suggestion for building up to be more an "System Mount Tool"
who be include submenu, and can maby handle partitions also + iso-mounting over http
and drive-mounting with ext234,xfs and other linuxpartitions also ?
Where support mounting and unmounting .. from diffrent possibilitys and not only iso's and v-drives ..
maybe an pluginsystem include who can build plugins for it,
who can write plugins for an plugin-api ..and with this can anyone write an plugin..
(hummm.. thinking... Total Mounter ?) who be maybe zip, 7zip(7zip.dll) and rpm's also mountable ..
maybe with write support ? via plugins ? (zipcreator as plugin, Iso-creator with caching via plugin.. read out in temp and write the iso +an lil addon for change the iso-configuration, maby make bootable, remove and add and so on .. via plugins.. so can anybody who can programming in dephi, or compatible Api-language from the plugin, create an plugin for mounting and r/w anyting .. via right mouse-klick on the files

if you are building up an new interface, why not then make an real System integrate program ..
an Command center as well as plug in support supported and an TMControl in Controlcenter(From WinNT) and right Mouseklick Submenu'ed *s* on the Explorer Program where Control all Mountable things .. from FS up to files and folders (subst and samba/network also on folders before the user login*)(and maybe others also and this with an plugin-system for enable to use single extensions and extensions-packs ...

(*so it is possible for the Admin, for create an nfs-like Filesystem and make it possible
for any Workstation to import/export this fs to an other Computer from the user.. maybe from an Server
who be all accounts stored .. like an Linux extports/NFS system with YP but this on Windows/WindowsNT-like , mounting/importing other extern folder to an internal folder it is possible like i know )


best regards
Blacky

Go erwan, go! 

 

Wonko

Hi Olof,

 

About the driver part, on windows 7, it was smooth and easy.

On XP, using the provide tool did not do the trick. Doing it manually did not do the trick.

At best does the driver appears in error (device not ready), even after a reboot.

 

Also got confused between winxp, winlh and winnet drivers?

 

Regards,

Erwan

Hi Olof,
 
About the driver part, on windows 7, it was smooth and easy.
On XP, using the provide tool did not do the trick.

The provided setup tools should install the correct versions. On XP, it might look a little confusing sometimes after setup when you look at Device Manager. But as far as I know, setup should work correctly on XP as well. A reboot might be needed to make everything look correct though. Do you see any error messages when you try to install on XP, or does it actually work anyway when you start the mount tool GUI after setup?

 

But let me check, I will start a Windows XP and try it!

 

Doing it manually did not do the trick. At best does the driver appears in error (device not ready), even after a reboot.
 
Also got confused between winxp, winlh and winnet drivers?

 
Manual installation
 
Win7 = kernel 760x versions. This is Windows 7 and Server 2008 R2, all editions. (Works on Windows 8 as well.)
WinLH = kernel 600x versions. This is "longhorn", that is Vista and Server 2008, all editions.
WinNET = kernel 3790 versions. This is Windows Server 2003 and Server 2003 R2, all editions, as well as 64 bit version of Windows XP Professional.
WinXP = kernel 2600 versions, that is all 32 bit editions of Windows XP.
 
(Don't blame me for this stupid notation. It is something that has been around in the WDK build scripts for quite a while.)
 
Storport editions of the driver
These versions simply install plug-and-play root devices. They can be installed by the Add Legacy Hardware wizard in Device Manager (right-click on root object i device tree), or Install New Hardware wizard in Control Panel. It is also possible to install this kind of driver using the devcon.exe command line tool. devcon install path\phdskmnt.inf ROOT\PhDskMnt or something like that should do.
 
These storport editions require storport.sys, so minimum kernel version is 3790, so called WinNET, that is Server 2003 or 64 bit XP. In case of Server 2003, KB932755 is needed as well.
 
Scsiport edition of the driver
32 bit editions of Windows XP lacks storport.sys, so in order to provide something roughly equivalent on Windows XP, there is a scsiport.sys edition of this driver for XP. It fakes a non-PnP ISA SCSI adapter driver and needs to be installed in a completely different way. Either right-click the .inf file in WinXP directory and select Install, or use the corresponding call with rundll32 setupapi.dll,InstallHinfSection DefaultInstall 132 path\phdskmnt.inf, or alternatively, copy the .sys driver file to system32\drivers and use sc create phdskmnt type= kernel ... etc command to manually create this driver. Once the driver has been installed, it needs to be loaded. Type net start phdskmnt to do that. Device Manager might look strange until next reboot, but the virtual device should be immediately available.

@Olof

A question and a suggestion.

 

Question:

I may be aging (and become forgetful), but if I stated a few days ago that a 2K version existed, it means that in a previous version of the site it did exist.

Or I really saw something that wasn't there?

 

Suggestion:

The idea of naming the new "assembled in a .7z" "DriverSetup.7z" is IMHO an amost foolproof way to create, before or later, some issue.

 

Would it not be more useful to name it something *like* "AIM_Driver_20130930.7z" ?

(and yes this is another question, but meant as a suggestion )

 

Wonko

@Olof
A question and a suggestion.
 
Question:
I may be aging (and become forgetful), but if I stated a few days ago that a 2K version existed, it means that in a previous version of the site it did exist.
Or I really saw something that wasn't there?

You are right. My build scripts accidentaly built a Win2K version as well. That was never anything that was meant to happen, though. Mostly because I don't think it would ever actually work...

 

 

Suggestion:
The idea of naming the new "assembled in a .7z" "DriverSetup.7z" is IMHO an amost foolproof way to create, before or later, some issue.

 

Yes. The only reason that was done was because I knew to little about GitHub to get it working correctly in the other way. GitHub translated CR/LF to CR in the .inf files I uploaded, which meant that when people downloaded the complete repository as a big .zip file, they got checksum mismatches for the digitally signed .inf/.cat combinations when they tried to install the driver...

 

I have now learned that there are ways to teach GitHub to behave in a more sane way and not touching anything with its own unfathomable logic. So, when I get some time I will simply fix that and upload a nice directory tree instead of the 7z file.

Olof Lagerkvist, on 30 Sept 2013 - 1:33 PM, said: I have now learned that there are ways to teach GitHub to behave in a more sane way and not touching anything with its own unfathomable logic. So, when I get some time I will simply fix that and upload a nice directory tree instead of the 7z file.

But the idea of having a single, compact and easy to download .7z is nice IMHO, if named/labeled not "anonymously such as "DriverSetup" or "Install" (or "EAT ME", "DRINK ME", etc. ):
http://en.wikipedia.org/wiki/Alice's_Adventures_in_Wonderland#Synopsis

http://en.wikipedia.org/wiki/Alice's_Adventures_in_Wonderland#Synopsis

 
 
The above links are posted mainly to show how the good (though demented) IPB guys have put up a board parser that fails to build a valid link whenever a not-so-strange char is in the page address.
 
 

http://en.wikipedia.org/wiki/Alice's_Adventures_in_Wonderland#Synopsis

The only way is to use manually the "link" tool:
<a data-ipb="nomediaparse" data-cke-saved-href="http://en.wikipedia.org/wiki/Alice" href="http://en.wikipedia.org/wiki/Alice" s_adventures_in_wonderland#synopsis"="">http://en.wikipedia.org/wiki/Alice's_Adventures_in_Wonderland#Synopsis
and even it is re-parsed incorrectly in case you edit the post, making it something as senseless as:

<a data-ipb="nomediaparse" data-cke-saved-href="http://en.wikipedia.org/wiki/Alice" href="http://en.wikipedia.org/wiki/Alice" s_adventures_in_wonderland#synopsis"="">http://en.wikipedia.org/wiki/Alice's_Adventures_in_Wonderland#Synopsis

Wonko

But the idea of having a single, compact and easy to download  .7z is nice IMHO, if named/labeled not "anonymously such as  "DriverSetup" or "Install" (or "EAT ME", "DRINK ME", etc. ):

http://en.wikipedia....erland#Synopsis

 

 

The problem with that, in a version controlled repository, is that you would not be able to track changes to individual files. But it is of course possible that this is not actually a problem. Changes are probably better tracked in the source tree, not through changes in the installer tree. Also, you would not be able to track changes to the setup archive itself, since there will be a new file without any history with each update.

 

But over the years I have sort of missed a version named installer for ImDisk as well. The problem in that case is the huge number of deep links to .../imdiskinst.exe everywhere, which is referred to as "latest version", regardless of what it really is.

The provided setup tools should install the correct versions. On XP, it might look a little confusing sometimes after setup when you look at Device Manager. But as far as I know, setup should work correctly on XP as well. A reboot might be needed to make everything look correct though. Do you see any error messages when you try to install on XP, or does it actually work anyway when you start the mount tool GUI after setup?

 

But let me check, I will start a Windows XP and try it!

 

 
Manual installation
 
Win7 = kernel 760x versions. This is Windows 7 and Server 2008 R2, all editions. (Works on Windows 8 as well.)
WinLH = kernel 600x versions. This is "longhorn", that is Vista and Server 2008, all editions.
WinNET = kernel 3790 versions. This is Windows Server 2003 and Server 2003 R2, all editions, as well as 64 bit version of Windows XP Professional.
WinXP = kernel 2600 versions, that is all 32 bit editions of Windows XP.
 
(Don't blame me for this stupid notation. It is something that has been around in the WDK build scripts for quite a while.)
 
Storport editions of the driver
These versions simply install plug-and-play root devices. They can be installed by the Add Legacy Hardware wizard in Device Manager (right-click on root object i device tree), or Install New Hardware wizard in Control Panel. It is also possible to install this kind of driver using the devcon.exe command line tool. devcon install path\phdskmnt.inf ROOT\PhDskMnt or something like that should do.
 
These storport editions require storport.sys, so minimum kernel version is 3790, so called WinNET, that is Server 2003 or 64 bit XP. In case of Server 2003, KB932755 is needed as well.
 
Scsiport edition of the driver
32 bit editions of Windows XP lacks storport.sys, so in order to provide something roughly equivalent on Windows XP, there is a scsiport.sys edition of this driver for XP. It fakes a non-PnP ISA SCSI adapter driver and needs to be installed in a completely different way. Either right-click the .inf file in WinXP directory and select Install, or use the corresponding call with rundll32 setupapi.dll,InstallHinfSection DefaultInstall 132 path\phdskmnt.inf, or alternatively, copy the .sys driver file to system32\drivers and use sc create phdskmnt type= kernel ... etc command to manually create this driver. Once the driver has been installed, it needs to be loaded. Type net start phdskmnt to do that. Device Manager might look strange until next reboot, but the virtual device should be immediately available.

 

Tried the rundll32 way and got a "installation failed".
Tried the sc way but i would need more details like group and depend parameters.

 
This should be enough, I think:
sc create phdskmnt type= kernel binpath= system32\drivers\phdskmnt.sys
net start phdskmnt

 

Will keep on investigating : for sure, this is not a given under xp
I am using my work laptop (xp...) : I miss my home windows 7 machine !

At least it used to work on XP, so it should be possible to get it up and running there now as well. Just that XP is surely a lot more complicated with these things.

Just tried the installer on Windows XP and it seemed to work correctly as far as I can see. Got the driver installed and the GUI mount tool did not complain either.

Unfortunately, the driver setup is rather complicated on XP. This is because on XP, it is impossible to send SCSI miniport commands to a driver that has no recognized devices (for example SCSI disks) connected. This means that for a virtual driver like this, we would lose possibility to request it to create one for us, if none already exists. To solve that, the driver always "recognizes" one fake "RAID array control device". This one in turn requires a driver to make Device Manager happy. That is just a so called null driver that does not do anything, but it needs to be installed as an additional step. This is what makes driver setup a much more complicated and slower process on XP compared to other versions.

This is specific to 32 bit XP. On 64 bit XP, all editions of Server 2003 or Vista or later this is not a problem since they have built-in support for virtual miniports through storport.sys instead.