46 replies to this topic

[was_jaclaz]

I received a mail by Olof Lagerkvist, the author of the newish IMDISK driver which, by explicit permission, reproduce here in parts:

I would just like to inform you about that the project has been updated recently. Some new functions have been added such as extending the size of a mounted virtual disk, virtual removable media disks and graphical user interfaces such as a Control Panel applet and context menus in Windows Explorer. Simple API functions are now exported so that a developer with the help of a .h and a .lib file can mount/dismount virtual disks directly from within applications. Should also be fairly easy to build .cmd scripts with the command line interface.

The I/O redirect functionality has undergone a major design change and it is now much more CPU-effective. It is possible to redirect I/O requests for the virtual disk over TCP/IP or Named Pipe so that the actual storage for the virtual disk can be on another computer or handled by another application.

The official page of the project is this one:
http://www.ltr-data.se/opencode.html
please update your old bookmarks, if any.

There are also lots of other ideas and half-way implemented and undocumented functionality, for example it is possible through registry settings to auto-start the driver when Windows starts up and let it automatically create a RAM-disk and pre-load it with the contents of an image file.

Besides the above,really interesting features, I want to underline the following:

I could create some sort of wish-list or something and possibly include some descriptions of these so long undocumented features, but I am searching for a forum where such a project can reach interested and possibly participating people.

Of course I offered Boot-land as a possible such forum board, inviting Olof to join and eventually give him some "dedicated" space, like we already have for tftp32 and Winimize.

I hope that he will accept my offer and soon join us, however all members interested in the above software is invited to test the new release and submit here questions, suggestions and ideas.

I am going to "sticky" this for a few days, giving it some better visibility......

UPDATE:
Olof joined us and project has it's own new place on the board.

For those wondering WHAT Filedisk and VDK are, reference to them and some more similar apps is here:
http://www.boot-land...vers-t1507.html

jaclaz

[Brito]

These are good news - I also like Olof's developments.

If needed, where would you recommend a new sub-forum and wich name to use?

Of course that we already have a Program support and discussion forum where Olof is very welcome to post a new topic refering to each new version/published program.



I've never tested imdisk, but this sounds a promissing feature:

The I/O redirect functionality has undergone a major design change and it is now much more CPU-effective. It is possible to redirect I/O requests for the virtual disk over TCP/IP or Named Pipe so that the actual storage for the virtual disk can be on another computer or handled by another application.

[Olof Lagerkvist]

Hi everyone,

I have read some of the discussions on this forum and thought that discussion about some of my projects, especially ImDisk, would fit quite well here.

There have been lots of e-mail with ideas about functions to add, such as support for mounting VMWare/Virtual PC virtual disk files and IMZ compressed images, encryption, easier remote mounting over TCP/IP, support for virtual audio CD:s, "virtual CD-burning", full read/write support for virtual DVD+RW media and lots of more. It would be next to impossible for me to add all functions to it myself so I would like to get a feeling about which functions are of highest interest.

As it is right now, lots of work has been done to make the driver easy to extend with other modules, both for custom storage support and for mounting/unmounting. This might make it possible for people not very used to writing driver code be able to extend the driver.

It is also no problem to create "commercialized spin-offs", with partly or fully protected code. There are some of that kind already with for example very special storage for the virtual disks. So, it is no problem to add closed-source modules either.

[Brito]

Hi Olof - welcome to our community!

Thanks for releasing your applications as freeware - It has been almost a year since Jaclaz first recommended your site: http://www.boot-land...p?showtopic=181

Your work is really interesting and we sure appreciate tools like ImDisk wich are very useful for our projects.

A new sub-forum can be added under the Programs category where you will be able to organize discussions and others give feedback about your tools (including myself..) - what do you think?

[MedEvil]

@jaclaz
I know what imDisk does. but could you give me an eyample, where it would be useful for PE purposes?

[was_jaclaz]

@jaclaz
I know what imDisk does. but could you give me an eyample, where it would be useful for PE purposes?

It's already here:
http://www.boot-land...?showtopic=1655
http://www.boot-land...?...=1580&st=40
http://www.boot-land...?showtopic=1441

Medevil, Medevil, bad boy, you should pay more attention....



jaclaz

[Oleg_II]

Hi jaclaz,
Still waiting for a free solution to boot my W2k into RAM Could you ask the author if this driver can be used for this?

Just saw Olof is also here

[Alexei]

Hi everyone,

I have read some of the discussions on this forum and thought that discussion about some of my projects, especially ImDisk, would fit quite well here.
...

Hi Olof,
1) I'm sorry, it may be already discussed, but is it possible to mount virtual disk at very early stages of boot process?
I want it to be ready when NT switches from accessing HDD via BIOS Int 13 to using HDD drivers.
2) Also, there is interesting product http://www.programur...t-connector.htm though it's not freeware
I'd like to have same thing for HDD, i.e. "physical drive over ethernet". Is it hard to implement?
Thanks in advance,
Alexei

[was_jaclaz]

OK, I did some testing of IMDISK, very preliminary, but enough to get an idea of the basic workings.

Since Olof asked for it, the poor guy most probably didn't fully envision the consequences of that action , here are my suggestions:

1) FEATURE REQUESTS (IMPORTANT):
a. possibility to specify an offset for the bootsector counted in either bytes or sectors (better) from the start of the image file to the location of the bootsector.
this will allow, besides providing a workaround for point #2)a below, a way to mount a number of non-RAW images, most notably SDI, VMDK, Virtualbox and QEMU ones (only the fixed-size type), see this for a quick reference:
http://www.911cd.net...o...19155&st=22
b. BE VERY CAREFUL, IMDISK will not recognize an image with a MBR, and thus once it is mounted, if you open the drive in Explorer it will prompt for FORMATTING it.
c. the IMDISK driver, UNLIKE filedisk and VDK, appears not create a \\.\PHYSICALDRIVEx link, thus this prevents direct access to it via a DISK EDITOR if not through the mounted letter.

2) FEATURE REQUESTS (that would be nice)
a. possibility, like VDK already does, to recognize and mount full hard-disk images, i.e. those with a MBR and hidden sectors, or, at least, have a dialog like WINIMAGE has to choose which partition to connect to
b. remove, switched by the .iso extension?, the right click option to "Mount as writeable IMDISK Virtual Disk" for .iso images
c. autodetect, from size of file?, 1.44 and 2.88 Floppy images and mount them starting with B: letter

3) REQUEST FOR DOCS
a. it seems I cannot find ANY docs/examples/howtos for the non-basic functions.

4) Bugs or however reports of something strange
a. Right click menu, it seems like IMDISK thinks that EVERY drive is an IMDISK drive, and of course it cannot unmount a non-IMDISK one
b. file association appear to be a bit messy or however non-existant, i.e. you can tell IMDISK to mount ANY file with right click (and if the file is not recognized a prompt for formatting the drive once opened in Explorer will pop up, see above), additionally a floppy image named "something.bin" was mounted by IMDISK as a CDROM drive

I'll have to find the time to do some more testing, this thingy appears to work itherwise flawlessly

A bit, but not much off-topic:

I'd like to have same thing for HDD, i.e. "physical drive over ethernet". Is it hard to implement?

Alexei, just a crazy idea, which I just had and didn't have the time to even try, is the following:
1) httpdisk by Bo Branten:
http://www.acc.umu.se/~bosse/
2) hfs tiny webserver:
http://www.rejetto.com/hfs/

I am testing just HFS right now and it appears like a real winner for a small project I am having a look at for a friend of mine, if the two work together, you could use http as the internal protocol, since HFS can allow only some IP addresses, it should be easy to "limit" it to intranet.

jaclaz

[Brito]

A new sub forum can be found here:
http://www.boot-land...ImDisk-f59.html

[Oleg_II]

Just a side note

Starting from PowerQuest DriveImage v7 this imaging solution has the ability to mount image files as virtual hard drives. It is possible with Ghost v9, 10, 11 and 12 and some more "professional" appz like Symantec LiveState Recovery and Backup Exec System Recovery. And the most interesting is: Ghost v12 and Symantec Backup Exec System Recovery v7 contain image file explorer that not only can browse what is inside an image file and mount it as a virutal hard disk, but also capable to convert this image file into VMDK file.
And what is the image file? It could be an image of a system partition (MBR can be included too if using switches during imaging process).

Maybe somebody can find it usefull

[windrv]

Just a side note

Starting from PowerQuest DriveImage v7 this imaging solution has the ability to mount image files as virtual hard drives. It is possible with Ghost v9, 10, 11 and 12 and some more "professional" appz like Symantec LiveState Recovery and Backup Exec System Recovery. And the most interesting is: Ghost v12 and Symantec Backup Exec System Recovery v7 contain image file explorer that not only can browse what is inside an image file and mount it as a virutal hard disk, but also capable to convert this image file into VMDK file.
And what is the image file? It could be an image of a system partition (MBR can be included too if using switches during imaging process).

Maybe somebody can find it usefull

Is it freeware or an advertisement?

Any free stuff replacement?

[Oleg_II]

It's not freeware but I don't suggest you to use it There are a lot of people who use Ghost and may not know about this possibility (I've been using PQ DeployCenter for 6 years by now but found this possibility only a mounth ago - DeployCenter images could be mounted as hard drives into my system by this utility since DriveImage v7).

And for some talented folks it may be a good idea to look into it and maybe create a free alternative too

[windrv]

It's not freeware but I don't suggest you to use it There are a lot of people who use Ghost and may not know about this possibility (I've been using PQ DeployCenter for 6 years by now but found this possibility only a mounth ago - DeployCenter images could be mounted as hard drives into my system by this utility since DriveImage v7).

And for some talented folks it may be a good idea to look into it and maybe create a free alternative too

Yes. I agree that talented folks would spend their time on creating free stuff rather than spending their invalueable time hunting for free stuff.

[Oleg_II]

Yes, you are right and Olof Lagerkvist is one of those who creates free stuff for those who's jobs and skills are not in computing but they are still computer geeks

[windrv]

Yes, you are right and Olof Lagerkvist is one of those who creates free stuff for those who's jobs and skills are not in computing but they are still computer giks

You are also a talent and your time is also very invalueable!

[was_jaclaz]

Topic moved to it's new "home".

jaclaz

[Olof Lagerkvist]

OK, I did some testing of IMDISK, very preliminary, but enough to get an idea of the basic workings.

Since Olof asked for it, the poor guy most probably didn't fully envision the consequences of that action , here are my suggestions:

Well well, no problem, at least so far

1) FEATURE REQUESTS (IMPORTANT):
a. possibility to specify an offset for the bootsector counted in either bytes or sectors (better) from the start of the image file to the location of the bootsector.
this will allow, besides providing a workaround for point #2)a below, a way to mount a number of non-RAW images, most notably SDI, VMDK, Virtualbox and QEMU ones (only the fixed-size type), see this for a quick reference:
http://www.911cd.net...o...19155&st=22

Yes, this is an interesting feature request and actually something I am missing myself every now and then so offset will be added in a near future.

There is a workaround you can use in the meantime though. Download devio.exe (the server part of ImDisk network redirected disks) from http://www.ltr-data.se/files/devio.exe

This is a command line tool. Basic syntax:

devio tcp-port filename size offset

Size and offset are in 512-byte blocks, not bytes. They can be suffixed by K, M, G for kilobytes, megabytes or gigabytes.

devio 20000 diskimage.img 20M 64

That is, start serving I/O for a 20 MB file called diskimage.img at offset 64 blocks (= 32768 bytes) on tcp port 20000.

Then use the imdisk.exe command line tool to actually mount it:

imdisk -a -t proxy -o ip -f localhost:20000 -m Q:

b. BE VERY CAREFUL, IMDISK will not recognize an image with a MBR, and thus once it is mounted, if you open the drive in Explorer it will prompt for FORMATTING it.

Yes, actually ImDisk does not recognize anything from the contents of the image file. The main reason is that it should be possible to use any filesystem on it, so if you are testing a new filesystem driver or something it should be possible to use it on an ImDisk virtual drive without ImDisk itself trying to read and recognize anything.

c. the IMDISK driver, UNLIKE filedisk and VDK, appears not create a \\.\PHYSICALDRIVEx link, thus this prevents direct access to it via a DISK EDITOR if not through the mounted letter.

Actually, PHYSICALDRIVEx links should only be created by drivers of partitionable disks and then linking to the "Partition0", that is direct access to the physical disk. For a driver like ImDisk that handles virtual partitions rather than virtual hard disks there is no reason to create a PHYSICALDRIVEx link as it is possible to access the drive directly though the drive letter, for example \\.\E:. If a virtual drive is created without a drive letter then a drive letter can be added to an existing ImDisk device, for example using the dosdev.exe tool http://www.ltr-data....iles/dosdev.zip and the following syntax:

dosdev -r Q: \Device\ImDisk0

This adds the drive letter Q: to ImDisk number 0. Type imdisk -l to list existing ImDisk virtual drives. Type dosdev -d Q: to remove it again. For developers: The library exports a function called ImDiskOpenDeviceByNumber() that can be used to open and communicate with a virtual drive even if it has no drive letter or if the drive letter is unknown.

2) FEATURE REQUESTS (that would be nice)
a. possibility, like VDK already does, to recognize and mount full hard-disk images, i.e. those with a MBR and hidden sectors, or, at least, have a dialog like WINIMAGE has to choose which partition to connect to

I will probably add something like that when I add offset support to the mounting process.

b. remove, switched by the .iso extension?, the right click option to "Mount as writeable IMDISK Virtual Disk" for .iso images

Would be nice. As it is right now this is not really possible to change... The menu items are only defined by two registry keys, HKCR\*\shell\ImDiskMountFile and HKCR\*\shell\ImDiskMountFileWriteable so it is not possible to make any differences for special filetypes. I am thinking about developing a real shell extension dll instead which would at least make this possible to solve.

c. autodetect, from size of file?, 1.44 and 2.88 Floppy images and mount them starting with B: letter

Auto-detection from file size only work if a device type is not manually specified and the image file name does not end with .iso or .bin, in which case a CD/DVD image is assumed. All common floppy sizes are otherwise auto-detected, however they do not use separate drive letters (and it would require lots of edit to change that since the drive letter is selected long before the auto-detection based on file size).

3) REQUEST FOR DOCS
a. it seems I cannot find ANY docs/examples/howtos for the non-basic functions.

That is mostly correct. The only current documentation is the output when running imdisk.exe or drvio.exe from the command line without parameters. I think the first steps towards documentation is to build a how-to archive or something. That could very well start with some of the q+a from this forum.

4) Bugs or however reports of something strange
a. Right click menu, it seems like IMDISK thinks that EVERY drive is an IMDISK drive, and of course it cannot unmount a non-IMDISK one

This is also based on the problem that I do not use shell extensions but rather just define a context menu item in the registry. That way I had to add the registry key as HKCR\Drive\shell\ImDiskUnmount which adds the menu item to all drives.

b. file association appear to be a bit messy or however non-existant, i.e. you can tell IMDISK to mount ANY file with right click (and if the file is not recognized a prompt for formatting the drive once opened in Explorer will pop up, see above), additionally a floppy image named "something.bin" was mounted by IMDISK as a CDROM drive

Yes, any file can be mounted as a virtual disk. This is because disk images can have almost any extension these days, if any extension at all. The only special treatment is that .bin and .iso extensions automatically create a virtual CD/DVD-ROM drive. Maybe I can change this autodetection so that it auto-detects floppy images based on size before auto-detecting CD/DVD-ROM images based on extension. That would at least solve your problem in this case.

[Brito]

Hi Olof!

About documentation: You're also welcome to use boot land's wiki as a repository for syntax and informations - other users (including myself) will be able to write their experience and complete the documentation as needed (or translate to other languages)

To keep away spammers, only forum members with more than 10 replies are allowed to edit or add pages on the wiki.

http://www.boot-land.net/wiki


btw: This sub-forum will be moved to main page..

[Olof Lagerkvist]

Hi Olof,
1) I'm sorry, it may be already discussed, but is it possible to mount virtual disk at very early stages of boot process?
I want it to be ready when NT switches from accessing HDD via BIOS Int 13 to using HDD drivers.

Yes this is possible. The driver can be setup to auto-load at system start and all settings for auto-creating virtual disks can be set in the registry. There are no documentation for this so far. I will try to write something down soon to describe how this works.

2) Also, there is interesting product http://www.programur...t-connector.htm though it's not freeware
I'd like to have same thing for HDD, i.e. "physical drive over ethernet". Is it hard to implement?

This works as it is right now, and it does not need to be Windows in the server end, it could be a computer started on a Live-CD with FreeBSD. One problem though is that such drives cannot be mounted in the early stages of the OS boot. The problem is that for redirecting I/O over network the driver needs to communicate with a user-mode helper service and that one cannot be started early in the boot process.

[was_jaclaz]

VERY good!

What I had in mind were just a few small checks:
1) check for 55AA as last two bytes of first sector (that should mean that the drive image ALREADY has either a MBR or bootsector
2) check the first, say, three or four bytes of first sector and compare them to a "database" actually a simple .ini type of file will be enough (easily expandable/updatable) containing the start code of "known" MBR's and bootsectors and those for Qemu, vdk and similar vurtual disk images
3) consequently, display a message, something like:
a. In case a MBR is recognized, a further check for partition entries and:

This image appears to have a xxx OS MBR, with following partition entries:
1) Type 06 - BIGDOS Start at 0-1-1 Size 2040192 sectors
2) EPBR
E1) Type 0b - FAT 32 Start at 127-1-1 Size 8401932 sectors
3) - No entry
4) - No entry
Please choose which partition to mount......

b. In case a bootrecord is identified display nothing and mount the image
c. In case some kind of VM image is recognized display something like:

This appears to be a VM disktype image.
Are you sure you want to mount the image starting from first sector?
(Y/N) No will mount the image starting from offset xxx.

and the program would loop to 3a. with the new offset
d. In case code is not identified, display something like:

The image appears to contain uknown code in the first sector.
Are you sure you want to mount it starting from first sector?
Press ENTER for yes or input the number of sectors to skip

4) of course a couple added parameters for command line is necessary, something like

-f or -force

to skip the check and a

-o numsectors or -offset numsectors

to force skipping to a given value

As always, I have no idea on the amount of work is needed to realize something like the above, and as said the thing I am really missing is just the offset possibility.

Maybe one could first thing implement the offset thingy, and only later, when time permits, expand it to the full detection routine, for the benefit of users with less knowledge of disk image internals.

Also, I guess it would be possible to do it as an "external" program, being it a "real" .exe or a batch file, if you think this is the correct way, I can volunteer for the batch.

Thanks a lot for the "devio workaround" I can't wait to find the time to test it!



jaclaz

[Alexei]

Yes this is possible. The driver can be setup to auto-load at system start and all settings for auto-creating virtual disks can be set in the registry. There are no documentation for this so far. I will try to write something down soon to describe how this works.
This works as it is right now, and it does not need to be Windows in the server end, it could be a computer started on a Live-CD with FreeBSD. One problem though is that such drives cannot be mounted in the early stages of the OS boot. The problem is that for redirecting I/O over network the driver needs to communicate with a user-mode helper service and that one cannot be started early in the boot process.

Documentation: I found that not much explanations are needed if you provide a lot of examples, starting with trivial sample and adding features one by one. It also helps in testing, as it's easy to play with a sample writing down what's happening
Also, you have a lot of useful comments - just mark them somehow, then automatically extract to the docs.

I think, it will be possible to boot from the network (PXE server). Eventually, I'll take care of that
I understand why it's not possible, but I think I know how to do it

I propose a feature (for better compatibility):
For each image create not only \\.\PHYSICALDRIVEx (drive), but also \\.\PHYSICALDRIVEn (volume),
where x-letter; n-number. The drive should have "fake MBR" with only one partition, which is the volume
All sectors from fake MBR till Boot Record are also supposed to be virtual (all 00h by default). MBR code, NT-signature, and (optionally) sectors are supposed to be user-definable.

Another idea is to add more control over data buffering:
- optional asynchronous load of a full image (in background)
- optional memory buffer with user-definable buffer size, policy (MRU?),
and percent of buffer which is non-pageable
- "fast mode" - return to caller right after writing to the buffer (temporary/dicardable data).
- "EWF mode" - changes are not reflected in source image
submodes: changes in memory, changes on another drive (local or virtual)

Other features:
- all command line options available from registry (just as one string?)
- Complete uninstall, as if it never been installed.
- user-definable policy on priorities of execution threads (useful?)

I took a quick look at your code. I can read "C", but not write
That's what I noticed (I may be wrong, though),
- IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS - not supported
- IOCTL_DISK_GET_DRIVE_GEOMETRY (etc.) does not return Media Type and Total Cyl
BTW, general expression: very good professional work

Regarding server-side of the communication proxy: I'd be interested in complete specification (I'm thinking about implementing it on windows). Even better it would be to have an API to a user-written DLL that provides virtual I/O (server side).
I mean something like this:
virtual disk<-->proxy client<-->transport<-->proxy server<--(API)-->UserDLL<-->whatever


Alexei

PS
@jaclaz,
In general, I dislike "automated logic" and numerous defaults. They may save 5min. of user time or make him for a day An option to disable all automation should be a nice compromize


Alexei

[Olof Lagerkvist]

(This post has been modified a bit since I first posted it.)
(28 July 2008 - further update, added "DriveLetterN" to the list of registry settings.)

Documentation: I found that not much explanations are needed if you provide a lot of examples, starting with trivial sample and adding features one by one. It also helps in testing, as it's easy to play with a sample writing down what's happening

All driver settings are in a very "techincal" format in the registry, basically based on the data structure the user mode applications send to the driver when a new virtual disk is created. You can create a registry key HKLM\SYSTEM\CurrentControlSet\Services\ImDisk\Parameters. Under that key the following values can be created.

MaxDevices
Optional value, default value is currently 16. If defined this should be a 32-bit (REG_DWORD) value and it can be used to specify the maximum number of virtual devices that can be created. This can be at most 32 because the driver keeps track of used device numbers in a 32-bit bit-field.

LoadDevices
This should be a 32-bit (REG_DWORD) value and it can be used to automatically create virtual disks when the driver is loaded. This means that if the driver is setup to load at system boot, virtual disks will be created at system boot. The value specifies the number of virtual disks to create in this way.

FileNameN
These values (string values of type REG_SZ) specifies which image file should be used for the virtual disks specified with the LoadDevices value. N should be replaced with 0 for the first virtual disk, 1 for the second and so on. Just like otherwise the filename is optional for RAM-disks and if a filename is specified for a RAM-disk this just specifies an image file to pre-load into the RAM-disk when it is created.

SizeN
These values should be 64-bit binary values and specifies the size for virtual disks. For file backed disks, the file size is adjusted to this size. N should be replaced with 0 for the first virtual disk, 1 for the second and so on. These parameters are optional for file backed virtual disks. Note that the Size values are specified in reverse byte order, the first byte represent the least significant byte of the 64-bit size.

FlagsN
These values specifies different options for creating the virtual disks. It should be a 32-bit (REG_DWORD) value. The flags are any reasonable combination of the flags specified in the source package in inc\imdisk.h (scroll down to "Bit constants for the Flags field in IMDISK_CREATE_DATA"). Note that most flags can be auto-selected so you will not always need to create the Flags value. For example, if you specify a size and no filename a r/w RAM-disk is created, if you specify a filename then a file backed virtual disk is created and so on.

DriveLetterN
These values should be string values where the first character is used to create a drive letter for the virtual disk. If no DriveLetter setting is present for a virtual disk, no drive letter is created for it.

As it is right now there is no error reporting whatsoever if any of the disks specified by LoadDevices cannot be created, neither to attached kernel debuggers nor to the system event log. I will add some error reporting in the future.

Also, you have a lot of useful comments - just mark them somehow, then automatically extract to the docs.

Yes, I have been thinking about generating docs from comments with Doxygen but it did not understand Windows driver code very well so... Do anyone know anything better?

I think, it will be possible to boot from the network (PXE server). Eventually, I'll take care of that
I understand why it's not possible, but I think I know how to do it

Good. I like such comments.

I propose a feature (for better compatibility):
For each image create not only \\.\PHYSICALDRIVEx (drive), but also \\.\PHYSICALDRIVEn (volume),
where x-letter; n-number. The drive should have "fake MBR" with only one partition, which is the volume
All sectors from fake MBR till Boot Record are also supposed to be virtual (all 00h by default). MBR code, NT-signature, and (optionally) sectors are supposed to be user-definable.

I can see that this could be useful but it requires lots of modifications both to the driver logic itself and to the way it registers the new virtual disks on the system so this will probably not be implemented in a near future.

Another idea is to add more control over data buffering:
- optional asynchronous load of a full image (in background)

This should not be very difficult to implement. The image file is read into the memory by the newly created worker thread, but currently the calling thread waits for this to finish. I suppose there would be no big issues with just modifying this a little bit so that the image read-in is delayed to after the point the caller is waiting for.

Other features:
- optional memory buffer with user-definable buffer size, policy (MRU?),
and percent of buffer which is non-pageable
- "fast mode" - return to caller right after writing to the buffer (temporary/dicardable data).
- "EWF mode" - changes are not reflected in source image
submodes: changes in memory, changes on another drive (local or virtual)

Interesting ideas. I have thought about some kind of write-copy mode that stores changed parts of the image in a memory buffer or in some kind of "undo file". The reason that is a heavy task to implement is that it needs some kind of cluster-indexing thingy in the background, some place where the driver can look in a table and see wether to read requested data from the original image file or from another place. But it is definitely something that I will try to implement in the future.

Other features:
- all command line options available from registry (just as one string?)
- Complete uninstall, as if it never been installed.
- user-definable policy on priorities of execution threads (useful?)

All command line options are available, not as one string, but as described above.

The current uninstall routines should leave no traces. Is there anything in particular you think of here?

User-defined worker thread priority would probably be useful, yes, and hopefully not very difficult to implement as each virtual disk creates a worker thread in the System process and it can set which priority it likes when it is started.

Other features:
I took a quick look at your code. I can read "C", but not write
That's what I noticed (I may be wrong, though),
- IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS - not supported
- IOCTL_DISK_GET_DRIVE_GEOMETRY (etc.) does not return Media Type and Total Cyl

IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS should only be implemented by disk drivers for volumes on hard disk drives and there are no hard disk drives associated with the ImDisk "partitions".

Which media type is returned depends on the drive type. It is for example 12 for virtual hard disk partitions and 11 for virtual CD/DVD-ROM drives. Total number of cylinders should be returned but possibly rounded down to nearest integer.

You can download my 'devioctl' tool: <a href="http://www.ltr-data....s/devioctl.zip" target="_blank"><a href="http://www.ltr-data....s/devioctl.zip" target="_blank">http://www.ltr-data.se/files/devioctl.zip</a></a>
Type for example

devioctl geometry C:

to see what disk.sys returns for drive C: and then

devioctl geometry E:

to see what imdisk.sys returns for an ImDisk drive E:.

BTW, general expression: very good professional work

Thanks a lot!

Regarding server-side of the communication proxy: I'd be interested in complete specification (I'm thinking about implementing it on windows). Even better it would be to have an API to a user-written DLL that provides virtual I/O (server side).
I mean something like this:
virtual disk<-->proxy client<-->transport<-->proxy server<--(API)-->UserDLL<-->whatever

The structures of the headers of the I/O packets sent between client and server are defined in inc\imdproxy.h and there is a sample implementation of it for TCP/IP redirection in the devio sub-directory. Most of the interesting stuff goes in devio/devio.c. I keep a compiled exe at <a href="http://www.ltr-data....iles/devio.exe" target="_blank"><a href="http://www.ltr-data....iles/devio.exe" target="_blank">http://www.ltr-data.se/files/devio.exe</a></a> too. The source for devio is pretty much POSIX-ish as it started off as a project for a FreeBSD Live CD but it should be fairly easy to start with that and extend it so that instead of opening a file or device it can call functions in a dll.

[Alexei]

@Olof,
Thanks for reg descriptions and everything

Alexei @ May 11 2007, 05:01 PM
Also, you have a lot of useful comments - just mark them somehow, then automatically extract to the docs.

Yes, I have been thinking about generating docs from comments with Doxygen but it did not understand Windows driver code very well so... Do anyone know anything better?

I'd say, Doxigen is "too much". Your source is not a part of a big project. Writing your own tool to extract what you mark may be faster then looking for something suitable on the web You can use any markers inside comments, such as <<<......>>> and just
copy extracted text to a text file. Next, it can be a batch that assembles everything, adds html header/trailer, etc

Alexei @ May 11 2007, 05:01 PM
I propose a feature (for better compatibility):
For each image create not only \\.\PHYSICALDRIVEx (drive), but also \\.\PHYSICALDRIVEn (volume)...

I can see that this could be useful but it requires lots of modifications both to the driver logic itself and to the way it registers the new virtual disks on the system so this will probably not be implemented in a near future.

I'm afraid, lack of the drive may create problems during startup. We'll see. Anyway, it's not required to be the same driver.
It can be additional driver that provides the drive. Each new leyer of mods adds complexity that provokes bugs...

Interesting ideas. I have thought about some kind of write-copy mode that stores changed parts of the image in a memory buffer or in some kind of "undo file". The reason that is a heavy task to implement is that it needs some kind of cluster-indexing thingy in the background, some place where the driver can look in a table and see wether to read requested data from the original image file or from another place. But it is definitely something that I will try to implement in the future.

I would use a bit mask for that (bit per cluster), probably in non-pageable.

The current uninstall routines should leave no traces. Is there anything in particular you think of here?

I'll check it up and provide the list, if any

IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS should only be implemented by disk drivers for volumes on hard disk drives and there are no hard disk drives associated with the ImDisk "partitions".

That's what Startup may dislike. We'll see

The structures of the headers of the I/O packets sent between client and server are defined in inc\imdproxy.h and there is a sample implementation of it for TCP/IP redirection in the devio sub-directory. Most of the interesting stuff goes in devio/devio.c. I keep a compiled exe at http://www.ltr-data.se/files/devio.exe too. The source for devio is pretty much POSIX-ish as it started off as a project for a FreeBSD Live CD but it should be fairly easy to start with that and extend it so that instead of opening a file or device it can call functions in a dll.

Yes, protocol is pretty simple. I'll play with it
Meanwhile, could you please add COM port to the list of supported transports?
I have personal interest in this function, as I began virtualizing my environment. COM may be faster then TCP/IP for host/guest communications. I'd like to use IMDISK to let Guest to access Hosts's HDD partitions "directly".


Alexei

[Olof Lagerkvist]

I'd say, Doxigen is "too much". Your source is not a part of a big project. Writing your own tool to extract what you mark may be faster then looking for something suitable on the web You can use any markers inside comments, such as <<<......>>> and just
copy extracted text to a text file. Next, it can be a batch that assembles everything, adds html header/trailer, etc

Sound like a good way to go. Think I will do something like that.

I'm afraid, lack of the drive may create problems during startup. We'll see. Anyway, it's not required to be the same driver.
It can be additional driver that provides the drive. Each new leyer of mods adds complexity that provokes bugs...

Yes, there is a risk that this will create problems during startup. But I think that, along with some other issues like incompatibility with the GUI defrag tool in Windows etc come from the problem that ImDisk does not implement interaction with the Volume Mount Manager, it acts more exactly like a disk driver did in Windows NT 4.0 and earlier.

To implement Volume Mount Manager support it would have to call the Mount Manager to notify it about new disks, create GUID values for each virtual volume, answer Mount Manager's callback ioctl codes (IOCTL_MOUNTDEV_x) etc and that makes the driver a lot more complex. Ken Kato's Virtual Disk Driver and Virtual Floppy Driver implement some support for these codes but not really exactly the way they should be implemented as the DDK documents say... I think I would like to do this in a correct manner if doing it at all. In most cases it is fully ok to not implement Volume Mount Manager support at all, or else implement everything that the Volume Mount Manager documents state as mandatory to support for a storage driver.

But as you say this could very well be implemented in a separate driver and that would keep the original ImDisk pretty much as it is right now for other purposes than where strict Volume Mount Manager support is really needed.

I would use a bit mask for that (bit per cluster), probably in non-pageable.

Yes, something like that. The kernel has very easy-to-use support functions for large bit-fields because that is very common for filesystem drivers to use. But it still requires a lot modification and I think I will implement things suchs as image file offsets and delayed read-in from image file to RAM-disks first.

Yes, protocol is pretty simple. I'll play with it
Meanwhile, could you please add COM port to the list of supported transports?
I have personal interest in this function, as I began virtualizing my environment. COM may be faster then TCP/IP for host/guest communications. I'd like to use IMDISK to let Guest to access Hosts's HDD partitions "directly".

COM ports are already supported... (At least, maybe...)

Actually this has hardly been tested at all but some kind of support for it is implemented. Example:

imdisk -a -t proxy -o comm -f "COM1: BAUD=256000 PARITY=N DATA=8 STOP=1" -m F:

This should connect to a server end of an ImDisk proxy through COM1. It initializes using BuildCommDCBAndTimeouts() so the syntax to the -f switch is like the mode com command. I have no sample server implementation for COM ports but the communication protocol is the same as over TCP/IP. Again, this is mostly not tested at all so anything could happen (almost)