Jump to content











Photo
- - - - -

ImDisk with a twist


  • Please log in to reply
6 replies to this topic

#1 knjers

knjers
  • Members
  • 2 posts
  •  
    Croatia

Posted 27 October 2012 - 06:36 PM

I am definitely in the wrong place. And probably asking a stupid question. So, I apologize. For everything. But anyway, do have to ask. After lurking here for a while I am really not sure this, to me an obvious tool, exists.

Looking for a SINGLE tool / driver that can mount raw .img, .vhd and .vmdk containers in device and volume mode. Preferably several concurrently And in per mounting object selectable RO or RW access, with fixed or removable flag. And if in device mode pass the MBR or GPT to Windoze.

With a modern UI.

This obviously includes being able to:
- mount NTFS sparse raw image as well as truncated raw image without unused blocks
- mount .vdh and .vmdk flat or growable, with optional undo and commit

A very big plus would be if it had some thingamagic to create mentioned containers from scratch with selectable size, type and flat/growable.
A mighty extreme plus if it could convert between some of those containers, but asking for a three floor house without even having enough cement to build a foundation is probably a bit too much. And I do have Ghost for that.

Has anybody used the german "rhymes with Aragorn" toolset lately? Can those do something like this? I remember trying them out several years ago but they required reciting three Hail Magnetos before pressing commit. And even then lady luck gave me better chances on winning the lottery than my data surviving.
Perhaps SSD do not require praying.


Maybe I am asking for too much. Maybe I am too old and senile. Maybe I have to many things on my mind to keep track of different cmd switches between devio and DiscUtilsDevio. But to anybody dealing with more than one machine, this should be on the keychain.

Thanks

#2 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14132 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 27 October 2012 - 07:12 PM

I presume it must run in DOS, Linux, all Windows NT flavours, including and up to 7, in Windows 8 and Windows RT, need not Administrator privileges, be digitally signed and available in both 32 and 64 bit versions :w00t:

When you find the thing you have described, please ask the developer to add to it an automatic making coffee feature (black and strong, two cups of sugar for me, thanks :)).

Confirmed :thumbup: (you are asking too much ;)).

:cheers:
Wonko



#3 Sha0

Sha0

    WinVBlock Dev

  • Developer
  • 1682 posts
  • Location:reboot.pro Forums
  • Interests:Booting
  •  
    Canada

Posted 27 October 2012 - 08:15 PM

A mighty extreme plus if it could convert between some of those containers...

qemu-img can help you to convert images to different formats.

#4 knjers

knjers
  • Members
  • 2 posts
  •  
    Croatia

Posted 27 October 2012 - 08:42 PM

So no such tool. There goes my dream of virtualised storage on the new and improved Windows RockaTansky.

Being more realistic, I obviously have to drop the device request.
Does ImDIsk really provide solid read write access to sparse .vmdk and .vhd volumes up to 100GB? Does it not blow up when doing a large Ghost32 transfer or serious I/O (like Ken Kato's VDK used to)? Does it survive repeated laptop standbys and hibernates on XP? How much GDI objects does it use per volume mounted?
Have more questions but being new here just wouldn't be right.

Obviously I can not use TotalMounter (no .vmdk) or OSFMount (no .vmdk write support).



I thought the suffusion was supposed to be yellow..


qemu-img can help you to convert images to different formats.


Thanks, I have read about that. But that is the last of my worries since I first have to create containers and mount them before stumbling into different virtualization platforms.

#5 Sha0

Sha0

    WinVBlock Dev

  • Developer
  • 1682 posts
  • Location:reboot.pro Forums
  • Interests:Booting
  •  
    Canada

Posted 27 October 2012 - 09:25 PM

Thanks, I have read about that. But that is the last of my worries since I first have to create containers and mount them before stumbling into different virtualization platforms.

For me, it's a matter of time and priorities. Eventual goals for WinVBlock have long been:
  • .VMDK support
  • .VHD support (not the silly "VHD" that so many people say when they usually mean "raw disk image", but the more complicated VHDs)
  • device-mapper-style features, for software RAID or taking disk ranges from multiple sources and exposing them as a whole disk
  • RAM disks loaded natively by NTLdr's /RDPATH=
  • Disk images embedded in the Registry
  • Support for Olof's devio disks
  • Maybe the other disk image formats supported by QEmu
  • Disk images via HTTP. I got this working in an experimental branch, but it is not really ready for release
  • Probably forgetting one at the moment
Maybe some day! Too bad, for now! :loleverybody:

#6 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14132 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 October 2012 - 08:52 AM

Well, for the record, similar to the "plain" VHD (which are a "plain" RAW image with a single sector appended) the "common" (again "plain", static) .VMDK's are RAW images with an external descriptor file, so this kind is compatible with any of the tools mentioned.

As mentioned several times (but finding no actual people listening and capable or willing to do something about it :() IMHO a line should be drawn somewhere.
Low level tools/drivers like Winvblock, Firadisk and Total Mounter are useful and good :worship: for booting and for some kind of "lower" level access, but most probably adding to any of them support for all kind of (mostly senseless) VM disk formats (if you sum the .vhd+.vmdk+.vdi you will get more than ten different formats) is complex.

On the other hand, making "higher level" drivers for them using any of the now available toolkits should be not that much difficult.

As a reminder we have available at least:
Dokan:
http://dokan-dev.net/en/
Eterlogic Virtual Drive SDK:
http://www.eterlogic...alDriveSDK.html
Pismotechnic Pismo File Mount Developer Kit
http://www.pismotechnic.com/

that should be capable of producing a driver/tool capable doing (not as "device" but at a higher level of volume or filesystem) most of the common chores (mount, copy/delete/paste files, etc.) on those "queer" image formats.


:cheers:
Wonko

#7 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1358 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 28 October 2012 - 09:22 PM

Just a couple of comments about what ImDisk or DiscUtils can be used for, among all requirements mentioned in this thread.

Looking for a SINGLE tool / driver that can mount raw .img, .vhd and .vmdk containers in device and volume mode. Preferably several concurrently


ImDisk can be used for mounting individual volumes only. Using proxy mode with DiscUtils library, it can access several disk image file formats.

And in per mounting object selectable RO or RW access,


Possible with ImDisk.

with fixed or removable flag.


Possible with ImDisk.

And if in device mode pass the MBR or GPT to Windoze.


Not possible with ImDisk, because it work with individual volumes only. It can use DiscUtils to read MBR or GPT partition tables to select individual volumes, though.

With a modern UI.


There is no modern UI for either ImDisk or DiscUtils. They both provide extensive API though, so it should be possible for someone to create a modern UI reasonably easy.

This obviously includes being able to:
- mount NTFS sparse raw image as well as truncated raw image without unused blocks
- mount .vdh and .vmdk flat or growable, with optional undo and commit


ImDisk/DiscUtils can do this.

A very big plus would be if it had some thingamagic to create mentioned containers from scratch with selectable size, type and flat/growable.


DiscUtils includes a tool called VHDCreate that can be used to create VHD files.

If you have Virtual Server or Hyper-V installed, you also have scriptable components installed that can create fixed, dynamic and differential VHD files. On Windows 8 or Server 2012, these components can also create files in the VHDX format used in Hyper-V on Server 2012 and Windows 8.

A mighty extreme plus if it could convert between some of those containers, but asking for a three floor house without even having enough cement to build a foundation is probably a bit too much. And I do have Ghost for that.


DiscUtils includes tools called VirtualDiskConvert, DiskClone and OSClone that can be used to convert between various image file formats, for example VHD, VMDK, VDI and XVA. They can also convert from or to physical disks, including iSCSI targets.

If you have Virtual Server or Hyper-V installed, you also have scriptable components installed that can convert between fixed, dynamic and differential VHD formats. On Windows 8 or Server 2012, these components can also convert between VHD and VHDX formats (used in Hyper-V on Server 2012 and Windows 8).

Maybe I am asking for too much. Maybe I am too old and senile. Maybe I have to many things on my mind to keep track of different cmd switches between devio and DiscUtilsDevio. But to anybody dealing with more than one machine, this should be on the keychain.


In general, what you are asking for is a huge swiss-army-nife like virtual disk tool that simply does not exist. Pretty much all of your requested features can be done through various open source libraries, drivers, applications, tools etc, so I would say that you could rather ask someone to build a nice-and-easy modern UI on top of everything. I do things like that a lot in commercial projects, but none of them have been nearly as big as the one you are asking for...

Does ImDIsk really provide solid read write access to sparse .vmdk and .vhd volumes up to 100GB?


I use that quite regularly myself so, yes. Note that this is entirely implemented using DiscUtils library, so as long as DiscUtils library can access your image file (read/write individual partitions), you can use it with ImDisk. http://discutils.codeplex.com

Does it not blow up when doing a large Ghost32 transfer or serious I/O (like Ken Kato's VDK used to)?


No. It seems to be safe nowadays. It had a problem initially with something like this, that particularly showed up on Vista and above. But I corrected a few buffer checks that checked against the wrong sizes so those problem seem to be gone now.

Does it survive repeated laptop standbys and hibernates on XP?


Yes. Also quite frequently used by myself. Note that ImDisk driver is not a plug-and-play driver at all, so it does not even get notified about power changes etc.

How much GDI objects does it use per volume mounted?


None. Why would a disk driver utilize GDI objects?

Or do I misunderstand something? Do you mean resource usage overall? Like kernel pool usages, virtual memory usage, handles and threads etc? I have not measured that lately, but could do that if there are many users interested. Generally, the both ImDisk and DiscUtils sources have been thoroughly examined for possible resource leaks, possible missing buffer checks etc by many different people over many years. All that has came up have been carefully corrected. The result is a driver and library that have been used by me and many other people for everything from everyday ram disk usage to remote mounting physical disks or image files and transfer of hundreds of GB of data, for example in projects like migrating physical servers to virtual and similar. It does not guarantee that everything is 100% safe and accurate, but it should give a hint about what level it stands at for the moment.

Have more questions but being new here just wouldn't be right.


Well, just ask! :)

To anyone in thoughts of writing a new virtual storage driver with various features, with full storage device stack integration to get interaction with Disk Management, Volume Shadow Copy Services, Windows version independent shell integration etc, I would strongly recommend Microsoft's StorPort virtual miniport sample. It is a good starting point for writing virtual miniports for the storage device stack to support Vista/Server 2003 and later Windows versions. Because it is implemented as a miniport, you automatically get all features and characteristics that physical SCSI devices have. The fact that it is a miniport also means that you would not need much experience in writing device drivers to get a working driver.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users