Jump to content











Photo
- - - - -

VMDK Proxy for ImDisk

vmdk proxy imdisk

  • Please log in to reply
60 replies to this topic

#1 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 21 July 2014 - 04:53 PM

Hi,

 

Following the EWF proxy for ImDisk I posted here based on libewf library, I thought I could do the same for VMDK based on libvmdk.

 

Supported (read) VMDK formats :

  • RAW (flat)
  • COWD version 1 (sparse)
  • VMDK version 1, 2 and 3 (sparse)

 

Find here attached the proxy.dll (and the source code) for ImDisk (and other software using sharedmemory proxy such as ImgMount / Arsenal driver).

 

To launch the proxy : devio --dll=proxy.dll;dllopen shm:test_proxy c:\test.vmdk.

To use the proxy from ImDisk : imdisk -a -t proxy -o shm -o ro -f test_proxy -m Z: .

 

Note the RO flag as the libvmdk does not support writing for now.

 

idpfrmR.png

 

Not fully related but now that I can manipulate vmdk files, I might add this feature to CloneDisk as well later on.

 

Regards,

Erwan

Attached Files


  • Olof Lagerkvist likes this

#2 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1366 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 21 July 2014 - 04:54 PM

Really nice and interesting work, thanks a lot for sharing it!



#3 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 21 July 2014 - 05:13 PM

Thanks Olof but the main credits actually go to you for your excellent softwares and also Joachim Metz for his many libraries :)



#4 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 21 July 2014 - 05:32 PM

Very good work :worship:

There are several other VERY interesting libraries by Joachim if you are into this. :)

 

Though many of these are still Alpha/experimental, having also the .vhd one, the .qcow one would be nice additions to Clonedisk.

 

And of course you know how I would favour this solution over the .NET based ones.

 

As side note I would also find interesting, the one for VSS copies, and of course, if it can work as well as a filesystem, see: http://reboot.pro/to...s-a-filesystem/  the REGF one would be IMHO useful :).

 

:duff:

Wonko



#5 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 21 July 2014 - 05:43 PM

Great minds think alike :)

I am already working on the VHD one and the REGF/VSS ones also are on my radar.

 

And indeed, my main driver is the native part as an alternative to .NET.

 

And I like the fact that these pieces of codes can be used for several purposes : imdisk proxies, clonedisk / imgmount units, etc...



#6 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 April 2015 - 01:15 PM

Ok, let's start. :frusty:

 

Got the download. :)

It doesn't work. :(

 

Devio.exe, when run as:

 

devio --dll=proxy.dll;dllopen shm:test_proxy C:\appoggio\scrambled\vmdkisotest\thesplit.vmdk

seemingly requires a msvcr120.dll.

 

Once this has been downloaded and added into the same directory where devio.exe is, it throws an error:

Cannot find entry point GetLogicalProcessorInformation in dynamic link library KERNEL32.DLL

 

Windows XP SP2 here (and YES, I know and NO, this is NOT negotiable).

 

Either devio.exe or *something else* :unsure: will probably need to be recompiled to become working on XP.  :angry7:

 

:duff:

Wonko



#7 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1366 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 28 April 2015 - 01:23 PM

devio.exe itself should run fine even on Windows NT 3.51, unless I have made some mistake very recently. But I don't think so because I cannot remember having even tried to build it with VC++ 2013. But depends.exe would show you exactly what is requiring something that your OS doesn't have.

#8 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 28 April 2015 - 01:49 PM

Ok, let's start. :frusty:

 

Got the download. :)

It doesn't work. :(

 

Devio.exe, when run as:

 

devio --dll=proxy.dll;dllopen shm:test_proxy C:\appoggio\scrambled\vmdkisotest\thesplit.vmdk

seemingly requires a msvcr120.dll.

 

Once this has been downloaded and added into the same directory where devio.exe is, it throws an error:

Cannot find entry point GetLogicalProcessorInformation in dynamic link library KERNEL32.DLL

 

Windows XP SP2 here (and YES, I know and NO, this is NOT negotiable).

 

Either devio.exe or *something else* :unsure: will probably need to be recompiled to become working on XP.  :angry7:

 

:duff:

Wonko

 

Must be a libvmdk.dll dependencies.

 

I should have distributed the msvc runtime dll in the zip file.

 

The GetLogicalProcessorInformation  should work with XP Sp3 thus ... (i know i know...).

And I cant tell yet for sure it is a dependency from libvmdk.dll.



#9 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 28 April 2015 - 01:59 PM

Must be a libvmdk.dll dependencies.

 

I should have distributed the msvc runtime dll in the zip file.

 

The GetLogicalProcessorInformation  should work with XP Sp3 thus ... (i know i know...).

And I cant tell yet for sure it is a dependency from libvmdk.dll.

 

quickly checking the imports section for devio.exe, proxy.dll and libvmdk.dll i cannot find any reference of this API ?

 

could it be the ms vc runtime?

checking....

 

EDIT : confirmed, the issue is with the MS VC runtime.

 

Try the previous version i.e msvcr110.dll? or even msvcr100.dll?

 

EaGaOdZ.png



#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 April 2015 - 02:02 PM

devio.exe itself should run fine even on Windows NT 3.51, unless I have made some mistake very recently. But I don't think so because I cannot remember having even tried to build it with VC++ 2013. But depends.exe would show you exactly what is requiring something that your OS doesn't have.

Yep :), that's the reason why I posted here and cited the *something else* as possible culprit, though I don't use often devio.exe, every time I used it (for simpler tests) it always worked :thumbup:.

 

The point is that IF I used a stupider, already much more bloated later Windows OS, I would already have the stupid .Net (possibly 4.0 version) installed and then I would use the IMDISK with discutils.dll (or whatever other stupid .Net  facility it would provide), but our good friend erwan.l :) offered this thing here as a native alternative to the .Net bloat, seemingly without checking if it worked at all, natively.

 

As you might be well aware I am a self-declared, grumpy, old, bastard, and when I download something described as "native", I expect and demand that it should §@ç#ing work out-of-the-box, or at least provide the list of what (the heck) it needs in order to work or state how it is (say) targeted only to Vista or 7 or later and won't work on XP or on XP SP3 only.

 

And yes I know that XP is out of extended support, and that I should get a later Windows OS or at least upgrade to SP3, but this isn't going to happen any soon.

 

@erwan.l

No prob about the msvc .dll (though it is a nuisance and makes very little sense) as it is easy to add it, while the GetLogicalProcessorInformation is a show stopper (at least for me) until some good soul will recompile (if possible at all) the libvmdk.dll (which BTW is now on github https://github.com/libyal/libvmdk) without these stupid dependencies.

 

 

:duff:

Wonko



#11 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1366 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 28 April 2015 - 02:08 PM

To build an application for Windows XP in VC++ 2013 you would need to specify a Platform Toolset that is XP compatible. "Visual studio 2013 - XP compatible" would usually work. But I strongly recommend using Visual studio 2010 or earlier, since they are out-of-the-box guaranteed to build for XP without any special tweaks needed.

#12 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 28 April 2015 - 02:28 PM

Issue is with msvcr120.dll which (hard)links GetLogicalProcessorInformation (only available on XP SP3).

 

Too bad as libvmdk does use this dll but does not use this api (and probably use basic C functions which are in msvcr100.dll).

 

Apart from rebuilding libvmdk.dll with a lower MS VC runtime (like msvcr110.dll), any other ideas?

I suppose renaming a msvcr110.dll to msvcr120.dll is a bad idea?

 

For the record this is a hack/patch for this issue here.



#13 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1366 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 28 April 2015 - 02:31 PM

No other ideas. Easiest is probably to build with 2010. If you have VS 2010 installed you can select it as Platform Toolkit in 2013 projects as well, so it's usually very easy.

#14 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 28 April 2015 - 02:35 PM

No other ideas. Easiest is probably to build with 2010. If you have VS 2010 installed you can select it as Platform Toolkit in 2013 projects as well, so it's usually very easy.

 

I use VS2013 on my machine and i am usually not into VC projects (except to build libvmdk and a few other projects).

I'll see what I can do but have little hope as I am not a VC expert :(

 

Any chance you would consider building the patch I posted in previous post ?

Then we could made a custom MSVCR120.DLL running on XP SP2 for Wonko : I feel he will need it more and more if he wants to stick to his grand pa' windows.

 

Dont tell him I called him that way ;)



#15 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1366 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 28 April 2015 - 02:42 PM

I could try that later. But anyway, all you would need to do actually is download and install Visual C++ 2010 Express open the solution and build. Doesn't that work?

#16 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 April 2015 - 03:12 PM

Double post, sorry.  :blush:

 

:duff:

Wonko



#17 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 April 2015 - 03:13 PM

Naah, no need to worry.

 

The msvcr100.dll does not work, but a msvcr110.dll copied to the directory and renamed to msvcr120.dll seems like working fine, at least with a "monolithic flat" image.

 

:duff:

Wonko



#18 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 28 April 2015 - 04:36 PM

Naah, no need to worry.

 

The msvcr100.dll does not work, but a msvcr110.dll copied to the directory and renamed to msvcr120.dll seems like working fine, at least with a "monolithic flat" image.

 

:duff:

Wonko

 

 

good to hear it works !

I'll include msvcr120.dll in the zip file and will also includ a msvc_xpSP2 folder with a short text file explaining why/what etc...

 

lets hear from you if you the split iso file works thru the vmdk (although a vmdk is initially meant for a harddrive image).



#19 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 April 2015 - 04:59 PM

Yep, verified :).

 

It works nicely :thumbup:.

 

And not only that, the libvmdk evidently uses (and parses) less  data from .vmdk descriptor file (and it needs not to be actually have a .vmdk extension), in other words it can use successfully a much simplified descriptor file.

 

I will soon post a few notes on the relevant thread:

http://reboot.pro/to...ng-split-image/

 

:duff:

Wonko



#20 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 28 April 2015 - 06:09 PM

Yep, verified :).

 

It works nicely :thumbup:.

 

And not only that, the libvmdk evidently uses (and parses) less  data from .vmdk descriptor file (and it needs not to be actually have a .vmdk extension), in other words it can use successfully a much simplified descriptor file.

 

I will soon post a few notes on the relevant thread:

http://reboot.pro/to...ng-split-image/

 

:duff:

Wonko

 

Excellent!

 

I truly enjoy these brainstorming/challenging sessions : what if... lets try this... lets do that...



#21 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 April 2015 - 06:33 PM

Excellent!

 

I truly enjoy these brainstorming/challenging sessions : what if... lets try this... lets do that...

Yep :smiling9:, this is where most of the fun is.

 

:duff:

Wonko



#22 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 29 April 2015 - 10:21 AM

@erwan.l
@Olof
The thingy seems like working fine, but right now (unless there is a trick/command parameter which I completely missed :w00t: :ph34r:) it is not usable from batch (or it is very inconvenient).

When I run the:
devio --dll=proxy.dll;dllopen shm:test_proxy c:\test.isi

A command window "waits" for the IMDISK connection.

So I have to open another command window to issue in it the:
imdisk -a -t proxy -o shm -o ro -o cd -f test_proxy -m Z:

And the first window remains open, logging (needlessly outside perhaps troubleshooting) read requests or *whatever*.

Ideally, in my perverted mind, there should be a mechanism to either automatically spawn a separate window to "wait for the connection" (windows that should close automatically as soon as the connection is established, unless a /log or /debug parameter is given explicitly, leaving the proxy enabled) or the proxy should just install/activate and then there would be the need for a separate command to check the status of the proxy and shut it down manually once the IMDISK connection is closed.

:duff:
Wonko

#23 erwan.l

erwan.l

    Gold Member

  • Developer
  • 2286 posts
  • Location:Nantes - France
  •  
    France

Posted 29 April 2015 - 10:40 AM

@erwan.l
@Olof
The thingy seems like working fine, but right now (unless there is a trick/command parameter which I completely missed :w00t: :ph34r:) it is not usable from batch (or it is very inconvenient).

When I run the:
devio --dll=proxy.dll;dllopen shm:test_proxy c:\test.isi

A command window "waits" for the IMDISK connection.

So I have to open another command window to issue in it the:
imdisk -a -t proxy -o shm -o ro -o cd -f test_proxy -m Z:

And the first window remains open, logging (needlessly outside perhaps troubleshooting) read requests or *whatever*.

Ideally, in my perverted mind, there should be a mechanism to either automatically spawn a separate window to "wait for the connection" (windows that should close automatically as soon as the connection is established, unless a /log or /debug parameter is given explicitly, leaving the proxy enabled) or the proxy should just install/activate and then there would be the need for a separate command to check the status of the proxy and shut it down manually once the IMDISK connection is closed.

:duff:
Wonko

 

At home (but not there right now) I have a batch that does it all (launch devio, launch imdisk).

From (sometimes failing) memory, I use the start (or call?) DOS keyword to spawn a separate process.

 

Devio needs to remain open as this is a daemon. It will close itself once the client closes the connection.

You can probably hide it still with the start command.

 

As for the debugging, it comes from the proxy but i could make it optional thus once you hide the window, you would not realy care about the debugging text it and could still be piped to a log file.



#24 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1366 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 29 April 2015 - 10:42 AM

@erwan.l
@Olof
The thingy seems like working fine, but right now (unless there is a trick/command parameter which I completely missed :w00t: :ph34r:) it is not usable from batch (or it is very inconvenient).

When I run the:
devio --dll=proxy.dll;dllopen shm:test_proxy c:\test.isi

A command window "waits" for the IMDISK connection.

So I have to open another command window to issue in it the:
imdisk -a -t proxy -o shm -o ro -o cd -f test_proxy -m Z:

And the first window remains open, logging (needlessly outside perhaps troubleshooting) read requests or *whatever*.


Well, I often script devio and imdisk myself and I use the following:
start devio shm:test_proxy c:\image.iso
imdisk -a -t proxy -o shm -o cd -f test_proxy -m Z:

Or similar. devio.exe displays some initial information about the image, whether it detected an MBR, any partitions, vhd structure etc. After ImDisk connects to it, it only logs errors. The window closes automatically when the ImDisk drive is dismounted. There is of course also the /b switch to start command and > nul tricks if you don't want a separate window and/or messages. There is of course also runwait /hide and similar if you don't want devio to create any window at all and if you need it to be completely disconnected from your original cmd window.
 

Ideally, in my perverted mind, there should be a mechanism to either automatically spawn a separate window to "wait for the connection" (windows that should close automatically as soon as the connection is established, unless a /log or /debug parameter is given explicitly, leaving the proxy enabled) or the proxy should just install/activate and then there would be the need for a separate command to check the status of the proxy and shut it down manually once the IMDISK connection is closed.


Good ideas, thanks. There are various tricks to find out whether a process owns the console window it launches within, or if it is shared with any other process such as cmd.exe. If shared, it could relaunch itself in a new window. Then, after client connection, it could try to find the console window handle and hide it. It should work, but it would require some window/console handle "magic". :)



#25 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1366 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 29 April 2015 - 10:45 AM

@All

By the way, I am about to recompile libewf, libvhdi, libvmdk, libsmraw and libodraw so that they use only system dlls and no particular VC++ runtime dlls. It looks right now like the x86 versions will require minimum Windows 2000 and the x64 minimum Windows Server 2003 or XP. I will also make some small corresponding libewf_devio.dll, libvmdk_devio.dll etc files for use with devio.

 

Should be ready pretty soon.







Also tagged with one or more of these keywords: vmdk, proxy, imdisk

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users