17 replies to this topic

[Olof Lagerkvist]

I have now built a few of the libyal libraries along with some test tools provided by libyal author. Then, I have added small "devio adapter dll files" along with each of them as well as devio.exe itself for convenience. The adapter dll files are called the same as the libyal library with _devio suffixed. For instance, to use libsmraw.dll with devio you use libsmraw_devio.dll. I hope I got everything right in the builds. No particular VC++ runtime dll files are needed, only system dll files are used.

 

x86 versions: http://ltr-data.se/f...ls_devio_x86.7z(require min Windows 2000)

x64 versions: http://ltr-data.se/f...ls_devio_x64.7z(require min Windows XP/Server 2003)

 

Example, to mount a multi-part image called image_multipart.001, image_multipart.002 and image_multipart.003:

start devio --dll=libsmraw_devio.dll;dllopen shm:proxy1 image_multipart.001+image_multipart.002+image_multipart.003
imdisk -a -t proxy -o shm -f proxy1 -m #:

Thanks to Joachim Metz for the great libyal libraries!

https://github.com/libyal/

 

[Wonko the Sane]

@Olof
The thingy seems to work fine (and no need for the stupid msvcr120.dll ), but there is still the "same" or "similar" issue:
start devio --dll=libsmraw_devio.dll;dllopen shm:proxy1 Core-current.iso1+Core-current.iso2
imdisk -a -t proxy -o shm -o ro -o cd -f proxy1 -m Z:

I can access the Z: drive fine, but the devio Window is filled with "Error reading image file" (not very different from the ones from erwan.l's proxy.dll).

"Your" proxy for libvmdk.dll fails for me:

devio --dll=libvmdk_devio.dll;dllopen shm:proxy1 Core-current.isi
No write support yet for vmdk files.
Library call failed to open 'Core-current.isi': m

 

 

Is there an added "read-only" parameter that can be passed?

 

Wonko

[Olof Lagerkvist]

@Olof
The thingy seems to work fine (and no need for the stupid msvcr120.dll ), but there is still the "same" or "similar" issue:
start devio --dll=libsmraw_devio.dll;dllopen shm:proxy1 Core-current.iso1+Core-current.iso2
imdisk -a -t proxy -o shm -o ro -o cd -f proxy1 -m Z:

I can access the Z: drive fine, but the devio Window is filled with "Error reading image file" (not very different from the ones from erwan.l's proxy.dll).

That sounds more like a devio.exe error message. Really strange that you see that there and the drive works anyway. I just tried now to split a couple of iso images and tried similar mounting but I did not see any error messages. I still think we should keep those error messages there because usually when such messages are displayed, something has gone wrong and the virtual disk is not working. In such cases, this is the only place where you can see any kind of explaining message about it. There is unfortunately no useful error message/code returned to the ImDisk driver so no error code can be forwarded to the filesystem and applications.

 

The question now is rather why you get error messages while the virtual disk is working anyway. That could be difficult to find out.
 

"Your" proxy for libvmdk.dll fails for me:
 
Is there an added "read-only" parameter that can be passed?

 
Yes, add -r to the command line, right before the same of the shared memory object (proxy1 in this case).
 

devio --dll=libvmdk_devio.dll;dllopen -r shm:proxy1 Core-current.isi

[Wonko the Sane]

With the -r added I get:

devio --dll=libvmdk_devio.dll;dllopen -r shm:proxy1 Core-current.isi
Opening image file...
'Core-current.isi'
Retrieving image virtual size

And then I have a error message from windows "An error occurred in devio.exe, the application will be closed" (or the like of it, the message is Italian) actual details are:

 

AppName: devio.exe AppVer: 0.0.0.0 ModName: ntdll.dll
ModVer: 5.1.2600.3520 Offset: 00018af2

 

Wonko

[Olof Lagerkvist]

With the -r added I get:

devio --dll=libvmdk_devio.dll;dllopen -r shm:proxy1 Core-current.isi
Opening image file...
'Core-current.isi'
Retrieving image virtual size

And then I have a error message from windows "An error occurred in devio.exe, the application will be closed" (or the like of it, the message is Italian) actual details are:

 

AppName: devio.exe AppVer: 0.0.0.0 ModName: ntdll.dll
ModVer: 5.1.2600.3520 Offset: 00018af2

 

Oops. I only tested with directly specifying a .vmdk file, because that's what I happened to have here. But I can check and see if something else needs to be done, or done in some different order, to make this library open images through isi files.

[Olof Lagerkvist]

Apparently it is a little bit more complicated than I first thought to support such descriptor files.

 

But just an idea, what if you use proxy.dll from Erwan's package and then libvmdk.dll from my package? I mean, maybe I am making things unnecessarily complicated and I really hate to reinvent wheels and such things. Erwans's various proxy dlls are probably working just fine and all I really wanted to participate with here were libyal versions without unnecessary OS/DLL/etc requirements. We could probably create a package together that solves both problems with a combination of our dll files.

[Wonko the Sane]

In the meantime I tested a reversed frankenproxy, using:

"your" libvmdk_devio.dll

"erwan.l's" libvmdk.dll version 20140421 (adding the stupid msvcr120.dll, actually a msvcr110.dll renamed)

 

The result is the same without the -r switch (which might mean that erwan.l "forced" it to be read only).

 

With the -r switch added the .iso works fine and I have the usual load of "Error reading image file" (which might mean that *whatever* libvmdk.dll signals, erwan.l proxy interprets it as "Read request - size:0 offset:0" and your proxy interpretes it as  "Error reading image file")

 

As  a side note, someone (any among you and erwan.l) will have before or later to explain to this hairy reasoner how comes that:

libvmdk_devio.dll 9.216 bytes

proxy.dll 396.288 bytes

 

I will try the other frankenproxy and report.

EDIT:

This is erwan.l's proxy.dll and your compiled libvmdk.dll

 

devio --dll=proxy.dll;dllopen shm:proxy1 Core-current.isi
File to open: Core-current.isi
TLibvmdk OK

then the same crash as before:

 

AppName: devio.exe AppVer: 0.0.0.0 ModName: ntdll.dll
ModVer: 5.1.2600.3520 Offset: 00018af2
 

Wonko

[erwan.l]

proxy.dll contains only a few lines of code which interfaces libvmdk.dll (compiled from github).

 

Yes i purposedly "nullify" the write requests by tricking the caller (devio) with sending the expecting return code.

Actually, i believe i could enable write requests in that proxy since libvmdk supports read/write.

 

The rather big size is a pitfall of delphi xe5 : even for a few lines of code, the binary is rather big

 

You lost me with the frankenproxy?

It goes devio->proxy.dll->libvmdk.dll (Olof and I should have the same binary I guess).

Only the proxy part will really differ I'd say.

 

Simple code for proxy.dll below.

library proxy;

uses
  SysUtils,
  Classes,
  windows,
  LibVMDK in 'libvmdk.pas';

{$R *.res}

type
  dllread_proc = function (handle:thandle; buf:pointer; size:cardinal; offset:int64): integer; cdecl;
  dllwrite_proc = function (handle:thandle; buf:pointer; size:cardinal; offset:int64): integer; cdecl;
  dllclose_proc = function (handle:thandle): integer; cdecl;

var
file_handle:thandle;
vmdk:TLibVMDK;


function my_read_proc(handle:thandle; buf:pointer; size:cardinal; offset:int64): integer; cdecl;
begin
	writeln('Read request - size:'+inttostr(size)+' offset:'+inttostr(offset));
	result:=vmdk.libvmdk_read_random (buf,size,offset);
end;

function my_write_proc(handle:thandle; buf:pointer; size:cardinal; offset:int64): integer; cdecl;
begin
  	writeln('Write request - size:'+inttostr(size)+' offset:'+inttostr(offset));
	//result:=ewf.libewf_write_random(buf,size,offset);
        result:=size;
end;

function my_close_proc(handle:thandle): integer; cdecl;
begin
	writeln('Close request');
	vmdk.destroy ;
	result:=0;
end;

function dllopen(filename:pchar; read_only:integer; var dllread:dllread_proc; var dllwrite:dllwrite_proc; var dllclose:dllclose_proc; var size:int64):thandle;cdecl;
var
value:ansistring;
begin
	writeln('File to open: '+ansistring(filename));

	dllread := my_read_proc;
	dllwrite := my_write_proc;
	dllclose := my_close_proc;

  vmdk:=TLibvmdk.create ;
  if vmdk.libvmdk_open_wide(widestring(filename),LIBVMDK_OPEN_READ )<>0
    then writeln('TLibvmdk NOK')
    else Writeln('TLibvmdk OK');
  if vmdk.libvmdk_open_extent_data_files <>1 then writeln('Unable to open extent data files');
  size:=vmdk.libvmdk_get_media_size ;
  writeln('size: '+inttostr(size));

	result:=1;
end;

exports
  dllopen index 1;

begin
end.

[Olof Lagerkvist]

In the meantime I tested a reversed frankenproxy, using:
"your" libvmdk_devio.dll
"erwan.l's" libvmdk.dll version 20140421 (adding the stupid msvcr120.dll, actually a msvcr110.dll renamed)

My suggestion was otherwise the quite opposite, like using "my" libvmdk.dll with erwan's proxy.dll. That way you don't need msvcr***.dll but should still be able to use the more "complete" integration with libvmdk API functions that erwan.i probably has in his proxy.dll. But I see further down that you tried that as well and got the same crash as with proxy.

 

I wonder if that could be an issue in recent libvmdk.dll code. I see that "my" libvmdk.dll seems to work fine for simply mounting .vmdk files. Maybe I can try with an older version of the code. I know that the versions of libyal libraries that are nowadays on GitHub sometimes have a few problems. Another one is that libewf.dll no longer works in write-overlay mode, only in read-only mode. On the other hand the most recent version supports the new .ex01 etc formats.
 

The result is the same without the -r switch (which might mean that erwan.l "forced" it to be read only).
 
With the -r switch added the .iso works fine and I have the usual load of "Error reading image file" (which might mean that *whatever* libvmdk.dll signals, erwan.l proxy interprets it as "Read request - size:0 offset:0" and your proxy interpretes it as  "Error reading image file")

That could be something. If my proxy dll simply forwards something that libvmdk.dll detects as "invalid" and responds with an error code and my proxy just forwards that error code back to devio instead of simply ignoring it and return success to devio, then it could explain something. Question is still why I cannot seem to reproduce any such things here. I have tried now on a couple of different Windows versions and - no error messages.
 

As  a side note, someone (any among you and erwan.l) will have before or later to explain to this hairy reasoner how comes that:
libvmdk_devio.dll 9.216 bytes
proxy.dll 396.288 bytes

Probably a huge runtime library built-in into erwan's dll. I pretty much never build things like that but instead use C++ runtime libraries included in Windows or if they don't provide what I need, I use msvcr100.dll or whatever could be needed, but that does not happen very often. That's why my exe and dll files are usually very small.
 

[erwan.l]

for the record, "my" libvmdk.dll was compiled using vs2013 on 20140421 .

"my" proxy.dll was compiled using delphi xe5.

[Olof Lagerkvist]

proxy.dll contains only a few lines of code which interfaces libvmdk.dll (compiled from github).

Thanks for the code. The only difference in my case is that I use the more "modern" version of Joachim's API model, with things like libvmdk_handle_open, libvmdk_handle_read_buffer_at_offset etc. But they should call the same internals in the dll as far as I can see in libvmdk sources.
 

Yes i purposedly "nullify" the write requests by tricking the caller (devio) with sending the expecting return code.

I see. Maybe I could try that as well. But I need reports from Wonko in particular to find out if that works, because I cannot seem to get such errors here for some reason.
 

Actually, i believe i could enable write requests in that proxy since libvmdk supports read/write.

It does? I didn't see any write support in the source code. Just a couple of stubs with "TODO" comments and "not yet supported" and things like that. Example: https://github.com/l...de/libvmdk.h.in

 

It looks like we might be using very different versions of libvmdk.dll. I downloaded the current version on GitHub.
 

[erwan.l]

 

It does? I didn't see any write support in the source code. Just a couple of stubs with "TODO" comments and "not yet supported" and things like that. Example: https://github.com/l...de/libvmdk.h.in

 

It looks like we might be using very different versions of libvmdk.dll. I downloaded the current version on GitHub.
 

 

my mistake sorry : got confused with libewf !!

[Wonko the Sane]

We are cross posting .

Lets see if I can recap.
Let's call Olibvmdk.dll the version of libvmdk.dll compiled by Olof and included in the download in this thread.

Let's call Elibvmdk.dll the version of libvmdk.dll compiled by Erwan.l and included in the download in the vmdk proxy thread.

 

#1 Plain erwan's:

Elibvmdk.dll

proxy.dll

msvcr120.dll (actually msvcr110.dll renamed)

Command Lines:

devio --dll=proxy.dll;dllopen shm:proxy1 Core-current.isi

imdisk -a -t proxy -o shm -o ro -o cd -f proxy1 -m Z:

 

DEVIO LOG:

File to open: Core-current.isi
TLibvmdk OK
size: 8286208
Successfully opened 'Core-current.isi'.
Read request - size:1536 offset:0
Image size used: 8286208 bytes.
Read request - size:512 offset:0
No master boot record detected. Using entire image.
Total size: 8286208 bytes. Using 8286208 bytes from offset 0.
Required alignment: 1 bytes.
Buffer size: 2097152 bytes.
Shared memory operation.
Waiting for connection on object proxy1. Press Ctrl+C to cancel.
Connection on object proxy1.
Read request - size:512 offset:0
Read request - size:2048 offset:32768
Read request - size:2048 offset:34816
Read request - size:2048 offset:36864
Read request - size:2048 offset:38912
Read request - size:2048 offset:40960
Read request - size:0 offset:0
Read request - size:4096 offset:0
Read request - size:0 offset:0
Read request - size:2048 offset:32768
Read request - size:2048 offset:34816
Read request - size:2048 offset:36864
Read request - size:2048 offset:65536
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:2048 offset:51200
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:0 offset:0
Read request - size:0 offset:0
Connection closed.
Close request
Image close result: 0

 

#2 Plain Olof's:

 

Olibvmdk.dll

libvmdk_devio.dll

 

Command Lines:

devio --dll=libvmdk_devio.dll;dllopen shm:proxy1 Core-current.isi

 

DEVIO LOG:

No write support yet for vmdk files.
Library call failed to open 'Core-current.isi': m

 

#3 Plain Olof's with added -r:

Olibvmdk.dll

libvmdk_devio.dll

 

Command Lines:

devio --dll=libvmdk_devio.dll;dllopen -r shm:proxy1 Core-current.isi

 

DEVIO LOG:

Opening image file...
'Core-current.isi'
Retrieving image virtual size

CRASH

 

#4 frankenproxy based on libvmdk_devio.dll as proxy

Elibvmdk.dll

libvmdk_devio.dll

msvcr120.dll (actually msvcr110.dll renamed)

 

 

Command Lines:

devio --dll=libvmdk_devio.dll;dllopen -r shm:proxy1 Core-current.isi

imdisk -a -t proxy -o shm -o ro -o cd -f proxy1 -m Z:

 

DEVIO LOG:

 

Opening image file...
'Core-current.isi'
Retrieving image virtual size
Image virtual size is: 8286208 bytes
Successfully opened 'Core-current.isi'.
Image size used: 8286208 bytes.
No master boot record detected. Using entire image.
Total size: 8286208 bytes. Using 8286208 bytes from offset 0.
Required alignment: 1 bytes.
Buffer size: 2097152 bytes.
Shared memory operation.
Waiting for connection on object proxy1. Press Ctrl+C to cancel.
Connection on object proxy1.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Error reading image file.
Connection closed.
Error closing image file.
Image close result: 0

 

 

#5 frankenproxy based on proxy.dll as proxy

Olibvmdk.dll

proxy.dll

 

Command Lines:

devio --dll=proxy.dll;dllopen shm:proxy1 Core-current.isi

 

DEVIO LOG:

File to open: Core-current.isi
TLibvmdk OK

CRASH

 

#1 and #4 Elibvmdk.dll allow me to mount with IMDISK and access normally the file in the split image file, though the "errors" in the LOG are different.

#2 Olibvmdk.dll does not work (but does not crash devio)

#3 and #5 Olibvmdk.dll crash devio 

 

It seems to me clear that the issue is *somehow* in Olibvmdk.dll, which might mean both that the compile is botched for any reason or that Joachim has introduced some changes in the source code since the time erwan.l compiled the Elibvmdk.dll

 

Wonko

[erwan.l]

As  a side note, someone (any among you and erwan.l) will have before or later to explain to this hairy reasoner how comes that:

libvmdk_devio.dll 9.216 bytes

proxy.dll 396.288 bytes

 

 

 

 

My developper personal feelings got hurt 

 

I decided to cleanup some dependencies, recompile the proxy.dll and include the msvcr120.dll (renamed msvcr110.dll in fact).

 

Now 90k !

Probably the best I can do.

 

Side note : i dont seem to be able to use libvmdk.dll on my brand new win 8.1 x64 system.

Still too early thus to blame either the O.S or my test vmdk file.

 

file uploaded here.

[Olof Lagerkvist]

I found a way to crash "my" version of libvmdk.dll as well. All I needed to do was to try the included vmdkinfo.exe tool on a .vmdk image. So, I downloaded and built a version of libvmdk from October 2014 and it seems to work better. At least it does not crash in that particular scenario.
 
I have also modified libvmdk_devio.dll so that it simply ignores zero length read and writes and no error message should show now in case such requests are sent to the proxy.
 
Download urls same as before.
x86 versions: http://ltr-data.se/f...ls_devio_x86.7z (requires min Windows 2000)
x64 versions: http://ltr-data.se/f...ls_devio_x64.7z (requires min Windows XP/Server 2003)

[Wonko the Sane]

I found a way to crash "my" version of libvmdk.dll as well. All I needed to do was to try the included vmdkinfo.exe tool on a .vmdk image. So, I downloaded a version of libvmdk from October 2014 and it seems to work better. At least it does not crash in that particular scenario.

So, the issue must be somewhere in the source library.
Anyway, now it works nicely here.
 

I have also modified libvmdk_devio.dll so that it simply ignores zero length read and writes and no error message should show now in case such requests are sent to the proxy.

Yep , now it works as it should.

@erwan.l
Nice try, really, I do appreciate your effort , but this is limbo :
http://en.wikipedia....i/Limbo_(dance)
the bar is now set at 10,000 bytes and Olof's proxy just managed to pass under it successfully (saving not only from the size of the proxy, but also the whole 860 Kb or so of the msvcr1x0.dll).
I do understand your "Delphi originated handicap", but there is no match .


Wonko

[erwan.l]

I do understand your "Delphi originated handicap", but there is no match .


Wonko

 

First my feelings got hurt now you just broke my heart 

 

See the positive side : it could be worse like written in .NET or Java 

[Wonko the Sane]

First my feelings got hurt now you just broke my heart 
 
See the positive side : it could be worse like written in .NET or Java 

Well, I fear that soon we might consider either .Net or Java as "simple" , as the good MS guys are seemingly well beyond that   see here (thanks to Formfiller for posting it on MSFN):
https://archive.is/DkDBq

Seemingly soon a number of things running on the new abomination (windows 10) will go through HyperV and emulate something else...

Isn't it funny how first we had the Internet (which is nothing much more, if you think about it, than Asimov's MultiVac ) and soon we will go to the original IBM approach of running VM's?


Wonko