23 replies to this topic

[kcom]

I have VirtualBox 1.6.0, host is Windows XP (on NTFS) and guest is Windows XP (on FAT32). After installign ImDisk I gone inside the folder with the xp.vdi virtual disk, right clicked it and mounted it. (Also did try to chose harddisk manually.)

It will mount the disk but Windows on host is asking to format the disk. So somehow I think it isn`t correctly mounted.

What can be wrong?

[was_jaclaz]

The OFFSET to the bootsector.

Read here:
http://www.boot-land...?showtopic=4127
http://www.boot-land...?showtopic=2220

Particularly here:
http://www.boot-land...topic=2220&st=3

AND here:
http://www.boot-land...?showtopic=4850

(to be able to supply an offset, you will need to use command line or the IMDISK Control Panel Applet, right-click won't work - it defaults to offset 0)

jaclaz

[Olof Lagerkvist]

(to be able to supply an offset, you will need to use command line or the IMDISK Control Panel Applet, right-click won't work - it defaults to offset 0)

Just an additional note here, with recent versions of ImDisk it is actually possible to use the "right-click" method with offsets too because it brings up the same dialog as used with the Control Panel applet, in the right-click case just with the filename automatically filled in.

[was_jaclaz]

I stand corrected .

jaclaz

[Brito]

Hi Olof,

Any plans for automatic recognition of a few popular file formats?

The automatic .vdi support would also be welcomed since it has become a very handy way of transferring files rather than using the emulated network link between the host and guest machines.

Even the other day had to look for a tool to convert a nero image back to ISO because I had forgotten the correct offset for this type of image.

For vmware and alike I know it would cause more work to recognize the proper geometry but wouldn't it be a worthy feature rather than using a wrapper for this task?

Just a few ideas.

[pscEx]

Any plans for automatic recognition of a few popular file formats?

The automatic .vdi support would also be welcomed since it has become a very handy way of transferring files rather than using the emulated network link between the host and guest machines.

Hi Olof, to give you a bit 'psychologic pressure':
This is a feature I also would like to have.

Peter

[Olof Lagerkvist]

Me too I miss this myself too, so I have done a few tests and it looks promising. I think I can include auto-offset for a few image formats pretty soon

[pscEx]

Me too I miss this myself too, so I have done a few tests and it looks promising. I think I can include auto-offset for a few image formats pretty soon

Peter

[kcom]

The links are not really about 'mounting VirtualBox vdi file' and none clearly saying what to enter where. I found this http://forums.virtua...6208c5b59198dbf

I have hexeditor HxD. There is a blue offset number 000012050 and the readable text is something with FAT32. Does not work when I enter it in the offset part...

[pscEx]

[quote name='kcom' post='37029' date='Jun 5 2008, 09:29 PM']The links are not really about 'mounting VirtualBox vdi file' and none clearly saying what to enter where. I found this http://forums.virtua...6208c5b59198dbf

I have hexeditor HxD. There is a blue offset number 000012050 and the readable text is something with FAT32. Does not work when I enter it in the offset part...[/quote]
I think that 'the links' are due to Jaclaz's post.

Please try to remember for future replies (to make it easier for other members to understand):
Use the 'Add Reply' in the very bottom of the post only, if your reply is a general issue for the whole topic.

If you want to reply to a specific post, click the 'Reply' at the bottom of this individual post.
And: To help the forum not to store too many unnecessary data, delete inside the [Quote] what is not necessary for your reply.

Peter

[Brito]

These are really good news!

Thank you Olof.

[pscEx]

Thank you Olof.

The boss already posted a 'Thanks'
Why I should do the same?
But I do!

Peter

[was_jaclaz]

The links are not really about 'mounting VirtualBox vdi file' and none clearly saying what to enter where. I found this http://forums.virtua...6208c5b59198dbf

I have hexeditor HxD. There is a blue offset number 000012050 and the readable text is something with FAT32. Does not work when I enter it in the offset part...

The links (the ones I posted) are EXACTLY the needed ones.

First one introduces the "concept" of OFFSET:
http://www.boot-land...?showtopic=4127

Second one gives values for some common formats:
http://www.boot-land...?showtopic=2220

Third one is a link to a post in the thread above:
http://www.boot-land...topic=2220&st=3
that introduces THREE infos:
1) a link to a post on 911CD forum where the OFFSET for a particular .vdi image is given
2) that not all .vdi images have the same OFFSET
3) consequently, a suggestion about a "general" way to find the needed OFFSET inside a "generic" image file

Fourth one is a link to a thread started by cdob where he shows a good example of a batch file to do what introduced in point 3) of the previous given link.

Re-check them now that you have had an explicit explanation of the logic behind them and the order in which they were given, maybe you will change your opinion about them being appropriate or not.


I apologize however if service was not up to your expectations...


jaclaz

[bilou_gateux]

Google search engine, search ".vdi"+"offset" found Utilisation des images VirtualBox (vdi)

Although written in French, it may help you and i can translate it to english.

vditool usage

vditool.exe (32bit Windows)


C:\Program Files\innotek VirtualBox>vditool dump vista.vdi
vditool Copyright © 2004-2005 innotek GmbH.

Dumping VDI image file="vista.vdi" into the log file...
Log created: 2008-04-07T22:21:24.901576000Z
Executable: /usr/lib/virtualbox/vditool
Arg[0]: vditool
Arg[1]: dump
Arg[2]: vista.vdi
--- Dumping VDI Disk, Images=1
Dumping VDI image "vista.vdi" mode=r/o fOpen=1 File=00000004
Header: Version=00010001 Type=2 Flags=0 Size=16106127360
Header: cbBlock=1048576 cbBlockExtra=0 cBlocks=15360 cBlocksAllocated=15360
Header: offBlocks=512 offData=61952
Header: Geometry: C/H/S=31207/16/63 cbSector=512 Mode=3
Header: uuidCreation={76a7017b-df0f-4a61-1a98-86d9abed5098}
Header: uuidModification={3f2921ea-ff02-4229-94ab-6960293cd315}
Header: uuidParent={00000000-0000-0000-0000-000000000000}
Header: uuidParentModification={00000000-0000-0000-0000-000000000000}
Image: fFlags=00000000 offStartBlocks=512 offStartData=61952
Image: uBlockMask=000FFFFF uShiftIndex2Offset=20 uShiftOffset2Index=20 offStartBlockData=0
The operation completed successfully!



Useful infos are bolded

Reading output, offStartData=61952 offset of image data from the begining of image file.

This value is MBR start.

+----------------------------------------------------------------------------------------------------------+
Google search engine, search site:http://forums.virtualbox.org vdi offset
+----------------------------------------------------------------------------------------------------------+
My 1st attempt:
This only works with a fixed-size virtual disk image!
I have created a fixed size disk named 4MB.vdi format /FS:NTFS

vditool	Copyright (c) 2004-2005 innotek GmbH.



Dumping VDI image file="R:\temp\4MB.vdi" into the log file...

Log created: 2008-06-06T18:32:35.718554200Z

Executable: C:\Program Files\innotek VirtualBox\vditool.exe

Commandline: vditool dump R:\temp\4MB.vdi 

--- Dumping VDI Disk, Images=1

Dumping VDI image "R:\temp\4MB.vdi" mode=r/o fOpen=1 File=00000798

Header: Version=00010001 Type=2 Flags=0 Size=4194304

Header: cbBlock=1048576 cbBlockExtra=0 cBlocks=4 cBlocksAllocated=4

Header: offBlocks=512 offData=1024

Header: Geometry: C/H/S=8/16/63 cbSector=512 Mode=3

Header: uuidCreation={a314b8c5-133e-47c5-b97b-07658c662e6e}

Header: uuidModification={bc1f033f-331d-416a-988a-4019fd7c6eb2}

Header: uuidParent={00000000-0000-0000-0000-000000000000}

Header: uuidParentModification={00000000-0000-0000-0000-000000000000}

Image:  fFlags=00000000 offStartBlocks=512 offStartData=1024

Image:  uBlockMask=000FFFFF uShiftIndex2Offset=20 uShiftOffset2Index=20 offStartBlockData=0

The operation completed successfully!

Search the magic string: "eb 52 90 4e 54 46 53" (= "ëR.NTFS")

gsar -b -s:xeb:x52:x90:x4e:x54:x46:x53 R:\temp\4MB.vdi

output

r:\temp\4MB.vdi: 0x8200
r:\temp\4MB.vdi: 0x308000
r:\temp\4MB.vdi: 2 matches found

HEX2DEC v1.00 - Hexadecimal to decimal number converter
Copyright © 1992, S.G. Gill

hex2dec 8200

output

hex2dec 8200

Hex:   8200
Dec:   33280

imdisk -a -t file -f R:\temp\4MB.vdi -b 33280 -m #:

Success!
+----------------------------------------------------------------------------------------------------------+
My 2nd attempt:
This only works with a fixed-size virtual disk image!
I have created a fixed size disk named NewHardDisk1G.vdi format /FS:NTFS

vditool	Copyright (c) 2004-2005 innotek GmbH.



Dumping VDI image file="R:\Temp\NewHardDisk1G.vdi" into the log file...

Log created: 2008-06-06T20:44:21.187304200Z

Executable: C:\Program Files\innotek VirtualBox\vditool.exe

Commandline: vditool dump R:\Temp\NewHardDisk1G.vdi 

--- Dumping VDI Disk, Images=1

Dumping VDI image "R:\Temp\NewHardDisk1G.vdi" mode=r/o fOpen=1 File=00000798

Header: Version=00010001 Type=2 Flags=0 Size=1073741824

Header: cbBlock=1048576 cbBlockExtra=0 cBlocks=1024 cBlocksAllocated=1024

Header: offBlocks=512 offData=4608

Header: Geometry: C/H/S=2080/16/63 cbSector=512 Mode=3

Header: uuidCreation={2e64246f-ad6e-459d-ab0f-04689763e252}

Header: uuidModification={fbb031be-6f34-4a81-a9bb-228ec39f2823}

Header: uuidParent={00000000-0000-0000-0000-000000000000}

Header: uuidParentModification={00000000-0000-0000-0000-000000000000}

Image:  fFlags=00000000 offStartBlocks=512 offStartData=4608

Image:  uBlockMask=000FFFFF uShiftIndex2Offset=20 uShiftOffset2Index=20 offStartBlockData=0

The operation completed successfully!

imdisk -a -t file -f NewHardDisk1G.vdi -b 36864 -m #:

Success!

+----------------------------------------------------------------------------------------------------------+

@cdob or @Jaclaz
Q: method to compute the offset 512*63+offStartData is correct ?

@Jaclaz
Q: Have you written a batch to do hex2dec conversion.
your mbrbatch.cmd already have a subroutine to do DEC2hex conversion

[jerh]

some information about the VDI header can be found here:

Tutorial: All about VDIs

I also posted there some code that could be of some help for mounting VDI files into imdisk

http://forums.virtua...p?p=31276#31276

(until native support in imdisk...)

[was_jaclaz]

Jerome, first thing welcome to the board.

Thanks for the informations.

Nice thingies you've put together.

Particularly, the idea of makesparsefile option to copy and update sparse files, something I have never found a smallish program for.



jaclaz

[TerryE]

I am a moderator on the VBox forum and the author of the above "All about VDIs" tutorial. I am just developing a section on how to access VDI contained file systems in the host and will be suggesting ImDisk for Windows users. So this is really just a courtesy to connect with this forum before doing so.

It might if I summarise the VDI format. All VDIs contain a prefix header, followed by 1 MByte pages. There is a fixed 512 byte preamble followed by a mapping page vector which is 4 bytes per page in the VDI; this is then rounded up to a sector boundary. The true offset is in the header as discussed in the above paper.

In the case of a static VDI, all pages are preallocated hence the mapping vector goes 0,1,2,3,... A 100Mybe static VDI will therefore have a 2 sector header with the MBR starting at offset 1024. Since most virtual geometries assume a track size of 63 sectors, the first partition will typically start at the 65th sector or at offset 33280.

A dynamic VDI is initialised with all entries = -1 (block not allocated). Reads from unallocated block return 0 sequences and writes to unallocated blocks create new pages on demand. You can also have snapshots and immutable disks which complicate things, though registering a VDI as writethough disables snapshotting for that disk.

So the general rule of thumb is to create and declare any FS that are to be shared this way as static writethough.

[pscEx]

Hi TerryE

First: Welcome in this forum!

Second: Some hope comes up that with your help I can do what I want to do since a long time: Mount a VDI with ImDisk.
You have to know: I'm an experienced programmer on the application side. But I'm a noob due to driver level.

And therefore I did not understand anything of the VBox forum's post mentioned above.

What I need is something like:

• Create the VDI drive with <syntax>
• Mount it with imdisk using <syntax> (or GUI settings)

Possible?
Would be great!

Peter

[TerryE]

I did not understand anything of the VBox forum's post mentioned above

Boo-Hoo Boo-Hoo. I am very upset.

Anyway, maybe I should say do you want to meet at you place or mine? I find this IP.Board GUI very dislocating after phpBB. Mind you I am still trying to the get the VBox boys to upgrade to phpBB3 so maybe I can't complain too much. But at least we have shortcut keys for bold etc.. Anyway back to topic.

First the VBox command to create and register a VDI container is VboxManage createvdi -filename vdiName -size vdiSize -static -register -type writethrough
but this still only creates an empty block device. Note that you must specify an absolute path in the vdiName because relative addresses are relative to the default VDI folder. Also the size is an (integer) number of MBytes, eg, 100 for 100 Mbytes. Also remember that the install does not add the C:\Program Files\Sun\xVM VirtualBox directory to the path, so you need to path this in yourself.

You then need to partition it and put on any file systems before it can be mounted. This is straight forward in Linux (but then again if my host was linux, I would just use a loop device anyway and not an imDisk. Given this the easiest way is then to allocate your VDI to a guest VM, boot it and allow the guest VM to partition it and initialise any filesystems, though I would strongly recommend sticking to a single partition per VDI in this sort of scenario.

Once the guest has initialised the FS on the VDI you can shut it down and then what you really want is to be able to automatically compute the offsets using some NT scripting language. Is it too much to assume that the punter has Perl installed? Well VDIs adopt a standard geometry so the 1st a partition always starts at offset 32256 + the header length. As long as the VDI < 256 Gbyte, the header length is just the VDI size mod 1048576. Well even I can write tis one in CMD language so I leave it to you to fill in the blanks :-)

//TerryE

[pscEx]

VboxManage createvdi -filename vdiName -size vdiSize -static -register -type writethrough
but this still only creates an empty block device.

Maybe that this the first step on my way to success.
Until now I used the VBox GUI to create the drive. Possibly in the result something is different!
Thanks.

You then need to partition it and put on any file systems before it can be mounted. This is straight forward in Linux (but then again if my host was linux, I would just use a loop device anyway and not an imDisk. Given this the easiest way is then to allocate your VDI to a guest VM, boot it and allow the guest VM to partition it and initialise any filesystems, though I would strongly recommend sticking to a single partition per VDI in this sort of scenario.

And here the noob is w/o knowledge how to go on.
Finally I want to mount the device with Imdisk. But how I can partition it, if I cannot mount? What's a loop device? ???

I do not want to bore you or waste your time in answering my questions. I'm going to Google etc to get more info. Maybe some unanswered questions appear, and I'll put them here then.


Peter

BTW: The current situation is very well known for me. In the past I often got questions which I answered technically correct. But the mistake I made was: I assumed that the people asking for help have the same level of knowledge like me.
As result:
I have been happy to have helped somebody.
The 'somebody' did not know anything more than before.

[was_jaclaz]

Maybe that this the first step on my way to success.
Until now I used the VBox GUI to create the drive. Possibly in the result something is different!
Thanks.


And here the noob is w/o knowledge how to go on.
Finally I want to mount the device with Imdisk. But how I can partition it, if I cannot mount? What's a loop device? ???

Peter,
We have a problem here.

IMDISK does NOT handle partitions.

It is a volume/partition driver NOT a hard disk image one.

VDI images are hard disk images.

There is NO WAY to use it to mount an unpartitioned drive image.
NOR to partition it.

Let's for a moment set aside VDI images and go back to conventional "RAW" hard drive images (like the ones VDK can mount), for the sake of simplicity, let's assume that this image has the "common" nx255x63 geometry:
1) first sector is the MBR
2) following 62 sectors are hidden sectors
3) following sector is the bootsector of first partition

To mount this image with IMDISK you have to use a parameter to skip first 63 blocks, or, if you prefer, 63x512=32256 bytes.

Since IMDISK is a partition/volume driver, it NEEDS the first sector it accesses to be the bootsector, ANYTHING preceding it is "a suffusion of yellow" for all it concerns it.

EVERYTHING it does is connected to "LogicalDrive", i.e. from the bootsector onwards.

Unlike IMDISK, VDK creates a //./PhysicalDriven, thus allows to access the image starting from first sector (the MBR), consequently allowing partitioning (read as "wtriting a valid MBR") and this is the method I use to create hard disk images in my MBRBATCH/MKIMG batches.

Nothing prevents from using the same approach to partition and format STATIC VDI images, if you are interested in such a feature, I can - time permitting - try to add the feature to the batches.

Once the image has been properly partitioned and formatted, it (to be more exact any of the partitions on the VDI hard-disk images) can be mounted and accessed normally through IMDISK, providing the correct OFFSET in sectors or bytes.

You can think of it like a tall box and two people trying to pick up something from it's bottom:
IMDISK is a guy with "shorter" arms and cannot reach the bottom of it
VDK is a guy with "longer" arms and can pick up things at the bottom allright

Hope the above clears the matter.



jaclaz

[pscEx]

Peter,
We have a problem here.

IMDISK does NOT handle partitions.

It is a volume/partition driver NOT a hard disk image one.

VDI images are hard disk images.

There is NO WAY to use it to mount an unpartitioned drive image.
NOR to partition it.

Let's for a moment set aside VDI images and go back to conventional "RAW" hard drive images (like the ones VDK can mount), for the sake of simplicity, let's assume that this image has the "common" nx255x63 geometry:
1) first sector is the MBR
2) following 62 sectors are hidden sectors
3) following sector is the bootsector of first partition

To mount this image with IMDISK you have to use a parameter to skip first 63 blocks, or, if you prefer, 63x512=32256 bytes.

Since IMDISK is a partition/volume driver, it NEEDS the first sector it accesses to be the bootsector, ANYTHING preceding it is "a suffusion of yellow" for all it concerns it.

EVERYTHING it does is connected to "LogicalDrive", i.e. from the bootsector onwards.

Unlike IMDISK, VDK creates a //./PhysicalDriven, thus allows to access the image starting from first sector (the MBR), consequently allowing partitioning (read as "wtriting a valid MBR") and this is the method I use to create hard disk images in my MBRBATCH/MKIMG batches.

Nothing prevents from using the same approach to partition and format STATIC VDI images, if you are interested in such a feature, I can - time permitting - try to add the feature to the batches.

Once the image has been properly partitioned and formatted, it (to be more exact any of the partitions on the VDI hard-disk images) can be mounted and accessed normally through IMDISK, providing the correct OFFSET in sectors or bytes.

You can think of it like a tall box and two people trying to pick up something from it's bottom:
IMDISK is a guy with "shorter" arms and cannot reach the bottom of it
VDK is a guy with "longer" arms and can pick up things at the bottom allright

Hope the above clears the matter.



jaclaz

Thanks, Jaclaz.

Your explanation is written in a level which can be understood also by an noob like me!

As a result: My dream can never become reality ! (?)

Peter

[TerryE]

I do not want to bore you or waste your time in answering my questions. I'm going to Google etc to get more info. Maybe some unanswered questions appear, and I'll put them here then.

The big problem with the FLOSS movement is getting people who are willing to answer the Qs pro bono. So when I see someone with your posting record (it doesn't matter what the forum is) then you have earned a lot of credits in my view. BTW looking at your profile we're quite similar. I am only 55 but then again I did first learn FORTRAN at 14! Anyway back to topic.

I think that we need to establish a level-set. If you are talking about VDIs then you must be planning to use VirtualBox. So this means that you have a host in mind and a guest, and what you are wanting to do is to share a FS in some way between the host and the guest. Am I right so far?

With VirtualBox their are two easy way to do this. The first is to use a standard network share using NT fileshares (and the *NIX compatibility product Samba on Linux). The second is to use the VirtualBox paravirtualised drivers (Guest Additions - GA) which you install on the Guest. These streamline the whole GUI interaction and also provide a mechanism for directly sharing file hierarchies on the Host with the guest much as if they were network fileshares but a lot faster.

Also most users use the compressed file system formats, so in general the only people that want to go down this route are if you are running an OS which does not support GA (e.g. Win9x) or you are debugging the install of say XP and you haven't yet got to the point where GA are installed and running.

If you are only talking about NT on NT then we need a slightly different conversation than one where Linux is involved. So which are you interested in?

On Linux systems the ImDisk concept is built in as standard. A Loop Device allows you to map any contiguous file to a pseudo device which you can then use as a file system. This allows you to nest file systems as files in other file systems.

If this conversation is primarily about using ImDisk then we are both learning, because I have yet to install it on one of my XP systems and here is the right place to have this discussion. If it is more about VBox, then I don't mind having the conversation here but we should not go too off topic. It would be better continuing it on my forum and we can always post a synopsis and link here :-)

[pscEx]

See my comments in blue

The big problem with the FLOSS movement is getting people who are willing to answer the Qs pro bono. So when I see someone with your posting record (it doesn't matter what the forum is) then you have earned a lot of credits in my view. BTW looking at your profile we're quite similar. I am only 55 but then again I did first learn FORTRAN at 14! Anyway back to topic.
Just to show blue

I think that we need to establish a level-set. If you are talking about VDIs then you must be planning to use VirtualBox. So this means that you have a host in mind and a guest, and what you are wanting to do is to share a FS in some way between the host and the guest. Am I right so far?
Youn got it 100%

With VirtualBox their are two easy way to do this. The first is to use a standard network share using NT fileshares (and the *NIX compatibility product Samba on Linux).
Not 'impossible', but not recommended. See below.
The second is to use the VirtualBox paravirtualised drivers (Guest Additions - GA) which you install on the Guest. These streamline the whole GUI interaction and also provide a mechanism for directly sharing file hierarchies on the Host with the guest much as if they were network fileshares but a lot faster.
Currently I do not know the GA.

But let me explain my needs:

I developed a WinBuilder project 'nativeEx_TrackWBInstall'
There inside a VM a PE comes up, installs a 'setup.exe' or whatelse and writes a WinBuilder script to a 'fixed' drive.
This script contains 'everything': Registry as well as copied files etc.

After leaving the PE / VM, you can copy the script to a different WB project and you have it ...

In order to catch 'everything' the PE is minimized. No network, no explorer, only cmd etc.

This makes the first alternity impossible.

Currently I'm writing to a VMWare disk. But there are some disadvantages I want to remove by use of a VDI:
Starting the PE including the actual 'setup.exe' can be done from a WB project on the host.
But when the VM client returns with the WB script in the VMWare drive, inside WinBuilder I do not have the possibility to catch this script and to write it into my project. I have to use a different GUI app 'outside' WinBuilder.
But I think, with mounting the VDI drive by a cmd line, I can catch the script and copy it to whereever I want.

If this conversation is primarily about using ImDisk then we are both learning, because I have yet to install it on one of my XP systems and here is the right place to have this discussion. If it is more about VBox, then I don't mind having the conversation here but we should not go too off topic. It would be better continuing it on my forum and we can always post a synopsis and link here :-)

Maybe the handicapped blind person with the handicapped lame one on his shoulder become a good team!

I'm going to have a qualified look into your forum tomorrow.

Peter