Jump to content











Photo

Boot Win 7 VHD on Bare Metal PC from Empty Drive

vhd bare metal boot win 7 grub4dos

  • Please log in to reply
321 replies to this topic

Poll: Are you interested in booting Windows from VHD? (86 member(s) have cast votes)

Would you try to install OS to and boot from a single portable VHD file (virtual disk) instead of hard drive?

  1. Yes (83 votes [96.51%])

    Percentage of vote: 96.51%

  2. No (3 votes [3.49%])

    Percentage of vote: 3.49%

Would you be interested to copy that VHD file to an empty USB Thumb or HD and boot from it on real PC?

  1. Yes (82 votes [97.62%])

    Percentage of vote: 97.62%

  2. No (2 votes [2.38%])

    Percentage of vote: 2.38%

Did you try to boot OS from VHD on real PC instead of Virtual Machine?

  1. Yes, I usually boot VHDs saved on an internal hard drive (36 votes [38.30%])

    Percentage of vote: 38.30%

  2. Yes, I usually boot VHDs saved on a USB drive or thumb (11 votes [11.70%])

    Percentage of vote: 11.70%

  3. Yes, I boot VHDs saved on drives of any type (13 votes [13.83%])

    Percentage of vote: 13.83%

  4. Not yet (34 votes [36.17%])

    Percentage of vote: 36.17%

Vote Guests cannot vote

#76 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 23 November 2011 - 03:47 PM

why you named my name ... connected to the lousy screenshot ... which includes a ... lousy icon ....


I think you answered your own question.

Your comparison of VHD format with ZIP is interesting, though some - not so trivial - details must be specified.
There are three types of VHD's:

  • Static Fixed
  • Dynamic
  • Differencing
The first type is nothing but a RAW disk image with a single description sector appended to it.
There are three sub -cases for it:
  • the disk image is "filled to the brim" by one (or more) partition(s)
  • there is some "unused space" at the end of it
  • there is some "unused space" after the MBR and before the PBR of first partition (with the exception of the "normal" 62 or 2047 hidden sectors), as an example you create two parittions on the disk and later you delete first one.


Originally I was going to analogy of a ZIP file made with the "Store" option (no compression what-so-sever) and some 1920x1200 true-color BMPs that were entirely black. But I wanted to keep it simple.

You could (however) compress those giant black BMPs down to next to no size, had you not been a total doofus at the time you made the ZIP.

And again, it's not the problem of the ZIP program to deal with doofus related errors

IMHO:
In case 1.1 (probably 99% of commonly used VHD's) there is no possible way to reduce the size of the image file without touching the inner partition data.
In case 1.2 it is trivial to truncate the image after the last sector used and re-add the footer
In case 1.3 there is no possible way to reduce the size of the image without touching the inner partition data.


I would agree, with you there. Re 1.2 - Given the VHD specification is in the public domain, it probably is quite trivial to truncate and change a few bytes. And I allow for that in my closing statement about "if there is an easy hack".

I am not familiar with Dynamic or Differencing VHD's, but if you don't touch their contents I doubt that you can in any way reduce their size. :unsure:


I would have agreed with you before today, but reading the MSDN document (which I pasted above a few post up), apparently you *can* compress differencing VHDs.

If I may introduce some anecdotal here-say: I heard someone suggest that differencing VHDs grow in such a way, that seems to indicate they are more of a "log of disk writes".

The example cited, was that if you start with an empty differencing VHD, and add a 1GB file, the VHD will be 1GB.
If you delete the 1GB file, the VHD will still be 1GB
If you overwrote the 1GB file with another 1GB file, the VHD would be 2GB.

I assume that if you ran a 13 pass file eraser over the file, that would mean you had a 14GB VHD.

Hence: compressing differential VHD files is a WMI option for VHD manipulation.

Certainly, I can't think of any other way you'd be able to optimize a differential VHD.

#77 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 23 November 2011 - 04:07 PM

Since Eraser fills with zeros the same disk sectors over and over, the number of vdisk sectors in Diff that are different from Base doesn't change, hence the Diff VHD shouldn't grow too.

Compaction is the term that was not eagily explained by MS (but seems to be collecting all zero sectors in a single empty file), while the opp itself may follow pre-compaction (filling empty sectors with zeros). It looks like in newer Hyper-V implementations both opps were combined in one. It is a method of data compression, but it doesn't affect max size of a dynamic disk - only min size, and this is a problem, since once a dynamic VHD boots it reserves its max set size on the underlying physical disk. When you drop such a disk onto a USB Thumb, it can fit easily, until one tries to boot from it. That's why resizing a VHD file is as important in many cases as compacting its content.

I.e. compacting a disk makes it handier to store, while resizing its file makes it possible to boot the VHD from a smaller physical drive. :suda:

#78 cdob

cdob

    Gold Member

  • Expert
  • 1469 posts

Posted 23 November 2011 - 04:10 PM

A source code example to access BCD using VBScript and WMI
http://www.codeproje...ic_BCDEDIT.aspx

#79 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 23 November 2011 - 04:14 PM

sfinktah

Currently available tools like VHD Resizer appear to do just that: copy a VHD file to a smaller file, if a disk or volume image inside it is smaller than the file. I think it's easy to implement in VHD One (if you like this name). :)


I'm usually good at naming things. But I'm open to suggestions, these were my "Final Four":

Posted Image

I guess if you wanted to optimize the Google aspect, you'd perhaps name it to encompass more of it's capabilities (whatever they will be), eg:

VHD-BCD-Windows-Boot-Maker or somesuch... VHD2BCD... Or perhaps spin off a series of products like Acronis did.... VHD One, BCD One.... or there's the Microsoft solution: VHD One, VHD Two, VHD 95, VHD 2012 Home Edition....

You might still be interested to ask VirtualPC Guy (really in his Blog or by email), why they don't offer API for such a basic scenario, despite offering similar Expand feature.


One of the things I fear most about this project, is that Microsoft will just come out with such an program rendering it useless. I noticed already, that using the Microsoft iSCSI Target software, there is an "Expand VHD" option... (Microsoft iSCSI loves VHDs)...

Posted Image

Since we're deep into porn zipping, I get curious how one can compact a zipped porn. :confused1: Can you clarify in the same clear manner, what exactly happens during compaction (that might be accompanied with precompaction)? :photo:


I thought my expamples were pretty clear... maybe you're just not a porn guy...
  • precompacting would involve not including she-male porn in the first place.
  • converting images & movies to better file formats (eg, converting MPEG2 to MP4), and all images to JPG.
  • generally tidying up files you don't need (thumbnails, bits of HTML, redundant ZIP files of extract porn)
All quite similar to what you'd do to a VHD. I went quite crazy on a Windows XP image that had grown from 800mb compressed, to almost 4gb - simply from having had SP3, .NET 3.0, .NET 3.5, .NET 4.0, and other required updates install). And managed to get it down to about 1.7GB compressed.

Not nearly as much fun as sorting porn, I can assure you.

Incidental question: Can you actually apply SP1 to Server 2008 R2, if it's in a VHD? Mine has taken to simply saying: "Can't update the boot drive". I would just install SP1 from ISO, but it's so hard to find OEM ISO images.

#80 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 23 November 2011 - 04:27 PM

There might be numerous issues with updating to a next SP even on real hardware - I had to Repair Install Win 7 from its original Setup DVD before doing it, so you may want to try that. All previous Win updates will be invalidated, but they're incorporated in SP1 anyway. Any installed apps won't be affected. Usually the update problem is caused by overactive disk and registry cleanups, resulting in the records of previous installs and updates dissapear.

You may try using Search feature of My Digital Life news website - it has everything of that kind and more..., plus a very specific forum. :)

I like VHD Director, since its easier to remember and assume what it does for most users.
  • sfinktah likes this

#81 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 23 November 2011 - 04:40 PM

Since Eraser fills with zeros the same disk sectors over and over, the number of vdisk sectors in Diff that are different from Base doesn't change, hence the Diff VHD shouldn't grow too.


shouldn't ... but is that your instinct or your experience talking?

Compaction is the term that was not eagily explained by MS (but seems to be collecting all zero sectors in a single empty file), while the opp itself may follow pre-compaction (filling empty sectors with zeros).


That was the impression I got, and since I couldn't run their pre-compact program outside of Hyper-V, I felt EraserPortable with a 0000 custom fill would achieve the same thing. Although I never got to find out where the actual compression happened.

It looks like in newer Hyper-V implementations both opps were combined in one. It is a method of data compression, but it doesn't affect max size of a dynamic disk - only min size, and this is a problem, since once a dynamic VHD boots it reserves its max set size on the underlying physical disk. When you drop such a disk onto a USB Thumb, it can fit easily, until one tries to boot from it. That's why resizing a VHD file is as important in many cases as compacting its content.

I.e. compacting a disk makes it handier to store, while resizing its file makes it possible to boot the VHD from a smaller physical drive. :suda:


Well, I guess we now delving into the subject matter of NTFS and sparse files. And lets say (for arguments sake) that when you create a fixed size VHD, it's stored as a sparse file.... even if that is true, when you copy that VHD to your USB drive, the resultant VHD is definately not going to be stored as a sparse file.

Now, if we were talking about VMware, which I am more familiar with... there are two main formats for a VMDK (ignoring the snapshot aspect).

a) pre-allocated
B) dynamic

Although the documentation seems to talk about "thin" and "thick" provisioning, "flat" and "sparse", and what have you.... but I don't believe any of that.

A VMDK is either pre-allocated, in which case it is a byte-for-byte copy that you can literally "dd" onto a real disk, or it's dynamic... starts at 0 bytes and grows.

Now if at any time, you happen to delete some files on your dynamic VMDK, you can do and do this:

Posted Image

"Clean up Virtual Machine"... and you can reclaim some space. This is actually the VM I trimmed down from 3.8GB, and after I had deleted all the crap, I reclaimed back about 2GB. That size of the .VMDK is 1.7GB, and if I were to check the contents, I would see exactly 1.7GB of files + reserved MFT area.

I've always assumed that these dynamic VMDK files are stored on disk something like this:

[here's 3gb of stuff] *STUFF* [now here's 1gb of 0's we can skip] [here's more stuff] *STUFF*

So, it makes sense to me, that VHD dynamic disks would work in exactly the same way. It's just a question of whether they understand NTFS.

You see, if they did understand NTFS, they could just store the used parts of the disk. (Like some of the better disk back/imaging tools).

BUT, if they were "filesystem agnostic", and didn't understand NTFS, then they could simply rely on skipping big chunks full of 0's.

Thus, the pre-compaction and actual compaction. The pre-compaction runs inside the VM, which is running Windows and has the NTFS drivers. So it can easily zero out the bits that have no data.

That allows the VHD compaction process to shrink the disk size down, without having to know anything about the filesystem. It just has to compress the 0's.

I've really forgotten my point, but I think I understand compaction better for having writen whatever it was I just wrote.

#82 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 23 November 2011 - 04:43 PM

I like VHD Director



Yeah, you could be right. It works for Acronis, after all. :) Certinaly better than "VHD Ghost", or (heaven forbid) "VHD Magic"

#83 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 23 November 2011 - 04:49 PM

A source code example to access BCD using VBScript and WMI
http://www.codeproje...ic_BCDEDIT.aspx


Yeah, I checked that one out a few days ago, but I don't have the stomach for VBscript. Although I've been forced to do a project in VB.NET, that's as close as I ever wish to get to anything with "VB" in the name.

At the end of the day, it's all WMI, and there's plenty of documentation about WMI on MSDN. Although that link is a great resource for someone who just wants to include a quick fix in a script. I myself have a small piece of vbs that does my OEM registration, but I stole that, and only had to add 3 lines to include my GIGABYTE board.

#84 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 23 November 2011 - 04:53 PM

I think you answered your own question.

Latin saying of the day:
Nomen est omen

:whistling:


I fail to see what fun there is in falsely attributing me something, though. :unsure:

Trivial as in:

Well, I guess we now delving into the subject matter of NTFS and sparse files. And lets say (for arguments sake) that when you create a fixed size VHD, it's stored as a sparse file.... even if that is true, when you copy that VHD to your USB drive, the resultant VHD is definately not going to be stored as a sparse file.

Myabe yes, maybe no.
http://www.flexhex.c...rse-files.phtml
http://forum.emule-p...showtopic=70486

:cheers:
Wonko

#85 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 24 November 2011 - 12:53 PM

Latin saying of the day:
Nomen est omen


"mein Name ist ein Omen"... yes, I'm sure you are very sane.

I fail to see what fun there is in falsely attributing me something, though. :unsure:


From your point of view, that's probably true.

Trivial as in:



Well yes, although you make it look more complicated that it really is. I read the Microsoft Technical Specification on the VHD format, it's actually a very short and easy read... not at all technical. I would recommend anyone who works with VHD disks to give it a once-over.

http://www.microsoft...ang=en&id=23850

It very neatly explains how differencing VHDs are linked to their parents, and how stuff is stored.

It makes me think of taking back what I said earlier, and getting my hands dirty to implement a few NTFS/MBR hacks into the code by directly manipulating the VHD file itself - rather than just calling Windows to do stuff.

Myabe yes, maybe no.
http://www.flexhex.c...rse-files.phtml



Can They Really Be That Large?

Jeffrey Richter and Luis Felipe Cabrera in their article "A File System for the 21st Century"


When Jeffrey Richter says anything, the only thing you should be doing is sitting down and taking notes. That guy wrote one of the best books on Windows programming (and hacking) that has ever been written. (see: Programming Application for Windows; Microsoft Press).

I've book marked the article for the code on determine real file length, but I don't plan to get into the sparse file creation business if I can help it.

#86 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 24 November 2011 - 01:03 PM

Here's something I just picked up from a Microsoft Whitepaper on VHD performance:

Note: there are types of compaction for dynamic or differencing VHDs. One is done when file system is presented (i.e. you mount the VHD and bring the disk online and the other is not (VHD is offline). We recommend mounting a VHD first before doing compaction to improve the efficiency. This is also the default behavior when a compaction is initiated in Hyper-V Manager. However, DiskPart allows you to do an offline compaction without prompting you to mount it first. When that is the case, only data blocks full of zero will get released since no file system is involved. This may limit the effectiveness of the compact operation. Note that online compaction will fail with a file system limitation error if any volume snapshots exist.



#87 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 24 November 2011 - 02:22 PM

Can you actually apply SP1 to Server 2008 R2, if it's in a VHD? Mine has taken to simply saying: "Can't update the boot drive". I would just install SP1 from ISO, but it's so hard to find OEM ISO images.


What MS doesn't address in the VHD Spec is their marketing strategy, affecting product differentiation and compatibility policies. They store the Creator VM type in the VHD footer for a reason - and not always a technical one. I wonder if you tried to boot a VHD image created in Win7 VPC in a Win Server 2008R2 Hyper-V VM (before and after sysprepping it)? This should generally work after sysprepping, but merely the need to sysprep would be due to intentionally different VM hardware to limit cross-platform VHD usability. AND, someone will hopefully test how it works for VHDs created in VPCs 2004 & 2007: will they boot in 2008 & 2008R2 Hyper-V or Win7 VPC when attached as IDE HDs to it? This would also indicate, whether the Spec you mentioned is the latest in use by MS.

As to running a Win Server 2008R2 Hyper-V hypervisor from a VHD (hopefully you tried already) on real hardware, were you able to run any VMs in that hypervisor? Generally, placing Hyper-V on a native boot VHD shouldn't affect Hyper-V ability to run VMs, despite it has strict hardware requirements, making it hardly possible to run inside another VM with a different virtual hardware. But running Hyper-V from a VHD may somewhat degrade its VMs performance, though insignificant for VHD One feature testing purposes.

#88 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 24 November 2011 - 07:52 PM

It makes me think of taking back what I said earlier, and getting my hands dirty to implement a few NTFS/MBR hacks into the code by directly manipulating the VHD file itself - rather than just calling Windows to do stuff.


That seems the most efficient compare to VHD Resizer file copying approach, and looks like already done in VHD Tool for expanding a dynamic VHD. Its desirable to implement also:

- fast shrinking of a dynamic VHD by directly editing its footer's disk max size value,
- adjusting on demand a VHD creator type to improve its compatibility in various VPCs.

#89 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 12:54 AM

Posted Image

#90 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 01:06 AM

What MS doesn't address in the VHD Spec is their marketing strategy, affecting product differentiation and compatibility policies. They store the Creator VM type in the VHD footer for a reason - and not always a technical one. I wonder if you tried to boot a VHD image created in Win7 VPC in a Win Server 2008R2 Hyper-V VM (before and after sysprepping it)? This should generally work after sysprepping, but merely the need to sysprep would be due to intentionally different VM hardware to limit cross-market VHD usability. AND, someone will hopefully test how it works for VHDs created in VPCs 2004 & 2007: will they boot in 2008 & 2008R2 Hyper-V or Win7 VPC when attached as IDE HDs to it? This would also indicate, whether the Spec you mentioned is the latest in use by MS.


I have found Paragon's "Adjust OS" to remove the need to sysprep before a standard migration, and it has a similar function when using it's P2V feature. It has a seperate option for Virtual PC, Virtual Server, and Hyper-V, and just one VMware. The only visible difference was IDE or SCSI, but I didn't check the configuration files it made.

With regards to VHD footers, I too was a little alarmed to note the amount of fields taken up by version flags, creator names, and a few other things... I checked a VHD i had just converted with qemu, and it had some suitably bizarre creator name. I don't think they'll intentionally sabotage VHD, but perhaps I'm biased from just reading this article:

http://msdn.microsof...e/dd569754.aspx

It was clear the VHD format that Microsoft acquired from Connectix was well suited to the direction that Microsoft wanted to take with virtualization, namely to turn it into a platform. Whereas the VMware virtual disk formats were proprietary and very convoluted, often changing wildly from one release to the next, the VHD format was from the start straightforward and flexible enough to stand the test of time. In the intervening years, the VHD format has proven itself again and again, having been adopted by other products and technologies within Microsoft and by other software companies, big and small.


For this reason I'm happy to see that Windows 7 supports the VHD format natively. This means that users and administrators can easily create and attach virtual disks as if they were additional physical storage devices, without installing any third-party drivers or tools. You can, for example, use the Disk Management MMC (Microsoft Management Console) snap-in or the DISKPART command-line tool to create and attach virtual disks. You can then partition, format, and use them like any other hard disk on your computer.


As to running a Win Server 2008R2 Hyper-V hypervisor from a VHD (hopefully you tried already) on real hardware, were you able to run any VMs in that hypervisor? Generally, placing Hyper-V on a native boot VHD shouldn't affect Hyper-V ability to run VMs, despite it has strict hardware requirements, making it hardly possible to run inside another VM with a different virtual hardware. But running Hyper-V from a VHD may somewhat degrade its VMs performance, though insignificant for VHD One feature testing purposes.


Funny you should ask, I actually just tried to add the Hyper-V role to my 2008 R2 VHD, and it failed to install. But given the difficulty I had with SP1, and a random number of Windows Updates that have gone through, I hadn't thought to attribute that failure to anything sinister. Although it did reboot a rediculous number of times (six), so it's not impossible it's trying to do something tricky like change a boot parameter or alter whatever passes for the NT KERNEL these days.

I should note, that my test platform a SandyBridge i7 2600k, with 8 GB of Corsair Vengeance, and 2 x 3TB SATA 3 drives in a RAID stripe.

If you're interested, there is a rather boring Microsoft .DOCX full of graphs calls "WS08 R2 VHD PERFORMANCE WHITEPAPER" that compared the speed of running Hyper-V machines on VHD, vs other storage methods.

Edited by sfinktah, 25 November 2011 - 01:16 AM.


#91 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 01:23 AM

- fast shrinking of a dynamic VHD by directly editing its footer,


That just isn't going to be possible, since a dynamic VHD won't have any empty space at the end which can be just chopped off. Unless you are referring to the theoretical space of (for example, your extra blank partition)... which is only theoretical until you have to do a native boot - then I guess it becomes a rather big problem.

- adjusting on demand a VHD creator type to improve its compatibility in various VPCs.


Yuck, a terrible thought... but the I can provide easy access to the fields. As long as the operator knows what values to put in them :)

#92 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 01:27 AM

There are almost more fields about who created the image, than what the image is.

Posted Image

I was thinking this morning, "What would happen if you just reduced the size of the drive by changing the size field, so that the disk stopped before that hypothetical empty partition?" Surely nothing a quick CHKDSK couldn't fix :P

Edited by sfinktah, 25 November 2011 - 01:30 AM.


#93 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 25 November 2011 - 05:51 AM

The size of a fixed VHD can hardly be reduced that way, since first your app would need to check, if a volume placed on the VHD doesn't have any files stored at the end of it (i.e. the volume would first need to be defragged and shrinked, and the availability of free space at the end verified by a user or your app). Even then I doubt one can simply chop the file, even with running CHKDSK or using Repair VHD feature of VHD Tool, and updating its size and checksum fields in the footer, though it needs testing. :) But it can be copied instead to a smaller size file as VHD Resizer does, or better extended (and possibly shrinked using the same method) instantly with VHD Tool without using NTFS commands - it extends the original fixed VHD rather than creating its copy.

It should also be possible to change a dynamic VHD's max size value by directly accessing or replacing its footer. If new max size value entered by a user is bigger than actual current size, the max size can presumably be changed by simply correcting the value and checksum in the footer (and its copy in the header) instead of copying the VHD to a new file of different max size (research how VHD Tool extends a VHD instantly), because it doesn't change real file size. Not sure however, what that parameter is: Original Size or Disk Geometry. The VHD spec doesn't seem to clearly name it, but its entered by a user when the disk is created. Since raw2vhd and VHD Tool can append a VHD footer to IMG file, similar approach can be used to update & replace a dynamic VHD's footer and header.

Sorry to say that Hyper-V won't run on your processor, since it doesn't support VT-d and Trusted Execution (DEP), and neither in any popular VM, but will run VMs when started from a native boot VHD on supported HW. I suggest to install Win7 Virtual PC SP1 for your Win7 platform (i.e. 64-bit, it doesn't require VT & DEP) and do all testing in it or on real HW. The VHDs should also run in Virtual PC 2007 SP1 that has more HW controls, VirtualBox and VMWare - but OS settings on the VHD may require be generalized or made universal. Ones installed, Win7 VPC main controls are shown on the toolbar of Virtual Machines folder - a special Win Explorer folder that opens instead of a VM Console when you click on VPC shortcut. If you don't see specific controls on its toolbar ( Open VM, Create VM, Settings), click Reset Folders in Folder Options - View.

#94 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 01:46 PM

Posted Image


nt4.com/vhd.director.003.exe

nt4.com/vhd.director.003.src.rar




Posted Image


* the broken links have been fixed


Edited by sfinktah, 25 November 2011 - 02:44 PM.


#95 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 02:38 PM

The size of a fixed VHD can hardly be reduced that way, since first your app would need to check, if a volume placed on the VHD doesn't have any files stored at the end of it (i.e. the volume would first need to be defragged and shrinked, and the availability of free space at the end verified by a user or your app). Even then I doubt one can simply chop the file, even with running CHKDSK, and updating its size and checksum fields in the footer, though it needs testing. ../public/style_emoticons/default/smile.png But it can be copyed instead to a smaller size file as VHD Resizer does.


Of course you can just chop of the end of disk. Obviously not without some measurement and careful checking, but It's never going to notice until it tries to go there... which it never will. Try cutting the end of a DVD or something. :P

Anyway, the concept of defragging, then pre-compression (since the defrag will leave more mess than it cleared up), then the post compression, followed by copying the VHD.... well, it's pretty gay.

Here's a much simpler way to ensure that there is no wasted space on your new DVD.
  • Create Blank VHD. (Call it V:)
  • Attach Blank VHD
  • xcopy /s c:*.* v:
  • Detach Blank VHD
Don't jump all over me about xcopy... or point out the little technical reasons it wouldn't work.... just think of the general concept I'm conveying.

"Just take the files you want, and put them neatly in a row, on a new VHD."

I was talking about changing a dynamic VHD's max size value by directly accessing its footer. If new max size value entered by a user is bigger than actual current size, the max size can presumably be changed by simply correcting the value and checksum in the footer (and its copy in the header) instead of copying the VHD to a new file of different max size (test how VHD Tool expands it instantly). Not sure however, what that parameter is: Original Size or Disk Geometry. The VHD spec doesn't seem to clearly name it, but its entered by a user when the disk is created.


Yup, that's what I figured you meant. And there are 3 fields that deal with that information, although how many are actually used is unknown. There in the list I pasted, and you'll also see them in VHD Director 0.03.

"Original Size" (Quite literally how big the physical hard drive was)
"Current Size" (how big the drive is now"
"Disk Geometry" (stored as C/H/S)

After using dd to do enough raw copies of VMDKs to USB drives, you tend to get the impression that the only thing CHS layout is used for is to calculate how big the disk is... I haven't broken that particular field down in VHD DIRECTOR 0.03, it's still in hex. But it's 0xCCCCHHSS if you want to read it, and if you need to create a new one, there's code for it at the end of Microsofts documentation.

It's on my list.

Sorry to say that Hyper-V won't run on your processor, since it doesn't support VT-d and Trusted Execution (DEP), and neither in any popular VM, but will run VMs when started from a native boot VHD on supported HW. I suggest to install Win7 Virtual PC SP1 for your Win7 platform (i.e. 64-bit, it doesn't require VT & DEP) and do all testing in it or on real HW (the VHDs should also run in Virtual PC 2007 SP1 that has more HW controls, VirtualBox and VMWare - but OS settings on the VHD may require be generalized or made universal). Ones installed, Win7 VPC main controls are shown on the toolbar of Virtual Machines folder - a special Win Explorer folder that opens instead of a VM Console when you click on VPC shortcut. If you don't see specific controls on its toolbar ( Open VM, Create VM, Settings), click Reset Folders in Folder Options - View.


Son of a bitch. That's only like the newest, fastest, most expensive "end user" CPU intel are currently releasing.

Well, not to fear, my main workstation, which has the cheapest chip, from the first series of the i7 chip (we're up to the third now), does have VT-d, and that's where I run all my VMs. http://ark.intel.com...Ts-Intel-QPI%29

It's been retrofitted with Mac OS X, so I use Fusion to run my virtual machines. But not to fear, it can copy a 4.4GB VMDK between drives in 24 seconds, and has 24 GB on board, so it won't leave me lacking for virtualization capability.

Posted Image

I installed Virtual Server for W7 the other day, it ran ... passably. How much can you really expect from a hypervisor running in a virtualised server :) Fortunately I don't need to actually use it for virtualization, I just need it for testing. I was rather much wondering "where it was" after I installed it, but I found a VHD or a VHC file and worked it out.

Crap, even the Dell laptop I bought in 2006 had VT-d. Intel must be getting cheap.

I was running a VM on the 2600k with the latest vSphere (ESXi 5.0) and it didn't complain... I know it wouldn't be running "optimally", but I would have though Hyper-V would still run.

Edited by sfinktah, 25 November 2011 - 02:43 PM.


#96 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 02:45 PM

* the broken links to vhd director have been fixed now

#97 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 02:58 PM

fixed the dataoffset to show -1, added CHS decoding of Disk Geometry.

Posted Image

#98 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 25 November 2011 - 03:09 PM

Not sure what app you mean under "Virtual Server for W7"... But Hyper-V won't run either on the proc you linked abobe, since it doesn't support DEP.

Since xcopy doesn't seem to preserve symbolic links, the size of a copied volume will increase notable, and there may be other issues - its not recommended for bootable disk cloning. I suggest to study source code of VHD Tool, as it can extend (and possibly being enabled to shrink using the same method space permitting) instantly an original VHD without using NTFS commands to copy it. And it can Repair a VHD (presumably if broken after being shrinked or similar manipulation). I edited the above post.

Original size in the footer presumably indicates max size of a dynamic VHD entered by a user when the file is created.

#99 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 25 November 2011 - 03:30 PM

I believe I specifically asked you not to mention XCOPY. :P

Yes, obviously you need to use something that will make an identical *file for file* (junction for junction, metadata, acl, etc, etc) copy. I would suggest the WIM format, but recall reading someone suggesting that it didn't handle (something or other) correctly (although find this hard to believe).

My point is very simple. You will only ever quick-shrink a VHD to the end of it's allocated diskspace. And that include that giant chunk of MFT reserved space that covers 12% of your disk, which can appear in the middle of an 80 GB partition, even before you start putting data on the drive.

I'm not saying that the quick trim is useless - I got a few GB out of it the other day (I mounted the VHD and used DISKPART to shrink the partition) - but if you are willing to wait the time it takes for a defrag and double compress and VHD copy, there are better ways to do it.

I do realise that if you did it "by hand" you wouldn't have a hope in hell of getting the VHD to slot back in a boot... but we're talking about an application that is capable of manipulating the GUID of the VHD and even altering the BCD store if required.

BTW - I've never really given VHD Resizer the full test treatment... I'm assuming it's limited to shrinking the image only as far as the last data on the disk? I have a feeling they just use standard windows APIs to create, mount, and copy the disk. Am I wrong?

#100 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 25 November 2011 - 04:08 PM

Before using VHD Resizer or VHD Tool to shrink a VHD, one must manually shrink its volume in Diskpart or such (that may require defrag as well, so be it).

BUT, VHD Resizer then creates a copy of VHD file of smaller or larger size, while VHD Tool doesn't - instead it changes the file size of the original VHD instantly without making its copy (possibly by updating its MFT record). There is no need to use any other utilities including WIM in this case.





Also tagged with one or more of these keywords: vhd, bare metal, boot, win 7, grub4dos

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users