Jump to content











Photo

Boot Win 7 VHD on Bare Metal PC from Empty Drive

vhd bare metal boot win 7 grub4dos

  • Please log in to reply
321 replies to this topic

Poll: Are you interested in booting Windows from VHD? (84 member(s) have cast votes)

Would you try to install OS to and boot from a single portable VHD file (virtual disk) instead of hard drive?

  1. Yes (81 votes [96.43%])

    Percentage of vote: 96.43%

  2. No (3 votes [3.57%])

    Percentage of vote: 3.57%

Would you be interested to copy that VHD file to an empty USB Thumb or HD and boot from it on real PC?

  1. Yes (80 votes [97.56%])

    Percentage of vote: 97.56%

  2. No (2 votes [2.44%])

    Percentage of vote: 2.44%

Did you try to boot OS from VHD on real PC instead of Virtual Machine?

  1. Yes, I usually boot VHDs saved on an internal hard drive (34 votes [36.96%])

    Percentage of vote: 36.96%

  2. Yes, I usually boot VHDs saved on a USB drive or thumb (11 votes [11.96%])

    Percentage of vote: 11.96%

  3. Yes, I boot VHDs saved on drives of any type (13 votes [14.13%])

    Percentage of vote: 14.13%

  4. Not yet (34 votes [36.96%])

    Percentage of vote: 36.96%

Vote Guests cannot vote

#126 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 28 November 2011 - 05:09 PM

Thanks, interesting Wayback doc.

Another issue, whether a dynamic VHD needs contiguous free space equal to its max size to be available on underlying physical disk to boot OS from it, or it may be scattered around the HD? A VHD is only treated as virtual disk, when mounted in running OS. When OS boots from VHD, it sees it as a physical disk of max volume size. So the OS volume strives to occupy all the disk, unless its size is restricted by a user or lack of suitable space, and would require contiguous space on the drive, as any volume.

When a detached dynamic VHD file is extended (i.e. its max size changed), it doesn't matter since its physical size doesn't change, but needs to be taken into account by a user, whether such extension would allow to boot it from a target physical drive, contiguous space permitting or possibly defrag is required before placing the VHD on it.

Diskpart Ref: If an offset parameter is not specified, the partition is placed in the first unoccupied contiguous disk extent that is large enough. If a size parameter is not listed, the partition may be extended to occupy the determined disk extent up to the size of the entire disk.

#127 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 29 November 2011 - 09:37 PM

Posted Image

#128 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 29 November 2011 - 10:34 PM

Of course, its only possible if that NTFS partition and system volume on it inside the VHD weren't damaged by directly editing VHD file in 3rd party tools.


I found out one reason that would cause failure, in the previously mentioned case of moving partitions.

"The first four bytes are said [MBR] disk signature ... These are followed by eight more bytes ... [which are] the byte offset of this partition.... Starting with Windows Vista, [this] ... is stored in the Boot Configuration Data (BCD) store and the boot process depends on it [20]. If [this value] changes, cannot be found or has a conflict, Windows is unable to boot."


Move a partition, invalidate it's signature. :P

Bootmgr starts winload.exe (no need to mount the VHD, since its already mounted), WinVBlock is loaded and mounts the VHD

Wow. Sha0 keeps busy. Very tidy explanation. So (unrelated tangent of interest to me).... and possible next guide for you: "Boot Win 7 VHD on Bare Metal PC via iSCSI with No Drive"...

These are my thoughts - based on my flawed understandings and impressions - please correct as required -

If grub is mapping a VHD to be the boot drive, via int 13h handler, (which is what i assume it does), then there's there's only two things stopping you from moving the VHD to a remote server...

a) grub won't do a live --map to a network device, unless it's preloaded as a ramdisk. (at least that's the case for PXE)
B) WinVBlock would have to co-operate and have capabilities to match.

Since WinVBlock is based on an AoE, it probably has the code (or close enough) [sha0 willing] to handle it, but it would have to get a hand-over from grub. Which isn't going to happen....

... unless, (and this is very hacky)... grub loads 200-300mb of the VHD, and presents it.... the bootmgr is in the first 100mb, and if the kernel, winvblock, and the other required drivers were carefully placed at the very start of the main windows drive, it might just be able to bootstrap itself into real mode, where winvblock would switch it to the full VHD (accessed remotely)....

... meh, i must be daydreaming... just because i can't get iSCSI to boot I'm constructing rediculous plans...

I read your guides, I didn't say I didn't skip over the boring bits though! :)

#129 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 29 November 2011 - 10:49 PM

Just been reading your chats about MFT and NTFS... good to see, since I didn't have time to follow it up myself... but knew it could be important. With regards to you queries about mac's and NTFS and virtual pc and what not... I'm not anti-NTFS. Although to be objective, I would have to say it's certainly showing it's age compared to newer and better designed file systems (ext4 for linux, HFS+ in osx). But i've nothing against it.

With regards to that MFT area appearing after a defrag - already tried that. But what I haven't tried, is mounting (or perhaps actually copying to real disk) that VHD, and trying it on another system.

Since the VM I was running was originally (the same) VHD as distributed by Microsoft, it is not impossible that there have been changed made to keep the MFT in check.

What did your research end up showing?

If an offset parameter is not specified, the partition is placed in the first unoccupied contiguous disk extent that is large enough. If a size parameter is not listed, the partition may be extended to occupy the determined disk extent up to the size of the entire disk.



Well, if it's that stupid that it can't map a virtual drive "on the fly", you can bet it's definately too stupid to map over any used blocks on the disk. And these VHDs that Microsoft give out are 127GB dynamic... heavens knows why. Maybe they really don't want people using them as bootable VHD drives or something. I already turned mine into an OEM version, I think it will make a very nice VHD when I can shrink it :)

#130 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 29 November 2011 - 11:18 PM

and FTR, i still remember fixmbr.exe and fixboot.exe, and all the way back to when Windows "Server" (aka Windows NT 3.5) added Service Pack 3, which included amongst it's features: "THE INTERNET!"

Posted Image

They used to pay me $45/hour to administer a few of those ugly things. I think I still have my MSCP for IIS 4 around somewhere. How embarassing.

#131 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 30 November 2011 - 12:31 AM

Well, I guess Wonko can show a few interesting picks of his hardware too. :dubbio:.

It looks like Sha0 added a bus to his WinVBlock driver to hook Grub4DOS mapped drives - would it count towards your iSCSI goal? May be someone on this board can suggest whether Grub4DOS can be tricked to mount an iSCSI drive, see posts here and there? It should probably work with iSCSI Boot Floppy anyway...

Chenall seems to be one of more active current Grub4DOS maintainers, so you can ask him for more iSCSI support, if required.

#132 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 30 November 2011 - 04:20 PM

Since the VM I was running was originally (the same) VHD as distributed by Microsoft, it is not impossible that there have been changed made to keep the MFT in check. What did your research end up showing?...These VHDs that Microsoft give out are 127GB dynamic... heavens knows why. Maybe they really don't want people using them as bootable VHD drives or something. I already turned mine into an OEM version, I think it will make a very nice VHD when I can shrink it

That was the exact idea to have such a tool - and demand for its features will only grow with VHD proliferation, as Hyper-V will now be shipped with Win 8. :) Yet another required feature is intelligent "in-place" Conversion from Dynamic to Fixed VHD type rather then conversion via Copying the file as VHD Resizer does. Its often needed since Grub4DOS doesn't support mounting dynamic VHDs at this point, and its development is very slow. But it require data removal from dirty sectors adjacent to the VHD file to allow its contiguous grows.

As to MFT shrinking, didn't download the Explorer VHD you linked since I don't have any use for it, but Wonko Search Engine® shown up an interesting method to shrink, move and surround the MFT. :thumbsup: I think, its typical for Win7 MFT to have next to no reserved MFT zone right after install, and later too, especially if maintained often, like this 100Gb Win7 64-bit volume with plentiful apps, Pagefile anf Hiberfil.sys, having just 100Mb MFT with no reserve.

Posted Image

And here's MFT & Zone of just defraged 6 Gb Win7 Embedded VHD. MFT Zone size fluctuates depending on what OS version has currently mounted the volume: same 100Gb volume shows a big MFT Zone when mounted in WinXP.

img20.mediafire.com/34570a48cff868c63096a9248d84b9a63865caacdfd8290f0d50aae11b68fda45g.jpg

But again, it was chkdsk'd in Win Server 2008R2 (works better) and defragged in PerfectDisk at boot time defrag ( I think, Win Defrag can move MFT Zone in a running system now) not long ago. PerfectDisk has Prep for Shrink mode that moves various size files into MFT Zone to cleanup space, presumably using Win Defrag API. Don't forget, a VHD file itself must also be contiguous to be mounted by Grub4DOS.

Apparently, current MS Data Access and Storage libraries, including Defrag API, give plenty of juice to work on the project.

#133 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 03 December 2011 - 02:50 AM

intelligent "in-place" Conversion from Dynamic to Fixed VHD type rather then conversion via Copying the file as VHD Resizer does. Its often needed since Grub4DOS doesn't support mounting dynamic VHDs at this point, and its development is very slow. But it require data removal from dirty sectors adjacent to the VHD file to allow its contiguous grows.


Dynamic VHD is a curious file format. It's theoretically possible for there to be any amount of unreferenced blocks... for example, I can write in a text file, "Hello Wonko" and it might store it in a new block and then make an entry in the block allocation table to say "This 2MB of disk lives at location #1".

If I then change the text file to say "Hello sambul61", it could write another copy of the same 2MB at the end of the file, and then update the block allocation table to say "This 2MB of disk lives at location #2".

The original 2MB containing "Hello Wonko" would become orphaned, and just sit there taking up space.

Of course, there's no reason that it would ever happen... until Microsoft or someone discovers it's faster to keep the drives heads positioned at the end of the the disk, caching the BAT, and cleaning up later. Which would certainly be true in high usage scenarios. But that's outside our scope.

I only mention that, because I mistakenly thought that it would be possible to convert a fixed VHD to a dynamic VHD that way to save time. But every 2MB block has to have a 512 byte sector map at the start, so it can't be done.

As for converting fixed to dynamic "in place", it is almost impossible. You have to copy to a second file. The reason is simple, a fixed VHD takes up more space than a dynamic one. So if (for example) you read the first 512 blocks (1 gigabyte) of a dynamic VHD, you'd probably end up writing more than 1 gigabyte of disk back out. That would erase the next block of data you needed to read.

That being said... if the VHD was sufficiently packed, so that there were absolutely NO contiguous 2MB free spaces, then it becomes plausible. Fixed VHDs are 512bytes smaller per 2MB block, and also don't have any headers at the start, which gives you 1024 bytes extra room, plus 512 bytes for every 2MB block.

It's certainly an interesting challenge... but don't think you'd see much of a speed increase. Probably be slower. The only reason it could possibly be a little faster, would be that the heads didn't have to move as far between source and destination. And if a conventional conversion used a big chunk of memory, it could read and write 1GB at a time, which means there would only be 80 moves of the head for a 40GB VHD. At a seek time of 5 - 12ms, that would be lucky to add up to 1 second total.

All this assumes the host drive is itself defragmented, and it assumes that the 2MB blocks of the dynamic VHD was allocated in sequence.

All in all, very messy. Unless you seriously did not have the disk space, it wouldn't be a desirable solution. Even if it were seriously well coded, it would probably resemble trying to defragment a disk that's 90% full. A *lot* of juggling and unnecessary hard drive activity.





Apparently, current MS Data Access and Storage libraries, including Defrag API, give plenty of juice to work on the project.


That is very good to know - I had 3 options I was considering:
  • Attempt to leverage a third party defrag tool via DLL trickery (O&O Defrag, or the one you mentioned - probaby has the same DLL)
  • Direct manipulation of the NTFS file system via third party modules (lacks proven reliability, but allows you to totally ignore any restrictions like "System Reserved")
  • Using an older defrag dll, since I noticed Paragon licensed one of Microsoft's (the one that used to show the pretty blocks), I figured they'd have left around an easy-to-use DLL.
But if I can do it via the official (and more importantly: up-to-date) defrag tools, that's a big bonus. I can also check out the apps you mentioned, and see what DLL calls they use to get the job done. I think there will be far better options to be found it what you have written. Although, O&O defrag does have a free 30 day trial - which is a nice way to use a commercial app with VHD Director without having to release it as warez.

I always install a copy after I finish setting up a VHD... since every new VHD is a new computer, and a new 30 days. :)

Oh, on the topic of making things quicker, when doing [whatever] to a VHD, you could potentially save time and (temporary) space by skipping over pagefile, hyberfil, and the DLLCACHE directory. I noticed that the VHD that microsoft distributed for XP didn't have anything in the DLLCACHE directory. I understand the OS automatically fills it with copies of the main DLLs automatically. Not sure if that applies outside of Windows XP though.

Using their example, I've equipped myself with an XP SP3 VMDK that zips to 349 MB. There's no way to boot XP from VHD, right? Unless Microsoft (or someone) make VHD drivers for XP, right? Unless WinVBlock could do it?

#134 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 03 December 2011 - 11:51 AM

Something else that would make for a good tool:


Create a Windows Server 2008 R2 Server Standard Core VHD, named foo.vhd
CSCRIPT WIM2VHD.WSF /WIM:X:sourcesinstall.wim /SKU:SERVERSTANDARDCORE /VHD:C:foo.vhd

Create a Windows Server 2008 R2 Server Standard VHD with an automatically generated name
CSCRIPT WIM2VHD.WSF /WIM:X:sourcesinstall.wim /SKU:SERVERSTANDARD

Create a Windows 7 Home Premium VHD with an automated setup answer file
CSCRIPT WIM2VHD.WSF /WIM:X:sourcesinstall.wim /SKU:HOMEPREMIUM /UNATTEND:C:stuffunattend.xml

Apply the first image in a custom WIM to a VHD named custom.vhd
CSCRIPT WIM2VHD.WSF /WIM:C:stuffcustom.wim /SKU:1 /VHD:C:custom.vhd

Create a Windows 7 Ultimate VHD, and copy a folder structure to the root of the drive
CSCRIPT WIM2VHD.WSF /WIM:X:sourcesinstall.wim /SKU:ULTIMATE /MERGEFOLDER:C:DeployVHDROOT

Create a Windows 7 Ultimate VHD without leaving a signature file on the disk
CSCRIPT WIM2VHD.WSF /WIM:X:sourcesinstall.wim /SKU:ULTIMATE /SIGNDISK:FALSE

Create a Windows 7 Ultimate VHD and display the output in verbose/trace mode
CSCRIPT WIM2VHD.WSF /WIM:X:sourcesinstall.wim /SKU:ULTIMATE /TRACE:TRUE

Copy all of the binaries that WIM2VHD.WSF requires to C:WIM2VHD, so that it can be easily moved from one machine to another
CSCRIPT WIM2VHD.WSF /WIM:X:sourcesinstall.wim /COPYLOCAL:C:WIM2VHD


Edited by sfinktah, 03 December 2011 - 11:52 AM.


#135 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 03 December 2011 - 01:49 PM

Yeah, this WIM2VHD tool may be quite handy for P2V migration, since it seems to combine ImageX, Sysprep and BCD Editor functionality when hooked with WAIK. :)

As to booting WinXP from VHD, its not much different from booting an unsupported for native boot Win7 version from VHD, and done via Grub4DOS, once you made a WinXP archive and restored it to an empty fixed VHD. One must install WinVBlock or FiraDisk driver to online WinXP before making its backup, or inject it with DISM to offline OS on an attached VHD. Grub4DOS Menu.lst section looks the same for both OSs. For WinXP boot its advisable to create an empty VHD in Virtual PC rather than Win7 Disk Management Console, as disk format approach is different in WinXP compare to Win7 & Vista, and some Grub4DOS & Virtual driver issues were reported when booting a WinXP VHD, formatted in Win7 DMC or with newer Diskpart.

There are ways and tools discussed on this forum to make such VHD universal and bootable with Grub4DOS on multiple PCs. :huh:

#136 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 01:34 PM

Its advisable to create an empty VHD in Virtual PC rather than Win7 Disk Management Console



Funny you should mention that.... I thought I would make a fresh Dynamic VHD to test VHD Director with, fill it up, then delete half the files, and go from there...

So I created a 1024 MB VHD, with a 512 MB NTFS partition all in diskmgmt.msc (aforementioned Disk Management Console, of the Windows 2008 R2 SP1 variety though), and then I try to copy 490 MB of files onto it, and it tells me I would need 8 MB more disk space. WTF.... Look at disk properties, see I only have 481 MB.

Scratch my head....

Back to diskmgmt.msc, reformat the drive again, and then look at properties:

Posted Image

"Mother-fracker!" I say, and I remember someone saying that the MFT depends on where the volume is mounted... so I make a new partition on the unused half of the VHD, and in the format options, and it asks me to pick a drive letter, or path to mount it on, I choose "Do not mount this mother-fracking disk anywhere you bastard." Or something like that.

Look at the properties, 30 MB missing again.

"Mother-fracker!" I repeat, kicking my Bill Gate effigy. That's 5.9% of my disk. (Interestingly, exactly half the 12% "minimum MFT reserved space" I recall reading on M$).

How big is the VHD you ask? 28 MB. What's in the VHD you ask? Well 66% of it contains "00 00 00 00 00 00 00 00 00 00 00 00 00", and most of the rest contains:


0015640: 200b 210b 220b 230b 240b 250b 260b 270b   .!.".#.$.%.&.'.

0015650: 280b 290b 2a0b 2b0b 2c0b 2d0b 2e0b 2f0b  (.).*.+.,.-.../.

0015660: 300b 310b 320b 330b 340b 350b 360b 370b  0.1.2.3.4.5.6.7.

0015670: 380b 390b 3a0b 3b0b 3c0b 3d0b 3e0b 3f0b  8.9.:.;.<.=.>.?.

0015680: 400b 410b 420b 430b 440b 450b 460b 470b  @.A.B.C.D.E.F.G.

0015690: 480b 490b 4a0b 4b0b 4c0b 4d0b 4e0b 4f0b  H.I.J.K.L.M.N.O.

00156a0: 500b 510b 520b 530b 540b 550b 560b 570b  P.Q.R.S.T.U.V.W.

00156b0: 580b 590b 5a0b 5b0b 5c0b 5d0b 5e0b 5f0b  X.Y.Z.[..].^._.

00156c0: 600b 610b 620b 630b 640b 650b 660b 670b  `.a.b.c.d.e.f.g.

00156d0: 680b 690b 6a0b 6b0b 6c0b 6d0b 6e0b 6f0b  h.i.j.k.l.m.n.o.

00156e0: 700b 710b 720b 730b 740b 750b 760b 770b  p.q.r.s.t.u.v.w.

00156f0: 780b 790b 7a0b 7b0b 7c0b 7d0b 7e0b 7f0b  x.y.z.{.|.}.~...

0015700: 800b 810b 820b 830b 840b 850b 860b 870b  ................

0015710: 880b 890b 8a0b 8b0b 8c0b 8d0b 8e0b 8f0b  ................

0015720: 900b 910b 920b 930b 940b 950b 960b 970b  ................

0015730: 980b 990b 9a0b 9b0b 9c0b 9d0b 9e0b 9f0b  ................

0015740: a00b a10b a20b a30b a40b a50b a60b a70b  ................

0015750: a80b a90b aa0b ab0b ac0b ad0b ae0b af0b  ................

0015760: b00b b10b b20b b30b b40b b50b b60b b70b  ................

0015770: b80b b90b ba0b bb0b bc0b bd0b be0b bf0b  ................

0015780: c00b c10b c20b c30b c40b c50b c60b c70b  ................

0015790: c80b c90b ca0b cb0b cc0b cd0b ce0b cf0b  ................

00157a0: d00b d10b d20b d30b d40b d50b d60b d70b  ................

00157b0: d80b d90b da0b db0b dc0b dd0b de0b df0b  ................

00157c0: e00b e10b e20b e30b e40b e50b e60b e70b  ................

00157d0: e80b e90b ea0b eb0b ec0b ed0b ee0b ef0b  ................

00157e0: f00b f10b f20b f30b f40b f50b f60b f70b  ................

00157f0: f80b f90b fa0b fb0b fc0b fd0b fe0b ff0b  ................


Edited by sfinktah, 04 December 2011 - 01:37 PM.


#137 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 04 December 2011 - 01:48 PM

Wouldn't you expect NTFS own files to occupy some space on its Volume? Which filesystem doesn't require any space at all for its own code on a hard drive? :)

After all, considering a filesystem be a database of file objects, with each having numerous descriptors - logical to assume they should be kept somewhere, preferably in continuous space for faster access, hence such storage space better be reserved beforehand - right? Especially keeping in mind, small files are entirely stored inside MFT. :dubbio: And its only a part of NTFS own code...

#138 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 01:57 PM


Microsoft Windows [Version 6.1.7601]

Copyright (c) 2009 Microsoft Corporation.  All rights reserved.



C:UsersAdministrator>format E: /q /fs:ntfs

The type of the file system is FAT.

The new file system is NTFS.

Enter current volume label for drive E: DosFormat



WARNING, ALL DATA ON NON-REMOVABLE DISK

DRIVE E: WILL BE LOST!

Proceed with Format (Y/N)? y

QuickFormatting 511M

Volume label (32 characters, ENTER for none)? DosFormat

Creating file system structures.

Format complete.

	 511.8 MB total disk space.

	 507.1 MB are available.



C:UsersAdministrator>mkdir e:temp



C:UsersAdministrator>dir e:

 Volume in drive E is DosFormat

 Volume Serial Number is 78E2-896B



 Directory of E:



12/05/2011  12:52 AM    <DIR>		  temp

			   0 File(s)			  0 bytes

			   1 Dir(s)	 504,930,304 bytes free



C:UsersAdministrator>


Son of a mother-fracker.

Posted Image


Now I'm pissed of. MFT is going down like yo' momma.

#139 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 02:03 PM

Okay, maybe I over-reacted .. but it's that pie charts fault.

I did the same test on a 127 GB dynamic VHD to see if it was proportional.

Posted Image

#140 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 04 December 2011 - 02:21 PM

So, its a bare minimum, required to Boot Win 7 VHD on Bare Metal PC from Empty Drive. :devil:

#141 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 02:23 PM

But I can over-react to this... I took the 64 GB "Windows XP IE 6 test platform" VHD, and reformatted it. And it uses the same 98 MB of space.

But I can't shrink it below 3 GB. Trying defrag now, to see if will work better on a non-system drive.

Edited by sfinktah, 04 December 2011 - 02:31 PM.


#142 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 02:24 PM

Nope... it only allowed me an extra 2 MB.

Posted Image

Time to see where that space lives and how to delete it.

Hmm... this is interesting, according to the Event Log, it's actually Defrag that does the shrinking. And it says:


A volume shrink analysis was initiated on volume New Volume (G:). This event log entry details information about the last unmovable file that could limit the maximum number of reclaimable bytes.



Diagnostic details:

- The last unmovable file appears to be: $BitMap::$DATA

- The last cluster of the file is: 0xbfe16

- Shrink potential target (LCN address): 0x55d2

- The NTFS file flags are: -S--D

- Shrink phase: <analysis>



To find more details about this file please use the "fsutil volume querycluster ?Volume{4e7058cc-199b-11e1-82d5-000c29ff3dac} 0xbfe16" command.


Those "more details" turned out to be:


G:>fsutil volume querycluster ?Volume{4e7058cc-199b-11e1-82d5-000c29ff3dac} 0xbfe16

Cluster 0x00000000000bfe16 used by -S--D $BitMap::$DATA


Still, did learn something...


C:UsersAdministrator>fsutil behavior query mftzone

MftZone = 0


That means (in theory) you could do:

C:UsersAdministrator>fsutil behavior set mftzone 0


Edited by sfinktah, 04 December 2011 - 02:34 PM.


#143 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 04 December 2011 - 02:51 PM

No, it means size parameter value=0 for MFT zone in Registry - meaning default MFT Zone size controlled by OS. You don't seem to read other posts apart from your own... :)

However, a simple solution to defrag and relocate MFT in a running system (or an offline OS volume on an attached VHD) exists: Windows 7 Defrag API. See also How do you defragment MFT on NTFS disk?

#144 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 05:25 PM

This is a breakdown of what the 30 MB was on the 512 MB partition, as well as the MFT Reserved (I'm not sure if they shows up in the used space or not). It certainly looks like a lot more than 6%.

Posted Image


This is the 127 GB dynamic:

Posted Image

And this is the same, after Disk Management has shrunk it to it's minimum size:

Posted Image

#145 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 05:31 PM

No, it means size parameter value=0 for MFT zone in Registry - meaning default MFT Zone size controlled by OS. You don't seem to read other posts apart from your own... :)

However, a simple solution to defrag and relocate MFT in a running system (or an offline OS volume on an attached VHD) exists: Windows 7 Defrag API. See also How do you defragment MFT on NTFS disk?


Hahaha, I do read them, I do! But I admit I only skimmed the MFT stuff because it looked very involved... and I knew you'd be there to help me when I got to it, with all the answers. :)

I definately noted your previous reference to the Windows 7 Defrag API, but I mentally devalued it when Defrag itself didn't fix the problem. But I will look at it now. I only just got the cluster maps done then, so until now I couldn't see the full extent of the issue. It's partially the MFT, and partially the MFT Reserved.

The difference between them being (I assume) that the areas of dark grey are all used by files (eg: $whatever.secret.crapMFTcrappy.log) and the pinky ones are just *reserved* ... with no actual files linked. I'll have refresh my memory as to what they're for. Or do I have to read posts... *sigh*

#146 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 04 December 2011 - 05:48 PM

Disk Zones color depends on a defrag app used. :) Disk Management uses a default defrag scenario for a presented disk structure after calculating its "dirty & clean" sector location stats (whatever scenario is defined in it). YOU can use Defrag API for more specific tasks instead.

More useful links:

Tools and APIs for Virtual Hard Disks
Virtual Disk Service (VDS) Protocol - might be useful to boot a VHD via iSCSI
SET_VIRTUAL_DISK_INFO - is useful to repair relationship link between a Differencing and Base VHD in VHD header, when broken due to the Dif or Base file moved to a different location

A MS VHD developer overview:

The Virtual Disk API In Windows 7

#147 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 05:54 PM

With regard to MyDefrag, I tried the Monthly System, and the example script "ForceTogether", neither of which did anything.

Has this program worked for you, and if so, what environment were you running it in?

#148 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 05:56 PM

Disk Zones color depends on a defrag app used. ../public/style_emoticons/default/smile.png


That why I included the legend, you should read my posts. :P

I'll investigate your links, although I'm loosing faith.

#149 sfinktah

sfinktah

    Frequent Member

  • Advanced user
  • 217 posts
  • Location:Der Äther
  • Interests:/(C(++|#)|P(HP|XE)|(OS|Linu)X|8051)/
  •  
    Australia

Posted 04 December 2011 - 06:03 PM

Tools and APIs for Virtual Hard Disks

Already read it, and it's two links.

Virtual Disk Service (VDS) Protocol - might be useful to boot a VHD via iSCSI

Requires Hyper-V to be installed.

SET_VIRTUAL_DISK_INFO - is useful to repair relationship link between a Differencing and Base VHD when broken due to the Dif or Base file moved to a different location

That part I have covered. :)

The Virtual Disk API In Windows 7

Already read it, good article, but it stops just short of the good stuff.


Thanks for trying though :)

BTW, VHD director can actually create VHDs already. It's this damnable shrinking that is causing grief.

#150 sambul61

sambul61

    Gold Member

  • Advanced user
  • 1568 posts
  •  
    American Samoa

Posted 04 December 2011 - 06:05 PM

although I'm loosing faith.

They always say it, when a task becomes more complex & unbearable to handle. :dubbio:

BUT: basic VHD tools already exist, no need for a repeat effort, unless its more intelligent & feature rich. :book:



Hence your TOOL would be expected to offer In-place VHD Upgrade and other advanced functionality:

- fast resize a fixed VHD (the VHD Tool approach needs more intelligence, i.e. without damaging its own volume data or HD volume data where the VHD is stored - may include defrag in and outside of the VHD, still much faster than copying a large VHD with OS on it, not an empty one)
- fast convert a fixed to dynamic VHD and vice versa
- fast resize a dynamic VHD max set size (shrinking may include its resident volume defrag, while increasing isn't a problem, since it will stop growing at first dirty HD sector anyway, when adding more data to it)
- verify and fix lost links in a VHD chain
- fix broken BCD entries resulting from a possible volume shift when converting a fixed VHD to dynamic and vice versa
- fix unmatched disk signature issues in BCD preventing OS booting from a VHD
- etc...

Since Hyper-V is included with Win8, MS may have lifted (or will lift) the processor restrictions, they imposed on production servers, similar to what they did with Windows VPC upon users demand. Hence, you can try working now with Win8 Developer Preview & Hyper-V to look at its VDS support for iSCSI (if functional enough at this stage), and see what happen.





Also tagged with one or more of these keywords: vhd, bare metal, boot, win 7, grub4dos

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users