System & Differencing VHDs synch and update strategy
Posted 10 September 2011 - 07:12 PM
The problem is how to sync Windows Updates and extra accessory soft that would inevitably need to be installed afterwards on each VHD or partition. Ideally, a different approach would be better suited, if all apps should be accessible from the same PC, and using VMs is not acceptable due to slow performance and lack of hardware support required by the apps. Namely, what if a Basic VHD or partition is prepared and than continues to be amended and updated with extra accessory soft and Win Updates. At the same time several Differencing (or Incremental) VHDs are prepared that all refer to the same Basic VHD, but different major apps are installed on each. Would this concept fly by allowing easy update of the Basic VHD, assuming the Diff VHDs are made persistent (don't disappear after dismount)?
Or, anyone is able to suggest another update strategy for all VHDs that would not require duplicate Win Update downloads and installs after booting to each VHD, while not affecting different major apps installed on each?
Posted 10 September 2011 - 09:31 PM
I'm not sure if some of the newest virtualization deployment tactics would let you basically share a main disk, and apply differences on top of it - sort of like the layered file system approach in Linux.
So, in the tradition of Wonko, what is the real issue/problem you are looking to solve, not how are we trying to solve it (with VHD's)...For example, is this just to isolate two (or more) programs that don't play nicely together? Or is it something else.
For example, if they each were in their own MINI-VM, then unless there was a bug that affected the application, you wouldn't have to update/patch it's VM...Afterall, most MS patchs are core OS security or specific in nature, and not really "required" for normal daily use (if we weren't worried about Malware, worms, virus, etc.)...
Anyway, more background might help us all here
Posted 10 September 2011 - 09:55 PM
And again, I consider Win Update important enough feature to avoid simply dropping it entirely. So you say, one can't have several differencing disks to a common base? Its kind of strange, if you think about multiple possible purposes of an incremental volume. Will have to post it to MS guys. And may be to VMWare guys - they are used to similar questions anyway. Can you suggest a different solution?
Posted 10 September 2011 - 11:57 PM
Not off the top of my head, and having come from the world of CAD (EDA) for over 25 yeras, I totally understand the issue. Of course most of our stuff was Linix/Unix and only recently at the "lower" end did we get into Windows/PC based software. Maybe there is some way to leverage file de-duplication, but with windows, there is so much in the registry and the like that needs to be kept synchronized with the "core" that I'm not sure how easy it could be...
Can you suggest a different solution?
But an interesting issue, especially for larger enterprises that need to leverage multiple environments on a single piece of HW...
Posted 11 September 2011 - 12:06 AM
" On volume V: we would like to create a read-only physically bootable Windows 7 parent virtual image, and one ore more differencing images based on that parent-image. The benefit of such scenario is obvious; we create the parent image including all commonly used stabilized components and using differencing images e.g. for beta products....The perfect scenario is rather to create a differencing vhd image and installing Beta bits on that differencing vhd. The parent vhd image is set to read-only and is not impacted by betas at all".
So it appears that at solution conceptualization stage it seems like some kind of approach worse to try experimenting with.
Posted 11 September 2011 - 12:25 AM
Also VBoot, support very well differencing VHD comparatively to MS VHD diff who lack some feature and some/all windows distribution, no VHD boot for Home user
one time by mistake or tentation I have boot the main VHD of alot of Diff one. On MS VHD no chance to boot a diff VHD anymore BSOD.
but with VBoot they forgot the BSOD part. But I have not tested a lot of sector change of my main VHD for then test the snapshot.
But with MS VHD no chance and VBoot no problemo.
Look like the MS VHD driver is very fragile and storage consommer mostly when it come to SSD and snapshot.
BTW, vboot will release the v2 of the vhd boot driver in 5 days and now with a lot of feature... ram caching file caching ram boot etc..
with desktop that can now upgrade up to 48Gb ram when 8Gb release. I prefert buy memory then storage like SSD.
and I almost forgot the best VHD diff is who have a parent at storage on SSD and it seft diff on ... ("ram") or pernament HD. ( remember what kill SSD ? writes, thats why TRIM exist, hopefully, but dont have to if just use SSD for fast ramdom read, then RAID0 of cheap SSDs would be very good, (but raid0 blow alway TRIM fonction).
and for conclude... why only one snapshot? limitation on MS driver... they dont imagine that home user could use a fonction that can save them lot of time, money and fustration of waiting a recovery !"*($Y!*&$! . so MS let home user play in a virus environement that they create expecialy for them and for there Geek Formater appliance shop
but thats why Win8 comming... tssss Vboot work also on XP, no need to buy new OS for compagny.
a snapshot of a snapshot why not ? testing program install, etc.... then merge the snapshots in one after aproval.
but certain people alway prefert a recovery waiting is important is money after all !
and sambul I dont understand your way of "colitioning" VHD appliance but I think I know what you search for.
and again is exactly what VBoot v.2 will do ... in the "possible" of thing is a file ram caching way and you have the choise of "aproval" change on disk after close the file program that were caching. ( is not clear for me too, after all Vboot V.2 is release only in about 1 week for now)
So think of a VHD booting on Ram only and commit live change to its VHD main/snapshot kick start.
this message was suposed to be very short at the beginning hihi
is all I have to said apart that I'm sorry for my bad english!
p.s. I not trying to sell you a driver, but test it when v.2 is available, I have test since v.1 and now v.2 bugfix all the hardware incompatibility and improve performance more !
and if your linux booting is perfect for VBoot who start any boot sector in any VHD with Grub2 ease interface, no need chainload Bootmgr from any grub/syslinux/boot manager also VHD chainloading is not possible anyway
Edited by Natrix, 11 September 2011 - 12:39 AM.
Posted 11 September 2011 - 12:33 AM
Speaking of differencing VHDs, I found this article Implementing a Windows 7 SteadyState by utilizing differencing VHD files and the “Boot from VHD” feature that seems to almost exactly address similar concept or task. I think, I'm going to try this approach.
Posted 11 September 2011 - 12:46 AM
Well, yes and no. I'm not sure how the whole issue of updates and patches comes into play here. Since *IF* the differencing images have all sorts of additions to the registry, I'm not sure you could easily take the base image and update it without having to totally recreate the new "differencing" images... Yes, if the foundation is solid and doesn't change, then this could possibly be a way to leverage things - i.e. instead of two or more different "beta" products, you had two or more different client programs that needed to be isolated.
So it appears that at solution conceptualization stage it seems like some kind of approach worse to try experimenting with.
But depending on if there is any overlap at all or dependency...i.e. A and C are in the base, and tightly coupled together. B depends on A and is in Add-on #1 C is in add-on #2 and is different that the C in the Base. If you update the base, and A and C both change, then #2 is totally broken until you also update it. And it may now need to have it's OWN A and C...
Take that out to the thousands of files and registry entries and dependencies in Windows and typical applications, and you have a big mess...This was something I struggled with for MANY years as we tried to deliver working suites of tools to customers and also manage patches and bug fixes.
Here is a simple picture...
New Bitmap Image.bmp 594.19KB 210 downloads
If "product" A is the circle of 1,4,5,7 and B is the circle of 2,5,6,7 and C is the circle of 3,4,6,7 and for laughs, 6 and 7 are in the same "dll", then you can see that updates to the "base" could have an impact on the whole continuing to work correctly...So, the only answer is to update (or re-place the creation of the "differencing" disk.
Maybe there are enough parts of the base that could be shared in the initial creation of the environments that it is a "win" on diskspace...BUT, I don't see any way out of the individually applying the patches to each environment - and then, at some point in the future, recreate a new base and a new set of differencing disks.
BUT, as I siad, a worthy thought process and if there is a way to make it work - GREAT!
Posted 11 September 2011 - 12:53 AM
But I was almost certain that just changing the BCD boot manager option of the location of the OS (NO VHD HERE) the os file like C: for BOOT in EasyBCD solve my problems... here in a VBoot way!
but for MS VHD boot BCD option are taken before VHD boot, and VBoot VHD boot .. after the vhd boot then it start the BCD boot manager
And for the old myth of never copy win OS from a PC or HD to another PC or HD on same computer because of certain parameter like disk signature I others things that I have forgot ... is no longer true with new OS.. I dont know for XP but 7 I never had problem.
maybe you can elaborated for my poor comprehension of the scare of the "Disk Signature collision" ( dont forgot that I use VBoot )
VBoot 2.0 supose have a 14 day trial full feature. but only few will be necesary and not for PC crash but pay cash!
p.s omg lOL Implementing a Windows 7 SteadyState is so ridicule ! MS could have write special program for do what a WinPE could do... but WinPE could also take lot of time booting in. WinPE 3.0 take long... and steadystate is something I already write myseft before they came with the idea.. and I'm not programmer.
VBoot have these feature but with a blink of a eye at the Grub2 Boot Manager just press I for immuable press S for doing another snapshot even is already one press R for restoring to..
OMG SteadyState suck and is time consuming for testing environnement. boot winpe for doing one opération is completely ridiculus if you know VBoot and know that time is money.
Edited by Natrix, 11 September 2011 - 01:08 AM.
Posted 11 September 2011 - 01:04 AM
Yes, you have a huge point. I don't think, MS guys thought it through that far. They address so far "read only" concept of a base disk. Despite one approach may be that at the time base disk is updated, these changes propagate to all differencing disks if required. On the other hand, each differencing disk should be self-sufficient, i.e. probably having a copy of Base Disk Registry and critical system files, and all changes to them on that disk. Once Base Disk is updated, a new copy of its Registry can presumingly propagate to a differencing disk. How it will affect changes made on it by the earlier installed app - who knows... How updating later that app will affect local Registry copy match with Base Disk - its another question.
It appears, Base VHD can not be updated in principle in MS concept or even booted to after differencing VHDs are created. All Windows Updates are to be applied only to differencing disks. This in turn partially negates the whole purpose of this solution, unless differencing disks updates are done via a local server, and possibly by a method other than traditional patch install.
MSDN VHD FAQ: "You should not modify the parent of a differencing VHD. If the parent VHD is changed or replaced by a different VHD (even if it has the same file name), the block structure between the parent and differencing VHD will no longer match and the differencing VHD will be corrupted.
You must keep both files (the parent VHD and the differencing VHD) in the same directory on a local volume for native-boot scenarios. For native-boot VHDs, the parent VHD and the differencing disk cannot reside on different volumes, even if they reside on the same local disk. However, when you attach a differencing VHD that is not used for native boot (for example, if you plan to use it for image management), the parent VHD can be in different directories, and on a different volume or even on a remote share."
As to Windows Update, it does work on a single PC without adversely affecting earlier installed apps on that system volume. Probably, because Windows now keeps multiple copies of each system dll, if different dll versions were installed by the apps. Anyway, did you notice, many outside folks talk about differencing VHDs from practical application standpoint, but not Microsoft. This is a sure sign of under-developed area. I wonder, what Ulli has to say about it. Updating VMware and VM OS installs should also affect differencing VMDKs.
14-day trial is too short for a boot app that requires a lot of new concepts thinking over from a novice or enterprise user - I say novice since experienced Grub4DOS users won't pay $80 for Vboot. As to collision avoidance, its not old staff. Just recently I was unable to boot Win7 after Restoring a Win7 Backup archive to another drive on the same PC with Acronis. I had to use Paragon Fix Boot Disk utility to fix Sig and Reg issues on the Restored volume. Also, besides booting from VHDs, they can be mounted on the same PC for data exchange. In that scenario collision avoidance is major issue as well, as after dismount they should remain bootable on the same PC, hence their Sigs should not change.. Of course, a known solution is Generalize (delete MountedDevices subkey) manually or with MS Sysprep before cloning, or for using the Clone on the same PC - just edit BCD Store of the cloned volume.
Posted 11 September 2011 - 01:32 AM
Use GImagex instead of Acronis is more commont lol
Sorry but you completely lost me when you said : Just recently I was unable to boot Win7 after Restoring a Win7 Backup archive to another drive on the same PC with Acronis.
So you a combining two almost crasp software Acronis..... and Win 7 Backup damn plz try someting else anyting ...is more flexyble I'm not saying is VBoot I telling you stop using "fake use tool software"
Paragon Tool is better so. but plz Acronis and Win7 Backup... damn I dont understand what you do on reboot.pro forum..... freedom you mind of your problem of the past and go find your future tool.... maybe you could start another topic forum for...
Edited by Natrix, 11 September 2011 - 01:34 AM.
Posted 11 September 2011 - 04:20 AM
You should not interfere with off topic staff and aggravate others, it violates forum rules, you can be banned if continue. This forum is not Vboot advertising platform, and you can't hijack others' threads just because no-one reads here about Vboot, since Grub4DOS and Burg are free. What exactly are you doing here?
Posted 11 September 2011 - 09:54 AM
Windows 7 can be installed allright in a Logical Volume inside extended (just like *any* NT family OS).
I see no actual problem in having a disk with as many logical volumes in Extended partition as you wish.
And I see no actual difference (in the sense of particularly complex procedures in managing a number of .vhd images or managing a number of logical volumes).
To all effects having several Windows 7 installs on several .vhd files or having them on several logical volumes does not really make a practical difference.
The problem of keeping them all updated to the same SP release/update/hotfix etc. seems to me exactly the same.
Basically you have several different Windows 7 installs (on the same machine) and it is not unlike having several Windows 7 installs on a set of identical machines.
This is a typical enterprise setting, where instead all clients have usually the same software.
I presume that the easiest could be using what is normally used in such a scenario, like WSUS or a similar solution. (this will eliminate the duplicate downloads, but of course each logical volume or vhd will need to connect to the WSUS server and install the updates).
Posted 11 September 2011 - 11:47 AM
Thanks for the WSUS hint! When starting this thread, it was unclear for me, whether VHDs can better serve the task. Now I see, the VHD concept needs further work from MS, since it doesn't fully address inspirations of multiple user groups as some web buzz reflects. Basically, they primarily developed the incremental feature for testing own staff. Let's try to discuss these avenues with MS guys. At times they deny or seemingly ignore even directly addressed feedback, only to resolve the issue in the next release. At times however it crosses their bosses "vision", and the issue is never addressed even if delivered to the right people. Will see...
I'd love to exploit VHD approach a bit more, as all current trends in business environment are towards virtualization, and native VHD boot delivers ideal solution in terms of very low productivity hit and overhead. However, Win and generic programs update on a base VHD volume, and then changed disk structure syncing with child volumes need to be addressed to show definitive advantages of VHD approach compare to multiple Win installs in logical partitions in described scenario.
Posted 11 September 2011 - 03:15 PM
I forget to mention however, Base & Differencing VHD concept has significant advantages compare to Multiple Logical Partitions approach. Namely, one needs to install OS and most generic apps only once on a Base VHD. And only potentially conflicting with each other apps are installed on several Differencing VHDs linked to the same Base. As a result, you need only single License for each app, no licensing issues ever, and required HD space to store all VHDs is drastically reduced compare to holding several logical partitions with OS and all generic apps installed on each.
So, the concept of VHDs is greatly beneficial compare to physical drives in such scenario. Except, it presumes, all future OS and generic & special apps updates will be done on each Differencing VHD, while Base VHD will remain Read Only. In that sense it offers no advantages compare to logical partitions. However, I failed to see any serious drawbacks either, so overall VHD approach seems to make a lot more sense.
Posted 11 September 2011 - 03:43 PM
How big needs to be a volume containing the OS + a given set of apps? (but NOT any data)
Say 30 Gb.
You can fit 15 of such volumes in a common US$ 40 500 Gb disk or 30 on a US$ 60 1Tb one.
Let's wait when (mind you I sincerely hope it won't happen and that you have anyway a solid backup/imaging procedure) one of the base or diifferencing .vhd - for -any reason - becomes corrupt.
If you read this:
that maybe will also help you understand the other issue you are having with .vhd booting and particularly this one:
particularly this part:
then go back to here:
and read the implications and caveats of using differencing .vhd's
particularly the part:
Store all critical data outside the native-boot VHDs. When you store critical data outside the VHD that contains the Windows boot image, it is easier to recover the data if the VHD becomes unusable.
Use fixed VHDs for production environments. You can use all three VHD file types (fixed, dynamically expanding, and differencing) to native boot, but we recommend using fixed VHDs for production and using dynamically expanding or differencing VHDs for development and test environments.
Remember that this is written by the same guys who created:
- native Windows 7 .vhd booting
- dynamic .vhd's
- differencing .vhd's
In theory they should know what they are talking about.
(they didn't create static .vhd, as they are simply a RAW image with a CONNECTIX footer)
You may understand how I would personally prefer volumes to .vhd's. (or use fixed .vhd's that are actually exactly like volumes/disks, and as such do not give any particular advantage in space occupied)
Posted 11 September 2011 - 07:14 PM
Posted 12 September 2011 - 02:37 PM
...Basically, they primarily developed the incremental feature for testing own staff...
VHDs were simply the HDD image format for Virtual PC. That is like VMDKs are the [primary] HDD image format for VMware VMs. Microsoft acquired Virtual PC. It seems to me that discussion about what you want to do with VHD images should not be confused with what they were designed for.
I also think it's important to understand block versus filesystem! VHDs are for disks (the "block" level). Concerns about DLL and OS component versions, Registry dependencies, program versions, etc., will obviously arise, but what do they have to do with VHDs? Even without VHDs, Microsoft has the Enhanced Write Filter, which is also at the "block" level; same concerns. Linux' device-mapper can provide the same functions and the same concerns.
Microsoft has the File-Based Write Filter which is at the "filesystem" level. They also have a Registry Filter. See here.
If I wanted to use differencing VHDs for booting Windows, this is how I might do it:
- Basic Windows installation VHD
- All applications and customizations VHD, which would be a delta from #1
- Model-specific VHD, which would be a delta from #2 with model-specific drivers, etc.
- Client-specific VHD, which would be a delta from #3 with a particular Computer Name and Active Directory identity
- Boot-specific VHD, which, if a "kiosk"-style environment was desired, would be discarded and rebuilt at each reboot, so viruses and user meddlings would not persist. It would be a fresh delta from #4
But those are for my needs, and obviously mightn't apply to your scenario(s).
Posted 12 September 2011 - 04:01 PM
And BTW, the good VMware guys have managed over the years more image types than you may want to know about, a good list is here as you should already know:
Also creating "internal" incompatibilities within their own products (see the .pln vs. .vmdk here):
And of course the ones with Redo-log or Snapshotting (corresponding to the "differencing" VHD's) suffer from the same kind of problems.
Posted 12 September 2011 - 05:34 PM
Its important to understand, what VHDs were designed for, and current targets (but not roadmaps) are presented in MS Devs blogs. In contrast with VM guys, MS delivered native boot VHDs that draw high interest from app usage standpoint due to low performance hit. BUT, MS guys didn't envision app usage as primary diff. VHD purpose, they claim its for testing rather then usage. Hence, no attention was paid to updating a Base VHD with Win Updates and adding new generic apps to it instead of changing Differencing ones, and propagating these changes in a non-destructive way to all Dif VHDs (syncing). That seems to be a weak spot so far.
I get your 1-5 structure idea, but which number is the Base here, when applied to a single PC? If only 1 is the Base, then all 1-5 must be present in the same folder on that PC - not sure how it will affect app performance? Any graphs, showing performance versus VHD chain length? I wonder, how internally this thing works - Registry on 5th would point to any of 1-5 depending on the request, and all 5 are mounted at the same time?
Yes, I know VM approach to this goes long way back. What draws attention to this VHD solution from many end users and infrastructure guys is NATIVE BOOT - no performance hit plus direct hardware support. Of course, enterprise server based VM solutions are popular, but they seems to work slow enough for some heavy apps, its mostly for office type apps., unless costly infrastructure is in place. But at a workstation level, native boot diff. VHD solution seems to be attractive enough to create plenty of buzz on the web.
Posted 12 September 2011 - 06:25 PM
Posted 15 September 2011 - 01:22 PM
Some VHD planning notes...
Create Base VHD as Expandable for smaller initial size - it won't affect its performance, since once all generic apps are installed on it, you'll lock it. Before you start creating Dif. VHDs, don't forget to deselect "Content Indexing" in Disk Properties, Defrag the Base, and assign its Paging file to a physical HD for faster performance and smaller Base size, make the VHD "Read Only". OS booted from VHDs don't support Hybernation and Hybrid Sleep features for now, so you can delete Hybernation file in the Base VHD. Differencing VHDs by default seems to have no Paging file, meaning paging is done via the Base VHD's Paging file, if its redirected by you to a HD volume. If you later unlock and change the Base VHD or ever boot from it, all its Differencing VHDs become corrupted, and would require be recreated with all their apps reinstalled. Hence, store all data used by VHDs on a physical partition. Updates of Windows and installed Apps must be repeated for now on each Dif. VHD, since the Base VHD must remain locked for as long as any of Dif. VHD files exist, even if they're not mounted.
If you want fast app performance, make Base VHD really basic, no extra apps except AV, and add Hyper-V role, then create a Dif VHD and install just one app and specific drivers on it you want to work with. On Win Server 2008R2 such setup will be really fast. Assuming, you have enough RAM and some other Windows images with all generic apps installed, start Hyper-V hosted or another VM inside the Dif. VHD, and run your preferred OS image inside it. This way you get access to 2 worlds at once: a) fast direct hardware access for your main app working inside very light clean system, AND c) access to all generic office apps you usually work with via a Hyper-V VM on the same PC.
Posted 21 February 2014 - 06:16 PM
This seems like a real bummer, I've got a base vhd I'd love to optimise, and run different 'profile'virtual machines as difference(d) disks, ie
(proposed) child 1: DAW
vsts and DAWs, midi drivers like rtp and coolsynth...
powerUser child 2
Office, photoshop, connectivity, media
testBed child 3
so, with each update, I have to install 3 times (or twice at least, or leave myself unupdated)
thats a shame