This is my attempt to make a serious "how to" contribution to the Windows 7 software development community, and, hopefully, make something that I learned the hard way simpler for someone else.
I have my Windows system on my work computer set up the way I want it, with all the utilities and programs that I use installed. But I wanted to run the new Beta of Visual Studio 2010. However, there's a standing (and understandable) rule: no beta software on computers that you're doing production work on. But my boss did give me permission to use it on the project I'm working on (since it won't be production ready until after the scheduled release of VS2010 in March) PROVIDED that I run it in a segregated environment such as a Virtual PC) so that it can't corrupt anything else.
I had run the beta of Windows 7 in a Virtual PC when it came out, but it was painfully slow. Windows 7 has the capability to boot up from a Virtual Hard Disk (VHD) as though it were actually a hard disk and run with minimal speed degradation (the disk access is somewhat slower, but everything else is running just as fast, because you are actually running on the hardware, not a program simulating the hardware). So I decided I would clone my existing system drive, which is configured the way I want it, and boot from that. Should be simple, right? WRONG! There are a number of hidden gotchas along the way, which this post is intended to walk you through.
However, a couple caveats before you start down this road with me: I run my systems a little different than it comes out of the box. I have my disk partitioned in 2, similar to the way I learned to do it under Unix: I have a system drive (C:), but all my user data is on a separate partition. I assign the drive letter U: (for Users) to this partition. There's a number of reasons I do this, but for the purposes of this discussion, it means my working data is available to me whether I'm booted into the real system drive or to the VHD drive. If you have your drive setup the way Windows does it out of the box, with everything on C:, and your user data in C:\Users\<username>, then it's going to be clumsy, at best, to access the files you write in one instance from the other instance.
The other caveat is that you need enough free space on your disk to hold the VHD in a fully expanded form. In practice, this probably means that if your disk is more than half full, this isn't going to work for you.
Okay, that being said, here's the steps I took (on about the 5th iteration of this process). (Also, be warned that a couple of these steps can take quite a bit of time - altogether, this process could take half a workday.) (Also, if you run as a non-privileged user, most of these steps need to be run as administrator.)
1. Clean your system disk. Get rid of all the junk on it, because you're going to be cloning the disk and you want it as clean as possible. I use a freeware program called "CCleaner" (http://www.ccleaner.com/) which empties the Recycle bin, clears your browser cache, gets rid of temporary files, etc. When you finish, look at the properties of the drive to determine how much space is actually being used and how much is free. This will be important later when we resize the cloned disk.
2. Defragment your disk. Even though Windows runs defrag periodically, and will probably tell you the disk doesn't need defragging, run it anyway, and specify the /X (consolidate free space) option. I open an Administrative cmd prompt window (type "cmd" into the Start menu search box, then right click the "cmd.exe" option that shows up, and select Run As Administrator), and type "defrag C: /U /V /X". The idea is that we want to end up with all, or as much as possible, of the free space at the end of the drive. (This process will take a fair amount of time - maybe 10-20 minutes if I remember right, depending on drive size and how fragmented it is.)
3. Download and run Disk2Vhd.exe (http://technet.microsoft.com/en-us/sysinternals/ee656415.aspx) from the SysInternals group at Microsoft. In the list of disk partitions it gives you, select just your system partion (C:). This will clone your system drive into a virtual hard disk file. It's okay to put it on the same drive you're cloning. This process will also take quite a while.
4. Once that is done, use Windows' Disk Manager to Attach the VHD as another hard drive. Right click My Computer, and select Manage. Then click Disk Management. When the storage manager comes up , Right click Disk Management, and select "Attach VHD". Find the VHD file you created in the previous step, and attach it. It will show it as an additional hard disk, but it will tell you it's offline, and help will say it has a serial number conflict with an existing drive. Well, since we cloned it from the C: drive, it has the same serial number, so that makes sense. The good news is that it's easy to fix. If you right click the newly added Disk symbol (probably Disk 1), and select "Online", it will cure the problem and put the VHD online.
5. If your drive had multiple partitions, you will see that the mounted VHD has space corresponding to the space those partitions took up. The space will be shown as "RAW" partions. We need to delete those. Right click each of them (except the one corresponding to your cloned system partition), and select "Delete Volume". The space will now be shown as "Unallocated", and adjacent Unallocated partitions, if they exist, will be merged.
6. Now right click the partition corresponding to your cloned system drive, and select "Shrink Volume". It will spend a couple minutes analyzing the partition to see how much it can shrink it, then will give you a dialog box showing you how much space is now being taken up (the full size of your original partition), and how much is available to shrink. There will be a textbox/spinner to let you select how much you actually want to shrink it, and then shows the resulting final size after the designated shrink is applied.
There's a trade off here. The size you shrink it to becomes, in effect, the "physical size" of the new VHD, and you want to give yourself some room for expansion. On the other hand, when you're booted to the VHD, the system will expand it on disk to the full "physical size" specified, and you have to have room on the disk for that expansion, or the boot will fail. So, for example, if I have a 100G C: drive, with 30G used, and I clone it, the clone file may actually only take up, say, 25G. But if the "physical size" that I've specified for the VHD is 50G, when I boot from it, the OS will expand that 25G to the full 50G, which still leaves 100-(50+30) = 20G free. But if I now copy 25G of video onto that physical disk, then the new free space is 20G (100G - (30G+20G+25G)), but when I boot into the VHD, the OS will try to expand it and run out of room, and the boot will fail. So, make the size big enough to accommodate reasonable expansion, but balance that against the free space on the disk.
7. Once you've finished the shrink operation, right click the Disk in Disk Management, and select "Detach VHD".
8. We've shrunk the operating system's conception of the file, but the VHD still thinks it is the size of the original physical disk. Download VHD_Resizer (http://vmtoolkit.com/files/folders/converters/entry87.aspx). (You will need to create an account, but it's free and appears to be innocuous.) In our example above, the VHD still thinks it's of a 100G drive, even though it may only have 50G of content on it. Run the resizer, select the VHD file that you've created, and specify a Save location for the new, re-sized VHD that it's going to create. You will end up, temporarily, with two copies of a pretty big file. If you don't have space available on another partition, you may need to temporarily attach an external drive. Specify the size that you shrunk the VHD to in the previous step as the size of the target file. The program is going to do a sector by sector copy of the source VHD to the target VHD, so this operation will also take a while. When you're done, you can delete the source VHD, and move the target VHD to it's final location. 9. We probably need to rewrite the boot sector, using a program from the boot subdirectory of the Windows install DVD called bootsect.exe. Attach the VHD using Disk Manager as before, then use the command "bootsect.exe /NT60 F: /FORCE /MBR" (substitute whatever drive letter the VHD gets attached as for the "F:"). Detach the VHD when you're done.
10. Now we need to modify the System Boot Configuration Data to add the VHD as a boot configuration. Scott Hanselman has an excellent blog post on this, and I'll refer you there (http://www.hanselman.com/blog/LessVirtualMoreMachineWindows7AndTheMagicOfBootToVHD.aspx) for instructions.
11. Once you have done that, you should be able to restart your system, and have the new VHD entry presented in the boot menu options. Go ahead and boot to it, then open a cmd shell and type "echo %SystemDrive%". The response is the drive letter that was assigned to the VHD. Don't modify anything. Restart the system and boot back to the native drive.
12. Final step. We need to modify the registry on the VHD so that it will cause the VHD to come up as the C: drive, and move the C: drive someplace else. We need to do this because the Registry on the VHD disk is a clone of the Registry that was on the native drive, and contains lots of paths that are hard-coded to the C: drive. If you don't modify where your VHD mounts, a lot of things that you think are pointed to VHD will actually point to the C: drive, and you won't have an isolated environment at all. Here's what we need to do.
Acknowledgements
I want to thank Scott Hanselman from Microsoft (@shanselman) for his attention and assistance during this process, and Garry Martin (@garrymartin) for helping me understand the VHD expansion on boot process, and the boot sector rewrite process. I would not have succeeded without their help.
Please feel free to leave comments or email me if this doesn't work for you.
I have my Windows system on my work computer set up the way I want it, with all the utilities and programs that I use installed. But I wanted to run the new Beta of Visual Studio 2010. However, there's a standing (and understandable) rule: no beta software on computers that you're doing production work on. But my boss did give me permission to use it on the project I'm working on (since it won't be production ready until after the scheduled release of VS2010 in March) PROVIDED that I run it in a segregated environment such as a Virtual PC) so that it can't corrupt anything else.
I had run the beta of Windows 7 in a Virtual PC when it came out, but it was painfully slow. Windows 7 has the capability to boot up from a Virtual Hard Disk (VHD) as though it were actually a hard disk and run with minimal speed degradation (the disk access is somewhat slower, but everything else is running just as fast, because you are actually running on the hardware, not a program simulating the hardware). So I decided I would clone my existing system drive, which is configured the way I want it, and boot from that. Should be simple, right? WRONG! There are a number of hidden gotchas along the way, which this post is intended to walk you through.
However, a couple caveats before you start down this road with me: I run my systems a little different than it comes out of the box. I have my disk partitioned in 2, similar to the way I learned to do it under Unix: I have a system drive (C:), but all my user data is on a separate partition. I assign the drive letter U: (for Users) to this partition. There's a number of reasons I do this, but for the purposes of this discussion, it means my working data is available to me whether I'm booted into the real system drive or to the VHD drive. If you have your drive setup the way Windows does it out of the box, with everything on C:, and your user data in C:\Users\<username>, then it's going to be clumsy, at best, to access the files you write in one instance from the other instance.
The other caveat is that you need enough free space on your disk to hold the VHD in a fully expanded form. In practice, this probably means that if your disk is more than half full, this isn't going to work for you.
Okay, that being said, here's the steps I took (on about the 5th iteration of this process). (Also, be warned that a couple of these steps can take quite a bit of time - altogether, this process could take half a workday.) (Also, if you run as a non-privileged user, most of these steps need to be run as administrator.)
1. Clean your system disk. Get rid of all the junk on it, because you're going to be cloning the disk and you want it as clean as possible. I use a freeware program called "CCleaner" (http://www.ccleaner.com/) which empties the Recycle bin, clears your browser cache, gets rid of temporary files, etc. When you finish, look at the properties of the drive to determine how much space is actually being used and how much is free. This will be important later when we resize the cloned disk.
2. Defragment your disk. Even though Windows runs defrag periodically, and will probably tell you the disk doesn't need defragging, run it anyway, and specify the /X (consolidate free space) option. I open an Administrative cmd prompt window (type "cmd" into the Start menu search box, then right click the "cmd.exe" option that shows up, and select Run As Administrator), and type "defrag C: /U /V /X". The idea is that we want to end up with all, or as much as possible, of the free space at the end of the drive. (This process will take a fair amount of time - maybe 10-20 minutes if I remember right, depending on drive size and how fragmented it is.)
3. Download and run Disk2Vhd.exe (http://technet.microsoft.com/en-us/sysinternals/ee656415.aspx) from the SysInternals group at Microsoft. In the list of disk partitions it gives you, select just your system partion (C:). This will clone your system drive into a virtual hard disk file. It's okay to put it on the same drive you're cloning. This process will also take quite a while.
4. Once that is done, use Windows' Disk Manager to Attach the VHD as another hard drive. Right click My Computer, and select Manage. Then click Disk Management. When the storage manager comes up , Right click Disk Management, and select "Attach VHD". Find the VHD file you created in the previous step, and attach it. It will show it as an additional hard disk, but it will tell you it's offline, and help will say it has a serial number conflict with an existing drive. Well, since we cloned it from the C: drive, it has the same serial number, so that makes sense. The good news is that it's easy to fix. If you right click the newly added Disk symbol (probably Disk 1), and select "Online", it will cure the problem and put the VHD online.
5. If your drive had multiple partitions, you will see that the mounted VHD has space corresponding to the space those partitions took up. The space will be shown as "RAW" partions. We need to delete those. Right click each of them (except the one corresponding to your cloned system partition), and select "Delete Volume". The space will now be shown as "Unallocated", and adjacent Unallocated partitions, if they exist, will be merged.
6. Now right click the partition corresponding to your cloned system drive, and select "Shrink Volume". It will spend a couple minutes analyzing the partition to see how much it can shrink it, then will give you a dialog box showing you how much space is now being taken up (the full size of your original partition), and how much is available to shrink. There will be a textbox/spinner to let you select how much you actually want to shrink it, and then shows the resulting final size after the designated shrink is applied.
There's a trade off here. The size you shrink it to becomes, in effect, the "physical size" of the new VHD, and you want to give yourself some room for expansion. On the other hand, when you're booted to the VHD, the system will expand it on disk to the full "physical size" specified, and you have to have room on the disk for that expansion, or the boot will fail. So, for example, if I have a 100G C: drive, with 30G used, and I clone it, the clone file may actually only take up, say, 25G. But if the "physical size" that I've specified for the VHD is 50G, when I boot from it, the OS will expand that 25G to the full 50G, which still leaves 100-(50+30) = 20G free. But if I now copy 25G of video onto that physical disk, then the new free space is 20G (100G - (30G+20G+25G)), but when I boot into the VHD, the OS will try to expand it and run out of room, and the boot will fail. So, make the size big enough to accommodate reasonable expansion, but balance that against the free space on the disk.
7. Once you've finished the shrink operation, right click the Disk in Disk Management, and select "Detach VHD".
8. We've shrunk the operating system's conception of the file, but the VHD still thinks it is the size of the original physical disk. Download VHD_Resizer (http://vmtoolkit.com/files/folders/converters/entry87.aspx). (You will need to create an account, but it's free and appears to be innocuous.) In our example above, the VHD still thinks it's of a 100G drive, even though it may only have 50G of content on it. Run the resizer, select the VHD file that you've created, and specify a Save location for the new, re-sized VHD that it's going to create. You will end up, temporarily, with two copies of a pretty big file. If you don't have space available on another partition, you may need to temporarily attach an external drive. Specify the size that you shrunk the VHD to in the previous step as the size of the target file. The program is going to do a sector by sector copy of the source VHD to the target VHD, so this operation will also take a while. When you're done, you can delete the source VHD, and move the target VHD to it's final location. 9. We probably need to rewrite the boot sector, using a program from the boot subdirectory of the Windows install DVD called bootsect.exe. Attach the VHD using Disk Manager as before, then use the command "bootsect.exe /NT60 F: /FORCE /MBR" (substitute whatever drive letter the VHD gets attached as for the "F:"). Detach the VHD when you're done.
10. Now we need to modify the System Boot Configuration Data to add the VHD as a boot configuration. Scott Hanselman has an excellent blog post on this, and I'll refer you there (http://www.hanselman.com/blog/LessVirtualMoreMachineWindows7AndTheMagicOfBootToVHD.aspx) for instructions.
11. Once you have done that, you should be able to restart your system, and have the new VHD entry presented in the boot menu options. Go ahead and boot to it, then open a cmd shell and type "echo %SystemDrive%". The response is the drive letter that was assigned to the VHD. Don't modify anything. Restart the system and boot back to the native drive.
12. Final step. We need to modify the registry on the VHD so that it will cause the VHD to come up as the C: drive, and move the C: drive someplace else. We need to do this because the Registry on the VHD disk is a clone of the Registry that was on the native drive, and contains lots of paths that are hard-coded to the C: drive. If you don't modify where your VHD mounts, a lot of things that you think are pointed to VHD will actually point to the C: drive, and you won't have an isolated environment at all. Here's what we need to do.
- Mount the VHD in Disk Manager again.
- Open regedit.
- Click HKEY_LOCAL_MACHINE
- Click File -> Load Hive
- In the browser window, navigate to F: (or whatever drive your VHD is mounted to)\Winows\System32\Config\SYSTEM, and click Open
- When it asks for a key name, make something up (I used "TempReg")
- Now Click the TempReg key that you just created, and under that click "MountedDevices"
- Right-click the key "DosDevices\C:", select Rename from the context menu, and change the C: to some unused drive letter on your system (I used Z:). This is where your native drive will now show up when you boot from the VHD.
- Right-click the key "DosDevices\<x>:", where <X> is whatever drive letter you found in the %SystemDrive% environment variable above. Relect Rename, and rename it to "DosDevices\C:". Now your VHD will boot as the C: Drive.
- Click the TempReg key, and then click File -> Unload Hive.
- Close the Registry Editor
- Detach the VHD in disk manager
Acknowledgements
I want to thank Scott Hanselman from Microsoft (@shanselman) for his attention and assistance during this process, and Garry Martin (@garrymartin) for helping me understand the VHD expansion on boot process, and the boot sector rewrite process. I would not have succeeded without their help.
Please feel free to leave comments or email me if this doesn't work for you.
Walking with God: Talk to Him. Hear from Him. Really., by John Eldredge
Posts
