39 replies to this topic

[sfinktah]

It worked



Highlighted is the spurious route it added for the NAS, but the ping (and the iSCSI install) work fine. I just modified the ipxe ibft code to copy the IP address of the host, instead of the gateway address (since that line was directly above and easy to copy).

Although the routing table still shows the gateway as being the real gateway, it obviously affected something internal, as the arp information for the iSCSI target is now populated, and correct.



If I have the time, I will try and implement a more detailed solution following the guide layed out in sanbootconf.


I justified the pain of going through another iSCSI install... I want to compare a freshly installed (but unbooted) iSCSI install, vs a copy has been installed in the standard fashion. I'm curious to see how easy it would be to to make the change afterwards. There are documents from Microsoft that explain how, but they are rediculously long.

[bwiese]

I want to compare a freshly installed (but unbooted) iSCSI install, vs a copy has been installed in the standard fashion. I'm curious to see how easy it would be to to make the change afterwards. There are documents from Microsoft that explain how, but they are rediculously long.

Well that sounds interesting. What are you hunting for?

[bwiese]

Are you using XPe for RDP? I looked briefly into XPe this week, since I had to run off a handful of identical XP VMs to do load testing. I ended up using this seemingly shoddy (but actually quite reliable) product from Singapore called "CCBoot." I'll not waste time praising it's amazing ability to clone a live machine to iSCSI with a few mouse clicks, but rather about how it's designed to use a single iSCSI drive for all clients (the whole thing is designed to run net-cafe's)

i cut my teeth on CCBoot, so I'm very familiar with it. I probably would have gone live on it, but I discovered a number of bugs that just kept getting in the way, and while the author spent an hour on IM with me going over them and seemed genuinely interested in fixing them, he never did, so I moved on. I also tried Double-Take's solution, which is terrible, mostly due to an unexplainably slow iSCSI Target. Finally I ended up on an all Microsoft build that, surprisingly, just works.

With Microsoft's iSCSI Software Target, you can also set up a single "gold image" VHD and then create differencing disks off that for mounting as individual target OS for write-backs. You can even mark the differencing VHD's as compressed (which actually gets you 90% of the space back). This works very, very well. The only problem is that creating the differencing VHD's must be done by Microsoft's iSCSI Software Target, and there is no UI or documentation for it, only a powershell commandlet and some blogging by a member of the (I think) product team.

I've tried many different thin zero maintenance clients, starting with several linux varieties. I used XP stripped down with nLite, to about 700 Mb on disk footprint, 60 Mb boot bandwidth, and <10 seconds boot time, which overall probably worked the smoothest. I still have those .vmdk files (from CCBoot), and converted .vhd files (which worked first-off on Microsoft iSCSI Software Target). Now I'm trying Windows Embedded Standard 7, mostly because of its superior hardware support for our company-wide dual monitors (XP just gets flaky sometimes). The VHD's are bigger (1.7 Gb vs 1 Gb on XP), and it boots a little slower, but it's actually more stable (virtually zero random blue screens on boot from what I assume are race conditions). WES7 is actually more resilient to recognizing a different NIC on boot than XP was. CCBoot used a modified version of sanbootconf (to stored the boot server IP in the registry), and he built his own CCBootPnP drivers which allowed for a completely new NIC to be detected, installed, and run on boot that made a single image work on much wider hardware. I think this was ultimately the best part of his product. Without that driver on WES7 I am forced to maintain multiple "gold images" for different NIC hardware, but it works well anyway.

Edited by bwiese, 09 December 2011 - 02:15 PM.

[sfinktah]

The only problem is that creating the differencing VHD's must be done by Microsoft's iSCSI Software Target, and there is no UI or documentation for it

Ahh, VHDs happen to be my particular area of expertise this month, as I'm developing an application specifically to manipulate them.

Lets do do the basic theory, even though you're probably familiar with it.

There are three kinds of VHD:

• fixed (these are just flat copies of a disk with a 512 byte footer at the end)
• dynamic (images of a disk, with all the unused areas skipped)
• differencing (images of the changed areas of a parent VHD).

As far as the technical layout and operating of differencing VHD goes, they are exactly the same as dynamic disks. Which we are all familiar with from virtualized machines. The only difference is that instead of only storing the sections of the used disk, they store only sections that have changed from their parent. (They can be nested 8 deep I think).

So in my pretty diagram below, the bottom line is what I call the "base" image. (that's why it's at the bottom). The two lines in the middle are differencing VHDs, and the top line is what you see if you looked at the disk when it was mounted.

▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ (What you get)
░░▓░░░░▓ ░░░▓░▓░░ ░░░░░░░░ ░░░░░░░░ ░░▓▓░░░░ ░░▓░░░░▓ ▓▓▓▓▓░░░ ░░░░░░░░ Differencing (Current)
░░░░░░░░ ░░░░░░░░ ░░░░░░░░ ░░░░░░░░ ░░▓▓▓▓▓▓ ░░▓▓▓▓▓▓ ░░░░░░░░ ░░░░░░░░ Differencing (Parent)
▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓▓▓▓▓▓ ▓▓▓░░░░░ ░░░░░░░░ ░░░░░░░░ Fixed (Grandparent)

Now, to make a differencing VHD (which just start off as empty files with a link to their parent), you can open DISKPART.EXE and something like:

create vdisk file=c:diff.vhd parent=c:previous_diff.vhd



After that is done, "diff.vhd" will appear to be identical to the previous vhd.

Does does that help, or are you trying to do something more complex?

BTW, the technical documentation you are missing is a 16 page word document, released as part of the Microsoft Open Anus Project or somesuch. It should be the top link on this page: delicious.com/sfinktah

[bwiese]

I'm pretty sure that Microsoft's iSCSI Software Target doesn't support differencing VHD's based on differencing VHD's. Only 1 layer is allowed. There are other constraints peculiar to iSCSI Software Target that are documented, but I was not exhaustive in my testing. Dynamics VHD's are not supported.

I should have been more specific. It is possible to create a differencing 'write-back' VHD against a 'golden image' parent VHD outside of iSCSI Software Target, and it will work fine if added as a device later. However, this can't be done while iSCSI Software Target is running, as it locks the parent VHD which prevents DISKPART from attaching to it. Since what I really wanted was to be able to discard and deploy an OS on the fly without shutting down the rest of the functioning iSCSI based machines, I needed iSCSI Software Target to do the creating of the child differencing VHD for me. Fortunately it does have a powershell commandlet that does this and attaches it as an iSCSI device, which can then be associated with an iSCSI target.

[sambul61]

This can't be done while iSCSI Software Target is running, as it locks the parent VHD which prevents DISKPART from attaching to it.

A parent VHD must be detached in order to create its child with Diskpart.

Could you guys give some examples, in what scenarios using iSCSI is preferable to other solutions? For what kind of companies, in what tasks, what OS versions, and how to better organize such setup? What are its primary benefits? What are major difficulties in setting it up? A kind of overview from an insider without marketing pitch...

I'm surprised, MS finally released its own iSCSI Target for servers seemingly contrary to its unti-monopoly settlements spirit.

Btw, for preparing and analyzing WinPE plugins and apps, NTCore Explorer Suite may be handy.

[bilou_gateux]

i've even written a moa241 iscsi plugin pack that gives you the complete iscsi targetting system in the PE envrionment. and it works - you can view all the iscsi targets you like - you just can't install to them because they don't have iBFTs.

Do you have detailed instructions on how to do this ? step-by-step instructions posted on sanbarrow forum or elsewhere...

[marek_sal]

Ahh, crap. I just hit back and deleted my big post. Too lazy to write it all again.

Thanks a lot for running that test for me. I have actually managed to get iSCSI install to work.

It turns out, that something was causing Windows to use the MAC address of my DHCP/DNS/TFTP/GATEWAY server to attempt to contact the iSCSI target.

I also noticed that Windows does it's own DHCP request, and doesn't request the extra options for iSCSI.

So I took the DHCP server out of the equation and wrote a script:

script.txt:

#!gpxe

ifopen net0

set net0/ip 192.168.1.115

set net0/netmask 255.255.255.0

set net0/gateway 192.168.1.138

set net0/dns 192.168.1.238

set keep-san 1

sanboot iscsi:192.168.1.8::::nas:iscsi

prompt --key 0x02 --timeout 2000 Press Ctrl-B for the iPXE command line... && shell || exit

and compiled into iPXE as an option rom for the virtual e1000 vmware nic:

make bin/8086100f.mrom DEBUG=scsi:3,iscsi:3 EMBED=script.txt

and inserted the required lines into the .vmx file for the virtual machine I was using:

ethernet0.opromsize = 262144

e1000bios.filename = "8086100f.mrom"

Set the boot order to DVD first, placed the official 7601.17514.101119-1850_x64fre_server_2008_r2_standard_enterprise_datacenter_and_web_with_sp1_x64_dvd_617601.iso (md5: 8dcde01d0da526100869e2457aafb7ca), and hit escape at boot to force it to load iPXE first.

That makes it drop nicely back to the DVD after the iSCSI initialization, and everything worked a charm...



Well... still haven't finished, but I can see my iscsi drive in diskpart now.



The moral of the story, is probably that I read somewhere in a Microsoft document that you should never use DHCP for iSCSI. I'm sure that isn't true entirely, but I think it would work a lot smoother if your DHCP or TFTP server were on the same IP as your iSCSI.

Hi sfinktah,

have you managed to enable the WinPE to install Windows on your iSCSI target?
I have the same results as you described in that post - after booting the WinPE, diskpart sees my iscsi target drive.

But when I run the Win7 x86 installer (from ISO file), installer doesn't allow me to install windows on iSCSI target.

I am using iPXE with these commands:

iPXE> dhcp
iPXE> clear dhcp.net0/gateway:ipv4
iPXE> sanhook --drive 0x81 iscsi:192.168.146.132:::iqn.2012-01.local:8gdisk (I can't hook to 0x80, as WinPE fails with 0xc000000e error)
iPXE> sanboot --drive 0xE0 http://192.168.146.134/winpex86-1.iso (my WinPE ISO, with iSCSI drivers and utilities)

I test it under VMware Workstation 8.

Regards,
Marek

[sfinktah]

i've even written a moa241 iscsi plugin pack that gives you the complete iscsi targetting system in the PE envrionment. and it works - you can view all the iscsi targets you like - you just can't install to them because they don't have iBFTs.

Do you have detailed instructions on how to do this ? step-by-step instructions posted on sanbarrow forum or elsewhere...

Ahh crap... to marek_sal and bilou... I don't have a blind clue how I did it, and I accidentally deleted the VM that I use as my "Technician PC."

But, I believe what I basically did, was take the instructions out of another guide (perhaps the "How to Boot Windows XP via iSCSI" one) and took the time to prepare the registry changes as scripts, and to take all the required DLL files and pack them all up into a moa pack.

I think I archived a copy though, the URL was posted earlier in the thread without any explanation, but it was: http://reboot.nt4.co...tiator.32_64.7z

I don't recall how moa241 plugins work, but I'm sure I designed it to be compatible with whatever standard sanbarrow created... a quick peek at it shows it extracts stuff like this:

./WIN7_add/amd64/Windows/System32/iscsicpl.exe

./WIN7_add/amd64/Windows/System32/iscsidsc.dll

./WIN7_add/amd64/Windows/System32/iscsiwmi.dll

./WIN7_reg

./WIN7_reg/x86

./WIN7_reg/x86/plugins

./WIN7_reg/x86/plugins/PE3-SYSTEM_iscsi_init.reg

./WIN7_reg/x86/plugins/PE3-SOFTWARE_iscsi_init.reg

./WIN7_reg/amd64

./WIN7_reg/amd64/plugins

./WIN7_reg/amd64/plugins/PE3-SYSTEM_iscsi_init.reg

./WIN7_reg/amd64/plugins/PE3-SOFTWARE_iscsi_init.reg

./WIN7_drivers

./WIN7_drivers/x86

./WIN7_drivers/x86/plugin_driver_folders_here.txt

./WIN7_drivers/ia64

./WIN7_drivers/ia64/plugin_

I certainly had no luck installing an O/S from that point, but I was more interested in confirming iSCSI connectivity at the time.

Marek - if I was to install another iSCSI based setup, and given that I've forgotten everything I learned (thank god for being able to read my own posts), then I would use a modified version of iPXE - or some tricky scripting.... and then just do a normal install from DVD to iSCSI.

It's not that I don't think a WinPE based installer won't work, it's just that it didn't seem to work for me. And that was an issue I never resolved. I never went back to try again after I found the root issue.

These days I'm happy with my WindowsXP-SP3-DN35.rar (530mb) and Windows7x32.rar (4.5gb).

Both are unlocked and tidied up versions of the Microsoft Browser Test VHD's, converted to run in VMware Fusion. I unzip, I play, I erase.

[marek_sal]

Recently iPXE team has launched the 'wimboot' pseudo-environment to boot the WinPE .wim file directly from eg. HTTP server:

http://ipxe.org/wimboot

It solves all my problems, because I can sanhook the drive to 0x80 address , then wimboot the WinPE image, and finally eg. 'net use' the samba share with any kind of new windows (7,8,2012) installer files and istall it to iSCSI target.

Cheers,
Marek

[Wonko the Sane]

Recently iPXE team has launched the 'wimboot' pseudo-environment to boot the WinPE .wim file directly from eg. HTTP server:

and, just to keep things as together as possible, the topic is discussed here:
http://reboot.pro/17520/


Wonko

[Sha0]

Although the routing table still shows the gateway as being the real gateway, it obviously affected something internal, as the arp information for the iSCSI target is now populated, and correct.

Please see "...creates an explicit route to your your iSCSI target..." under "What to do if the installation can't find the iSCSI disk" for a little more information. Although on the gPXE wiki, there is much information there that applies to iPXE, as well.

...
I have the same results as you described in that post - after booting the WinPE, diskpart sees my iscsi target drive.

But when I run the Win7 x86 installer (from ISO file), installer doesn't allow me to install windows on iSCSI target.

I am using iPXE with these commands:

iPXE> dhcp
iPXE> clear dhcp.net0/gateway:ipv4
iPXE> sanhook --drive 0x81 iscsi:192.168.146.132:::iqn.2012-01.local:8gdisk (I can't hook to 0x80, as WinPE fails with 0xc000000e error)
iPXE> sanboot --drive 0xE0 http://192.168.146.134/winpex86-1.iso (my WinPE ISO, with iSCSI drivers and utilities)
...

There was a bug in iPXE that was just fixed a few days ago, regarding the drive number issue you mention above.

I'm not sure what you are expecting this to accomplish, however. I believe that only the sanboot command (not sanhook) will establish an iBFT for Windows to be able to re-establish the SAN connection. In your example above, there would only be an iBFT for the optical disc, but Windows does not understand non-HDD iSCSI SANs, as far as I know. The Windows PE environment would be established, but the rest of the optical disc would be unavailable. That is probably not want you want. You probably want the HDD SAN's iBFT and probably want the rest of your optical disc to be available, with its utilities.

...
It solves all my problems, because I can sanhook the drive to 0x80 address , then wimboot the WinPE image, and finally eg. 'net use' the samba share with any kind of new windows (7,8,2012) installer files and istall it to iSCSI target.

Using 0x80 for the HDD SAN and 0xE0 for the installation disc should now be fixed in iPXE.

[sfinktah]

I can't really comment as to what I was talking about, since that was surely centuries ago, but perhaps some elegant verse from K&R will explain it.

--- ./src/drivers/block/.backups/ibft.c;1    2011-12-07 23:37:15.000000000 +1100

+++ src/drivers/block/ibft.c    2011-12-09 16:12:02.000000000 +1100

@@ -233,30 +233,36 @@

     /* Fill in common header */

     nic->header.structure_id = IBFT_STRUCTURE_ID_NIC;

     nic->header.version = 1;

     nic->header.length = cpu_to_le16 ( sizeof ( *nic ) );

     nic->header.flags = ( IBFT_FL_NIC_BLOCK_VALID |

            	   IBFT_FL_NIC_FIRMWARE_BOOT_SELECTED );

 

     /* Extract values from configuration settings */

     ibft_set_ipaddr_setting ( &nic->ip_address, &ip_setting, 1 );

     DBG ( "iBFT NIC IP = %sn", ibft_ipaddr ( &nic->ip_address ) );

-    ibft_set_ipaddr_setting ( &nic->gateway, &gateway_setting, 1 );

+

+    ibft_set_ipaddr_setting ( &nic->gateway, &ip_setting, 1 );

     DBG ( "iBFT NIC gateway = %sn", ibft_ipaddr ( &nic->gateway ) );

+

+    // ibft_set_ipaddr_setting ( &nic->gateway, &gateway_setting, 1 );

+    // DBG ( "iBFT NIC gateway = %sn", ibft_ipaddr ( &nic->gateway ) );

+

     ibft_set_ipaddr_setting ( &nic->dns[0], &dns_setting,

                   ( sizeof ( nic->dns ) /

                	 sizeof ( nic->dns[0] ) ) );

     DBG ( "iBFT NIC DNS = %s", ibft_ipaddr ( &nic->dns[0] ) );

     DBG ( ", %sn", ibft_ipaddr ( &nic->dns[1] ) );

-    if ( ( rc = ibft_set_string_setting ( strings, &nic->hostname,

-                   	   &hostname_setting ) ) != 0 )

+

+    if ( ( rc = ibft_set_string_setting ( strings, &nic->hostname, &hostname_setting ) ) != 0 )

         return rc;

+

     DBG ( "iBFT NIC hostname = %sn",

    	   ibft_string ( strings, &nic->hostname ) );

 

     /* Derive subnet mask prefix from subnet mask */

     fetch_ipv4_setting ( NULL, &netmask_setting, &netmask_addr );

     while ( netmask_addr.s_addr ) {

         if ( netmask_addr.s_addr & 0x1 )

             netmask_count++;

         netmask_addr.s_addr >>= 1;

     }

@@ -471,10 +477,11 @@

         return rc;

     if ( ( rc = ibft_fill_initiator ( &ibft->initiator, &strings,

                       iscsi ) ) != 0 )

         return rc;

     if ( ( rc = ibft_fill_target ( &ibft->target, &strings,

                	    iscsi ) ) != 0 )

         return rc;

 

     return 0;

 }

+/* vim: set ts=8 sts=0 sw=3 noet: */

Although I made these changes a long time ago, the relevant code hasn't changed (I just pulled the latest source). That's not to say that the issue wasn't fixed elsewhere since then, and I'm certainly not suggesting this was anything but a quick hack to suit my particular situation.

But hopefully it explains the issue, and how I dealt with it at the time.

[Sha0]

I can't really comment as to what I was talking about, since that was surely centuries ago, but perhaps some elegant verse from K&R will explain it.

...some iPXE source code...

While that work-around might work, it doesn't "understand" the problem, and the problem only exists in a limited sense. I submit that referring to the above-referenced Etherboot Wiki page and section can yield better insight.

[sfinktah]

I was planning on coding a proper solution, I recall that I had mapped out a reasonable logic flow to apply it only when necessary, .. however I haven't a clue what that was anymore.

The reference to the wiki is good, now that it's in this thread, it might actually get seen. I know I googled the bejesus out my browser looking for answers, and did come across it.

BTW, it did work. That's how I finally managed to install Windows 7.