Jump to content











Photo
- - - - -

Over-the-network Windows 10 installation into iSCSI LUN

iscsi ipxe tiny pxe server windows server 2012 r2 diskless win10

  • Please log in to reply
18 replies to this topic

#1 scan80269

scan80269
  • Members
  • 5 posts
  •  
    United States

Posted 15 August 2016 - 05:44 AM

I have figured out how to install Windows 10 directly into iSCSI LUNs for diskless clients.  Tiny PXE Server is set up on the iSCSI server.  A customized WinPE with iSCSI Initiator enabled, along with Win10 OS installation files, are put into a small (8-10GB) iSCSI LUN to support over-the-network OS installation.  iPXE is used on the diskless clients. The high level steps are as follows:

 

 

1. Set up iSCSI target server
- Windows Server 2012 R2, Tiny PXE Server & iPXE used in this procedure
 
2. Set up one iSCSI LUN for each diskless client
- Recommend using dynamically expanding VHDX file with Windows Server 2012 R2
  A. Create and quick format iSCSI LUN disk partition to NTFS
  B. Add client MAC address to IQN for its dedicated iSCSI LUN
 
3. Set up a small (8-10GB) bootable iSCSI LUN for Windows OS installation
  A. Add client MAC addresses to IQN for this iSCSI LUN
  B. Prepare customized WinPE with iSCSI Initiator enabled
  - See add_iscsi.cmd batch file example to customize WinPE
  C. Mount iSCSI LUN to drive letter
  D. Transfer customized WinPE and OS installation files into iSCSI LUN
  - WinPE files to root (\sources, \boot, \efi, \bootmgr, \bootmgr.efi, etc.)
  - OS installation files to a folder, e.g. \Win10
  E. Set iSCSI LUN disk partition to Active for booting
  - e.g. DISKPART - sel dis 1, sel par 1, active
  F. Unmount/Eject iSCSI LUN
 
4. Set up iPXE & Tiny PXE Server to support PXE->iPXE->WinPE booting on diskless clients
- Configure iPXE boot script to support both OS installation (once per client) and diskless booting for clients
- See iscsi.ipxe script example
 
5. Initiate PXE boot on diskless client to boot to WinPE command prompt
 
6. Launch iSCSI Initiator (iscsicpl) and connect to iSCSI LUN prepared in step #2
 
7. Change drive & folder to OS installation files
- e.g. C:\Win10\sources
 
8. Launch setup.exe to start OS installation.  Select the iSCSI LUN disk partition to install OS into.

 

Here's a sample iscsi.ipxe file:

================================================
#!ipxe

echo iscsi.ipxe script running...
ifopen net0
dhcp
set net0/gateway 0.0.0.0
echo IP: ${net0/ip}, Gateway: ${net0/gateway}
set net0/keep-san 1
echo keep-san: ${net0/keep-san}
sanboot iscsi:<server IP>::::iqn.1991-05.com.microsoft:<server name>-win10-client1-target ||
sanboot iscsi:<server IP>::::iqn.1991-05.com.microsoft:<server name>-win10-client2-target ||
sanboot iscsi:<server IP>::::iqn.1991-05.com.microsoft:<server name>-win10-install-target
================================================

Note:
- replace <server IP> with iSCSI server static IP address: e.g. 192.168.0.100
- replace <server name> with iSCSI server name, e.g. iSCSIserver. This name must match what the client sees within iSCSI Initiator
- setting gateway address to 0.0.0.0 is a workaround to avoid Windows 10 suffering an "INACCESSIBLE BOOT DEVICE" blue screen crash during boot

 

This script supports a one-time OS installation into a diskless client, as well as the subsequent OS bootups.  The above example has iSCSI LUNs set up for two clients, but can be extended to an arbitrary number of clients.  The iSCSI LUNs are set up to match the MAC address of the intended clients, so a given client will always sanboot from its corresponding iSCSI LUN.

 

By pre-formatting the iSCSI LUN disk partitions (for each client), the first time the client PXE boots, the sanboot command for the iSCSI LUN will fail, since the disk is still blank (though formatted), so the iPXE script falls through to the last line which connects to the "OS install" iSCSI LUN and boots to a WinPE command prompt.  After running iSCSI Initiator (iscsicpl) to connect to the iSCSI LUN, OS installation can start in much the same way as for a local physical disk.

 

I have been successfully using ipxe-undionly.kpxe included in Tiny PXE Server 1.0.0.19.

 

Following is a batch file used to customize WinPE (64-bit) to add iSCSI Initiator support:

Add_iSCSI.cmd:

===================================
set WindowsPE=C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs
if not exist C:\Mounted_WIM md C:\Mounted_WIM
Dism /Mount-Image /ImageFile:C:\WinPE\media\sources\boot.wim /Index:1 /MountDir:C:\Mounted_WIM
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\WinPE-WMI.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\en-us\WinPE-WMI_en-us.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\WinPE-NetFX.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\en-us\WinPE-NetFX_en-us.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\WinPE-Scripting.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\en-us\WinPE-Scripting_en-us.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\WinPE-PowerShell.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\en-us\WinPE-PowerSHell_en-us.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\WinPE-StorageWMI.cab"
Dism /Add-Package /Image:C:\Mounted_WIM /PackagePath:"%WindowsPE%\en-us\WinPE-StorageWMI_en-us.cab"
copy C:\windows\system32\iscsicpl.dll C:\Mounted_WIM\windows\system32
copy C:\windows\system32\iscsicpl.exe C:\Mounted_WIM\windows\system32
copy C:\windows\system32\en-us\iscsicpl.dll.mui C:\Mounted_WIM\windows\system32\en-us
copy C:\windows\system32\en-us\iscsicpl.exe.mui C:\Mounted_WIM\windows\system32\en-us
Dism /Get-Packages /Image:C:\Mounted_WIM
pause
Dism /Unmount-Image /MountDir:C:\Mounted_WIM /commit
===================================

 

This batch file works with Windows 10 ADK (Assessment and Deployment Kit) which can be downloaded from Microsoft.

 

A firewall opening for Tiny PXE Server is needed.  This type of remote OS booting from iSCSI LUN apparently works only if the network interconnecting the server and clients is set to "private" (instead of "public") at the server.  If Windows Server OS see the network as "unidentified", run gpedit.msc to change the network type from "public" to "private".

 

I'm working on a more detailed step-by-step procedure to be available later.

 

A big thanks to Erwan for Tiny PXE Server in making this possible.

 



#2 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13305 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 15 August 2016 - 08:40 AM

Nice. :)

Can you post also some info on the hardware you used for the tests?

I mean, is the iSCSI LUN  a self-standing device or a "normal" PC/server?

Once the install is finalized, if I get this right, you have one partition on the iSCSI LUN device for each client, correct? :unsure:

 

:duff:

Wonko



#3 scan80269

scan80269
  • Members
  • 5 posts
  •  
    United States

Posted 15 August 2016 - 02:11 PM

I've tested using mostly Intel NUC boxes:

- iSCSI target server: NUC5i5MYBE (in Akasa fanless chassis, Micron M600 512GB M.2 SSD, 16GB DDR3), Windows Server 2012 R2, Tiny PXE Server

- client PCs: NUC6i7KYK, NUC6i5SYH, NUC5i5MYBE, DH87RL motherboard, HP consumer tower ("Thimphu" Skylake motherboard)

- Ethernet switch: Netgear GS108

- Wireless router (as DHCP server & connection to internet): D-Link DIR-868L

 

The iSCSI LUN for each client is contained in a separate VHDX file on the server.  I chose dynamically expanding VHDX, and the file sizes are just below 17GB for clean Win10 OS plus device drivers.  OS boot time (excluding PXE->iPXE boot times, just sanboot of iSCSI LUN) over gigabit is 20-30 seconds.

 

I tested with latest Windows 10 Anniversary Edition (Version 1607) a.k.a. "Redstone 1" and the corresponding ADK, but did not try older Version 1511 or original 10240 build.  This Windows 10 comes with inbox Intel NIC drivers so there was no need to offline inject drivers into OS files (install.wim) to make this work.  I suppose driver offline injection will necessary for NICs that have no Win10 inbox driver support.

 

Currently, only legacy PXE boot can work on the client side.  This is because iPXE supports sanboot with legacy BIOS but not with UEFI.  I look forward to seeing iPXE support sanboot with UEFI.



#4 scan80269

scan80269
  • Members
  • 5 posts
  •  
    United States

Posted 15 August 2016 - 05:41 PM

One correction to the last line of the iscsi.ipxe script file:

 

sanboot --drive 0x81 iscsi:<server ip>::::iqn.1991-05.com.Microsoft:<server name>-win10-install-target

 

Since this "Install OS" iSCSI LUN is mounted as the second iSCSI drive (first drive is the designated OS boot drive), the "--drive 0x81" is needed, as the sanboot line for the designated boot drive is already mapped to drive 0x80.



#5 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13305 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 15 August 2016 - 07:36 PM

I am clearly missing something :w00t: , but a diskless device with a 512 Gb SSD seems to me not-so-diskless... :unsure:

 

:duff:

Wonko



#6 scan80269

scan80269
  • Members
  • 5 posts
  •  
    United States

Posted 15 August 2016 - 08:25 PM

By diskless client I mean PCs running Win10 with no local storage attached. The server carrying the remote disks for the clients obviously cannot be diskless.  The clients have no local disk attached as the server is acting as remote storage accessed over Ethernet.  This is not a scheme for making Win10 run entirely in memory.

 

I've been able to boot Win10 via iSCSI disk since early this year, but only recently learned how to do over-the-network OS install.  I used to have to install Win10 into a local hard disk first, then clone it to the iSCSI disk, then detach the local hard disk.  Now, I don't need to make temporary use of a physical hard disk.  No DVD install media or USB flash drive needed for OS installation either, though a set of Win10 OS installation files and ADK are needed for setup.



#7 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13305 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 16 August 2016 - 09:37 AM

By diskless client I mean PCs running Win10 with no local storage attached. The server carrying the remote disks for the clients obviously cannot be diskless.  The clients have no local disk attached as the server is acting as remote storage accessed over Ethernet.  This is not a scheme for making Win10 run entirely in memory.

I understand what you mean :), but diskless has a meaning (rather intuitive BTW) which means another thing.

And the 512 Gb SSD physically present on the machine is still "local storage" (and quite a lot of it in absolute terms).

Just trying to make things as clear as possible and disambiguate (if needed).

As an example most "diskless" thingies I am familiar with may be actually "diskless" or have a local CF card or SD card (let's say 64 Mb or 128 Mb in size) to store a handful of settings that are machine/station specific.

 

:duff:

Wonko



#8 tal.aloni

tal.aloni

    Newbie

  • Developer
  • 20 posts
  •  
    Israel

Posted 19 August 2016 - 08:31 PM

Windows 10 RTM is indeed broken and need some trick for direct install to iSCSI to work.

Fortunately, Windows 10 v1607 fixed this issue.

You simply need to chainload iPXE and "sanhook" to the iSCSI target and then boot Windows PE v1607 and start the installation. that's it.



#9 scan80269

scan80269
  • Members
  • 5 posts
  •  
    United States

Posted 22 August 2016 - 05:49 PM

Yes.  Direct OS install to iSCSI works for Windows 10 Anniversary Edition (Version 1607 "Redstone 1") but fails for Windows 10 Version 1511 ("Threshold 2") and Windows 10 RTM (build 10240).  Based on my testing, I'm convinced the difference is in the OS itself.  I even tried offline injecting appropriate NIC drivers into OS install files (boot.wim & install.wim) but it didn't help.

 

I also tried directly installing Windows Server 2012 R2 into iSCSI, but that didn't work either.



#10 Ohana

Ohana
  • Members
  • 3 posts
  •  
    Canada

Posted 07 September 2016 - 12:37 AM

I used Jentu to direct installl windows 10, it was just like local disk but faster.
I got it bitlocked as well :)


Edited by Ohana, 07 September 2016 - 12:38 AM.


#11 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13305 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 07 September 2016 - 07:40 AM

I used Jentu to direct installl windows 10, it was just like local disk but faster.
I got it bitlocked as well :)

How nice of you to join the board only to recommend a proprietary/Commercial tool, it must be very good if you felt like sharing it.

I am sure everyone will be interested in a software for which there is no price, and just some videos and interactive demos on a site that seems designed to sell shoes (or smartphones, or whatever, to millennials).

Their KB site - at least - is readable and has some content:
http://kb.jentu.net/index.html

 

Here is an excerpt of this page:

http://kb.jentu.net/...-with-wifi.html

 

 

Example use case

An agency with 25,000 workstations of varying makes and models across 120 geographically independent offices (an average of 207 workstations each) has trouble maintaining a consistent environment across all devices. Management would like to use a locked-down and easily maintained OS management platform, but cannot afford expensive virtualization and storage servers or replacement of desktop workstations. Reimaging every workstation is out of the question because it takes 4+ hours per workstation.

 

The Jentu Distributed VDI solution repurposes existing server hardware deemed too slow for virtualization purposes as Jentu storage nodes. Three to five servers at each location provides consistent and reliable access to locked down Microsoft Windows environment(s). Jentu Distributed VDI Workstations can be brought online in approximately 1 hour or less. All 207 workstations in each location run load-balanced across all 3-5 servers in their local office building, the only configuration required is to install and run a VM hypervisor configured to boot from network. VMware Workstation, Parallels, or Oracle VirtualBox are example hypervisors. Initial configuration of a Jentu Distributed VDI workstation takes approximately 1 hour or less.

 

In each office, 200 unique, custom built PCs all share a single image, enabling their network administrator to affect all user environments securely in just minutes.

 

I am rather sure that most Senior IT managers with operations in this size of magnitude are reboot.pro members and will be interested in the product.

 

:duff:

Wonko



#12 Alexander Phang

Alexander Phang
  • Members
  • 1 posts
  •  
    China

Posted 27 November 2016 - 09:09 AM

Hi,

I've been working on this for weeks, I deployed this kind of diskless desktop computers in company months ago, hosting about 30 computers. Everything works fine until Microsoft release cumulative update for Windows 10 version 1607. Computers which have version 1511 installed download the update and keep trying to update to 1607. But the "Cumulative Update" requires a WinPE environment to install(I guess), then the update failed and leaving the computer "preparing for automatic repair".

Now I'm considering two solutions:

1. Deploy Windows 10 Version 1607 again and pray. Microsoft may solve this problem in version 1607, so the next cumulative update will success without problem.

2. Buy a iSCSI HBA card for every computer. I even have chosen a model QLE4060c form QLogic. Buying a used card is cheap form taobao.com(similar with eBay). The HBAs card will connect to iSCSI target and emulate a local storage device for booting operating system, block I/O etc. It work just like a local drive.

 

Any suggestions?



#13 Ohana

Ohana
  • Members
  • 3 posts
  •  
    Canada

Posted 27 November 2016 - 11:19 AM

 

1. Deploy Windows 10 Version 1607 again and pray. Microsoft may solve this problem in version 1607, so the next cumulative update will success without problem.

2. Buy a iSCSI HBA card for every computer. I even have chosen a model QLE4060c form QLogic. Buying a used card is cheap form taobao.com(similar with eBay). The HBAs card will connect to iSCSI target and emulate a local storage device for booting operating system, block I/O etc. It work just like a local drive.

 

Any suggestions?

Mr. Phang
Option 2 will offload the TCP engine changing the NIC sequence decoupling the NIC driver from the windows kernel, but it wouldn't solve the boot error you encountering.

Best cause of action would be to roll back to last known good and update aa single "unforked" image then roll out to the 30 production machines.

Based on my experience version 1607 fixed the TCP offload order and now supports iSCSI boot natively.

Bare in mind Microsoft has dropped licensing for Diskless in windows 10 (link below):
https://community.sp...ence-with-jentu 

Are you serving each endpoint a dedicated image?


Edited by Ohana, 27 November 2016 - 11:20 AM.


#14 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13305 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 27 November 2016 - 11:31 AM

Bare in mind Microsoft has dropped licensing for Diskless in windows 10 (link below):
https://community.sp...ence-with-jentu 

 

I can find nowhere in that any actual verified statement about that, all I can see is some loose talks about various licensing forms, some - as obvious - promotion for Jentu and a general consensus that noone - not even MS representatives - actually understand Windows licenses or - even if they do - they cannot speak for the company.

 

:duff:

Wonko



#15 Ohana

Ohana
  • Members
  • 3 posts
  •  
    Canada

Posted 27 November 2016 - 12:02 PM

Wonko,

No need to be spiteful, Jentu does secure disk streaming, with a functional feature set, that allows administrators to streamline desktop roll-out and management in a concise package, with the ability to quickly mitigate damages or downtime caused through software updates.

In this thread a SPiceworks Moderator shared his experience with the platform, no compensation was offered in return only the chance to finally get diskless endpoint with 99.999 uptime.

 

.
Chris (Microsoft) Jun 22, 2016 at 6:51 PM 
 

Brand Representative for Microsoft

Is the MS rep.



#16 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13305 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 27 November 2016 - 01:18 PM

Maybe I missed it but all I can see in that post by Chris:

https://community.sp...ic/post/5937395

is that diskless devices that came with 7 or 8 can be upgraded to 10 according to some given licensing program, whilst some XP and Vista ones cannot (thankfully I may add).

 

As a side note I am personally (trusting your word for it) rather confident that Jentu :) is probably the fourth best thing in the world  (after - say - bread, icecream and grub4dos), only it is a high-end, top-level, "corporate" tool suitable for multi-national enterprises, and as such is not available to "common mortals" (like - I believe - most of the members on reboot.pro are).

 

In one of my usual automotive comparisons, you are effectively advertising and promoting Bentley's and Rolls Royce's (and there excellences) in a country with no roads and where the average income is lower than or around the poverty threshold.

 

By all means you are perfectly free to do so :good: , but don't be so surprised that the natives look at you strangely.  ;)

 

:duff:

Wonko



#17 disik

disik
  • Members
  • 2 posts
  •  
    Russian Federation

Posted 18 January 2017 - 06:21 AM

Thanks to the topic starter, I have successfully installed Windows 10 1067 with the DVD disc directly to iSCSI, without any WinPE creation. I was lucky to have my NIC drivers included into ISO by Microsoft itself. My iPXE config is

#!ipxe

dhcp net0
set net0/gateway 0.0.0.0
set initiator-iqn iqn.2017-01.servername:win10-${net0/mac:hexhyp}
set root-path iscsi:servername::::iqn.2017-01.servername:win10-${net0/mac:hexhyp}
set username win10
set password win10password
set keep-san 1
sanboot --drive 0x80 ${root-path}

  • ZEE likes this

#18 disik

disik
  • Members
  • 2 posts
  •  
    Russian Federation

Posted 19 January 2017 - 11:08 AM

Magically the same worked with Windows 7 64 bit SP1 vanilla ISO with manual loading of the NIC driver on pre-setup stage.


  • ZEE likes this

#19 ZEE

ZEE

    Member

  • Members
  • 65 posts
  •  
    Portugal

Posted A week ago

Yep.. .this works... at least in Virtualbox...

Booting a Windows/PC needs the NIC drivers 'injected' or boot will hang when passing control to Windows Kernel...

-_-

 

Thanks to the topic starter, I have successfully installed Windows 10 1067 with the DVD disc directly to iSCSI, without any WinPE creation. I was lucky to have my NIC drivers included into ISO by Microsoft itself. My iPXE config is

#!ipxe

dhcp net0
set net0/gateway 0.0.0.0
set initiator-iqn iqn.2017-01.servername:win10-${net0/mac:hexhyp}
set root-path iscsi:servername::::iqn.2017-01.servername:win10-${net0/mac:hexhyp}
set username win10
set password win10password
set keep-san 1
sanboot --drive 0x80 ${root-path}

 


Edited by ZEE, A week ago.






Also tagged with one or more of these keywords: iscsi, ipxe, tiny pxe server, windows server 2012 r2, diskless win10

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users