Jump to content











Photo

Ubuntu install & boot on iSCSI over PXE

pxe iscsi pytarget ubuntu

  • Please log in to reply
9 replies to this topic

#1 vsoraas

vsoraas
  • Members
  • 2 posts
  •  
    Norway

Posted 09 October 2017 - 10:44 PM

Hello everyone!
 
Before anything else, BIG thanks to Erwan who made TPS, and to everyone (Misty and Erwan in particular as far as I can tell) for all the various how-to's and guides.
 
The goal of this post:  
1. PXE-boot a client system
2. Load an ubuntu installer over the web
3. Install ubuntu on an iSCSI target
4. PXE-boot the client into the new ubuntu installation
 
Instructions were largely taken from here:
 
 
I had some trouble figuring it all out and it took me several hours to cobble everything together. Maybe due to my particular circumstance of desired outcome and lack of previous knowledge in this area, I had trouble understanding everything correctly and spent a lot of time with trial and error to figure out some basic things about PXE. In the end I had to combine things from several resources (mostly the above ones, which I now see that I wouldn't have needed if I had more knowledge about PXE before starting this project.
 
For this reason, I hope my post here can help other newcomers connect the dots. I spent hours stuck on the most basic things (didn't understand what to use for initiator-iqn, for example) - so don't be like me, read my story and learn from my mistake instead :)
 
Regardless - here's how I made it all work in the end, dummies version:
 
 
First - my setup
 
Host system: 
192.168.1.10 
Running Win10, TinyPXEServer, pyTarget
 
Client system:
192.168.1.15
Intel NUC
 
Local gateway (also the DHCP server):
192.168.1.1
pfSense 2.3.1-RELEASE
 
 
Configs
 
TinyPXE's config.ini:
[dhcp]
root=E:\TinyPXEServer\files\
filename=ipxe.pxe
altfilename=ubuntu1604netboot.ipxe
httpd=0
binl=0
start=0
dnsd=0
proxydhcp=1
bind=1
smb=0
optextra=175.6.1.1.1.8.1.1
log=0
opt1=255.255.255.0
opt3=192.168.1.1
opt6=192.168.1.1
opt28=192.168.1.255
opt43=0
opt51=3600
opt54=192.168.1.10
next-server=192.168.1.10
opt60=PXEClient
poolstart=192.168.1.210
poolsize=10
[frmDHCPServer]
top=476
left=3206
 
TinyPXE's ubuntu1604netboot.ipxe:
#!ipxe
set boot-url http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64
kernel ${boot-url}/linux vga=788 --- quiet 
initrd ${boot-url}/initrd.gz
boot
 
pyTarget's config.xml:
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<root>
    <console ip="" port="3260" debug_level="6" />

    <target name="iscsi" ip="192.168.1.10" port="3260" portal="1">
        <host name="iqn.2006-11.1" target_pwd="" initiator_pwd="">
            <lun id="0" type="0" path="E:\pyTarget\ubuntu16.ima" capacity="81943040" />
        </host>

        <item key="MaxConnections" value="8" />
        <item key="InitialR2T" value="Yes" />
        <item key="ImmediateData" value="Yes" />
        <item key="FirstBurstLength" value="65536" />
        <item key="MaxBurstLength" value="262144" />
        <item key="DefaultTime2Wait" value="2" />    
        <item key="DefaultTime2Retain" value="20" />        
        <item key="MaxOutstandingR2T" value="16" />  
        <item key="DataPDUInOrder" value="Yes" />  
        <item key="DataSequenceInOrder" value="Yes" />     
        <item key="ErrorRecoveryLevel" value="2" />
        <item key="MaxRecvDataSegmentLength" value="65536" />
        <item key="AuthMethod" value="none" /> 
    </target>
</root>
 
Installation procedure
 
1. Start pyTarget
2. Start TinyPXEServer
3. Verify that both services' consoles show everything is OK
4. Boot client
5. Wait for the files to load & ubuntu installation to start
6. Once you're at the part to select installation directory, select "Manual" and then "Configure iSCSI volumes".
7. Choose "Log in", enter IP and IQN (copy TARGET NAME value from your config.xml)
8. If you left the password field empty in config.xml, when ubuntu now prompts you for a password you have to press SPACEBAR a single time and then press ENTER.
9. Go back and select "Guided" (unless you really want to install using manual... it's your tears, not mine, so go ahead)
10. Install as usual
 
Installation in progress *as seen from the host consoles*:  
 
Wireshark wasn't necessary, but I'd used it to troubleshoot a problem I had to figure out after I made the configs poorly so I figured I would look at the network as well during the install.
 
 
Post-install setup of TinyPXE to allow the booting
 
This file can also be used to launch the installation instead of ubuntu1604netboot.ipxe, but I hadn't made this file at that point in time.
 
bootmenu.ipxe:
#!ipxe
set boot-url http://${next-server}
# iqn must be same as TARGET NAME in pytarget config.xml
set iqn iscsi
#set iscsi-target iscsi:${boot-url}::::${iqn}
set iscsi-target iscsi:192.168.1.10::::iscsi

#================ Main Menu =================
:menu
menu iPXE MENU
item ubuntuiscsi Boot ubuntu 16.04 via iSCSI
item ubuntunetboot Install ubuntu 16.04 via netboot
item shell          Drop iPXE shell
choose target && goto ${target}
#============== End Main Menu ===============

:ubuntuiscsi
set gateway 0.0.0.0
# initiator-iqn must be same as HOST NAME under the TARGET in config.xml
set initiator-iqn iqn.2006-11.1 
set keep-san 1
sanboot ${iscsi-target}
boot

:ubuntunetboot
set boot-url http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64
kernel ${boot-url}/linux vga=788 --- quiet 
initrd ${boot-url}/initrd.gz
boot

:shell
echo Type exit to get the back to the menu
shell
set menu-timeout 0
 
 
So far, so good
 
But we are not there yet... We need to set iscsi properties, so that GRUB knows how to load the OS. They can be set at boot time as well as in a persistent configuration file.
 
For the boot arguments as well as persistence, follow the bottom instructions here:
 
To get to the place where you can enter the boot arguments for the first time, press E when you're at the startup menu where you can select to start ubuntu or choose advanced options for ubuntu.
 
Ensure that all your configs are correct and that both TPS and pyTarget have reloaded the configs. By now, you should be able to boot to desktop! :)

Edited by vsoraas, 09 October 2017 - 10:50 PM.

  • Brito likes this

#2 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 10 October 2017 - 05:25 PM

Hi Vsoraas,

 

Thanks for sharing, very good tutorial.

 

Some remarks :

 

About TPS

-next-server=192.168.1.10 is not needed - would be if your next-server was different from your proxy-dhcp server

-opt3=192.168.1.1 is not needed as well (leave this to your dhcp server) or set it to 0.0.0.0

-opt6=192.168.1.1 is not needed as well (leave this to your dhcp server)

In general, in proxy-dhcp mode, leave the main dhcp options to your dhcp server and focus on your pxe boot options only

 

About pytarget

I wonder what happens if you try host name="*" or host name="" - indeed, hardcoding the initiator name is not always handy and you then have to hardcode it again in your ipxe script - a constraint we dont have with other iscsi target softwares.

Very nice thus to write a tutorial based on pytarget !

 

About grub

i am surprised you had to fiddle with it - i never encountered this requirement - can you tell us more?

 

Thanks again for sharing.

 

Erwan



#3 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 10 October 2017 - 05:34 PM

Also, for clarity in your pxe script, you may want to re order and rename the labels (stage1-install-iscsi, then stage2-boot-iscsi).

 

Another minor detail : you probably dont need the instruction "boot" after "sanboot".

 

Suggest IPXE script modifications :

:stage1-install-iscsi
set boot-url http://archive.ubuntu.com/ubuntu/dists/xenial-updates/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64
kernel ${boot-url}/linux vga=788 --- quiet 
initrd ${boot-url}/initrd.gz
boot

:stage2-boot-iscsi
set gateway 0.0.0.0
# initiator-iqn must be same as HOST NAME under the TARGET in config.xml
set initiator-iqn iqn.2006-11.1 
set keep-san 1
sanboot ${iscsi-target}


#4 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 10 October 2017 - 07:42 PM

Note for later :

-not only can we use option 17 to set the iscsi target

-we can also use dhcp option 203 to set the client iqn : 203.4.65.65.65.65 -> initiator-iqn:string = AAAA



#5 vsoraas

vsoraas
  • Members
  • 2 posts
  •  
    Norway

Posted 11 October 2017 - 04:15 PM

About TPS

-next-server=192.168.1.10 is not needed - would be if your next-server was different from your proxy-dhcp server

-opt3=192.168.1.1 is not needed as well (leave this to your dhcp server) or set it to 0.0.0.0

-opt6=192.168.1.1 is not needed as well (leave this to your dhcp server)

In general, in proxy-dhcp mode, leave the main dhcp options to your dhcp server and focus on your pxe boot options only

 

I had input next-server only because it seemed suggested when using proxy-dhcp. I was unsure if I needed it, so it was there just in case. But I can remove it :)

 

The other values were populated into TPS automatically I think? Not by me manually anyway.

 

 

About pytarget

I wonder what happens if you try host name="*" or host name="" - indeed, hardcoding the initiator name is not always handy and you then have to hardcode it again in your ipxe script - a constraint we dont have with other iscsi target softwares.

Very nice thus to write a tutorial based on pytarget !

 

Yeah, pyTarget config is very poorly documented! The AuthMethod key-value pair to disable CHAP for example, I gleaned from pyTarget console only... it's not mentioned anywhere in the documentation (and no results for me on google). Maybe some time we will learn more about pyTarget and can document it somewhere.

 

 

About grub

i am surprised you had to fiddle with it - i never encountered this requirement - can you tell us more?

 

I'm actually not quite sure why it happened this way. I configured iSCSI before installing, so it's not like the installer shouldn't already be aware that I am installing to a remote drive. All I know is that after the regular install, GRUB installed also in its normal manner with no prompts or errors, and when it came time to reboot I only got initramfs shell with no possibility to boot to desktop.

 

ash shell is rather limited, but https://unix.stackex...a/127188/138805 show you to use command df to list partitions. I found no sign of the iSCSI volume, so I figured it had to be a failure to mount. It took some deliberation before I stumbled upon the solution in one of misty's write-ups about preparing GRUB for networked boot. I followed those instructions exactly and I achieved boot with the proper grub arguments, and then persistently by tweaking grub.cfg after a successful desktop boot. 

 


Edited by vsoraas, 11 October 2017 - 04:15 PM.

  • erwan.l likes this

#6 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 11 October 2017 - 04:58 PM

 

Yeah, pyTarget config is very poorly documented! The AuthMethod key-value pair to disable CHAP for example, I gleaned from pyTarget console only... it's not mentioned anywhere in the documentation (and no results for me on google). Maybe some time we will learn more about pyTarget and can document it somewhere.

 

 

pyTarget is a nice alternative: it is free, open source and portable.

Downside is that it is poorly documented and probably less performing compared to other solutions.

 

Starwind iscsi target is my favorite one but not portable and not open source (free thus in a limited thus plenty acceptable version).



#7 tinoy69

tinoy69

    Member

  • Members
  • 30 posts
  •  
    Philippines

Posted 12 October 2017 - 02:50 AM

Note for later :

-not only can we use option 17 to set the iscsi target

-we can also use dhcp option 203 to set the client iqn : 203.4.65.65.65.65 -> initiator-iqn:string = AAAA

 

how can i specify the iscsi port to option 17? i know that it defaults to 3260

 

can i specify the full uri rootpath (like the one in the ipxe script (sanboot command))?

 

server::port::iqn

 

like msdhcp here

http://ipxe.org/sanuri



#8 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 12 October 2017 - 09:05 AM

iscsi syntax is iscsi:192.168.1.1:tcp:3260:0:iqn.iSCSI-target1.

 

in your case i believe you can drop the iscsi prefix and therefore use 192.168.1.1:tcp:3260:0:iqn.iSCSI-target1



#9 tinoy69

tinoy69

    Member

  • Members
  • 30 posts
  •  
    Philippines

Posted 16 October 2017 - 03:51 PM

erwan, rootpath needs the full syntax, (with scsi: or aoe: prefix), already tested it, please do confirm if that is the case...

without "iscsi:" it gives out an error (mines, could not open san device: error 0x3f122003)

but with "iscsi:" it boots the image with no problem...


  • erwan.l likes this

#10 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 16 October 2017 - 04:13 PM

erwan, rootpath needs the full syntax, (with scsi: or aoe: prefix), already tested it, please do confirm if that is the case...

without "iscsi:" it gives out an error (mines, could not open san device: error 0x3f122003)

but with "iscsi:" it boots the image with no problem...

 

You are right.

I checked an old faq I have written here (see bullet #6) and indeed you need to use the full syntax e.g iscsi:192.168.1.100:tcp:3260:0:test






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users