kexec-loader changes

I am considering making a change to the way kexec-loader mounts filesystems and locates files. It currently mounts filesystems at the points you tell it, the same as a full Linux distro does, I may change the behaviour to be more GRUB like. For example:

rootfs /dev/sda1
mount /dev/sda2 /initrd
kernel /boot/vmlinuz
initrd /initrd/initrd.gz

would become:

kernel (sda1)/boot/vmlinuz
initrd (sda2)/initrd.gz

I can see the following pros/cons in this approach:

Pros:
Slightly simler to use.
Much cleaner code.

Cons:
Relative symlinks between filesystems cannot be used, although these are probably rare anyway.

Please post your thoughts on this change, if I don't any arguments for keeping the current code I will start changing it after 2009-01-20.

Looks good.

Support for UUIDs and LABELs would be nice too, so you can make a iso with the kexec-loader kernel only, while you put the config file on an USB drive. Then you can boot your USB drive on every PC, by booting from the CD. And it doesn't matter how the USB drive is detected (/dev/sda, /dev/sdb, /dev/sdc).

# UUID of a ext3 partition
kernel (777cf276-eb04-4c5f-a50c-5c251e76e8d1)/boot/vmlinuz
# UUID of a fat partition
initrd (47A3-186B)/boot/initrd.gz

# LABEL of a ext3 partition
kernel (Ubuntu Hardy)/boot/vmlinuz
# LABEL of a fat partition
initrd (MP3 PLAYER)/boot/initrd.gz

For UUID use, a blkid like command would be very useful to get the UUIDs of all partitions.

So if I understand right, we would end up with an generic kicker disc to boot any linux distribution on any device (no matter if IDE, SATA, USB, SCSI or whatever)?

...and as soon kexec supports Windows (if this will even happen) we can even boot this?

Sounds very interesting.

Support for UUIDs and LABELs would be nice too, so you can make a iso with the kexec-loader kernel only, while you put the config file on an USB drive. Then you can boot your USB drive on every PC, by booting from the CD. And it doesn't matter how the USB drive is detected (/dev/sda, /dev/sdb, /dev/sdc).

That's a good idea, (putting the kernel on a CD and the config on a USB drive). I might make ISOs in the future, although deciding which modules to include may be a problem.

I plan to add support for referencing filesystems by UUID/LABEL later.

So if I understand right, we would end up with an generic kicker disc to boot any linux distribution on any device (no matter if IDE, SATA, USB, SCSI or whatever)?

...and as soon kexec supports Windows (if this will even happen) we can even boot this?

Sounds very interesting.

It can already boot any Linux system, but certain parts, like the shell is very buggy right now. I've made a lot of improvements to the shell code which will be available in about 4 hours when the nightly build script runs, I'll post a link to the disk image when it's done.

Booting windows requires chainloading, somthing which I've tried implementing, but is very difficult due to how chainloading works. With the help of a kernel dev I managed to write a Linux patch which theoretically chainloads, but it crashes the system, we think it's because the BIOS disk number you're chainloading needs to be loaded into a register, this is very difficult from inside Linux, impossible for devices which the BIOS does not support.

You can boot windows already if you make a kicker disk (Fake Signature Method Image) for the windows installation.

Grub4dos (for use with kexec): grub.exe
Fake Signature Kicker Image for windows (gzipped): kicker.gz

kernel (sda1)/grub.exe --config-file="map (rd) (hd0); map --hook; rootnoverify (hd0); chainloader (hd0)+1"
initrd (sda1)/kicker.gz

The following is the command used for mapping the initrd (= (sda1)/fsmi.gz ), which can be accessed by using (rd) in grub4dos, to (hd0). The rest of the commands should be similar to the kicker disk for Windows commands (can contain errors, didn't test it myself):
map (rd) (hd0); map --hook; rootnoverify (hd0); chainloader (hd0)+1

Untill kexec-loader is updated, use (adapt /dev/sda1 if nessecary):

rootfs /dev/sda1

kernel /grub.exe --config-file="map (rd) (hd0); map --hook; rootnoverify (hd0); chainloader (hd0)+1"

initrd /kicker.gz

The forum wraps the code. it must look like:

rootfs ...

kernel ...

initrd ...

If solemnwarning is interested, the windows kicker disk image can be made on boot time. Because grub4dos accepts an initrd, kexec-loader can make it on the fly.

• It can mount (rw) an existing kicker disk from the USB hard drive.
• Some files (like boot.ini and the registry hives) can be updated on the fly in this image.
• This image must be passed as initrd to grub.exe.

It can grab the necessary files from the USB drive on boot time, so it always contains the last registry hives, so they don't get destroyed (maybe there is a solution for this problem already, but I didn't read the whole tread). kexec-loader can also make the fake signature on the fly so you can use kexec-loader to boot every Windows installation without making a kicker image for Windows for each new USB hard disk that you want to use.


The feature to make an initrd (file image formatted in fat13, fat32, ntfs or ext2) on the fly, by placing several files (the needed image files that you want to hook and the menu.lst file for grub4dos) in it, would be useful in combination with grub4dos. e.g.: boot dos based boot images from USB hard drive without USB support.

This doesn't have the changes I mentioned, but it does a lot of bugfixes/enhancements over the current release version:

kexec-loader-r424-floppy.img.gz

Stuff

I've read some of this thread:

http://www.911cd.net...showtopic=21242

If I'm understanding correctly this method can be used to boot NT from any hardware it has drivers for, if so I will write any required programs to maintain the stage1 image.

I'm still reading, but if anyone knows of a good tutorial to this and/or the NT boot proccess, can you give me a link?

A link to the tread for a tool to make the kicker disk image with winbuilder:
http://www.boot-land...amp;#entry54710

The file /ChainbootXP/Projects/ChainbootXP/ChainBootXP.script inside the zip will give you some info.

Can you add a link to your kexec-loader site in your signature?

I'm still reading, but if anyone knows of a good tutorial to this and/or the NT boot proccess, can you give me a link?

http://www.geocities.../nt_core18.html
http://en.wikipedia....startup_process
http://en.wikipedia....ws_Boot_Manager

Alex Ionescu, developer of TinyKrnl project developed an full open source replacement of the windows 2003 files involved in the initial boot process and is also a member here on boot land - http://www.boot-land...?showtopic=3791.

Maybe he can clear out in detail your doubts.

Does the filesystem of the stage1 image have to use the same format as the filesystem windows is installed on?

Does the filesystem of the stage1 image have to use the same format as the filesystem windows is installed on?

Yes.
cdob posted more info and links to source code at http://www.boot-land...amp;#entry54755.

idea #1: Maybe it would be good to have something like
menu.cfg /<uuid>/grub/menu.cfg?

idea #2: What about integrating this project after it's in production state into grub4dos?

kecec 1

# means that if booting linux (using kernel and initrd commands) will prefer to use kexec to boot it and not 'normal grub method'

title		Ubuntu 8.04.1, kernel 2.6.24-23-generic using grub

kexec 0

root			(hd0,1)

kernel		/boot/vmlinuz-2.6.24-23-generic root=UUID=288a980f-c95c-48c9-87a0-bdb4eb98ddb3 ro quiet splash

initrd		/boot/initrd.img-2.6.24-23-generic

quiet

title		Ubuntu 8.04.1, kernel 2.6.24-23-generic using kexec

kexec 1

root			(hd0,1)

kernel		/boot/vmlinuz-2.6.24-23-generic root=UUID=288a980f-c95c-48c9-87a0-bdb4eb98ddb3 ro quiet splash

initrd		/boot/initrd.img-2.6.24-23-generic

quiet

So it would become a real drop in replacement, you could easily try to switch the option.

That's a good idea, (putting the kernel on a CD and the config on a USB drive). I might make ISOs in the future, although deciding which modules to include may be a problem.

On floppy I wish modules for:
- CD-ROM
- USB
- then space will be full?

On CD I wish as much as modules as space is available (and as long it doesn't make troubles).
- any storage device
- USB keyboard / mouse
- linux encryption modules

I plan to add support for referencing filesystems by UUID/LABEL later.

Cool

Booting windows requires chainloading, somthing which I've tried implementing, but is very difficult due to how chainloading works. With the help of a kernel dev I managed to write a Linux patch which theoretically chainloads, but it crashes the system, we think it's because the BIOS disk number you're chainloading needs to be loaded into a register, this is very difficult from inside Linux, impossible for devices which the BIOS does not support.

I understand it's hard. Please don't overwork yourself while trying to establish the "kexec supports now windows" patch project. Imho this "kexec supports now windows" patch is a whole other project then "kexec as grub linux booting drop in replacement".

You can boot windows already if you make a kicker disk (Fake Signature Method Image) for the windows installation.

I've read some of this thread:

http://www.911cd.net...showtopic=21242

If I'm understanding correctly this method can be used to boot NT from any hardware it has drivers for, if so I will write any required programs to maintain the stage1 image.

Yes, this method is very nice because it can have great success for starting Windows.

Real chainloading is still a good anyway, because it's theoretical more generic (support not only one version of Windows, but any operating system) and could even work together if full disk encryption is involved.

idea #1: Maybe it would be good to have something like
menu.cfg /<uuid>/grub/menu.cfg?

What?

idea #2: What about integrating this project after it's in production state into grub4dos?

*examples*

Integrating them isn't really feasible as they have totally different designs/purposes, grub4dos runs in real mode and only accesses BIOS drives, the same as GRUB, whreas kexec-loader runs under Linux and accesses everything the Linux block and vfs layer supports.

On floppy I wish modules for:
- CD-ROM
- USB
- then space will be full?

The floppies already have USB support built into the kernel, otherwise using a USB floppy drive would be impossible.

I understand it's hard. Please don't overwork yourself while trying to establish the "kexec supports now windows" patch project. Imho this "kexec supports now windows" patch is a whole other project then "kexec as grub linux booting drop in replacement".

Due to the number of programs required to generate stage1 images for the fake signature method, it may not be possible for me to embed them in kexec-loader without taking up all the module space on the floppy. (mkntfs, etc)

Maybe you have though about it already, but just in case that you didn't think about it, you can use ntfscp (part of ntfsprogs) to copy files to a ntfs partition without mounting it (so no need for ntfs-3g and fuse ==> less space needed):
http://www.linux-ntfs.org/doku.php
http://man.linux-ntf...g/ntfscp.8.html

It isn't necessary that all programs that you need, are available in the floppy image. You can load them from the USB drive once kexec-loader has been booted, if necessary. Or you can also provide a 2.88MB floppy image that can be booted by memdisk or grub4dos from the internal hard drive which includes every module and program.

About the chain loading stuff, I think it is better to boot grub4dos from kexec-loader to do something like that. It is more designed to be a general boot loader.

What?

grublegacy has a command called configfile. You can include another configuration execute it.

I wish to use for kexec-loader simply
configfile <uuid-of-device, device label or linux device>/GRUB/menu.cfg.

This way kexec-loader could be used as a drop in replacement for grub. kexec-loader could use exactly the same menu entry like grub.

Integrating them isn't really feasible as they have totally different designs/purposes, grub4dos runs in real mode and only accesses BIOS drives, the same as GRUB, whreas kexec-loader runs under Linux and accesses everything the Linux block and vfs layer supports.

You are right. grub4dos is a real mode application and the menu and the bootloader stuff is all in one?

While kexec boots into Linux and then pops up the menu.

Would be pretty hard to separate the menu in realmode and to load the correct module (+ Linux).

Indeed, it's only a cool feature but not really a top priority. More important is the ability to chainload kexec-loader from grub4dos, but I guess that won't become a problem as it's using linux kernel format.

The floppies already have USB support built into the kernel, otherwise using a USB floppy drive would be impossible.

Ah sorry, I've forgotten to add:
- IDE / SATA support
(currently not in?)

Due to the number of programs required to generate stage1 images for the fake signature method, it may not be possible for me to embed them in kexec-loader without taking up all the module space on the floppy. (mkntfs, etc)

If space is a matter then reserve this feature to to iso edition only.

grublegacy has a command called configfile. You can include another configuration execute it.

I wish to use for kexec-loader simply
configfile <uuid-of-device, device label or linux device>/GRUB/menu.cfg.

This way kexec-loader could be used as a drop in replacement for grub. kexec-loader could use exactly the same menu entry like grub.

I'll probably implement a command like that in a later version

Ah sorry, I've forgotten to add:
- IDE / SATA support
(currently not in?)

The kernel I use for the floppy supports PC floppy, USB and FAT, everything else can be added from the modules tarball on the download page.

It can grab the necessary files from the USB drive on boot time, so it always contains the last registry hives, so they don't get destroyed (maybe there is a solution for this problem already, but I didn't read the whole tread). kexec-loader can also make the fake signature on the fly so you can use kexec-loader to boot every Windows installation without making a kicker image for Windows for each new USB hard disk that you want to use.

Currently I'm parsing registry to detect early boot drivers.
Can you parse registry too?

If not, a fixed list can be prepared.
A fixed list won't match all configurations. However a almost all working solution should be possible.

grublegacy has a command called configfile. You can include another configuration execute it.

grub4dos has the configfile command as well.

The problem with the uuid approach is that you also need a command similar to root to let GRUB/grub4dos know on "which" drive it is working, and be able to use as much as possible "relative" paths (easier to copy and paste between different menu.lst's and generally more "elegant".

Using a simple "tag" file, you can already use the configfile command together with the root one.
So all it's needed is a single, 0 byte, tag file, that possibly can have the name=uuid on the root of the drive, or even more practical, rename the anyname.lst to <uuid>.lst I guess that solemnwarning can write a small tool to create a "uuid-named" file in the root of any drive.

A "general" menu.lst like the following:

title DriveUUID

find --set-root <uuid>.lst

configfile <uuid>.lst

will allow for selecting the "right" drive and menu.lst

...just an idea, mind you.

Indeed, it's only a cool feature but not really a top priority. More important is the ability to chainload kexec-loader from grub4dos, but I guess that won't become a problem as it's using linux kernel format.

Cross-loading of kexec-loader and grub4dos was already tested and it works nicely:
http://www.boot-land...?showtopic=4636

jaclaz

I have no idea how to parse the registry from outside windows, I don't think I know anywhere near enough about NT to implement a stage1 builder.

On the UUID/label front, I can now get the UUID from an ext and the label from an ext/vfat filesystem, but I've not managed to find any others yet. Does anyone know where to find these values in other filesystems, their limits, how they're terminated, etc?

On the DOS/NT side (read FAT16/FAT32/NTFS), apart the MBR Disk Signature, which identifies the PHYSICALDRIVE and not the partition, there are two values, LABEL and Volume Serial.
You can find detains on where to find them in the bootrecord of the partitions on the Starman's pages:
http://mirror.href.c.../mbr/NTFSBR.htm
http://mirror.href.c...mbr/MSWIN41.htm

a FAT16 bootsector is here:
http://www.ntfs.com/...tion-sector.htm

jaclaz

On the UUID/label front, I can now get the UUID from an ext and the label from an ext/vfat filesystem, but I've not managed to find any others yet. Does anyone know where to find these values in other filesystems, their limits, how they're terminated, etc?

Use libblkid (http://e2fsprogs.sourceforge.net/) to get the UUID and LABEL Or you can use blkid from busybox.

Use libblkid (http://e2fsprogs.sourceforge.net/) to get the UUID and LABEL Or you can use blkid from busybox.

That looks like it's only for ext filesystems?

If you download the e2fsprogs-1.41.3 archive and extract it to ./e2fsprogs-1.41.3/

If you do grep "static int probe" * -A2 in ./e2fsprogs-1.41.3/lib/blkid:
[codebox]devname.c:static int probe_all(blkid_cache cache, int only_if_new)devname.c-{devname.c- FILE *proc;--probe.c:static int probe_ext4dev(struct blkid_probe *probe,probe.c- struct blkid_magic *id,probe.c- unsigned char *buf)--probe.c:static int probe_ext4(struct blkid_probe *probe, struct blkid_magic *id,probe.c- unsigned char *buf)probe.c-{--probe.c:static int probe_ext3(struct blkid_probe *probe, struct blkid_magic *id,probe.c- unsigned char *buf)probe.c-{--probe.c:static int probe_ext2(struct blkid_probe *probe, struct blkid_magic *id,probe.c- unsigned char *buf)probe.c-{--probe.c:static int probe_jbd(struct blkid_probe *probe, struct blkid_magic *id,probe.c- unsigned char *buf)probe.c-{--probe.c:static int probe_fat(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_fat_nomagic(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_ntfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_xfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_reiserfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id, unsigned char *buf)probe.c-{--probe.c:static int probe_reiserfs4(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_jfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_zfs(struct blkid_probe *probe, struct blkid_magic *id,probe.c- unsigned char *buf)probe.c-{--probe.c:static int probe_luks(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_romfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_cramfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_swap0(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf __BLKID_ATTR((unused)))--probe.c:static int probe_swap1(struct blkid_probe *probe,probe.c- struct blkid_magic *id,probe.c- unsigned char *buf __BLKID_ATTR((unused)))--probe.c:static int probe_iso9660(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_udf(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf __BLKID_ATTR((unused)))--probe.c:static int probe_ocfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_ocfs2(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_oracleasm(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_gfs(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_gfs2(struct blkid_probe *probe,probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_hfs(struct blkid_probe *probe __BLKID_ATTR((unused)),probe.c- struct blkid_magic *id __BLKID_ATTR((unused)),probe.c- unsigned char *buf)--probe.c:static int probe_hfsplus(struct blkid_probe *probe,probe.c- struct blkid_magic *id,probe.c- unsigned char *buf)--probe.c:static int probe_lvm2(struct blkid_probe *probe,probe.c- struct blkid_magic *id,probe.c- unsigned char *buf)[/codebox]So it seems to be able to get the UUID and LABEL from a awful lot of different file systems.If you look in [b]/e2fsprogs-1.41.3/lib/blkid/probe.c[/b], you can see the following at the end of the [i]static int probe_ntfs(struct blkid_probe *probe, struct blkid_magic *id __BLKID_ATTR((unused)), unsigned char *buf)[/i] code:[code]sprintf&#40;uuid_str, &#34;%016llX&#34;, blkid_le64&#40;ns->volume_serial&#41;&#41;;blkid_set_tag&#40;probe->dev, &#34;UUID&#34;, uuid_str, 0&#41;;if &#40;label_str&#91;0&#93;&#41; blkid_set_tag&#40;probe->dev, &#34;LABEL&#34;, label_str, 0&#41;;return 0;[/code]
I can't program in C, but it seems to me that this will return the UUID and LABEL of the NTFS partition.

edit by mr_,
-offtopic and solved-

New version here:
http://www.boot-land...?showtopic=6989

jaclaz