Jump to content











Photo
- - - - -

Ubuntu 16.04 - boot from hdd using grub4dos with persistence


  • Please log in to reply
33 replies to this topic

#1 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 04:44 PM

I have successfully created an environment with a laptop so that I can dual boot between windows 7 and ubuntu desktop 16.04 using grub4dos.
It is fully documented on another topic here:
http://reboot.pro/to...un-it-from-hdd/
All is working fine. Both windows 7 and ubuntu can boot up properly and I can use them with no issues.
But I am still missing something on ubuntu: persistence across reboots.
When I try to add persistence to the ubuntu iso through the initramfs options it fails miserably on me by simply booting without the persistence file mounted.
As I have stated before, I am running things on different partitions so I will make a quick drawing of what the partitions/folders/files currently look like.

500GB hdd
|
|_100MB (system partition)
|   |_______grub (folder)
|   |         |_grldr (file)
|   |         |_menu.lst (file)
|   |_boot.ini (file)
|
|_250GB (boot partition aka C:)
|   |_Windows OS
|
|_240GB (data/baks partition aka D:)
|   |_______OS (folder)
|   |        |________live (folder)
|   |        |          |_________ubuntu-rw (folder)
|   |        |          |             |_casper-rw (file)
|   |        |          |
|   |        |          |_ubuntu-16.04.1-desktop-amd64.iso (file)
|   |        |          |_manjaro-bspwm-community-16.06-x86_64.iso (file)
|   |        |          |_etc...

And this is how each file looks like right now.

file -> boot.ini
[boot loader]
Timeout=30
default=C:\grub\grldr
[operating systems]
C:\grub\grldr="grub4dos"
file -> menu.lst
title Back to Windows Boot Manager
find --set-root /bootmgr
chainloader /bootmgr
title Ubuntu
find --set-root /OS/live/ubuntu-16.04.1-desktop-amd64.iso
map /OS/live/ubuntu-16.04.1-desktop-amd64.iso (0xff) || map --mem /OS/live/ubuntu-16.04.1-desktop-amd64.iso (0xff)
map --hook
root (0xff)
kernel /casper/vmlinuz.efi file=/cdrom/preseed/ubuntu.seed noprompt boot=casper iso-scan/filename=/OS/live/ubuntu-16.04.1-desktop-amd64.iso persistent persistent-path=/OS/live/ubuntu-rw quiet --
initrd /casper/initrd.lz

I would post a journalctl -b0 but it is too long and the forum doesn't allow it.

Also ubuntu 16.04 seems to have ditched /var/log/boot.log so I am clueless of what to do to debug this.

Please let me know if you spot something wrong.

Bear in mind that this same iso file is also on a usb easy2boot pendrive and it is working just fine with persistence. This means it's not a bug in this release or something and since easy2boot uses grub4dos I guess it also rules out any grub4dos influence.

Thanks in advance.

Cheers

 

PS: oops, forgot to put the tags in the post and now the forum won't let me put them when I edit the post.


Edited by john3voltas, 25 September 2016 - 04:45 PM.


#2 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13596 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 September 2016 - 05:27 PM

Does the "casper-rw" work on NTFS formatted volumes? :dubbio:

http://askubuntu.com...ges-on-disk-hdd

 

:duff:

Wonko



#3 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 05:46 PM

I guess it does work on NTFS. I mean, I also have a easy2boot pendrive formated in NTFS and casper-rw is working just fine on the pen.

The easy2boot pen has so many menu.lst that I can't understand how it is working on the pen... :-(



#4 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13596 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 September 2016 - 05:58 PM

I guess it does work on NTFS. I mean, I also have a easy2boot pendrive formated in NTFS and casper-rw is working just fine on the pen.

The easy2boot pen has so many menu.lst that I can't understand how it is working on the pen... :-(

Well, it must have not that many lines containing the word "persistent" or "casper", a grep or similar on *.lst should find them alright.

 

:duff:

Wonko



#5 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 25 September 2016 - 06:24 PM

Easy2Boot creates a partition entry for the casper-rw file using grub4dos partnew command.

When linux boots, it just sees the partition entry in the partition table and mounts the partition as a normal ext filesystem.

In this way, the kernel does not need to support NTFS or other filesystems because it just sees an ext filesystem.

The initial kernel does not support NTFS, so you need to create a FAT32 volume and store the file in there (or create an ext3 partition).



#6 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 06:29 PM

Hi steve6375

Thanks for joining us.

I don't know if I could follow your line of thought.

You said that using partnew in grub4dos, ubuntu then simply sees the ext3 partition without any trouble.

But then you say that the initial kernel (ubuntu's?) does not support NTFS...

What am I reading wrong? :dubbio:



#7 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13596 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 September 2016 - 06:36 PM

Hi steve6375

Thanks for joining us.

I don't know if I could follow your line of thought.

You said that using partnew in grub4dos, ubuntu then simply sees the ext3 partition without any trouble.

But then you say that the initial kernel (ubuntu's?) does not support NTFS...

What am I reading wrong? :dubbio:

You are not reading anything wrong :), you simply do not know (yet ;)) what partnew does, see:

http://reboot.pro/to...brided/?p=88531

 

Basically the extents of the casper-rw file are entered in the MBR partition table and the Linux doesn't use the casper-rw file, but rather the casper-rw (fake) partition.

 

:duff:

Wonko



#8 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 25 September 2016 - 06:44 PM

You are not reading anything wrong :), you simply do not know (yet ;)) what partnew does, see:

http://reboot.pro/to...brided/?p=88531

 

Basically the extents of the casper-rw file are entered in the MBR partition table and the Linux doesn't use the casper-rw file, but rather the casper-rw (fake) partition.

 

:duff:

Wonko

Yes, E2B does not use the persistent-path=xxxx cheat code - Ubuntu just looks for a partition with a volume name of casper-rw and uses that as the persistent volume.



#9 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 06:55 PM

Interesting reading Wonko, but don't forget that I am not experienced enough to grasp all mentioned in the post you linked.

With that said, I must emphasize that in the past I have had a laptop with Windows XP, grub4dos (renaming grldr to ntldr) with 2 NTFS partitions and dualbooting from windows into several flavors of Linux (Puppy Slacko, AlphaOS and a Debian-based frugal Debian called Pussy IIRC) and I never had to have FAT32 partitions...

The only difference is that instead of loading the ISO, I would rather extract it to a folder and then issue the usual kernel and initrd commands in grub4dos.

Can't understand why now I can't use NTFS when I did a few years ago. Unfortunately that was all set up by a friend who is no longer with us and I don't know what he did because I don't have that laptop any longer.

Are we positive that there is an issue with persistence and NTFS? :(

 

EDIT: okay, I've searched and found evidence that ubuntu has (or at least had, there's no date on the wiki docs where I read this) a problem with persistence and NTFS.

Also, I have found evidence that for instance Puppy can be completely run off a NTFS partition with a persistence file.

http://murga-linux.c...p=620229#620229

 

EDIT2: Porteus can do it on NTFS too. Maybe it's an issue with ubuntu only or maybe with debian derivatives only.

http://www.porteus.o...v-10.html#App-B


Edited by john3voltas, 25 September 2016 - 07:24 PM.


#10 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 07:03 PM

Yes, E2B does not use the persistent-path=xxxx cheat code - Ubuntu just looks for a partition with a volume name of casper-rw and uses that as the persistent volume.

steve6375, I was writing when you posted.

What exactly is preventing me to achieve with an hdd and grub4dos what you can achieve with a usb pendisk and easy2boot?

I mean, easy2boot doesn't create ext partitions on my usb pendisk. Both are using grub4dos and both have NTFS as the base filesystem.

I stand clueless... :(

 

EDIT: I just edited my previous post with some important information.


Edited by john3voltas, 25 September 2016 - 07:20 PM.


#11 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13596 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 September 2016 - 07:20 PM

Windows considers the 00 Partition ID as "not a partition".

 

Linux instead, considers the 00 Partition ID as "a partition" as long as the CHS and LBA data (I believe actually only the LBA data) in the partition entry point to an actual filesystem that it can recognize.

 

If you think about it a partition is defined in the MBR by:

1) an ID <- which is a "protective ID"

2) a start address for a contiguous extent on the disk

3) a length for the contiguous extent on the disk

 

Nothing prohibits to have the SAME extent mapped to a file in a filesystem (if not the risk of deleting accidentally something due to this "overlap").

 

The other way round, your file "casper-rw" is a (contiguous) extent on disk, indexed with the name "casper-rw" in the NTFS filesystem.

 

Nothing prevents from mapping the same extent as a partition in the MBR (Windows won't see this mapping).

 

Of course it would be not very smart to - say - defrag the NTFS volume from the booted UBUNTU or deleting the casper-rw file from the filesystem ...

 

:duff:

Wonko


  • imnothing likes this

#12 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13596 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 25 September 2016 - 07:28 PM

 

EDIT: okay, I've searched and found evidence that ubuntu has (or at least had, there's no date on the wiki docs where I read this) a problem with persistence and NTFS.

 

 

Really? :unsure:

Does the "casper-rw" work on NTFS formatted volumes? :dubbio:

http://askubuntu.com...ges-on-disk-hdd

 

In that there is a 2015 date.

 

:duff:

Wonko



#13 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 25 September 2016 - 07:35 PM

steve6375, I was writing when you posted.

What exactly is preventing me to achieve with an hdd and grub4dos what you can achieve with a usb pendisk and easy2boot?

I mean, easy2boot doesn't create ext partitions on my usb pendisk.

err. yes it does! E2B creates an entry in the partition table (ID type 0) which points to the file.

 

Try this.

Run E2B with persistence using the .mnu file so you boot to a persistent linux.

Shutdown linux

Now - without booting to E2B again - run RMPrepUSB and do a Drive Info - 0 

You should see that the third entry in the partition table (casper file) points inside the E2B partition and the fourth entry (ISO file) also points inside the E2B partition.

These entries are wiped when you boot to E2B (as they are ID type 0, it is safe to delete them).

 

So you could do the same (here be dragons!), but you must have an empty table entry in the partition table of your hdd.



#14 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 07:35 PM

First of all, thank you all for your patience. I can be very stubborn...on occasions :P

Let me see if I get this straight. Where is my limitation?

Is this limitation supposed to be ubuntu's lack of NTFS support for persistence files?

Is it grub4dos that might not support it?

Could it be both?

Right now my understanding is that this is all up to ubuntu's lack of support for persistence on NTFS but I am just trying to make sure I didn't miss something you've been trying to tell me.

TIA

Cheers



#15 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 07:38 PM

So you could do the same (here be dragons!), but you must have an empty table entry in the partition table of your hdd.

 

I guess that would be a bit too risky on my behalf.

Now I see the difference between E2B and my setup. :)



#16 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 25 September 2016 - 07:45 PM

 

Right now my understanding is that this is all up to ubuntu's lack of support for persistence on NTFS.

 

yes - some boot kernels do support NTFS file systems and some don't.

For instance, for some linux distros, they will only boot on FAT32 volumes, if you extract the files from an ISO and placed them on an NTFS volume and booted using grub4dos, it would not find the squashfs file because it would be on an NTFS volume which it cannot mount. In the case of Ubuntu, I think it just has a problem with the casper file being needed at an early stage??

This is the main reason why utilities such as YUMI which extract files from the ISO and support persistence, use FAT32 only.

 

https://sites.google...utorials/ubuntu


Edited by steve6375, 25 September 2016 - 07:50 PM.


#17 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 25 September 2016 - 08:04 PM

From the link posted by Wonko

The persistent-path does not allow any explicit disk reference, so should be unique across all FAT partitions. Tested with 1 or 2 FAT partitions (one being the EFI partition). Will not work on an ext2 or ntfs filesystem instead of FAT. If you also add the "toram" word on the same line as "persistent", your compressed filesystem on the slow USB will be copied into ram and give much better performance, however, there seems to be a shutdown issue, with the FAT partition not being cleanly unmounted (which does not seem to cause any problems but...)

 

 

 

That it won't work on an ext2 partition seems odd to me (as it must be capable of booting from ext2)? Maybe, it is just scripted to look at FAT partitions?

 

Some distros which support the iso-filename type of cheat code can boot from an ISO file on a FAT32 drive but not an NTFS drive because it cannot find the ISO file on an NTFS drive - e.g. deftz, Fedora_live_security, NodeZero, CentOS, Gentoo, Sabayon.

 

P.S. I edited my post above, as I think Ubuntu can find the squashfs on an NTFS volume.



#18 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 08:44 PM

P.S. I edited my post above, as I think Ubuntu can find the squashfs on an NTFS volume.

 

You mean that I could try and extract the vmlinuz and initrd - instead of just asking grub4dos to load the iso - and that might just work?



#19 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 25 September 2016 - 10:16 PM

It will boot (probably), but persistence won't work.



#20 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 25 September 2016 - 10:20 PM

It will boot (probably), but persistence won't work.

 

Sorry, I was under the impression that *that* way it would work...

I guess I'll have to find something else that works.

Do you know if Debian is supposed to work? Debian being the upstream of ubuntu I guess it shouldn't, right?

Cheers



#21 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13596 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 26 September 2016 - 07:12 AM

Out of curiosity, why not Porteus?

 

:duff:

Wonko



#22 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 26 September 2016 - 07:21 PM

Of course, why not.
Well, I've never fiddled with Porteus, and the truth is I've never fiddled with Ubuntu either.
But Ubuntu is Debian-based which I still remember a bit from the days of PussyOS whereas Porteus is Slackware-based and I only remember slackware from...1998...
Never seen any of those booting up from their ISO with persistence.
The only ones I know reasonably well on a grub4dos+persistence are Puppy which without a proper package manager doesn't fill my needs and then there's both PussyOS and AlphaOS. The first is abandonware and the second is abandonware but no one told it. It's userbase is less than 20 users.
So I guess I should listen to others opinions on this matter. An OS that I can copy the ISO to an NTFS partition, boot it up with grub4dos and have changes persistence across reboots.
I've been testing Porteus since last night but still haven't managed to even boot it up. And then there is still the persistence...
Any good tips with Porteus or any other major desktop distros like Fedora, Debian, Manjaro, Mint, LXLE, etc?
TIA
Cheers

#23 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13596 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 27 September 2016 - 09:09 AM

I am not much into Linux distro's but Ed (Ed_P over at 911cd.net) has a particular liking for Porteus, so I believe it is a "viable" distro.

 

The ways to boot it via grub4dos have been detailed on there and on Porteus Forum:

http://forum.porteus...php?f=75&t=3384
http://forum.porteus...php?f=48&t=3425

That should cover the basics.

 

:duff:

Wonko



#24 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 27 September 2016 - 09:22 AM

E2B has a Sample .mnu file for Porteus at \_ISO\docs\Sample mnu files\linux\Porteus-v3_Persistent.mnu



#25 john3voltas

john3voltas

    Member

  • Members
  • 87 posts
  •  
    Portugal

Posted 09 October 2016 - 01:54 PM

Steve6375, is there anyway I can boot from a E2B pendisk but simply point to the iso and to the persistence file on a ntfs volume?

I don't have a problem in booting from my E2B pendrive however I do have an issue with it being so slow. If the bootup to grub4dos was to be accomplished on the pendrive, would it be possible to point it to a casper-rw persistence file which is on a ntfs filesystem?

Thanks and sorry for my persistence with this issue ;)






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users