33 replies to this topic

[RandomUser]

Well, this is embarrassing. I need help with the grub4DOS error that reads "inconsistent filesystem" and found this topic very similar to the error that I have and did not want to revive such an old thread. Except that thread is for using USB drive, I am having this problem on an internal SAS drive.

All of my old images loads great in Grub4DOS, however all newly created images, Grub4DOS throws out the aforementioned error. Of course all new images, when created get stored on the very end of all the other files stored in a free sector space. Meaning towards the end of the hard drive, where the older images are stored closer to the beginning of the hard drive. So perhaps grub4dos cannot see past 1.6 or 1.7TB of data?

My hard drive is 4TB and it is 1.7TB full and it's formatted as GPT NTFS file system.

Using Chenall grub4dos-0.4.6a-2016-03-26.

Using a separate hard drive as a grub4dos booter drive formatted as FAT32 file system, and is able to load images off of the NTFS formatted drive specified above.

Thanks in advance.

[Wonko the Sane]

You have a 4Tb hard disk and you made a single huge 4 Tb NTFS partition on it? Really?

 

Which specific grub4dos command creates the error?

 

Or HOW EXACTLY are you attempting to boot those images?

 

And HOW EXACTLY are you having the grub4dos grldr load from the GPT style disk? (Hybrid MBR, chainloaded from something else, etc.)

 

Basically an image is an "extent" on the disk (if contiguous) or a series of extents if fragmented, there is no real reason why a "same" grub4dos can access an extent (or a multi extent) past a given size when booted from the "other" (smaller, FAT32) device, unless - maybe - the way it is loaded makes a difference.

 

Can you try booting "normally" the grldr on the huge GPT disk/NTFS partition, from it chainload the grldr on the small (I presume MBR) disk/FAT32 partition and from the latter try loafing those same images and report what happens?

 

I mean, maybe is an issue with boot disk or "first" hard disk?

 

Wonko

[RandomUser]

Actually the 4TB disk has two extra partitions, one for system partition, one for efi partition, and the rest of the disk is data partition where the OS resides.

grldr loads completely off the full MBR FAT32 booter drive, which is the first disk it boots off of on the system. If I want to use the 4TB persistent OS, I use grub4dos to load clover and with that I use it to boot the OS, I am finding myself using RAMdisk booted OS more often then the OS on the 4TB drive as I liked the fact it gives a fresh install feels every time. I mainly use the OS on the 4TB drive when I need to use the password saved on it for (important) site logins that requires it, such as a banking institution websites.

The reason I did it this way, is due to grub4dos back in the days did not support GPT disk and this was the workaround, and am to lazy to change the setup.

All the images are in one contagious chain, however multiple images are stored different area of the hard drives.

The setup is really basic and it works well. as follows:

title Acronis 2015
map --top --mem (hd1,2)/vrd/Acronis2015.gz (hd0)
map --hook
rootnoverify  (hd0,0)
chainloader  (hd0,0)/bootmgr

Or It could be:

map --top --mem (hd1,3)/vrd/Acronis2015.gz (hd0)

map --hook
rootnoverify  (hd0,0)
chainloader  (hd0,0)/bootmgr

These command works great on older images made that are stored closer to the beginning of the hard drive and the newer images stored closer to the end however does not. Now if I were to throw the newly created image to the FAT32 drive by simply copy from 4TB drive and pasting it to FAT32 drive, it will load without a problem.

Heck even copying the older image and pasting it on the same 4TB drive, the pasted image makes grub4dos throw out the error.

I tried running chkdsk /F on the 4TB drive and it came out clean. The command above is finding the images from the 4TB GPT drive.

Perhaps I have an underlying hardware issue somewhere?

[steve6375]

grub4dos would use BIOS calls to access the disk.

So wouldn't this limit it to 2TB?

[steve6375]

Might be interesting to try

blocklist (hd1,3)/vrd/Acronis2015.gz

and see what is returned?

[RandomUser]

Now that I think about it, When I first put together this computer and bought the 4TB hard drive, it would not even detect it, so I had to flash a new firmware to the SAS controllers, to be able to use the full 4TB. The SAS controller is integrated to the motherboard, and so I guess that it has a separate firmware from the BIOS, but it is called MPT 2 BIOS. All SATA drives and such goes through BIOS, so since grub4dos uses BIOS calls, I did indeed hit the 2TB limit even though I am using less then 2TB of space or would grub4dos uses LSI SAS calls? This would explain the error I have been getting, but it's weird how it can read previous image from the 4TB drive.

Tried the blocklist (hd1,3)/vrd/Acronis2015.gz command, and the same error as before.

This is a very weird issue, perhaps I will have to buy another smaller drive just for images storage?

Thanks again to all.

[steve6375]

You need to take into account the preceding partitions

hd1,3 implies 4 partitions (you implied there were only 3?), so the file will be at 

 

ptn4 start + offset to file.

 

blocklist will return the sector position(s) (+ size) relative to the start of the partition. Try it on a file that works on that partition.

 

You can maybe use a disk editor to find where each file actually starts (and ends) on the disk?

[RandomUser]

Okay, tried the command on a known working file and here is the result:

(hd1,3)3405929744+32466127

This was tested on an image that was last created and working.

now, the Acronis2015 image as per above post, while grub4dos wouldn't give me a read out, but found a utility that will and the result is as follows:

3606670320-3607462591
or converted to grub4dos format 3606670320+792271

As for the partition, Windows disk manager shows only 3 partitions, and forgot there is a hidden MSR partition that the manager cannot see so (hd1,3) is correct.

As you can see the Acronis image is way closer towards the end of the hard drive.

[steve6375]

3607462591  == 1720GB, but I think that is relative to the start of the 4th partition.

Do you know where the partition starts? (You can use RMPrepUSB - Drive Info to list the partitions or some other MBR editor.)

[steve6375]

The first file ends at 1639GB (relative to start of ptn4).

[RandomUser]

Okay, found out using Partition Guru that the starting sector of Partition 4 is 1081344.

How do you calculate the size in GB from block?

[steve6375]

a sector is 1/2K, so divide by 2048 to get MB.

Highest sector possible for BIOS access is FFFFFFFF or 4294967295 which is much higher than 3607462591 so not sure why that fails but 3438395871 is OK...

[RandomUser]

Thank you.

Yeah, it is baffling. I did found this and while I don't use those filesystem let alone Linux, I did however suffered a catastrophic hard drive failure and the failure was sneaky too, My data got corrupted during a defrag when the defragger moved the data to the bad part of the disk and that bad part of the disk was growing in size. SAS SMART is different from SATA SMART, so the bad sector reporting is not obvious. Good thing I had a backup, so after RMAing the drive and receiving the replacement drive I cloned the drive from my backup. I wonder if this is similar to what's in the link? Awww cra*, I really don't want to have to reformat the drive, especially if it doesn't fix my issue LOL.

[Wonko the Sane]

Okay, found out using Partition Guru that the starting sector of Partition 4 is 1081344.

How do you calculate the size in GB from block?

Which would mean that the three first partition combined use 1081344*512=553648128 bytes or roughly 500 Mb, are you sure of that?

(it sounds to me a tadbit on the "low" side, usually the MSR partition is 128 Mb, the "system" partition (the compulsory FAT32 bit EFI one for "boot") between 100 and 300 Mb, how big is the third one?)

Can you run in grub4dos this command?

cat --hex --skip=28 --length=8 (hd1,3)0+1

and post result?

(this will output the "sectors before" in the NTFS bootsector of the fourth partition)

 

Wonko

[RandomUser]

Just a little over 500MB, more like 527MB based off of the partition guru on those multiple partitions. The third partition is the EFI partition and that is 99MB. I let the Windows setup automatically created all the partitions, and I don't know if it makes a difference, I used the server version of Windows installer. I needed the Windows Server as that was the only version that supports more than two physical CPUs.

Anyhoo I ran that command in grub4dos and here's the output:

0x0000001C: 00 80 10 00 00 00 00 00

Edited by RandomUser, 12 April 2016 - 09:05 PM.

[karyonix]

That confirms the start of partition at 553,648,128 byte. Let's calculate position of both file on disk.
 

| -                | start sector |   end sector |    start byte |      end byte |
| working file     |   3407011088 |   3439477215 | 1744389677056 | 1761012334080 |
| non-working file |   3607751664 |   3608543935 | 1847168851968 | 1847574494720 |

BIOS' ability to read between 1.7 TB and 1.9 TB should not be different.

Maybe MFT record of the non-working file or directory is beyond 2TB and either BIOS or GRUB4DOS cannot read past 2TB, so metadata about the file could not be read.
MFT is usually around the middle of NTFS partition when formatted. It is likely that MFT starts below 2TB and crosses the 2TB boundary.

BIOS INT13h Extensions use 16-byte Disk Address Packet structure.
starting_LBA field is 64-bit.
The API can support disk larger than 2TB. But BIOS implementation may not support it.

I am not sure whether GRUB4DOS can use higher than 32-bit sector LBA when BIOS support it.

[RandomUser]

^This, I didn't think about the MFT record going across the boundary. So my solution should be simple then. Just buy another hard drive, which is doable, because the smaller hard drives are not expensive, and my computer has plenty of space for them. I wouldn't have to mess with my primary hdd by doing this. Or at least it's possible that grub4DOS does not support 48-bit LBA or 64-bit LBA. Perhaps this is the reason why I am experiencing the issue? Does anyone other then me uses 4TB hard drive or more to be able to test this?

Edited by RandomUser, 13 April 2016 - 03:42 AM.

[steve6375]

You can try

blocklist /$MFT

To see if where the MFT is.

[Wonko the Sane]

Yep, confirmed , 0x108000 means 1081344 or 553648128 bytes before.

 

Typically a $MFT is not "in the middle" of the NTFS volume, for any volume bigger than 5 or 6 Gb it goes normally on cluster 786432, and given that a cluster is normally 8 sectors the start of the $MFT should be 6291456 sectors, aka 3221225472 bytes or roughly 3 Gb into the volume.

 

With such a large volume (almost 4 Tb ) it is however possible that the $MFT behaviour changes and the $MFT is farther inside the volume.

 

Just for the record, there is a trick (or two) to force a $MFT to start very early (sometimes used for large images on small sticks) see:
http://reboot.pro/to...disk-emulation/

but I don't think it can be done without re-formatting the volume (with built-in tools) and personally I wouldn't risk to see if a third party tool works on such a big volume.

The added small hard disk (though not particularly "elegant") seems to me like the easiest and safest workaround.

 

Wonko

[tinybit]

@karyonix

I think grub4dos already has support for 64-bit LBA. If you found it buggy, you may fix it. The problem is, whether or not BIOS would support sector past 2T.

[steve6375]

You can try reading sectors using a syntax like...

cat --hex (hd1)3607462591+1

and see if you get an error or not. Increase the number until you hit a problem.

[RandomUser]

Okay, according to grub4DOS it state:

(hd1,3)5903488+1081856

that would mean that the $MFT is roughly 2.8GB deep from the start of the partition. So it should be on par what Wonko stated.

I also tried that cat --hex command, and don't seem to be hitting any problem. I even tried using a value as high as this one:

cat --hex (hd1)4607462591+1

and Grub4DOS was able to read the disk, albeit with zeros though.

It's looking more and more like I will have to get another hard drive or start putting newer images on that smaller hard disk that's already installed for safest workaround.

[steve6375]

Well, if grub4dos can read those sectors, it may indicate that there is a bug in the GPT or NTFS filesystem handling code in grub4dos?

[steve6375]

Maybe try using dd to  copy the contents of one file into another?

If it works with files below 1.7TB but not with a file above 1.7TB then that might prove that there is a bug in grub4dos?

 

dd if=(hd1,3)/test.iso of=(hd1,0)/smallfile.txt

 

smallfile.txt just needs to be a MB or so, as it will only copy data up to the size of that file and will probably fail if the source file is past 1.7TB??

[RandomUser]

Tried that command and I think it's baffling me, so perhaps it is the BIOS, however I can see and access all the files just fine in Windows.

anyhoo here it is:

buf_0x10000, loops=0x295 in_pos=0x0, out_pos=0x0
0x00000290 0x00000291 0x00000292 0x00000293 0x00000294 0x0000028D 0x0000028E 0x0000028F
Bytes read / written = 0x294B0000 / 0x294B0000

The iso file was beyond the 1.7TB range in the volume .

EDIT:

That was a fluke, Somehow the iso file apparently was not beyond the 1.7TB range, don't know how I misread the numbers, but this time was sure it was beyond the 1.7TB range and after running the command the second time, I get that aforementioned error.

Edited by RandomUser, 14 April 2016 - 05:36 AM.