Jump to content











Photo
- - - - -

Grubutil FAT reports wrong 'free space' on FAT32 drive

grub4dos

  • Please log in to reply
3 replies to this topic

#1 deomsh

deomsh

    Frequent Member

  • Advanced user
  • 196 posts
  •  
    Netherlands

Posted 04 December 2020 - 12:00 AM

While testing a script which is build around 'FAT', one of Chenall's Grubutils, I did some copying with bigger files and I ran into problems.

 

While copying a 1GB test-file to a 40GB FAT32 drive, 'FAT' gave me following:

 

'Need more space: 802336 KB,Current available drive space: 246240 KB

FAT Error (7): Access denied due to prohibited access or directory full'

 

This was strange, because the disk had still plenty of free space, about 37GB.

 

Also the size of the test-file was not the problem, just before I copied a 1GB file from a FAT16 drive to the same 40 GB FAT 32 drive. Using 'FAT copy' standalone gave the same error. :(

 

FAT info (hd0,0) gave the same free space, but also a strange result:

 

Fat sub-type: FAT32

Sectors per cluster: 64

Sectors per FAT: 10238

Number of free clusters: 1187343

Total Clusters: 1310231

41927392 Total drive space.

246240 KB available

 

Everything matched with the CHKDSK-report in MS-DOS afterwards, except of course the free space, which should be 37994976 KB according to CHKDSK. This is exactly the same calculated from FAT info above: Number of free clusters * Sectors per cluster * 512 bytes per sector / 1024 = REAL KB available.

 

To be sure I tested latest version of FAT too: found in g4dext-2019-07-14.zip on https://github.com/g...butils/releases but no difference.

 

I tested FAT first in VBOX 6.0.18 r136238 (Qt5.6.2) on VHD's, which gave report above.

 

Later 'in the wild' on a 30GB SATA hard-drive. On the SATA hard-drive I could copy three files of 1GB-size without problems, the fourth 1GB-file gave Error (7). At that time the free size was 16,2 GB. 'FAT info' reported 209520 KB available, and 1061671 free clusters of 32 sectors per cluster. After a full reboot the error remained. :wacko:

 

I don't know how to upload print-screens on this forum, so to be sure here's the output inserted as code, I made with help of GRAB.G4B (but cleaned from the annoying spacing between each line of output).

 

First one inside VBOX with my script FATCOPY.G4B:

 grub> fatcopy.g4b (hd2,0)/1GBFAT16.VHD (hd0,0) /v                               
                                                                                 
 >copy 1GBFAT16.VHD                                                              
 Need more space: 802336 KB,Current available drive space: 246240 KB             
 FAT Error: (7) Acces denied due to prohibited access or directory full          
 grub>                                                                           
 grub> fat info (hd0,0)                                                          
 FAT sub-type:   FAT32                                                           
 Sectors per cluster: 64                                                         
 Sectors per FAT: 10238                                                          
 Number of free clusters: 1187343                                                
 Total clusters: 1310231                                                         
 FAT start sector: 32                                                            
 41927392 KB total drive space.                                                  
 246240 KB available.                                                            
 grub>                                                                           

Second one 'in the wild' with 'FAT copy' only.

 grub> fat info (hd0,0)                                                          
 FAT sub-type:   FAT32                                                           
 Sectors per cluster: 32                                                         
 Sectors per FAT: 15259                                                          
 Number of free clusters: 1061671                                                
 Total clusters: 1952948                                                         
 FAT start sector: 32                                                            
 31247168 KB total drive space.                                                  
 209520 KB available.                                                            
 grub>                                                                           
 grub> fat copy (hd1,4)/hdd-1gb.img (hd0,0)/hdd-1gb4.img                         
 Need more space: 838800 KB,Current available drive space: 209520 KB             
 FAT Error: (7) Acces denied due to prohibited access or directory full          
 grub>                                                                           

I hope someone can help me with this problem, or reproduce it. :unsure:

 



#2 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 04 December 2020 - 12:59 PM

From the little I can understand from :unsure::

https://github.com/g...4dext/fat/fat.c

 

 

 

printf("Sectors per cluster: %d\n"  
"Sectors per FAT: %d\n"  
"Number of free clusters: %d\n"  
"Total clusters: %d\n"  
"FAT start sector: %d\n"  
"%u KB total drive space.\n"  
"%u KB available.\n",  
fs.csize,fs.fsize,fs.free_clust,fs.n_fatent - 2,fs.fatbase,(fs.n_fatent - 2) * fs.csize>>1,free_space >> 10);   return 0;   }

Sectors per cluster: fs.csize

Number of free clusters:fs.free_clust

KB available: free_space >> 10

 

BUT free_space seemingly comes from:

 

*free_space = fre_clust * fs.csize << 9;

 

so, for some reasons, fre_clust is NOT the same value as fs.free_clust

 

Given that in example 

free_space=246240 KB available

246240 KB -> 252149760 bytes

and that cluster size is (in bytes) 64*512=32768 bytes

fre_clust must be 252149760/32768= 7695 (clusters) whilst fs.free_clust is 1187343

 

So, since 1187343 (fs.free_clust) is correct, the issue must be in the way the "other" variable fre_clust is calculated.

 

1187343 is hex 0x121E0F

7695 is hex 0x1E0F

 

So it seems like the cluster number is truncated at 4 bytes, i.e. 0xFFFF=65535

 

In the second example

1061671 is hex 0x103327

0x3327 -> 13095*32*512=214548480 and 214548480/1024=209520

 

No idea what actually needs to be corrected to avoid this truncation. :dubbio:

 

 

:duff:

Wonko



#3 deomsh

deomsh

    Frequent Member

  • Advanced user
  • 196 posts
  •  
    Netherlands

Posted 06 December 2020 - 02:13 PM

From the little I can understand from :unsure::
https://github.com/g...4dext/fat/fat.c

Sectors per cluster: fs.csize
Number of free clusters:fs.free_clust
KB available: free_space >> 10

BUT free_space seemingly comes from:

so, for some reasons, fre_clust is NOT the same value as fs.free_clust

Given that in example
free_space=246240 KB available
246240 KB -> 252149760 bytes
and that cluster size is (in bytes) 64*512=32768 bytes
fre_clust must be 252149760/32768= 7695 (clusters) whilst fs.free_clust is 1187343

So, since 1187343 (fs.free_clust) is correct, the issue must be in the way the "other" variable fre_clust is calculated.

1187343 is hex 0x121E0F
7695 is hex 0x1E0F

So it seems like the cluster number is truncated at 4 bytes, i.e. 0xFFFF=65535

In the second example
1061671 is hex 0x103327
0x3327 -> 13095*32*512=214548480 and 214548480/1024=209520

No idea what actually needs to be corrected to avoid this truncation. :dubbio:


:duff:
Wonko

This is very helpful, thanks a lot :punk: At least I know that my script isn't the problem.

I took a look at the source code, but my knowledge of C is at the level of 'Hello world' only. Although I can compile FAT in a Ubuntu-VM, I do not understand the way variables are declared/calculated in C. :pulpfiction:

But I found a workaround for the time-being: FATMINI has no check on free space, so after coping a next big file, there is again enough space on my drive according to fat info. According to crc32 the copying succeeded.

 grub> fat copy (hd2,0)/1gbfat16.vhd (hd0,0)                                     
 Need more space: 802336 KB,Current available drive space: 246240 KB             
 FAT Error: (7) Acces denied due to prohibited access or directory full          
 grub>                                                                           
 grub> fat copy (hd2,0)/1gbfat16.vhd (hd0,0) > nul || fatmini copy (hd2,0)/1gbf> 
 grub>                                                                           
 grub> ls (hd2,0)/1gbfat16.vhd                                                   
  1GBFAT16.vhd                                                                   
 grub> ls (hd0,0)/1gbfat16.vhd                                                   
  1GBFAT16.VHD                                                                   
 grub> crc32 (hd2,0)/1gbfat16.vhd                                                
 33a26e2b                                                                        
 grub> crc32 (hd0,0)/1gbfat16.vhd                                                
 33a26e2b                                                                        
 grub>                                                                           

So I added a subroutine to my script as a workaround, only in case the Target is FAT32 > 4GB, including the readout of free clusters from the FAT32 boot sector.

The subroutine is called in case fat copy gives an error, but this time redirected to null. Earlier I restored FAT output of errors, especially because is there was not enough free space left.

To me thing seems good now, but feel free to criticize!

The sub-routine needed only one new global variable (echo's are only for quick debugging).After some tedious testing I found that fat info gives free space between a few hundred MB's and 4GB, so your suspicion that only 16-bit values are used seems to be right. :smart:

:faterror
#new subroutine for FAT32 partitions above 4GB
setlocal && set * && set tdevice=%tdevice% && set argsize=%argsize% && set arg=%arg%
set message1=%~1
set message2=%~2
if not exist tdevice && endlocal && set gotomsg=%message2% && echo Unspecified FAT-error %act%omit %arg% && goto :eof
#echo tdevice=%tdevice%
# Extra % needed after tdevice=(hdx,y)  => %tdevice%1+1 is NOT good?? Only %tdevice%%1+1 is oke!
cat --hex --skip=0x1E8 --length=4 %tdevice%%1+1 | set freeclus=
#echo freeclus=%freeclus%
set /a freeclus=0x%freeclus:~19,2%%%freeclus:~16,2%%%freeclus:~13,2%%%freeclus:~10,2%
cat --hex --skip=0x0D --length=1 %tdevice%+1 | set secpclus=
#echo secpclus=%secpclus%
set /a secpclus=0x%secpclus:~10,2%
cat --hex --skip=0x0B --length=2 %tdevice%+1 | set bytepsec=
#echo bytepsec=%bytepsec%
set /a bytepsec=0x%bytepsec:~13,2%%%bytepsec:~10,2%
echo bytepsec=%bytepsec% secpclus=%secpclus% freeclus=%freeclus%
set /a "needclus=%argsize% / %bytepsec% + 1 / %secpclus% + 1"
set /a "frespace=%freeclus% - 1 * %secpclus% * %bytepsec%"
# Negative argsize with almost 4 GB-file, above 0x7FFFFFF same limitations CALC & @retval ??
echo argsize=%argsize% needclus=%needclus% frespace=%frespace%
if %needclus%>=%freeclus% && echo %act%omit %arg% needed %argsize% bytes, free space availaible %frespace% bytes && endlocal && set gotomsg=%message2% && goto :eof
echo fatmini %message1%
fatmini %message1%
set gotomsg=
endlocal && set gotomsg=%gotomsg%
goto :eof

Below some (cleaned) output from GRAB.G4B. To gain speed I used this time a 2Gb-file. The copytime seems very low, but the file is in fact an empty image, so it seems almost no bytes are actually written to disk. :eek:

 grub> fat info (hd0,0)                                                          
 FAT sub-type:   FAT32                                                           
 Sectors per cluster: 64                                                         
 Sectors per FAT: 10238                                                          
 Number of free clusters: 862441                                                 
 Total clusters: 1310231                                                         
 FAT start sector: 32                                                            
 41927392 KB total drive space.                                                  
 2432288 KB available.  
                                                         
 grub> fatcopy (hd2,0)/2gbfat16.vhd (hd0,0)/2gbf1602.vhd                         
                                                                                 
 Does 2gbf1602.vhd specify a file name or directory on the target                
 (F=file or D=directory or Q=quit)?:F                                            
                                                                                 
 >copy 2gbfat16.vhd 2gbf1602.vhd                                                 
 Copied 2085877 KB in 00:03:15                                                   
 Total files copied/overwritten/skipped: 1/0/0

 grub> fat info (hd0,0)                                                          
 FAT sub-type:   FAT32                                                           
 Sectors per cluster: 64                                                         
 Sectors per FAT: 10238                                                          
 Number of free clusters: 797257                                                 
 Total clusters: 1310231                                                         
 FAT start sector: 32                                                            
 41927392 KB total drive space.                                                  
 346400 KB available. 
                                                           
 grub> fatcopy (hd2,0)/2gbfat16.vhd (hd0,0)/2gbf1603.vhd                         
                                                                                 
 Does 2gbf1603.vhd specify a file name or directory on the target                
 (F=file or D=directory or Q=quit)?:F                                            
                                                                                 
 >copy 2gbfat16.vhd 2gbf1603.vhd                                                 
 bytepsec=512 secpclus=64 freeclus=797257                                        
 argsize=2135938560 needclus=65184 frespace=26124484608                          
 fatmini copy  (hd2,0)/2gbfat16.vhd (hd0,0)/2gbf1603.vhd                         
 Copied 2085877 KB in 00:03:31                                                   
 Total files copied/overwritten/skipped: 1/0/0

 grub> fat info (hd0,0)                                                          
 FAT sub-type:   FAT32                                                           
 Sectors per cluster: 64                                                         
 Sectors per FAT: 10238                                                          
 Number of free clusters: 732073                                                 
 Total clusters: 1310231                                                         
 FAT start sector: 32                                                            
 41927392 KB total drive space.                                                  
 2454816 KB available. 
                                                          
 grub> fatcopy (hd2,0)/2gbfat16.vhd (hd0,0)/2gbf1604.vhd                         
                                                                                 
 Does 2gbf1604.vhd specify a file name or directory on the target                
 (F=file or D=directory or Q=quit)?:F                                            
                                                                                 
 >copy 2gbfat16.vhd 2gbf1604.vhd                                                 
 Copied 2085877 KB in 00:03:01                                                   
 Total files copied/overwritten/skipped: 1/0/0
 grub> fat info (hd0,0)                                                          
 FAT sub-type:   FAT32                                                           
 Sectors per cluster: 64                                                         
 Sectors per FAT: 10238                                                          
 Number of free clusters: 666889                                                 
 Total clusters: 1310231                                                         
 FAT start sector: 32                                                            
 41927392 KB total drive space.                                                  
 368928 KB available.                                                             

                                    

And so on, undefintely.... :sleeping:


Edited by deomsh, 06 December 2020 - 02:29 PM.


#4 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 06 December 2020 - 06:51 PM

I took a look at the source code, but my knowledge of C is at the level of 'Hello world' only. Although I can compile FAT in a Ubuntu-VM, I do not understand the way variables are declared/calculated in C.  :pulpfiction:

You have the advantage on me, rest assured :).

 

As said I have no idea where/why the truncation occurs, the variables seem like being defined as DWORDS or unsigned long (i.e. 32 bit values):

DWORD fre_clust, fre_sect;
unsigned long long f_pos = 0;
	unsigned long long fil_size;
	UINT br, bw;/*br 读取字节数;bw 写入字节数。*/

Maybe it is the "br" defines as UINT? :unsure:

 

But it shouldn't be related. :dubbio:

 

:duff:

Wonko







Also tagged with one or more of these keywords: grub4dos

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users