Jump to content











Photo
- - - - -

Copying files to a NTFS compressed drive/vhd - size related.


  • Please log in to reply
14 replies to this topic

#1 gbrao

gbrao

    Frequent Member

  • Advanced user
  • 354 posts
  •  
    India

Posted 22 July 2017 - 07:09 AM

I have faced this problem when copying files/folders from a uncompressed source to a compressed destination. E.g. : There are folders+files whose uncompressed size is 150MB and whose compressed size is (say) 100MB. If I try to copy these files to a NTFS compressed drive or vhd whose size is (say) 125MB, I get a message 'insufficient space on destination'. Because the copy program takes the uncompressed size into account. Is there a way/program to do such a copy? btw I face this problem when copying windows files to a compressed vhd ( to boot from memory ).

#2 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 22 July 2017 - 11:20 AM

Use robocopy. Compression happens in the background so you need to retry. Robocopy with the correct options will retry.



#3 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 22 July 2017 - 11:31 AM

You can also use my compcopy program which I created just for this problem ( it waits to compress each file as it copies ):

 

http://wb.paraglider...les/Compcopy.7z

 

Executed from a wb script like this:

 

ShellExecute,Open,"%ProjectTemp%\CompCopy.exe","#$q%TargetImage%\Programs#$q y:\programs /s"

 

It only supports copying all files from a source folder to a target folder ( no wild cards should be specified):

 

CompCopy <sourceDir> <targetDir> /s
 


  • Nuno Brito and gbrao like this

#4 gbrao

gbrao

    Frequent Member

  • Advanced user
  • 354 posts
  •  
    India

Posted 23 July 2017 - 02:20 PM

CompCopy only worked with *.* appended to source folder.

e.g. :
compcopy l:\windows\*.* m:\windows /s

#5 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13750 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 23 July 2017 - 02:51 PM

CompCopy only worked with *.* appended to source folder.

e.g. :
compcopy l:\windows\*.* m:\windows /s

Interesting, it seems like the exact opposite of what the Author stated.


It doesn't work on XP (just for the record).

:duff:
Wonko

#6 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 24 July 2017 - 09:11 AM

It does not require any wild cards or trailing \. Try:

 

Compcopy i:\windows m:\windows /s

 

Its linked with the XP runtime but probably linker does not set the correct flags to run on XP. I checked it ran on win7.



#7 gbrao

gbrao

    Frequent Member

  • Advanced user
  • 354 posts
  •  
    India

Posted 24 July 2017 - 09:23 AM

without *.*


with *.*



I'm also facing some permission/acl issues with the copied files

EDIT : I just cannot attach images, trying.

#8 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 24 July 2017 - 09:30 AM

I suggest you use robocopy for copying windows. I just verified program works on other folders without any trailing \ or wild cards. It does not support copying of file permissions or hardlinks. It was not designed for copying windows.



#9 gbrao

gbrao

    Frequent Member

  • Advanced user
  • 354 posts
  •  
    India

Posted 24 July 2017 - 09:34 AM

OK. Could you tell me how compcopy works - the method/logic. I could try and do something similar.

#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13750 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 24 July 2017 - 09:56 AM

@paraglider
JFYI:
Translated from Italian:
Cannot find entry point GetFileInformationByHandleEx in the dynamic link library KERNEL32.DLL.
Then:
Application not correctly initialized (0xc0000005)

:duff:
Wonko

#11 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 24 July 2017 - 04:54 PM

Yes. That function is only available on vista and later. Win xp is a dead os. Don't see any need to support it.



#12 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 24 July 2017 - 04:57 PM

It uses 2 threads. One reads the files and creates a chain of buffers which are read by the second thread to write the data. It uses this code to compress each file after its written:

 

void CompressFile(TCHAR * fileName, DWORD attributes)
{
    if (attributes & FILE_ATTRIBUTE_READONLY)
    {
        SetFileAttributes(fileName, FILE_ATTRIBUTE_NORMAL);
    }
    HANDLE file = CreateFile(fileName,
        GENERIC_WRITE | GENERIC_READ,
        FILE_SHARE_READ,
        NULL,
        OPEN_EXISTING,
        FILE_FLAG_NO_BUFFERING | FILE_FLAG_WRITE_THROUGH,
        NULL);
    if (INVALID_HANDLE_VALUE == file)
    {
        LogMessage(herr, "Failed to open file %s, ret = %u\n", fileName, GetLastError());
    }
    else
    {
        DWORD BytesReturned;
        USHORT state = COMPRESSION_FORMAT_DEFAULT;
        BOOL ret = DeviceIoControl(
            (HANDLE)file,               // handle to file or directory
            FSCTL_SET_COMPRESSION,       // dwIoControlCode
            (LPVOID)&state,             // input buffer
            (DWORD) sizeof(state),       // size of input buffer
            NULL,                        // lpOutBuffer
            0,                           // nOutBufferSize
            &BytesReturned,              // number of bytes returned
            NULL                         // OVERLAPPED structure
            );
        if (!ret)
        {
            LogMessage(herr, "Failed to compress file %s, ret = %u\n", fileName, GetLastError());
        }
        CloseHandle(file);
    }
    if (attributes & FILE_ATTRIBUTE_READONLY)
    {
        SetFileAttributes(fileName, attributes);
    }
}


  • gbrao likes this

#13 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13750 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 24 July 2017 - 05:17 PM

Yes. That function is only available on vista and later. Win xp is a dead os. Don't see any need to support it.

Sure :), and noone asked for supporting it, I just mentioned the fact for the record, but it is not the linker's fault, it is by design.  .

 

:duff:

Wonko



#14 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 25 July 2017 - 08:50 AM

It was an accident I used a function that was new to Vista.



#15 paraglider

paraglider

    Gold Member

  • .script developer
  • 1716 posts
  • Location:NC,USA
  •  
    United States

Posted 25 July 2017 - 09:08 AM

Looks like the reason I used that function was SetFileInformationByHandle. There is no equivalent function exported in XP by kernel32.dll. Its quite difficult to set file access times in XP in a way that is compatible with later OS.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users