69 replies to this topic

[joakim]

In Windows 8 many things changed, including bootmgr. Previously, bootmgr was compressed with lznt1, but is now compressed with Xpress Huffman. I have previously described how to modify the Windows 7 and previous versions: http://reboot.pro/fi...ls-bootmgrntfs/

For the new compression I made a separate program; http://www.mediafire...3d3jb8a9f13dytr (outdated)

Update version 2, 20.05.12: A program to recompile bootmgr.exe into bootmgr, ready for booting: http://reboot.pro/fi...mgr-recompiler/ or http://www.mediafire...d9pyrn578bvxbww

It uses RtlCompressBuffer and RtlGetCompressionWorkSpaceSize in ntdll.dll. The parameters are supplied through an inputbox, and is the CompressionFormatAndEngine bitmask. It can be supplied in decimal or in hex (prefixed with 0x). Obviously, you must run the attached program in Windows 8, unless you just want compression with LZNT1. Explanation of the possible valid parameters:

COMPRESSION_FORMAT_NONE = 0x0000

COMPRESSION_FORMAT_DEFAULT = 0x0001

COMPRESSION_FORMAT_LZNT1 = 0x0002

COMPRESSION_FORMAT_XPRESS = 0x0003

COMPRESSION_FORMAT_XPRESS_HUFF = 0x0004

COMPRESSION_ENGINE_STANDARD = 0x0000

COMPRESSION_ENGINE_MAXIMUM = 0x0100

So to compress with Xpress Huffman supply either 260 or 0x104.

Basic structure of bootmgr:

1. 16-bit stub with code to unpack, evaluate and execute the compressed part. About 2x KB.

2. A 16 byte section with some information that the 16-bit stub evaluates

3. A tiny 8192 byte PE image, with unknown function. It's content is not evaluated.

4. The compressed 32-bit executable bootmgr.exe

This is in accordance with previous versions, but with the main difference being the compression engine used on the 32-bit executable. Another small difference is the small section right before the tiny PE image. Take a look at this image:


At offset 0x68a0 I have identified 4 values:

byte 1-4: a signature

byte 5-8: the compressed size of bootmgr.exe

byte 9-12: the uncompressed size of bootmgr.exe

byte 13-16: the relative offset to the compressed data calculated from the start of this section (ie the signature)

When we modify the 32-bit executable, we must deal with an incorrect checksum of the executable as well as an invalidated digital signature. For the checksum the easiest is to just update it. To deal with the digital signature we have 2 options:

1. Configure TESTSIGNING in BCD on the entry for {bootmgr}

or

2. Change code in bootmgr.exe to deactivate the check.

For the sake of a challenge we choose number 2 and modify code. First we remove the digital signature because it is no longer needed. The relevant parts to change for version 6.2.8250.0 is:
At VA 401263 change these 6 bytes:

0f8862020000 -> 909090909090

(a conditional jump is nopped out)

Furthermore, to deactivate the signature check of winload.exe we need to change a few more bytes.
At VA 42935e change these 2 bytes:

7415 -> eb15

(a conditional jump is changed to a short jump)

At VA 429385 change these 2 bytes:

7507 -> eb07

(a conditional jump is changed to a short jump)

So to reassemble bootmgr after modification:
Use the above linked tool; http://reboot.pro/fi...mgr-recompiler/ or http://www.mediafire...d9pyrn578bvxbww

Have fun!

[coderforlife]

Thanks Joakim! This is a great post. Thanks for finding the tiny portion with the information about the size and offset of the compressed part as I was having trouble finding it (it was much easier before).

I have just posted the first iteration of my compression and decompression code on github: https://github.com/c...ife/ms-compress

No precompiled binaries yet, but coming soon! Otherwise you could compile it yourself - it creates a DLL for now. It all should work, but please tell me of problems. I will be updating it in the next few days with license information (probably GPL).

The API is quite simple, much like the RTL functions:

size_t compress (CompressionFormat format, const_bytes in, size_t in_len, bytes out, size_t out_len);
size_t decompress(CompressionFormat format, const_bytes in, size_t in_len, bytes out, size_t out_len);

The actual used size of out is returned instead of being an argument. Decompress reverses the order of the input and output arguments from RTL but otherwise the same (well, also missing some silly arguments). The types const_bytes and bytes are defined in my program simply as unsigned char* (with or without a const). The RTL format definitions also work, but I have also defined them in compression.h:

COMPRESSION_NONE = 0, // == COMPRESSION_FORMAT_NONE
COMPRESSION_LZX = 1, // replaces COMPRESSION_FORMAT_DEFAULT
COMPRESSION_LZNT1 = 2, // == COMPRESSION_FORMAT_LZNT1
COMPRESSION_XPRESS = 3, // == COMPRESSION_FORMAT_XPRESS
COMPRESSION_XPRESS_HUFF = 4, // == COMPRESSION_FORMAT_XPRESS_HUFF

You can also call the desired compression function directly (along with some additional helpers). For example, for XPRESS:

size_t xpress_compress(const_bytes in, size_t in_len, bytes out, size_t out_len);
size_t xpress_max_compressed_size(size_t in_len);

size_t xpress_decompress(const_bytes in, size_t in_len, bytes out, size_t out_len);
size_t xpress_uncompressed_size(const_bytes in, size_t in_len);


As you can tell from this, the uncompressed size is actually determinable just from the compressed data! However it is better to use it if available since it would be faster to not have to scan through the entire compressed data stream.


I am not finished writing the code, but here is the status of the algorithms:

• LZX: This is currently just the 7zip code with some minor tweaks. Untested against RTL.
• LZNT1:
  
  • Compression is much faster than RTL (~50x), but takes more memory (avg ~1 MB total), and has a much nicer worst-case upper limit on size
  • Decompression is slightly faster than RTL (~1.4x) and gets faster with better compressed data (RTL is reversed)
• XPRESS:
  
  • Compression is slower than RTL (~0.6x) but has marginally better compression ratio and uses the same amount of memory
  • Decompression is almost as fast as RTL (~0.9x)
• XPRESS_HUFF:
  
  • Has a similar compression ratio to WIMGAPI
  • Time not tested, and not compared to RTL since RTL-XPRESS-HUFF is broken

As you can see, XPRESS is what needs my attention first, although it isn't as bad as it was originally: at one point my compression code was about 0.01x the speed of RTL!

Have fun!

[joakim]

That's great Jeff! Thanks for sharing. Will try it out.

[homes32]

any ideas as to what the mystery PE is for?

[joakim]

any ideas as to what the mystery PE is for?

As noted in the first post, it is not evaluated and hence not used for anything. You can easily verify that by zeroing out all bytes for that tiny PE.

[coderforlife]

If it is the same as Windows 7 bootmgr tiny PE then that PE file seems to be only for version information. It contains no code and one resource: version information. The Windows 7 one was a bit strangely setup (didn't have the language and name tiers of resource information) but it was obvious that the PE file was only for versioning information, presumably so you could get it without decompressing the actual PE so updatin would be easier.

[joakim]

After some silly mistakes I got it working. Only tried xpress_huff_compress, and although it compressed, I did not manage to do it properly. How do you get "size_t out_len" ? I thought it was supposed to be calculated, but apparently not.. So I end up with a file with the compressed chunk, and depending on the size I manually put on it, the rest is filled with zeros. Another thing, compared to MS' implementation, yours seem to compress better. Have you tested compatibility with bootmgr's stub?

[joakim]

Btw, bootmgr stub is not happy with the compressed exe.

[coderforlife]

I haven't completely finished testing but I have been checking that a compress/decompress cycle results in no changes to the bytes, including for the bootmgr data.

The out_len is simply the length of the given out buffer so that the program does not overrun the buffer. The return value is actually how many bytes were written to the out buffer. So it just needs to be large enough. For other types I have added a function for getting the max compressed size (if you gave the program completely uncrompressible data) but I haven't made one for Xpress Huff yet.

There still may be bugs in the compression. Let me check.

[joakim]

So I retried with both compress and decompress and they both work great. Out_len was of course generated as expected, but for some reason did not work yesterday. It is now all good, and original bytes are the exact same after decompression. Well done

[joakim]

I whipped together a sample frontend program for your excellent library; http://www.mediafire...3zge3atj7ge6dwu

I included both 32-bit and 64-bit compiled dlls along with the license .

It is written in AutoIt, which many of the members here use, including myself.

Only tested and verified working Windows 7 x64.

Edit:
@all
This means we can modify Windows 8 bootmgr from within Windows 7 (for example).

[joakim]

Some more info:

Actually you can copy ntdll.dll from Windows 8 into Windows 7, and call RtlCompressBuffer on the copied dll. That way you can use MS native Xpress and XpressHuff implementation i Windows 7 too. Useful for this topic, since the XpressHuff compression implementation of Jeff's library currently is slightly incompatible with bootmgr 8's stub (ie execution breaks at the integrity checking part).

@Jeff
Do you know the details of the difference?

[coderforlife]

I thought it was working? I don't know the problems I guess. I will look into it in the coming week. Just to make sure, you are updating the compressed length in the data right (like after the BM thingee)?

[joakim]

Sorry for giving misleading information. I never double checked the values in 16 byte section, and so it turned out the correct values are;

byte 1-4: a signature

byte 5-8: the compressed size of bootmgr.exe

byte 9-12: the uncompressed size of bootmgr.exe

byte 13-16: the relative offset to the compressed data calculated from the start of this section (ie the signature)

And it also turned out that your XpressHuff solution works absolutely fine with bootmgr And your implementation has a better compression ratio

[joakim]

I created an application to recompile bootmgr.exe into bootmgr; http://www.mediafire...cz9ncm1cvm9eeri Should be compatible with any version of bootmgr, and also corrects its checksum.

It works great on Windows 7 x64.

@Jeff
It seems that your library is not happy on XP 32-bit. Is it even supposed to be?

[coderforlife]

Your original values did seem a little off... Now they are nicely multiples of 4!

Which program is not working with Windows XP? The compression library should run in anything, it only uses the standard C/C++ library. I believe you can even compile it on Linux and have it work, one I the major reasons I wanted to have so many different methods implemented in an open source way. However it is possible that the C/C++ library linked to is not compatible/present in Windows 7.

Did you compile using Visual Studio or MinGW? If you used Visual Studio it may have linked to VC 2010 library, which comes with Windows 7 and nothing earlier. You can download it from Microsoft. If you used MinGW it may link to specific Windows 7 kernel functions. The solution is to compile on Windows XP.

If you are having a problem with bmzip, I thin I did some weird things there. I recommend just not using it and using the compression library which does most of the work of bmzip and is actually now faster.

[joakim]

Maybe I'm doing something wrong, but in MinGW it fails to compile. If built from VS2010 on XP x86, the program using the library will crash. Hmm, maybe it's something with the datatypes or whatever in the program using the library that is the issue on x86 platforms..

[joakim]

In the meantime, on x86, copy the 32-bit version of ntdll.dll from Windows 8, to your 32-bit Windows version and use my tool there. That works, verified on XP. Remember to keep the dlls in the same directory as the tool.

[joakim]

New updated version, and probably last one; http://www.mediafire...d9pyrn578bvxbww

It supports both LZNT1 and XpressHuff generated bootmgr's, and the tool can be run on any Windows version from XP and later. Read the readme about the required libraries though. MSCompression.dll is included in the download.

So in short you can generate bootmgr of Vista with XpressHuff or Windows 8 version with LZNT1, or any combination you can think of Nice to have, sometimes.

Just included it in the download section too.

[city24]

So, how to decompress the Windows 8 bootmgr into bootmgr.exe? I haven't found anything explaining how to actually do that.

Edited by city24, 16 September 2012 - 09:35 AM.

[joakim]

The easiest is probably to use Jeff's bmzip; https://github.com/coderforlife/bmzip However you could also use my frontend mscompression that uses Jeff's library, but that would require to extract the correct bytes before decompression. I guess I could have made something similar, by using the later Windows 8 ntdll.dll, but not now as there exist several ways already.

[city24]

The easiest is probably to use Jeff's bmzip; https://github.com/coderforlife/bmzip However you could also use my frontend mscompression that uses Jeff's library, but that would require to extract the correct bytes before decompression. I guess I could have made something similar, by using the later Windows 8 ntdll.dll, but not now as there exist several ways already.

Thanks. It worked.

[joakim]

Good. Let me know if you run into problems modifying it.

[amir_82111]

Hi

I want to edit bootmgr of windows 8 to change  "\boot\bcd" to other path for example:

\boot\bc1

or

boot1\bcd

 

how do it?

please...

[LeightonJames]

Hi all

 

Apologies for dragging this thread up but i'm at my wits end. Has anyone got a pre-compiled version of BMZip they can upload? I am having serious problems getting it to compile.

 

Thanks