Jump to content











Photo
- - - - -

RegKeyFixer


  • Please log in to reply
30 replies to this topic

#1 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 19 August 2012 - 08:31 PM

Posted Image

File Name: RegKeyFixer
File Submitter: joakim
File Submitted: 19 Aug 2012
File Updated: 30 Aug 2012
File Category: Tools

Dealing with invalid registry keys

Background:
Here's Mark Russinovich's explanation of the issue; In the Win32 API strings are interpreted as NULL-terminated ANSI (8-bit) or wide character (16-bit) strings. In the Native API names are counted Unicode (16-bit) strings. While this distinction is usually not important, it leaves open an interesting situation: there is a class of names that can be referenced using the Native API, but that cannot be described using the Win32 API.

In short that means that native functions (for instance kernel mode) can deal with embedded null terminated ansi strings, whereas win32 api can't. So by using native functions it is possible to create names (for instance a registry key) that become invalid when accessed in usermode (regedit). That text was taken from the description of RegDelNull which really sparked this off; http://technet.micro...ernals/bb897448 That tool has its shortcomings, and I am certain RegKeyFixer can perform better at dealing with these invalid key names.

Proof of Concept:
Inspired by good old RegHide; http://technet.micro...s/dd581628.aspx I wrote my own (CreateInvalidKey.exe) which is included in the download. Run the PoC and verify with regedit that you have an invalid registry key. Then run RegKeyFixer and specify the correct path, and remember to specify -r as switch (rename), to convert the key into a valid one. RegDelNull seems completely broken on x64, and halfbroken on x86 (could identify and delete the key, but not rename it). To fix the invalid key run this from the commandline;

RegKeyFixer64.exe \Registry\Machine\software\joakim -r -n


Now try access the renamed key, and verify that there exist some data that was previously not visible.

Details:
The included tools utilizes some powerfull native functions in ntdll.dll. Theses functions are what lets you deal with invalid key names, because we can interact with the OBJECT_ATTRIBUTES structure; http://msdn.microsof...9(v=vs.85).aspx .
Since it uses native NT functions, it does not work with user friendly registry names like HKEY_LOCAL_MACHINE, HKCU etc. It will only take the Windows internal registry names, those starting with \Registry\... Below is a listing of the most important translations:

HKEY_LOCAL_MACHINE			 \registry\machine

HKEY_USERS					 \registry\user

HKEY_CURRENT_USER			 \registry\user\user_sid

HKEY_CLASSES_ROOT			 \registry\machine\software\classes

HKEY_CURRENT_CONFIG			\Registry\Machine\System\CurrentControlSet\Hardware Profiles\Current


The user sid is the one similar to this: S-1-5-21-2895024241-3518395705-1366494917-288

The syntax is:
RegKeyFixer.exe path -switch1 -switch2

Path can be in the format specified above

Switch1:
  • -r switch is for renaming invalid key names
  • -d switch is for deleting invalid key names
  • -f switch is for just finding the invalid key names (just search, no repair)
Switch2
  • -s switch is for recursive mode
  • -n switch is for regular enumeration of subkeys

Some examples:

Recursively find and rename invalid keys found at HKLM\SOFTWARE and all its subkeys

RegKeyFixer.exe \Registry\Machine\SOFTWARE -r -s


Just find invalid key names at HKLM\SYSTEM\ControlSet001\services (no recursion, just enumerate 1 level down)

RegKeyFixer.exe \Registry\Machine\SYSTEM\ControlSet001\services -f -n



The tools have been tested on Windows 7 SP1 x64 and XP SP3 x86.

Extra PoC:
There is a second PoC included that generate keys that are more or less impossible to fix on a live system. I suspect the only way to fix such keys, are to do it in offline mode. That means my program is not able to fix those keys. If you know about a program than can fix those keys, then let me know.

Click here to download this file

#2 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 21 August 2012 - 09:22 PM

Btw, another class of invalid registry keys are proven to be even more difficult for Windows to handle. Specifically this is when approaching the upper limit of characters in a name. What may look like a bug to me is that the native registry functions will let you create key names up to 255 characters long, and throw an error when above that. Smart since regedit can't handle longer key names. However, the check fail to take into account that the path also takes up some characters. Thus as long as key name is less than 255 you will be able to create the key, although turns invalid when name+path exceeds 255. The nice side effect is of having a registry key that can't be deleted. Add embedded null termination to the equation and even the native functions in ntdll.dll will have severe issues with deleting this key. They may be readable though, although tricky, and definetely not using regedit..

To me it seems like a broken logic somewhere in ntdll.dll.

Edit: Tested on Windows 7 x64.

#3 kgursu

kgursu
  • Members
  • 7 posts
  •  
    Turkey

Posted 25 August 2012 - 02:19 PM

An error occurred, how to solve this?


Invalid keyname in hex: 7B00440045003800360039003900440032002D003800410030003500
2D0034002000400093000100FF00FF00FF00FF000100000000000000F8007A00C300010000000600
07000000FF00FF00
Number of invalid charaters in keyname: 10
Error in NtCreateKey3 : 0xC000009A -> ¦stenen hizmeti tamamlamak iin yeterli si
stem kayna­² yok.


Thanks.

#4 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 25 August 2012 - 05:05 PM

Thanks for reporting. That was certainly a funny key name, and a strange error..

I tried to reproduce this, but was not able to. I mean I created a key with a name based on the hex values in the output you posted, which of course regedit will compalin about. But my tool was able to correctly rename it to a valid key name.

Can you please provide some more information about what you did. For instance the full command line supplied, your OS and architecture, where did this key originate from (if you know) etc.

It looks like part of a guid, so unless it's from malware, I would bet a system failure during an installation of something..

Also, if your OS is x64 then make sure you are running RegKeyFixer64.exe

#5 Harry Callahan

Harry Callahan
  • Members
  • 6 posts
  •  
    United States

Posted 25 August 2012 - 07:40 PM

I got a similar error when I tried to input your fix command manually. After copying and pasting the command worked fine. must of had typos.

But my questions are:

Is there a way to scan the entire registry for invalid entries or is this target specific only?

If it is target specific how would you know where the target is; when and where to find the problem reg entry?

#6 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 25 August 2012 - 09:04 PM

Until recursive search is implemented, it is assumed that you know where the buggy key is located.

#7 kgursu

kgursu
  • Members
  • 7 posts
  •  
    Turkey

Posted 26 August 2012 - 04:42 AM

Thanks for reporting. That was certainly a funny key name, and a strange error..

I tried to reproduce this, but was not able to. I mean I created a key with a name based on the hex values in the output you posted, which of course regedit will compalin about. But my tool was able to correctly rename it to a valid key name.

Can you please provide some more information about what you did. For instance the full command line supplied, your OS and architecture, where did this key originate from (if you know) etc.

It looks like part of a guid, so unless it's from malware, I would bet a system failure during an installation of something..

Also, if your OS is x64 then make sure you are running RegKeyFixer64.exe


Hi!

My system is a victim of too many crashes after the install of Windows 7 x86 Enterprise. The key "HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindows NTCurrentVersion1TaskCache1" is a remnant of corrupted Task Scheduler registry key, with two changes, those two key names "1"s, which I renamed that time. I couldn't delete the upper "1" key that time and tried after the launch of your tool. So, I tried RegKeyFixer.exe "RegistryMachinesoftwareMicrosoftWindows NTCurrentVersion1" -d command first, but it failed without any errors. Then, I tried deleting the exact root of the problem with this command: RegKeyFixer.exe "RegistryMachinesoftwareMicrosoftWindows NTCurrentVersion1TaskCache1" -d
After that trial the posted error appeared.

So, what can I do to get it deleted?

#8 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 26 August 2012 - 07:18 AM

It is very strange because I tried reproducing the exact key location too, but can't reproduce the problem. I created the funny key name at "RegistryMachinesoftwareMicrosoftWindows NTCurrentVersion1TaskCache1", but had no problems deleting it. My test was on Windows 7 x64 though.

I therefore don't quite understand why you are having troubles.. Can you try the "-r" switch to see if renaming the key makes any difference?

I don't think there's permission issues as I would have expected something like "access denied" then.

I will have the recursive search option implemented later today I think. If you're still facing issues, try that one when finished.

#9 kgursu

kgursu
  • Members
  • 7 posts
  •  
    Turkey

Posted 26 August 2012 - 12:52 PM

It is very strange because I tried reproducing the exact key location too, but can't reproduce the problem. I created the funny key name at "RegistryMachinesoftwareMicrosoftWindows NTCurrentVersion1TaskCache1", but had no problems deleting it. My test was on Windows 7 x64 though.

I therefore don't quite understand why you are having troubles.. Can you try the "-r" switch to see if renaming the key makes any difference?

I don't think there's permission issues as I would have expected something like "access denied" then.

I will have the recursive search option implemented later today I think. If you're still facing issues, try that one when finished.


Hmm,

So, do you think they weren't somewhat stayed locked also? Or, should there be the same "access denied" message then?
In between, I already tried renaming keys without any success...

Edited by kgursu, 26 August 2012 - 12:54 PM.


#10 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 26 August 2012 - 08:11 PM

I have implemented recursive search in a separate tool for now, called RegSearch. Due to the risky nature of having NtDeleteKey and NtRenameKey implemented in recursive mode, I will not merge it into RegKeyFixer until it's rock solid and fully tested.

For instance I've found many strange things, especially inside RegistryMachineSYSTEMCurrentControlSetControlDeviceClasses. And also the RegistryMachineSOFTWAREWow6432Node on 64-bit is not interpreted correctly, and will display strange. Anyways, it will give you an insght into some strange things. I see quit a few keys with control code characters and/or corrupted unicode strings (resulting in chineese looking characters). Anyone else witnessed this?

@kgursu
Although I could not reproduce with the data you posted, I have found something similar on one of my systems that produces the same error. I am currently not aware of what the issue actually is. But can you do me a favour and post the output of the command;
"reg query HKEY_LOCAL_MACHINEsoftwareMicrosoftWindows NTCurrentVersion1TaskCache1 /s"

#11 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 26 August 2012 - 09:23 PM

Here's a sample key name from inside one of the subkeys in DeviceClasses;


230023003F00230057007000640042007500730045006E0075006D0052006F006F007400230055004D0042002300320026003300370063003100380036006200260030002600530054004F005200410047004500230056004F004C0055004D00450023005F003F003F005F00550053004200530054004F00520023004400490053004B002600560045004E005F00530041004E004400490053004B002600500052004F0044005F004300520055005A00450052005F004D004900430052004F0026005200450056005F0030002E00320023003200300030003400310031003000310035003200300037003500460044003000460032003300310026003000230023007B000D0A0000B259553BB7000090E072320200000000606D320200000000E06D320200000000406E320200000000E06B320200000000206C3202000000004071320200000000C076320200000000A0703202000000004074320200000000


Not too easy to spot at first glance, but look towards the end of the string and you will notice unicode corruption. As stated earlier, I have no idea what is causing this. Some of them are picky, while some are not. Will keep investigating...

#12 lamanch

lamanch
  • Members
  • 2 posts
  •  
    Uzbekistan

Posted 27 August 2012 - 09:03 AM

Forgive, but there is a utility with similar functionality + GUI;
Registry Trash Keys Finder.
http://www.databack4u.com/snc/

#13 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 27 August 2012 - 10:23 AM

Seriously, what's that for any good?
  • It is closed source.
  • You have to pay for the full version.
  • It can't delete null embedded keys (at least the one I tried, which is a simpel one)
  • The program crash when I access one of my trickiest test keys.

On the good side, it has a nice looking gui (although it took several minutes to start on the second attempt after the crash)

I also see it has some other functionality, but I did not look at that though.

#14 kgursu

kgursu
  • Members
  • 7 posts
  •  
    Turkey

Posted 27 August 2012 - 11:29 AM

I have implemented recursive search in a separate tool for now, called RegSearch. Due to the risky nature of having NtDeleteKey and NtRenameKey implemented in recursive mode, I will not merge it into RegKeyFixer until it's rock solid and fully tested.

For instance I've found many strange things, especially inside RegistryMachineSYSTEMCurrentControlSetControlDeviceClasses. And also the RegistryMachineSOFTWAREWow6432Node on 64-bit is not interpreted correctly, and will display strange. Anyways, it will give you an insght into some strange things. I see quit a few keys with control code characters and/or corrupted unicode strings (resulting in chineese looking characters). Anyone else witnessed this?

@kgursu
Although I could not reproduce with the data you posted, I have found something similar on one of my systems that produces the same error. I am currently not aware of what the issue actually is. But can you do me a favour and post the output of the command;
"reg query HKEY_LOCAL_MACHINEsoftwareMicrosoftWindows NTCurrentVersion1TaskCache1 /s"


The resulted message is:

HATA: Geçersiz sözdizimi.
Kullanım için "REG QUERY /?" yazın.

Sorry, but it is in Turkish. Translation:

ERROR: Invalid syntax.


So, I guess this reg key is inaccessable, isn't it?

#15 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 27 August 2012 - 11:54 AM

No it means you supplied the command with wrong syntax. I am bad in Turkish, but maybe you forgot to put the registry path inside quotation marks (if it contains spaces).

#16 lamanch

lamanch
  • Members
  • 2 posts
  •  
    Uzbekistan

Posted 28 August 2012 - 04:36 AM

Excuse Joakim, the author of the program has really made the program paid for everything, except Russian-speaking users. The Russian version of the program the full-function and free. You can show a test reg-file which this program could not remove? On mine XP32bit this program easy deletes similar keys, including keys created RegHide. I do not criticise your program, I want to compare possibilities of these programs.

#17 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 28 August 2012 - 09:58 AM

Forgive, but there is a utility with similar functionality + GUI;
Registry Trash Keys Finder.
http://www.databack4u.com/snc/

Excuse Joakim, the author of the program has really made the program paid for everything, except Russian-speaking users. The Russian version of the program the full-function and free. You can show a test reg-file which this program could not remove? On mine XP32bit this program easy deletes similar keys, including keys created RegHide. I do not criticise your program, I want to compare possibilities of these programs.


I did not interpret your post as criticism. I was just wondering why it was proposed as alternative working equally well.

First of all it is closed source which makes it less appealing right away. Second, it turned out to perform worse at deleting invalid keys. Third, it seems to crash more than not crash on Windows 7 x64 (I have only tested it on this OS).

Regarding the generation of invalid keys, it is not possible to create using a reg file. You will need to have special purpose program. If you look inside package there is included a PoC that will write such a key. I don't know if you're the developer of that program, but if you are, then maybe you want to reproduce one of the more trickier key. If so, I can send you a special purpose program to generate such keys, so you can see for yourself what happens.

#18 kgursu

kgursu
  • Members
  • 7 posts
  •  
    Turkey

Posted 28 August 2012 - 10:07 AM

No it means you supplied the command with wrong syntax. I am bad in Turkish, but maybe you forgot to put the registry path inside quotation marks (if it contains spaces).


Ok, here is the result of the command:

HKEY_LOCAL_MACHINEsoftwareMicrosoftWindows NTCurrentVersion1TaskCache1{D
E8699D2-8A05-4 @?☺ÿÿÿÿ☺



#19 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 28 August 2012 - 06:10 PM

@kgursu
Certain tricky keys, like the one you're struggling with, can't be handled with native functions in ntdll.dll. I am of the impression that these keys needs to be handled in a very different way, and I currently don't know how (at least on a live system). In offline mode it's different, as you can access the hive as a file, and modify it just like you want. However that approach is very different, and it is unlikely that I will create one such in the short run.

But I will do some more testing, and maybe I find a solution.. At least I know how to programatically produce these keys, so we'll see..
  • kgursu likes this

#20 kgursu

kgursu
  • Members
  • 7 posts
  •  
    Turkey

Posted 28 August 2012 - 08:10 PM

@kgursu
Certain tricky keys, like the one you're struggling with, can't be handled with native functions in ntdll.dll. I am of the impression that these keys needs to be handled in a very different way, and I currently don't know how (at least on a live system). In offline mode it's different, as you can access the hive as a file, and modify it just like you want. However that approach is very different, and it is unlikely that I will create one such in the short run.

But I will do some more testing, and maybe I find a solution.. At least I know how to programatically produce these keys, so we'll see..


So, do you know any program that can do this offline right now? It should be freeware from my point of view. :)
And, thanks for trying to solve that in the future...

#21 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 28 August 2012 - 08:43 PM

No, I don't know about such a program.

#22 pscEx

pscEx

    Platinum Member

  • Team Reboot
  • 12707 posts
  • Location:Korschenbroich, Germany
  • Interests:What somebody else cannot do.
  •  
    European Union

Posted 29 August 2012 - 04:02 PM

@Joakim,

perhaps a bit OffTopic, but maybe you can help me.

The issue is that I want to read EDID monitor data. This can be usually done by some setupapi functions.

In the special case, however, because that is during boot, some registry information is not (yet) inited, and the setupapi functions fail.
I get in every case a valid registry key RegistryMachineBlaBla, where the needed data are really stored.

How can I access these data with native functions?

Peter

EDIT:
When writing, I think I got an idea.

Just with win32 api functions open the key BlaBla under root HKLM
Tomorrow I'll try.

#23 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 29 August 2012 - 07:02 PM

Should be fully possible by using the native functions, since they are designed to work without the win32 subsystem. Functions of interest would be;

NtCreateKey
NtOpenKey (maybe not?)
NtQueryValueKey
NtClose

Here's a link to something related (although no registry handling); http://hex.pp.ua/nt-...s-shell-eng.php

#24 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 29 August 2012 - 07:05 PM

But beware, I just posted about something that looks like a bug to me; http://forum.sysinte...topic28455.html

#25 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 29 August 2012 - 08:12 PM

@all
Don't use RegSearcher until new versioon is ready. I will remove the current version of it.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users