24 replies to this topic

[Technotika]

Hi chaps

I have been asked to recover files from a formatted SD card.
I went for my best known tool Photorec and it appeared to run fine and recovered loads of files.

I have been told that not all of the files are there. There are some missing although there are little thumbnails of the missing files?
Does any one know anything more thorough that might give better results?

Thanks guys !

[MedEvil]

I would suggest that you first get a list of the names of the missing files. Then you can test all kinds of tools, to see, if any manages to show those files.

That said, i'm very happy with r-studio.

Though if you really wanna get each last picture, even those that the automatic tools can not recover. Then some manual work is in order.

- Create an image of your sd card.
- Unformat it with a tool of your choice.
- Make a list of the names of the pictures missing.
- Úse a disk editor to rebuild the fat table for the missing files.

[Nuno Brito]

I agree with MedEvil, the first step you should do is creating a byte-per-byte image of your SD card.

[Wonko the Sane]

Some hints in the hope of disambiguation.

• As it was suggested, first thing do a "dd-like" or "forensic sound" image of the SD card, better if two, always work on the image and never on the original (PHOTOREC is OK since it won't write to the same device)
  
• If a filesystem has been formatted the filenames are NOT there anymore in an accessible way.
  
• If the actual files were in ROOT of the filesystems the names are lost (forever).
  
• If the actual files were in a directory under ROOT (or in any further subdirectory) they can normally be recoverd (filenames I mean).
  
• If the filesystem was formatted under Vista or 7 without the "Quick" option, anything will not be recoverable (but this is not the case at hand since you managed to recover some with PHOTOREC ).
  
• PHOTOREC is a file-based recovery tool, the amount and accuracy of what it can be found by it depends on level of fragmentation of the filesystem, basically anything that was contiguous will be recovered, anything that was not won't.

Rebuilding a FAT table with a hex editor is not "easy-peasy", you will have to learn how to do it and this will take a long time.


There are several tools that may be able to access both the filenames and the actual files, or if you prefer, PHOTOREC is "last possibility".

Tools that you may want to try using ON THE IMAGE AND NOT on the original SD card:
DMDE
http://softdm.com/
FAT walker (I am presuming that the SD card was formatted with a FAT16 or 32 filesystem):
http://dmitrybrant.com/fatwalker

Generic advice is given here:
http://www.msfn.org/...ool/page__st__7

If the images were jpeg, there are dedicated tools to try and recover them (Commercial only AFAICR).

Depending on HOW the images were created (i.e. if the SD card was in a digital camera and the camera actually "wrote" them directly) there is the possibility through some "peculiar" behaviour of some camera's in the way the images were named, the way the EXIF data is written and the way the thumbnail was made inside them to find a few more "hits" than if the SD card was written in a PC with jpeg (say) taken from the internet or created by a graphic program.


Wonko

[MedEvil]

PHOTOREC is a file-based recovery tool, the amount and accuracy of what it can be found by it depends on level of fragmentation of the filesystem, basically anything that was contiguous will be recovered, anything that was not won't.

Are you sure? Those kind of tools usually recover everything, as if the fat chain would be continuous. So some files could end up broken, but should be recovered nontheless.

Technotika said, that some files are missing completely.

Does Photrec check the files and only recovers those, which are in prefect order?

[Wonko the Sane]

Does Photrec check the files and only recovers those, which are in prefect order?

Not "exactly" this simple AFAICR.
PHOTOREC looks for "file headers" or file signatures (or both) thus a "perfect file" will be recovered, depending on the actual data found (and the way it is interpreted) a "broken" file may be recovered (broken ) or it can be even "overlooked". (this is particularly true for JPEG images that have an internal thumbnail that can well confuse the "parser")

But even if the "broken" file is "recovered" (in the sense that a bunch of sectors are copied to the new filesystem under a new random name):

• it will have NOT it's original name
  
• it won't be viewable/accessible by any JPEG viewer

thus there will be "missing files" (and possibly just some thumbnails of them) which is exactly what Technotika reported.



It is just a different use of terms , to me "a bunch of sectors copied under a random name and not viewable/accessible" are not "recovered files", they remain "a bunch of sectors copied under a random name and not viewable/accessible" .



Wonko

[MedEvil]

But even if the "broken" file is "recovered" (in the sense that a bunch of sectors are copied to the new filesystem under a new random name):

• it will have NOT it's original name
  
• it won't be viewable/accessible by any JPEG viewer

thus there will be "missing files" (and possibly just some thumbnails of them) which is exactly what Technotika reported.

Not quite. Depending on the program, the fitrst character of the filename will be eighter replaced with some default character or a dialog is poped up, asking for the correct character.
More importantly though, jpeg is a streaming format, thus as long as one has the first sector any decent viewer will show it.
A different story would for instance bmp be, which is not a sreaming format and has the header at the end and is therefore way harder to recover.

[steve6375]

PhotoRec_Win is good and I can recommend it from personal experience of recovering an SD card - it worked brilliantly! See http://sites.google....s/file_recovery for a guide

[Technotika]

thanks for all the top tips - I'm gonna get the DD image today as the card is coming back to me then see what more I can do

will get the image I need
sudo dd if=/dev/disk1 of=/Users/abcdef/PlugPBX/betaimageDec18-nodev.img (changed to suit needs of course)

or can I use a dd rescue image from within windows?

[Wonko the Sane]

will get the image I need
sudo dd if=/dev/disk1 of=/Users/abcdef/PlugPBX/betaimageDec18-nodev.img (changed to suit needs of course)

Under which OS?
Under Linux usually "disk" is hda, hdb, sda, sdb, etcetera.

or can I use a dd rescue image from within windows?

Sure:
http://reboot.pro/7783/

Not quite. Depending on the program, the fitrst character of the filename will be eighter replaced with some default character or a dialog is poped up, asking for the correct character.
More importantly though, jpeg is a streaming format, thus as long as one has the first sector any decent viewer will show it.
A different story would for instance bmp be, which is not a sreaming format and has the header at the end and is therefore way harder to recover.

Well, NO.

The "replacement character" is about DELETED files, NOT about re-formatted media.

Your experience with JPEG is very UNLIKE mine (and actually unlike the one this guy had):
http://www.impulsead...jpeg-photo.html

You should play a little bit with this little program:
http://www.s2service...agefreeware.htm
http://www.s2service...repair-jpeg.exe
then come back.

Since you are a programmer, you may like also these:
http://www.a-kat.com...g/JPEGDump.html
http://www.imperialv...rg/binary/jpeg/


Wonko

[Technotika]

anyone know a free .IMG mounter tool - I'm using partitiom find and mount and free version limits speed to 512k - really slwoing

[Wonko the Sane]

anyone know a free .IMG mounter tool - I'm using partitiom find and mount and free version limits speed to 512k - really slwoing

Are you JOKING or WHAT?
We are FULL of such programs .
For the scope you need it, easiest is IMDISK.
If you need a \\.\PhysicalDrive access you may want to use VDK or the MS VSS services.
Find them and many other ones here:
http://reboot.pro/1507/page__st__1

But, at least for using dmde and fatwalker and testdisk/photorec there should be NO need for it as they can mount images allright.


Wonko

[Technotika]

oh jeeez I was joking (not!)

totally forgot about IMDISK !!!

of course

(walks away shaking head)

[Technotika]

As per the threads posted kindly, I can confirm that I now have A LOT more of the photo's back. I'd say 80/85%.

I have a DD image still which I mounted with IMDISK, and ran various tools mentioned.
The best results were (in order) with File Scavenger, R-Studio, Runtime get data back and the least being Photorec in this case although has been the best in the past.

I'm not sure on the gains to efforts ratio on trying to rebuild the fat table properly, although it would be good to learn.

I have since found that there was an initial problem internally with the camera. One day it turned on with an error saying "Problem with SD Card, please format"

From the pattern of the photo's recovered it seems perhaps the original photo's prior to formatting were corrupted.

If you get all the recovered photo's in one folder and put thumbnail view on you would'nt be wrong in thinking everything was recovered.
it's when you maximise the photo's you see where things are awry. On some, once expanded, have about 85% of the image is just a block of grey and maybe 2cms thick across the top of peoples heads.
Others open but they are made up blocks of the other photo's and some open up "colorised" or kind of "negagtive" all from good looking thumbnails. But like I said most of them are ok.
From what I report what do you think is the cause the strange photo's recovered, are they recoverable perhaps in some more advanced environs, or could they actually be broken from the internal camera error.

Ultimately as always I came here with 45% and with your tips I got to 85%, so kudos to the masters once again!

(tried some of the jpeg repair tools but no luck yet)

[Wonko the Sane]

If you get all the recovered photo's in one folder and put thumbnail view on you would'nt be wrong in thinking everything was recovered.

Read (attentively) challenge #1:
http://reboot.pro/14958/

it's when you maximise the photo's you see where things are awry. On some, once expanded, have about 85% of the image is just a block of grey and maybe 2cms thick across the top of peoples heads.
Others open but they are made up blocks of the other photo's and some open up "colorised" or kind of "negagtive" all from good looking thumbnails. But like I said most of them are ok.

As expected by at least one of the "masters"

From what I report what do you think is the cause the strange photo's recovered, are they recoverable perhaps in some more advanced environs, or could they actually be broken from the internal camera error.

Re-read (attentively) post #10:
http://reboot.pro/15233/page__st__9
and post #6:
http://reboot.pro/15233/page__st__5
you recovered some photos and a number of "a bunch of sectors copied under a random name and not viewable/accessible" items
As you can see, these "bunches of bytes" can in some cases be opened/viewed partially, in some cases simply don't show anything, in some cases they look good but with "wrong" colours/light, in some cases can be made "almost good" by cropping the good part, etc., etc. but they represent NOT a "recovered picture".

This means logically that:

• NOT all corruption is "the same"
  
• NOT all images will ever be recoverable from the "bunch of bytes"
  
• the subset that CAN be recovered will need anyway several DIFFERENT approaches in order to be recovered

Simple FAQ/FGA:
Q1: Can I recover a corrupted jpeg image?
A1. Yes/No it greatly depends on the type and extension of the corruption
Q2. Are there specialized softwares to do so?
A2. Yes.
Q3. Are there freeware/open source tools among them?
A3. Yes.
Q4. Do they work (I mean the Freeware/Open Source ones)?
A4. No, normally they don't unless the corruption is really of trivial entity.
Q5. Do they work (I mean the Commercial ones)?
A5. No, normally they don't unless the corruption is really of trivial entity.
Q6. I really need some of these pictures to be recovered, which would be best approach?
A6. Your best bet (of course depending on the value you give to the actual picture) is to look for professional service. The alternative being that of writing your own software and/or learn to use each and every trick and the usage of each and every available tool and attempt doing the recovery manually. Please understand how each and every corrupted image may need different techniques and that any "single" existing program (freeware/OpenSource or Commercial) is likely to have only one or a few of these techniques implemented.
Q7. What if I rebuild the FAT table? Would this guarantee that I can recover these corrupted images?
A7. Not necessarily, if a byte (yes, a single byte in some cases can cause HAVOC) is corrupted in one of the sectors the images spans across, having the FAT perfectly rebuilt won't change anything. On the other hand, if the problem is "mixed" sectors due to fragmentation, then rebuiding the FAT table might allow the recovery of the image.
Q8. Ok, so let's say I want to buy a Commercial program, will it do what it is advertised about?
A8. Usually, NO. (see above)
Q9. Then how do I choose a program?
A9. You try it, most will give you a demo/trial function before actually paying for the license.
Q10. Well, but finding all the available tools and trying them will take ages.
A10. Yes.
Q11. What should I do?
A11. Take the needed time or use a service, usually you pay them proportionally to results.


Wonko

[MedEvil]

Technotika, Wonko is italian so he loves to use lots of words to say simple things.

Pictures, with strange colors (only). - a bit has fliped (on disk). Not a problem one can fix with better recovery.
The picture needs fixing. No idea though, how to do that.

Pictures with gray blocks or bits and pieces of other pictures in them. - The recovered fat chain is wrong.
The picture was fragmented and the recovery program guessed the wrong order.
This can only be fixed manually, by trail and error.

Use a disk editor.
- Go to the directory - go to the file name - there you find the start cluster and the size of the picture.
- check for the used cluster size on the partition.
- go to the FAT table and create a chain starting from the 'start cluster' of appropriate length.
- try viewing the file in a picture viewer
- make an educated guess at which cluster the chain starts to go wrong
- change the chain, so that it has now a gap where the problem was.
- check again - still not better? - do above again
- - better? move on to the part were the next problem is

Depending on how big and fragmented the pisture was, this can be really hidious.


TIPS:
- A FAT chain moves always forward, never backwards or jumps around. Is the end of the partition reached, it continues at the beginning.
- Manual recovery is easier, if you can cross of your list, all the clusters you know definitly do not belong to the sought picture. For that, do an in place recovery or undelete and redelete all pictures that are not perfect.

[Wonko the Sane]

Pictures, with strange colors (only). - a bit has fliped (on disk).

Are you sure that a bit was "flipped"?
Which ONE?
HOW it was "flipped"?

The picture needs fixing. No idea though, how to do that.

Short, simple, effective.

Pictures with gray blocks or bits and pieces of other pictures in them. - The recovered fat chain is wrong.

NO, not necessarily/always. (see already given link):
http://www.impulsead...jpeg-photo.html

Gray Areas in your Image

In various forums on the internet, people have incorrectly stated that photos are unrecoverable if they contain gray areas. This is not usually true. Some JPEG decoders are very picky with how they recover from errors and simply report Drawing Failed (in Windows Picture and Fax Viewer), while others might draw a middle-gray region or just display a "Red X". In some cases the gray area is in fact completely damaged (overwritten by unrelated data), but this is not always the case.

Wonko

[MedEvil]

Yes, there exist a lot of poorly written picture viewers, which are not suited for manual picture recovery.
So what? There exist also a lot of tools, which are not suited for file recovery in general, like a hammer or a screwdriver for instance.

I'm confident that Technotika has enough brains, to not base his work on the windows Fax viewer or it's later equivalents.

[Wonko the Sane]

So what? There exist also a lot of tools, which are not suited for file recovery in general, like a hammer or a screwdriver for instance.

Hmmm.
I agree somewhat on the hammer.
But maybe a screwdriver can have some uses (if you know how to use it) example:
http://hddguru.com/a...dstack-Q-and-A/

Spoiler

Figure 25 — after heads had been replaced, author would have to adjust manually a gradient angle of a head stack axis in WD200BB hard drive to get data

Kids, don't even think of doing this at home!


As often happens, again I don't agree fully with you.
In my experience "badly programmed" (in the sense of "less known" or "minor" or "non standard" or "less featuresd" software) can sometimes parse things that "orthodox" software cannot, of course in the case of the windows fax viewer we have an example of "orthodox badly programmed" software , so we do agree specifically .

As a completely unrelated example , there is an old, obsolete, preliminary version of SPREAD32 that can open partially corrupted .xls files and even .xls files with sheet password protection.

Any info on the flipped bit?


Wonko

[Technotika]

Well the way I see it, they are not my photo's, so I've done my "bit". Literally. If they want more, refer to "adopting a service".

I really haven't the inclination to stitch busted photo's back together, albeit it be a good skill (for some one with more patience)

Cheers!
Have good w/e

[MedEvil]

In my experience "badly programmed" (in the sense of "less known" or "minor" or "non standard" or "less featuresd" software) can sometimes parse things that "orthodox" software cannot,

Sorry, i used "badly programmed" with my own connotations. Refering to any software, that does things on it's own, that it shouldn't.
Like for instance, changing the first letter of each line to capital, hiding things from the user or not allowing the user certain operations, no matter, how many dialogs he confirms and so on.

Any info on the flipped bit?

Yep, the fifth from the right, seventh row from the top. That's the guilty bit. It viciously droped a banana peel, so that the bit to it right steped on it and fliped!

What do you wanna hear? How to fix a jpeg stream?
Sorry, never done that.
I just know, that a wrong clusterchain will not cause this problem. Hence it is not a problem that can be fixed by creating a better FAT chain.




btw. Using a html wysiwyg editor, makes working with this new board bearable.

[Wonko the Sane]

I was confused by the "bit" (as opposed to "byte") and to your certainty that a "flipped bit" will cause "strange colours".

It is now clear that you posted some "generic" advice and not anything that you actually know the details of, something like:

if there is something wrong with the image, I have no idea what it can be, but the whatever is the problem is not connected with a wrong cluster chain, maybe,or maybe not, I have no real data/experience to support this statement.

(translated from italian )

Probably in German is one single word :

IchhabekeineAhnungaberichwollteetwasschreiben

Wonko

[MedEvil]

Well if there is a problem with a jpeg, it is WAY more likely, that a single bit has fliped, than that 8 in a row have fliped. Wouldn't you agree?

[TheHive]

You can try this FREE program called "Zero Assumption Digital Image Recovery"
Read the reviews. Recover pictures from media cards

http://www.snapfiles...gerecovery.html

[Wonko the Sane]

Well if there is a problem with a jpeg, it is WAY more likely, that a single bit has fliped, than that 8 in a row have fliped. Wouldn't you agree?

Not really.
Since most (read as "all") "disk operation" is made at sector or at byte level (and not at bit level) AFAIK.

There is a nice exception that I happened to find a couple of times, that was a (on IDE/ATA) a damaged connector on the HD.

On some makes the space around the connector is too big and allows to connect the cable 1 pin shifted.
If you couple this with one of those IDE cables with no hole on pin20 (and no external "position key" and with an unexperienced but physically strong user, the result can be that of flatting out or break pin 18, which leads to a bit-wise corruption.


I guess the same may happen on the serial line of a SD card with some strange problem in the actual controller, but this would be hardware and probably, just like the mentioned example all values would have been corrupted by the missing bit.


Wonko