Savannah Photographer Dale Reagan

There are a number of Open Source tools that you may find need for when your CF card (or other camera storage media) becomes corrupted (fully or partially.)  In all cases for digital camera media I encourage folks to:

1. use the camera tools to ‘format’ (erase, clean) the media after you copy your images to your hard drive; if you use your computer to ‘format’ the media it may lead to problems (for various and nebulous reasons) and
2. once you get your images onto your computer disk then make another copy to a backup drive OR to CD/DVD.

Ok – but we seem to have a problem – we can’t read the media - how do we get the images off of the ‘bad’ CF (or other) card?  There are a number of commercial tools (some free – check with the maker of your media) that you can try OR you can explore the Open Source data recovery tools below.  Note that this type of activity is not for the faint-of-heart and, if you have images with commercial value (i.e. taken on assignment or during studio work) then you may want to contract with a service that specializes in data recovery.

• photorec - best for image files; relatively simple to use and designed specifically to recover image files.  Two resources with more information include this Wikipedia post and this CGSeurity page.

• ntfsundelete - restore NTFS drives (and possibly files)

• testdisk - Restore partitions (Unix/Linux); actually another tool related to photorec.

• foremost – Configurable – restores files by type from a disk partition.  The recovered files will have numeric names but appropriate extensions (i.e.  .jpg, .gif, .pdf, etc.)  The bad news is your folder structure is lost.

When using these types of tools, I encourage folks to select options that copy lost files from the damaged media to a good drive – don’t try to rebuild, repair the damaged media.  Get your files first and then try formatting in your camera and taking some test images using the suspect media card and then copy the test images to your PC.  In most cases, once I recover any lost images I would discard the potentially problematic media…

The Linux manual page for PhotoRec is quite brief so you may want to explore the links above for more detailed information on using this tool.

---

PHOTOREC(1)                  Administration Tools                  PHOTOREC(1)

NAME
photorec – Recover lost files from harddisk, digital camera and cdrom

SYNOPSIS
photorec [/log] [/debug] [/d recup_dir] [device|image.dd|image.e01]

photorec /version

DESCRIPTION
PhotoRec  is file data recovery software designed to recover lost files including video, documents and archives from Hard Disks and CDRom and lost pictures (Photo Recovery) from digital camera memory. PhotoRec ignores the filesystem and goes after the under-lying data, so it’ll work even if your media’s filesystem is severely damaged or formatted. PhotoRec is safe to use, it will never attempt to write to the drive or memory support you are about to recover lost data from.

OPTIONS
/log   create a photorec.log file

/debug add debug information

SEE ALSO
testdisk(1), fdisk(1).

For a detailed example using hard disk tools the post on my technology blog may be of interest.

Before starting Adobe Lightroom2 and in general, I would suggest using some sort of workflow that protects you from image-file loss sooo, backups are a must.  While Lightroom can be used to ‘import’ files I find that it is a slower process than what I outline below.   After a photo session I usually proceed with something like this:

1. copy  images from camera media to work area on hard drive (using Windows XP File Explorer I open both the destination drive and the source drive – my USB card reader and the destination drive – I also create a new folder to place the images in and usually, I will use a date-based descriptive name like:  \my_image_folder\0801_tybee_client_camera_model\)
2. proceed with basic selection/evaluation of the images on the hard disk – erase any out-takes, rename the files, add meta-data (keywords, location, photographer, client information, etc.)   I use Adobe Photoshop Bridge for this (faster, lighter weight processing than using Lightroom – at least for my environment…)
3. copy ‘keepers’ to secondary, backup, external hard drive (which is only powered on long enough for the copy process to complete)
4. remove the files from the camera media after confirming that both hard disk sets are readable (i.e. using the camera tools I ‘format’ the camera storage card)
5. proceed with image editing (development changes, generate final images or prints) – for a long photo session this can take several hours/days; this is done in Lightroom)
6. copy processed images to external hard drive (remove the 1st, un-processed folders and replace with the final images)
7. create CD/DVD/Some_other_media permanent (read-only) backups of final image sets or just the best images
8. remove the ‘work’ images from step 1 (as long as you no longer need them and have adequate and tested backups on disk or external media)

Now things get more interesting.  All of the above was done using computer A.  Sometimes I want to use a second computer to access the files (on the computer and in the location where they already exist.)  I use the second computer because it has additional compute resources (more CPU power and RAM) so it will simply process things more quickly (in Lightroom.)  In such a case I would begin using the second computer at step #5 (image editing.)

Why use this or similar approaches?

• all files for a photo session are under the same folder tree,
• backup and replication of the file sets is simplified,
• Lightroom performance will actually be better since you are using a relatively small image database (i.e. limited to the size of the photo session as opposed to all of your images; the down side: you won’t have a master Lightroom catalog unless you create a separate catalog and import the images.  Based on my experience this can work fairly well if you limit the size and types of images in your master catalog, i.e. use smallish JPEGs, no RAW files,  with the same folder structure but on a separate file tree or master drive; I typically generate such smaller images during my workflow – your mileage may vary…)
• you don’t have to search for the Lightroom data – it’s all under the same folder tree.

When I started using Lightroom I had catalogs all over the place (wherever I had disk space) – now I use this scheme and life is simply better. 

The approach outlined below should work for either accessing Lightroom catalogs on external drives as well as network drives.  Based on my limited testing this works with Windows XP (and it may also work with Vista.)  It should however, work with any externally directly attached USB/Firewire disk drive.

The bad news - Lightroom does not support using ‘catalogs’ on ‘network drives‘.   I will guess that performance and potential file conflict issues are the reasons for this – if two computers attempt to access the same set of data then problems can result if network file locking is not being used (which I am guessing is part of the problem.)  The other item of concern would be network performance issues (i.e. it might ‘run’ really ’slow’…)

So, how do web get Lightroom2 to use a ‘network drive’? We trick to OS into showing that we are using a local drive by using the subst command (a DOS command.)

Some strategies for flexible image access

• use external drives
• network drives (using the ’subst’ command)
• use folder structures (an example below)

I use a naming convention similar to the sample below.

• job_id-source-folder_n+0
• job_id-source-folder_n+1
• LR-job_id_catalog
• LR-exports

Using the DOS subst command (virtual vs network drives)

Help subst – Associates a path with a drive letter.

SUBST [drive1: [drive2:]path]
SUBST drive1: /D

drive1: Specifies a virtual drive to which you want to assign a path.
[drive2:]path Specifies a physical drive and path you want to assign to a virtual drive.
/D Deletes a substituted (virtual) drive.

Type SUBST with no parameters to display a list of current virtual drives.

---

On Windows XP – using: Start –> Run –> cmd (your drive letter may be different – open a command shell, change to the ‘D‘ drive, change to the ‘Clients‘ folder, and then use the ‘dir‘ command to see a list of folders. In this example the Lightroom catalog is stored in the folder LS-client_name_09-07-30. Exports (from Lightroom) are stored in the LR-exports folder and the folders that begin with numbers (i.e. 0729*) contain the images from the photo session.

1. Start –> Run –> cmd <enter>
2. d: <enter>
3. cd Clients_2009\0729_client_name <enter>
4. dir <enter>

Sample output from the dir command is shown below. (Yes, you can see similar information using your file system browser and no, you don’t need to do all of these steps.)

Volume in drive D is Client_Photos

Volume Serial Number is E011-4DE4
Directory of D:\Clients_2009\0729_client_name

07/30/2009  10:06 PM              0729_client_camera-01
07/30/2009  11:09 PM              0729_client_camera-02
07/31/2009  10:11 PM            13,698 Adobe Bridge Cache.bc
07/30/2009  09:58 PM                 8 Adobe Bridge Cache.bct
08/04/2009  02:55 PM              LR-exports
08/04/2009  03:42 PM              LS-client_name-09-07-30
3 File(s)         13,706 bytes
6 Dir(s)  39,526,641,664 bytes free

Before you can use the subst command you need to share the folder on the host computer. You need to give the share a unique name – I use simple names like ‘X’ or ‘Y’ (drive names NOT already used on the host computer.)  You must also have Windows configured for local network use on both computers.  Provided your Windows network is configured properly for resource sharing the steps are:

1. share resource on host computer
2. use subst command on second computer on the same network
   • subst  x:  \\2nd_computer_name\X
3. open the newly connected virtual drive in Windows Xp – browse the folders
4. open Lightroom and access the drive as if it were a ‘local’ drive

---

Notes on this approach

• you can loose files/data – both systems must remain on during any period where shared files are being used
• this will be slower than using a directly attached drive or slower than using files in a local folder
• if you need speed then you should only use files in local folders – you could copy before and after edits between the two systems
• if you connect a network drive using the command:  net use X:  \\2nd_computer_name\X (or if you use the GUI and connect a network drive via browsing) then it is treated as a network drive; a virtual drive (created using subst) is treated as a local drive – that’s the reason Lightroom will work with it…
• you can use:  Start –> Run –> subst  x:  \\2nd_computer_name\X  OR
• you can use:  Start –> Run –> cmd (and then run subst  x:  \\2nd_computer_name\X from the DOS shell)
• if you use the Windows file browser the connected drive will show up as a network drive (somewhat confusing perhaps, but in this case what we care about is how Lightroom ’sees’ the drive.)
  

Which Statement is True?

A Camera Never Lies… or

A Print is only one possible interpretation of an image…

Both have merit and both have depend upon human and medium variables.  In the old (film) days there (very broadly speaking) were two types of photographers:

1. those who created photos and hired out developing and printing and
2. those that handled print creation from start to finish.

The basic limits of what could be done with a print were tied to the  quality of the original negative image;  the printing process allowed for an additional creative touch to be applied to the final print.  Those photographers who handled all of these steps could of course have maximum creative input into the final output.   It should be noted that most (if not all) aspects of image creation and printing were controlled by the photographer or printer.  The choice of film brand, type, ISO as well as the choice of chemicals and papers used all affect the final outcome and, in  many cases become a type of signature for a photographer or printer.

In the digital photo world these factors are still in play (or they can be.)  Camera manufactures have made tremendous, perhaps even miraculous progress in developing current digital cameras (entry level to pro dSLR systems.)  Photo-printer manufactures have made similar levels of improvements.  It is now possible to click the shutter button, wait a few seconds for the image to pass through the air to your wireless network, reach your computer and then be shuttled to your inkjet printer – all  without you having/needing to decide on any camera settings, computer settings, printer settings, nada!   Provided that your ink levels are good and your photo-printer paper is of reasonable quality you will most likely wind up with a print that is acceptable and it may even be a photo that your share with others.

If you take the ‘no-hands’ approach described above then for each auto setting you use you are actually allowing the camera (i.e. the knowledge/wisdom/skill of the camera, printer, computer & software engineers) to pick for you.  There is nothing wrong with this – it’s just a choice that you make; most serious photographers (pro and non-pro) will make these choices (i.e. they won’t use any automatic settings.)  Does this really make any difference?  So far, no, not much difference – well, not until you start doing truly creative things with the image/print.

Cameras (film or digital) are really, really, really dumb – compared to the combination of the human eye and brain.  The camera will never ’see’ the level of image complexity that we perceive in a scene (our eye/brain combination is a 3d view; a camera only has a 1d, flat view.)   So, a camera never lies really means that a camera never sees the whole truth…  How about:  the print never lies? The camera is much closer to the original image – any print that we encounter is in fact, one interpretation of any given image and as such, is much less likely to be close to any sort of truth – a digital negative (file) is much closer to the truth (errr, the camera’s truth.)

JPEG, TIFF, RAW – the Digital Negative

Camera manufactures offer you choices – most cameras include an option for RAW files.  The choice of digital negative file type impacts image quality and places practical limits on what can be done with your negative.  JPEG  files are cooked – i.e. they are sort of like using other automatic settings – they limit your creative input control.  RAW files allow the most flexibility for post-processing.  A simple example is the choice of light temperature (i.e. setting the camera for daylight, night, cloudy, etc.)

If you have a RAW file then you can change this after the photo has been taken; if you use a JPEG file then you have to re-process the file (i.e. create a new file to use) to fix a problem like this.  Since a RAW file is never a work-file, you could argue that there is no difference.  In addition, the latest version of Adobe Lightroom treats all negative files as if they were RAW files (any changes result in the creation of a new file so the original is never changed.)  If your camera supports it I suggest that you try using a setting that provides you with both a RAW file and and a JPEG file – only then will you be able to see the differences in both types of capture.

NOTE – since I first wrote this many software products have changed – many current products allow you to treat a JPEG file like a RAW file, i.e. any changes are made to a copy of the original file so your ‘negative image‘ is preserved as long as you use software that does not change it…