{"id":182,"date":"2011-05-07T13:57:51","date_gmt":"2011-05-07T19:57:51","guid":{"rendered":"https:\/\/www.joshualyman.com\/?p=182"},"modified":"2013-03-23T05:46:58","modified_gmt":"2013-03-23T11:46:58","slug":"using-photorec-and-a-bash-script-to-recover-51407-images","status":"publish","type":"post","link":"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/","title":{"rendered":"Using Photorec and a bash script to recover 51,407 images"},"content":{"rendered":"<p>A sad thing happened to my mother-in-law the other day. Somehow, one of her 1 TB hard drives that stores mountains of images (she is a professional photographer) got reformatted to a nice, fresh state, but certainly not what was wanted. She has a client that needs the images in just a couple of days, and so the race was on to see what I could do (remotely, using LogMeIn no less) to recover the data.<\/p>\n<p>I turned to <a href=\"http:\/\/www.cgsecurity.org\/wiki\/PhotoRec\">Photorec<\/a>, and absolutely fantastic command line program that does a hard core scavenge of any disk, looking for known file headers, and tries to reconstruct a file based on its findings. I felt that, since the drive had simply been reformatted and not zeroed, with only one file being placed on the drive since the formatting, that the chances of finding the files was pretty good. Using another external hard drive as storage for recovered files, 15 hours later it had recovered 51,407 .JPGs, .NEFs, and .PSDs, woohoo!<\/p>\n<p>Now one unfortunate byproduct of Photorec recovery is that, while much of the file&#8217;s metadata is still in place (modified\/accessed timestamps, EXIF data, etc.), it didn&#8217;t recover filenames or folder structure. So I had over 51,000 images with random filenames in 130 different &#8220;recup&#8221; folders, mostly in chunks of the same creation date, but not entirely.<\/p>\n<p>To solve this, I wrote a small little bash script to sort files into folders by modified date. I&#8217;m not super skilled at bash scripting, so it took me longer than it probably should of and is probably not very elegant, but it worked quite well in my case! I pointed the script at the recovered directories, ran it, and 3 minutes later it had created over 1300 folders based on the created\/modified dates of the files, moved the files into the folders, and bam! I was pretty impressed with myself. \ud83d\ude42<\/p>\n<p>If anyone is looking for a similar script (couldn&#8217;t find one after an initial Google search last night) then feel free to take this one and modify to suit your needs. Obviously I can&#8217;t guarantee anything with it, but it shouldn&#8217;t eat your files if you set up the directories correctly. If you are nervous, change the &#8220;mv&#8221; command on line 27 to &#8220;cp&#8221; and you should be safer (although the script will of necessity run much slower). There are certainly some things that could be improved about the script (not hardcoding the 300 of ## number, not cd&#8217;ing into the source directory, etc.) but it worked in a pinch and didn&#8217;t need to be elegant. Hope it helps!<\/p>\n<pre lang=\"bash\" line=\"1\">#!\/bin\/bash\r\n# Sort files into folders based on their creation date.\r\n#\r\n\r\n# Indicate a path to the source folder of data to be sorted, and a destination folder where you want the sorted folders to be.\r\n# Note: this will work better if the sorted directory is NOT inside the source directory, otherwise recursion might occur and the world might blow up.\r\nSOURCE_DIR=~\/Desktop\/testsrc\/\r\nSORTED_DIR=~\/Desktop\/sorted\/\r\nsorted_count=0\r\n\r\n# go into the first folder of the source folder\r\ncd \"$SOURCE_DIR\";\r\n\r\n# for each item in this folder\r\nfind . -type f -print | while read file; do \r\n\r\n\t# Get the creation date of the item and store it in a variable\r\n\tfile_date=`stat -f '%Sm' -t \"%F\" \"$file\"`;\r\n\r\n\t# Does a folder with this date already exist?\r\n\tif [ ! -d \"$SORTED_DIR\/$file_date\" ]; then\r\n\t\t# If no, create the folder\r\n\t\tmkdir -p $SORTED_DIR\/$file_date\r\n\tfi\r\n\r\n\t# Now move the file to the folder\r\n\tmv \"$file\" \"$SORTED_DIR\/$file_date\/\"\r\n\r\n        # Provide some feedback to the user on our progress\r\n\tlet \"sorted_count += 1\"\r\n\tif [[ \"$sorted_count % 100\" -eq \"0\" ]]; then\r\n\t\techo \"Sorted $sorted_count of 51,407\" # Obviously the of count is hard coded \r\n\tfi\r\n\r\ndone\r\n\r\necho \"Finished.\"<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>How do you recover thousands of images from a 1 TB formatted hard drive and make sense out of them? Use Photorec to recover the images, and then write a small bash script to sort those gibberish files into folders created based on date modified! Sample script included.<\/p>\n<div class=\"more-link-wrapper\"><a class=\"more-link\" href=\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/\">Continue reading<span class=\"screen-reader-text\">Using Photorec and a bash script to recover 51,407 images<\/span><\/a><\/div>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[10],"tags":[85,91,22,90,88,89,87,86],"jetpack_publicize_connections":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Using Photorec and a bash script to recover 51,407 images - Joshua Lyman.com<\/title>\n<meta name=\"description\" content=\"Bash script to sort files into folders created from modified date on the file, used to recover thousands of images from a Photorec data dump.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"Using Photorec and a bash script to recover 51,407 images - Joshua Lyman.com\" \/>\n<meta name=\"twitter:description\" content=\"Bash script to sort files into folders created from modified date on the file, used to recover thousands of images from a Photorec data dump.\" \/>\n<meta name=\"twitter:creator\" content=\"@jlyman\" \/>\n<meta name=\"twitter:site\" content=\"@jlyman\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Joshua\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/\"},\"author\":{\"name\":\"Joshua\",\"@id\":\"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e\"},\"headline\":\"Using Photorec and a bash script to recover 51,407 images\",\"datePublished\":\"2011-05-07T19:57:51+00:00\",\"dateModified\":\"2013-03-23T11:46:58+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/\"},\"wordCount\":434,\"commentCount\":4,\"publisher\":{\"@id\":\"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e\"},\"keywords\":[\"bash\",\"images\",\"linkedin\",\"photography\",\"photorec\",\"recovery\",\"scripting\",\"scripts\"],\"articleSection\":[\"Programming\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/\",\"url\":\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/\",\"name\":\"Using Photorec and a bash script to recover 51,407 images - Joshua Lyman.com\",\"isPartOf\":{\"@id\":\"https:\/\/www.joshualyman.com\/#website\"},\"datePublished\":\"2011-05-07T19:57:51+00:00\",\"dateModified\":\"2013-03-23T11:46:58+00:00\",\"description\":\"Bash script to sort files into folders created from modified date on the file, used to recover thousands of images from a Photorec data dump.\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.joshualyman.com\/#website\",\"url\":\"https:\/\/www.joshualyman.com\/\",\"name\":\"Joshua Lyman.com\",\"description\":\"Technologist, founder, and teacher\",\"publisher\":{\"@id\":\"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.joshualyman.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e\",\"name\":\"Joshua\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.joshualyman.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/i1.wp.com\/www.joshualyman.com\/wp-content\/uploads\/2020\/12\/Headshot-2020-Square-Small.jpg?fit=1000%2C1000\",\"contentUrl\":\"https:\/\/i1.wp.com\/www.joshualyman.com\/wp-content\/uploads\/2020\/12\/Headshot-2020-Square-Small.jpg?fit=1000%2C1000\",\"width\":1000,\"height\":1000,\"caption\":\"Joshua\"},\"logo\":{\"@id\":\"https:\/\/www.joshualyman.com\/#\/schema\/person\/image\/\"},\"sameAs\":[\"https:\/\/www.joshualyman.com\",\"https:\/\/twitter.com\/jlyman\"],\"url\":\"https:\/\/www.joshualyman.com\/author\/jladmin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Using Photorec and a bash script to recover 51,407 images - Joshua Lyman.com","description":"Bash script to sort files into folders created from modified date on the file, used to recover thousands of images from a Photorec data dump.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/","twitter_card":"summary_large_image","twitter_title":"Using Photorec and a bash script to recover 51,407 images - Joshua Lyman.com","twitter_description":"Bash script to sort files into folders created from modified date on the file, used to recover thousands of images from a Photorec data dump.","twitter_creator":"@jlyman","twitter_site":"@jlyman","twitter_misc":{"Written by":"Joshua","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/#article","isPartOf":{"@id":"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/"},"author":{"name":"Joshua","@id":"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e"},"headline":"Using Photorec and a bash script to recover 51,407 images","datePublished":"2011-05-07T19:57:51+00:00","dateModified":"2013-03-23T11:46:58+00:00","mainEntityOfPage":{"@id":"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/"},"wordCount":434,"commentCount":4,"publisher":{"@id":"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e"},"keywords":["bash","images","linkedin","photography","photorec","recovery","scripting","scripts"],"articleSection":["Programming"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/","url":"https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/","name":"Using Photorec and a bash script to recover 51,407 images - Joshua Lyman.com","isPartOf":{"@id":"https:\/\/www.joshualyman.com\/#website"},"datePublished":"2011-05-07T19:57:51+00:00","dateModified":"2013-03-23T11:46:58+00:00","description":"Bash script to sort files into folders created from modified date on the file, used to recover thousands of images from a Photorec data dump.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.joshualyman.com\/2011\/05\/using-photorec-and-a-bash-script-to-recover-51407-images\/"]}]},{"@type":"WebSite","@id":"https:\/\/www.joshualyman.com\/#website","url":"https:\/\/www.joshualyman.com\/","name":"Joshua Lyman.com","description":"Technologist, founder, and teacher","publisher":{"@id":"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.joshualyman.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/www.joshualyman.com\/#\/schema\/person\/4679b2b5b142a2c904bb46f97a04bd9e","name":"Joshua","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.joshualyman.com\/#\/schema\/person\/image\/","url":"https:\/\/i1.wp.com\/www.joshualyman.com\/wp-content\/uploads\/2020\/12\/Headshot-2020-Square-Small.jpg?fit=1000%2C1000","contentUrl":"https:\/\/i1.wp.com\/www.joshualyman.com\/wp-content\/uploads\/2020\/12\/Headshot-2020-Square-Small.jpg?fit=1000%2C1000","width":1000,"height":1000,"caption":"Joshua"},"logo":{"@id":"https:\/\/www.joshualyman.com\/#\/schema\/person\/image\/"},"sameAs":["https:\/\/www.joshualyman.com","https:\/\/twitter.com\/jlyman"],"url":"https:\/\/www.joshualyman.com\/author\/jladmin\/"}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7V8cO-2W","_links":{"self":[{"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/posts\/182"}],"collection":[{"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/comments?post=182"}],"version-history":[{"count":13,"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/posts\/182\/revisions"}],"predecessor-version":[{"id":402,"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/posts\/182\/revisions\/402"}],"wp:attachment":[{"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/media?parent=182"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/categories?post=182"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.joshualyman.com\/wp-json\/wp\/v2\/tags?post=182"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}