• Home »
  • Security »
  • Backtrack 4: Information Gathering: Archive: Metagoofil – Extract metadata from public documents

Backtrack 4: Information Gathering: Archive: Metagoofil – Extract metadata from public documents

One good thing about writing articles on tools is you get to test out lots of different stuff you may not have normally used. One of these tools for me was Metagoofil. Metagoofil is a tool for written in Python for extracting the metadata from public documents (pdf,doc,xls,ppt) available in the target websites. This information could be useful because you can get valid usernames, or people names, for using later in brute force password attacks (vpn, ftp, webapps etc.). The tool first queries Google for different filetypes that can have useful metadata (pdf, doc, xls,ppt,etc), then it downloads those documents to the disk and run the program “extract” on every file. It will generate a HTML page with the results of the metadata extracted, plus a list of potential usernames.

Lets take a look at the help menu:

bash

  1. root@666:/pentest/enumeration/google/metagoofil# ./metagoofil.py -h
  2.  
  3. *************************************
  4. *MetaGooFil Ver. 1.4b               *
  5. *Coded by Christian Martorella      *
  6. *Edge-Security Research             *
  7. *cmartorella@edge-security.com      *
  8. *************************************
  9.  
  10. MetaGooFil 1.4
  11.  
  12. usage: metagoofil options
  13.  
  14.         -d: domain to search
  15.         -f: filetype to download (all,pdf,doc,xls,ppt,odp,ods, etc)
  16.         -l: limit of results to work with (default 100)
  17.         -o: output file, html format.
  18.         -t: target directory to download files.
  19.  
  20.         Example: metagoofil.py -d microsoft.com -l 20 -f all -o micro.html -t micro-files

Lets run a check for pdf’s on the EPA website where I am certain they have lots of PDF files:

bash

  1. root@666:/pentest/enumeration/google/metagoofil# ./metagoofil.py -d www.epa.gov -l 100 -f pdf -o example.html -t deleteme
  2.  
  3. *************************************
  4. *MetaGooFil Ver. 1.4b               *
  5. *Coded by Christian Martorella      *
  6. *Edge-Security Research             *
  7. *cmartorella@edge-security.com      *
  8. *************************************
  9.  
  10. [+] Command extract found, proceeding with leeching
  11. [+] Searching in www.epa.gov for: pdf
  12.  350000
  13. [+] Total results in google: 350000
  14. [+] Limit:  100
  15. [+] Searching results: 0
  16. [+] Searching results: 20
  17. [+] Searching results: 40
  18. [+] Searching results: 60
  19. [+] Searching results: 80
  20. [+] Directory deleteme already exist, reusing it
  21.         [ 1/100 ] http://www.epa.gov/clearskies/Air_005.pdf
  22.         [ 2/100 ] http://www.epa.gov/cmop/docs/022red.pdf
  23.         [ 3/100 ] http://www.epa.gov/nps/natlstormwater03/25Neiswender.pdf
  24.         [ 4/100 ] http://www.epa.gov/nps/natlstormwater03/41Weinstein.pdf
  25.         [ 5/100 ] http://www.epa.gov/PR_Notices/pr2001-4.pdf
  26.         [ 6/100 ] http://www.epa.gov/nps/natlstormwater03/17Hillegass.pdf
  27.         [ 7/100 ] http://www.epa.gov/waters/tmdldocs/11536_CaneyForkSed_080105.pdf
  28.         [ 8/100 ] http://www.epa.gov/PR_Notices/pr2000-8.pdf
  29.         [ 9/100 ] http://www.epa.gov/nps/natlstormwater03/07Comstock.pdf
  30.         [ 10/100 ] http://www.epa.gov/PR_Notices/pr2000-9.pdf
  31.         [ 11/100 ] http://www.epa.gov/nps/natlstormwater03/08Dorava.pdf
  32.         [ 12/100 ] http://www.epa.gov/fedfac/pdf/uxo_risk_assmnt_rvw_2004.pdf
  33.         [ 13/100 ] http://www.epa.gov/PR_Notices/pr98-4.pdf
  34.         [ 14/100 ] http://www.epa.gov/cmop/docs/pol006.pdf
  35.         [ 15/100 ] http://www.epa.gov/nps/natlstormwater03/40Tuomari.pdf
  36.         [ 16/100 ] http://www.epa.gov/nps/natlstormwater03/29Reese.pdf
  37.         [ 17/100 ] http://www.epa.gov/ttncatc1/dir1/fsetling.pdf
  38.         [ 18/100 ] http://www.epa.gov/nps/natlstormwater03/14Greer.pdf
  39.         [ 19/100 ] http://www.epa.gov/cmop/docs/002red.pdf
  40.         [ 20/100 ] http://www.epa.gov/opp00001/regulating/fifra.pdf
  41.         [ 21/100 ] http://www.epa.gov/nps/natlstormwater03/45Hollister.pdf
  42.         [ 22/100 ] http://www.epa.gov/nps/natlstormwater03/10Duma.pdf
  43.         [ 23/100 ] http://www.epa.gov/lead/pubs/span_web_secure.pdf
  44.         [ 24/100 ] http://www.epa.gov/endo/pubs/notes_for_appendix_9.pdf
  45.         [ 25/100 ] http://www.epa.gov/oppsrrd1/REDs/0630red.pdf
  46.         [ 26/100 ] http://www.epa.gov/nps/natlstormwater03/38Stephens.pdf
  47.         [ 27/100 ] http://www.epa.gov/nhsrc/pubs/600r04065.pdf
  48.         [ 28/100 ] http://www.epa.gov/asbestos/pubs/vairesearchmethodfinal.pdf
  49.         [ 29/100 ] http://www.epa.gov/nps/natlstormwater03/34Shepard.pdf
  50.         [ 30/100 ] http://www.epa.gov/cmop/docs/013red.pdf
  51.         [ 31/100 ] http://www.epa.gov/endo/pubs/male_pubertal_lit_study_descriptions_table_final.pdf
  52.         [ 32/100 ] http://www.epa.gov/ocr/docs/42usc2000d.pdf
  53.         [ 33/100 ] http://www.epa.gov/cpd/pdf/maccppfinal.pdf
  54.         [ 34/100 ] http://www.epa.gov/nhsrc/pubs/vrUltrastrip032704.pdf
  55.         [ 35/100 ] http://www.epa.gov/ttncatc1/dir1/fsprytwr.pdf
  56.         [ 36/100 ] http://www.epa.gov/nps/natlstormwater03/12Gabbard.pdf
  57.         [ 37/100 ] http://www.epa.gov/endo/pubs/appendix_iv_feed_analysis_reports_rti_fp.pdf
  58.         [ 38/100 ] http://www.epa.gov/nps/natlstormwater03/32Sands.pdf
  59.         [ 39/100 ] http://www.epa.gov/nps/natlstormwater03/11Echols.pdf
  60.         [ 40/100 ] http://www.epa.gov/dced/pdf/ptfd_primer.pdf
  61.         [ 41/100 ] http://www.epa.gov/nps/natlstormwater03/02Booth.pdf
  62.         [ 42/100 ] http://www.epa.gov/ocr/docs/40p0007.pdf
  63.         [ 43/100 ] http://www.epa.gov/ttncatc1/dir1/rblc2002.pdf
  64.         [ 44/100 ] http://www.epa.gov/oppsrrd1/REDs/3082red.pdf
  65.         [ 45/100 ] http://www.epa.gov/cmop/docs/pol003.pdf
  66.         [ 46/100 ] http://www.epa.gov/PR_Notices/pr97-2.pdf
  67.         [ 47/100 ] http://www.epa.gov/nps/natlstormwater03/47Strecker.pdf
  68.         [ 48/100 ] http://www.epa.gov/PR_Notices/pr2001-3.pdf
  69.         [ 49/100 ] http://www.epa.gov/nps/natlstormwater03/15Groner.pdf
  70.         [ 50/100 ] http://www.epa.gov/nps/natlstormwater03/31Roa.pdf
  71.         [ 51/100 ] http://www.epa.gov/ttncatc1/dir1/cs6ch2.pdf
  72.         [ 52/100 ] http://www.epa.gov/endo/pubs/notes_for_appendix_6.pdf
  73.         [ 53/100 ] http://www.epa.gov/oust/mtbe/oxytable.pdf
  74.         [ 54/100 ] http://www.epa.gov/nps/natlstormwater03/28Pitt.pdf
  75.         [ 55/100 ] http://www.epa.gov/cmop/docs/001red.pdf
  76.         [ 56/100 ] http://www.epa.gov/cmop/docs/pol002.pdf
  77.         [ 57/100 ] http://www.epa.gov/ogc/china/eis.pdf
  78.         [ 58/100 ] http://www.epa.gov/msbasin/pdf/symposia_ia_presentations.pdf
  79.         [ 59/100 ] http://www.epa.gov/npdescan/FL0000701FP.pdf
  80. Florida Department of Environmental Protection
  81. Title(Rayonier Performance Fibers, LLC)
  82.         [ 60/100 ] http://www.epa.gov/nps/natlstormwater03/27Claytor.pdf
  83.         [ 61/100 ] http://www.epa.gov/nps/natlstormwater03/Johnsposter.pdf
  84.         [ 62/100 ] http://www.epa.gov/ttncatc1/dir1/fwespwpl.pdf
  85.         [ 63/100 ] http://www.epa.gov/ogd/forms/Buy_Am.pdf
  86.         [ 64/100 ] http://www.epa.gov/cmop/docs/pol005.pdf
  87.         [ 65/100 ] http://www.epa.gov/ocr/docs/33usc1251.pdf
  88.         [ 66/100 ] http://www.epa.gov/ttncatc1/dir1/icboiler.pdf
  89.         [ 67/100 ] http://www.epa.gov/nps/natlstormwater03/Mullinposter.pdf
  90.         [ 68/100 ] http://www.epa.gov/nhsrc/pubs/vrWatts062404.pdf
  91.         [ 69/100 ] http://www.epa.gov/watersense/docs/AWWA_Journal_showerheads.pdf
  92.         [ 70/100 ] http://www.epa.gov/asbestos/pubs/aherarequirements.pdf
  93.         [ 71/100 ] http://www.epa.gov/endo/pubs/trc_fr_101.pdf
  94.         [ 72/100 ] http://www.epa.gov/oust/mtbe/omethods.pdf
  95.         [ 73/100 ] http://www.epa.gov/greenchill/downloads/Bohn_Secondary_Loop_WP.pdf
  96.         [ 74/100 ] http://www.epa.gov/nps/natlstormwater03/35Sloan.pdf
  97.         [ 75/100 ] http://www.epa.gov/ttncatc1/dir1/fmechan.pdf
  98.         [ 76/100 ] http://www.epa.gov/nps/natlstormwater03/03Bretsch.pdf
  99.         [ 77/100 ] http://www.epa.gov/opprd001/factsheets/diclosulam.pdf
  100.         [ 78/100 ] http://www.epa.gov/hurricane/pdf/homeleadremodeling_brochure.pdf
  101.         [ 79/100 ] http://www.epa.gov/cmop/docs/red001.pdf
  102.         [ 80/100 ] http://www.epa.gov/nhsrc/pubs/vrSears090704.pdf
  103.         [ 81/100 ] http://www.epa.gov/PR_Notices/pr98-3.pdf
  104.         [ 82/100 ] http://www.epa.gov/agstar/pdf/wefjune2003.pdf
  105.         [ 83/100 ] http://www.epa.gov/nps/natlstormwater03/13Gentile.pdf
  106.         [ 84/100 ] http://www.epa.gov/nps/natlstormwater03/09Dreyfuss.pdf
  107.         [ 85/100 ] http://www.epa.gov/PR_Notices/pr2002-2.pdf
  108.         [ 86/100 ] http://www.epa.gov/PR_Notices/pr2001-6.pdf
  109.         [ 87/100 ] http://www.epa.gov/nps/natlstormwater03/16Hackett.pdf
  110.         [ 88/100 ] http://www.epa.gov/nps/natlstormwater03/33Shapiro.pdf
  111.         [ 89/100 ] http://www.epa.gov/nps/natlstormwater03/36Solek.pdf
  112.         [ 90/100 ] http://www.epa.gov/waters/tmdldocs/22726_HiwasseeSed.pdf
  113.         [ 91/100 ] http://www.epa.gov/endo/pubs/attachment_a1_ama_test_method.pdf
  114.         [ 92/100 ] http://www.epa.gov/nps/natlstormwater03/21Malec.pdf
  115.         [ 93/100 ] http://www.epa.gov/nhsrc/pubs/vsUltrastrip032704.pdf
  116.         [ 94/100 ] http://www.epa.gov/nps/natlstormwater03/20Liptan.pdf
  117.         [ 95/100 ] http://www.epa.gov/glnpo/lakesuperior/epo1998.pdf
  118.         [ 96/100 ] http://www.epa.gov/npdescan/okg950000gfp.pdf
  119.         [ 97/100 ] http://www.epa.gov/region4/waste/martincs.pdf
  120.         [ 98/100 ] http://www.epa.gov/endo/pubs/fish_assay_charge_questions.pdf
  121.         [ 99/100 ] http://www.epa.gov/oust/mtbe/mtbemap.pdf
  122.         [ 100/100 ] http://www.epa.gov/gasstar/documents/cast_iron_mains.pdf
  123.  
  124. Usernames found:
  125. ================
  126. BG34061
  127. "ÂÃòT▒3U
  128. Author(Florida Department of Environmental Protection)Florida Department of Environmental Protection
  129. ÂÂ3Ã^ÃÃÂWÿÃSÂÃéNúB«`Â,ÂÃs
  130. Âë!çì»eú§:ÂÃàLÂ9ÂÃ
  131. Blumenstein
  132. Ã\(¤qÃÃÂ`¶Ã°ÃÂWÂÃÂ2þ
  133. Ãt+´HºÂÂ%ÃÃ3»
  134.  
  135. Paths found:
  136. ============
  137. \
  138. Title(Rayonier Performance Fibers, LLC)/Author(Florida Department of Environmental Protection)/Keywords(NPDES Permit, Rayonier Performance Fibers LLC, Fermandina Beach, Nassau County, Florida, WWTP, Wastewater Treatment Plant, wastewater)/Subject(NPDES Permit for Rayonier Performance Fibers, LLC in Fermandina Beach, Nass

So now we can look in the same directory that we ran the tool in for our html file which will be called example.html:

bash

  1. root@666:/pentest/enumeration/google/metagoofil# ls -la
  2. total 76
  3. drwxr-xr-x 3  502 root  4096 May 28 09:16 .
  4. drwxr-xr-x 5 root root  4096 May 28 09:09 ..
  5. -rwxr-xr-x 1  502 root 15238 May 11 19:38 COPYING
  6. -rwxr-xr-x 1  502 root    97 May 11 19:38 LICENSES
  7. -rwxr-xr-x 1  502 root  2226 May 11 19:38 README
  8. drwxr-xr-x 2 root root  4096 May 28 09:19 deleteme
  9. -rw-r--r-- 1 root root 27067 May 28 14:20 example.html
  10. -rwxr-xr-x 1  502 root 11926 May 28 09:10 metagoofil.py

Now we can open the file with anyweb browser and get a nice graphical metadata breakdown of each file:
5-28-2010-2-26-09-PM

Metagoofil can be used with any of the file types listed or by simply passing the “-f all” argument you can have it check for other file types. I will most definitely be using this tool in the future.

Metadata (Paperback)


List Price: $72.00 USD
New From: $60.09 USD In Stock
Used from: $44.99 USD In Stock

Google Hacking for Penetration Testers (Paperback)


List Price: $49.95
New From: $35.40 USD In Stock
Used from: $2.99 USD In Stock

Share