Analyzing “ROBOTS.TXT” & Decrypting Hash Function Method

As per Spiffomatic64 said in the message at the starting that he has found message digest. So what it is?

A cryptographic hash function is a hash function that can be defined as a deterministic procedure that takes an arbitrary block of data and returns a fixed-size bit string, the (cryptographic) hash value, such that an accidental or intentional change to the data will change the hash value. The data to be encoded is often called the “message,” and the hash value is sometimes called the message digest or simply digests.

There are many websites that shows their password in hash format and if it is decrypt able hash function u can always crack it using different software or making own C++ programs

I can’t stress this enough but one of the first things you should do after viewing every page is to view the source code of every page. Unfortunately for us this reveals nothing.
You take a look at the News page and you decided to read it. There are some things in there that you think might give you a clue like the administrator’s girlfriends name. But this is all useless. But then you read:
“Google was grabbing links it shouldn’t be so I have taken extra precautions”
Now this might be interesting. In order to hide files from search engine spiders you need a robots.txt file. So what is basically robots.txt do?

Think of search engine spiders as very simple and automated data retrieval program, traveling the web to find information and links. They only absorb what they can see, and while a picture is worth a thousand words to a person, its worth zero to a search engine. They can only read and understand text, and then only if its laid out in a format that is tuned to their needs. Ensuring that they can access and read all the content from within a web site must be a core part of any search engine optimization strategy.

When a web page is submitted to a search engine, the url is added to the search engine spiders queue of websites to visit. Even if you don’t directly submit a website, or the web pages within a website, most robots will find the content within your website if other websites link to it. Thats part of a process referred to as building reciprocal links. This is one of the reasons why it is crucial to build the link popularity for a website, and to get links from other topical sites back to yours. It should be part of any website marketing strategy you opt in for.

When a search engine spider arrives at a website, the bots are supposed to check to see if you have a robots.txt file. This file is used to tell spiders which areas of your site are off-limits to them. Typically these may be directories containing files the spider doesn’t need to concern itself with. Some spiders will ignore these files. However, all search engine spiders do look for the file. Every website should have one, even if it is blank. Its just one of the things that the search engines look for.

So let’s take a look at what they are hiding:

http://www.hackthissite.org/missions/realistic/5/robots.txt

will show you the following:

User-agent: *
Disallow: /lib
Disallow: /secret

This indicates that there are two directories /lib and /secret the administrator does not want search engines to look at. So let us look at them.

We check out secret first for obvious reasons. We look at admin.php and get an “Invalid Password” message. We look at admin.bak.php and get:

error matching hash 23587b1fe357d4c1e79145d66ffdf670

This hash might be different for you. If we remember what Spiffomatic64 said about the hash being a “message digest” (MD), which might be the password you need to enter.

But what algorithm do we use? Let’s take a look at the other directory /lib inside there is a hash file. Lets download and take a look. Wow a lot of garbage. Take a good look through the file and you might come across this:

$FreeBSD: src/lib/csu/i386-elf/crti.S,v 1.7 2005/05/19 07:31:06 dfr Exp $ %02x
Error: MDupdate MD already done.
Error: MDupdate called with illegal count value %d. MD4 time trial. Processing 1 million 64-character blocks… is digest of 64M byte test input. Seconds to process test input: %g

If you noticed I bolded MD4. That means that this hash was encrypted with the MD4 algorithm. You can use any MD4 collision finder to crack the hash.

A popular one is MDCrack NG. Here is the usage and output of MDCrack:
Usage:

MDCrack-sse.exe –algorithm=MD4 23587b1fe357d4c1e79145d66ffdf670

Output:

MDCrack-sse.exe –algorithm=MD4 23587b1fe357d4c1e79145d66ffdf670

System / Starting MDCrack v1.8(3)
System / Running as MDCrack-sse.exe –algorithm=MD4 23587b1fe357d4c1e79145d66ffdf670
System / Charset is: abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ
System / Detected processor(s): 2 x INTEL Itanium | MMX | SSE | SSE2
System / Target hash: 23587b1fe357d4c1e79145d66ffdf670
System / >> Using MD4 cores: maximal candidate/user salt size: 16/54 bytes
Info   / Press ESC for available runtime shortcuts (Ctrl-c to quit)
Info   / Thread #0: >> Using Core 1
Info   / Thread #1: >> Using Core 1
Info   / Thread #0: Candidate size:  1 ( + user salt: 0 )
Info   / Thread #1: Candidate size:  1 ( + user salt: 0 )
Info   / Thread #0: Candidate size:  2 ( + user salt: 0 )
Info   / Thread #1: Candidate size:  2 ( + user salt: 0 )
Info   / Thread #0: Candidate size:  3 ( + user salt: 0 )
Info   / Thread #1: Candidate size:  3 ( + user salt: 0 )
Info   / Thread #1: Candidate size:  4 ( + user salt: 0 )
Info   / Thread #0: Candidate size:  4 ( + user salt: 0 )
Info   / Thread #1: Candidate size:  5 ( + user salt: 0 )
Info   / Thread #0: Candidate size:  5 ( + user salt: 0 )
———————————————————-/ Thread #1 (Success) —-
System / Thread #1: Collision found: d61bb
Info   / Thread #1: Candidate/Hash pairs tested: 14 583 394 ( 1.46e+007 ) in 968ms
Info   / Thread #1: Allocated key space: 2.42e+028 candidates, 0.00% done
Info   / Thread #1: Average speed: ~ 15 054 401 ( 1.51e+007 ) h/s

Where it says Collision found: d61bb is the password. You can also use Cain and Able password cracker or you can always write your own programming if you understand well the mechanisms.

About the Author

Aloha, I'm Amit Ghosh, a web entrepreneur and avid blogger. Bitten by entrepreneurial bug, I got kicked out from college and ended up being millionaire and running a digital media company named Aeron7 headquartered at Lithuania.

Related Posts

Basic Mission 2   I can again call you a pretty dumb as per you are now again viewing the walk...