What is robots.txt. What is a Web Bot? What does it do? What does a robots.txt file contain? Where to store robots.txt file? What happened if there is no such a file called robots.txt.