Hidden Resource in Robots.txt

Impact: Medium


Hidden resources in robots.txt refer to sensitive paths or directories that are inadvertently exposed in the robots.txt file. The robots.txt file is used to instruct web robots on which parts of a website to avoid crawling or indexing. While intended for cooperation with search engine crawlers, disclosing sensitive paths can provide malicious actors with valuable information that could be exploited for unauthorized access or to identify potential attack vectors.


To mitigate the risk of exposing hidden resources in robots.txt, carefully review and sanitize the contents of the file to avoid revealing sensitive paths or directories. Ensure that only necessary and safe paths are included in the robots.txt file, and regularly review and update it to remove any inadvertently disclosed information.


Last updated on May 13, 2024

Use SmartScanner Free version to test for this issue