Perhaps it’s just coincidence, but we’ve been noticing a rather alarming trend of late, namely robots.txt files being used to block all search engine robots from accessing what should otherwise be a fully accessible web site. In each case, the discovery has come about after a client asking us why their web site has suddenly [...]