Story of a post-modernist partnership tackling life's little roundabouts.
Tuesday, August 06, 2024
Alvin's Not So Smart Web-Trap
Alvin starts to realise his smart Web-Trap (using robot.txt) to block webcrawlers etc, stops his website being indexed and thus no one can find him, but the AI scrapers can just ignore it.
No comments:
Post a Comment