I went looking for something on a website that’s been taken down. No problem, right? I’ll just go into the Wayback Machine. I’ve gotten it off the Wayback Machine before; I know it’s there. I even bookmarked it! Except what happens when I go to that bookmark? “Can’t display because of site’s robots.txt.”
If they’re going to retroactively remove content that was freely available simply because someone– probably someone entirely different– bought up a domain just for squatting purposes and put that in there, they don’t deserve to call themselves an archive. Archives are about preservation, and there’s been enough outcry on their own forums that they have to be aware of the problem. But they keep holding that it’s some kind of ethical bound, to let people– who may not even be the originators of the content– remove things from the archive via an automated process. I see this as Big Brother, and a sign of the coming media control over all our thoughts. When you put things on the internet, they’re there forever, and we’re all told that repeatedly when it comes to putting things up there; that it will never go away. Except now here they are participating in making stuff go away. Just like Winston altering the news.
Since I get “cannot find server” at the website when I click on their link to view the robots.txt directly, that suggests something even more insidious: that they’re removing websites that no longer exist from their archive, thereby directly violating their own preservationist mission statement. And if they’re doing that? There’s no real reason for them to exist. It serves no real use to see the history of existing website– certainly they change, but not all that dynamically. The real value has always been in the knowledge that what they archive stays around, that they’re dedicated to keeping it there. Except, apparently, they no longer are.
Certainly they should cease calling themselves an archive, and if they’ve lost their way so badly as to stop being one, perhaps they should consider whether they ought to continue existing at all, or step aside and leave room for someone more dedicated to the principles to stand up in their place. Because this? This is an unforgivable betrayal.