Reply to this post | See parent post | Go Back

View Post [edit]

Poster:	Brak	Date:	Jan 30, 2005 2:31am
Forum:	web	Subject:	Re: Full-text search of the Wayback archive

Now that's a fascinating idea. It's not the reason it was taken down. We hope to have that functionality available again.

Doing a distributed index sounds like a worthy p2p project for all sorts of reasons. The problem with projects of this scale might be the sheer size of the data. If the index were, let's say, 25 Terabytes, then if each user donated 20GB of hard drive space, it would take, in theory, a minimum of 1250 users, and more realistically 5000 users (taking into account redundancy and load sharing). That's a taller order than most p2p apps. Anything is possible, however.

For now, stay tuned...