It never ends!

[Goshen]

Check this out...

If you go to
http://pages.alexa.com/associates/sitereport.html

And ask it to spider your site, it follows ALL of your book links.

I am worried it will never end, I had 2000 book in the bookshop, right now it has been crawling for over an hour and now I have 3321 books and it is still going up.

[Greg]

Yeah, this turned out to be pretty cool code I guess. :)

[Goshen]

ok, it ended it times out thankfully :), here is part of the report.

7078 unique URLs were crawled for this report.

We may not have retrieved all urls from your site due to a
timeout error. Here are possible reasons why a timeout
occurred: 1) your site has a large number of pages, 2) your
site contains dynamic functionality which our software cannot
understand.

[Greg]

hehe, nice crawl. I'm waiting for the day someone has a cool story about a rediculously deep crawl. Should be fun.