|
|
|
|
|
User-Agents Database |
User Agent Details of Search.Aus-AU.COM
| Name: | Search.Aus-AU.COM |
| URL: | http://search.aus-au.com/ |
| User Agent Key: | |
| User Agents: | |
| IP Ranges: | |
| Adheres robots.txt: | Yes |
| Adheres META tag: | Yes |
| Type(s): |
IndexingTool
|
| Verified: | | | Info: | Search-AU is a development tool I have built to investigate the power of a search engine and web crawler to give me access to a database of web content ( html / url's ) and address's etc from which I hope to build more accurate stats about the .au zone's web content. the robot started crawling from http://www.geko.net.au/ on march 1st, 1998 and after nine days had 70mb of compressed ascii in a database to work with. i hope to run a refresh of the crawl every month initially, and soon every week bandwidth and cpu allowing. if the project warrants further development, i will turn it into an australian ( .au ) zone search engine and make it commercially available for advertising to cover the costs which are starting to mount up. --dez (980313 - black friday!) | | Date Added: | 27/07/2005 23:35:45 | | |  |
User Agents - Search
|
|
