IBM DeveloperWorks explains how to build a web spider on Linux:

Web spiders are software agents that traverse the Internet gathering, filtering, and potentially aggregating information for a user. Using common scripting languages and their collection of Web modules, you can easily develop Web spiders. This article shows you how to build spiders and scrapers for Linux® to crawl a Web site and gather information, stock data, in this case.

Ruby and Python are the languages used.

This entry was posted on Monday, November 20th, 2006 at 10:15 pm and is filed under Linux, Programming, Python, Ruby. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.