Academic Journals Database
Disseminating quality controlled scientific knowledge

Compression Technique on Crawled Data for High Performance of Web Spider

Author(s): Avanish Kumar Singh | Pratap Singh Patwal

Journal: International Journal of Artificial Intelligence & Knowledge Discovery
ISSN 2231-2021

Volume: 1;
Issue: 1;
Start page: 3;
Date: 2011;
Original page

Web spider is one of the most important components of the search engine, which has been used for accessing the web pages through the World Wide Web(WWW). When we crawl the web pages with different data sizes, then we can access limited web pages according to transfer rate of total data size in per unit time. In case of increasing the data size of web pages, number of web pagesdecrease. So it affects the performance of web spider. This paper describes the issue of data size of web pages, crawled by the web spider. This problem has been implemented here regarding to achieve high performance of web spider by using compression technique, which reduces the data size with big compression ratio of crawled web pages.

Tango Rapperswil
Tango Rapperswil

     Affiliate Program