Academic Journals Database
Disseminating quality controlled scientific knowledge

Parallel and Distributed Algorithms for Frequent Pattern Mining in Large Databases

Author(s): Tanbeer Syed | Ahmed Chowdhury | Jeong Byeong-Soo

Journal: IETE Technical Review
ISSN 0256-4602

Volume: 26;
Issue: 1;
Start page: 55;
Date: 2009;
Original page

Keywords: Data mining | Frequent patterns | Knowledge discovery | Large-scale databases | Parallel and distributed processing | Tree restructuring.

Mining frequent patterns (FP) from large-scale databases has emerged as an important problem in the data mining and knowledge discovery research community. A significant number of parallel and distributed FP mining algorithms have been proposed, when the database is large and/or distributed. Among them, parallelization of the FP-growth algorithm using the FP-tree has been proved to be more efficient, when compared to the Apriori -based approaches. However, the FP-tree- based techniques suffer from two major limitations - multiple database scans requirement (i.e., high I/O cost) and huge communication overhead. Therefore, in this paper, we propose a novel tree structure, called PP-tree (Parallel Pattern tree) that significantly reduces the I/O cost by capturing the database contents with a single scan and facilitates efficient FP-growth mining on it. Our parallel algorithm works independently at each local site and merges the locally generated global frequent patterns at the final stage, thereby reducing inter-processor communication overhead and getting a high degree of parallelism. Extensive experimental study on datasets of different types reflects that parallel and distributed FP mining with our PP-tree is highly efficient on large databases.
Save time & money - Smart Internet Solutions      Why do you need a reservation system?