Integrated intelligent algorithms for adaptive clustering and classification of structured data from unstructured web documents

The proliferation of software applications has resulted in the ubiquitousness of the so-called big data from several unstructured web documents obtained from the rapidly growing Internet. Information extraction from these unstructured web documents has posed several challenges with several proposed methodologies from different quarters where data clustering and classification form the basis for the big data analysis. This paper presents integrated intelligent algorithms for the adaptive clustering and classification of structured data from unstructured web documents. The integrated intelligent algorithms consist of two parts. Firstly, two adaptive clustering algorithms based on Mamdani-type rule-based fuzzy model are proposed, namely: adaptive fuzzy C-means clustering (AFCC) and adaptive mountain climbing clustering (AMCC). Secondly, a new neural network-based Bayesian linear regression classification algorithm using function based approach with Gaussian mixture model. The major advantage of the newly proposed rule-based integrated intelligent algorithms is that the set of rules are automatically inferred and incorporated into the clustering algorithm which enhances the automatic regression vector construction for the neural network-based implementation of the proposed Bayesian linear regression classification based on the function space algorithm using the Gaussian mixture models. The implementation and application of the integrated intelligent algorithms to five case studies from the Kogi State University (KSU) Anyigba, Nigeria website shows the superior performances of the AMCC over the AFCC in terms of mean square errors of the training and testing data as well as the percentage accuracies and probability distributions. The aforementioned results render the proposed integrated intelligent algorithms based on the AMCC and the neural network-based Bayesian classification algorithms suitable for online adaptive clustering and classification of big data from unstructured web documents from dynamic webpages.