data mining is a business intelligence application

This chapter provides an overview of existing research in the area of BDQ definitions and measurement for data mining for BI, analyzes the gaps therein, and provides a direction for future research and practice in this area. Defining the factors of DQ for data mining and their measurement for a BI system has been one of the major challenges for researchers as well as practitioners. However, there is no agreement yet on definitions or measurements or interpretations of DQ for data mining. Objective measures, training/modeling approaches, and subjective measures are three major approaches that exist to measure DQ for data mining. DQ for data mining in business intelligence applications should be aligned with the objectives of the BI application. Data quality in data mining refers to the quality of the patterns or results of the models built using mining algorithms. However, very limited advancement has been made in research and industry in the topic of BDQ and their relevance and criticality for data mining and BI applications.

These have created a need for expanding the traditional data quality (DQ) factors into BDQ (big data quality) factors besides the need for new BDQ assessment and measurement frameworks for data mining and BI applications. These new complexities and challenges arise from the underlying dimensions of big data (volume, variety, velocity, and value) together with the ability to consume data at various stages of transition from raw data to standardized datasets.

In a big data environment, data is consumed in various states and various forms serving as input for data mining, and this is the main source of added complexity.

AbstractStudy of data quality for data mining application has always been a complex topic in the recent years, this topic has gained further complexity with the advent of big data as the source for data mining and business intelligence (BI) applications.

Data mining is a business intelligence application