Data Mining Basics
Definition and Purpose of Data Mining:
Data mining is a relatively new term that refers to the process by which predictive patterns are extracted from information.
Data is often stored in large, relational databases and the amount of information stored can be substantial. But what does this data mean? How can a company or organization figure out patterns that are critical to its performance and then take action based on these patterns? To manually wade through the information stored in a large database and then figure out what is important to your organization can be next to impossible.
This is where data mining techniques come to the rescue! Data mining software analyzes huge quantities of data and then determines predictive patterns by examining relationships.
Data Mining Techniques:
There are numerous data mining (DM) techniques and the type of data being examined strongly influences the type of data mining technique used.
Note that the nature of data mining is constantly evolving and new DM techniques are being implemented all the time.
Generally speaking, there are several main techniques used by data mining software: clustering, classification, regression and association methods.
Clustering:
Clustering refers to the formation of data clusters that are grouped together by some sort of relationship that identifies that data as being similar. An example of this would be sales data that is clustered into specific markets.
Classification:
Data is grouped together by applying known structure to the data warehouse being examined. This method is great for categorical information and uses one or more algorithms such as decision tree learning, neural networks and “nearest neighbor” methods.
Regression:
Regression utilizes mathematical formulas and is superb for numerical information. It basically looks at the numerical data and then attempts to apply a formula that fits that data.
New data can then be plugged into the formula, which results in predictive analysis.
Association:
Often referred to as “association rule learning,” this method is popular and entails the discovery of interesting relationships between variables in the data warehouse (where the data is stored for analysis). Once an association “rule” has been established, predictions can then be made and acted upon. An example of this is shopping: if people buy a particular item then there may be a high chance that they also buy another specific item (the store manager could then make sure these items are located near each other).
Data Mining and the Business Intelligence Stack:
Business intelligence refers to the gathering, storing and analyzing of data for the purpose of making intelligent business decisions. Business intelligence is commonly divided into several layers, all of which constitute the business intelligence “stack.”
The BI (business intelligence) stack consists of: a data layer, analytics layer and presentation layer.
The analytics layer is responsible for data analysis and it is this layer where data mining occurs within the stack. Other elements that are part of the analytics layer are predictive analysis and KPI (key performance indicator) formation.
Data mining is a critical part of business intelligence, providing key relationships between groups of data that is then displayed to end users via data visualization (part of the BI stack’s presentation layer). Individuals can then quickly view these relationships in a graphical manner and take some sort of action based on the data being displayed.