What is Multivariate Data Analysis

What is multivariate data?

A variate is a weighted combination of variables. Multivariate, bivariate, or univariate are used to refer to a classification of data on the basis of the number of variables. The variables are actually the number of objects that are considered as samples in any experiment. The data sets can be of three different types. 

The univariate data is very simple to analyse. The analysis is made on the basis of only one variable. Bivariate data would be used in a little more complex analysis as compared to univariate data. The bivariate data would constitute data where the analysis would be based on two variables simultaneously.

Similarly, multivariate data is that data where the analysis would be based on more than two variables for each observation. The multivariate data is used for explanatory purposes.
 
Multivariate analysis is based on multivariate statistics. This type of analysis involves the observation of more than a single statistical outcome at a time. This technique comes into use for performing trade studies across multiple designs in design and analysis. The effects of all variables would be taken into account on the responses of interest.
 
The terms 'independent variables' and 'dependent variables' also come into play. The distinction is somewhat blurred in the multivariate designs, especially in those situations where it is observational rather than experimental. The independent variable is manipulated by the researcher. 

The correlation between the independent and the dependent variable with such control would be accompanied by the control of extraneous variables. The method of collection of data does not influence the choice of the analytic tool. It can be analysed through regression analysis or ANOVA. The independent variables are often referred to as 'predictors' while the dependent variables are 'criterion variables'.

There are certain questions that can be answered with simpler statistics. This would be truer for data generated under controlled conditions. Many interesting research questions are very complex and demand multivariate models and multivariate statistics. 

High-speed computers are now available and so are multivariate software. This means, such questions can be answered by many people today through multivariate techniques that weren't accessible to most formerly. An increased interest can also be seen in the recent quasi-experimental and observational research methods.

It is often argued that multivariate analyses like multiple regression and ANCOVA can be used for providing statistical control of the extraneous variables. Multivariate analysis is quite varied and there can be variety of ways to go within one general type. This means two analyzers can reach different conclusions easily while independently analyzing the same data. 

Getting a computer to do multivariate analysis is relatively easy to learn. However, it might not be so easy to correctly interpret the output of multivariate software packages. While the conceptual world is three-dimensional and many are just comfortable with the two-dimensional space, multivariate statistics can take us into hyperspace, which is much different from where our cognitive faculties had evolved.

 

How to do a multivariate analysis?

Multivariate analysis can be performed in different ways with the available statistical tools at disposal. The tool to be used often depends on the data available. It is important to understand the appropriate uses of each technique. The purpose of the analysis is to find the best combination of the weights. Before the analysis technique is started, a clear understanding of the form and quality of data is quite essential.

• Multiple Regression Analysis - One of the most commonly used multivariate technique is multiple regression. The relationship between a single metric dependent variable and two or more independent variables is examined. The technique is quite dependent on determining the linear relationship with the lowest sum of the square variances. The assumptions of linearity, normality, and equal variances are observed quite carefully.
• Logistic Regression Analysis - The technique can be called a variation of multiple regression, allowing the prediction of an event, sometimes referred to as the 'choice models'. Utilizing non-metric dependent variables is allowed as the sole objective is to get a probabilistic assessment of the binary choice. The independent variables can be continuous or discrete. The tool helps to predict the choices consumers might make when alternatives are presented.
• Discriminant Analysis - The discriminant analysis correctly classifies people or observations into homogenous groups. The independent variables must have a high degree of normality and must be metric. The discriminant analysis is used to make a linear discriminant function that can then be used to classify the observations. The tool can be used to categorize people and put them into groups like buyers and non-buyers.
• Multivariate Analysis of Variance (MANOVA) - While ANOVA assesses the difference between groups, MANOVA is used to examine the dependence relationship between a set of dependent measures across a set of groups. Normality of the dependent variables is important in MANOVA. The null hypothesis can be rejected and the treatment differences would be determined if a significant difference can be found in the means.
• Factor Analysis - If there are too many variables in a research design, the variables can be reduced to a smaller set of factors. This is an independent technique and there is no dependent variable. The researcher looks for the underlying structure of the data matrix. The independent variables are ideally normal and continuous and at least 3 to 4 variables load onto a factor. Multicollinearity is mostly preferred between the variables as the correlations are key to the reduction of data.
• Cluster Analysis - Cluster Analysis is used to reduce the large data sets to meaningful subgroups or individual objects. This division can be performed on the similarity of the objects across a set of specified characteristics. Outliers can be a problem with this technique and can often be caused by too many irrelevant variables. The sample should represent the population and it is quite desirable to have the uncorrelated factors.

Frequently Asked Tibco Spotfire Interview Questions & Answers

 

These are some of the ways multivariate analysis can be performed.

1. What is Line Similarity?

The Line Similarity tool offers us the option to compare the lines in a line chart with a selected master line. Two new columns would be generated as a result of this. A similarity column would be generated first, which would present the similarity to the master line for each individual row. The second would be a rank column. 

The line most similar to the master line would receive the rank 1. Euclidean distance or correlation would be used to measure the distances. The empty values are generally replaced using row interpolation and are similar to something in the visualisation. The rows can also be excluded if necessary while performing calculations.

One thing that you need to keep in mind is that the Line Similarity tool cannot be used unless a suitable line chart has been created on which the calculation can be based upon. Multiple Y-axes cannot be put on an X-axis if it is both continuous and binned when a line similarity comparison is being performed. 

2. What is Clustering?

Clustering involves grouping a particular set of objects based on the characteristics and aggregating them based on the similarities. The methodology would partition the data depending on a join algorithm and is highly suited for the analysis of the desired information. This type of clustering analysis would allow the object to not be a part of a cluster or belong to it strictly, and is known as hard partitioning. 

The soft partitioning requires every object to belong to a cluster in a determined degree. There could even be more specific divisions where the objects can belong to multiple clusters. An object can be forced to participate in one cluster, group relationships, or hierarchical trees.

The partitioning can be implemented in several different ways based on the distinct models. Distinct algorithms are applied to each model and the results and properties are differentiated. The models are distinguished by the relationship and organization between them. Some of the important clustering types are:

• Centralized - Each of the clusters would be represented by a single vector mean and these mean values are used for comparison by an object value.
• Distributed - Statistical distributions dictate the building of the clusters
• Group - Only group information is available
• Connectivity - A distance function would dictate the connectivity between the models
• Density - The members of the cluster are grouped by regions where the observations found to be similar as well as dense
• Graph - Relationship and cluster organization between the members are fined by a structure linked by a graph

Different clustering algorithms exist in data mining. There is a lot that can be applied to a data set based on these cluster models. It is important to note that every method has its pros and cons. The choice of an algorithm would always depend on the characteristics of the data set and what we need to do with that.

• Centroid-based - Every cluster is referenced by a vector of values in this type of grouping method. Each object is a part of the cluster where the value difference is minimal and is compared to other clusters. The number of clusters should be defined beforehand and this is one of the biggest shortcomings of this class of algorithms. It is vastly used for optimization problems.
• Connectivity-based - Every object is related to its neighbours and would depend on the degree of that relationship on the distance between them. Clusters are created with the nearby objects and would be described as the maximum distance limit. These clusters mostly have hierarchical relationships.
• Distributed-based - The distributed methodology would combine those objects whose values belong to the same distribution. The process would need a complex and well-defined model for the random nature of value generation. These processes would achieve an optimal solution and the dependencies and correlations would be calculated.
• Density-based - This group of algorithms would create the clusters according to the high density of members of a data set in a particular location. Some distance notion is aggregated to a density standard level to the group members in the clusters. These kinds of processes might have less performance in detecting the limited areas of the group.

3. Why Clustering?

Clustering is actually a very valuable data analysis technique. There are several different applications in the world of sciences. Every data set of information that is large can be processed through this type of analysis. The produced results would be great and have many distinct types of data. It can be said that one of the most important applications is related to image processing and detecting distinct patterns in image data. It would be very effective in biological researches and distinguishing objects and identifying patterns. Another use would be the classification of medical exams.

Personal data in shopping, location, actions, interest, and a lot of other indicators can be combined for analysis with this methodology to offer very good insight and trends. Examples include market research, web analytics, market strategies, and others. Other applications that are based on clustering algorithms include robotics, recommender systems, climatology, statistical and mathematical analysis. It offers a broad spectrum of utilization.

4. What is K-means Clustering?

K-means clustering is a type of supervised learning and is used when unlabeled data is present, i.e. data does not have defined groups or categories. The goal of the algorithm is used to find groups in the data with the number of groups that represent the variable K. The algorithm performs iteratively and assigns each of the data points to the suitable K group based on the provided features. Feature similarity is the basis on which the data points are clustered. The results include:

• Centroids of the K clusters that might be put to use for labeling new data
• Labels of the training data where each data point is assigned to a single cluster

Instead of defining the groups before you look at the data, you would be allowed to find and analyze the groups that have formed organically. The number of groups can also be determined. Each centroid of the cluster is a collection of features which would define the resulting groups. The examining centroid feature weights can be used to qualitatively interpret what kind of group each cluster would be representing.

The K-means clustering algorithm would be employed when the aim is to find groups which have not been labelled in the data explicitly. It can be used for the confirmation of business assumptions about the types of groups that exist and to identify the unknown groups in the complex data sets. New data can be easily assigned to the correct group once the algorithm has been run and the groups have been identified.

It is guaranteed that this algorithm would converge to a result. However, the result might be a local optimum and not necessarily the best possible outcome. More than a single run of the algorithm with randomized starting centroids might be a better outcome.  The algorithm would find the clusters and the data set labels for a particular K, the value of which must have been chosen earlier. 

The user would need to run the K-means clustering algorithm for a range of K values to find the number of clusters in the data and then the results are to be compared. There is no exact method for determining the exact value of K but an accurate estimate can be obtained.

 

The mean distance between the data points and the cluster centroid is used for comparing the results across different values of K. As the number of clusters would always be reducing the distance to data points, increasing the value of K would be decreasing the metric to 0, which happens when K equals the number of data points. The mean distance to the centroid is plotted as a function of K and the 'elbow point' where there is a sharp shift in the rate of decrease is used for roughly determining K.

There are some other techniques as well for validating K like information criteria, cross-validation, the information theoretic jump method, the G-means algorithm, and the silhouette method.

5. What is Hierarchical Clustering?

Hierarchical cluster analysis, or simple hierarchical clustering, is an algorithm that can be used to combine similar objects into groups known as clusters. The endpoint obtained is a set of clusters and each cluster would be distinct from the other. The objects within each cluster would be very similar to each other. It is possible to perform hierarchical clustering with raw data as well as a distance matrix. The distance matrix can be calculated from the raw data itself.

Hierarchical clustering starts with the treatment of each observation as a separate cluster. Following that, two clusters that are the closest are identified, and then the two most similar clusters are merged. This process is continued until all the clusters are merged together. The output that is obtained from the hierarchical clustering is a dendrogram, showing the relationship between the clusters.

The distance between the two clusters is generally computed by drawing a straight line between the clusters, i.e., the Euclidean distance. Many other distance metrics have been developed along the way. The choice of the distance metric must be made on theoretical grounds that are based on the domain of study. If there exists no theoretical justification for an alternative, the Euclidean distance must be generally preferred.

Hierarchical clustering might have single linkage where the distance between the clusters is defined as the shortest distance between two data points in the cluster. In complete linkage type, however, the longest distance between any two points in the clusters are considered. The average values can be considered as well.

After the distance metric has been chosen, it would be necessary to determine from where the distance is to be computed. The two most similar parts or the two least similar bits can be chosen. The centre of the clusters or some other criterion might be chosen as well. Other linkage criteria are also available. The choice of linkage criteria should also be made on the basis of theoretical considerations that are based on the domain of application. 

In the top-down or divisive method, all the observations are assigned to a single cluster. Then the cluster is partitioned in the two least similar clusters. This procedure is performed recursively until just one cluster remains. It has been evident that the divisive algorithms produce more accurate hierarchies than what is obtained through the agglomerative algorithms. However, it might prove to be more complex.
 
Hierarchical clustering might have single linkage where the distance between the clusters is defined as the shortest distance between two data points in the cluster. In complete linkage type, however, the longest distance between any two points in the clusters is considered. The average values can be considered as well.

Difference between K-means and Hierarchical Clustering

There are a few differences between these two type of clustering. The applications depend on their individual characteristics. While in certain scenarios, K-means clustering can be preferred, and, while in others, it is hierarchical clustering.

	Hierarchical Clustering	K-means Clustering
1	It is not possible to handle big data with Hierarchical Clustering. It has a quadratic time complexity.	K-means clustering has a linear time complexity and can handle big data.
2	Results are reproducible in hierarchical clustering.	The choice of clusters is random and the results would differ when run multiple times.
3	Hierarchical clustering works uniformly with all data.	K-means clustering work well if the shape of the cluster is hyperspherical - circular in 2D and spherical in 3D
4	It is possible to use any number of clusters by properly interpreting the dendrogram.	It is necessary to have prior knowledge of K in K-means clustering.

 

Conclusion

Thus, the two types of clustering can be differentiated on the basis of where they should be applied. The application of multivariate analysis is dependent depends on the type of data available and the requirements of the sample analysis.