"Learning by kernels has interested researchers for many years and various types of kernel learning algorithms have been developed under different kinds of numerical optimization methods. Because of the heterogeneity of the real world data, combination of different kernels has been studied for binary class problems over the last decade. However, in reality, not every case is binary. Indeed, there are multiclass classification problems in engineering and applied sciences such as biomedical imaging and facial expressions. For such problems hierarchical classification methods have been proposed to predict multiclass problems. As against hierarchical methods, Error Correcting Output Code (ECOC) has been developed to avoid solving multiclass problems directly by breaking the problems into dichotomies instead. Each dichotomy consists of a binary output code from a matrix, the so called ECOC matrix, where each column of ECOC matrix defines the binary classification problem. SVM, one of the most powerful methods in ML, will be employed as ECOC binary classifiers. The decision on the class of test point is evaluated with respect to a combination of binary classifiers, which is often called ensemble classifier. This decision on the test point is affected by each binary classifier error, and hence the diversity of the binary classifiers has an impact on overall accuracy. Different methodologies have been proposed for the combination of classifiers, e.g., weighted combination, where the weights of ensembles can be found heuristically or via optimization modelling. The scientific objective of this proposal can be summarized as follows: 1) Develop novel and effective ensemble classifier systems by optimizing diversity-accuracy trade off 2) Improve the time complexity of model selection, 3) Generalize the overall model via multiple kernel learning for heterogeneous data from real world scenarios and experiment on classifying facial expressions."