Optimizing cluster analysis
For complex dynamical systems consisting of many interacting subsystems it is a general challenge to work out methods to reduce the high dimensionality of the system to a few dominating variables that are able to characterize the system. Cluster analysis is a commonly accepted method to group elements in those which are more similar than others. Although the big achievements of the cluster analysis in applications there is the inaccuracy as different algorithms may lead to different clusters.
Cluster analysis does not aim for grasping dynamical effects. Combining it with stochastic methods shows that an improved dynamical cluster classification can be derived.
Our work is based on the analysis of financial stocks for which market states could be identified using clustering approaches. The stochastic data analysis we use here enables to separate between the stochastic and the deterministic part of the dynamics. The deterministic part gives access to the stability of the clusters and thus allows a new interpretation of the latter. Thus an optimizing of the cluster analysis can be derived, as well as emerging and disappearing cluster can be identified.