Estimation of the number of clusters using multiple clustering validity indices

Krzysztof Kryszczuk; Paul Hurley

doi:10.1007/978-3-642-12127-2_12

Estimation of the number of clusters using multiple clustering validity indices

Krzysztof Kryszczuk, Paul Hurley

IBM

Research output: Chapter in Book / Conference Paper › Conference Paper › peer-review

62 Citations (Scopus)

Abstract

One of the challenges in unsupervised machine learning is finding the number of clusters in a dataset. Clustering Validity Indices (CVI) are popular tools used to address this problem. A large number of CVIs have been proposed, and reports that compare different CVIs suggest that no single CVI can always outperform others. Following suggestions found in prior art, in this paper we formalize the concept of using multiple CVIs for cluster number estimation in the framework of multi-classifier fusion. Using a large number of datasets, we show that decision-level fusion of multiple CVIs can lead to significant gains in accuracy in estimating the number of clusters, in particular for high-dimensional datasets with large number of clusters.

Original language	English
Title of host publication	Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings
Publisher	Springer Verlag
Pages	114-123
Number of pages	10
ISBN (Print)	3642121268, 9783642121265
DOIs	https://doi.org/10.1007/978-3-642-12127-2_12
Publication status	Published - 2010
Externally published	Yes
Event	9th International Workshop on Multiple Classifier Systems, MCS 2010 - Cairo, Egypt Duration: 7 Apr 2010 → 9 Apr 2010

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5997 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	9th International Workshop on Multiple Classifier Systems, MCS 2010
Country/Territory	Egypt
City	Cairo
Period	7/04/10 → 9/04/10

Keywords

Clustering
Clustering validity indices
Multiple classifier

Access to Document

10.1007/978-3-642-12127-2_12

Cite this

Kryszczuk, K., & Hurley, P. (2010). Estimation of the number of clusters using multiple clustering validity indices. In Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings (pp. 114-123). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5997 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-642-12127-2_12

Kryszczuk, Krzysztof ; Hurley, Paul. / Estimation of the number of clusters using multiple clustering validity indices. Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings. Springer Verlag, 2010. pp. 114-123 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{27b56a7a9fbd4f44838359e25c65f315,

title = "Estimation of the number of clusters using multiple clustering validity indices",

abstract = "One of the challenges in unsupervised machine learning is finding the number of clusters in a dataset. Clustering Validity Indices (CVI) are popular tools used to address this problem. A large number of CVIs have been proposed, and reports that compare different CVIs suggest that no single CVI can always outperform others. Following suggestions found in prior art, in this paper we formalize the concept of using multiple CVIs for cluster number estimation in the framework of multi-classifier fusion. Using a large number of datasets, we show that decision-level fusion of multiple CVIs can lead to significant gains in accuracy in estimating the number of clusters, in particular for high-dimensional datasets with large number of clusters.",

keywords = "Clustering, Clustering validity indices, Multiple classifier",

author = "Krzysztof Kryszczuk and Paul Hurley",

year = "2010",

doi = "10.1007/978-3-642-12127-2_12",

language = "English",

isbn = "3642121268",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "114--123",

booktitle = "Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings",

note = "9th International Workshop on Multiple Classifier Systems, MCS 2010 ; Conference date: 07-04-2010 Through 09-04-2010",

}

Kryszczuk, K & Hurley, P 2010, Estimation of the number of clusters using multiple clustering validity indices. in Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5997 LNCS, Springer Verlag, pp. 114-123, 9th International Workshop on Multiple Classifier Systems, MCS 2010, Cairo, Egypt, 7/04/10. https://doi.org/10.1007/978-3-642-12127-2_12

Estimation of the number of clusters using multiple clustering validity indices. / Kryszczuk, Krzysztof; Hurley, Paul.
Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings. Springer Verlag, 2010. p. 114-123 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5997 LNCS).

Research output: Chapter in Book / Conference Paper › Conference Paper › peer-review

TY - GEN

T1 - Estimation of the number of clusters using multiple clustering validity indices

AU - Kryszczuk, Krzysztof

AU - Hurley, Paul

PY - 2010

Y1 - 2010

N2 - One of the challenges in unsupervised machine learning is finding the number of clusters in a dataset. Clustering Validity Indices (CVI) are popular tools used to address this problem. A large number of CVIs have been proposed, and reports that compare different CVIs suggest that no single CVI can always outperform others. Following suggestions found in prior art, in this paper we formalize the concept of using multiple CVIs for cluster number estimation in the framework of multi-classifier fusion. Using a large number of datasets, we show that decision-level fusion of multiple CVIs can lead to significant gains in accuracy in estimating the number of clusters, in particular for high-dimensional datasets with large number of clusters.

AB - One of the challenges in unsupervised machine learning is finding the number of clusters in a dataset. Clustering Validity Indices (CVI) are popular tools used to address this problem. A large number of CVIs have been proposed, and reports that compare different CVIs suggest that no single CVI can always outperform others. Following suggestions found in prior art, in this paper we formalize the concept of using multiple CVIs for cluster number estimation in the framework of multi-classifier fusion. Using a large number of datasets, we show that decision-level fusion of multiple CVIs can lead to significant gains in accuracy in estimating the number of clusters, in particular for high-dimensional datasets with large number of clusters.

KW - Clustering

KW - Clustering validity indices

KW - Multiple classifier

UR - http://www.scopus.com/inward/record.url?scp=77952046019&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-12127-2_12

DO - 10.1007/978-3-642-12127-2_12

M3 - Conference Paper

AN - SCOPUS:77952046019

SN - 3642121268

SN - 9783642121265

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 114

EP - 123

BT - Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings

PB - Springer Verlag

T2 - 9th International Workshop on Multiple Classifier Systems, MCS 2010

Y2 - 7 April 2010 through 9 April 2010

ER -

Kryszczuk K, Hurley P. Estimation of the number of clusters using multiple clustering validity indices. In Multiple Classifier Systems - 9th International Workshop, MCS 2010, Proceedings. Springer Verlag. 2010. p. 114-123. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-12127-2_12

Estimation of the number of clusters using multiple clustering validity indices

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this