Mutual Information Estimation in Higher Dimensions: A Speed-Up of a k-Nearest Neighbor Based Estimator.

Mutual Information Estimation in Higher Dimensions: A Speed-Up of a k-Nearest Neighbor Based Estimator.

Abstract

We focus on the recently introduced nearest neighbor based entropy estimator from Kraskov, Stögbauer and Grassberger (KSG), the nearest neighbor search of which is performed by the so called box assisted algorithm. We compare the performance of KSG with respect to three spatial indexing methods: box-assisted, k-D trie and projection method, on a problem of mutual information estimation of a variety of pdfs and dimensionalities. We conclude that the k-D trie method is significantly faster then box-assisted search in fixed-mass and fixed-radius neighborhood searches in higher dimensions. The projection method is much slower than both alternatives and not recommended for practical use.

Grafik Top
Authors
  • Vejmelka, Martin
  • Schindlerova, Katerina
Grafik Top
Shortfacts
Category
Paper in Conference Proceedings or in Workshop Proceedings (Full Paper in Proceedings)
Event Title
Mutual Information Estimation in Higher Dimensions: A Speed-Up of a k-Nearest Neighbor Based Estimator.
Divisions
Data Mining
Event Type
Conference
Event Dates
April 2007
Series Name
Adaptive and Natural Computing Algorithms
Page Range
pp. 790-797
Date
April 2007
Export
Grafik Top