,
Wenxuan Li [ctb],
Nan Xiao [ctb, cph],
Zachary Deane-Mayer [ctb]| Title: | Local Fisher Discriminant Analysis |
|---|---|
| Description: | Functions for performing and visualizing Local Fisher Discriminant Analysis(LFDA), Kernel Fisher Discriminant Analysis(KLFDA), and Semi-supervised Local Fisher Discriminant Analysis(SELF). |
| Authors: | Yuan Tang [aut, cre] ,
Wenxuan Li [ctb],
Nan Xiao [ctb, cph],
Zachary Deane-Mayer [ctb] |
| Maintainer: | Yuan Tang <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.1.3 |
| Built: | 2024-11-08 04:12:40 UTC |
| Source: | https://github.com/terrytangyuan/lfda |
This function defines operation for negative one half matrix power operator
x %^% nx %^% n
x |
the matrix we want to operate on |
n |
the exponent |
the matrix after negative one half power
This function assigns a color to each distinct value in the given vector.
Cols(vec)Cols(vec)
vec |
The vector where each distinct value will be assigned a color. |
The colors for each element in the given vector
This function returns an affinity matrix within knn-nearest neighbors from the distance matrix.
getAffinityMatrix(distance2, knn, nc)getAffinityMatrix(distance2, knn, nc)
distance2 |
The distance matrix for each observation |
knn |
The number of nearest neighbors |
nc |
The number of observations for data in this class |
an affinity matrix - the larger the element in the matrix, the closer two data points are
This function returns the requested type of transforming metric.
getMetricOfType(metric, eigVec, eigVal, total)getMetricOfType(metric, eigVec, eigVal, total)
metric |
The type of metric to be requested |
eigVec |
The eigenvectors of the problem |
eigVal |
The eigenvalues of the problem |
total |
The number of total rows to be used for weighting denominator |
The transformation metric in requested type
Performs kernel local fisher discriminant analysis on the given data,
which is the non-linear version of LFDA (see details lfda).
klfda(k, y, r, metric = c("weighted", "orthonormalized", "plain"), knn = 6, reg = 0.001)klfda(k, y, r, metric = c("weighted", "orthonormalized", "plain"), knn = 6, reg = 0.001)
k |
n x n kernel matrix. Result of the |
y |
n dimensional vector of class labels |
r |
dimensionality of reduced space (default: d) |
metric |
type of metric in the embedding space (default: 'weighted') 'weighted' — weighted eigenvectors 'orthonormalized' — orthonormalized 'plain' — raw eigenvectors |
knn |
parameter used in local scaling method (default: 6) |
reg |
regularization parameter (default: 0.001) |
list of the LFDA results:
T |
d x r transformation matrix (Z = t(T) * X) |
Z |
r x n matrix of dimensionality reduced samples |
Yuan Tang
Sugiyama, M (2007). - contain implementation Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research, vol.8, 1027–1061.
Sugiyama, M (2006). Local Fisher discriminant analysis for supervised dimensionality reduction. In W. W. Cohen and A. Moore (Eds.), Proceedings of 23rd International Conference on Machine Learning (ICML2006), 905–912.
Original Matlab Implementation: http://www.ms.k.u-tokyo.ac.jp/software.html#LFDA
See lfda for the linear version.
k <- kmatrixGauss(iris[, -5]) y <- iris[, 5] r <- 3 klfda(k, y, r, metric = "plain")k <- kmatrixGauss(iris[, -5]) y <- iris[, 5] r <- 3 klfda(k, y, r, metric = "plain")
Gaussian kernel computation for klfda, which maps the original data space to non-linear and higher dimensions.
kmatrixGauss(x, sigma = 1)kmatrixGauss(x, sigma = 1)
x |
n x d matrix of original samples. n is the number of samples. |
sigma |
dimensionality of reduced space. (default: 1) |
K n x n kernel matrix. n is the number of samples.
Yuan Tang
Sugiyama, M (2007). Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research, vol.8, 1027–1061.
Sugiyama, M (2006). Local Fisher discriminant analysis for supervised dimensionality reduction. In W. W. Cohen and A. Moore (Eds.), Proceedings of 23rd International Conference on Machine Learning (ICML2006), 905–912.
https://shapeofdata.wordpress.com/2013/07/23/gaussian-kernels/
See klfda for the computation of
kernel local fisher discriminant analysis
kmatrixGauss(iris[, -5])kmatrixGauss(iris[, -5])
Performs local fisher discriminant analysis (LFDA) on the given data.
lfda(x, y, r, metric = c("orthonormalized", "plain", "weighted"), knn = 5)lfda(x, y, r, metric = c("orthonormalized", "plain", "weighted"), knn = 5)
x |
n x d matrix of original samples. n is the number of samples. |
y |
length n vector of class labels |
r |
dimensionality of reduced space (default: d) |
metric |
type of metric in the embedding space (no default) 'weighted' — weighted eigenvectors 'orthonormalized' — orthonormalized 'plain' — raw eigenvectors |
knn |
parameter used in local scaling method (default: 5) |
LFDA is a method for linear dimensionality reduction that maximizes between-class scatter and minimizes within-class scatter while at the same time maintain the local structure of the data so that multimodal data can be embedded appropriately. Its limitation is that it only looks for linear boundaries between clusters. In this case, a non-linear version called kernel LFDA will be used instead. Three metric types can be used if needed.
list of the LFDA results:
T |
d x r transformation matrix (Z = x * T) |
Z |
n x r matrix of dimensionality reduced samples |
Yuan Tang
Sugiyama, M (2007). Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research, vol.8, 1027–1061.
Sugiyama, M (2006). Local Fisher discriminant analysis for supervised dimensionality reduction. In W. W. Cohen and A. Moore (Eds.), Proceedings of 23rd International Conference on Machine Learning (ICML2006), 905–912.
See klfda for the kernelized variant of
LFDA (Kernel LFDA).
k <- iris[, -5] y <- iris[, 5] r <- 3 lfda(k, y, r, metric = "plain")k <- iris[, -5] y <- iris[, 5] r <- 3 lfda(k, y, r, metric = "plain")
This function plot 3 dimensions of the lfda/klfda result.
## S3 method for class 'lfda' plot(x, labels, cleanText = FALSE, ...)## S3 method for class 'lfda' plot(x, labels, cleanText = FALSE, ...)
x |
The lfda/klfda result. |
labels |
A list of class labels used for lfda/klfda training. |
cleanText |
A boolean value to specify whether to make the labels in the plot cleaner (default: FALSE) |
... |
Additional arguments |
See lfda and klfda for the metric learning method used for this visualization.
This function transforms a data set, usually a testing set, using the trained LFDA metric
## S3 method for class 'lfda' predict(object, newdata = NULL, type = "raw", ...)## S3 method for class 'lfda' predict(object, newdata = NULL, type = "raw", ...)
object |
The result from lfda function, which contains a transformed data and a transforming matrix that can be used for transforming testing set |
newdata |
The data to be transformed |
type |
The output type, in this case it defaults to "raw" since the output is a matrix |
... |
Additional arguments |
the transformed matrix
Yuan Tang
k <- iris[, -5] y <- iris[, 5] r <- 3 model <- lfda(k, y, r = 4, metric = "plain") predict(model, iris[, -5])k <- iris[, -5] y <- iris[, 5] r <- 3 model <- lfda(k, y, r = 4, metric = "plain") predict(model, iris[, -5])
Print an lfda object
## S3 method for class 'lfda' print(x, ...)## S3 method for class 'lfda' print(x, ...)
x |
The result from lfda function, which contains a transformed data and a transforming |
... |
ignored |
This function mimics the behavior and syntax of repmat() in Matlab it generates a large matrix consisting of an N-by-M tiling copies of A
repmat(A, N, M)repmat(A, N, M)
A |
original matrix to be used as copies |
N |
the number of rows of tiling copies of A |
M |
the number of columns of tiling copies of A |
matrix consisting of an N-by-M tiling copies of A
Performs semi-supervised local fisher discriminant analysis (SELF) on the given data. SELF is a linear semi-supervised dimensionality reduction method smoothly bridges supervised LFDA and unsupervised principal component analysis, by which a natural regularization effect can be obtained when only a small number of labeled samples are available.
self(X, Y, beta = 0.5, r, metric = c("orthonormalized", "plain", "weighted"), kNN = 5, minObsPerLabel = 5)self(X, Y, beta = 0.5, r, metric = c("orthonormalized", "plain", "weighted"), kNN = 5, minObsPerLabel = 5)
X |
n x d matrix of original samples. n is the number of samples. |
Y |
length n vector of class labels |
beta |
degree of semi-supervisedness (0 <= beta <= 1; default is 0.5 ) 0: totally supervised (discard all unlabeled samples) 1: totally unsupervised (discard all label information) |
r |
dimensionality of reduced space (default: d) |
metric |
type of metric in the embedding space (no default) 'weighted' — weighted eigenvectors 'orthonormalized' — orthonormalized 'plain' — raw eigenvectors |
kNN |
parameter used in local scaling method (default: 5) |
minObsPerLabel |
the minimum number observations required for each different label(default: 5) |
list of the SELF results:
T |
d x r transformation matrix (Z = x * T) |
Z |
n x r matrix of dimensionality reduced samples |
Yuan Tang
Sugiyama, Masashi, et al (2010). Semi-supervised local Fisher discriminant analysis for dimensionality reduction. Machine learning 78.1-2: 35-61.
Sugiyama, M (2007). Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research, vol.8, 1027–1061.
Sugiyama, M (2006). Local Fisher discriminant analysis for supervised dimensionality reduction. In W. W. Cohen and A. Moore (Eds.), Proceedings of 23rd International Conference on Machine Learning (ICML2006), 905–912.
See lfda for LFDA and klfda for the kernelized variant of
LFDA (Kernel LFDA).
x <- iris[, -5] y <- iris[, 5] self(x, y, beta = 0.1, r = 3, metric = "plain")x <- iris[, -5] y <- iris[, 5] self(x, y, beta = 0.1, r = 3, metric = "plain")