Covariance function - Out of memory
up vote
0
down vote
favorite
I am trying to perform PCA on the MNIST data set. I have the following code so far.
...load data into MATLAB
% Centre data matrix
imagesMean = mean(images);
imagesShifted = images - imagesMean;
% Compute covariance matrix of mean shifted images
covariance = cov(imagesShifted);
Trying to do this gives me the following response:
Out of memory. Type "help memory" for your options.
Error in cov (line 155) c = (xc' * xc) ./ denom;
Error in PCA (line 27) covariance = cov(imagesShifted);
imagesShifted is a 784x60000 double matrix.
I am using a MacBook Pro 2015 with 16GB RAM and a 2.8 GHz processor and a dedicated graphics card.
I looked under the help menu for the memory command but the information only seems relevant to Windows machines. Also looked at the MathWorks website for resolving out of memory issues but wasn't sure how to proceed based on that information.
How can I get around this issue?
matlab statistics pca covariance
asked Nov 4 at 9:47
Sithling
305110
add a comment |
up vote
0
down vote
favorite
I am trying to perform PCA on the MNIST data set. I have the following code so far.
...load data into MATLAB
% Centre data matrix
imagesMean = mean(images);
imagesShifted = images - imagesMean;
% Compute covariance matrix of mean shifted images
covariance = cov(imagesShifted);
Trying to do this gives me the following response:
Out of memory. Type "help memory" for your options.
Error in cov (line 155) c = (xc' * xc) ./ denom;
Error in PCA (line 27) covariance = cov(imagesShifted);
imagesShifted is a 784x60000 double matrix.
I am using a MacBook Pro 2015 with 16GB RAM and a 2.8 GHz processor and a dedicated graphics card.
I looked under the help menu for the memory command but the information only seems relevant to Windows machines. Also looked at the MathWorks website for resolving out of memory issues but wasn't sure how to proceed based on that information.
How can I get around this issue?
matlab statistics pca covariance
asked Nov 4 at 9:47
Sithling
305110
1
Do you need a 60,000x60,000 matrix out or a 784x784 matrix out? If so you'll need to transpose your matrix before you put into the covariance function. Also, MATLAB has a dedicated pca function mathworks.com/help/stats/pca.html
– Durkee
Nov 4 at 13:41
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am trying to perform PCA on the MNIST data set. I have the following code so far.
...load data into MATLAB
% Centre data matrix
imagesMean = mean(images);
imagesShifted = images - imagesMean;
% Compute covariance matrix of mean shifted images
covariance = cov(imagesShifted);
Trying to do this gives me the following response:
Out of memory. Type "help memory" for your options.
Error in cov (line 155) c = (xc' * xc) ./ denom;
Error in PCA (line 27) covariance = cov(imagesShifted);
imagesShifted is a 784x60000 double matrix.
I am using a MacBook Pro 2015 with 16GB RAM and a 2.8 GHz processor and a dedicated graphics card.
I looked under the help menu for the memory command but the information only seems relevant to Windows machines. Also looked at the MathWorks website for resolving out of memory issues but wasn't sure how to proceed based on that information.
How can I get around this issue?
matlab statistics pca covariance
asked Nov 4 at 9:47
Sithling
305110
I am trying to perform PCA on the MNIST data set. I have the following code so far.
...load data into MATLAB
% Centre data matrix
imagesMean = mean(images);
imagesShifted = images - imagesMean;
% Compute covariance matrix of mean shifted images
covariance = cov(imagesShifted);
Trying to do this gives me the following response:
Out of memory. Type "help memory" for your options.
Error in cov (line 155) c = (xc' * xc) ./ denom;
Error in PCA (line 27) covariance = cov(imagesShifted);
imagesShifted is a 784x60000 double matrix.
I am using a MacBook Pro 2015 with 16GB RAM and a 2.8 GHz processor and a dedicated graphics card.
I looked under the help menu for the memory command but the information only seems relevant to Windows machines. Also looked at the MathWorks website for resolving out of memory issues but wasn't sure how to proceed based on that information.
How can I get around this issue?
matlab statistics pca covariance
matlab statistics pca covariance
asked Nov 4 at 9:47
Sithling
305110
asked Nov 4 at 9:47
Sithling
305110
asked Nov 4 at 9:47
Sithling
305110
asked Nov 4 at 9:47
Sithling
305110
asked Nov 4 at 9:47
Sithling
305110
305110
1
Do you need a 60,000x60,000 matrix out or a 784x784 matrix out? If so you'll need to transpose your matrix before you put into the covariance function. Also, MATLAB has a dedicated pca function mathworks.com/help/stats/pca.html
– Durkee
Nov 4 at 13:41
add a comment |
1
Do you need a 60,000x60,000 matrix out or a 784x784 matrix out? If so you'll need to transpose your matrix before you put into the covariance function. Also, MATLAB has a dedicated pca function mathworks.com/help/stats/pca.html
– Durkee
Nov 4 at 13:41
1
1
Do you need a 60,000x60,000 matrix out or a 784x784 matrix out? If so you'll need to transpose your matrix before you put into the covariance function. Also, MATLAB has a dedicated pca function mathworks.com/help/stats/pca.html
– Durkee
Nov 4 at 13:41
Do you need a 60,000x60,000 matrix out or a 784x784 matrix out? If so you'll need to transpose your matrix before you put into the covariance function. Also, MATLAB has a dedicated pca function mathworks.com/help/stats/pca.html
– Durkee
Nov 4 at 13:41
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
For large data sets, I suggest you to use the princomp function of matlab, with the flag 'econ' activated.
https://es.mathworks.com/help/stats/princomp.html
Or the pca function with the flag 'economy' or indicating the 'NumComponents' you wish.
https://es.mathworks.com/help/stats/pca.html
answered Nov 4 at 18:21
Sergio
613
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
princomp has been removed
– Durkee
2 days ago
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
For large data sets, I suggest you to use the princomp function of matlab, with the flag 'econ' activated.
https://es.mathworks.com/help/stats/princomp.html
Or the pca function with the flag 'economy' or indicating the 'NumComponents' you wish.
https://es.mathworks.com/help/stats/pca.html
answered Nov 4 at 18:21
Sergio
613
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
princomp has been removed
– Durkee
2 days ago
add a comment |
up vote
1
down vote
For large data sets, I suggest you to use the princomp function of matlab, with the flag 'econ' activated.
https://es.mathworks.com/help/stats/princomp.html
Or the pca function with the flag 'economy' or indicating the 'NumComponents' you wish.
https://es.mathworks.com/help/stats/pca.html
answered Nov 4 at 18:21
Sergio
613
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
princomp has been removed
– Durkee
2 days ago
add a comment |
up vote
1
down vote
up vote
1
down vote
For large data sets, I suggest you to use the princomp function of matlab, with the flag 'econ' activated.
https://es.mathworks.com/help/stats/princomp.html
Or the pca function with the flag 'economy' or indicating the 'NumComponents' you wish.
https://es.mathworks.com/help/stats/pca.html
answered Nov 4 at 18:21
Sergio
613
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
For large data sets, I suggest you to use the princomp function of matlab, with the flag 'econ' activated.
https://es.mathworks.com/help/stats/princomp.html
Or the pca function with the flag 'economy' or indicating the 'NumComponents' you wish.
https://es.mathworks.com/help/stats/pca.html
answered Nov 4 at 18:21
Sergio
613
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered Nov 4 at 18:21
Sergio
613
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered Nov 4 at 18:21
Sergio
613
answered Nov 4 at 18:21
Sergio
613
613
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Sergio is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
princomp has been removed
– Durkee
2 days ago
add a comment |
princomp has been removed
– Durkee
2 days ago
princomp has been removed
– Durkee
2 days ago
princomp has been removed
– Durkee
2 days ago
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53139501%2fcovariance-function-out-of-memory%23new-answer', 'question_page');
}
);
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
1
Do you need a 60,000x60,000 matrix out or a 784x784 matrix out? If so you'll need to transpose your matrix before you put into the covariance function. Also, MATLAB has a dedicated pca function mathworks.com/help/stats/pca.html
– Durkee
Nov 4 at 13:41