Running quicker for Numpy and Pandas( installed via conda) than via pip?
up vote
3
down vote
favorite
I have two Python environments (3.6).
I installed packages in one environment via conda, and in another environment via pip.
Then I found that conda-provided numpy and pandas run quicker than pip version (for most of the scenario).
Just wonder why this happens. Is that because they use different configuration during compilation?
python pandas numpy anaconda
add a comment |
up vote
3
down vote
favorite
I have two Python environments (3.6).
I installed packages in one environment via conda, and in another environment via pip.
Then I found that conda-provided numpy and pandas run quicker than pip version (for most of the scenario).
Just wonder why this happens. Is that because they use different configuration during compilation?
python pandas numpy anaconda
1
Most likely it is due to a different BLAS linked to numpy - e.g., see markus-beuckelmann.de/blog/boosting-numpy-blas.html
– chrisb
Nov 7 at 21:21
add a comment |
up vote
3
down vote
favorite
up vote
3
down vote
favorite
I have two Python environments (3.6).
I installed packages in one environment via conda, and in another environment via pip.
Then I found that conda-provided numpy and pandas run quicker than pip version (for most of the scenario).
Just wonder why this happens. Is that because they use different configuration during compilation?
python pandas numpy anaconda
I have two Python environments (3.6).
I installed packages in one environment via conda, and in another environment via pip.
Then I found that conda-provided numpy and pandas run quicker than pip version (for most of the scenario).
Just wonder why this happens. Is that because they use different configuration during compilation?
python pandas numpy anaconda
python pandas numpy anaconda
asked Nov 7 at 20:29
MTANG
1368
1368
1
Most likely it is due to a different BLAS linked to numpy - e.g., see markus-beuckelmann.de/blog/boosting-numpy-blas.html
– chrisb
Nov 7 at 21:21
add a comment |
1
Most likely it is due to a different BLAS linked to numpy - e.g., see markus-beuckelmann.de/blog/boosting-numpy-blas.html
– chrisb
Nov 7 at 21:21
1
1
Most likely it is due to a different BLAS linked to numpy - e.g., see markus-beuckelmann.de/blog/boosting-numpy-blas.html
– chrisb
Nov 7 at 21:21
Most likely it is due to a different BLAS linked to numpy - e.g., see markus-beuckelmann.de/blog/boosting-numpy-blas.html
– chrisb
Nov 7 at 21:21
add a comment |
1 Answer
1
active
oldest
votes
up vote
2
down vote
Yes, most likely this difference is due to different compilation configurations. Numpy makes use of the low-level fortran libraries BLAS and LAPACK. The numpy that you get from pip is compiled with different implementations of BLAS and LAPACK than the numpy that you get from conda. The latter uses the MKL implementation which is optimized specifically for Intel processors.
You can check the implementation that numpy is using in a particular environment with:
import numpy as np
np.show_config()
Some comparison benchmarks can be found here (as already noted by crisb).
Thanks. I think it is exactly the case. I'm doing some heavy linear algebra calculation via numpy.
– MTANG
Nov 7 at 22:35
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
Yes, most likely this difference is due to different compilation configurations. Numpy makes use of the low-level fortran libraries BLAS and LAPACK. The numpy that you get from pip is compiled with different implementations of BLAS and LAPACK than the numpy that you get from conda. The latter uses the MKL implementation which is optimized specifically for Intel processors.
You can check the implementation that numpy is using in a particular environment with:
import numpy as np
np.show_config()
Some comparison benchmarks can be found here (as already noted by crisb).
Thanks. I think it is exactly the case. I'm doing some heavy linear algebra calculation via numpy.
– MTANG
Nov 7 at 22:35
add a comment |
up vote
2
down vote
Yes, most likely this difference is due to different compilation configurations. Numpy makes use of the low-level fortran libraries BLAS and LAPACK. The numpy that you get from pip is compiled with different implementations of BLAS and LAPACK than the numpy that you get from conda. The latter uses the MKL implementation which is optimized specifically for Intel processors.
You can check the implementation that numpy is using in a particular environment with:
import numpy as np
np.show_config()
Some comparison benchmarks can be found here (as already noted by crisb).
Thanks. I think it is exactly the case. I'm doing some heavy linear algebra calculation via numpy.
– MTANG
Nov 7 at 22:35
add a comment |
up vote
2
down vote
up vote
2
down vote
Yes, most likely this difference is due to different compilation configurations. Numpy makes use of the low-level fortran libraries BLAS and LAPACK. The numpy that you get from pip is compiled with different implementations of BLAS and LAPACK than the numpy that you get from conda. The latter uses the MKL implementation which is optimized specifically for Intel processors.
You can check the implementation that numpy is using in a particular environment with:
import numpy as np
np.show_config()
Some comparison benchmarks can be found here (as already noted by crisb).
Yes, most likely this difference is due to different compilation configurations. Numpy makes use of the low-level fortran libraries BLAS and LAPACK. The numpy that you get from pip is compiled with different implementations of BLAS and LAPACK than the numpy that you get from conda. The latter uses the MKL implementation which is optimized specifically for Intel processors.
You can check the implementation that numpy is using in a particular environment with:
import numpy as np
np.show_config()
Some comparison benchmarks can be found here (as already noted by crisb).
edited Nov 7 at 21:47
answered Nov 7 at 21:39
Xukrao
1,6181622
1,6181622
Thanks. I think it is exactly the case. I'm doing some heavy linear algebra calculation via numpy.
– MTANG
Nov 7 at 22:35
add a comment |
Thanks. I think it is exactly the case. I'm doing some heavy linear algebra calculation via numpy.
– MTANG
Nov 7 at 22:35
Thanks. I think it is exactly the case. I'm doing some heavy linear algebra calculation via numpy.
– MTANG
Nov 7 at 22:35
Thanks. I think it is exactly the case. I'm doing some heavy linear algebra calculation via numpy.
– MTANG
Nov 7 at 22:35
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53197337%2frunning-quicker-for-numpy-and-pandas-installed-via-conda-than-via-pip%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Most likely it is due to a different BLAS linked to numpy - e.g., see markus-beuckelmann.de/blog/boosting-numpy-blas.html
– chrisb
Nov 7 at 21:21