sklearn error ValueError: Input contains NaN, infinity or a value too large for dtype('float32')
up vote
0
down vote
favorite
I am not familiar with python and am trying to run a decision tree classifier in python using SKLEARN library and when I run the code, I encounters the error:
ValueError: Input contains NaN, infinity or a value too large for dtype('float32')
I have tried using a smaller subset of my excel datasheet and the code is able to execute with the results I want. So I suspect the problem is that my data set is too big. Here is my code that causes the crash:
df_X = data_train[['DayOfWeek', 'Promo', 'StateHoliday']]
df_Y = data_train[['Sales_band']]
X_train, X_test, y_train, y_test = train_test_split(df_X, df_Y, random_state=1)
model = tree.DecisionTreeClassifier()
model.fit(X_train, y_train) // Line that causes crash
y_predict = model.predict(X_test)
print('The accuracy of the Decision Tree is', accuracy_score(y_test, y_predict))
python pandas numpy scikit-learn sklearn-pandas
asked Nov 7 at 10:22
Jia Hao Lim
1
add a comment |
up vote
0
down vote
favorite
I am not familiar with python and am trying to run a decision tree classifier in python using SKLEARN library and when I run the code, I encounters the error:
ValueError: Input contains NaN, infinity or a value too large for dtype('float32')
I have tried using a smaller subset of my excel datasheet and the code is able to execute with the results I want. So I suspect the problem is that my data set is too big. Here is my code that causes the crash:
df_X = data_train[['DayOfWeek', 'Promo', 'StateHoliday']]
df_Y = data_train[['Sales_band']]
X_train, X_test, y_train, y_test = train_test_split(df_X, df_Y, random_state=1)
model = tree.DecisionTreeClassifier()
model.fit(X_train, y_train) // Line that causes crash
y_predict = model.predict(X_test)
print('The accuracy of the Decision Tree is', accuracy_score(y_test, y_predict))
python pandas numpy scikit-learn sklearn-pandas
asked Nov 7 at 10:22
Jia Hao Lim
1
2
The error message seems to suggest that your dataset is not too big; rather that one of the values of your dataset is either: Not a number, infity or a number too large to fit into a floating point number of type float32. I would suggest checking your data for missing values/nan's as a first step.
– Pallie
Nov 7 at 10:44
Oh, you are right. Thank you
– Jia Hao Lim
Nov 7 at 10:48
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I am not familiar with python and am trying to run a decision tree classifier in python using SKLEARN library and when I run the code, I encounters the error:
ValueError: Input contains NaN, infinity or a value too large for dtype('float32')
I have tried using a smaller subset of my excel datasheet and the code is able to execute with the results I want. So I suspect the problem is that my data set is too big. Here is my code that causes the crash:
df_X = data_train[['DayOfWeek', 'Promo', 'StateHoliday']]
df_Y = data_train[['Sales_band']]
X_train, X_test, y_train, y_test = train_test_split(df_X, df_Y, random_state=1)
model = tree.DecisionTreeClassifier()
model.fit(X_train, y_train) // Line that causes crash
y_predict = model.predict(X_test)
print('The accuracy of the Decision Tree is', accuracy_score(y_test, y_predict))
python pandas numpy scikit-learn sklearn-pandas
asked Nov 7 at 10:22
Jia Hao Lim
1
I am not familiar with python and am trying to run a decision tree classifier in python using SKLEARN library and when I run the code, I encounters the error:
ValueError: Input contains NaN, infinity or a value too large for dtype('float32')
I have tried using a smaller subset of my excel datasheet and the code is able to execute with the results I want. So I suspect the problem is that my data set is too big. Here is my code that causes the crash:
df_X = data_train[['DayOfWeek', 'Promo', 'StateHoliday']]
df_Y = data_train[['Sales_band']]
X_train, X_test, y_train, y_test = train_test_split(df_X, df_Y, random_state=1)
model = tree.DecisionTreeClassifier()
model.fit(X_train, y_train) // Line that causes crash
y_predict = model.predict(X_test)
print('The accuracy of the Decision Tree is', accuracy_score(y_test, y_predict))
python pandas numpy scikit-learn sklearn-pandas
python pandas numpy scikit-learn sklearn-pandas
asked Nov 7 at 10:22
Jia Hao Lim
1
asked Nov 7 at 10:22
Jia Hao Lim
1
edited Nov 7 at 10:38
asked Nov 7 at 10:22
Jia Hao Lim
1
asked Nov 7 at 10:22
Jia Hao Lim
1
asked Nov 7 at 10:22
Jia Hao Lim
1
1
2
The error message seems to suggest that your dataset is not too big; rather that one of the values of your dataset is either: Not a number, infity or a number too large to fit into a floating point number of type float32. I would suggest checking your data for missing values/nan's as a first step.
– Pallie
Nov 7 at 10:44
Oh, you are right. Thank you
– Jia Hao Lim
Nov 7 at 10:48
add a comment |
2
The error message seems to suggest that your dataset is not too big; rather that one of the values of your dataset is either: Not a number, infity or a number too large to fit into a floating point number of type float32. I would suggest checking your data for missing values/nan's as a first step.
– Pallie
Nov 7 at 10:44
Oh, you are right. Thank you
– Jia Hao Lim
Nov 7 at 10:48
2
2
The error message seems to suggest that your dataset is not too big; rather that one of the values of your dataset is either: Not a number, infity or a number too large to fit into a floating point number of type float32. I would suggest checking your data for missing values/nan's as a first step.
– Pallie
Nov 7 at 10:44
The error message seems to suggest that your dataset is not too big; rather that one of the values of your dataset is either: Not a number, infity or a number too large to fit into a floating point number of type float32. I would suggest checking your data for missing values/nan's as a first step.
– Pallie
Nov 7 at 10:44
Oh, you are right. Thank you
– Jia Hao Lim
Nov 7 at 10:48
Oh, you are right. Thank you
– Jia Hao Lim
Nov 7 at 10:48
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
You may have missing values in your dataset. You may want to use dropna() to remove all rows containing missing values if it won't affect the quality of your prediction/accuracy of prediction
answered Nov 10 at 21:51
isaac-moore
194
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
You may have missing values in your dataset. You may want to use dropna() to remove all rows containing missing values if it won't affect the quality of your prediction/accuracy of prediction
answered Nov 10 at 21:51
isaac-moore
194
add a comment |
up vote
0
down vote
You may have missing values in your dataset. You may want to use dropna() to remove all rows containing missing values if it won't affect the quality of your prediction/accuracy of prediction
answered Nov 10 at 21:51
isaac-moore
194
add a comment |
up vote
0
down vote
up vote
0
down vote
You may have missing values in your dataset. You may want to use dropna() to remove all rows containing missing values if it won't affect the quality of your prediction/accuracy of prediction
answered Nov 10 at 21:51
isaac-moore
194
You may have missing values in your dataset. You may want to use dropna() to remove all rows containing missing values if it won't affect the quality of your prediction/accuracy of prediction
answered Nov 10 at 21:51
isaac-moore
194
answered Nov 10 at 21:51
isaac-moore
194
answered Nov 10 at 21:51
isaac-moore
194
answered Nov 10 at 21:51
isaac-moore
194
194
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53187515%2fsklearn-error-valueerror-input-contains-nan-infinity-or-a-value-too-large-for%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
The error message seems to suggest that your dataset is not too big; rather that one of the values of your dataset is either: Not a number, infity or a number too large to fit into a floating point number of type float32. I would suggest checking your data for missing values/nan's as a first step.
– Pallie
Nov 7 at 10:44
Oh, you are right. Thank you
– Jia Hao Lim
Nov 7 at 10:48