How to prevent Keras predict_generator from shuffling data?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
1
I created a deep learning model, and I want to check the performance of the model by using predict_generator. I am using the following code which compares the images' labels with the predicted classes and then returns the prediction error.
validation_generator = validation_datagen.flow_from_directory(
validation_dir,
target_size=(image_size, image_size),
batch_size=val_batchsize,
class_mode='categorical',
shuffle=False)
# Get the filenames from the generator
fnames = validation_generator.filenames
# Get the ground truth from generator
ground_truth = validation_generator.classes
# Get the label to class mapping from the generator
label2index = validation_generator.class_indices
# Getting the mapping from class index to class label
idx2label = dict((v,k) for k,v in label2index.items())
# Get the predictions from the model using the generator
predictions = model.predict_generator(validation_generator, steps=validation_generator.samples/validation_generator.batch_size,verbose=1)
predicted_classes = np.argmax(predictions,axis=1)
errors = np.where(predicted_classes != ground_truth)[0]
print("No of errors = {}/{}".format(len(errors),validation_generator.samples))
# Show the errors
for i in range(len(errors)):
pred_class = np.argmax(predictions[errors[i]])
pred_label = idx2label[pred_class]
title = 'Original label:{}, Prediction :{}, confidence : {:.3f}'.format(
fnames[errors[i]].split('/')[0],
pred_label,
predictions[errors[i]][pred_class])
original = load_img('{}/{}'.format(validation_dir,fnames[errors[i]]))
plt.figure(figsize=[7,7])
plt.axis('off')
plt.title(title)
plt.imshow(original)
plt.show()
validation_generator.classes is arranged but predicted_classes is not arranged.
I take the code from here https://www.learnopencv.com/keras-tutorial-fine-tuning-using-pre-trained-models/
How can I prevent predict_generator from shuffling data?
machine-learning keras neural-network deep-learning
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
asked Nov 24 '18 at 12:15
NoranNoran
239110
|
show 1 more comment
1
I created a deep learning model, and I want to check the performance of the model by using predict_generator. I am using the following code which compares the images' labels with the predicted classes and then returns the prediction error.
validation_generator = validation_datagen.flow_from_directory(
validation_dir,
target_size=(image_size, image_size),
batch_size=val_batchsize,
class_mode='categorical',
shuffle=False)
# Get the filenames from the generator
fnames = validation_generator.filenames
# Get the ground truth from generator
ground_truth = validation_generator.classes
# Get the label to class mapping from the generator
label2index = validation_generator.class_indices
# Getting the mapping from class index to class label
idx2label = dict((v,k) for k,v in label2index.items())
# Get the predictions from the model using the generator
predictions = model.predict_generator(validation_generator, steps=validation_generator.samples/validation_generator.batch_size,verbose=1)
predicted_classes = np.argmax(predictions,axis=1)
errors = np.where(predicted_classes != ground_truth)[0]
print("No of errors = {}/{}".format(len(errors),validation_generator.samples))
# Show the errors
for i in range(len(errors)):
pred_class = np.argmax(predictions[errors[i]])
pred_label = idx2label[pred_class]
title = 'Original label:{}, Prediction :{}, confidence : {:.3f}'.format(
fnames[errors[i]].split('/')[0],
pred_label,
predictions[errors[i]][pred_class])
original = load_img('{}/{}'.format(validation_dir,fnames[errors[i]]))
plt.figure(figsize=[7,7])
plt.axis('off')
plt.title(title)
plt.imshow(original)
plt.show()
validation_generator.classes is arranged but predicted_classes is not arranged.
I take the code from here https://www.learnopencv.com/keras-tutorial-fine-tuning-using-pre-trained-models/
How can I prevent predict_generator from shuffling data?
machine-learning keras neural-network deep-learning
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
asked Nov 24 '18 at 12:15
NoranNoran
239110
I don't get what you mean by X is arranged but Y is not arranged.
– Matias Valdenegro
Nov 24 '18 at 13:18
@MatiasValdenegro the order of the items in validation_generator.classes is same as the order of the data in dataset. But predict_generator does not do the predication in the same order of the dataset.
– Noran
Nov 24 '18 at 13:58
Please add detail of this, the classes attribute in the generator gives you the class indices, are you saying that the class indices do not match the output of the generator? If so, add examples of this. Also you should make sure that there are no new or unseen classes in the validation data (this would change class indices).
– Matias Valdenegro
Nov 24 '18 at 14:12
My code is same as the code in the link.. The data in validation_generator.classes is [0,0,0,0,....1,1,1,1,........2,2,2..] But predict_generator changes the order of the data and make the predication for the images as [1,1,1,...2,2,2,...,0,0,0] or [2,2,2... 0,0,0....1,1,1...2,2.. ]
– Noran
Nov 24 '18 at 14:20
I think the tutorial is not doing it right, it shouldn't be using generator.classes, but instead each time the generator is called it returns the image data and the corresponding label.
– Matias Valdenegro
Nov 24 '18 at 16:59
|
show 1 more comment
1
1
1
I created a deep learning model, and I want to check the performance of the model by using predict_generator. I am using the following code which compares the images' labels with the predicted classes and then returns the prediction error.
validation_generator = validation_datagen.flow_from_directory(
validation_dir,
target_size=(image_size, image_size),
batch_size=val_batchsize,
class_mode='categorical',
shuffle=False)
# Get the filenames from the generator
fnames = validation_generator.filenames
# Get the ground truth from generator
ground_truth = validation_generator.classes
# Get the label to class mapping from the generator
label2index = validation_generator.class_indices
# Getting the mapping from class index to class label
idx2label = dict((v,k) for k,v in label2index.items())
# Get the predictions from the model using the generator
predictions = model.predict_generator(validation_generator, steps=validation_generator.samples/validation_generator.batch_size,verbose=1)
predicted_classes = np.argmax(predictions,axis=1)
errors = np.where(predicted_classes != ground_truth)[0]
print("No of errors = {}/{}".format(len(errors),validation_generator.samples))
# Show the errors
for i in range(len(errors)):
pred_class = np.argmax(predictions[errors[i]])
pred_label = idx2label[pred_class]
title = 'Original label:{}, Prediction :{}, confidence : {:.3f}'.format(
fnames[errors[i]].split('/')[0],
pred_label,
predictions[errors[i]][pred_class])
original = load_img('{}/{}'.format(validation_dir,fnames[errors[i]]))
plt.figure(figsize=[7,7])
plt.axis('off')
plt.title(title)
plt.imshow(original)
plt.show()
validation_generator.classes is arranged but predicted_classes is not arranged.
I take the code from here https://www.learnopencv.com/keras-tutorial-fine-tuning-using-pre-trained-models/
How can I prevent predict_generator from shuffling data?
machine-learning keras neural-network deep-learning
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
asked Nov 24 '18 at 12:15
NoranNoran
239110
I created a deep learning model, and I want to check the performance of the model by using predict_generator. I am using the following code which compares the images' labels with the predicted classes and then returns the prediction error.
validation_generator = validation_datagen.flow_from_directory(
validation_dir,
target_size=(image_size, image_size),
batch_size=val_batchsize,
class_mode='categorical',
shuffle=False)
# Get the filenames from the generator
fnames = validation_generator.filenames
# Get the ground truth from generator
ground_truth = validation_generator.classes
# Get the label to class mapping from the generator
label2index = validation_generator.class_indices
# Getting the mapping from class index to class label
idx2label = dict((v,k) for k,v in label2index.items())
# Get the predictions from the model using the generator
predictions = model.predict_generator(validation_generator, steps=validation_generator.samples/validation_generator.batch_size,verbose=1)
predicted_classes = np.argmax(predictions,axis=1)
errors = np.where(predicted_classes != ground_truth)[0]
print("No of errors = {}/{}".format(len(errors),validation_generator.samples))
# Show the errors
for i in range(len(errors)):
pred_class = np.argmax(predictions[errors[i]])
pred_label = idx2label[pred_class]
title = 'Original label:{}, Prediction :{}, confidence : {:.3f}'.format(
fnames[errors[i]].split('/')[0],
pred_label,
predictions[errors[i]][pred_class])
original = load_img('{}/{}'.format(validation_dir,fnames[errors[i]]))
plt.figure(figsize=[7,7])
plt.axis('off')
plt.title(title)
plt.imshow(original)
plt.show()
validation_generator.classes is arranged but predicted_classes is not arranged.
I take the code from here https://www.learnopencv.com/keras-tutorial-fine-tuning-using-pre-trained-models/
How can I prevent predict_generator from shuffling data?
machine-learning keras neural-network deep-learning
machine-learning keras neural-network deep-learning
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
asked Nov 24 '18 at 12:15
NoranNoran
239110
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
asked Nov 24 '18 at 12:15
NoranNoran
239110
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
edited Nov 24 '18 at 13:39
desertnaut
20.9k84579
20.9k84579
asked Nov 24 '18 at 12:15
NoranNoran
239110
asked Nov 24 '18 at 12:15
NoranNoran
239110
asked Nov 24 '18 at 12:15
NoranNoran
239110
239110
I don't get what you mean by X is arranged but Y is not arranged.
– Matias Valdenegro
Nov 24 '18 at 13:18
@MatiasValdenegro the order of the items in validation_generator.classes is same as the order of the data in dataset. But predict_generator does not do the predication in the same order of the dataset.
– Noran
Nov 24 '18 at 13:58
Please add detail of this, the classes attribute in the generator gives you the class indices, are you saying that the class indices do not match the output of the generator? If so, add examples of this. Also you should make sure that there are no new or unseen classes in the validation data (this would change class indices).
– Matias Valdenegro
Nov 24 '18 at 14:12
My code is same as the code in the link.. The data in validation_generator.classes is [0,0,0,0,....1,1,1,1,........2,2,2..] But predict_generator changes the order of the data and make the predication for the images as [1,1,1,...2,2,2,...,0,0,0] or [2,2,2... 0,0,0....1,1,1...2,2.. ]
– Noran
Nov 24 '18 at 14:20
I think the tutorial is not doing it right, it shouldn't be using generator.classes, but instead each time the generator is called it returns the image data and the corresponding label.
– Matias Valdenegro
Nov 24 '18 at 16:59
|
show 1 more comment
I don't get what you mean by X is arranged but Y is not arranged.
– Matias Valdenegro
Nov 24 '18 at 13:18
@MatiasValdenegro the order of the items in validation_generator.classes is same as the order of the data in dataset. But predict_generator does not do the predication in the same order of the dataset.
– Noran
Nov 24 '18 at 13:58
Please add detail of this, the classes attribute in the generator gives you the class indices, are you saying that the class indices do not match the output of the generator? If so, add examples of this. Also you should make sure that there are no new or unseen classes in the validation data (this would change class indices).
– Matias Valdenegro
Nov 24 '18 at 14:12
My code is same as the code in the link.. The data in validation_generator.classes is [0,0,0,0,....1,1,1,1,........2,2,2..] But predict_generator changes the order of the data and make the predication for the images as [1,1,1,...2,2,2,...,0,0,0] or [2,2,2... 0,0,0....1,1,1...2,2.. ]
– Noran
Nov 24 '18 at 14:20
I think the tutorial is not doing it right, it shouldn't be using generator.classes, but instead each time the generator is called it returns the image data and the corresponding label.
– Matias Valdenegro
Nov 24 '18 at 16:59
I don't get what you mean by X is arranged but Y is not arranged.
– Matias Valdenegro
Nov 24 '18 at 13:18
I don't get what you mean by X is arranged but Y is not arranged.
– Matias Valdenegro
Nov 24 '18 at 13:18
@MatiasValdenegro the order of the items in validation_generator.classes is same as the order of the data in dataset. But predict_generator does not do the predication in the same order of the dataset.
– Noran
Nov 24 '18 at 13:58
@MatiasValdenegro the order of the items in validation_generator.classes is same as the order of the data in dataset. But predict_generator does not do the predication in the same order of the dataset.
– Noran
Nov 24 '18 at 13:58
Please add detail of this, the classes attribute in the generator gives you the class indices, are you saying that the class indices do not match the output of the generator? If so, add examples of this. Also you should make sure that there are no new or unseen classes in the validation data (this would change class indices).
– Matias Valdenegro
Nov 24 '18 at 14:12
Please add detail of this, the classes attribute in the generator gives you the class indices, are you saying that the class indices do not match the output of the generator? If so, add examples of this. Also you should make sure that there are no new or unseen classes in the validation data (this would change class indices).
– Matias Valdenegro
Nov 24 '18 at 14:12
My code is same as the code in the link.. The data in validation_generator.classes is [0,0,0,0,....1,1,1,1,........2,2,2..] But predict_generator changes the order of the data and make the predication for the images as [1,1,1,...2,2,2,...,0,0,0] or [2,2,2... 0,0,0....1,1,1...2,2.. ]
– Noran
Nov 24 '18 at 14:20
My code is same as the code in the link.. The data in validation_generator.classes is [0,0,0,0,....1,1,1,1,........2,2,2..] But predict_generator changes the order of the data and make the predication for the images as [1,1,1,...2,2,2,...,0,0,0] or [2,2,2... 0,0,0....1,1,1...2,2.. ]
– Noran
Nov 24 '18 at 14:20
I think the tutorial is not doing it right, it shouldn't be using generator.classes, but instead each time the generator is called it returns the image data and the corresponding label.
– Matias Valdenegro
Nov 24 '18 at 16:59
I think the tutorial is not doing it right, it shouldn't be using generator.classes, but instead each time the generator is called it returns the image data and the corresponding label.
– Matias Valdenegro
Nov 24 '18 at 16:59
|
show 1 more comment
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53458051%2fhow-to-prevent-keras-predict-generator-from-shuffling-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53458051%2fhow-to-prevent-keras-predict-generator-from-shuffling-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I don't get what you mean by X is arranged but Y is not arranged.
– Matias Valdenegro
Nov 24 '18 at 13:18
@MatiasValdenegro the order of the items in validation_generator.classes is same as the order of the data in dataset. But predict_generator does not do the predication in the same order of the dataset.
– Noran
Nov 24 '18 at 13:58
Please add detail of this, the classes attribute in the generator gives you the class indices, are you saying that the class indices do not match the output of the generator? If so, add examples of this. Also you should make sure that there are no new or unseen classes in the validation data (this would change class indices).
– Matias Valdenegro
Nov 24 '18 at 14:12
My code is same as the code in the link.. The data in validation_generator.classes is [0,0,0,0,....1,1,1,1,........2,2,2..] But predict_generator changes the order of the data and make the predication for the images as [1,1,1,...2,2,2,...,0,0,0] or [2,2,2... 0,0,0....1,1,1...2,2.. ]
– Noran
Nov 24 '18 at 14:20
I think the tutorial is not doing it right, it shouldn't be using generator.classes, but instead each time the generator is called it returns the image data and the corresponding label.
– Matias Valdenegro
Nov 24 '18 at 16:59