How do I handle async tests in python?
I'm trying to write a test to make sure that an entity cannot be added to google cloud datastore twice according to some validation (or unique key made up by multiple fields).
To give you an example, let's say I want to save a simple image_url and a type. Now there can only be one image that has a certain URL and a certain type.
This is easy enough to check (at least for now I verify that my datastore does not find an entity with such values).
But when writing a test, I get inconsistent results due to latency I suppose. So something like:
def test_foo_bar(self):
self.datastore.save_image(self.image_metadata)
with self.assertRaises(ValueError):
self.datastore.save_image(self.image_metadata)
Now the problem seems to be that by the time I check if an entity already exists, the just created entity is not there yet (?). So it doesn't throw an error, creates a duplicate and the test fails. This far the only way to get some consistency was to add a time.sleep(1) to give it more time, but still, at times the test doesn't pass (although it does most of the time). I personally don't like this inconsistency.
What is a good approach to avoid this behavior?
python python-3.x google-cloud-datastore
edited Nov 16 '18 at 1:22
Esref
2491515
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
|
show 5 more comments
I'm trying to write a test to make sure that an entity cannot be added to google cloud datastore twice according to some validation (or unique key made up by multiple fields).
To give you an example, let's say I want to save a simple image_url and a type. Now there can only be one image that has a certain URL and a certain type.
This is easy enough to check (at least for now I verify that my datastore does not find an entity with such values).
But when writing a test, I get inconsistent results due to latency I suppose. So something like:
def test_foo_bar(self):
self.datastore.save_image(self.image_metadata)
with self.assertRaises(ValueError):
self.datastore.save_image(self.image_metadata)
Now the problem seems to be that by the time I check if an entity already exists, the just created entity is not there yet (?). So it doesn't throw an error, creates a duplicate and the test fails. This far the only way to get some consistency was to add a time.sleep(1) to give it more time, but still, at times the test doesn't pass (although it does most of the time). I personally don't like this inconsistency.
What is a good approach to avoid this behavior?
python python-3.x google-cloud-datastore
edited Nov 16 '18 at 1:22
Esref
2491515
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
1
The most important quesion is: what is guaranteed here? You seem to say that there is no guarantee that the image is saved after.save_imagecall returns. So what else can you do? If there is no guarantee, there is nothing you can do. If there is a guarantee, for example "image will be saved after 10s" then you should try after 10s. Also asynchronity is one thing, incosistency the other. Why does.save_imagemethod create duplicate? This shouldn't happen, no matter when the actual saving happens.
– freakish
Nov 15 '18 at 21:32
I see, there's no safe guarantee but one can only hope that it will never take more than, for example, 5s. Other than that I can only completely mock the datastore so that it all happens locally without any actual connection to it and leave that for e2e testing. Would that be a good idea ?
– G4bri3l
Nov 15 '18 at 21:36
In regards to duplicates, the only way to prevent duplicate creation is by checking if an entity with that data has been created before. In order for me to test that I have to try and create the same resource twice and get an error. So I'm not sure what you mean there, it can only happen AFTER a resource has been saved already. Maybe I'm not understanding your point.
– G4bri3l
Nov 15 '18 at 21:41
(1) Sure, you can do e2e testing, but you will have the same problem there. You don't solve anything. (2) You talk about "duplicates". If I try to add the same row twice (with the same id) I will get an error (assuming id is unique). No duplicate can happen. And saving being synchronous or not doesn't matter. So what I don't understand: how can you get duplicates in this scenario? One of the saves should fail eventually. Unless its a limitation of google cloud datastore?
– freakish
Nov 15 '18 at 21:44
Oh ok, so datastore is quite flexible. When I create a resource I just specify values for each column (as you would expect) and the id is autogenerated for you so there's never conflicts on ids (as any DB system would do). So I can create 1000 entries with same url and type and they will all have different ids. One thing I can do is to generate an id based on the unique fields (maybe a hash of url and type) so that duplicates will have conflicting ids and datastore will not allow it to begin with. Datastore does not guarantee uniqueness unless you manually specify an id, which I think I'll do.
– G4bri3l
Nov 15 '18 at 21:52
|
show 5 more comments
I'm trying to write a test to make sure that an entity cannot be added to google cloud datastore twice according to some validation (or unique key made up by multiple fields).
To give you an example, let's say I want to save a simple image_url and a type. Now there can only be one image that has a certain URL and a certain type.
This is easy enough to check (at least for now I verify that my datastore does not find an entity with such values).
But when writing a test, I get inconsistent results due to latency I suppose. So something like:
def test_foo_bar(self):
self.datastore.save_image(self.image_metadata)
with self.assertRaises(ValueError):
self.datastore.save_image(self.image_metadata)
Now the problem seems to be that by the time I check if an entity already exists, the just created entity is not there yet (?). So it doesn't throw an error, creates a duplicate and the test fails. This far the only way to get some consistency was to add a time.sleep(1) to give it more time, but still, at times the test doesn't pass (although it does most of the time). I personally don't like this inconsistency.
What is a good approach to avoid this behavior?
python python-3.x google-cloud-datastore
edited Nov 16 '18 at 1:22
Esref
2491515
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
I'm trying to write a test to make sure that an entity cannot be added to google cloud datastore twice according to some validation (or unique key made up by multiple fields).
To give you an example, let's say I want to save a simple image_url and a type. Now there can only be one image that has a certain URL and a certain type.
This is easy enough to check (at least for now I verify that my datastore does not find an entity with such values).
But when writing a test, I get inconsistent results due to latency I suppose. So something like:
def test_foo_bar(self):
self.datastore.save_image(self.image_metadata)
with self.assertRaises(ValueError):
self.datastore.save_image(self.image_metadata)
Now the problem seems to be that by the time I check if an entity already exists, the just created entity is not there yet (?). So it doesn't throw an error, creates a duplicate and the test fails. This far the only way to get some consistency was to add a time.sleep(1) to give it more time, but still, at times the test doesn't pass (although it does most of the time). I personally don't like this inconsistency.
What is a good approach to avoid this behavior?
python python-3.x google-cloud-datastore
python python-3.x google-cloud-datastore
edited Nov 16 '18 at 1:22
Esref
2491515
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
edited Nov 16 '18 at 1:22
Esref
2491515
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
edited Nov 16 '18 at 1:22
Esref
2491515
edited Nov 16 '18 at 1:22
Esref
2491515
edited Nov 16 '18 at 1:22
Esref
2491515
2491515
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
asked Nov 15 '18 at 21:28
G4bri3lG4bri3l
1,81921937
1,81921937
1
The most important quesion is: what is guaranteed here? You seem to say that there is no guarantee that the image is saved after.save_imagecall returns. So what else can you do? If there is no guarantee, there is nothing you can do. If there is a guarantee, for example "image will be saved after 10s" then you should try after 10s. Also asynchronity is one thing, incosistency the other. Why does.save_imagemethod create duplicate? This shouldn't happen, no matter when the actual saving happens.
– freakish
Nov 15 '18 at 21:32
I see, there's no safe guarantee but one can only hope that it will never take more than, for example, 5s. Other than that I can only completely mock the datastore so that it all happens locally without any actual connection to it and leave that for e2e testing. Would that be a good idea ?
– G4bri3l
Nov 15 '18 at 21:36
In regards to duplicates, the only way to prevent duplicate creation is by checking if an entity with that data has been created before. In order for me to test that I have to try and create the same resource twice and get an error. So I'm not sure what you mean there, it can only happen AFTER a resource has been saved already. Maybe I'm not understanding your point.
– G4bri3l
Nov 15 '18 at 21:41
(1) Sure, you can do e2e testing, but you will have the same problem there. You don't solve anything. (2) You talk about "duplicates". If I try to add the same row twice (with the same id) I will get an error (assuming id is unique). No duplicate can happen. And saving being synchronous or not doesn't matter. So what I don't understand: how can you get duplicates in this scenario? One of the saves should fail eventually. Unless its a limitation of google cloud datastore?
– freakish
Nov 15 '18 at 21:44
Oh ok, so datastore is quite flexible. When I create a resource I just specify values for each column (as you would expect) and the id is autogenerated for you so there's never conflicts on ids (as any DB system would do). So I can create 1000 entries with same url and type and they will all have different ids. One thing I can do is to generate an id based on the unique fields (maybe a hash of url and type) so that duplicates will have conflicting ids and datastore will not allow it to begin with. Datastore does not guarantee uniqueness unless you manually specify an id, which I think I'll do.
– G4bri3l
Nov 15 '18 at 21:52
|
show 5 more comments
1
The most important quesion is: what is guaranteed here? You seem to say that there is no guarantee that the image is saved after.save_imagecall returns. So what else can you do? If there is no guarantee, there is nothing you can do. If there is a guarantee, for example "image will be saved after 10s" then you should try after 10s. Also asynchronity is one thing, incosistency the other. Why does.save_imagemethod create duplicate? This shouldn't happen, no matter when the actual saving happens.
– freakish
Nov 15 '18 at 21:32
I see, there's no safe guarantee but one can only hope that it will never take more than, for example, 5s. Other than that I can only completely mock the datastore so that it all happens locally without any actual connection to it and leave that for e2e testing. Would that be a good idea ?
– G4bri3l
Nov 15 '18 at 21:36
In regards to duplicates, the only way to prevent duplicate creation is by checking if an entity with that data has been created before. In order for me to test that I have to try and create the same resource twice and get an error. So I'm not sure what you mean there, it can only happen AFTER a resource has been saved already. Maybe I'm not understanding your point.
– G4bri3l
Nov 15 '18 at 21:41
(1) Sure, you can do e2e testing, but you will have the same problem there. You don't solve anything. (2) You talk about "duplicates". If I try to add the same row twice (with the same id) I will get an error (assuming id is unique). No duplicate can happen. And saving being synchronous or not doesn't matter. So what I don't understand: how can you get duplicates in this scenario? One of the saves should fail eventually. Unless its a limitation of google cloud datastore?
– freakish
Nov 15 '18 at 21:44
Oh ok, so datastore is quite flexible. When I create a resource I just specify values for each column (as you would expect) and the id is autogenerated for you so there's never conflicts on ids (as any DB system would do). So I can create 1000 entries with same url and type and they will all have different ids. One thing I can do is to generate an id based on the unique fields (maybe a hash of url and type) so that duplicates will have conflicting ids and datastore will not allow it to begin with. Datastore does not guarantee uniqueness unless you manually specify an id, which I think I'll do.
– G4bri3l
Nov 15 '18 at 21:52
1
1
The most important quesion is: what is guaranteed here? You seem to say that there is no guarantee that the image is saved after
.save_image call returns. So what else can you do? If there is no guarantee, there is nothing you can do. If there is a guarantee, for example "image will be saved after 10s" then you should try after 10s. Also asynchronity is one thing, incosistency the other. Why does .save_image method create duplicate? This shouldn't happen, no matter when the actual saving happens.– freakish
Nov 15 '18 at 21:32
The most important quesion is: what is guaranteed here? You seem to say that there is no guarantee that the image is saved after
.save_image call returns. So what else can you do? If there is no guarantee, there is nothing you can do. If there is a guarantee, for example "image will be saved after 10s" then you should try after 10s. Also asynchronity is one thing, incosistency the other. Why does .save_image method create duplicate? This shouldn't happen, no matter when the actual saving happens.– freakish
Nov 15 '18 at 21:32
I see, there's no safe guarantee but one can only hope that it will never take more than, for example, 5s. Other than that I can only completely mock the datastore so that it all happens locally without any actual connection to it and leave that for e2e testing. Would that be a good idea ?
– G4bri3l
Nov 15 '18 at 21:36
I see, there's no safe guarantee but one can only hope that it will never take more than, for example, 5s. Other than that I can only completely mock the datastore so that it all happens locally without any actual connection to it and leave that for e2e testing. Would that be a good idea ?
– G4bri3l
Nov 15 '18 at 21:36
In regards to duplicates, the only way to prevent duplicate creation is by checking if an entity with that data has been created before. In order for me to test that I have to try and create the same resource twice and get an error. So I'm not sure what you mean there, it can only happen AFTER a resource has been saved already. Maybe I'm not understanding your point.
– G4bri3l
Nov 15 '18 at 21:41
In regards to duplicates, the only way to prevent duplicate creation is by checking if an entity with that data has been created before. In order for me to test that I have to try and create the same resource twice and get an error. So I'm not sure what you mean there, it can only happen AFTER a resource has been saved already. Maybe I'm not understanding your point.
– G4bri3l
Nov 15 '18 at 21:41
(1) Sure, you can do e2e testing, but you will have the same problem there. You don't solve anything. (2) You talk about "duplicates". If I try to add the same row twice (with the same id) I will get an error (assuming id is unique). No duplicate can happen. And saving being synchronous or not doesn't matter. So what I don't understand: how can you get duplicates in this scenario? One of the saves should fail eventually. Unless its a limitation of google cloud datastore?
– freakish
Nov 15 '18 at 21:44
(1) Sure, you can do e2e testing, but you will have the same problem there. You don't solve anything. (2) You talk about "duplicates". If I try to add the same row twice (with the same id) I will get an error (assuming id is unique). No duplicate can happen. And saving being synchronous or not doesn't matter. So what I don't understand: how can you get duplicates in this scenario? One of the saves should fail eventually. Unless its a limitation of google cloud datastore?
– freakish
Nov 15 '18 at 21:44
Oh ok, so datastore is quite flexible. When I create a resource I just specify values for each column (as you would expect) and the id is autogenerated for you so there's never conflicts on ids (as any DB system would do). So I can create 1000 entries with same url and type and they will all have different ids. One thing I can do is to generate an id based on the unique fields (maybe a hash of url and type) so that duplicates will have conflicting ids and datastore will not allow it to begin with. Datastore does not guarantee uniqueness unless you manually specify an id, which I think I'll do.
– G4bri3l
Nov 15 '18 at 21:52
Oh ok, so datastore is quite flexible. When I create a resource I just specify values for each column (as you would expect) and the id is autogenerated for you so there's never conflicts on ids (as any DB system would do). So I can create 1000 entries with same url and type and they will all have different ids. One thing I can do is to generate an id based on the unique fields (maybe a hash of url and type) so that duplicates will have conflicting ids and datastore will not allow it to begin with. Datastore does not guarantee uniqueness unless you manually specify an id, which I think I'll do.
– G4bri3l
Nov 15 '18 at 21:52
|
show 5 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53328145%2fhow-do-i-handle-async-tests-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53328145%2fhow-do-i-handle-async-tests-in-python%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
The most important quesion is: what is guaranteed here? You seem to say that there is no guarantee that the image is saved after
.save_imagecall returns. So what else can you do? If there is no guarantee, there is nothing you can do. If there is a guarantee, for example "image will be saved after 10s" then you should try after 10s. Also asynchronity is one thing, incosistency the other. Why does.save_imagemethod create duplicate? This shouldn't happen, no matter when the actual saving happens.– freakish
Nov 15 '18 at 21:32
I see, there's no safe guarantee but one can only hope that it will never take more than, for example, 5s. Other than that I can only completely mock the datastore so that it all happens locally without any actual connection to it and leave that for e2e testing. Would that be a good idea ?
– G4bri3l
Nov 15 '18 at 21:36
In regards to duplicates, the only way to prevent duplicate creation is by checking if an entity with that data has been created before. In order for me to test that I have to try and create the same resource twice and get an error. So I'm not sure what you mean there, it can only happen AFTER a resource has been saved already. Maybe I'm not understanding your point.
– G4bri3l
Nov 15 '18 at 21:41
(1) Sure, you can do e2e testing, but you will have the same problem there. You don't solve anything. (2) You talk about "duplicates". If I try to add the same row twice (with the same id) I will get an error (assuming id is unique). No duplicate can happen. And saving being synchronous or not doesn't matter. So what I don't understand: how can you get duplicates in this scenario? One of the saves should fail eventually. Unless its a limitation of google cloud datastore?
– freakish
Nov 15 '18 at 21:44
Oh ok, so datastore is quite flexible. When I create a resource I just specify values for each column (as you would expect) and the id is autogenerated for you so there's never conflicts on ids (as any DB system would do). So I can create 1000 entries with same url and type and they will all have different ids. One thing I can do is to generate an id based on the unique fields (maybe a hash of url and type) so that duplicates will have conflicting ids and datastore will not allow it to begin with. Datastore does not guarantee uniqueness unless you manually specify an id, which I think I'll do.
– G4bri3l
Nov 15 '18 at 21:52