ObjectCreated:Post on S3 Console upload?
0
My S3 Lambda Event listener is only seeing ObjectCreated:Put events when a file is uploaded via the S3 console. This is both for new files and overwriting existing files. Is this the expected behavior?
It seems like a new file upload should generate ObjectCreated:Post in keeping with the POST == Create, PUT == Update norm.
amazon-web-services amazon-s3 aws-lambda aws-sdk
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
add a comment |
0
My S3 Lambda Event listener is only seeing ObjectCreated:Put events when a file is uploaded via the S3 console. This is both for new files and overwriting existing files. Is this the expected behavior?
It seems like a new file upload should generate ObjectCreated:Post in keeping with the POST == Create, PUT == Update norm.
amazon-web-services amazon-s3 aws-lambda aws-sdk
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
add a comment |
0
0
0
My S3 Lambda Event listener is only seeing ObjectCreated:Put events when a file is uploaded via the S3 console. This is both for new files and overwriting existing files. Is this the expected behavior?
It seems like a new file upload should generate ObjectCreated:Post in keeping with the POST == Create, PUT == Update norm.
amazon-web-services amazon-s3 aws-lambda aws-sdk
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
My S3 Lambda Event listener is only seeing ObjectCreated:Put events when a file is uploaded via the S3 console. This is both for new files and overwriting existing files. Is this the expected behavior?
It seems like a new file upload should generate ObjectCreated:Post in keeping with the POST == Create, PUT == Update norm.
amazon-web-services amazon-s3 aws-lambda aws-sdk
amazon-web-services amazon-s3 aws-lambda aws-sdk
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
asked Nov 21 '18 at 22:49
Mike SummersMike Summers
1,23131941
1,23131941
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
0
S3 has 4 APIs for object creation:
PUT is used for requests that send only the raw object bytes in the HTTP request body. It is the most common API used for creation of objects up to 5 GB in size.- POST uses specially-crafted HTML forms with attributes, authentication, and a file all as part of a
multipart/form-dataHTTP request body.
Copy is used where the source bytes come from an existing object in HTTP (which incidentally also uses HTTPPUTon the wire, but is its own event type). The Copy API is also used any time you edit the metadata of an existing object: once stored in S3, objects and their metadata are completely immutable. The console allows you to "edit" metadata, but it accomplishes this by copying the object on top of itself (which is a safe operation in S3, even when bucket versioning is not enabled, because the old object is untouched until the new object creation has succeeded) while supplying revised metadata. S3 does not support move or rename -- these are done with a copy followed by a delete. The maximum size of object that can be copied with the Copy API is 5 GB.
Multipart, which is mandatory for creating objects exceeding 5 GB and recommended for multi-megabyte objects. Multipart can be used for objects of any size, but each part (other than the last) must be at least 5 MiB in size, so it is not typically used for smaller uploads. This API also allows safe retrying of any parts that failed, uploading parts in parallel, and has multiple integrity checks to prevent any defects from appearing in the object that S3 reassembles. Multipart is also used to copy large objects
The console communicates with S3 using the standard public APIs, the same as the SDKs use, and uses either PUT or multipart, depending on the object size, and Copy for editing object metadata, as mentioned above.
For best results, always use the s3:ObjectCreated:* event, unless you have a specific reason not to.
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
Thanks, I familiar with the API. My underlying question is why the console doesn't hide the implementation leakage and behave as expected from an RFC 2616 point of view. The current behavior pushes logic into the app tracking new vs update that belongs in the console.
– Mike Summers
Nov 22 '18 at 15:21
I don't understand why you believe the behavior of S3's implementation is inconsistent with RFC 2616, which allowsPUTto be used for resource creation, not just replacement. "If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."
– Michael - sqlbot
Nov 22 '18 at 15:47
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53421520%2fobjectcreatedpost-on-s3-console-upload%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
S3 has 4 APIs for object creation:
PUT is used for requests that send only the raw object bytes in the HTTP request body. It is the most common API used for creation of objects up to 5 GB in size.- POST uses specially-crafted HTML forms with attributes, authentication, and a file all as part of a
multipart/form-dataHTTP request body.
Copy is used where the source bytes come from an existing object in HTTP (which incidentally also uses HTTPPUTon the wire, but is its own event type). The Copy API is also used any time you edit the metadata of an existing object: once stored in S3, objects and their metadata are completely immutable. The console allows you to "edit" metadata, but it accomplishes this by copying the object on top of itself (which is a safe operation in S3, even when bucket versioning is not enabled, because the old object is untouched until the new object creation has succeeded) while supplying revised metadata. S3 does not support move or rename -- these are done with a copy followed by a delete. The maximum size of object that can be copied with the Copy API is 5 GB.
Multipart, which is mandatory for creating objects exceeding 5 GB and recommended for multi-megabyte objects. Multipart can be used for objects of any size, but each part (other than the last) must be at least 5 MiB in size, so it is not typically used for smaller uploads. This API also allows safe retrying of any parts that failed, uploading parts in parallel, and has multiple integrity checks to prevent any defects from appearing in the object that S3 reassembles. Multipart is also used to copy large objects
The console communicates with S3 using the standard public APIs, the same as the SDKs use, and uses either PUT or multipart, depending on the object size, and Copy for editing object metadata, as mentioned above.
For best results, always use the s3:ObjectCreated:* event, unless you have a specific reason not to.
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
Thanks, I familiar with the API. My underlying question is why the console doesn't hide the implementation leakage and behave as expected from an RFC 2616 point of view. The current behavior pushes logic into the app tracking new vs update that belongs in the console.
– Mike Summers
Nov 22 '18 at 15:21
I don't understand why you believe the behavior of S3's implementation is inconsistent with RFC 2616, which allowsPUTto be used for resource creation, not just replacement. "If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."
– Michael - sqlbot
Nov 22 '18 at 15:47
add a comment |
0
S3 has 4 APIs for object creation:
PUT is used for requests that send only the raw object bytes in the HTTP request body. It is the most common API used for creation of objects up to 5 GB in size.- POST uses specially-crafted HTML forms with attributes, authentication, and a file all as part of a
multipart/form-dataHTTP request body.
Copy is used where the source bytes come from an existing object in HTTP (which incidentally also uses HTTPPUTon the wire, but is its own event type). The Copy API is also used any time you edit the metadata of an existing object: once stored in S3, objects and their metadata are completely immutable. The console allows you to "edit" metadata, but it accomplishes this by copying the object on top of itself (which is a safe operation in S3, even when bucket versioning is not enabled, because the old object is untouched until the new object creation has succeeded) while supplying revised metadata. S3 does not support move or rename -- these are done with a copy followed by a delete. The maximum size of object that can be copied with the Copy API is 5 GB.
Multipart, which is mandatory for creating objects exceeding 5 GB and recommended for multi-megabyte objects. Multipart can be used for objects of any size, but each part (other than the last) must be at least 5 MiB in size, so it is not typically used for smaller uploads. This API also allows safe retrying of any parts that failed, uploading parts in parallel, and has multiple integrity checks to prevent any defects from appearing in the object that S3 reassembles. Multipart is also used to copy large objects
The console communicates with S3 using the standard public APIs, the same as the SDKs use, and uses either PUT or multipart, depending on the object size, and Copy for editing object metadata, as mentioned above.
For best results, always use the s3:ObjectCreated:* event, unless you have a specific reason not to.
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
Thanks, I familiar with the API. My underlying question is why the console doesn't hide the implementation leakage and behave as expected from an RFC 2616 point of view. The current behavior pushes logic into the app tracking new vs update that belongs in the console.
– Mike Summers
Nov 22 '18 at 15:21
I don't understand why you believe the behavior of S3's implementation is inconsistent with RFC 2616, which allowsPUTto be used for resource creation, not just replacement. "If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."
– Michael - sqlbot
Nov 22 '18 at 15:47
add a comment |
0
0
0
S3 has 4 APIs for object creation:
PUT is used for requests that send only the raw object bytes in the HTTP request body. It is the most common API used for creation of objects up to 5 GB in size.- POST uses specially-crafted HTML forms with attributes, authentication, and a file all as part of a
multipart/form-dataHTTP request body.
Copy is used where the source bytes come from an existing object in HTTP (which incidentally also uses HTTPPUTon the wire, but is its own event type). The Copy API is also used any time you edit the metadata of an existing object: once stored in S3, objects and their metadata are completely immutable. The console allows you to "edit" metadata, but it accomplishes this by copying the object on top of itself (which is a safe operation in S3, even when bucket versioning is not enabled, because the old object is untouched until the new object creation has succeeded) while supplying revised metadata. S3 does not support move or rename -- these are done with a copy followed by a delete. The maximum size of object that can be copied with the Copy API is 5 GB.
Multipart, which is mandatory for creating objects exceeding 5 GB and recommended for multi-megabyte objects. Multipart can be used for objects of any size, but each part (other than the last) must be at least 5 MiB in size, so it is not typically used for smaller uploads. This API also allows safe retrying of any parts that failed, uploading parts in parallel, and has multiple integrity checks to prevent any defects from appearing in the object that S3 reassembles. Multipart is also used to copy large objects
The console communicates with S3 using the standard public APIs, the same as the SDKs use, and uses either PUT or multipart, depending on the object size, and Copy for editing object metadata, as mentioned above.
For best results, always use the s3:ObjectCreated:* event, unless you have a specific reason not to.
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
S3 has 4 APIs for object creation:
PUT is used for requests that send only the raw object bytes in the HTTP request body. It is the most common API used for creation of objects up to 5 GB in size.- POST uses specially-crafted HTML forms with attributes, authentication, and a file all as part of a
multipart/form-dataHTTP request body.
Copy is used where the source bytes come from an existing object in HTTP (which incidentally also uses HTTPPUTon the wire, but is its own event type). The Copy API is also used any time you edit the metadata of an existing object: once stored in S3, objects and their metadata are completely immutable. The console allows you to "edit" metadata, but it accomplishes this by copying the object on top of itself (which is a safe operation in S3, even when bucket versioning is not enabled, because the old object is untouched until the new object creation has succeeded) while supplying revised metadata. S3 does not support move or rename -- these are done with a copy followed by a delete. The maximum size of object that can be copied with the Copy API is 5 GB.
Multipart, which is mandatory for creating objects exceeding 5 GB and recommended for multi-megabyte objects. Multipart can be used for objects of any size, but each part (other than the last) must be at least 5 MiB in size, so it is not typically used for smaller uploads. This API also allows safe retrying of any parts that failed, uploading parts in parallel, and has multiple integrity checks to prevent any defects from appearing in the object that S3 reassembles. Multipart is also used to copy large objects
The console communicates with S3 using the standard public APIs, the same as the SDKs use, and uses either PUT or multipart, depending on the object size, and Copy for editing object metadata, as mentioned above.
For best results, always use the s3:ObjectCreated:* event, unless you have a specific reason not to.
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
answered Nov 22 '18 at 1:25
Michael - sqlbotMichael - sqlbot
93.7k13140203
93.7k13140203
Thanks, I familiar with the API. My underlying question is why the console doesn't hide the implementation leakage and behave as expected from an RFC 2616 point of view. The current behavior pushes logic into the app tracking new vs update that belongs in the console.
– Mike Summers
Nov 22 '18 at 15:21
I don't understand why you believe the behavior of S3's implementation is inconsistent with RFC 2616, which allowsPUTto be used for resource creation, not just replacement. "If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."
– Michael - sqlbot
Nov 22 '18 at 15:47
add a comment |
Thanks, I familiar with the API. My underlying question is why the console doesn't hide the implementation leakage and behave as expected from an RFC 2616 point of view. The current behavior pushes logic into the app tracking new vs update that belongs in the console.
– Mike Summers
Nov 22 '18 at 15:21
I don't understand why you believe the behavior of S3's implementation is inconsistent with RFC 2616, which allowsPUTto be used for resource creation, not just replacement. "If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."
– Michael - sqlbot
Nov 22 '18 at 15:47
Thanks, I familiar with the API. My underlying question is why the console doesn't hide the implementation leakage and behave as expected from an RFC 2616 point of view. The current behavior pushes logic into the app tracking new vs update that belongs in the console.
– Mike Summers
Nov 22 '18 at 15:21
Thanks, I familiar with the API. My underlying question is why the console doesn't hide the implementation leakage and behave as expected from an RFC 2616 point of view. The current behavior pushes logic into the app tracking new vs update that belongs in the console.
– Mike Summers
Nov 22 '18 at 15:21
I don't understand why you believe the behavior of S3's implementation is inconsistent with RFC 2616, which allows
PUT to be used for resource creation, not just replacement. "If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."– Michael - sqlbot
Nov 22 '18 at 15:47
I don't understand why you believe the behavior of S3's implementation is inconsistent with RFC 2616, which allows
PUT to be used for resource creation, not just replacement. "If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."– Michael - sqlbot
Nov 22 '18 at 15:47
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53421520%2fobjectcreatedpost-on-s3-console-upload%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
