How can we exclude some data nodes only from one namespace in hdfs federation?
I was checking the hadoop federation. Here the data nodes are common, only namespace will be different. How can we exclude some datanodes only from one namespace.
hadoop hive hbase namenode datanode
edited Nov 14 '18 at 8:43
VIN
13311
asked Nov 13 '18 at 11:42
RahulRahul
330111
add a comment |
I was checking the hadoop federation. Here the data nodes are common, only namespace will be different. How can we exclude some datanodes only from one namespace.
hadoop hive hbase namenode datanode
edited Nov 14 '18 at 8:43
VIN
13311
asked Nov 13 '18 at 11:42
RahulRahul
330111
add a comment |
I was checking the hadoop federation. Here the data nodes are common, only namespace will be different. How can we exclude some datanodes only from one namespace.
hadoop hive hbase namenode datanode
edited Nov 14 '18 at 8:43
VIN
13311
asked Nov 13 '18 at 11:42
RahulRahul
330111
I was checking the hadoop federation. Here the data nodes are common, only namespace will be different. How can we exclude some datanodes only from one namespace.
hadoop hive hbase namenode datanode
hadoop hive hbase namenode datanode
edited Nov 14 '18 at 8:43
VIN
13311
asked Nov 13 '18 at 11:42
RahulRahul
330111
edited Nov 14 '18 at 8:43
VIN
13311
asked Nov 13 '18 at 11:42
RahulRahul
330111
edited Nov 14 '18 at 8:43
VIN
13311
edited Nov 14 '18 at 8:43
VIN
13311
edited Nov 14 '18 at 8:43
VIN
13311
13311
asked Nov 13 '18 at 11:42
RahulRahul
330111
asked Nov 13 '18 at 11:42
RahulRahul
330111
asked Nov 13 '18 at 11:42
RahulRahul
330111
330111
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In decommissioning the nodes that need to be decommissioned are added to the exclude file at all of the Namenodes. Each Namenode decommissions its Block Pool. When all the Namenodes finish decommissioning a Datanode, the Datanode is considered decommissioned.
For more details, you can have a look at: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/Federation.html#Decommissioning
answered Nov 13 '18 at 14:56
VINVIN
13311
Dear Vin, as i mentioned i want to remove a datanode from only one of the namenode not from both.
– Rahul
Nov 13 '18 at 20:09
@Rahul As per my understanding, you can't do that because either that data node will be connected to the cluster or not. And if it is there in the cluster it should get updated in each namespace otherwise it violates the data integrity
– VIN
Nov 13 '18 at 20:36
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53280288%2fhow-can-we-exclude-some-data-nodes-only-from-one-namespace-in-hdfs-federation%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In decommissioning the nodes that need to be decommissioned are added to the exclude file at all of the Namenodes. Each Namenode decommissions its Block Pool. When all the Namenodes finish decommissioning a Datanode, the Datanode is considered decommissioned.
For more details, you can have a look at: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/Federation.html#Decommissioning
answered Nov 13 '18 at 14:56
VINVIN
13311
Dear Vin, as i mentioned i want to remove a datanode from only one of the namenode not from both.
– Rahul
Nov 13 '18 at 20:09
@Rahul As per my understanding, you can't do that because either that data node will be connected to the cluster or not. And if it is there in the cluster it should get updated in each namespace otherwise it violates the data integrity
– VIN
Nov 13 '18 at 20:36
add a comment |
In decommissioning the nodes that need to be decommissioned are added to the exclude file at all of the Namenodes. Each Namenode decommissions its Block Pool. When all the Namenodes finish decommissioning a Datanode, the Datanode is considered decommissioned.
For more details, you can have a look at: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/Federation.html#Decommissioning
answered Nov 13 '18 at 14:56
VINVIN
13311
Dear Vin, as i mentioned i want to remove a datanode from only one of the namenode not from both.
– Rahul
Nov 13 '18 at 20:09
@Rahul As per my understanding, you can't do that because either that data node will be connected to the cluster or not. And if it is there in the cluster it should get updated in each namespace otherwise it violates the data integrity
– VIN
Nov 13 '18 at 20:36
add a comment |
In decommissioning the nodes that need to be decommissioned are added to the exclude file at all of the Namenodes. Each Namenode decommissions its Block Pool. When all the Namenodes finish decommissioning a Datanode, the Datanode is considered decommissioned.
For more details, you can have a look at: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/Federation.html#Decommissioning
answered Nov 13 '18 at 14:56
VINVIN
13311
In decommissioning the nodes that need to be decommissioned are added to the exclude file at all of the Namenodes. Each Namenode decommissions its Block Pool. When all the Namenodes finish decommissioning a Datanode, the Datanode is considered decommissioned.
For more details, you can have a look at: https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-hdfs/Federation.html#Decommissioning
answered Nov 13 '18 at 14:56
VINVIN
13311
answered Nov 13 '18 at 14:56
VINVIN
13311
answered Nov 13 '18 at 14:56
VINVIN
13311
answered Nov 13 '18 at 14:56
VINVIN
13311
13311
Dear Vin, as i mentioned i want to remove a datanode from only one of the namenode not from both.
– Rahul
Nov 13 '18 at 20:09
@Rahul As per my understanding, you can't do that because either that data node will be connected to the cluster or not. And if it is there in the cluster it should get updated in each namespace otherwise it violates the data integrity
– VIN
Nov 13 '18 at 20:36
add a comment |
Dear Vin, as i mentioned i want to remove a datanode from only one of the namenode not from both.
– Rahul
Nov 13 '18 at 20:09
@Rahul As per my understanding, you can't do that because either that data node will be connected to the cluster or not. And if it is there in the cluster it should get updated in each namespace otherwise it violates the data integrity
– VIN
Nov 13 '18 at 20:36
Dear Vin, as i mentioned i want to remove a datanode from only one of the namenode not from both.
– Rahul
Nov 13 '18 at 20:09
Dear Vin, as i mentioned i want to remove a datanode from only one of the namenode not from both.
– Rahul
Nov 13 '18 at 20:09
@Rahul As per my understanding, you can't do that because either that data node will be connected to the cluster or not. And if it is there in the cluster it should get updated in each namespace otherwise it violates the data integrity
– VIN
Nov 13 '18 at 20:36
@Rahul As per my understanding, you can't do that because either that data node will be connected to the cluster or not. And if it is there in the cluster it should get updated in each namespace otherwise it violates the data integrity
– VIN
Nov 13 '18 at 20:36
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53280288%2fhow-can-we-exclude-some-data-nodes-only-from-one-namespace-in-hdfs-federation%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
