OutOfMemoryError in Bufferedmutator of HBase
2
I am getting below error when data insertion in HBase through BufferedMutator,
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.sendMultiAction(AsyncRequestFutureImpl.java:547)
at org.apache.hadoop.hbase.client.AsyncProcess.submitMultiActions(AsyncProcess.java:337)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:320)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:228)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.doFlush(BufferedMutatorImpl.java:303)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.close(BufferedMutatorImpl.java:241)
Here I am Inserting data of length 200-300 bytes per request with single Connection. And Synchronized queries are generated. Also find below thread dump. where Timed waiting threads are of htable-pool.
Thread Dump
Can you please answer about how to tackle this Exception.
Sample code -
try(BufferedMutator mutator = Connection.creatConnection().getBufferedMutator(TableName.valueOf(tableName)) {
Put put = new Put(Bytes.toBytes(rowKey));
map.forEach(element->put.addColumn(CF,element.getKey(),element.getValue());
mutator.mutate(put);
} catch (Exception e) {
e.printStackTrace();
}
java hbase threadpool
asked Nov 21 '18 at 9:10
APLAPL
116
add a comment |
2
I am getting below error when data insertion in HBase through BufferedMutator,
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.sendMultiAction(AsyncRequestFutureImpl.java:547)
at org.apache.hadoop.hbase.client.AsyncProcess.submitMultiActions(AsyncProcess.java:337)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:320)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:228)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.doFlush(BufferedMutatorImpl.java:303)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.close(BufferedMutatorImpl.java:241)
Here I am Inserting data of length 200-300 bytes per request with single Connection. And Synchronized queries are generated. Also find below thread dump. where Timed waiting threads are of htable-pool.
Thread Dump
Can you please answer about how to tackle this Exception.
Sample code -
try(BufferedMutator mutator = Connection.creatConnection().getBufferedMutator(TableName.valueOf(tableName)) {
Put put = new Put(Bytes.toBytes(rowKey));
map.forEach(element->put.addColumn(CF,element.getKey(),element.getValue());
mutator.mutate(put);
} catch (Exception e) {
e.printStackTrace();
}
java hbase threadpool
asked Nov 21 '18 at 9:10
APLAPL
116
Please provide the relevant part of your code. Add a Minimal, Complete, and Verifiable example including proper example input/output data.
– Bsquare ℬℬ
Nov 21 '18 at 9:26
@Bsquare Added Sample code in question and input is String map with 100 entries.
– APL
Nov 21 '18 at 11:44
How do you launch your application? What is the configuration of HBase? In particular how many memory? On which OS are you working?
– Bsquare ℬℬ
Nov 21 '18 at 13:37
Simple Java Script is launched by java command From one server to HBase server. HBase have 1 Master and 6 Region servers. At the time of script running server have 25 - 30 GB free memory and running from ubuntu OS.
– APL
Nov 22 '18 at 4:55
add a comment |
2
2
2
2
I am getting below error when data insertion in HBase through BufferedMutator,
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.sendMultiAction(AsyncRequestFutureImpl.java:547)
at org.apache.hadoop.hbase.client.AsyncProcess.submitMultiActions(AsyncProcess.java:337)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:320)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:228)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.doFlush(BufferedMutatorImpl.java:303)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.close(BufferedMutatorImpl.java:241)
Here I am Inserting data of length 200-300 bytes per request with single Connection. And Synchronized queries are generated. Also find below thread dump. where Timed waiting threads are of htable-pool.
Thread Dump
Can you please answer about how to tackle this Exception.
Sample code -
try(BufferedMutator mutator = Connection.creatConnection().getBufferedMutator(TableName.valueOf(tableName)) {
Put put = new Put(Bytes.toBytes(rowKey));
map.forEach(element->put.addColumn(CF,element.getKey(),element.getValue());
mutator.mutate(put);
} catch (Exception e) {
e.printStackTrace();
}
java hbase threadpool
asked Nov 21 '18 at 9:10
APLAPL
116
I am getting below error when data insertion in HBase through BufferedMutator,
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:717)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:957)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.sendMultiAction(AsyncRequestFutureImpl.java:547)
at org.apache.hadoop.hbase.client.AsyncProcess.submitMultiActions(AsyncProcess.java:337)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:320)
at org.apache.hadoop.hbase.client.AsyncProcess.submit(AsyncProcess.java:228)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.doFlush(BufferedMutatorImpl.java:303)
at org.apache.hadoop.hbase.client.BufferedMutatorImpl.close(BufferedMutatorImpl.java:241)
Here I am Inserting data of length 200-300 bytes per request with single Connection. And Synchronized queries are generated. Also find below thread dump. where Timed waiting threads are of htable-pool.
Thread Dump
Can you please answer about how to tackle this Exception.
Sample code -
try(BufferedMutator mutator = Connection.creatConnection().getBufferedMutator(TableName.valueOf(tableName)) {
Put put = new Put(Bytes.toBytes(rowKey));
map.forEach(element->put.addColumn(CF,element.getKey(),element.getValue());
mutator.mutate(put);
} catch (Exception e) {
e.printStackTrace();
}
java hbase threadpool
java hbase threadpool
asked Nov 21 '18 at 9:10
APLAPL
116
asked Nov 21 '18 at 9:10
APLAPL
116
edited Nov 21 '18 at 11:40
APL
asked Nov 21 '18 at 9:10
APLAPL
116
asked Nov 21 '18 at 9:10
APLAPL
116
asked Nov 21 '18 at 9:10
APLAPL
116
116
Please provide the relevant part of your code. Add a Minimal, Complete, and Verifiable example including proper example input/output data.
– Bsquare ℬℬ
Nov 21 '18 at 9:26
@Bsquare Added Sample code in question and input is String map with 100 entries.
– APL
Nov 21 '18 at 11:44
How do you launch your application? What is the configuration of HBase? In particular how many memory? On which OS are you working?
– Bsquare ℬℬ
Nov 21 '18 at 13:37
Simple Java Script is launched by java command From one server to HBase server. HBase have 1 Master and 6 Region servers. At the time of script running server have 25 - 30 GB free memory and running from ubuntu OS.
– APL
Nov 22 '18 at 4:55
add a comment |
Please provide the relevant part of your code. Add a Minimal, Complete, and Verifiable example including proper example input/output data.
– Bsquare ℬℬ
Nov 21 '18 at 9:26
@Bsquare Added Sample code in question and input is String map with 100 entries.
– APL
Nov 21 '18 at 11:44
How do you launch your application? What is the configuration of HBase? In particular how many memory? On which OS are you working?
– Bsquare ℬℬ
Nov 21 '18 at 13:37
Simple Java Script is launched by java command From one server to HBase server. HBase have 1 Master and 6 Region servers. At the time of script running server have 25 - 30 GB free memory and running from ubuntu OS.
– APL
Nov 22 '18 at 4:55
Please provide the relevant part of your code. Add a Minimal, Complete, and Verifiable example including proper example input/output data.
– Bsquare ℬℬ
Nov 21 '18 at 9:26
Please provide the relevant part of your code. Add a Minimal, Complete, and Verifiable example including proper example input/output data.
– Bsquare ℬℬ
Nov 21 '18 at 9:26
@Bsquare Added Sample code in question and input is String map with 100 entries.
– APL
Nov 21 '18 at 11:44
@Bsquare Added Sample code in question and input is String map with 100 entries.
– APL
Nov 21 '18 at 11:44
How do you launch your application? What is the configuration of HBase? In particular how many memory? On which OS are you working?
– Bsquare ℬℬ
Nov 21 '18 at 13:37
How do you launch your application? What is the configuration of HBase? In particular how many memory? On which OS are you working?
– Bsquare ℬℬ
Nov 21 '18 at 13:37
Simple Java Script is launched by java command From one server to HBase server. HBase have 1 Master and 6 Region servers. At the time of script running server have 25 - 30 GB free memory and running from ubuntu OS.
– APL
Nov 22 '18 at 4:55
Simple Java Script is launched by java command From one server to HBase server. HBase have 1 Master and 6 Region servers. At the time of script running server have 25 - 30 GB free memory and running from ubuntu OS.
– APL
Nov 22 '18 at 4:55
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53408585%2foutofmemoryerror-in-bufferedmutator-of-hbase%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53408585%2foutofmemoryerror-in-bufferedmutator-of-hbase%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Please provide the relevant part of your code. Add a Minimal, Complete, and Verifiable example including proper example input/output data.
– Bsquare ℬℬ
Nov 21 '18 at 9:26
@Bsquare Added Sample code in question and input is String map with 100 entries.
– APL
Nov 21 '18 at 11:44
How do you launch your application? What is the configuration of HBase? In particular how many memory? On which OS are you working?
– Bsquare ℬℬ
Nov 21 '18 at 13:37
Simple Java Script is launched by java command From one server to HBase server. HBase have 1 Master and 6 Region servers. At the time of script running server have 25 - 30 GB free memory and running from ubuntu OS.
– APL
Nov 22 '18 at 4:55