SparkR - Error in db.readTypedVector(con, colType, numRows) : Unsupported type for deserialization: “some...

up vote
0
down vote

favorite

I have sparkDataFrame and I am running a function using gapplyCollect. Very often this fails with below error

Error in handleErrors(returnStatus, conn) : 

  org.apache.spark.SparkException: Job aborted due to stage failure: Task 25 in stage 694.0 failed 4 times, most recent failure: Lost task 25.3 in stage 694.0 (TID 24447, 10.139.64.4, executor 0): org.apache.spark.SparkException: R computation failed with

 Error in db.readTypedVector(con, colType, numRows) : 

  Unsupported type for deserialization: Some message from one of the columns. Calls: <Anonymous> -> lapply -> lapply -> FUN -> db.readTypedVector

Execution halted

    at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:455)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:432)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

Any thoughts on what could be wrong?
Message is short but has ,::, and - character. If I strip message to just 10 characters it works.
I am using Azure databricks.

edited Nov 7 at 18:42

asked Nov 7 at 17:57

Krunal

add a comment |

up vote
0
down vote

favorite

I have sparkDataFrame and I am running a function using gapplyCollect. Very often this fails with below error

Error in handleErrors(returnStatus, conn) : 

  org.apache.spark.SparkException: Job aborted due to stage failure: Task 25 in stage 694.0 failed 4 times, most recent failure: Lost task 25.3 in stage 694.0 (TID 24447, 10.139.64.4, executor 0): org.apache.spark.SparkException: R computation failed with

 Error in db.readTypedVector(con, colType, numRows) : 

  Unsupported type for deserialization: Some message from one of the columns. Calls: <Anonymous> -> lapply -> lapply -> FUN -> db.readTypedVector

Execution halted

    at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:455)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:432)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

Any thoughts on what could be wrong?
Message is short but has ,::, and - character. If I strip message to just 10 characters it works.
I am using Azure databricks.

edited Nov 7 at 18:42

asked Nov 7 at 17:57

Krunal

add a comment |

up vote
0
down vote

favorite

I have sparkDataFrame and I am running a function using gapplyCollect. Very often this fails with below error

Error in handleErrors(returnStatus, conn) : 

  org.apache.spark.SparkException: Job aborted due to stage failure: Task 25 in stage 694.0 failed 4 times, most recent failure: Lost task 25.3 in stage 694.0 (TID 24447, 10.139.64.4, executor 0): org.apache.spark.SparkException: R computation failed with

 Error in db.readTypedVector(con, colType, numRows) : 

  Unsupported type for deserialization: Some message from one of the columns. Calls: <Anonymous> -> lapply -> lapply -> FUN -> db.readTypedVector

Execution halted

    at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:455)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:432)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

Any thoughts on what could be wrong?
Message is short but has ,::, and - character. If I strip message to just 10 characters it works.
I am using Azure databricks.

edited Nov 7 at 18:42

asked Nov 7 at 17:57

Krunal

I have sparkDataFrame and I am running a function using gapplyCollect. Very often this fails with below error

Error in handleErrors(returnStatus, conn) : 

  org.apache.spark.SparkException: Job aborted due to stage failure: Task 25 in stage 694.0 failed 4 times, most recent failure: Lost task 25.3 in stage 694.0 (TID 24447, 10.139.64.4, executor 0): org.apache.spark.SparkException: R computation failed with

 Error in db.readTypedVector(con, colType, numRows) : 

  Unsupported type for deserialization: Some message from one of the columns. Calls: <Anonymous> -> lapply -> lapply -> FUN -> db.readTypedVector

Execution halted

    at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:455)

    at org.apache.spark.sql.execution.FlatMapGroupsInRExec$$anonfun$14.apply(objects.scala:432)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

    at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$25.apply(RDD.scala:842)

Any thoughts on what could be wrong?
Message is short but has ,::, and - character. If I strip message to just 10 characters it works.
I am using Azure databricks.

r apache-spark sparkr databricks

edited Nov 7 at 18:42

asked Nov 7 at 17:57

Krunal

edited Nov 7 at 18:42

asked Nov 7 at 17:57

Krunal

edited Nov 7 at 18:42

asked Nov 7 at 17:57

Krunal

asked Nov 7 at 17:57

Krunal

asked Nov 7 at 17:57

Krunal

add a comment |

active

oldest

votes

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53195110%2fsparkr-error-in-db-readtypedvectorcon-coltype-numrows-unsupported-type-f%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

active

oldest

votes

draft saved

draft discarded

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Wsrtjtyk