Remove duplicate values from JSON with jq
1
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
asked Nov 21 '18 at 10:23
DirkDirk
106110
|
show 1 more comment
1
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
asked Nov 21 '18 at 10:23
DirkDirk
106110
You'll want to useunique_by(path_expr)
– Aaron
Nov 21 '18 at 13:24
What result are you expecting?
– peak
Nov 21 '18 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 '18 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 '18 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 '18 at 21:39
|
show 1 more comment
1
1
1
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
asked Nov 21 '18 at 10:23
DirkDirk
106110
i'm merging two json-files with jq
(group_by( [."contraction", "definitions"]) | map((.[0]|del(."definitions" )) + { "definitions": (map(."definitions" )) }))
which leads to the following result:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
},
{
"search": "1",
"replace": "1"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
},
{
"search": "A",
"replace": ""
}
]
}
]
but now i want to get rid of the duplicate entries which have the same string in their search-attribute.
I've tried unique and unique_by filters, but it leads to complie errors.
the result , should be:
[
{
"contraction": "TEST_1",
"definitions": [
{
"search": "1",
"replace": "12"
},
{
"search": "3",
"replace": "4"
},
{
"search": "6",
"replace": "2"
}
]
},
{
"contraction": "TEST_2",
"definitions": [
{
"search": "A",
"replace": "post"
},
{
"search": "B",
"replace": "prae"
}
]
}
]
is this possible with jq? because i'm not trying to filter keys but normal values. any ideas?
The code on JQPlay
unix jq
unix jq
asked Nov 21 '18 at 10:23
DirkDirk
106110
asked Nov 21 '18 at 10:23
DirkDirk
106110
edited Nov 21 '18 at 21:39
Dirk
asked Nov 21 '18 at 10:23
DirkDirk
106110
asked Nov 21 '18 at 10:23
DirkDirk
106110
asked Nov 21 '18 at 10:23
DirkDirk
106110
106110
You'll want to useunique_by(path_expr)
– Aaron
Nov 21 '18 at 13:24
What result are you expecting?
– peak
Nov 21 '18 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 '18 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 '18 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 '18 at 21:39
|
show 1 more comment
You'll want to useunique_by(path_expr)
– Aaron
Nov 21 '18 at 13:24
What result are you expecting?
– peak
Nov 21 '18 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 '18 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 '18 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 '18 at 21:39
You'll want to use
unique_by(path_expr)– Aaron
Nov 21 '18 at 13:24
You'll want to use
unique_by(path_expr)– Aaron
Nov 21 '18 at 13:24
What result are you expecting?
– peak
Nov 21 '18 at 16:20
What result are you expecting?
– peak
Nov 21 '18 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 '18 at 21:30
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 '18 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 '18 at 21:31
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 '18 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 '18 at 21:39
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 '18 at 21:39
|
show 1 more comment
1 Answer
1
active
oldest
votes
1
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 '18 at 21:57
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53409951%2fremove-duplicate-values-from-json-with-jq%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 '18 at 21:57
add a comment |
1
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 '18 at 21:57
add a comment |
1
1
1
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
You could simply extend your filter with this:
map( .definitions |= unique_by(.search) )
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
answered Nov 21 '18 at 21:47
peakpeak
33.4k94160
33.4k94160
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 '18 at 21:57
add a comment |
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 '18 at 21:57
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 '18 at 21:57
wow, thanks a lot. jqplay.org/s/_dv-6iUaku
– Dirk
Nov 21 '18 at 21:57
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53409951%2fremove-duplicate-values-from-json-with-jq%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
You'll want to use
unique_by(path_expr)– Aaron
Nov 21 '18 at 13:24
What result are you expecting?
– peak
Nov 21 '18 at 16:20
@peak thanks for your reply. i've edit my posting and now it shows what i'm expecting.
– Dirk
Nov 21 '18 at 21:30
@Aaron how do you mean it? Like a filter?
– Dirk
Nov 21 '18 at 21:31
Yes, the steps producing the description arrays should be piped into unique_by. If you add your input(s) to your post I'll be able to test it and post an answer.
– Aaron
Nov 21 '18 at 21:39