Assigning values to a column based on relative magnitudes of other columns
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
Please consider the series:
series = pd.Series(np.random.normal(0, 1, 100))
Here's a moving average df that contains moving averages of this series, with each column corresponding to moving averages using one value in window = [2, 5, 10, 15, 20, 25]
ma_df = pd.DataFrame()
for i in window:
ma_df['ma'+ str(i)] = series.rolling(window = i).mean()
df = pd.concat([ma_df, series], axis =1)
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series
95 -0.66 -0.15 0.15 -0.02 -0.09 -0.16 0.11
96 0.28 -0.09 0.11 0.02 -0.05 -0.14 0.46
97 0.76 -0.09 0.28 0.08 -0.04 -0.07 1.06
98 1.10 0.27 0.32 0.19 0.02 -0.01 1.13
99 1.03 0.74 0.39 0.32 0.13 0.01 0.94
Question: I wish to create a column df['relative_positions']
which takes different values depending on the relative magnitude of different columns of df. For example,df['positions']
=0 if the following conditions are met:
df['relative_positions'] [(df.series > df.ma5) & (df.series > df.ma10) & (df.series < df.ma15) & (df.series > df.ma25) & (df.ma10 > df.ma25) & ....] = 0
or df['positions']
= 1 if:
df['relative_positions'] [(df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25) & ....] = 1
or df['positions']
= 2 if:
df['relative_positions'] [(df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5) & ....] = 2
and so on so forth. In each set of conditions, all columns must be compared to all other columns, with both <
and >
signs.
In short, df['relative_positions']
takes different values (don't matter what) depending on the magnitude of the columns relative to one another (larger or smaller).
Issue:
As you can see, writing these conditions by hand is close to impossible. Is there any way I can automatically populate such lists of permutations of conditions?
I'm sorry if the question is vague. Will be happy to clarify anything.
python-3.x pandas
add a comment |
Please consider the series:
series = pd.Series(np.random.normal(0, 1, 100))
Here's a moving average df that contains moving averages of this series, with each column corresponding to moving averages using one value in window = [2, 5, 10, 15, 20, 25]
ma_df = pd.DataFrame()
for i in window:
ma_df['ma'+ str(i)] = series.rolling(window = i).mean()
df = pd.concat([ma_df, series], axis =1)
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series
95 -0.66 -0.15 0.15 -0.02 -0.09 -0.16 0.11
96 0.28 -0.09 0.11 0.02 -0.05 -0.14 0.46
97 0.76 -0.09 0.28 0.08 -0.04 -0.07 1.06
98 1.10 0.27 0.32 0.19 0.02 -0.01 1.13
99 1.03 0.74 0.39 0.32 0.13 0.01 0.94
Question: I wish to create a column df['relative_positions']
which takes different values depending on the relative magnitude of different columns of df. For example,df['positions']
=0 if the following conditions are met:
df['relative_positions'] [(df.series > df.ma5) & (df.series > df.ma10) & (df.series < df.ma15) & (df.series > df.ma25) & (df.ma10 > df.ma25) & ....] = 0
or df['positions']
= 1 if:
df['relative_positions'] [(df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25) & ....] = 1
or df['positions']
= 2 if:
df['relative_positions'] [(df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5) & ....] = 2
and so on so forth. In each set of conditions, all columns must be compared to all other columns, with both <
and >
signs.
In short, df['relative_positions']
takes different values (don't matter what) depending on the magnitude of the columns relative to one another (larger or smaller).
Issue:
As you can see, writing these conditions by hand is close to impossible. Is there any way I can automatically populate such lists of permutations of conditions?
I'm sorry if the question is vague. Will be happy to clarify anything.
python-3.x pandas
add a comment |
Please consider the series:
series = pd.Series(np.random.normal(0, 1, 100))
Here's a moving average df that contains moving averages of this series, with each column corresponding to moving averages using one value in window = [2, 5, 10, 15, 20, 25]
ma_df = pd.DataFrame()
for i in window:
ma_df['ma'+ str(i)] = series.rolling(window = i).mean()
df = pd.concat([ma_df, series], axis =1)
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series
95 -0.66 -0.15 0.15 -0.02 -0.09 -0.16 0.11
96 0.28 -0.09 0.11 0.02 -0.05 -0.14 0.46
97 0.76 -0.09 0.28 0.08 -0.04 -0.07 1.06
98 1.10 0.27 0.32 0.19 0.02 -0.01 1.13
99 1.03 0.74 0.39 0.32 0.13 0.01 0.94
Question: I wish to create a column df['relative_positions']
which takes different values depending on the relative magnitude of different columns of df. For example,df['positions']
=0 if the following conditions are met:
df['relative_positions'] [(df.series > df.ma5) & (df.series > df.ma10) & (df.series < df.ma15) & (df.series > df.ma25) & (df.ma10 > df.ma25) & ....] = 0
or df['positions']
= 1 if:
df['relative_positions'] [(df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25) & ....] = 1
or df['positions']
= 2 if:
df['relative_positions'] [(df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5) & ....] = 2
and so on so forth. In each set of conditions, all columns must be compared to all other columns, with both <
and >
signs.
In short, df['relative_positions']
takes different values (don't matter what) depending on the magnitude of the columns relative to one another (larger or smaller).
Issue:
As you can see, writing these conditions by hand is close to impossible. Is there any way I can automatically populate such lists of permutations of conditions?
I'm sorry if the question is vague. Will be happy to clarify anything.
python-3.x pandas
Please consider the series:
series = pd.Series(np.random.normal(0, 1, 100))
Here's a moving average df that contains moving averages of this series, with each column corresponding to moving averages using one value in window = [2, 5, 10, 15, 20, 25]
ma_df = pd.DataFrame()
for i in window:
ma_df['ma'+ str(i)] = series.rolling(window = i).mean()
df = pd.concat([ma_df, series], axis =1)
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series
95 -0.66 -0.15 0.15 -0.02 -0.09 -0.16 0.11
96 0.28 -0.09 0.11 0.02 -0.05 -0.14 0.46
97 0.76 -0.09 0.28 0.08 -0.04 -0.07 1.06
98 1.10 0.27 0.32 0.19 0.02 -0.01 1.13
99 1.03 0.74 0.39 0.32 0.13 0.01 0.94
Question: I wish to create a column df['relative_positions']
which takes different values depending on the relative magnitude of different columns of df. For example,df['positions']
=0 if the following conditions are met:
df['relative_positions'] [(df.series > df.ma5) & (df.series > df.ma10) & (df.series < df.ma15) & (df.series > df.ma25) & (df.ma10 > df.ma25) & ....] = 0
or df['positions']
= 1 if:
df['relative_positions'] [(df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25) & ....] = 1
or df['positions']
= 2 if:
df['relative_positions'] [(df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5) & ....] = 2
and so on so forth. In each set of conditions, all columns must be compared to all other columns, with both <
and >
signs.
In short, df['relative_positions']
takes different values (don't matter what) depending on the magnitude of the columns relative to one another (larger or smaller).
Issue:
As you can see, writing these conditions by hand is close to impossible. Is there any way I can automatically populate such lists of permutations of conditions?
I'm sorry if the question is vague. Will be happy to clarify anything.
python-3.x pandas
python-3.x pandas
edited Nov 27 '18 at 16:17
Saeed
asked Nov 25 '18 at 3:40
SaeedSaeed
554414
554414
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Your comparisons are yielding nothing, because df.series
is always higher than the rolling means in the corresponding columns. Nevertheless, you might need np.where
to do your comparisons as:
df['relative_positions'] = np.where((df.series > df.ma5) & (df.series > df.ma10) & (**df.series > df.ma15**) & (df.series > df.ma25) & (df.ma10 > df.ma25), 0,
np.where((df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25), 1, np.where((df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5), 2, 'empty')))
For testing purposes I changed (**df.series > df.ma15**)
to larger instead of lower. Otherwise you get empty's only.
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series relative_positions
95 94.5 93.0 90.5 88.0 85.5 83.0 95 0
96 95.5 94.0 91.5 89.0 86.5 84.0 96 0
97 96.5 95.0 92.5 90.0 87.5 85.0 97 0
98 97.5 96.0 93.5 91.0 88.5 86.0 98 0
99 98.5 97.0 94.5 92.0 89.5 87.0 99 0
I'm sorry it was my oversight in reproducing the issue. The reason the comparisons yielded nothing was that the series was all positive increasing numbers. Changing that to some normally-distributed numbers solves the issue. I edited the questions. Anyways, your answer does not address my question. My issue is not usingnp.where
to find where in the data the conditions I typed work. My issue is populating an all-inclusive, permutation-like list of those conditions.
– Saeed
Nov 27 '18 at 16:15
@Saeed then you need to provide an example of the desired result
– Zanshin
Nov 27 '18 at 16:45
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53464458%2fassigning-values-to-a-column-based-on-relative-magnitudes-of-other-columns%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Your comparisons are yielding nothing, because df.series
is always higher than the rolling means in the corresponding columns. Nevertheless, you might need np.where
to do your comparisons as:
df['relative_positions'] = np.where((df.series > df.ma5) & (df.series > df.ma10) & (**df.series > df.ma15**) & (df.series > df.ma25) & (df.ma10 > df.ma25), 0,
np.where((df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25), 1, np.where((df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5), 2, 'empty')))
For testing purposes I changed (**df.series > df.ma15**)
to larger instead of lower. Otherwise you get empty's only.
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series relative_positions
95 94.5 93.0 90.5 88.0 85.5 83.0 95 0
96 95.5 94.0 91.5 89.0 86.5 84.0 96 0
97 96.5 95.0 92.5 90.0 87.5 85.0 97 0
98 97.5 96.0 93.5 91.0 88.5 86.0 98 0
99 98.5 97.0 94.5 92.0 89.5 87.0 99 0
I'm sorry it was my oversight in reproducing the issue. The reason the comparisons yielded nothing was that the series was all positive increasing numbers. Changing that to some normally-distributed numbers solves the issue. I edited the questions. Anyways, your answer does not address my question. My issue is not usingnp.where
to find where in the data the conditions I typed work. My issue is populating an all-inclusive, permutation-like list of those conditions.
– Saeed
Nov 27 '18 at 16:15
@Saeed then you need to provide an example of the desired result
– Zanshin
Nov 27 '18 at 16:45
add a comment |
Your comparisons are yielding nothing, because df.series
is always higher than the rolling means in the corresponding columns. Nevertheless, you might need np.where
to do your comparisons as:
df['relative_positions'] = np.where((df.series > df.ma5) & (df.series > df.ma10) & (**df.series > df.ma15**) & (df.series > df.ma25) & (df.ma10 > df.ma25), 0,
np.where((df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25), 1, np.where((df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5), 2, 'empty')))
For testing purposes I changed (**df.series > df.ma15**)
to larger instead of lower. Otherwise you get empty's only.
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series relative_positions
95 94.5 93.0 90.5 88.0 85.5 83.0 95 0
96 95.5 94.0 91.5 89.0 86.5 84.0 96 0
97 96.5 95.0 92.5 90.0 87.5 85.0 97 0
98 97.5 96.0 93.5 91.0 88.5 86.0 98 0
99 98.5 97.0 94.5 92.0 89.5 87.0 99 0
I'm sorry it was my oversight in reproducing the issue. The reason the comparisons yielded nothing was that the series was all positive increasing numbers. Changing that to some normally-distributed numbers solves the issue. I edited the questions. Anyways, your answer does not address my question. My issue is not usingnp.where
to find where in the data the conditions I typed work. My issue is populating an all-inclusive, permutation-like list of those conditions.
– Saeed
Nov 27 '18 at 16:15
@Saeed then you need to provide an example of the desired result
– Zanshin
Nov 27 '18 at 16:45
add a comment |
Your comparisons are yielding nothing, because df.series
is always higher than the rolling means in the corresponding columns. Nevertheless, you might need np.where
to do your comparisons as:
df['relative_positions'] = np.where((df.series > df.ma5) & (df.series > df.ma10) & (**df.series > df.ma15**) & (df.series > df.ma25) & (df.ma10 > df.ma25), 0,
np.where((df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25), 1, np.where((df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5), 2, 'empty')))
For testing purposes I changed (**df.series > df.ma15**)
to larger instead of lower. Otherwise you get empty's only.
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series relative_positions
95 94.5 93.0 90.5 88.0 85.5 83.0 95 0
96 95.5 94.0 91.5 89.0 86.5 84.0 96 0
97 96.5 95.0 92.5 90.0 87.5 85.0 97 0
98 97.5 96.0 93.5 91.0 88.5 86.0 98 0
99 98.5 97.0 94.5 92.0 89.5 87.0 99 0
Your comparisons are yielding nothing, because df.series
is always higher than the rolling means in the corresponding columns. Nevertheless, you might need np.where
to do your comparisons as:
df['relative_positions'] = np.where((df.series > df.ma5) & (df.series > df.ma10) & (**df.series > df.ma15**) & (df.series > df.ma25) & (df.ma10 > df.ma25), 0,
np.where((df.series < df.ma5) & (df.series< df.ma10) & (df.ma25 < df.ma15) & (df.ma10 < df.ma25) & (df.series < df.ma25), 1, np.where((df.ma20 < df.ma5) & (df.series > df.ma20) & (df.ma20 < df.ma15) & (df.ma25 < df.series) & (df.series < df.ma5), 2, 'empty')))
For testing purposes I changed (**df.series > df.ma15**)
to larger instead of lower. Otherwise you get empty's only.
df.tail()
ma2 ma5 ma10 ma15 ma20 ma25 series relative_positions
95 94.5 93.0 90.5 88.0 85.5 83.0 95 0
96 95.5 94.0 91.5 89.0 86.5 84.0 96 0
97 96.5 95.0 92.5 90.0 87.5 85.0 97 0
98 97.5 96.0 93.5 91.0 88.5 86.0 98 0
99 98.5 97.0 94.5 92.0 89.5 87.0 99 0
answered Nov 27 '18 at 11:38
ZanshinZanshin
7601523
7601523
I'm sorry it was my oversight in reproducing the issue. The reason the comparisons yielded nothing was that the series was all positive increasing numbers. Changing that to some normally-distributed numbers solves the issue. I edited the questions. Anyways, your answer does not address my question. My issue is not usingnp.where
to find where in the data the conditions I typed work. My issue is populating an all-inclusive, permutation-like list of those conditions.
– Saeed
Nov 27 '18 at 16:15
@Saeed then you need to provide an example of the desired result
– Zanshin
Nov 27 '18 at 16:45
add a comment |
I'm sorry it was my oversight in reproducing the issue. The reason the comparisons yielded nothing was that the series was all positive increasing numbers. Changing that to some normally-distributed numbers solves the issue. I edited the questions. Anyways, your answer does not address my question. My issue is not usingnp.where
to find where in the data the conditions I typed work. My issue is populating an all-inclusive, permutation-like list of those conditions.
– Saeed
Nov 27 '18 at 16:15
@Saeed then you need to provide an example of the desired result
– Zanshin
Nov 27 '18 at 16:45
I'm sorry it was my oversight in reproducing the issue. The reason the comparisons yielded nothing was that the series was all positive increasing numbers. Changing that to some normally-distributed numbers solves the issue. I edited the questions. Anyways, your answer does not address my question. My issue is not using
np.where
to find where in the data the conditions I typed work. My issue is populating an all-inclusive, permutation-like list of those conditions.– Saeed
Nov 27 '18 at 16:15
I'm sorry it was my oversight in reproducing the issue. The reason the comparisons yielded nothing was that the series was all positive increasing numbers. Changing that to some normally-distributed numbers solves the issue. I edited the questions. Anyways, your answer does not address my question. My issue is not using
np.where
to find where in the data the conditions I typed work. My issue is populating an all-inclusive, permutation-like list of those conditions.– Saeed
Nov 27 '18 at 16:15
@Saeed then you need to provide an example of the desired result
– Zanshin
Nov 27 '18 at 16:45
@Saeed then you need to provide an example of the desired result
– Zanshin
Nov 27 '18 at 16:45
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53464458%2fassigning-values-to-a-column-based-on-relative-magnitudes-of-other-columns%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown