Post http://localhost:9093/api/v1/alerts: dial tcp [::1]:9093: connect: connection refused
0
i start prometheus and alertmanager on the same pod in k8s cluster,both container start normally, but i find prometheus can not send post request to alertmanager why?
on prometheus container
level=info ts=2018-11-15T07:02:02.147783133Z caller=main.go:238 msg="Starting Prometheus" version="(version=2.4.3, branch=HEAD, revision=167a4b4e73a8eca8df648d2d2043e21bdb9a7449)"
level=info ts=2018-11-15T07:02:02.147861401Z caller=main.go:239 build_context="(go=go1.11.1, user=root@1e42b46043e9, date=20181004-08:42:02)"
level=info ts=2018-11-15T07:02:02.147879439Z caller=main.go:240 host_details="(Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64 k8s-monitor-6d9cd9cd55-vzlmq (none))"
level=info ts=2018-11-15T07:02:02.147895594Z caller=main.go:241 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2018-11-15T07:02:02.147908172Z caller=main.go:242 vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2018-11-15T07:02:02.151206196Z caller=main.go:554 msg="Starting TSDB ..."
level=info ts=2018-11-15T07:02:02.151347322Z caller=web.go:397 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-11-15T07:02:02.15948818Z caller=main.go:564 msg="TSDB started"
level=info ts=2018-11-15T07:02:02.159530481Z caller=main.go:624 msg="Loading configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.164675684Z caller=main.go:650 msg="Completed loading of configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.16470252Z caller=main.go:523 msg="Server is ready to receive web requests."
level=error ts=2018-11-15T07:02:02.165912877Z caller=notifier.go:473 component=notifier alertmanager=http://localhost:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://localhost:9093/api/v1/alerts: dial tcp [::1]:9093: connect: connection refused"
on alertmanager container
level=info ts=2018-11-15T07:02:02.498442497Z caller=main.go:174 msg="Starting Alertmanager" version="(version=0.15.2, branch=HEAD, revision=d19fae3bae451940b8470abb680cfdd59bfa7cfa)"
level=info ts=2018-11-15T07:02:02.498524847Z caller=main.go:175 build_context="(go=go1.10.3, user=root@3101e5b68a55, date=20180814-10:53:39)"
level=info ts=2018-11-15T07:02:02.505830622Z caller=cluster.go:155 component=cluster msg="setting advertise address explicitly" addr=192.168.216.185 port=9094
level=info ts=2018-11-15T07:02:02.50851545Z caller=main.go:322 msg="Loading configuration file" file=/opt/alertmanager/alertmanager.yml
level=info ts=2018-11-15T07:02:02.508720329Z caller=cluster.go:570 component=cluster msg="Waiting for gossip to settle..." interval=2s
level=info ts=2018-11-15T07:02:02.512111679Z caller=main.go:398 msg=Listening address=:9093
level=info ts=2018-11-15T07:02:04.508881502Z caller=cluster.go:595 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000097885s
level=info ts=2018-11-15T07:02:12.50938298Z caller=cluster.go:587 component=cluster msg="gossip settled; proceeding" elapsed=10.000595281s
in the pod
[root@k8s-monitor-6d9cd9cd55-vzlmq /]# netstat -utlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp6 0 0 :::9090 :::* LISTEN 5/prometheus
tcp6 0 0 :::9093 :::* LISTEN -
tcp6 0 0 :::9094 :::* LISTEN -
udp6 0 0 :::9094 :::* -
prometheus
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
add a comment |
0
i start prometheus and alertmanager on the same pod in k8s cluster,both container start normally, but i find prometheus can not send post request to alertmanager why?
on prometheus container
level=info ts=2018-11-15T07:02:02.147783133Z caller=main.go:238 msg="Starting Prometheus" version="(version=2.4.3, branch=HEAD, revision=167a4b4e73a8eca8df648d2d2043e21bdb9a7449)"
level=info ts=2018-11-15T07:02:02.147861401Z caller=main.go:239 build_context="(go=go1.11.1, user=root@1e42b46043e9, date=20181004-08:42:02)"
level=info ts=2018-11-15T07:02:02.147879439Z caller=main.go:240 host_details="(Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64 k8s-monitor-6d9cd9cd55-vzlmq (none))"
level=info ts=2018-11-15T07:02:02.147895594Z caller=main.go:241 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2018-11-15T07:02:02.147908172Z caller=main.go:242 vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2018-11-15T07:02:02.151206196Z caller=main.go:554 msg="Starting TSDB ..."
level=info ts=2018-11-15T07:02:02.151347322Z caller=web.go:397 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-11-15T07:02:02.15948818Z caller=main.go:564 msg="TSDB started"
level=info ts=2018-11-15T07:02:02.159530481Z caller=main.go:624 msg="Loading configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.164675684Z caller=main.go:650 msg="Completed loading of configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.16470252Z caller=main.go:523 msg="Server is ready to receive web requests."
level=error ts=2018-11-15T07:02:02.165912877Z caller=notifier.go:473 component=notifier alertmanager=http://localhost:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://localhost:9093/api/v1/alerts: dial tcp [::1]:9093: connect: connection refused"
on alertmanager container
level=info ts=2018-11-15T07:02:02.498442497Z caller=main.go:174 msg="Starting Alertmanager" version="(version=0.15.2, branch=HEAD, revision=d19fae3bae451940b8470abb680cfdd59bfa7cfa)"
level=info ts=2018-11-15T07:02:02.498524847Z caller=main.go:175 build_context="(go=go1.10.3, user=root@3101e5b68a55, date=20180814-10:53:39)"
level=info ts=2018-11-15T07:02:02.505830622Z caller=cluster.go:155 component=cluster msg="setting advertise address explicitly" addr=192.168.216.185 port=9094
level=info ts=2018-11-15T07:02:02.50851545Z caller=main.go:322 msg="Loading configuration file" file=/opt/alertmanager/alertmanager.yml
level=info ts=2018-11-15T07:02:02.508720329Z caller=cluster.go:570 component=cluster msg="Waiting for gossip to settle..." interval=2s
level=info ts=2018-11-15T07:02:02.512111679Z caller=main.go:398 msg=Listening address=:9093
level=info ts=2018-11-15T07:02:04.508881502Z caller=cluster.go:595 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000097885s
level=info ts=2018-11-15T07:02:12.50938298Z caller=cluster.go:587 component=cluster msg="gossip settled; proceeding" elapsed=10.000595281s
in the pod
[root@k8s-monitor-6d9cd9cd55-vzlmq /]# netstat -utlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp6 0 0 :::9090 :::* LISTEN 5/prometheus
tcp6 0 0 :::9093 :::* LISTEN -
tcp6 0 0 :::9094 :::* LISTEN -
udp6 0 0 :::9094 :::* -
prometheus
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
add a comment |
0
0
0
i start prometheus and alertmanager on the same pod in k8s cluster,both container start normally, but i find prometheus can not send post request to alertmanager why?
on prometheus container
level=info ts=2018-11-15T07:02:02.147783133Z caller=main.go:238 msg="Starting Prometheus" version="(version=2.4.3, branch=HEAD, revision=167a4b4e73a8eca8df648d2d2043e21bdb9a7449)"
level=info ts=2018-11-15T07:02:02.147861401Z caller=main.go:239 build_context="(go=go1.11.1, user=root@1e42b46043e9, date=20181004-08:42:02)"
level=info ts=2018-11-15T07:02:02.147879439Z caller=main.go:240 host_details="(Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64 k8s-monitor-6d9cd9cd55-vzlmq (none))"
level=info ts=2018-11-15T07:02:02.147895594Z caller=main.go:241 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2018-11-15T07:02:02.147908172Z caller=main.go:242 vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2018-11-15T07:02:02.151206196Z caller=main.go:554 msg="Starting TSDB ..."
level=info ts=2018-11-15T07:02:02.151347322Z caller=web.go:397 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-11-15T07:02:02.15948818Z caller=main.go:564 msg="TSDB started"
level=info ts=2018-11-15T07:02:02.159530481Z caller=main.go:624 msg="Loading configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.164675684Z caller=main.go:650 msg="Completed loading of configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.16470252Z caller=main.go:523 msg="Server is ready to receive web requests."
level=error ts=2018-11-15T07:02:02.165912877Z caller=notifier.go:473 component=notifier alertmanager=http://localhost:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://localhost:9093/api/v1/alerts: dial tcp [::1]:9093: connect: connection refused"
on alertmanager container
level=info ts=2018-11-15T07:02:02.498442497Z caller=main.go:174 msg="Starting Alertmanager" version="(version=0.15.2, branch=HEAD, revision=d19fae3bae451940b8470abb680cfdd59bfa7cfa)"
level=info ts=2018-11-15T07:02:02.498524847Z caller=main.go:175 build_context="(go=go1.10.3, user=root@3101e5b68a55, date=20180814-10:53:39)"
level=info ts=2018-11-15T07:02:02.505830622Z caller=cluster.go:155 component=cluster msg="setting advertise address explicitly" addr=192.168.216.185 port=9094
level=info ts=2018-11-15T07:02:02.50851545Z caller=main.go:322 msg="Loading configuration file" file=/opt/alertmanager/alertmanager.yml
level=info ts=2018-11-15T07:02:02.508720329Z caller=cluster.go:570 component=cluster msg="Waiting for gossip to settle..." interval=2s
level=info ts=2018-11-15T07:02:02.512111679Z caller=main.go:398 msg=Listening address=:9093
level=info ts=2018-11-15T07:02:04.508881502Z caller=cluster.go:595 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000097885s
level=info ts=2018-11-15T07:02:12.50938298Z caller=cluster.go:587 component=cluster msg="gossip settled; proceeding" elapsed=10.000595281s
in the pod
[root@k8s-monitor-6d9cd9cd55-vzlmq /]# netstat -utlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp6 0 0 :::9090 :::* LISTEN 5/prometheus
tcp6 0 0 :::9093 :::* LISTEN -
tcp6 0 0 :::9094 :::* LISTEN -
udp6 0 0 :::9094 :::* -
prometheus
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
i start prometheus and alertmanager on the same pod in k8s cluster,both container start normally, but i find prometheus can not send post request to alertmanager why?
on prometheus container
level=info ts=2018-11-15T07:02:02.147783133Z caller=main.go:238 msg="Starting Prometheus" version="(version=2.4.3, branch=HEAD, revision=167a4b4e73a8eca8df648d2d2043e21bdb9a7449)"
level=info ts=2018-11-15T07:02:02.147861401Z caller=main.go:239 build_context="(go=go1.11.1, user=root@1e42b46043e9, date=20181004-08:42:02)"
level=info ts=2018-11-15T07:02:02.147879439Z caller=main.go:240 host_details="(Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Thu Jan 25 20:13:58 UTC 2018 x86_64 k8s-monitor-6d9cd9cd55-vzlmq (none))"
level=info ts=2018-11-15T07:02:02.147895594Z caller=main.go:241 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2018-11-15T07:02:02.147908172Z caller=main.go:242 vm_limits="(soft=unlimited, hard=unlimited)"
level=info ts=2018-11-15T07:02:02.151206196Z caller=main.go:554 msg="Starting TSDB ..."
level=info ts=2018-11-15T07:02:02.151347322Z caller=web.go:397 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-11-15T07:02:02.15948818Z caller=main.go:564 msg="TSDB started"
level=info ts=2018-11-15T07:02:02.159530481Z caller=main.go:624 msg="Loading configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.164675684Z caller=main.go:650 msg="Completed loading of configuration file" filename=/opt/prometheus-2.4.3.linux-amd64/prometheus.yml
level=info ts=2018-11-15T07:02:02.16470252Z caller=main.go:523 msg="Server is ready to receive web requests."
level=error ts=2018-11-15T07:02:02.165912877Z caller=notifier.go:473 component=notifier alertmanager=http://localhost:9093/api/v1/alerts count=0 msg="Error sending alert" err="Post http://localhost:9093/api/v1/alerts: dial tcp [::1]:9093: connect: connection refused"
on alertmanager container
level=info ts=2018-11-15T07:02:02.498442497Z caller=main.go:174 msg="Starting Alertmanager" version="(version=0.15.2, branch=HEAD, revision=d19fae3bae451940b8470abb680cfdd59bfa7cfa)"
level=info ts=2018-11-15T07:02:02.498524847Z caller=main.go:175 build_context="(go=go1.10.3, user=root@3101e5b68a55, date=20180814-10:53:39)"
level=info ts=2018-11-15T07:02:02.505830622Z caller=cluster.go:155 component=cluster msg="setting advertise address explicitly" addr=192.168.216.185 port=9094
level=info ts=2018-11-15T07:02:02.50851545Z caller=main.go:322 msg="Loading configuration file" file=/opt/alertmanager/alertmanager.yml
level=info ts=2018-11-15T07:02:02.508720329Z caller=cluster.go:570 component=cluster msg="Waiting for gossip to settle..." interval=2s
level=info ts=2018-11-15T07:02:02.512111679Z caller=main.go:398 msg=Listening address=:9093
level=info ts=2018-11-15T07:02:04.508881502Z caller=cluster.go:595 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000097885s
level=info ts=2018-11-15T07:02:12.50938298Z caller=cluster.go:587 component=cluster msg="gossip settled; proceeding" elapsed=10.000595281s
in the pod
[root@k8s-monitor-6d9cd9cd55-vzlmq /]# netstat -utlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp6 0 0 :::9090 :::* LISTEN 5/prometheus
tcp6 0 0 :::9093 :::* LISTEN -
tcp6 0 0 :::9094 :::* LISTEN -
udp6 0 0 :::9094 :::* -
prometheus
prometheus
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
asked Nov 15 '18 at 7:50
user1208081user1208081
1742318
1742318
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53314642%2fpost-http-localhost9093-api-v1-alerts-dial-tcp-19093-connect-connecti%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53314642%2fpost-http-localhost9093-api-v1-alerts-dial-tcp-19093-connect-connecti%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
