Selenium lagging when opening more sessions
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
add a comment |
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
Have you tried addingchrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')
– Moshe Slavin
Nov 22 '18 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 '18 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 '18 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 '18 at 11:05
add a comment |
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
I am using VPS with 8 CPU cores.
Problem started when I opened around 30 sessions of Chrome web driver. I minimized all browsers to reduce CPU usage and I was tracking tests inside my prompt.
Usually 1 test lasts 55 seconds, but when I have more sessions they vary from 1 minute to 3 minutes sometimes. What is cause of the problem here? My CPU usage is around 20 percent of all time.
python selenium web selenium-webdriver driver
python selenium web selenium-webdriver driver
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
asked Nov 22 '18 at 9:38
Mark LettermanMark Letterman
161
161
Have you tried addingchrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')
– Moshe Slavin
Nov 22 '18 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 '18 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 '18 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 '18 at 11:05
add a comment |
Have you tried addingchrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')
– Moshe Slavin
Nov 22 '18 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 '18 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 '18 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 '18 at 11:05
Have you tried adding
chrome_options? you can use '--headless' like this chrome_options.add_argument('--headless')– Moshe Slavin
Nov 22 '18 at 10:52
Have you tried adding
chrome_options? you can use '--headless' like this chrome_options.add_argument('--headless')– Moshe Slavin
Nov 22 '18 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 '18 at 10:57
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 '18 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 '18 at 10:59
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 '18 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 '18 at 11:05
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 '18 at 11:05
add a comment |
1 Answer
1
active
oldest
votes
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 '18 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 '18 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 '18 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 '18 at 20:11
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53427858%2fselenium-lagging-when-opening-more-sessions%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 '18 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 '18 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 '18 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 '18 at 20:11
add a comment |
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 '18 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 '18 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 '18 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 '18 at 20:11
add a comment |
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
The cause of the problem most probably is RAM usage. For browser execution RAM is crucial and when you got to the limit, the tests start to perform slower. Check out the usage of RAM and most likely you will see that 30 instances of a browser use it all.
As a suggestion play around the number of browsers and try to keep an optimal percentage of RAM usage. In your case maybe limiting the number of parallel threads to 25 or even 20 will make entire suite run faster than in 30. But of course, if there is an option it's better to just add more memory and proceed with fast automation ;)
some more details about FF and Chrome RAM usage
Edit after a comment that Ram is only used by 15%
It could simply be an app, that slows down under load. Yep, 30 threads do not look anyhow valuable but it may depend on the app-environment hardware or just that some part of an app's performance has an issue. We had a case when our tests were working fine on one environment in 200 threads, but when data center was migrating, we switched to different environments (that were configured differently) and we found that at the beginning the new environment only could normally work under 30-50 threads (comparing to usual 200). We reported it as an issue and it was confirmed and solved by IT team.
As a suggestion for this case - try to monitor when does the slowdown happen, is there any patterns for example - 'after 10 minutes of test suite run' or 'after some action on the app being called'. This info may help to investigate the potential problem on an app side.
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
edited Nov 23 '18 at 11:06
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
answered Nov 22 '18 at 15:43
Vladimir EfimovVladimir Efimov
700513
700513
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 '18 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 '18 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 '18 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 '18 at 20:11
add a comment |
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 '18 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 '18 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 '18 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 '18 at 20:11
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 '18 at 22:47
I have 40gb of ram, and when i run tests, only 15% is used.
– Mark Letterman
Nov 22 '18 at 22:47
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 '18 at 5:24
@MarkLetterman updated my answer with a second suggestion as the RAM does not seem to be the case
– Vladimir Efimov
Nov 23 '18 at 5:24
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 '18 at 21:00
I noticed what is the problem that is causing lag. It is .sendkeys() command when i try to write something by finding element. This function is very slow on Chrome when is minimized. I wonder how can i speed it up?
– Mark Letterman
Nov 24 '18 at 21:00
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 '18 at 20:11
@MarkLetterman the only thing I can think of is replacing sendKeys method call with javascript. it should look something like WebElement el = ........; ((JavascriptExecutor) driver).executeScript("arguments[0].value='text you want to enter'", el)
– Vladimir Efimov
Nov 25 '18 at 20:11
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53427858%2fselenium-lagging-when-opening-more-sessions%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Have you tried adding
chrome_options? you can use'--headless'like thischrome_options.add_argument('--headless')– Moshe Slavin
Nov 22 '18 at 10:52
I did, tests work great then but the problem is CPU usage. I can't handle more then 5 headless chromes and my VPS is pretty strong.
– Mark Letterman
Nov 22 '18 at 10:57
Do all the 30 sessions need to run in parallel?
– Moshe Slavin
Nov 22 '18 at 10:59
Yes they do. It is crucial for me that tests run in period of 55 seconds - 1 minute max.
– Mark Letterman
Nov 22 '18 at 11:05