10k Concurrent connections with jmeter












1















I have a 32GB, i7 core processor running on windows 10 and I am trying to generate 10kVU concurrent load via jmeter. For some reason I am unable to go beyond 1k concurrent and I start getting BindException error or Socket connection error. Can someone help me with the settings to achieve that kind of load? Also if someone is up for freelancing I am happy to consider that as well. Any help would be great as I am nearing production and am unable to load test this use case. If you guys have any other tools that I can use effectively, that would also help.










share|improve this question



























    1















    I have a 32GB, i7 core processor running on windows 10 and I am trying to generate 10kVU concurrent load via jmeter. For some reason I am unable to go beyond 1k concurrent and I start getting BindException error or Socket connection error. Can someone help me with the settings to achieve that kind of load? Also if someone is up for freelancing I am happy to consider that as well. Any help would be great as I am nearing production and am unable to load test this use case. If you guys have any other tools that I can use effectively, that would also help.










    share|improve this question

























      1












      1








      1








      I have a 32GB, i7 core processor running on windows 10 and I am trying to generate 10kVU concurrent load via jmeter. For some reason I am unable to go beyond 1k concurrent and I start getting BindException error or Socket connection error. Can someone help me with the settings to achieve that kind of load? Also if someone is up for freelancing I am happy to consider that as well. Any help would be great as I am nearing production and am unable to load test this use case. If you guys have any other tools that I can use effectively, that would also help.










      share|improve this question














      I have a 32GB, i7 core processor running on windows 10 and I am trying to generate 10kVU concurrent load via jmeter. For some reason I am unable to go beyond 1k concurrent and I start getting BindException error or Socket connection error. Can someone help me with the settings to achieve that kind of load? Also if someone is up for freelancing I am happy to consider that as well. Any help would be great as I am nearing production and am unable to load test this use case. If you guys have any other tools that I can use effectively, that would also help.







      jmeter performance-testing load-testing






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 18 '18 at 7:51









      nikhil umraliyanikhil umraliya

      65




      65
























          2 Answers
          2






          active

          oldest

          votes


















          0














          You reach the limit of 1 computer, thus you must execute in distributed environment of multiple computers.



          You can setup JMeter's distributed testing on your own environment or use blazemeter or other cloud based load testing tool




          we can use BlazeMeter, which provides us with an easy way to handle our load tests. All we need to do is to upload our JMX file to BlazeMeter. We can also upload a consolidated CSV file with all the necessary data and BlazeMeter will take care of splitting it, depending on the amount of engines we have set.



          On BlazeMeter we can set the amount of users or the combination of engines (slave systems) and threads that we want to apply to our tests. We can also configure additional values like multi locations.







          share|improve this answer
























          • Blazemeter is quite expensive. I have considered that as a last resort. Need to try every possible option before that

            – nikhil umraliya
            Nov 19 '18 at 5:59











          • @nikhilumraliya Other options are Loadium,flood.io, RedLine13,..

            – user7294900
            Nov 19 '18 at 6:04





















          0














          1k concurrent sounds low enough that it's something else ... it's also the default amount of open file descriptor limits on a lot of Linux distributions so maybe try to raise the limit.



          ulimit -Sn  


          will show you your current limit and



          ulimit -Hn


          will show you the hard limit you can go before you have to touch configuration files. Editing /etc/security/limits.conf as root and setting something like



          yourusername soft nofile 50000
          yourusername hard nofile 50000


          yourusername - will have to be the username of the user which with you run jmeter.



          After this you will probably have to restart in order for the changes to take effect. If not on Linux I don't know how to actually do this you will have to google :D



          Recommendation:
          As a k6 developer I can propose it as an alternative tool, but running 10k VUs on a single machine will be hard with it as well. Every VU will take some memory - like at least 1-3mb and this will go up the larger your script is. But with 32gb you could still run upto 1-2kVUs and use http.batch to make concurrent requests which might simulate the 10k VUs depending on what your actual workflow is like.



          I managed to run the stages sample with 300VUs on a single 3770 i7 CPU and 4gb of ram in a virtual machine and got 6.5k+ rps to another virtual machine on a neighboring physical machine (the latency is very low) so maybe 1.5-2kVUs with a a somewhat more interesting script and some higher latency as this will give time to the golang to actually run GC while waiting for tcp packets. I highly recommend using discardResponseBodies if you don't need them and even if you need some to actually get the response for them. This helps a lot with the memory consumption a each VU






          share|improve this answer

























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53358888%2f10k-concurrent-connections-with-jmeter%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            You reach the limit of 1 computer, thus you must execute in distributed environment of multiple computers.



            You can setup JMeter's distributed testing on your own environment or use blazemeter or other cloud based load testing tool




            we can use BlazeMeter, which provides us with an easy way to handle our load tests. All we need to do is to upload our JMX file to BlazeMeter. We can also upload a consolidated CSV file with all the necessary data and BlazeMeter will take care of splitting it, depending on the amount of engines we have set.



            On BlazeMeter we can set the amount of users or the combination of engines (slave systems) and threads that we want to apply to our tests. We can also configure additional values like multi locations.







            share|improve this answer
























            • Blazemeter is quite expensive. I have considered that as a last resort. Need to try every possible option before that

              – nikhil umraliya
              Nov 19 '18 at 5:59











            • @nikhilumraliya Other options are Loadium,flood.io, RedLine13,..

              – user7294900
              Nov 19 '18 at 6:04


















            0














            You reach the limit of 1 computer, thus you must execute in distributed environment of multiple computers.



            You can setup JMeter's distributed testing on your own environment or use blazemeter or other cloud based load testing tool




            we can use BlazeMeter, which provides us with an easy way to handle our load tests. All we need to do is to upload our JMX file to BlazeMeter. We can also upload a consolidated CSV file with all the necessary data and BlazeMeter will take care of splitting it, depending on the amount of engines we have set.



            On BlazeMeter we can set the amount of users or the combination of engines (slave systems) and threads that we want to apply to our tests. We can also configure additional values like multi locations.







            share|improve this answer
























            • Blazemeter is quite expensive. I have considered that as a last resort. Need to try every possible option before that

              – nikhil umraliya
              Nov 19 '18 at 5:59











            • @nikhilumraliya Other options are Loadium,flood.io, RedLine13,..

              – user7294900
              Nov 19 '18 at 6:04
















            0












            0








            0







            You reach the limit of 1 computer, thus you must execute in distributed environment of multiple computers.



            You can setup JMeter's distributed testing on your own environment or use blazemeter or other cloud based load testing tool




            we can use BlazeMeter, which provides us with an easy way to handle our load tests. All we need to do is to upload our JMX file to BlazeMeter. We can also upload a consolidated CSV file with all the necessary data and BlazeMeter will take care of splitting it, depending on the amount of engines we have set.



            On BlazeMeter we can set the amount of users or the combination of engines (slave systems) and threads that we want to apply to our tests. We can also configure additional values like multi locations.







            share|improve this answer













            You reach the limit of 1 computer, thus you must execute in distributed environment of multiple computers.



            You can setup JMeter's distributed testing on your own environment or use blazemeter or other cloud based load testing tool




            we can use BlazeMeter, which provides us with an easy way to handle our load tests. All we need to do is to upload our JMX file to BlazeMeter. We can also upload a consolidated CSV file with all the necessary data and BlazeMeter will take care of splitting it, depending on the amount of engines we have set.



            On BlazeMeter we can set the amount of users or the combination of engines (slave systems) and threads that we want to apply to our tests. We can also configure additional values like multi locations.








            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Nov 18 '18 at 9:13









            user7294900user7294900

            22.1k113259




            22.1k113259













            • Blazemeter is quite expensive. I have considered that as a last resort. Need to try every possible option before that

              – nikhil umraliya
              Nov 19 '18 at 5:59











            • @nikhilumraliya Other options are Loadium,flood.io, RedLine13,..

              – user7294900
              Nov 19 '18 at 6:04





















            • Blazemeter is quite expensive. I have considered that as a last resort. Need to try every possible option before that

              – nikhil umraliya
              Nov 19 '18 at 5:59











            • @nikhilumraliya Other options are Loadium,flood.io, RedLine13,..

              – user7294900
              Nov 19 '18 at 6:04



















            Blazemeter is quite expensive. I have considered that as a last resort. Need to try every possible option before that

            – nikhil umraliya
            Nov 19 '18 at 5:59





            Blazemeter is quite expensive. I have considered that as a last resort. Need to try every possible option before that

            – nikhil umraliya
            Nov 19 '18 at 5:59













            @nikhilumraliya Other options are Loadium,flood.io, RedLine13,..

            – user7294900
            Nov 19 '18 at 6:04







            @nikhilumraliya Other options are Loadium,flood.io, RedLine13,..

            – user7294900
            Nov 19 '18 at 6:04















            0














            1k concurrent sounds low enough that it's something else ... it's also the default amount of open file descriptor limits on a lot of Linux distributions so maybe try to raise the limit.



            ulimit -Sn  


            will show you your current limit and



            ulimit -Hn


            will show you the hard limit you can go before you have to touch configuration files. Editing /etc/security/limits.conf as root and setting something like



            yourusername soft nofile 50000
            yourusername hard nofile 50000


            yourusername - will have to be the username of the user which with you run jmeter.



            After this you will probably have to restart in order for the changes to take effect. If not on Linux I don't know how to actually do this you will have to google :D



            Recommendation:
            As a k6 developer I can propose it as an alternative tool, but running 10k VUs on a single machine will be hard with it as well. Every VU will take some memory - like at least 1-3mb and this will go up the larger your script is. But with 32gb you could still run upto 1-2kVUs and use http.batch to make concurrent requests which might simulate the 10k VUs depending on what your actual workflow is like.



            I managed to run the stages sample with 300VUs on a single 3770 i7 CPU and 4gb of ram in a virtual machine and got 6.5k+ rps to another virtual machine on a neighboring physical machine (the latency is very low) so maybe 1.5-2kVUs with a a somewhat more interesting script and some higher latency as this will give time to the golang to actually run GC while waiting for tcp packets. I highly recommend using discardResponseBodies if you don't need them and even if you need some to actually get the response for them. This helps a lot with the memory consumption a each VU






            share|improve this answer






























              0














              1k concurrent sounds low enough that it's something else ... it's also the default amount of open file descriptor limits on a lot of Linux distributions so maybe try to raise the limit.



              ulimit -Sn  


              will show you your current limit and



              ulimit -Hn


              will show you the hard limit you can go before you have to touch configuration files. Editing /etc/security/limits.conf as root and setting something like



              yourusername soft nofile 50000
              yourusername hard nofile 50000


              yourusername - will have to be the username of the user which with you run jmeter.



              After this you will probably have to restart in order for the changes to take effect. If not on Linux I don't know how to actually do this you will have to google :D



              Recommendation:
              As a k6 developer I can propose it as an alternative tool, but running 10k VUs on a single machine will be hard with it as well. Every VU will take some memory - like at least 1-3mb and this will go up the larger your script is. But with 32gb you could still run upto 1-2kVUs and use http.batch to make concurrent requests which might simulate the 10k VUs depending on what your actual workflow is like.



              I managed to run the stages sample with 300VUs on a single 3770 i7 CPU and 4gb of ram in a virtual machine and got 6.5k+ rps to another virtual machine on a neighboring physical machine (the latency is very low) so maybe 1.5-2kVUs with a a somewhat more interesting script and some higher latency as this will give time to the golang to actually run GC while waiting for tcp packets. I highly recommend using discardResponseBodies if you don't need them and even if you need some to actually get the response for them. This helps a lot with the memory consumption a each VU






              share|improve this answer




























                0












                0








                0







                1k concurrent sounds low enough that it's something else ... it's also the default amount of open file descriptor limits on a lot of Linux distributions so maybe try to raise the limit.



                ulimit -Sn  


                will show you your current limit and



                ulimit -Hn


                will show you the hard limit you can go before you have to touch configuration files. Editing /etc/security/limits.conf as root and setting something like



                yourusername soft nofile 50000
                yourusername hard nofile 50000


                yourusername - will have to be the username of the user which with you run jmeter.



                After this you will probably have to restart in order for the changes to take effect. If not on Linux I don't know how to actually do this you will have to google :D



                Recommendation:
                As a k6 developer I can propose it as an alternative tool, but running 10k VUs on a single machine will be hard with it as well. Every VU will take some memory - like at least 1-3mb and this will go up the larger your script is. But with 32gb you could still run upto 1-2kVUs and use http.batch to make concurrent requests which might simulate the 10k VUs depending on what your actual workflow is like.



                I managed to run the stages sample with 300VUs on a single 3770 i7 CPU and 4gb of ram in a virtual machine and got 6.5k+ rps to another virtual machine on a neighboring physical machine (the latency is very low) so maybe 1.5-2kVUs with a a somewhat more interesting script and some higher latency as this will give time to the golang to actually run GC while waiting for tcp packets. I highly recommend using discardResponseBodies if you don't need them and even if you need some to actually get the response for them. This helps a lot with the memory consumption a each VU






                share|improve this answer















                1k concurrent sounds low enough that it's something else ... it's also the default amount of open file descriptor limits on a lot of Linux distributions so maybe try to raise the limit.



                ulimit -Sn  


                will show you your current limit and



                ulimit -Hn


                will show you the hard limit you can go before you have to touch configuration files. Editing /etc/security/limits.conf as root and setting something like



                yourusername soft nofile 50000
                yourusername hard nofile 50000


                yourusername - will have to be the username of the user which with you run jmeter.



                After this you will probably have to restart in order for the changes to take effect. If not on Linux I don't know how to actually do this you will have to google :D



                Recommendation:
                As a k6 developer I can propose it as an alternative tool, but running 10k VUs on a single machine will be hard with it as well. Every VU will take some memory - like at least 1-3mb and this will go up the larger your script is. But with 32gb you could still run upto 1-2kVUs and use http.batch to make concurrent requests which might simulate the 10k VUs depending on what your actual workflow is like.



                I managed to run the stages sample with 300VUs on a single 3770 i7 CPU and 4gb of ram in a virtual machine and got 6.5k+ rps to another virtual machine on a neighboring physical machine (the latency is very low) so maybe 1.5-2kVUs with a a somewhat more interesting script and some higher latency as this will give time to the golang to actually run GC while waiting for tcp packets. I highly recommend using discardResponseBodies if you don't need them and even if you need some to actually get the response for them. This helps a lot with the memory consumption a each VU







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Nov 22 '18 at 14:03

























                answered Nov 22 '18 at 13:37









                Михаил СтойковМихаил Стойков

                4115




                4115






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53358888%2f10k-concurrent-connections-with-jmeter%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    這個網誌中的熱門文章

                    Tangent Lines Diagram Along Smooth Curve

                    Yusuf al-Mu'taman ibn Hud

                    Zucchini