Jenkins as JobServer on Hadoop EdgeNode











up vote
0
down vote

favorite












I´m not sure that someone can help me but I´ll take a try.



I´m running Jenkins on an Openshift-Cluster to use it for Deployment and as a jobserver for running ETL-Jobs. These jobs are transferring data from flatfiles to databases and from db to db.



Now, I should expand the system to transfer data to a hadoop cluster using MapR.
What I would like to know is, how can I use a new Jenkins-Slave as a jobserver on an EdgeNode from the hadoop-cluster using MapR. Do I need the Jenkins on the EdgeNode or am I able to use MapR from my existing Jenkins-Jobserver?



Mabye, someone is able to help me or has some informations/links how to solve it.



Thx to all....






hadoop jenkins mapr







share|improve this question




























    up vote
    0
    down vote

    favorite












    I´m not sure that someone can help me but I´ll take a try.



    I´m running Jenkins on an Openshift-Cluster to use it for Deployment and as a jobserver for running ETL-Jobs. These jobs are transferring data from flatfiles to databases and from db to db.



    Now, I should expand the system to transfer data to a hadoop cluster using MapR.
    What I would like to know is, how can I use a new Jenkins-Slave as a jobserver on an EdgeNode from the hadoop-cluster using MapR. Do I need the Jenkins on the EdgeNode or am I able to use MapR from my existing Jenkins-Jobserver?



    Mabye, someone is able to help me or has some informations/links how to solve it.



    Thx to all....






    hadoop jenkins mapr







    share|improve this question


























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I´m not sure that someone can help me but I´ll take a try.



      I´m running Jenkins on an Openshift-Cluster to use it for Deployment and as a jobserver for running ETL-Jobs. These jobs are transferring data from flatfiles to databases and from db to db.



      Now, I should expand the system to transfer data to a hadoop cluster using MapR.
      What I would like to know is, how can I use a new Jenkins-Slave as a jobserver on an EdgeNode from the hadoop-cluster using MapR. Do I need the Jenkins on the EdgeNode or am I able to use MapR from my existing Jenkins-Jobserver?



      Mabye, someone is able to help me or has some informations/links how to solve it.



      Thx to all....






      hadoop jenkins mapr







      share|improve this question















      I´m not sure that someone can help me but I´ll take a try.



      I´m running Jenkins on an Openshift-Cluster to use it for Deployment and as a jobserver for running ETL-Jobs. These jobs are transferring data from flatfiles to databases and from db to db.



      Now, I should expand the system to transfer data to a hadoop cluster using MapR.
      What I would like to know is, how can I use a new Jenkins-Slave as a jobserver on an EdgeNode from the hadoop-cluster using MapR. Do I need the Jenkins on the EdgeNode or am I able to use MapR from my existing Jenkins-Jobserver?



      Mabye, someone is able to help me or has some informations/links how to solve it.



      Thx to all....






      hadoop jenkins mapr




      hadoop jenkins mapr






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 7 at 15:33









      cricket_007

      76.5k1042106




      76.5k1042106










      asked Nov 7 at 10:18









      Alex

      61




      61
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          0
          down vote













          "Use MapR" isn't quite clear to me because I just view it as Hadoop at the end of the day, but you can effectively make your Jenkins slave an "edge node" by installing only the Hadoop Java (maybe also MapR) client utilities plus any XML configuration files from the other edge nodes that define how to communicate with the cluster.



          Then, Jenkins would be able to run sh("hadoop jar app.jar"), for example



          If you're using Openshift, you might also try putting a Hadoop client inside a Docker image that could run in Jenkins, or anywhere else






          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53187451%2fjenkins-as-jobserver-on-hadoop-edgenode%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            0
            down vote













            "Use MapR" isn't quite clear to me because I just view it as Hadoop at the end of the day, but you can effectively make your Jenkins slave an "edge node" by installing only the Hadoop Java (maybe also MapR) client utilities plus any XML configuration files from the other edge nodes that define how to communicate with the cluster.



            Then, Jenkins would be able to run sh("hadoop jar app.jar"), for example



            If you're using Openshift, you might also try putting a Hadoop client inside a Docker image that could run in Jenkins, or anywhere else






            share|improve this answer

























              up vote
              0
              down vote













              "Use MapR" isn't quite clear to me because I just view it as Hadoop at the end of the day, but you can effectively make your Jenkins slave an "edge node" by installing only the Hadoop Java (maybe also MapR) client utilities plus any XML configuration files from the other edge nodes that define how to communicate with the cluster.



              Then, Jenkins would be able to run sh("hadoop jar app.jar"), for example



              If you're using Openshift, you might also try putting a Hadoop client inside a Docker image that could run in Jenkins, or anywhere else






              share|improve this answer























                up vote
                0
                down vote










                up vote
                0
                down vote









                "Use MapR" isn't quite clear to me because I just view it as Hadoop at the end of the day, but you can effectively make your Jenkins slave an "edge node" by installing only the Hadoop Java (maybe also MapR) client utilities plus any XML configuration files from the other edge nodes that define how to communicate with the cluster.



                Then, Jenkins would be able to run sh("hadoop jar app.jar"), for example



                If you're using Openshift, you might also try putting a Hadoop client inside a Docker image that could run in Jenkins, or anywhere else






                share|improve this answer












                "Use MapR" isn't quite clear to me because I just view it as Hadoop at the end of the day, but you can effectively make your Jenkins slave an "edge node" by installing only the Hadoop Java (maybe also MapR) client utilities plus any XML configuration files from the other edge nodes that define how to communicate with the cluster.



                Then, Jenkins would be able to run sh("hadoop jar app.jar"), for example



                If you're using Openshift, you might also try putting a Hadoop client inside a Docker image that could run in Jenkins, or anywhere else







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 7 at 15:32









                cricket_007

                76.5k1042106




                76.5k1042106






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53187451%2fjenkins-as-jobserver-on-hadoop-edgenode%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    這個網誌中的熱門文章

                    Xamarin.form Move up view when keyboard appear

                    圖片
                    .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 0 I'm trying to build a Chat app UI, the idea of the Layout was pretty simple: When the input bar is focused, keyboard show up and "push" up the chat bar, as it's a grid, the ListView will resize to fit the screen: I update the input bar's margin to "push" it up: NSValue result = (NSValue)args.Notification.UserInfo.ObjectForKey(new NSString(UIKeyboard.FrameEndUserInfoKey)); CGSize keyboardSize = result.RectangleFValue.Size; if (Element != null){ Element.Margin = new Thickness(0, 0, 0,keyboardSize.Height); //push the entry up to keyboard height when keyboard is activated } And this is ...
                    閱讀完整內容

                    Post-Redirect-Get with Spring WebFlux and Thymeleaf

                    圖片
                    .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 2 Can someone explain to me how to implement the post-redirect-get pattern in Spring WebFlux and Thymeleaf? What subscribes on the database save method? @GetMapping("/register") public String showRegisterForm(Model model) { model.addAttribute("user", new User()); return "register"; } @PostMapping public String processRegisterForm(@Valid User user, BindingResult bindingResult) { if (bindingResult.hasErrors()) { return "register"; } else { userRepository.save(user); //what subscribes on this? //how to redirect on e.g. "/login"? ...
                    閱讀完整內容

                    Anylogic : not able to use stopDelay()

                    圖片
                    -2 I am building a railway station model. I want a train to depart when I click a button. I want to call stopDelay() for that but I am not sure how to do that and from which place. anylogic share | improve this question asked Nov 23 '18 at 10:12 Anupam Singh Anupam Singh 1 Hi Anupam, welcome to SOF. Please check how to ask good questions, see link below. Your question is...
                    閱讀完整內容