Terminating docker container on SIGTERM on warping script











up vote
0
down vote

favorite












I have the following Python script that warps a docker container run:



import subprocess
import sys
import signal

container_id = None

def catch_term(signum, frame):
sys.stderr.write('Caught SIGTERM. Stopping containern')
if container_id != None:
cmd = ['docker', 'rm', '-f', container_id]
subprocess.call(cmd)
sys.exit(143)

signal.signal(signal.SIGTERM, catch_term)
cmd = ['docker', 'run', '--cidfile', cidfile, '-i', image_name, container_process_cmd]
p = subprocess.Popen(cmd)
# This function waits for the file to contain 64 bytes and then reads the container id from it
container_id = get_container_id_from_cidfile(cidfile)
p.communicate()
sys.exit(p.returncode)


When I run the Python process on interactive mode, and kill that warping script during the run (kill <PID>), the catch_term function is called and the container is removed. Then my process exits.



However, when I run the warping process on the background, i.e. my_warping_script_above.py &, and kill it in the same way, the catch_term function is not called at all and my script keeps running, including the container.




  • Any idea how to make the catch_term function be called even when the script runs on background?


  • I think it has something to do with the fact that docker run is a "protected" process. It should only be able to get killed using a docker command, such as docker rm -f <CONTAINER ID>, or using Ctrl+C if called with -i. That's the reason why I'm trying to catch the TERM signal and call docker rm, but my function is not called when the script runs on background... (I think it tries to first kill the child process, then as it fails, it does not reach to stage of calling my registered function).











share|improve this question




























    up vote
    0
    down vote

    favorite












    I have the following Python script that warps a docker container run:



    import subprocess
    import sys
    import signal

    container_id = None

    def catch_term(signum, frame):
    sys.stderr.write('Caught SIGTERM. Stopping containern')
    if container_id != None:
    cmd = ['docker', 'rm', '-f', container_id]
    subprocess.call(cmd)
    sys.exit(143)

    signal.signal(signal.SIGTERM, catch_term)
    cmd = ['docker', 'run', '--cidfile', cidfile, '-i', image_name, container_process_cmd]
    p = subprocess.Popen(cmd)
    # This function waits for the file to contain 64 bytes and then reads the container id from it
    container_id = get_container_id_from_cidfile(cidfile)
    p.communicate()
    sys.exit(p.returncode)


    When I run the Python process on interactive mode, and kill that warping script during the run (kill <PID>), the catch_term function is called and the container is removed. Then my process exits.



    However, when I run the warping process on the background, i.e. my_warping_script_above.py &, and kill it in the same way, the catch_term function is not called at all and my script keeps running, including the container.




    • Any idea how to make the catch_term function be called even when the script runs on background?


    • I think it has something to do with the fact that docker run is a "protected" process. It should only be able to get killed using a docker command, such as docker rm -f <CONTAINER ID>, or using Ctrl+C if called with -i. That's the reason why I'm trying to catch the TERM signal and call docker rm, but my function is not called when the script runs on background... (I think it tries to first kill the child process, then as it fails, it does not reach to stage of calling my registered function).











    share|improve this question


























      up vote
      0
      down vote

      favorite









      up vote
      0
      down vote

      favorite











      I have the following Python script that warps a docker container run:



      import subprocess
      import sys
      import signal

      container_id = None

      def catch_term(signum, frame):
      sys.stderr.write('Caught SIGTERM. Stopping containern')
      if container_id != None:
      cmd = ['docker', 'rm', '-f', container_id]
      subprocess.call(cmd)
      sys.exit(143)

      signal.signal(signal.SIGTERM, catch_term)
      cmd = ['docker', 'run', '--cidfile', cidfile, '-i', image_name, container_process_cmd]
      p = subprocess.Popen(cmd)
      # This function waits for the file to contain 64 bytes and then reads the container id from it
      container_id = get_container_id_from_cidfile(cidfile)
      p.communicate()
      sys.exit(p.returncode)


      When I run the Python process on interactive mode, and kill that warping script during the run (kill <PID>), the catch_term function is called and the container is removed. Then my process exits.



      However, when I run the warping process on the background, i.e. my_warping_script_above.py &, and kill it in the same way, the catch_term function is not called at all and my script keeps running, including the container.




      • Any idea how to make the catch_term function be called even when the script runs on background?


      • I think it has something to do with the fact that docker run is a "protected" process. It should only be able to get killed using a docker command, such as docker rm -f <CONTAINER ID>, or using Ctrl+C if called with -i. That's the reason why I'm trying to catch the TERM signal and call docker rm, but my function is not called when the script runs on background... (I think it tries to first kill the child process, then as it fails, it does not reach to stage of calling my registered function).











      share|improve this question















      I have the following Python script that warps a docker container run:



      import subprocess
      import sys
      import signal

      container_id = None

      def catch_term(signum, frame):
      sys.stderr.write('Caught SIGTERM. Stopping containern')
      if container_id != None:
      cmd = ['docker', 'rm', '-f', container_id]
      subprocess.call(cmd)
      sys.exit(143)

      signal.signal(signal.SIGTERM, catch_term)
      cmd = ['docker', 'run', '--cidfile', cidfile, '-i', image_name, container_process_cmd]
      p = subprocess.Popen(cmd)
      # This function waits for the file to contain 64 bytes and then reads the container id from it
      container_id = get_container_id_from_cidfile(cidfile)
      p.communicate()
      sys.exit(p.returncode)


      When I run the Python process on interactive mode, and kill that warping script during the run (kill <PID>), the catch_term function is called and the container is removed. Then my process exits.



      However, when I run the warping process on the background, i.e. my_warping_script_above.py &, and kill it in the same way, the catch_term function is not called at all and my script keeps running, including the container.




      • Any idea how to make the catch_term function be called even when the script runs on background?


      • I think it has something to do with the fact that docker run is a "protected" process. It should only be able to get killed using a docker command, such as docker rm -f <CONTAINER ID>, or using Ctrl+C if called with -i. That's the reason why I'm trying to catch the TERM signal and call docker rm, but my function is not called when the script runs on background... (I think it tries to first kill the child process, then as it fails, it does not reach to stage of calling my registered function).








      python docker subprocess background-process






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 11 at 16:24

























      asked Nov 7 at 20:47









      SomethingSomething

      4,08173465




      4,08173465
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          0
          down vote



          accepted










          Answering myself:



          This trick solved my problem: adding preexec_fn=os.setsid to the subprocess.Popen function:



          p = subprocess.Popen(cmd, preexec_fn=os.setsid)





          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53197564%2fterminating-docker-container-on-sigterm-on-warping-script%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            0
            down vote



            accepted










            Answering myself:



            This trick solved my problem: adding preexec_fn=os.setsid to the subprocess.Popen function:



            p = subprocess.Popen(cmd, preexec_fn=os.setsid)





            share|improve this answer

























              up vote
              0
              down vote



              accepted










              Answering myself:



              This trick solved my problem: adding preexec_fn=os.setsid to the subprocess.Popen function:



              p = subprocess.Popen(cmd, preexec_fn=os.setsid)





              share|improve this answer























                up vote
                0
                down vote



                accepted







                up vote
                0
                down vote



                accepted






                Answering myself:



                This trick solved my problem: adding preexec_fn=os.setsid to the subprocess.Popen function:



                p = subprocess.Popen(cmd, preexec_fn=os.setsid)





                share|improve this answer












                Answering myself:



                This trick solved my problem: adding preexec_fn=os.setsid to the subprocess.Popen function:



                p = subprocess.Popen(cmd, preexec_fn=os.setsid)






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 7 at 20:55









                SomethingSomething

                4,08173465




                4,08173465






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53197564%2fterminating-docker-container-on-sigterm-on-warping-script%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    這個網誌中的熱門文章

                    Hercules Kyvelos

                    Tangent Lines Diagram Along Smooth Curve

                    Yusuf al-Mu'taman ibn Hud