How to use the requests module to skip connection timeout urls











up vote
2
down vote

favorite












Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



def req():

with open('demofile.txt','r') as http:

    for url in http.readlines():

        req = url.strip()

        print(req)

        page=requests.get("http://"+req,verify=False)

        if page.status_code == 400:

            break

        else:

            continue

        time.sleep(1)





python python-3.x python-requests







share|improve this question


























    up vote
    2
    down vote

    favorite












    Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



    def req():
    
with open('demofile.txt','r') as http:
    
    for url in http.readlines():
    
        req = url.strip()
    
        print(req)
    
        page=requests.get("http://"+req,verify=False)
    
        if page.status_code == 400:
    
            break
    
        else:
    
            continue
    
        time.sleep(1)





    python python-3.x python-requests







    share|improve this question
























      up vote
      2
      down vote

      favorite









      up vote
      2
      down vote

      favorite











      Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



      def req():
      
with open('demofile.txt','r') as http:
      
    for url in http.readlines():
      
        req = url.strip()
      
        print(req)
      
        page=requests.get("http://"+req,verify=False)
      
        if page.status_code == 400:
      
            break
      
        else:
      
            continue
      
        time.sleep(1)





      python python-3.x python-requests







      share|improve this question













      Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



      def req():
      
with open('demofile.txt','r') as http:
      
    for url in http.readlines():
      
        req = url.strip()
      
        print(req)
      
        page=requests.get("http://"+req,verify=False)
      
        if page.status_code == 400:
      
            break
      
        else:
      
            continue
      
        time.sleep(1)





      python python-3.x python-requests




      python python-3.x python-requests






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 2 days ago









      Rahul

      243




      243
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          2
          down vote



          accepted










          You can raise exception if there is a timeout and continue using finally for next request,



          import requests
          
import logging
          

          
timeout = 0.00001
          

          
try:
          
    response = requests.get(url="https://google.com", timeout=timeout)
          
except requests.exceptions.ConnectTimeout as e:
          
    logging.error("Time out!")
          
finally:
          
    # continue request here
          
    print("hello")
          

          

          
# output,
          
ERROR:root:Time out!
          
hello





          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53139649%2fhow-to-use-the-requests-module-to-skip-connection-timeout-urls%23new-answer', 'question_page');
            }
            );

            Post as a guest


















            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            2
            down vote



            accepted










            You can raise exception if there is a timeout and continue using finally for next request,



            import requests
            
import logging
            

            
timeout = 0.00001
            

            
try:
            
    response = requests.get(url="https://google.com", timeout=timeout)
            
except requests.exceptions.ConnectTimeout as e:
            
    logging.error("Time out!")
            
finally:
            
    # continue request here
            
    print("hello")
            

            

            
# output,
            
ERROR:root:Time out!
            
hello





            share|improve this answer



























              up vote
              2
              down vote



              accepted










              You can raise exception if there is a timeout and continue using finally for next request,



              import requests
              
import logging
              

              
timeout = 0.00001
              

              
try:
              
    response = requests.get(url="https://google.com", timeout=timeout)
              
except requests.exceptions.ConnectTimeout as e:
              
    logging.error("Time out!")
              
finally:
              
    # continue request here
              
    print("hello")
              

              

              
# output,
              
ERROR:root:Time out!
              
hello





              share|improve this answer

























                up vote
                2
                down vote



                accepted







                up vote
                2
                down vote



                accepted






                You can raise exception if there is a timeout and continue using finally for next request,



                import requests
                
import logging
                

                
timeout = 0.00001
                

                
try:
                
    response = requests.get(url="https://google.com", timeout=timeout)
                
except requests.exceptions.ConnectTimeout as e:
                
    logging.error("Time out!")
                
finally:
                
    # continue request here
                
    print("hello")
                

                

                
# output,
                
ERROR:root:Time out!
                
hello





                share|improve this answer














                You can raise exception if there is a timeout and continue using finally for next request,



                import requests
                
import logging
                

                
timeout = 0.00001
                

                
try:
                
    response = requests.get(url="https://google.com", timeout=timeout)
                
except requests.exceptions.ConnectTimeout as e:
                
    logging.error("Time out!")
                
finally:
                
    # continue request here
                
    print("hello")
                

                

                
# output,
                
ERROR:root:Time out!
                
hello






                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited 2 days ago

























                answered 2 days ago









                Sufiyan Ghori

                10.7k95580




                10.7k95580






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53139649%2fhow-to-use-the-requests-module-to-skip-connection-timeout-urls%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest






































































                    -

                    這個網誌中的熱門文章

                    Tangent Lines Diagram Along Smooth Curve

                    圖片
                    7 1 I am trying to replicate a diagram and have a minimal example. I don't know how to add additional tangents between the specified points and preserve smoothness. documentclass{article} usepackage{tikz} usetikzlibrary{calc,intersections} begin{document} begin{tikzpicture} % Axes draw [->, name path=x] (-1,0) -- (11,0) node [right] {$x$}; draw [->] (0,-1) -- (0,6) node [above] {$y$}; % Origin node at (0,0) [below left] {$0$}; % Points coordinate (start) at (1,-0.8); coordinate (c1) at (3,3); coordinate (c2) at (5.5,1.5); coordinate (c3) at (8,4); coordinate (end) at (10.5,-0.8); % show the points foreach n in {start,c1,c2,c3,end} fill [black] (n) circle (2pt) node [below] {}; % join the coordinates draw [thick,name path=curve] (start) to[out=70,in=180] (c1) to[out=0,in=180]
                    閱讀完整內容

                    Yusuf al-Mu'taman ibn Hud

                    圖片
                    Main article: Taifa of Zaragoza Al-Mu'taman Ruler of Zaragoza Reign 1081–1085 Predecessor Ahmad al-Muqtadir Successor Al-Mustain II Born Zaragoza Died 1085 Full name Yusuf ibn Ahmad al-Mu'taman ibn Hūd House Banu Hud Yusuf ibn Ahmad al-Mu'taman ibn Hūd (Arabic: المؤتمن بالله يوسف إبن أحمد إبن هود ‎, al-Mutaman bi l-Lah , died c. 1085) was the third king of the Banu Hud dynasty who ruled over the Taifa of Zaragoza from 1081 to 1085. Yusuf al-Mutaman reigned during the height of power of Muslim Zaragoza , following the thriving period of his father Ahmad al-Muqtadir. He continued his father's efforts and created around him a court of intellectuals, living in the beautiful palace of Aljafería, nicknamed as "the palace of joy". Al-Mu'taman was also a scholarly king and a patron of science, philosophy and arts. As an example of a wise king, he knew astrology, philosophy, and especially mathematics, a discipline in which
                    閱讀完整內容

                    Zucchini

                    圖片
                    This article is about the vegetable. For other uses, see Zucchini (disambiguation). Zucchini (courgette) A striped and a uniform color zucchini Genus Cucurbita Species Cucurbita pepo Origin 19th century northern Italy The zucchini ( / z uː ˈ k iː n i / , American English) or courgette ( / k ʊər ˈ ʒ ɛ t / , British English) is a summer squash which can reach nearly 1 metre (100 cm; 39 in) in length, but is usually harvested when still immature at about 15 to 25 cm (6 to 10 in). [1] A zucchini is a thin-skinned cultivar of what in Britain and Ireland is referred to as a marrow. [2] [3] In South Africa, a zucchini is known as a baby marrow. Along with certain other squashes and pumpkins, the zucchini belongs to the species Cucurbita pepo . It can be dark or light green. A related hybrid, the golden zucchini, is a deep yellow or orange color. [4] In a culinary context, the zucchini is treated as a vegetable; it is usually cooked and presented as a savory dish o
                    閱讀完整內容