How to use the requests module to skip connection timeout urls











up vote
2
down vote

favorite












Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



def req():

with open('demofile.txt','r') as http:

    for url in http.readlines():

        req = url.strip()

        print(req)

        page=requests.get("http://"+req,verify=False)

        if page.status_code == 400:

            break

        else:

            continue

        time.sleep(1)





python python-3.x python-requests







share|improve this question


























    up vote
    2
    down vote

    favorite












    Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



    def req():
    
with open('demofile.txt','r') as http:
    
    for url in http.readlines():
    
        req = url.strip()
    
        print(req)
    
        page=requests.get("http://"+req,verify=False)
    
        if page.status_code == 400:
    
            break
    
        else:
    
            continue
    
        time.sleep(1)





    python python-3.x python-requests







    share|improve this question
























      up vote
      2
      down vote

      favorite









      up vote
      2
      down vote

      favorite











      Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



      def req():
      
with open('demofile.txt','r') as http:
      
    for url in http.readlines():
      
        req = url.strip()
      
        print(req)
      
        page=requests.get("http://"+req,verify=False)
      
        if page.status_code == 400:
      
            break
      
        else:
      
            continue
      
        time.sleep(1)





      python python-3.x python-requests







      share|improve this question













      Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one



      def req():
      
with open('demofile.txt','r') as http:
      
    for url in http.readlines():
      
        req = url.strip()
      
        print(req)
      
        page=requests.get("http://"+req,verify=False)
      
        if page.status_code == 400:
      
            break
      
        else:
      
            continue
      
        time.sleep(1)





      python python-3.x python-requests




      python python-3.x python-requests






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 2 days ago









      Rahul

      243




      243
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          2
          down vote



          accepted










          You can raise exception if there is a timeout and continue using finally for next request,



          import requests
          
import logging
          

          
timeout = 0.00001
          

          
try:
          
    response = requests.get(url="https://google.com", timeout=timeout)
          
except requests.exceptions.ConnectTimeout as e:
          
    logging.error("Time out!")
          
finally:
          
    # continue request here
          
    print("hello")
          

          

          
# output,
          
ERROR:root:Time out!
          
hello





          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














             

            draft saved


            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53139649%2fhow-to-use-the-requests-module-to-skip-connection-timeout-urls%23new-answer', 'question_page');
            }
            );

            Post as a guest


















            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            2
            down vote



            accepted










            You can raise exception if there is a timeout and continue using finally for next request,



            import requests
            
import logging
            

            
timeout = 0.00001
            

            
try:
            
    response = requests.get(url="https://google.com", timeout=timeout)
            
except requests.exceptions.ConnectTimeout as e:
            
    logging.error("Time out!")
            
finally:
            
    # continue request here
            
    print("hello")
            

            

            
# output,
            
ERROR:root:Time out!
            
hello





            share|improve this answer



























              up vote
              2
              down vote



              accepted










              You can raise exception if there is a timeout and continue using finally for next request,



              import requests
              
import logging
              

              
timeout = 0.00001
              

              
try:
              
    response = requests.get(url="https://google.com", timeout=timeout)
              
except requests.exceptions.ConnectTimeout as e:
              
    logging.error("Time out!")
              
finally:
              
    # continue request here
              
    print("hello")
              

              

              
# output,
              
ERROR:root:Time out!
              
hello





              share|improve this answer

























                up vote
                2
                down vote



                accepted







                up vote
                2
                down vote



                accepted






                You can raise exception if there is a timeout and continue using finally for next request,



                import requests
                
import logging
                

                
timeout = 0.00001
                

                
try:
                
    response = requests.get(url="https://google.com", timeout=timeout)
                
except requests.exceptions.ConnectTimeout as e:
                
    logging.error("Time out!")
                
finally:
                
    # continue request here
                
    print("hello")
                

                

                
# output,
                
ERROR:root:Time out!
                
hello





                share|improve this answer














                You can raise exception if there is a timeout and continue using finally for next request,



                import requests
                
import logging
                

                
timeout = 0.00001
                

                
try:
                
    response = requests.get(url="https://google.com", timeout=timeout)
                
except requests.exceptions.ConnectTimeout as e:
                
    logging.error("Time out!")
                
finally:
                
    # continue request here
                
    print("hello")
                

                

                
# output,
                
ERROR:root:Time out!
                
hello






                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited 2 days ago

























                answered 2 days ago









                Sufiyan Ghori

                10.7k95580




                10.7k95580






























                     

                    draft saved


                    draft discarded



















































                     


                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53139649%2fhow-to-use-the-requests-module-to-skip-connection-timeout-urls%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest






































































                    -

                    這個網誌中的熱門文章

                    Xamarin.form Move up view when keyboard appear

                    圖片
                    .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 0 I'm trying to build a Chat app UI, the idea of the Layout was pretty simple: When the input bar is focused, keyboard show up and "push" up the chat bar, as it's a grid, the ListView will resize to fit the screen: I update the input bar's margin to "push" it up: NSValue result = (NSValue)args.Notification.UserInfo.ObjectForKey(new NSString(UIKeyboard.FrameEndUserInfoKey)); CGSize keyboardSize = result.RectangleFValue.Size; if (Element != null){ Element.Margin = new Thickness(0, 0, 0,keyboardSize.Height); //push the entry up to keyboard height when keyboard is activated } And this is ...
                    閱讀完整內容

                    Post-Redirect-Get with Spring WebFlux and Thymeleaf

                    圖片
                    .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 2 Can someone explain to me how to implement the post-redirect-get pattern in Spring WebFlux and Thymeleaf? What subscribes on the database save method? @GetMapping("/register") public String showRegisterForm(Model model) { model.addAttribute("user", new User()); return "register"; } @PostMapping public String processRegisterForm(@Valid User user, BindingResult bindingResult) { if (bindingResult.hasErrors()) { return "register"; } else { userRepository.save(user); //what subscribes on this? //how to redirect on e.g. "/login"? ...
                    閱讀完整內容

                    Anylogic : not able to use stopDelay()

                    圖片
                    -2 I am building a railway station model. I want a train to depart when I click a button. I want to call stopDelay() for that but I am not sure how to do that and from which place. anylogic share | improve this question asked Nov 23 '18 at 10:12 Anupam Singh Anupam Singh 1 Hi Anupam, welcome to SOF. Please check how to ask good questions, see link below. Your question is...
                    閱讀完整內容