Are global variables thread safe in flask? How do I share data between requests?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}







46















In my app the state of a common object is changed by making requests, and the response depends on the state.



class SomeObj():
def __init__(self, param):
self.param = param
def query(self):
self.param += 1
return self.param

global_obj = SomeObj(0)

@app.route('/')
def home():
flash(global_obj.query())
render_template('index.html')


If I run this on my development server, I expect to get 1, 2, 3 and so on. If requests are made from 100 different clients simultaneously, can something go wrong? The expected result would be that the 100 different clients each see a unique number from 1 to 100. Or will something like this happen:




  1. Client 1 queries. self.param is incremented by 1.

  2. Before the return statement can be executed, the thread switches over to client 2. self.param is incremented again.

  3. The thread switches back to client 1, and the client is returned the number 2, say.

  4. Now the thread moves to client 2 and returns him/her the number 3.


Since there were only two clients, the expected results were 1 and 2, not 2 and 3. A number was skipped.



Will this actually happen as I scale up my application? What alternatives to a global variable should I look at?










share|improve this question































    46















    In my app the state of a common object is changed by making requests, and the response depends on the state.



    class SomeObj():
    def __init__(self, param):
    self.param = param
    def query(self):
    self.param += 1
    return self.param

    global_obj = SomeObj(0)

    @app.route('/')
    def home():
    flash(global_obj.query())
    render_template('index.html')


    If I run this on my development server, I expect to get 1, 2, 3 and so on. If requests are made from 100 different clients simultaneously, can something go wrong? The expected result would be that the 100 different clients each see a unique number from 1 to 100. Or will something like this happen:




    1. Client 1 queries. self.param is incremented by 1.

    2. Before the return statement can be executed, the thread switches over to client 2. self.param is incremented again.

    3. The thread switches back to client 1, and the client is returned the number 2, say.

    4. Now the thread moves to client 2 and returns him/her the number 3.


    Since there were only two clients, the expected results were 1 and 2, not 2 and 3. A number was skipped.



    Will this actually happen as I scale up my application? What alternatives to a global variable should I look at?










    share|improve this question



























      46












      46








      46


      7






      In my app the state of a common object is changed by making requests, and the response depends on the state.



      class SomeObj():
      def __init__(self, param):
      self.param = param
      def query(self):
      self.param += 1
      return self.param

      global_obj = SomeObj(0)

      @app.route('/')
      def home():
      flash(global_obj.query())
      render_template('index.html')


      If I run this on my development server, I expect to get 1, 2, 3 and so on. If requests are made from 100 different clients simultaneously, can something go wrong? The expected result would be that the 100 different clients each see a unique number from 1 to 100. Or will something like this happen:




      1. Client 1 queries. self.param is incremented by 1.

      2. Before the return statement can be executed, the thread switches over to client 2. self.param is incremented again.

      3. The thread switches back to client 1, and the client is returned the number 2, say.

      4. Now the thread moves to client 2 and returns him/her the number 3.


      Since there were only two clients, the expected results were 1 and 2, not 2 and 3. A number was skipped.



      Will this actually happen as I scale up my application? What alternatives to a global variable should I look at?










      share|improve this question
















      In my app the state of a common object is changed by making requests, and the response depends on the state.



      class SomeObj():
      def __init__(self, param):
      self.param = param
      def query(self):
      self.param += 1
      return self.param

      global_obj = SomeObj(0)

      @app.route('/')
      def home():
      flash(global_obj.query())
      render_template('index.html')


      If I run this on my development server, I expect to get 1, 2, 3 and so on. If requests are made from 100 different clients simultaneously, can something go wrong? The expected result would be that the 100 different clients each see a unique number from 1 to 100. Or will something like this happen:




      1. Client 1 queries. self.param is incremented by 1.

      2. Before the return statement can be executed, the thread switches over to client 2. self.param is incremented again.

      3. The thread switches back to client 1, and the client is returned the number 2, say.

      4. Now the thread moves to client 2 and returns him/her the number 3.


      Since there were only two clients, the expected results were 1 and 2, not 2 and 3. A number was skipped.



      Will this actually happen as I scale up my application? What alternatives to a global variable should I look at?







      python flask thread-safety






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jun 1 '18 at 16:38









      davidism

      66.6k12187196




      66.6k12187196










      asked Sep 28 '15 at 3:52









      sayantankhansayantankhan

      5111612




      5111612
























          2 Answers
          2






          active

          oldest

          votes


















          46














          You can't use global variables to hold this sort of data. Not only is it not thread safe, it's not process safe, and WSGI servers in production spawn multiple processes. Not only would your counts be wrong if you were using threads to handle requests, they would also vary depending on which process handled the request.



          Use a data source outside of Flask to hold global data. A database, memcached, or redis are all appropriate separate storage areas, depending on your needs. If you need to load and access Python data, consider multiprocessing.Manager. You could also use the session for simple data that is per-user.





          The development server may run in single thread and process. You won't see the behavior you describe since each request will be handled synchronously. Enable threads or processes and you will see it. app.run(threaded=True) or app.run(processes=10). (In 1.0 the server is threaded by default.)





          Some WSGI servers may support gevent or another async worker. Global variables are still not thread safe because there's still no protection against most race conditions. You can still have a scenario where one worker gets a value, yields, another modifies it, yields, then the first worker also modifies it.





          If you need to store some global data during a request, you may use Flask's g object. Another common case is some top-level object that manages database connections. The distinction for this type of "global" is that it's unique to each request, not used between requests, and there's something managing the set up and teardown of the resource.






          share|improve this answer

































            2














            This is not really an answer to thread safety of globals.



            But I think it is important to mention sessions here.
            You are looking for a way to store client-specific data. Every connection should have access to its own pool of data, in a threadsafe way.



            This is possible with server-side sessions, and they are available in a very neat flask plugin: https://pythonhosted.org/Flask-Session/



            If you set up sessions, a session variable is available in all your routes and it behaves like a dictionary. The data stored in this dictionary is individual for each connecting client.



            Here is a short demo:



            from flask import Flask, session
            from flask_session import Session

            app = Flask(__name__)
            # Check Configuration section for more details
            SESSION_TYPE = 'filesystem'
            app.config.from_object(__name__)
            Session(app)

            @app.route('/')
            def reset():
            session["counter"]=0

            return "counters was reset"

            @app.route('/inc')
            def routeA():
            if not "counter" in session:
            session["counter"]=0

            session["counter"]+=1

            return "counter is {}".format(session["counter"])

            @app.route('/dec')
            def routeB():
            if not "counter" in session:
            session["counter"] = 0

            session["counter"] -= 1

            return "counter is {}".format(session["counter"])


            if __name__ == '__main__':
            app.run()


            After pip install Flask-Session, you should be able to run this. Try accessing it from different browsers, you'll see that the counter is not shared between them.






            share|improve this answer
























            • This is very cool! Why don't you set it up as its own Q&A?

              – jtlz2
              Apr 1 at 14:35












            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f32815451%2fare-global-variables-thread-safe-in-flask-how-do-i-share-data-between-requests%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            46














            You can't use global variables to hold this sort of data. Not only is it not thread safe, it's not process safe, and WSGI servers in production spawn multiple processes. Not only would your counts be wrong if you were using threads to handle requests, they would also vary depending on which process handled the request.



            Use a data source outside of Flask to hold global data. A database, memcached, or redis are all appropriate separate storage areas, depending on your needs. If you need to load and access Python data, consider multiprocessing.Manager. You could also use the session for simple data that is per-user.





            The development server may run in single thread and process. You won't see the behavior you describe since each request will be handled synchronously. Enable threads or processes and you will see it. app.run(threaded=True) or app.run(processes=10). (In 1.0 the server is threaded by default.)





            Some WSGI servers may support gevent or another async worker. Global variables are still not thread safe because there's still no protection against most race conditions. You can still have a scenario where one worker gets a value, yields, another modifies it, yields, then the first worker also modifies it.





            If you need to store some global data during a request, you may use Flask's g object. Another common case is some top-level object that manages database connections. The distinction for this type of "global" is that it's unique to each request, not used between requests, and there's something managing the set up and teardown of the resource.






            share|improve this answer






























              46














              You can't use global variables to hold this sort of data. Not only is it not thread safe, it's not process safe, and WSGI servers in production spawn multiple processes. Not only would your counts be wrong if you were using threads to handle requests, they would also vary depending on which process handled the request.



              Use a data source outside of Flask to hold global data. A database, memcached, or redis are all appropriate separate storage areas, depending on your needs. If you need to load and access Python data, consider multiprocessing.Manager. You could also use the session for simple data that is per-user.





              The development server may run in single thread and process. You won't see the behavior you describe since each request will be handled synchronously. Enable threads or processes and you will see it. app.run(threaded=True) or app.run(processes=10). (In 1.0 the server is threaded by default.)





              Some WSGI servers may support gevent or another async worker. Global variables are still not thread safe because there's still no protection against most race conditions. You can still have a scenario where one worker gets a value, yields, another modifies it, yields, then the first worker also modifies it.





              If you need to store some global data during a request, you may use Flask's g object. Another common case is some top-level object that manages database connections. The distinction for this type of "global" is that it's unique to each request, not used between requests, and there's something managing the set up and teardown of the resource.






              share|improve this answer




























                46












                46








                46







                You can't use global variables to hold this sort of data. Not only is it not thread safe, it's not process safe, and WSGI servers in production spawn multiple processes. Not only would your counts be wrong if you were using threads to handle requests, they would also vary depending on which process handled the request.



                Use a data source outside of Flask to hold global data. A database, memcached, or redis are all appropriate separate storage areas, depending on your needs. If you need to load and access Python data, consider multiprocessing.Manager. You could also use the session for simple data that is per-user.





                The development server may run in single thread and process. You won't see the behavior you describe since each request will be handled synchronously. Enable threads or processes and you will see it. app.run(threaded=True) or app.run(processes=10). (In 1.0 the server is threaded by default.)





                Some WSGI servers may support gevent or another async worker. Global variables are still not thread safe because there's still no protection against most race conditions. You can still have a scenario where one worker gets a value, yields, another modifies it, yields, then the first worker also modifies it.





                If you need to store some global data during a request, you may use Flask's g object. Another common case is some top-level object that manages database connections. The distinction for this type of "global" is that it's unique to each request, not used between requests, and there's something managing the set up and teardown of the resource.






                share|improve this answer















                You can't use global variables to hold this sort of data. Not only is it not thread safe, it's not process safe, and WSGI servers in production spawn multiple processes. Not only would your counts be wrong if you were using threads to handle requests, they would also vary depending on which process handled the request.



                Use a data source outside of Flask to hold global data. A database, memcached, or redis are all appropriate separate storage areas, depending on your needs. If you need to load and access Python data, consider multiprocessing.Manager. You could also use the session for simple data that is per-user.





                The development server may run in single thread and process. You won't see the behavior you describe since each request will be handled synchronously. Enable threads or processes and you will see it. app.run(threaded=True) or app.run(processes=10). (In 1.0 the server is threaded by default.)





                Some WSGI servers may support gevent or another async worker. Global variables are still not thread safe because there's still no protection against most race conditions. You can still have a scenario where one worker gets a value, yields, another modifies it, yields, then the first worker also modifies it.





                If you need to store some global data during a request, you may use Flask's g object. Another common case is some top-level object that manages database connections. The distinction for this type of "global" is that it's unique to each request, not used between requests, and there's something managing the set up and teardown of the resource.







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Aug 16 '18 at 14:09

























                answered Sep 28 '15 at 14:26









                davidismdavidism

                66.6k12187196




                66.6k12187196

























                    2














                    This is not really an answer to thread safety of globals.



                    But I think it is important to mention sessions here.
                    You are looking for a way to store client-specific data. Every connection should have access to its own pool of data, in a threadsafe way.



                    This is possible with server-side sessions, and they are available in a very neat flask plugin: https://pythonhosted.org/Flask-Session/



                    If you set up sessions, a session variable is available in all your routes and it behaves like a dictionary. The data stored in this dictionary is individual for each connecting client.



                    Here is a short demo:



                    from flask import Flask, session
                    from flask_session import Session

                    app = Flask(__name__)
                    # Check Configuration section for more details
                    SESSION_TYPE = 'filesystem'
                    app.config.from_object(__name__)
                    Session(app)

                    @app.route('/')
                    def reset():
                    session["counter"]=0

                    return "counters was reset"

                    @app.route('/inc')
                    def routeA():
                    if not "counter" in session:
                    session["counter"]=0

                    session["counter"]+=1

                    return "counter is {}".format(session["counter"])

                    @app.route('/dec')
                    def routeB():
                    if not "counter" in session:
                    session["counter"] = 0

                    session["counter"] -= 1

                    return "counter is {}".format(session["counter"])


                    if __name__ == '__main__':
                    app.run()


                    After pip install Flask-Session, you should be able to run this. Try accessing it from different browsers, you'll see that the counter is not shared between them.






                    share|improve this answer
























                    • This is very cool! Why don't you set it up as its own Q&A?

                      – jtlz2
                      Apr 1 at 14:35
















                    2














                    This is not really an answer to thread safety of globals.



                    But I think it is important to mention sessions here.
                    You are looking for a way to store client-specific data. Every connection should have access to its own pool of data, in a threadsafe way.



                    This is possible with server-side sessions, and they are available in a very neat flask plugin: https://pythonhosted.org/Flask-Session/



                    If you set up sessions, a session variable is available in all your routes and it behaves like a dictionary. The data stored in this dictionary is individual for each connecting client.



                    Here is a short demo:



                    from flask import Flask, session
                    from flask_session import Session

                    app = Flask(__name__)
                    # Check Configuration section for more details
                    SESSION_TYPE = 'filesystem'
                    app.config.from_object(__name__)
                    Session(app)

                    @app.route('/')
                    def reset():
                    session["counter"]=0

                    return "counters was reset"

                    @app.route('/inc')
                    def routeA():
                    if not "counter" in session:
                    session["counter"]=0

                    session["counter"]+=1

                    return "counter is {}".format(session["counter"])

                    @app.route('/dec')
                    def routeB():
                    if not "counter" in session:
                    session["counter"] = 0

                    session["counter"] -= 1

                    return "counter is {}".format(session["counter"])


                    if __name__ == '__main__':
                    app.run()


                    After pip install Flask-Session, you should be able to run this. Try accessing it from different browsers, you'll see that the counter is not shared between them.






                    share|improve this answer
























                    • This is very cool! Why don't you set it up as its own Q&A?

                      – jtlz2
                      Apr 1 at 14:35














                    2












                    2








                    2







                    This is not really an answer to thread safety of globals.



                    But I think it is important to mention sessions here.
                    You are looking for a way to store client-specific data. Every connection should have access to its own pool of data, in a threadsafe way.



                    This is possible with server-side sessions, and they are available in a very neat flask plugin: https://pythonhosted.org/Flask-Session/



                    If you set up sessions, a session variable is available in all your routes and it behaves like a dictionary. The data stored in this dictionary is individual for each connecting client.



                    Here is a short demo:



                    from flask import Flask, session
                    from flask_session import Session

                    app = Flask(__name__)
                    # Check Configuration section for more details
                    SESSION_TYPE = 'filesystem'
                    app.config.from_object(__name__)
                    Session(app)

                    @app.route('/')
                    def reset():
                    session["counter"]=0

                    return "counters was reset"

                    @app.route('/inc')
                    def routeA():
                    if not "counter" in session:
                    session["counter"]=0

                    session["counter"]+=1

                    return "counter is {}".format(session["counter"])

                    @app.route('/dec')
                    def routeB():
                    if not "counter" in session:
                    session["counter"] = 0

                    session["counter"] -= 1

                    return "counter is {}".format(session["counter"])


                    if __name__ == '__main__':
                    app.run()


                    After pip install Flask-Session, you should be able to run this. Try accessing it from different browsers, you'll see that the counter is not shared between them.






                    share|improve this answer













                    This is not really an answer to thread safety of globals.



                    But I think it is important to mention sessions here.
                    You are looking for a way to store client-specific data. Every connection should have access to its own pool of data, in a threadsafe way.



                    This is possible with server-side sessions, and they are available in a very neat flask plugin: https://pythonhosted.org/Flask-Session/



                    If you set up sessions, a session variable is available in all your routes and it behaves like a dictionary. The data stored in this dictionary is individual for each connecting client.



                    Here is a short demo:



                    from flask import Flask, session
                    from flask_session import Session

                    app = Flask(__name__)
                    # Check Configuration section for more details
                    SESSION_TYPE = 'filesystem'
                    app.config.from_object(__name__)
                    Session(app)

                    @app.route('/')
                    def reset():
                    session["counter"]=0

                    return "counters was reset"

                    @app.route('/inc')
                    def routeA():
                    if not "counter" in session:
                    session["counter"]=0

                    session["counter"]+=1

                    return "counter is {}".format(session["counter"])

                    @app.route('/dec')
                    def routeB():
                    if not "counter" in session:
                    session["counter"] = 0

                    session["counter"] -= 1

                    return "counter is {}".format(session["counter"])


                    if __name__ == '__main__':
                    app.run()


                    After pip install Flask-Session, you should be able to run this. Try accessing it from different browsers, you'll see that the counter is not shared between them.







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Nov 5 '18 at 10:21









                    lhklhk

                    7,330105793




                    7,330105793













                    • This is very cool! Why don't you set it up as its own Q&A?

                      – jtlz2
                      Apr 1 at 14:35



















                    • This is very cool! Why don't you set it up as its own Q&A?

                      – jtlz2
                      Apr 1 at 14:35

















                    This is very cool! Why don't you set it up as its own Q&A?

                    – jtlz2
                    Apr 1 at 14:35





                    This is very cool! Why don't you set it up as its own Q&A?

                    – jtlz2
                    Apr 1 at 14:35


















                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f32815451%2fare-global-variables-thread-safe-in-flask-how-do-i-share-data-between-requests%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    這個網誌中的熱門文章

                    Tangent Lines Diagram Along Smooth Curve

                    Yusuf al-Mu'taman ibn Hud

                    Zucchini