Learn more about these different git repos.
Other Git URLs
We use requests extensively in Hubs to talk to other services. We should use a thread-local session so we benefit from TCP connection pooling.
It would also be good if we exposed some of the session configuration options in the Hubs configuration, since it lets you configure default headers, proxies, retries, max redirects allowed, etc. for all outgoing requests using that session.
We could probably add this type of function in hubs/app.py :
hubs/app.py
@app.before_request def requests_session(): flask.g.requests_session = requests.Session() # Do more configuration based on app.config["REQUESTS"] for example
And then the session would be available in the thread-local flask.g singleton.
flask.g
Metadata Update from @abompard: - Issue priority set to: 1
Hmm. This would create a new session for every request, right?
That would be an improvement, but I think if we implemented a session registry as SQLAlchemy does for database sessions, we could share across requests that occur in the same thread (I'm assuming the WSGI processes get used for more than one request).
This would actually be generally useful since I know anitya had a similar issue.
this is the answer about decorator and calling the deferred http://flask.pocoo.org/docs/0.12/patterns/deferredcallbacks/
What I have in mind is something like
import threading import requests class RequestsSessionRegistry(object): def __init__(self, session_factory=None): self.registry = threading.local() if session_factory is None: self.session_factory = requests.Session def __call__(self, *args, **kwargs): """ Create or get the current thread-local requests session """ if not hasattr(self.registry, 'session'): self.registry.session = self.session_factory(*args, **kwargs) elif args or kwargs: # args or kwargs can't be provided if there's already a session raise ValueError('args and kwargs must be none if the registry has' ' a session already. Call `remove` first.') return self.registry.session def remove(self): """ Close all open connections and remove the thread-local session. """ if hasattr(self.registry, 'session'): self.registry.session.close() del self.registry.session
which would be used very much like a SQLAlchemy scoped session:
# Done at the module level, everyone just imports this Session RequestsSession = RequestsSessionRegistry() session = RequestsSession() session.get('http://example.com')
Pr #337 has this with some tweaks for bugs I found while writing tests.
Metadata Update from @sayanchowdhury: - Issue priority set to: Normal (was: High) - Issue tagged with: enhancement
Log in to comment on this ticket.