This adds monitor module providing Prometheus export for the common set of metrics (as of now).
rebased onto 76bbe41a72ed4d0d8684ab9561a5fd7213122a40
rebased onto 56871d8edb2419dc0759de9610a3662ed9d6b45f
can we make this consistent with Greenwave? Usually WaiverDB and Greenwave have the same paths since are "twins" :) Greenwave exposes the metrics at /metrics
Why is this import here and not at the begging of the file?
What "messaging_sent_failed_counter" stands for? If it is the number of messages not sent I wouldn't call it "messaging sent", since it wasn't actually sent.
Why do you have this counter if you already have the "messaging_sent_passed_counter"?
You are increasing "messaging_sent_counter" before this, if it will hit the "continue" it won't publish a message and it wouldn't make sense to increase "messaging_sent_counter"
why changing this logic?
I'm not sure we want to actually raise an exception if an error occurs, if we don't raise an error it will just continue with the next item of the loop. It would be ok to log it.
where is this used?
rebased onto 16f9e456e0d5f4b421a4a09452fd658f81714904
I can do it. I was thinking forward on the /monitor/ endpoint as a "home" for other things like healthchecks etc., as proposed in "Factory 2.0 Monitoring docs". If we do it the other way round, I'd prepare a PR for changing it in Greenwave.
/monitor/
What "messaging_sent_failed_counter" stands for? If it is the number of messages not sent I wouldn't call it "messaging sent", since it wasn't actually sent. Why do you have this counter if you already have the "messaging_sent_passed_counter"? You are increasing "messaging_sent_counter" before this, if it will hit the "continue" it won't publish a message and it wouldn't make sense to increase "messaging_sent_counter"
Sorry, the naming was already fixed in the monitor module, but not here.. -> Fixed now.
This is a generic set of metrics, not used on waiverdb.
There shouldn't be changes in logic. Just changes to de-indent: if...else -> if not
if...else
if not
As I added one more level, it started to be unreadable.
There shouldn't be any difference. If the sender(s) raised an exception before, this just re-raises it... The only difference is that it gets caught and counters are incremented...
It is fine if we use this approach, but yeah, I would like it to be the same for both WaiverDB and Greenwave. We just have to make sure no one is already using /metrics in Greenwave (I don't think so).
where is this used? This is a generic set of metrics, not used on waiverdb.
So is it useful to keep it if we don't use it?
where is this used? This is a generic set of metrics, not used on waiverdb. So is it useful to keep it if we don't use it?
It'd more difficult to maintain various versions of the monitor module for each service if the generic part would be different. Most of other services publish/consume messages. But WaiverDB just publishes (from what I know). I am open to ideas of keeping it slim but also maintainable.
I think that internal deployment would need to make slight modifications to annotations in the templates, and that should be all.
rebased onto e0054e8008f31b6c34c53354478d6fa98b70eceb
rebased onto 1f6c2aea3220950c000ef3893a712c52cd2156d0
rebased onto 8c534bb85808701287f8230632db6ea75444e259
This is causing also failings on our CIs... https://jenkins-waiverdb-test.cloud.paas.upshift.redhat.com/job/waiverdb-test/job/waiverdb-test-waiverdb-dev/162/console I understand your point, but wiaverdb is still a independent service and I'm not really happy about having stuff that we don't need in the code.
Optional: You could just use raise instead of needing the e variable
raise
e
Why not just use the path /monitor/metrics that points directly to this view? I don't see any benefit from making path a variable to parse. It just adds complexity in my opinion.
/monitor/metrics
path
rebased onto bc5a25cc1c0ac1de04a66fb46858dd5bfc22cafc
@mprahl @gnaponie I shrank the code to fit just the waiverdb needs. The etalon can be found in a separate repo, see monitor.py's header.
monitor.py
rebased onto 96aa03b62d5ff6d2bfb1f075ea4521fed7d33b3d
The CI is still failing... these errors: 08:43:09 [Invoke Pylint] E: 11, 0: Unable to import 'prometheus_client' (import-error) 08:43:09 [Invoke Pylint] W: 76, 0: Unused argument 'kw' (unused-argument) 08:43:09 [Invoke Pylint] W: 80,31: Unused argument 'conn' (unused-argument) 08:43:09 [Invoke Pylint] W: 80,37: Unused argument 'branch' (unused-argument) 08:43:09 [Invoke Pylint] W: 84,29: Unused argument 'exception_context' (unused-argument) 08:43:09 [Invoke Pylint] W: 88,25: Unused argument 'conn' (unused-argument) 08:43:09 [Invoke Pylint] W: 76, 4: Unused variable 'receive_dbapi_error' (unused-variable) 08:43:09 [Invoke Pylint] W: 80, 4: Unused variable 'receive_engine_connect' (unused-variable) 08:43:09 [Invoke Pylint] W: 84, 4: Unused variable 'receive_handle_error' (unused-variable) 08:43:09 [Invoke Pylint] W: 88, 4: Unused variable 'receive_rollback' (unused-variable) 08:43:09 [Invoke Pylint] W: 11, 0: Unused Histogram imported from prometheus_client (unused-import)
See PR#275 which block this PR on the CI. Then we can continue here...
You should add the keyword argument content_type=prometheus_client.CONTENT_TYPE_LATEST since this will set the content type of plain text with the metrics output version that python-prometheus_client is using.
content_type=prometheus_client.CONTENT_TYPE_LATEST
Thanks for pointing this out. I have to do it F2.0-projects-wide. :)
rebased onto 391b5f0ea0ab052175cca632300e093047eb7ba1
rebased onto dbec3f98f5f38f15acc40421b25292f9e5493d77
1 new commit added
[WIP] Silence pylint-3
Fix tests
Debug failing tests in CI
Just check our (F2.0) counters in the tests
The CI failed again, it looks like an error in Jenkins itself. Could you re-trigger the job?
rebased onto 4a731ea1b3e966b7b46b496a1aab8310bc1eb9ab
rebased onto 8e51a0534e317ca1896d3fd257cadba5a40822d5
@gnaponie Please for final review here. All comments have been addressed.
Why this is not imported at the beginning of the life?
From PEP8 doc: Imports should be grouped in the following order:
Standard library imports. Related third party imports. Local application/library specific imports. You should put a blank line between each group of imports.
Why is that before "from flask import current_app"? Is there some issue with circular imports?
Just to avoid cyclic imports. But the noqa seems to be some relic from previous revisions.
rebased onto 85cc21669c138fb09590f8a89b1f5eff60f44acb
Can we add the log of the error here?
Logging is not related to this PR, but I'll add some _log.exception there.
_log.exception
Beside the minor comments it looks good +1
fixup this
rebased onto 62e73883637a526a3b08575c11c44c451b938672
@gnaponie Build 181 Finished: SUCCESS for 62e7388.
Finished: SUCCESS
looks good to me +1
Commit 5265247c fixes this pull-request
Pull-Request has been merged by gnaponie
This adds monitor module providing Prometheus export for the common set of metrics (as of now).