#12171 Direct database access to message bus date for Greg Sutcliffe
Closed: Fixed 5 months ago by kevin. Opened 9 months ago by mattdm.

Greg (@gwmngilfen) is a data scientist interested in helping with metrics, statistics, and insights from our message bus. As we all know, the datagrepper interface is painfully slow. Could he have direct access to the database, please?

Thank you!


Thanks @mattdm - obviously read-only would be fine, I have no interest in altering anything ;)

Metadata Update from @phsmoura:
- Issue priority set to: Waiting on Reporter (was: Needs Review)
- Issue tagged with: low-gain, low-trouble, ops

9 months ago

@kevin that would be fine, but I've not seen much movement on it? I've also heard talk of some synthetic data approach that apparently @smilner was involved in, so I will go ask about that route too.

Ultimately, I just want to start trialling approaches to analysing the data while we build these better access systems - so even a onetime dump of a few days data would probably be enough. It's about understanding the structure and seeing what we can do with it.

@kevin that would be fine, but I've not seen much movement on it?

Well, we are busy? it's not super high on the priority list over say... getting releases out. ;)

I've also heard talk of some synthetic data approach that apparently @smilner was involved in, so I will go ask about that route too.

Interesting. I had not heard about that. Steve is actually out this week and next, but we can ask him when he's back.

Ultimately, I just want to start trialling approaches to analysing the data while we build these better access systems - so even a onetime dump of a few days data would probably be enough. It's about understanding the structure and seeing what we can do with it.

Our entire gigantic datanommer db is available:

https://infrastructure.fedoraproject.org/infra/db-dumps/

Unfortunately I see it's currently truncated/messed up... will fix that.

Well, we are busy? it's not super high on the priority list over say... getting releases out. ;)

oooh, I consider myself well and truly told off :) - but entirely fairly, that is more important! That came out more snarky that I meant it to, so my apologies. It was my own frustration at being stuck leaking out, and it has no place here.

Interesting. I had not heard about that

I believe I have the right Steve - I'm referring to CommOps meeting notes from after Flock, but as I couldn't make it this year, I'm just repeating what others have written. I'm out next week anyway, so I'll follow up when I get back on the 30th.

Our entire gigantic datanommer db is available:

If only I'd known! I still think releases have higher priority that fixing it though :stuck_out_tongue:

So, we do have folks working on this now... @dkirwan and @phsmoura so hopefully that will be up soon.

However, also we can just probibly get you access directly since you are on the team now. ;)

So, lets close this now and track the other db in those tickets and ping me for other access.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

5 months ago

Log in to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog