#9913 ro-access to running datanommer db
Closed: Fixed 3 years ago by kevin. Opened 3 years ago by astepano.

Describe what you would like us to do:


Hello, is there a ro access to running postgresql that hosts datanommer db?

Dumps for this db are located at: https://infrastructure.fedoraproject.org/infra/db-dumps/

I want to query on datanommer db.

There are 2 situations where we need this:

  1. when we lost some message, we want to dump it.
  2. I want to make queries to look what are active topics over some period of time.

Thank you.


For case 1, our other apps just do a http datagrepper query...

ie, see https://apps.fedoraproject.org/datagrepper/

For 2, @asaleh has been working on some openshift metrics/datagrepper query dashboards, perhaps we could share those and they might meet your needs.

If not, where would you want this access from?

Metadata Update from @kevin:
- Issue priority set to: Waiting on Reporter (was: Needs Review)
- Issue tagged with: medium-gain, medium-trouble, ops

3 years ago

Hello
dumping existing messages from HTTP requires much more time if to compare psql.

This I can say with 100% confidence.

https://apps.fedoraproject.org/datagrepper/ is super slow, even when I specify start=&end= window.

Just to compare in downstream: https://apps.fedoraproject.org/datagrepper/ -- dumps necessary messages in 2 days, where psql can dump the same messages in 10 min. (please believe me)
+ https://apps.fedoraproject.org/datagrepper/ replies with very high error rate.

Access from from batcave is fine.

The RO user exists and is in ansible. If you can login, you can try

https://monitor-dashboard-web-monitor-dashboard.app.os.fedoraproject.org/explore

as long as you have at least one project on openshift, you should be able to log-in, and the grafana instance running there has the datanommer configured as a datasource.

Ah yeah, would this meet your needs?

:-) hello

I still need some more info.

I can login into:

https://monitor-dashboard-web-monitor-dashboard.app.os.fedoraproject.org/explore

I can see PostgreSQL Connection in Settings.
I have "hostname" and "username" from this dialog.
I cannot see password.

Now I tried to use this, and without success.

batcave01 doesn't have psql installed command

I tried to spin a container: https://quay.io/repository/bitnami/postgresql
on https://console.app.os.fedoraproject.org/

I logged into: oc login --token=<TOKEN> --server=https://os.fedoraproject.org:443

I created a project.

I spinned container:

oc get pods
NAME READY STATUS RESTARTS AGE
command-demo-0 1/1 Running 0 14m

oc rsh command-demo-0
Error from server (BadRequest): Upgrade request required

:-) For me It sounds that I am doing something not right. + password :-)

Sorry, I was not clear.

I meant... does doing queries via the dashboard get you what you need? Or do you have to get raw data for some other purpose?
I didn't mean to spin up other instances or try and use the same user as the dashboard. :)

Hi :-) Is it possible to get list of all known topics from Graphana?
Or dump a raw messages with help of Graphana?
I already know: host name with db, username (that has RO access).
All I need: have psql on batcave01 and password for this user :-) That would be enough.

ok. I guess it's just easier to set you up with psql...

So, I have installed pqsl on batcave01, and put the ro user db password in your home directory in a file called 'db-pass'. Please keep this secret.

Let us know if you need anything further.

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

I confirm it works:

psql .....

datanommer=> SELECT msg_id from messages LIMIT 1;
                  msg_id                   
-------------------------------------------
 2020-81c09db0-6120-46ad-8695-b0fd63d58392
(1 row)

Log in to comment on this ticket.

Metadata
Boards 1
ops Status: Done