#30 Optimize syncing http logs
Closed 3 years ago by nphilipp. Opened 3 years ago by nphilipp.

User Story

As someone who needs http logs analyzed,

I want that daily sync runs take less than 6 hours,

because 6 HOURS ARE YOU KIDDING ME?!

Acceptance Criteria

A script exists which doesn't use one rsync call per synced log file and which processes several hosts in parallel.

Background

The existing syncHttpLogs.sh script polls logs for the past three days, for each host one at a time and one file after another. Using one call to rsync per host/day (i.e. directory) and working on several hosts in parallel should speed things up.


Metadata Update from @nphilipp:
- Issue tagged with: DNF Counting

3 years ago

Metadata Update from @nphilipp:
- Issue assigned to nphilipp

3 years ago

Issue status updated to: Closed (was: Open)

3 years ago

Log in to comment on this ticket.

Metadata
Boards 1