#437 Add list of metrics we might want to collect, and privacy/transparency checklist
Merged 4 months ago by catanzaro. Opened 4 months ago by catanzaro.

@@ -0,0 +1,43 @@ 

+ = Privacy and Transparency Checklist =

+ 

+ The proposal owners commit to completing the following as part of implementing the metrics change proposal.

+ 

+ == Before deployment ==

+ 

+ Governance arrangements:

+ 

+ * Metrics SIG up and running in the open

+ ** Docs on how the community can join

+ ** Process for community members to request new data to be collected

+ ** Members of the SIG have access to the database data

+ ** Process for community members to get snapshots of the database

+ * Rules on which data can and cannot be collected

+ * Formal approval of the initial metric set

+ * Approval procedure in place for any changes to the metrics system, including changes to which metrics are collected (changes will require FESCo approval)

+ 

+ Transparency measures:

+ 

+ * Document which data will be collected and what the purpose of each metric will be

+ * All code is open source

+ * Document the role of each component

+ * Method for users to view the local data on their machines

+ * Publish raw data from system testing

+ 

+ User control features:

+ 

+ * Metrics collection off by default

+ * Metrics collection enabled through opt in only

+ 

+ Privacy measures:

+ 

+ * Client side: filter collected data to only include known variables

+ * Server side: prevent cross-referencing of anonymous data

+ * Server side: publicize server setup once it is ready, to get feedback

+ * System testing prior to deployment - verify that only anonymous data is collected

+ 

+ == After deployment ==

+ 

+ * Publish data analysis once collection is up and running.

+ * Liaise with developers and partners: has the data actually been useful? Circulate the results of these conversations in order to assess whether the service has value or not.

+ * Review metrics to see which ones don't need to be collected on a long-term basis.

+ 

@@ -0,0 +1,256 @@ 

+ # Metrics list

+ 

+ A tentative list of the metrics Fedora Workstation might want to collect.

+ 

+ 

+ ## System details

+ 

+  * Fedora version - needed to interpret other results

+  * X11/Wayland - to assess Wayland success

+  * GNOME/Classic session - to determine whether classic mode is important

+  * Form factor - to inform hardware enablement priorities

+  * HSI level - to inform HSI and device security design (see https://fwupd.github.io/libfwupdplugin/hsi.html)

+  * Secure boot status - to inform device security strategy

+  * Virtual or bare metal - needed to interpret other results

+    * If virtual, virtualization platform - to inform engineering priorities

+  * Chassis-type (whether this is a laptop, server, tablet, etc... see "hostnamectl status")

+ 

+ 

+ ## Hardware

+ 

+ To inform hardware enablement priorities and improve presentation of hardware details (by enabling us to build a hardware metadata database - https://github.com/systemd/systemd/blob/main/hwdb.d/20-dmi-id.hwdb)

+ 

+  * Manufacturer

+  * Model

+  * Memory

+  * Disk capacity

+  * CPU

+  * How many displays

+  * For each display:

+      * Physical size

+      * Resolution

+      * Scaling factor

+  * How many batteries

+  * Remaining capacity of each battery

+  * Input devices present

+  * Peripherals present

+    * Type

+    * Model

+    * Connection type

+  * Printers added

+ 

+ 

+ ## Graphics

+ 

+ To direct graphics development work.

+ 

+ * Graphics hardware

+ * Driver

+ 

+ 

+ ## Disk configuration

+ 

+ To inform future configuration decisions and installer design.

+ 

+  * Filesystem

+  * Partitioning scheme

+  * /home on separate volume/partition?

+  * Disk encrypted?

+ 

+ 

+ ## Performance data

+ 

+ To direct future performance work.

+ 

+ * Disk space used/free

+ * Disk space used by Flatpak

+ * Memory load stats

+ * Memory footprint of specific components - gnome-shell, PackageKit, etc

+ 

+ 

+ ## Updates & upgrades

+ 

+ To inform: updates and upgrades design, decisions about support periods, packaging and testing strategy.

+ 

+  * Kernel version

+  * Install date

+  * Fedora version at install time

+  * Length of time since last update

+  * Date/time of last update check

+  * Updates staged locally?

+  * Automatic updates on/off?

+  * Repos enabled (RPM, Flatpak, Snap, etc)

+ 

+ 

+ ## Problems

+ 

+ To alert developers to problem areas and inform OOM design changes.

+ 

+  * How many oom kills?

+  * Frequency of oom kills?

+  * How many shell crashes

+  * Shell crashes over time

+ 

+ 

+ ## Internationalization

+ 

+ To inform i18n development priorities and design decisions.

+ 

+  * Display language

+  * Format locale

+  * Enabled input sources

+  * Default input source

+  * Country where device is physically located  (this will need discussion)

+  

+ (Ideally we'd be able to cross-reference these - it would be useful to know the input sources used with a particular display language, and so on.)

+ 

+ 

+ ## Codecs

+ 

+ To inform multimedia support efforts.

+ 

+ * Which codecs installed

+   * Where installed from

+ 

+ 

+ ## Apps

+ 

+ To inform app development priorities, gnome-software design, dash defaults.

+ 

+ Installation:

+  * Which apps installed

+  * Format of installed apps

+  * Flatpak runtimes installed

+ 

+ Usage:

+  * Number of times each app launched

+  * Length of time each app running

+  * Length of time each app focused

+ 

+ Defaults:

+  * Default browser

+  * Default mimetype handlers

+  * Default URL handlers

+ 

+ 

+ ## GNOME Shell usage

+ 

+ To inform future design changes to GNOME Shell.

+ 

+ App launching:

+  * Frequency of app launch from search

+  * Frequency of app launch from the dash

+  * Frequency of app launch from the app grid

+  

+ Window and workspace switching:

+  * Frequency of window switching:

+      * Using super+tab / alt+tab shortcut

+      * Using the activities overview

+  * Frequency of workspace switching:

+      * With keyboard shortcut

+      * With mouse wheel shortcut

+      * With activities overview

+ 

+ Dash and app grid:

+  * Number of default apps removed from dash

+  * Number of apps added to the dash

+  * Times apps moved in app grid

+  * Number of app folders added

+ 

+ General usage patterns:

+  * Number of windows open over time

+  * Number of workspaces open over time

+  * Number of apps running over time

+  * Number of windows per app

+  * Number of times activities overview opened

+    * For each one - how was the overview closed?

+ 

+ Calendar popover:

+    * How many events listed?

+    * How many world clocks listed?

+    * Weather indicated?

+ 

+ Search

+  * Which search providers enabled

+  * Number of times items (results or "more" link) from each search provider selected

+ 

+ 

+ ## Desktop configuration

+ 

+ To inform desktop development priorities and default desktop configuration.

+ 

+ Multitasking settings:

+ 

+  * Workspaces span displays?

+  * Window-based alt tab?

+  * Fixed number of workspaces?

+  * App switching: all workspaces or current workspace?

+  * Focus on hover?

+ 

+ User accounts

+ 

+  * How many regular user accounts

+  * How many enterprise login accounts

+  

+ Sharing settings

+ 

+  * File sharing on/off

+  * Remote desktop on/off

+  * Multimedia sharing on/off

+  * Remote login on/off

+ 

+ Other settings:

+ 

+  * Number and type of online accounts enabled

+  * Accessibility features enabled

+  * Location services enabled?

+  * Default apps? (web, mail, calendar, music...)

+ 

+ 

+ ## Shell extensions

+ 

+  * List of enabled extensions

+    * Installation format of each

+ 

+ 

+ ## App-specific metrics

+ 

+ To inform future design changes to apps and desktop user docs.

+ 

+ ## Settings (gnome-control-center)

+ 

+  * Number of times each panel opened

+ 

+ ## Files (nautilus)

+ 

+  * Number of bookmarks

+  * Number of starred items

+  * Number of remote filesystems mounted

+  * Times the following locations are opened: recents, starred, other locations, other xdg folders

+  * Preferences:

+      * Sort folders before files

+      * Expandable folders in list view

+      * Action to open items

+ 

+ ## Boxes (gnome-boxes)

+ 

+  * Number of boxes

+  * Is the icon-view or the list-view used?

+  * Are VMs configured to run in the background?

+  * Are VMs installed with UEFI or legacy BIOS?

+  

+ ## Connections (gnome-connections)

+  

+  * Number of remote connections

+  * Number of remote connections per protocol (RDP vs VNC)

+ 

+ ## Help (yelp)

+ 

+  * Number of times opened

+  * Pages viewed

+ 

+ ## Toolbx

+ 

+  * Number of toolbxs

+    * Type of each toolbx

+ 

no initial comment

1 new commit added

  • Add metrics privacy/transparency checklist
4 months ago

Pull-Request has been merged by catanzaro

4 months ago