@npmccallum asked us if we could help improve Bluetooth testing. The idea we came up with is to add some test cases covering particular Bluetooth workflows that can be included as Optional level tests in the desktop validation matrix.
Here's his list of flows to cover:
Headsets: 1. Pair device 2. Newly paired bluetooth device should be the default audio source (sound settings) 3. If the device supports A2DP, A2DP Sink should be default (sound settings) 4. Run the audio test ("Test Speakers" in Sound Settings) 5. Test music playback 6. If the device has a mic, make sure the mic works in a VOIP/meeting app (ex. Bluejeans) 7. Check any media buttons on the headset (play, pause, back, forward) 8. Power cycle the headset. Make sure device reconnects automatically. Run tests again. 9. Reboot system. Make sure device reconnects automatically. Run tests again.
Keyboards: 1. Pair device 2. Test each key 3. Test any media buttons present 4. Power cycle the keyboard. Make sure the device reconnects automatically. Run tests again. 5. Reboot system. Make sure device reconnects automatically. Run tests again.
Mice: 1. Pair device 2. Test mouse movement 3. Test all mouse buttons 4. Power cycle the mouse. Make sure the device reconnects automatically. Run tests again. 5. Reboot system. Make sure device reconnects automatically. Run tests again.
These should be turned into test cases - following the SOP , also use existing desktop test cases as references. Then the test cases can be added to the matrix. Thanks!
Nat, something that would help is if you can provide some info we can turn into a debugging guide along the lines of https://fedoraproject.org/wiki/How_to_debug_Wayland_problems - that will help people file better reports when they find bugs.
Metadata Update from @adamwill: - Issue priority set to: 1
Metadata Update from @adamwill: - Issue tagged with: test cases
I've asked @dzickus to work up a debug page for bluetooth.
Hi Adam. Would you also add audio bluetooth like phone and some sound speakers as well. Although they work as intended, the missing setting like volume and proper assignement of icons are one of most complains. Additionally, file transfer from bluetooth device seems broken.
Hey, adding myself to the ticket. I am in the new HW enablement team and there is likely some overlap with what we are trying to achieve. For example one thing I am personally going to focus on is (semi-)automated testing of laptops where bluetooth tests also play a role.
Anyway, need to run out, will have a closer look at this later.
Does there need to be consideration for different verisons of bluetooth and profiles for input devices? For example I have lots of issues with my Logitech MX Anywhere 2 which apparently using 'smart bluetooth', I've not found a good guide to troubleshooting it yet and instead am using the logitech wireless dongle or the time being.
I'd like for things to be intentionally broad. One thing that is important is how to measure the user experience. Many users won't know or care to know the distinction between different types of devices. They just want to know if their keyboard works nor not.
I just tested Logitech Wireless combo mk220[1] . Tested all the keys of the keyboard and the mouse. The scroll up and down wheel doesn't work. The function keys work fine.
[1]https://www.logitech.com/en-in/product/wireless-combo-mk220
Hi Adam,
Sorry for the late response Trying to find time to put this together. Clicking on above links for reference takes me back to this doc again. Any better links for SOP or 'desktop validation matrix'.
Right now it seems you steps above is a good start and we just need a 'debug' guide (like the Wayland example). Is that the expectation?
We might need some steps for firmware rollback as Intel seems to be getting sloppy with their firmware releases lately. But we need a baseline first.
Thoughts?
Cheers, Don
That's weird, the link targets are correct in the comment source, but they're rendered as links back here...how odd. Anyway, the correct URLs are:
https://fedoraproject.org/wiki/Template:Desktop_test_matrix https://fedoraproject.org/wiki/QA:SOP_test_case_creation
@dzickus ping? what's the current status on this? Thanks!
@dzickus Any chance we can get that bluetooth debug page? I think it would make a big difference in quality overall and (hopefully) reduce your workload.
@npmccallum Sorry I have been busy with other stuff to sit down and put that together. I agree a couple hours of work on that page could save hours in the long run. I will try to focus on it on Friday as next week has to much going on. I will at least get a rough template to start and poke at it over time.
Ok, here is a start here for debugging. https://fedoraproject.org/wiki/How_to_debug_Bluetooth_problems
Excellent! Thanks @dzickus!
@adamwill It looks like we have enough to get started on some official tests. What else do we need?
Metadata Update from @adamwill: - Issue assigned to sumantrom
@sumantrom , I believe this was for you - of course, if there's anyone else (e.g. a contributor) interested in taking it, that'd be awesome also.
@sumantrom , if you're too busy for this, maybe @coremodule or someone else could take it?
Metadata Update from @coremodule: - Issue assigned to coremodule (was: sumantrom)
@npmccallum
Would you mind reviewing this? If it's along the lines of what you were expecting, I will publish this along with several other cases to the wiki. [0]
[0]https://fedoraproject.org/wiki/QA:Testcase_bluetooth_headset
@coremodule I made one small change on the wiki. Other than that, it looks good.
well, hey, we got one test case out of this, at least. That's not nothing! I went ahead and added the test case to the appropriate category and to the desktop test matrix. Now on the current pace we have another ten years to get keyboards and mice done...
Login to comment on this ticket.