OK, did the upgrade. A few observations on the Chromebook we have:
- When "Testing Gestures" The delay is probably 30-45 seconds, nowhere near 10-20.
- About 1 in 5 times MS partially crashes when entering the Test Gestures mode (the screen goes blank for a few seconds then comes back to either the songs page or to the gestures settings page - a bit inconsistent on where it recovered to).
- It cannot reliable recognize any of the gestures. It does best with head turning and head up/down but even then it's probably 75% at best. I started to fool with the configuration options (size and duration) but test phase was unreliable enough I gave up. The lip move and eye winks were very low (never did get the lip move to work). While the head turns are most reliable, my wife thinks this would be the worst gesture to use because of the obvious unwanted recognition during normal music playing at concerts, etc. Not an MS issue but certainly an issue.
This is a very interesting idea/concept but at this time my wife cannot use it because it can't reliably recognize the gestures. If there is anything we can do to help test and/or debug, we'd be more than happy to.
As I mentioned the fixes I just implemented that are included with version 3.8.24 will address the delay you are seeing. It should initialize instantly with the next update. As far as the recognition, it works fine on my Chromebook, so you'll have to let me know if it's any better with the next update. It's very important your face is mostly pointed towards the camera and optimally in the center of the frame. The lighting also matters. Additionally, try to be within a couple of feet to the device (within a foot or so is optimal) in order to increase the accuracy of the detection. I've also seen issues with the orientation of the device - it works best in portrait. Flipped portrait can cause issues on some devices. Landscape should work, but it's still better in portrait I believe.
Mike
Just tried this for a "laugh"
Works ok for "raised eyebrows", can't use most of the other events because I might accidentally do them ! Also don't think open mouth will work if one sings! (but haven't tried it)
"Tilt Head Up" to go back a page also works for me.
Thought I'd try "Tilt Head Down" for forward a page and "Tilt Head Up" for back a page but found that having tilted my head one way, it then triggered the reverse action when I "removed" the tilting.
So going to try Eyebrows for moving forward and tilt up for backward.
The duration slide is a bit small for fat fingers if one is trying to set an exact time (but then again, it probably doesn't matter if you are 0.1/0.2 out from where you thought you would set it).
Thanks for adding this feature Mike; I've no idea if it will work for me but I'm sure it will for several others
Cheers
Geoff
(12-22-2023, 05:57 AM)Zubersoft Wrote: [ -> ]As I mentioned the fixes I just implemented that are included with version 3.8.24 will address the delay you are seeing. It should initialize instantly with the next update. As far as the recognition, it works fine on my Chromebook, so you'll have to let me know if it's any better with the next update. It's very important your face is mostly pointed towards the camera and optimally in the center of the frame. The lighting also matters. Additionally, try to be within a couple of feet to the device (within a foot or so is optimal) in order to increase the accuracy of the detection. I've also seen issues with the orientation of the device - it works best in portrait. Flipped portrait can cause issues on some devices. Landscape should work, but it's still better in portrait I believe.
Mike
Understood about recent release, etc. Just observing, not complaining - it is what it is. I assume that MS has zero to do with the facial recog library's capabilities. As far as distance, both my wife and I were at the "natural" distance for using MS in a musical setting. I bet it would work way better if we were closer and 1' is very close but that is not practical for the use-model. Our faces probably filled perhaps 1/5 the frame and we were about 3' away (hence the small -frame fill) (imagine sitting in a chair with the chromebook on a music stand.) We'll keep an eye out for updates but if you are stuck with the google lib's features, I assume it would not improve much?
Scott
Three feet away may not work well unless you have a very good camera on the device. I can use the feature at three feet away with my Galaxy Tab S8 Ultra, but it has a very good camera on it. Certain gestures don't work as well at that range though, such as raising eyebrows. You can also try increasing the image resolution to see if that helps (but this is also dependent on what the device's camera supports, as you can pick a high resolution but the library will fallback to the highest resolution your camera supports). If you can test it about 1.5 feet away, let me know if it works properly at that range. I just want to have a sense of whether there is a potential solution for your device or not.
I can't change how Google's library works, but the one thing I have control over is the threshold for deciding when a gesture is occurring or not. If I let users have control over the exact parameters for each gesture (so you can set the threshold at which the gesture triggers), this may allow you to find a solution. It may also result in a lot of false positives though. I'm going to add these more advanced settings when time allows.
Mike
Had to disable face gestures (Raised Eyebrows/ Tilt Head Up/ Tilt Head Down)
While this all worked fine when playing a setlist (including looking down sideways at my fingers on the uke frets!), the tablet kept turning pages when I was trying to compare song versions on my desktop and tablet when side by side. Strange really, as I didn't have the side turns enabled.
Geoff
I'm currently not checking to see if the user is looking directly at the device. This can lead to issues like you are describing. I may change the logic to require that the face is mostly looking at the camera in order to trigger gestures.
Mike
Thanks Mike
Merry Christmas and a Happy New Year to you and your family (and anyone who reads this!)
Geoff
Merry Christmas and a Happy New Year to you and yours as well!
Mike
What face gesture are other people using?
I've tried the Smile gesture which is pretty good but was just wandering what other people's preferences are.
Which one is the best in your opinion?
(12-31-2023, 01:58 PM)Frankos Wrote: [ -> ]What face gesture are other people using?
I've tried the Smile gesture which is pretty good but was just wandering what other people's preferences are.
Which one is the best in your opinion?
I use :
Smile
Mouth Open
Lips Left
Lips right
With reliability that is really quite good, but for me the most reliable, without any doubt, is Mouth Open.
I don't know why but I cannot get Mouth Open to work at all.
I've tried it on my Android Tablet and on my Android Phone, no response on both. Other face gesture work fine though.
Hi Mike
Retrying face gestures now that you have updated code to ignore when not looking at the camera.
It's better but Istill get a lot of false detections on head up/down.
I have the metronome turned on on all my songs.
When using a move to back to a 2 page "previous song", the metronome is not started
(Actually, this is also true when using tap actions to go back to last page of a multipage song)
I'm guessing the metronome code is only checked when the first page is displayed.
I'm not sure you need to change the code - just making you aware.
Looks like the only gesture I can probably live with is raised eyebrows to "scroll down or turn page if at bottom"
Actually, this is effectively a scroll as it never seems to turn to the top of a page (possibly because "always load whole setlist" is checked)
What is predictive scroll and turn?
Cheers
Geoff
Hello Geoff,
The problem is that I don't want to limit the angles too much, otherwise it can be very difficult to trigger gestures if the user isn't squarely looking at the device. Perhaps I'll eventually have to provide more options to configure this. I might advise not assigning any actions to the head up/down then, as then that shouldn't cause any problems.
You are correct the metronome is only started on the first song of a page - this is by design based on feedback I've gotten from users.
You would have to trigger the gesture twice - once to scroll to the bottom, then again to turn the page.
Predictive scroll and turn is similar, but instead of you setting the amount to scroll, it tries to figure out how many pixels to scroll in order to jump to the next measure.
Mike
Thanks for the explanation
I wear glasses so winking doesn't work
Turning Head left and right won't work because I may look around the audience
Open Mouth no use because may be singing
Doesn't easily detect lips left/right and doubt I can sing while contorting my lips for what seems a long time.
If you advising that I don't use head up/down then that only leaves eyebrows!
I can't think of any other actions one can do with one's face (tongue out? - but the audience might not appreciate it!)
So it looks like I'll have to stick to just using eyebrows.
While it is not as versatile as I would have liked, well done for implemementing this as I'm sure many others users will find it very useful.
Cheers
Geoff