Zubersoft Wrote:In this case, yes, I should have tested the application on multiple versions of Windows 10, but I wasn't even aware that I was accessing functionality that wasn't supported across all versions of Windows. I was just calling a simple method in Visual Studio, and there is no information in any of Microsoft's APIs in Visual Studio indicating there is a problem. Calling IsLoaded on a UI component seems like an innocent thing to do that should be included in the base SDK.
Trust nothing from Microsoft. At minimum, "smoke test" a new version with each supported version of Windows before making a release; this would have caught the recent failure. I make it a practice to step through all changed code in the debugger to ensure that it behaves as expected.
Zubersoft Wrote:3. I was just suggesting that it could be beneficial to have willing users test out builds before the official release. As I mentioned earlier, there are a huge number of settings, and with user generated libraries, it's not plausible for me to test every possible combination.
Beta testing can indeed be useful, but there's natural tendency for it to reduce the developer's attention to testing. Unless you beta test with thousands of users -- which is not practical -- there's no reason to believe that your beta testers will check significantly more combinations than you would.
Zubersoft Wrote:4. I will plan on setting up VMs for specific versions of Windows 10 and just run them with no internet. I'll also set up the updates to be postponed just in case.
Maintaining control of your test environments is essential.
Zubersoft Wrote:I will do everything I can in the future to prevent these kinds of issues when possible. The issue some users are experiencing with the app not loading now (separate from the issue with incompatible API call) is something that I'm not sure I could ever prevent. I have tested on multiple versions of Windows on 6 devices and none of them demonstrate this issue. Users that have been experiencing it have said it suddenly goes away. Others update Windows 10 and the issue vanishes. I have one user who is experiencing this issue who is testing builds for me, but so far no answers have been found. It's issues like these that, if I can't reproduce it, I can't prevent it from occurring. I can only respond proactively to try and address the issue for the small number of users who are encountering it.
A practice that has helped me enormously is to build error logging and progress logging into each application. When any error or inconsistency is detected, a date-and-time-stamped entry is written to the error log. There's also a "log debugging info" option that the user can enable; when this option is enabled, every non-trivial or "risky" function in the application writes an entry to the errorlog.txt file on entry (noting its parameters) and on exit (noting its results). Complex functions write entries documenting intermediate results. When a user reports behavior that you cannot replicate, they may have an errorlog they can send you; if not, you can ask them to enable "log debugging info", repeat the failing scenario, and then send you the resulting error log. Depending upon the size of your application, this may seem an overwhelming task; start by introducing logging to the top ten most complex or "risky" functions, and then address the rest gradually over time.
Dave