Time To Drop The Eye-Patch: UAT Needs Binocular Vision

The User Acceptance Testing, or short UAT, is one of the critical steps in implementing a new piece of software, so time management or payroll for our purposes. It takes place once the setup has been finalised and before full-scale testing (or “parallel runs”) take place. Its purpose is to identify situations in which the software does not behave as expected. Quite often this is due to setup rules not being sufficiently refined or conflicting with each other.

In order to identify these situations, so-called scenarios are built. They simulate real life situations, from the very basic all the way up to the very complex ones. Usually, the client is entrusted with building the scenario on the basis that it is they who know their policies best and know from experience which rules are likely to conflict with one another. On the surface of things, this seems logical: the client will indeed know which rules are applicable to which population, which one should be prioritised in case of conflict (e.g., sick leave over annual leave), etc.

However, I believe this approach has one major blind spot: what about the behavioural oddities or flaws inherent to the software itself? What if, for example, the time management software notoriously struggles with sick leaves for night shift workers? What if the payroll software is known for having difficulties calculating the proper sick pay when a multitude of seniority rules come into play? These situations are not linked to the client’s policies, but rather to the provider’s software. It would be reasonable to assume that a vendor who has implemented the same piece of software with multiple clients across several industries understands its strengths and weaknesses. Therefore, they should provide new clients with default scenarios to identify potential issues specific to the software.

Unfortunately, that does not seem to be the case with the majority of vendors I have worked with so far. This lack of proactive support often leads to unpleasant surprises after going live, when known issues manifest as erratic behaviour. This not only leads to additional work on both sides but also to frustration – both of which could have been avoided.

So, time to use both eyes for the UAT: the vendor's eye on the software and the client's eye on the policies. Do you agree? Please let me know your opinion in the comments.