Performance management systems - how to run a successful pilot

Roly Walter
30th April 2020

We occasionally get requests from prospective customers to run a pilot of Appraisd, our online performance management system.

Quite reasonably, they want to check for themselves that it will improve manager-employee conversations, help people define their goals and give and receive feedback before they commit to the cost of a full roll-out.

Rather than just putting up a demo site and sending out a few emails, we have found that following a comprehensive pilot programme template leads to much greater success. In all cases, the client has been able to fully appreciate the value of Appraisd and what it will and won't do for them. A successful pilot leads to a successful roll out. Since a pilot involves not just the Appraisd system but also many hours of our Customer Success service, we believe it makes sense to charge a fee for any pilot: this way we can ensure we dedicate full resources at the project to give the client the most realistic outcome.

Our Customer Success team has completed many successful pilots and refined its playbook over the years. Here we've extracted the key points into a guide that anyone can use.

How you'll measure success of the pilot

It's vital to know how you and any other stakeholders are going to evaluate the pilot. Make sure you get agreement from everyone so there's no arguing later.

  • Maybe you'd like to know if the system is really going to change manager behaviour for the better? So you need to ensure your pilot is long enough to measure this.
  • Maybe you feel you've definitely settled on the right product, but you'd like to try a small roll-out to iron out any technical or cultural issues before unleashing it on everyone else?
  • Maybe you'd like to evaluate not just the system, but also your new approach to performance management? Make sure you're able to split the feedback from users between these two areas so you don't end up throwing out the system just because your review forms were far too complex.
  • And as ever, you should be aware of your organisation's long-term people-related goals. Are you trying to shift the culture? Are you growing fast and need to set good practices in place? Are you trying to modernise? Think about how the pilot will help you work towards those.

Try writing some success statements which can be used as questions in a survey later, such as:

  • Pilot users found the system easy to use without much training
  • Where pilot users got stuck, help was easy to find
  • Pilot users found it easy to complete the challenges set for them (see below)
  • Pilot users feel the system would help them conduct their 1:1s with their team members
  • Pilot users feel the system gives them more ownership of the review and feedback process and clarity on their goals
  • The onboarding process with the vendor was straightforward and well-managed, and a good working relationship is easy to imagine

What's involved in the pilot

Having established the success criteria, move on to the detail. You need to establish the following, assuming any contractual discussions have been completed:

  • Key point of contact on the client and vendor side. How often should they check-in? We suggest weekly as a minimum.
  • Pilot users. Who are they, how will they be briefed? What contact directly with Appraisd will they have? We find it helpful if we can speak directly to pilot users if necessary to help them out.
  • Dates: briefing, launch, cut-off and evaluation. We prefer a short, intensive pilot with a small, committed user group rather than a long pilot involving many people.
  • Anticipate technical issues. Ensure anything that could cause unnecessary friction are removed, so make sure that single sign-on is enabled, that email addresses are whitelisted and that the employee data is accurate.
  • Appraisd configuration. Try to disable features that require a lot of coordination in timing, such as cascading objectives. Keep review forms as short as possible - you can always add to them later. Remove, rather than add, multiple levels of sign off and approval: I try to encourage clients to take an optimisic approach and assume people will generally try to do the right thing and so the workflow can be kept to a minimum. Enable features that encourage little-and-often usage, such as instant feedback and check-ins.

Pilot users' challenges

Be clear on what the pilot users need to do during the pilot, and how much of their time is required. Send out a list of tasks to achieve, and back them up with a brief webinar to answer any questions. A typical set of challenges for Appraisd pilot users includes:

  • Login (and say hi to the Appraisd Customer Success team over web chat!)
  • Set up 3 new objectives for yourself
  • Give 3 peers some feedback or praise
  • Book and conduct a check-in with everyone you manage within 3 days (agree this date with all pilot users). If you're a team member, complete your preparation 2 days beforehand.
  • Use the video calling features if working remotely
  • View your Team page to see who's been doing what
  • View your direct reports' objectives and add a comment to each one

For Administrators, we suggest you include the following:

  • Observe the progress the pilot users are making in setting objectives by using the metrics and org-chart reports
  • Get comfortable with adding and updating users
  • View an individual's profile and understand how to use it to understand their engagement with the pilot
  • Produce a report on the ratings users have given to a multiple choice question in a check-in
  • Produce a report on progress made on objectives

Common mistakes to avoid in running a pilot

In general, try to make sure the pilot doesn't fail for reasons other than the software. Avoidable reasons for failure include:

  • Lack of clarity for pilot users on what they need to do, by when
  • Lack of commitment from stakeholders to the aims of the pilot
  • Technical problems such as emails going to spam, or people unable to log in
  • Review forms were too complex / do not tackle what really matters
  • Guidance too longwinded
  • Workflows on reviews are overly complex
  • Pilot user group not committed to spending time conducting a check-in during the pilot period
  • No clear project manager overseeing the pilot
  • No clear Administrator committed to keeping the system up to date, producing reports

I hope you find this article useful. If you'd like to talk to us about a pilot of Appraisd, please contact us: +44 (0)20 7202 7979, or email