The six step framework: Build a Pilot

With what we discussed in the last two chapters (Assess and Identify) you’ve now been able to identify your champion use cases. Now you’re ready to go into project mode and start building a pilot. 

You’ve also prepared yourself and you’re now familiar with the basics of AI and what you can expect from an AI solution. And with the rest of your organization getting training to start using AI and the tools that are going to be introduced. Now is a good time to very specifically write down what you expect from the champion AI tool.

The requirements document

A requirements document is a clear description of what your AI solution needs to do, how users will interact with it and how it will interact with the rest of your infrastructure. The requirements are written in a “functional” way. This means that each of the requirements is written as something that the software needs to do, a task it needs to complete, a way in which it interacts with customers or data or machines or …

It’s best not to describe technical solutions in the requirements document unless it’s absolutely vital. Often times, leaving out the technical details will give a more creative result by the development team (a cheaper, faster of more stable approach). On the other hand, it’s a good idea to include an expected timing for certain features or requirements. Or to give some expected usage statistics and how they will grow in the future.

A requirements document gives the team that needs to build the software a clear guide on what needs to be built and removes a lot of uncertainty and doubt.

‍

Example requirements document for a Netflix-like service

…

Streaming video

• The user needs to be able to watch the available video content from a recent browser  on different devices
• In this phase, the supported devices should be: laptop, desktop computer, mobile phone (both Apple and Android) and tablet (both Apple and Android)
• In later phases this will need to be extended to gaming consoles
• Once the video starts playing, users can pause, fast forward, fast backward or stop the video playback.

Video recommendations

• The system needs to recommend video’s to the user based on the content he watched earlier
• To make a good recommendation, look at the viewing behavior of other users who have watched the same content.
• Recommend videos to the user that he has not watched yet, but similar users have (=watched the same content)

Recommendation approval rating

• Users can indicate whether a video recommendation was to their liking or not
• Users can indicate their approval by clicking a thumbs-up or thumbs-down button
• User approval feedback is collected so it can be used to measure the recommendation engine performance

…

‍

With this requirements document ready, it’s now time to start building a pilot! There are several ways to move forward with building a pilot: use external consultants, make your own team or do a combination of the two.

External consultants or internal specialists

Working with external consultants to build a pilot (and work on the subsequent iterations) is by far the easiest route to take. You buy in all the expertise you need to get the project off the ground. This is especially true when you work with a large organization like Cronos - which we at SUMSUM.digital are a part of - since we have people specialized in almost every system or discipline in house. Using external consultants gives you speed, expertise, flexibility and experience.

You can also go a completely different route and that is building your own specialized team in-house. This takes longer to set up - especially if you don’t yet have a development team - and requires a lot of investment both in time and money. You need to find the right talent and wait for them to end their notice period at their previous employer. You’ll also need to give them the right tools and additional training and they still have to learn to work together. On the other hand, in the long term, they will be more intimately familiar with your business and internal stakeholders.

There is also some middle ground: build your own team but kick-start it with consultants. This is a very interesting route to take. The consultants can start work on the tool you envision, build up the basic structure of the team you need and provide all the necessary expertise. In the meantime, you can start recruiting the talent you need and replace the consultants one by one until you have an internal team.

So we’re done now?

This is the point where the difference can be made between a successful and a failed AI project. The best path to failure is to conclude that since the technical teams now know what to do, you just have to wait for the end result and your work is done.

To make that AI project a success, some very crucial work starts now: preparation for the iterations (see next chapter) and roll-out of the solution. Let’s start with a maybe boring but recently even more important topic: data and AI governance and compliance!

Data and AI governance and compliance are the internal systems and processes you set up to keep track of what data and what AI models you use, what you use them for and how you use them. This applies also for software you buy. It’s very important to integrate the all of these governance processes into your overall AI strategy and approach.

The EU AI Act has become law just recently and introduces - on top of the GDPR regulations - another layer of due diligence you need to perform. This AI Act introduces risk levels - risk with regards to EU citizens - to the usage of AI models in specific tasks. For some tasks there is no risk and no requirement for you other than documenting that there is no risk (e.g. monitoring industrial processes). The risk levels go up to “unacceptable risk” in which case you’re not allowed to operate the AI tool that poses this risk. It’s your responsibility to assess and document this risk and, in the case of “high risk”, take the necessary precautions to minimize the risk and document all the steps you’ve taken.

Dealing with the GDPR and EU AI Act regulations is something on which many pages can be written and it will be far out of scope for this series of articles but, you will need to start thinking about and setting up all the processes to deal with the requirements of these new rules. We are not specialists in risk, governance and compliance ourselves, we rely on several experts to help us out at our customers.

Introducing compliance and governance processes is hard work and it will take some of your available resources to set this up properly, but it’s something that you’re going to have to do anyway. The good news is that, together with meeting your legal obligations and reducing risk, having good governance processes is also good business! It’ll increase the performance and reliance of the AI models you introduce into your business and it’ll give you and your organization better insights into and a better understanding of the AI products you buy or build.

In the next chapter, iterating, we’re going further into monitoring, measuring and and determining KPIs for your project so you can match the resources you’re investing to the expected gains. In other words we’re going to guard your ROI.

‍