About nine months ago, I saw my first demo of computer vision – an AI navigating a standard webpage without user assistance. It was a rough prototype that didn’t work perfectly, but you could already sense its potential. At the time, I thought, when they get this right, it could spell the end of e-learning as we know it.
Well, last week, they kinda got it right. I spent my weekend exploring the results, and honestly, it’s both brilliant and terrifying.
The demos and use-case examples for Operator (the tool in question) are primarily geared towards B2C tasks like booking holidays. However, I immediately saw its application in B2B scenarios – think form-filling tasks like, “Operator, submit my expenses claim.”
But here’s where it gets controversial: could Operator complete your e-learning for you? Naturally, I had to test it out.
Putting Operator to the Test
Test 1: Navigating a Simple E-learning Module
For the first experiment, I chose a well-designed “comic book” style e-learning module built with Articulate tools (by far the most popular tools for creating self-paced online learning). It must be pointed out that this module (like the others used in this blog) was a lot more polished than the usual workplace CBTs (computer-based trainings). I’m not here to cast aspersions on the quality of these examples, they are just the first ones I found free online (and I haven’t asked permission…).
I asked Operator to navigate through the interaction and complete it.
- It prompted me to see if I wanted to add my name to personalise the experience – I told it to use ‘Ben’.
- It paused once to ask for guidance on a multiple-choice question. I told it to use its best judgment and proceed without further queries. Operator used that prompt to complete the rest of the exercise unprompted.
- The format of the e-learning is well presented, but pretty samey – you’ll get the idea quick enough. I haven’t edited the video, but the video output from Operator itself is screenshot based, so doesn’t show every interaction it made with the screen.
Here’s how it performed:
Test 2: Tackling a More Complex Module
Next, I upped the difficulty. I tested Operator on a module about Climate Change from my old employer, Learning Pool, built using the Adapt framework. Adapt courses are more nuanced – users typically need to click every interactive element on the page to mark it as complete. This should represent a sterner test than most ‘standard’ e-learning.
- Operator handled this admirably. When prompted with multiple-choice questions, I instructed it to get all answers correct, which it did.
- It got stuck navigating once, but opened itself a new tab and carried on where it had previously got to (most e-learning works like this and it recognises the Operator as the same user as previous).
- There are lots of different types of interactions in an Adapt module, many more than in most basic CBTs. It was good with these.
- I haven’t made any cuts in the video but did stop it about 20 seconds from the end as it got pretty repetitive.
Watch its performance here:
Test 3: Attempting a Proper Assessment
To push Operator further, I turned to Learnosity’s demo assessments on the Solar System. These included a wide range of question types. Here’s what happened:
- Operator successfully completed most tasks but struggled with some of the more nuanced questions, especially those that required precise sequences of clicks.
- It couldn’t complete the second drag-and-drop question. The draggable elements reset dynamically after each interaction, which Operator couldn’t handle. I’ve edited the video here as it tried numerous times (and in numerous ways) before giving up.
- It also couldn’t complete audio-recording tasks. Operator prompted me to take control but lacked the capability to process audio inputs (or generate its own).
The results show the current limitations of AI in e-learning – but I was blown-away by its performance in some really quite tricky interactions:
Test 4: AI vs. AI Roleplay
In a final experiment, I used Operator to engage with an AI-powered roleplay tool – AI vs AI. This required login credentials, which I entered manually. Operator performed the roleplay seamlessly, scoring high with minimal input from me. While I can’t share a video due to proprietary reasons, this test highlighted how effortlessly Operator can handle tasks that simulate human interaction or require significant text input.
Implications for the Future
Operator-like tools will become indispensable in the workplace. Banning them outright won’t work, as they’ll be too useful for mundane tasks. For statutory training, where the primary objective is an audit trail rather than actual learning, tools like Operator may render traditional e-learning irrelevant. Businesses will need to rethink their approach.
For now, I foresee several immediate reactions:
- Policy Updates: Companies will create policies forbidding the use of Operator-like tools for completing mandatory training, potentially making it a fireable offense.
- Human Verification: Employers may add attestations or human-verification steps, such as requiring users to confirm they are not using AI.
- A Return to the Classroom: Some businesses may shift back to in-person training to ensure compliance.
- AI-Driven Prevention: Instead of relying on training to prevent bad behaviors (like cheating on expense reports), businesses could implement better input controls and automated checks to prevent errors or fraud.
Rethinking the Purpose of Training
The rise of Operator underscores a deeper truth: much of today’s statutory training doesn’t achieve its goals. The focus has been on checking boxes for compliance rather than creating meaningful learning experiences. The real opportunity lies in shifting from training-as-a-formality to training-as-transformation.
AI tools like Operator are a wake-up call. They force us to reevaluate the value of our training systems and find ways to make learning genuinely engaging and effective. The age of “click-next” e-learning is coming to an end, and honestly, it’s about time.