Orrrrr, would pro-active actions work? As in, "players have to hit the boss with particular abilities which will result in one of several responses from the boss, which then determine what players gotta do next".
This is closer to something that works in my experience.
A different way to think of this might be what I do for training mode in fighters.
Boss has Inputs A, B, C, D, E chosen completely randomly.
Boss will do something different when B follows A, than when B follows B. That gives you 25 options.
But let's say that whatever follows BB takes so long that it 'skips a count'. So you can only see BB twice in a row if it 'selected B four times'.
Then add some player inputs in terms of what abilities they hit with. You can do it by expanding it to FGHI, or just 'if you hit the boss a certain way while it's selecting B it will change B to A'. So you could 'make sure that the boss only chooses AA, AB, AC, AD, AE, BUT you could only do this if you 'knew it was going to select B' or 'did the ability because you saw it use CA and know that AB will kill you but AA wouldn't' (while still having to worry about AC, AD, AE).
A human will adapt to this using their intent, status of the team, and teamwork, in a different way than most bots will, but still not be in a situation where they have no control and zero predictability, nor (if you found this to be negative) in a situation where 'absolute randomness whenever' was possible.
In this situation a Tracker would do much less for you in terms of 'figuring out the boss' because you would already know the odds of any specific thing happening (they're all about 1 in 25 and you can reduce them through your own actions when your abilities to influence it aren't on cooldown, to get it down to 1 in 5).
But now the player's side is constantly finding having to find new ways to adapt to whatever is happening, and the Tracker is only doing what I (personally) am used to it being for. Allowing the player to look back at a specific point in a long fight and think "Oh, I could have done this better even considering the randomness", or of course 'Dammit Carl, you missed the proc six times this fight, you know you're supposed to leave the adds to someone else when EB is happening' (some people need a tracker for this, some don't).
Devs could then 'lower the odds of CA, technically, by making it so that CB and CC take so long that any CA following it never does CA'. Maybe if it's like, a dragon, CCC causes it to do an extra long breath blast, etc etc.
Then just set whatever win conditions you like and change 'damage' to be 'something that affects the selection'.instead of 'something that kills the boss'. Cooldowns alone would generally mean that you need the same number of people anyway, but now you've removed that visible 'Input->Output' part of the Runes thing.
Another useful part of it in Ashes (for example) is that you COULD theoretically spawn a boss with a different subset of A, B, C, D, E each time, and the combinations change, without players having to literally guess every moment. e.g. if you see Ice Breath, you know that at least two of the boss' 'slots' are 'Push' and 'Ice', so you can immediately start guessing/adapting to what the rest of their kit will be (this has nothing to do with bots, they would do that too, this is just a bonus).