I appreciate them writing this and taking it on the chin here but honestly there is so much in here that’s pretty damning about their design process. Nadu was a massive mistake but one that seems negligibly correctable.
Mistake 1: not enough playtesting!
Majors describes the testing they did. He cites no actual numbers or anything but I think that makes it pretty clear how deficient it is/was:
For both Modern Horizons 2 and Modern Horizons 3, we brought in a small group of contractors and worked on the set in a dedicated sprint as a collaboration between that group and a small number of play designers. The playtesting time is more dense, as the group is singularly focused on the set without other responsibilities, but shorter in terms of weeks.
Emphasis mine. I would wager that this is probably 2-3 contractors who played for probably a few weeks with the people who designed the set, a group who is obviously stuck in groupthink and can’t see differently. There’s no wonder they miss issues like this.
Mistake 2: changing cards again without testing
Again we have a card that was changed at the last minute and shipped as-is.
I missed the interaction with zero-mana abilities that are so problematic. The last round of folks who were shown the card in the building missed it too. We didn’t playtest with Nadu’s final iteration, as we were too far along in the process, and it shipped as-is.
How many times does this need to bite them before they just actually playtest every single text change? It’s crazy to me.
Mistake 3: no automatic flags for certain high-risk abilities
They all missed the interaction with 0-mana abilities, OK fine. But why is there no automatic flag for high risk abilities? Off the top of my head:
unbounded triggers
triggers that draw you cards
triggers that put cards directly into play
triggers that occur whenever the permanent is targeted
triggers that give things to all of your creatures
Original Nadu had every one of these, and there are no doubt many many more things that should automatically create a higher scrutiny/testing regimen. They added the 2-times-per-turn cap at the last minute but removed the by-an-opponent limiter!
Software can easily flag cards that should be tested more fully, or recommend problematic interactions. They don’t seem to be doing this at all, instead just accepting a certain % of failure/risk. This is so preventable.
Mistake 4: putting Commander cards in Modern sets
This one is a personal pet peeve of mine and really irritates me more than everything else. This card was designed specifically for Commander yet it went into a set ostensibly for Modern/Legacy.
In one of these meetings, there was a great deal of concern raised by Nadu’s flash-granting ability for Commander play. After removing the ability, it wasn’t clear that the card would have an audience or a home, something that is important for every card we make. Ultimately, my intention was to create a build-around aimed at Commander play, which resulted in the final text.
If the card is for Commander, put it into a Commander set! This is also extremely damning from a design perspective–they removed the flash ability and then didn’t think the card would be played at all!
So much of this seems to be preventable with better processes or using technology in even basic ways, ways they are no doubt not even close to taking advantage of. It’s a shame because these sets could be better and the playerbase wouldn’t have months of crap like this, or a Pro Tour absolutely ruined by a preventable card.
I can forgive a lot of the mistakes they made here; we’re all only human and I’m not going to pretend I’ve never shipped code with a bug that I really ought to have caught. They probably should never change any text without playtesting (unless it’s to make a card strictly worse, and even then you need to be pretty sure about the “strictly”), but I’ll bet they’ve gotten away with that a lot more than we realize.
But I think the cardinal sin here, and the mistake I can’t really understand, is that they changed an ability and “how does this interact with zero-mana spells or abilities” wasn’t in their top three considerations. Like, Cephalid Breakfast is over 20 years old. That type of interaction is not new or obscure; you can’t not have it on your radar.
Exactly, the fact that “interacts with 0-mana abilities” isn’t on a list of mandatory checks is just crazy to me. It just signals that they don’t have any kind of process/infrastructure in place to help them see these things.
Agreed on the red-flagging of abilities. Nadu had so many red flags it really should not have been allowed out untested. If the lands had even entered tapped it might have been okay but they missed so many possible limits on his abilities
I appreciate them writing this and taking it on the chin here but honestly there is so much in here that’s pretty damning about their design process. Nadu was a massive mistake but one that seems negligibly correctable.
Mistake 1: not enough playtesting!
Majors describes the testing they did. He cites no actual numbers or anything but I think that makes it pretty clear how deficient it is/was:
Emphasis mine. I would wager that this is probably 2-3 contractors who played for probably a few weeks with the people who designed the set, a group who is obviously stuck in groupthink and can’t see differently. There’s no wonder they miss issues like this.
Mistake 2: changing cards again without testing
Again we have a card that was changed at the last minute and shipped as-is.
How many times does this need to bite them before they just actually playtest every single text change? It’s crazy to me.
Mistake 3: no automatic flags for certain high-risk abilities
They all missed the interaction with 0-mana abilities, OK fine. But why is there no automatic flag for high risk abilities? Off the top of my head:
Original Nadu had every one of these, and there are no doubt many many more things that should automatically create a higher scrutiny/testing regimen. They added the 2-times-per-turn cap at the last minute but removed the by-an-opponent limiter!
Software can easily flag cards that should be tested more fully, or recommend problematic interactions. They don’t seem to be doing this at all, instead just accepting a certain % of failure/risk. This is so preventable.
Mistake 4: putting Commander cards in Modern sets
This one is a personal pet peeve of mine and really irritates me more than everything else. This card was designed specifically for Commander yet it went into a set ostensibly for Modern/Legacy.
If the card is for Commander, put it into a Commander set! This is also extremely damning from a design perspective–they removed the flash ability and then didn’t think the card would be played at all!
So much of this seems to be preventable with better processes or using technology in even basic ways, ways they are no doubt not even close to taking advantage of. It’s a shame because these sets could be better and the playerbase wouldn’t have months of crap like this, or a Pro Tour absolutely ruined by a preventable card.
I can forgive a lot of the mistakes they made here; we’re all only human and I’m not going to pretend I’ve never shipped code with a bug that I really ought to have caught. They probably should never change any text without playtesting (unless it’s to make a card strictly worse, and even then you need to be pretty sure about the “strictly”), but I’ll bet they’ve gotten away with that a lot more than we realize.
But I think the cardinal sin here, and the mistake I can’t really understand, is that they changed an ability and “how does this interact with zero-mana spells or abilities” wasn’t in their top three considerations. Like, Cephalid Breakfast is over 20 years old. That type of interaction is not new or obscure; you can’t not have it on your radar.
Exactly, the fact that “interacts with 0-mana abilities” isn’t on a list of mandatory checks is just crazy to me. It just signals that they don’t have any kind of process/infrastructure in place to help them see these things.
Agreed on the red-flagging of abilities. Nadu had so many red flags it really should not have been allowed out untested. If the lands had even entered tapped it might have been okay but they missed so many possible limits on his abilities