Draft simulators are widely used within the MTG community and there has been great interest in improving the quality of these AI drafters.
I've created a new draft simulator with bots that draft playable two-color decks. To do this, the drafting process is divided into three phases:
1. Speculation: Take the highest rated card out of each pack, with a preference for on-color cards
2. Commitment: Commit to drafting a two-color deck with a very strong color preference
3. Deck construction: Construct a deck with the 23 most powerful cards in a two-color pair
This model is also used to offer users pick suggestions and automatic deck construction.
Are you using print runs? The choices don't feel very similar to the ones you see in a real draft. And, I realize that the rare print runs are super obtuse, but seeing things like runner runner Outland Colossuses in the bots' picks is probably fixable just by making sure multiple drafters in a pod don't open the same rare in, say, pack 2 (which is approximately true IRL).
Tried forcing RB super hard, the autobuild feature came up with:
which I feel like is probably the right build given what I gave it to work with, so kudos on that.
The bots, going around to my left, drafted GU, UW, WG, WG, RW, RW, RB, (and then me, RB), so I pretty much utterly failed to read signals as well as I was sending them, apparently. Which I guess was kind of the point.
The UI is a little unwieldy. If you could put the various sets of cards into spoilerable boxes it would make the page a lot less annoyingly tall. Also if you could put markers between different packs so it's easy to see where P1p14 goes to P2p1, et al, it would make it easier to look at.
tl;dr the AI seems good, the UI needs work, and using print runs, even ones you make up yourself, would make the draft feel more authentic. If that's not too much harder than I assume it would be.
EDIT: tried out a sealed pool, the autobuild doesn't account for the need to evaluate creature more highly than combat tricks if you have a pool with anomalously few creatures. I wouldn't have expected it to, but it's something to keep in mind.
Well, that boils down to two things: first, that the lack of print runs increases variance, making it so a few commons will appear more than they usually would, while others will appear less, and second, that the bots aren't programmed to recognize a glut of cards or do anything about it. The draft I did above, I saw 6 Undead Servants and took 5 of them. Few real draft pods would allow that.
When I look at the suggestions during drafting at the bottom of the page, the primary places I disagree with the computer are over synergy picks, and also over hate drafts.
Notably absent from the deck are two Sigil of the Empty Throne and a second Auramancer. I feel like Veteran's Sidearm, Mighty Leap, Knight of the Pilgrim's Road, Ampryn Tactician, and Guardian Automaton are all plausible cuts for those three cards, given the context of "enchantments mattering" a lot more than usual in a deck with double Knightly Valor and a Call of the Full Moon.
Now, that's not to say that I know for sure my build (probably -1 Pilgrim, -1 Mighty Leap, -1 Sidearm, +2 Sigil, +1 Auramancer) is a better build, but I suspect that there are no draft picks I could make that would cause the feature to prioritize Sigil over Ampryn Tactician, even though such pools clearly exist.
The other thing is, for example, when the computer suggests that I pick a Blood-Cursed Knight over a Bounding Krasis because I'm solidly WR. Krasis is miles better than Knight, and there's obviously nothing in the pack I'll play, so I should hate the Krasis as the best card in the pack, but the computer wants to take Knight because I share a color with it, which is terrible reasoning late in the draft. Similarly, it suggests taking stone unplayables like Bellows Lizard over hateable cards like a very late Separatist Voidmage, which has gotta be wrong. Cards that should generally have permanent homes in sideboards shouldn't be getting a bonus for being on-color, or at least not one that's as big as the bonus for maindeckables.
Noticed Eyeblight Massacre has a base rating of only 2.4, seems very low relative to the other ratings I'm seeing (with a lot of "filler" quality creatures receiving higher ratings). Where are these base ratings coming from? Are they in some way adaptive to how highly cards are being picked by actual drafters?
I might be a good idea to get like seven different people to look at all your ratings and adjust them up or down as they see fit, to get a little variety in your bots. It would be another thing to help with authenticity.
- The recommended pick/deck feature is interesting. Again, tying this into something more powerful (i.e. what I mentioned earlier about having it more accurately read/reflect people's picks) could really put this site over the top. That way I could see what the "average" player would do in a given situation vs what I would do. That said, as it stands I am still finding the recommendations to be mostly accurate, so you did a nice job to start.
- Some simple sorting utilities (color, cost, maybe type) when viewing cards picked would be very useful in deck building and even during the draft.
- The "print runs" here do seem a little wacky. Seemed like I was seeing a disproportionate # of duplicate uncommons/rares.
The bots are an improvement on bestiaire's bots, considering color in addition to raw card power level.
The community would really like to see a better approximation of human drafting and some work on curve and synergy considerations is in the works. Hopefully that will be out by battle for zendikar, maybe sooner.
Print runs reduce the chances of duplicates in boosters and we want to take this into account in the model
Yeah, just looking at the bot drafted card pools I was impressed, definitely seems like your method is working well there.
When it comes to print runs the other thing I've noticed is just some very heavily skewed packs, with like 6 cards of one color and none of another. Not that big of a deal, but worth looking into if you are continuing to tweak it.
This simulator seems really good. Thanks for making it. However, it seems like the bots take artifact-related cards in packs 2 and 3 even when they have no syenrgies and then don't try to get any synergies. They also take the gold uncommons too highly.
Yeah, just looking at the bot drafted card pools I was impressed, definitely seems like your method is working well there.
When it comes to print runs the other thing I've noticed is just some very heavily skewed packs, with like 6 cards of one color and none of another. Not that big of a deal, but worth looking into if you are continuing to tweak it.
In regards to the first, maybe too well? Having all seven bots with identical evaluations may be resulting in an unrealistic amount of cooperation. I'd be interested to see what it would do to the drafts if the bots were a little different from each other.
To the second, earlier today I was trying a draft as RW, opened my pack 3 to see nothing but blue, green, and black cards. Not even any artifacts to take.
This simulator seems really good. Thanks for making it. However, it seems like the bots take artifact-related cards in packs 2 and 3 even when they have no syenrgies and then don't try to get any synergies. They also take the gold uncommons too highly.
I agree that artifacts and gold cards are both weaker areas for the bots. Artifacts kinda get double-whammied because the bots are incapable of evaluating synergies and also because the bots never hate draft and therefore overvalue colorless cards late in the draft. This makes UR a particularly bad archetype to try for, when drafting with these bots.
Random pack generation can lead to some skewed packs. I hope to smooth the common color distribution out.
Also, what do you think of having the bots rare-draft from time to time?
Rare-drafting seems like a terrible idea for this. It should be drafting the ideal scenarios that you would see at a competitive level, not what people at your LGS might do, especially since it depends on a pick-to-pick and a person-to-person basis.
Edit: If you are going to try to fix the problems of some cards appearing too much or too little, there seem to also be way too many Enshrouding Mists.
I'm curious to see how you tweak it to handle more synergy oriented sets, seems kinda tricky. I mean do you manually create tags/links between cards that care about each other and assign a value? Or do you design a system that reads player choices and how they change for given cards based on the cards they already have?
I'm curious to see how you tweak it to handle more synergy oriented sets, seems kinda tricky. I mean do you manually create tags/links between cards that care about each other and assign a value? Or do you design a system that reads player choices and how they change for given cards based on the cards they already have?
I think you have to add more dimensions.
You have some dimensions, like colour-red for example that are linear. The more red cards you have the more highly that causes you to rate future red cards.
Other dimensions have diminishing returns, like 5-drops. Once you have 3 good 5-drops, the bar really goes up before you want more of them. So you need different drafting logic (basically a negative modifier) for them.
So when you want combo cards like Act of Treason and Sacrifice effects, I think you have two dimensions: threaten effects that play well with sac cards, and sac cards that play well with threaten effects.
Yeah, I guess that is what I meant with the tags thing. You could tag the Act with a rider that looks for sac outlets, so that if you have one the value of the other goes up some amount. But that could be a LOT of work, and difficult to pin down in terms of the right #'s, in a set with a more complex/important web of synergies.
Also I don't know that the goal is to make genius-bots. I mean, it'd be cool to *truly* optimize the drafting experience but given that nobody really ever does that with the much more adaptable human brain, I kinda doubt that an algorithm brain is going to manage it. You can write in some kind of value-adjustments based on all sorts of criteria, but actually getting them to adjust correctly gets harder with every variable you add. I mean, ORI is a good format for working out the basics of a bot like this because splashes are basically never a thing and synergy is a low priority. It'll be much harder to make the bots even as realistic as they are now, in a more complex environment. I think bots that can analyze a curve and can usefully evaluate gold/colorless cards probably puts them in a really good spot for players to learn to draft a format. Getting more complex than that is probably a job for actual human players.
First off, thanks for making this, it is quite good.
The thing for origins draft that is missing is synergy. Cards should have a synergy rating (of how dependent they are on synergy), and then the bots and the suggested pick needs to look at the synergy of the cards already drafted. This is also a problem with the autobuild feature.
Also, having some of the bots being different in some way would feel better (perhaps some of them over/under value synergy, some of them prefer certain colors, etc.)
Here is my example of the missing synergy I drafted a RU artifacts deck, that looks like a lot of fun. The autobuild was quite different from my deck.
I'm curious to see how you tweak it to handle more synergy oriented sets, seems kinda tricky. I mean do you manually create tags/links between cards that care about each other and assign a value? Or do you design a system that reads player choices and how they change for given cards based on the cards they already have?
I think you have to add more dimensions.
You have some dimensions, like colour-red for example that are linear. The more red cards you have the more highly that causes you to rate future red cards.
Other dimensions have diminishing returns, like 5-drops. Once you have 3 good 5-drops, the bar really goes up before you want more of them. So you need different drafting logic (basically a negative modifier) for them.
So when you want combo cards like Act of Treason and Sacrifice effects, I think you have two dimensions: threaten effects that play well with sac cards, and sac cards that play well with threaten effects.
Looking into this! Stayed tuned in the next few weeks/months for the implementation.
Also I don't know that the goal is to make genius-bots. I mean, it'd be cool to *truly* optimize the drafting experience but given that nobody really ever does that with the much more adaptable human brain, I kinda doubt that an algorithm brain is going to manage it. You can write in some kind of value-adjustments based on all sorts of criteria, but actually getting them to adjust correctly gets harder with every variable you add. I mean, ORI is a good format for working out the basics of a bot like this because splashes are basically never a thing and synergy is a low priority. It'll be much harder to make the bots even as realistic as they are now, in a more complex environment. I think bots that can analyze a curve and can usefully evaluate gold/colorless cards probably puts them in a really good spot for players to learn to draft a format. Getting more complex than that is probably a job for actual human players.
The goal is to make the bots as smart as possible based on a model that accounts for:
1. power level
2. color
3. curve
4. synergy
The current live version accounts for 1 and 2. Making a Ben Stark quality bot would be quite ambitious, although a winning 8-4 level bot is probably tractable. And yes, figuring out some of the more subtle nuances of drafting is much of the fun!
Differences are -1 Plains, -1 Chandra's Fury, -1 Alchemists Vial, -1 Guardians of Meletis, -2 Bellows Lizard, +2 War Horn, +2 Valor in Akros, +2 Infectious Bloodlust.
I actually think this deck is pretty reasonable, despite the unintuitive picks. 8 2-drops and 5 3-drops makes for a strong curve and five mid-range mass pump effects means it'd be pretty hard to stabilize against this. Add infinite Bloodlusts/Auramancers for big hasty finishes and I think there's a real plan here. I might actually try drafting a deck like this sometime.
EDIT: just had a completely insane attempt at drafting mill. Started with the GB gold elf with Talent of the Telepath in the deck, second pick took Sphinx's Revelation over *another* Talent of the Telepath, and wheeled both Talents. Ended up with:
Draft simulators are widely used within the MTG community and there has been great interest in improving the quality of these AI drafters.
I've created a new draft simulator with bots that draft playable two-color decks. To do this, the drafting process is divided into three phases:
1. Speculation: Take the highest rated card out of each pack, with a preference for on-color cards
2. Commitment: Commit to drafting a two-color deck with a very strong color preference
3. Deck construction: Construct a deck with the 23 most powerful cards in a two-color pair
This model is also used to offer users pick suggestions and automatic deck construction.
You can visit the site at:
http://draftsim.com
I hope that your find this a useful tool for drafting practice and sealed deck generation. Post some sweet draft decks in this thread!
Best,
-Dan
Tried forcing RB super hard, the autobuild feature came up with:
1 Fetid Imp
1 Subterranean Scout
1 Malakir Cullblade
1 Despoiler of Souls
1 Liliana, Heretical Healer
1 Ghirapur Gearcrafter
2 Deadbridge Shaman
1 Nantuko Husk
1 Eyeblight Assassin
1 Blazing Hellhound
5 Undead Servant
1 Prickleboar
1 Fiery Conclusion
1 Infernal Scarring
1 Read the Bones
9 Swamp
8 Mountain
which I feel like is probably the right build given what I gave it to work with, so kudos on that.
The bots, going around to my left, drafted GU, UW, WG, WG, RW, RW, RB, (and then me, RB), so I pretty much utterly failed to read signals as well as I was sending them, apparently. Which I guess was kind of the point.
The UI is a little unwieldy. If you could put the various sets of cards into spoilerable boxes it would make the page a lot less annoyingly tall. Also if you could put markers between different packs so it's easy to see where P1p14 goes to P2p1, et al, it would make it easier to look at.
tl;dr the AI seems good, the UI needs work, and using print runs, even ones you make up yourself, would make the draft feel more authentic. If that's not too much harder than I assume it would be.
EDIT: tried out a sealed pool, the autobuild doesn't account for the need to evaluate creature more highly than combat tricks if you have a pool with anomalously few creatures. I wouldn't have expected it to, but it's something to keep in mind.
Looking into print runs and models for creature count/curve
Tried another draft with autobuild:
1 Topan Freeblade
3 Cleric of the Forward Order
1 Thopter Engineer
1 Ghirapur Gearcrafter
1 Knight of the Pilgrim's Road
1 Auramancer
1 Ampryn Tactician
1 Guardian Automaton
2 Prickleboar
1 Gold-Forged Sentinel
1 Swift Reckoning
2 Celestial Flare
1 Call of the Full Moon
1 Mighty Leap
1 Veteran's Sidearm
1 Suppression Bonds
2 Knightly Valor
1 Evolving Wilds
8 Plains
8 Mountain
Notably absent from the deck are two Sigil of the Empty Throne and a second Auramancer. I feel like Veteran's Sidearm, Mighty Leap, Knight of the Pilgrim's Road, Ampryn Tactician, and Guardian Automaton are all plausible cuts for those three cards, given the context of "enchantments mattering" a lot more than usual in a deck with double Knightly Valor and a Call of the Full Moon.
Now, that's not to say that I know for sure my build (probably -1 Pilgrim, -1 Mighty Leap, -1 Sidearm, +2 Sigil, +1 Auramancer) is a better build, but I suspect that there are no draft picks I could make that would cause the feature to prioritize Sigil over Ampryn Tactician, even though such pools clearly exist.
The other thing is, for example, when the computer suggests that I pick a Blood-Cursed Knight over a Bounding Krasis because I'm solidly WR. Krasis is miles better than Knight, and there's obviously nothing in the pack I'll play, so I should hate the Krasis as the best card in the pack, but the computer wants to take Knight because I share a color with it, which is terrible reasoning late in the draft. Similarly, it suggests taking stone unplayables like Bellows Lizard over hateable cards like a very late Separatist Voidmage, which has gotta be wrong. Cards that should generally have permanent homes in sideboards shouldn't be getting a bonus for being on-color, or at least not one that's as big as the bonus for maindeckables.
- The recommended pick/deck feature is interesting. Again, tying this into something more powerful (i.e. what I mentioned earlier about having it more accurately read/reflect people's picks) could really put this site over the top. That way I could see what the "average" player would do in a given situation vs what I would do. That said, as it stands I am still finding the recommendations to be mostly accurate, so you did a nice job to start.
- Some simple sorting utilities (color, cost, maybe type) when viewing cards picked would be very useful in deck building and even during the draft.
- The "print runs" here do seem a little wacky. Seemed like I was seeing a disproportionate # of duplicate uncommons/rares.
The pick orders are primarily based off the pick orders from Frank Karsten, found here:
http://www.channelfireball.com/articles/a-pick-order-list-for-magic-origins/
The bots are an improvement on bestiaire's bots, considering color in addition to raw card power level.
The community would really like to see a better approximation of human drafting and some work on curve and synergy considerations is in the works. Hopefully that will be out by battle for zendikar, maybe sooner.
Print runs reduce the chances of duplicates in boosters and we want to take this into account in the model
When it comes to print runs the other thing I've noticed is just some very heavily skewed packs, with like 6 cards of one color and none of another. Not that big of a deal, but worth looking into if you are continuing to tweak it.
Storm Crow is strictly worse than Seacoast Drake.
In regards to the first, maybe too well? Having all seven bots with identical evaluations may be resulting in an unrealistic amount of cooperation. I'd be interested to see what it would do to the drafts if the bots were a little different from each other.
To the second, earlier today I was trying a draft as RW, opened my pack 3 to see nothing but blue, green, and black cards. Not even any artifacts to take.
I agree that artifacts and gold cards are both weaker areas for the bots. Artifacts kinda get double-whammied because the bots are incapable of evaluating synergies and also because the bots never hate draft and therefore overvalue colorless cards late in the draft. This makes UR a particularly bad archetype to try for, when drafting with these bots.
I've gotten like 4 Orbs of Warding out of about 60 packs, which is a lot.
There also seem to be an above average amount of Fiery Impulses passed to you in pack 1 pick 2.
Storm Crow is strictly worse than Seacoast Drake.
Also, what do you think of having the bots rare-draft from time to time?
Rare-drafting seems like a terrible idea for this. It should be drafting the ideal scenarios that you would see at a competitive level, not what people at your LGS might do, especially since it depends on a pick-to-pick and a person-to-person basis.
Edit: If you are going to try to fix the problems of some cards appearing too much or too little, there seem to also be way too many Enshrouding Mists.
Storm Crow is strictly worse than Seacoast Drake.
Gotta be careful not to accidentally draft Healing Hands + Tainted Remedy combo.
2 Cleric of the Forward Order
1 Consul's Lieutenant
1 Iroas's Champion
1 Knight of the Pilgrim's Road
2 Ampryn Tactician
3 Charging Griffin
2 Firefiend Elemental
1 Prickleboar
1 Aven Battle Priest
1 Sentinel of the Eternal Watch
1 Volcanic Rambler
1 Titan's Strength
1 Celestial Flare
1 Swift Reckoning
1 Exquisite Firecraft
1 Kytheon's Tactics
1 Suppression Bonds
10 Plains
7 Mountain
That's the deck I built, I didn't use the autodeck function here. If I got passed these cards in a real draft I'd be *very* happy...
I think you have to add more dimensions.
You have some dimensions, like colour-red for example that are linear. The more red cards you have the more highly that causes you to rate future red cards.
Other dimensions have diminishing returns, like 5-drops. Once you have 3 good 5-drops, the bar really goes up before you want more of them. So you need different drafting logic (basically a negative modifier) for them.
So when you want combo cards like Act of Treason and Sacrifice effects, I think you have two dimensions: threaten effects that play well with sac cards, and sac cards that play well with threaten effects.
The thing for origins draft that is missing is synergy. Cards should have a synergy rating (of how dependent they are on synergy), and then the bots and the suggested pick needs to look at the synergy of the cards already drafted. This is also a problem with the autobuild feature.
Also, having some of the bots being different in some way would feel better (perhaps some of them over/under value synergy, some of them prefer certain colors, etc.)
Here is my example of the missing synergy I drafted a RU artifacts deck, that looks like a lot of fun. The autobuild was quite different from my deck.
Autobuild:
2 Mage-Ring Bully
1 Subterranean Scout
1 Akroan Sergeant
1 Boggart Brute
1 Chief of the Foundry
1 Scrapskin Drake
1 Thopter Engineer
3 Aspiring Aeronaut
1 Enthralling Victor
1 Guardian Automaton
1 Reclusive Artificer
1 Whirler Rogue
1 Prickleboar
1 Fiery Impulse
1 Molten Vortex
2 Titan's Strength
1 Dragon Fodder
1 Turn to Frog
1 Act of Treason
9 Mountain
8 Island
My Build:
1 Mage-Ring Bully
1 Subterranean Scout
1 Akroan Sergeant
1 Boggart Brute
1 Chief of the Foundry
1 Thopter Engineer
3 Aspiring Aeronaut
1 Guardian Automaton
4 Reclusive Artificer
1 Whirler Rogue
1 Fiery Impulse
1 Molten Vortex
1 Titan's Strength
1 Alchemist's Vial
1 Dragon Fodder
2 War Horn
1 Meteorite
9 Mountain
8 Island
Looking into this! Stayed tuned in the next few weeks/months for the implementation.
The goal is to make the bots as smart as possible based on a model that accounts for:
1. power level
2. color
3. curve
4. synergy
The current live version accounts for 1 and 2. Making a Ben Stark quality bot would be quite ambitious, although a winning 8-4 level bot is probably tractable. And yes, figuring out some of the more subtle nuances of drafting is much of the fun!
1 Cleric of the Forward Order
1 Hangarback Walker
1 Mage-Ring Bully
1 Subterranean Scout
1 Akroan Sergeant
2 Auramancer
1 Boggart Brute
1 Guardians of Meletis
1 Stalwart Aven
1 Ampryn Tactician
1 Alchemist's Vial
4 Dragon Fodder
1 Grasp of the Hieromancer
1 Infectious Bloodlust
1 Exquisite Firecraft
1 Chandra's Fury
1 Knightly Valor
9 Mountain
8 Plains
1 Hangarback Walker
1 Mage-Ring Bully
1 Subterranean Scout
1 Akroan Sergeant
2 Auramancer
1 Boggart Brute
1 Stalwart Aven
1 Ampryn Tactician
4 Dragon Fodder
1 Grasp of the Hieromancer
3 Infectious Bloodlust
2 War Horn
1 Exquisite Firecraft
2 Valor In Akros
1 Knightly Valor
9 Mountain
7 Plains
Differences are -1 Plains, -1 Chandra's Fury, -1 Alchemists Vial, -1 Guardians of Meletis, -2 Bellows Lizard, +2 War Horn, +2 Valor in Akros, +2 Infectious Bloodlust.
I actually think this deck is pretty reasonable, despite the unintuitive picks. 8 2-drops and 5 3-drops makes for a strong curve and five mid-range mass pump effects means it'd be pretty hard to stabilize against this. Add infinite Bloodlusts/Auramancers for big hasty finishes and I think there's a real plan here. I might actually try drafting a deck like this sometime.
EDIT: just had a completely insane attempt at drafting mill. Started with the GB gold elf with Talent of the Telepath in the deck, second pick took Sphinx's Revelation over *another* Talent of the Telepath, and wheeled both Talents. Ended up with:
1 Maritime Guard
1 Sigiled Starfish
1 Guardians of Meletis
3 Undead Servant
2 Thopter Spy Network
3 Talent of the Telepath
1 Disciple of the Ring
2 Displacement Wave
2 Sphinx's Tutelage
1 Hydrolash
1 Disperse
1 Veteran's Sidearm
1 Macabre Waltz
1 Anchor to the AEther
10 Island
7 Swamp
This is probably absolutely awful, but I couldn't resist seeing what it would look like.