This link should be an up-to-date tutorial, considering that I just wrote it a few days ago. I'd be very appreciative if you could test it out and provide feedback on any major shortcomings. I can try to provide some additional tech support if you're having trouble making GPUs work on a native Linux install.
I'm considering a major overhaul to the format, so if you want to be a cutting-edge alpha tester you could wait until tonight, when hopefully I can push out a preliminary version. If you want to donate a lot of time, you could also help by testing out 'one card at a time' training mode, which is going to be massively slow.
These things are still being developed - I will post new changes as they happen.
If I were to redefine my format regarding types I'd completely let go of the distinction super/plain/sub, it is unimportant. You can rebuild the order of the magic card at postprocessing. What influences the card's contents the most? That's the focus.
Likewise for planeswalkers: even if you like them the net doesn't care specially for them. They're too rare so they must be squeezed in the general format. Not a problem if the initial loyalty looks like an equipment cost or whatever works.
I have to disagree with you here. I won't claim that separating the different kinds of types makes a difference one way or another, as the only way to get a satisfactory answer would be to generate some data. But, I think the cost of adding structure is very small. Separating into 3 fields only costs two more delimiters, and it has vastly simplified what the network needs to generate. Correctly finishing the type field is extremely predictable, rather that dragging in fairly open-ended creature subtypes.
The thing is the network is good at delimited fields. Extremely good. It can generate 65000 cards with only one or two mistakes due to badly delimited fields, and those might have been the incomplete cards on the end, I don't remember. And it's also pretty good at realizing that creatures have subtypes while other types of cards usually don't, except arcane, and other little things like that.
We certainly don't want to make planeswalkers look like other cards. If they're too different, then the best approach is just to get rid of them and admit that we can't generate them at all. But this isn't what we observe: the network is quite capable of generating working planeswalkers, it just trips up sometimes because it's reluctant to make them long enough to have 3 abilities. It usually gets the loyalty costs right.
TL;DR - Structure is a good things. Networks are good at learning structure, but bad at dealing with complex relationships in pure text, like kicker. So the more we can do to add predictable structure to the text of a card, the better.
Sorry, I don't know enough of the wiring of the net, it's just a general feeling that we observe outputs that are carried away by what was decided just before. Dashed text beginning always with the same words? Learnt perfectly even with a few cards. Something even short is there before the recurring pattern recurs? Needs hundreds of cards to be learnt partially. I'd put that next to X and quote observations. Something's wrong somewhere. Wouldn't low temp favor local effects? But why can't we go to temp 1?
Well maybe training for 500 epochs works too
You're fine!
I hypothesize that the problem with X costs (as a prime example of the problems we have with the network) is three-fold:
* First, X costs can occur in the mana cost, leading to situations where X doesn't need to be defined in the body text (e.g. Blaze). When the network generates a card with an undefined X cost, it may mistakenly think that it already tied X to the mana cost.
* Second, definitions of X almost always follow commas (e.g. Accelerated Mutation, and commas indicate topic shifts, where the network expects to have some leeway as to what comes next. That is misleading for the network.
* Third, the length of the dependency can be highly variable. Sometimes the distance between X and its definition is short (e.g. Aven Shrine), and sometimes it's longer (e.g. Aura Mutation).
That second and third problems are also related to why the network struggles with kicker costs. There's always some intervening text that, when viewed without regards to the kicker ability, could go in virtually any direction. That creates noise that suggests that the network has more latitude than it actually does.
There are trade-offs here, of course. The network wants to be the best it can at predicting the text of cards. We would like for it to be great at predicting cards with X-costs, but not at the expense of its ability to predict other kinds of cards. X-cost cards are particularly tricky because they represent a minority of cards and because they exhibit context-sensitive dependencies. We want the network to give us the best approximation it can of such cards without overfitting on them and without learning rules that bleed into the majority of cards that do not exhibit their features.
Don't think of it as a dumb mistake, think of it as a calculated sacrifice.
Now, there are ways of making life easier for the network, such as flattening the grammar, boiling things down to the essentials where needed. That's helpful. But we should be clear that for any non-trivial, complex language, there will always be some amount of "garbage" output due to strategic underapproximation (especially since we have such a severe input shortage). This "weakness" arises out of the greatest strength of stochastic models like our RNN, which, I might add, is one of the most powerful and sophisticated models that I have ever encountered.
As we've discussed, there are ways for us to specialize the architecture to better solve our problem. Better training regimes, better input preprocessing, etc. We could also add modules that refine the output, like a grammar checker or a semantics approximator. After all, the secret to good writing is rewriting.
I'd like to add my own observations / musings about the format.
The interesting thing about Magic cards is that some parts depend on others. A lot of these dependencies are pretty simple to detect and explain, which is encouraging.
There's (at least) two very different kinds of dependencies. First, some parts of a card might impose a constraint. For example, if the card has type 'creature', then it had better not have any lines of text like "draw a card", and it probably should have a subtype or two. Meanwhile if a card has type 'instant', it can have a line that just says "draw a card", but it shouldn't have activated abilities or effects that trigger on entering the battlefield.
Second, some parts of card might impose a requirement. The classic examples here are things like X, kicker, an being an equipment. If a card costs X, then it had better do something that involves X. If it doesn't, but it still uses X in an ability, then it needs to define what X is afterward. If a card is an equipment, then it really should say "equip <COST>" somewhere.
Constraints are about what a card can't have. Requirements are about what it needs to have.
There's a certain duality here: if the network is producing a card, and it has already made it a creature, then that sets constraints on the card's text; on the other hand if it's writing the text and gives it flying, then that's a requirement that the card be a creature. What's a constraint in one direction is often a requirement in the other. Note that since we control the order in which the network generates parts of a card, we have a great deal of control here, for better or for worse. Experimentally, throwing it away and allowing the fields to be in a random but labeled order makes it much harder for the network to generate well-formed cards.
Here's what I observe. Usually the networks are pretty good with constraints. We do still see mistakes where sorceries have activated abilities and so on, but there are plenty of reasonable cards, too. Requirements are a much, much bigger problem. For some of them the hit rate is near zero, like kicker and some situations involving X. Others are actually success stories though, like equip. By putting it first thing in the text, right after the types, most modern networks are pretty good at coming up with working equipment cards.
So, some requirements are much harder to get right than others. This actually makes perfect sense. To paraphrase jml, if you require something to happen right at the start of a line, or right immediately after some sort of context switch, the network does much better. For obvious reasons. It knows that it needs to insert something, so if there are convenient points available to start inserting it, then it will. Of all the characters it could start with after the context switch, the ones that make it start meeting the requirement are the most likely. If there's a lot of other stuff in the way, however, then the network has a very hard time. Imagine you're a network generating a card, and you stick X in the cost. There are only so many lines of text in a card, and it's very hard to tell from the first character if that line will use the X from the casting cost or not. It's not that you, the network, don't know that you should use the X, it's just that when you start making a line it's very hard to guarantee that that line will be able to use X, and once you've gotten a little ways into the text of the line there are much more immediate restrictions to the text you can generate. You can't begin with "uncast target" and then just throw X on, you're probably going to say something like "spell\".
The goal, then, is to come up with a format that minimizes the bad requirements. We want things that impose constraints to come first, and things that would impose bad requirements to come early enough that those requirements are either converted into constraints or are less bad.
So, the important bits of the format are probably:
|type|subtypes|p/t|text||cost|
Type first, because it imposes constraints on everything. I think it makes sense to put subtypes and p/t before anything else, as they are so strongly influenced by type. Next is the text, and finally the cost. If we put it at the end, hopefully X cards will be easier, because the requirement is a lot simpler that way. All the network has to do is stick an X somewhere in the cost, as opposed to fabricating an entire line of text.
You could put text immediately after types, which might help issues with cards having inappropriate abilities, but text is long and arbitrary so it would probably make subtypes and p/t less consistent. It would be interesting to see if a properly trained network could remember things over a large distance like that.
Supertypes should go near type. You could put it first, but I'm more inclined to put supertypes after type and let type be the primary decider of what kind of card it is. Loyalty should probably go near p/t, you could reuse the field but I think it's less confusing for the network if you don't, and the extra delimiter isn't a huge price.
Rarity and name can go anywhere. Both are extremely nebulous constraints; that makes me want to tack them on the end. Or, you could put rarity first, otherwise the distribution of rarities will be skewed toward whatever the network is more likely to create in the absence of rarity information. For set generation, this probably doesn't matter, as we can overgenerate and filter to get the distribution we want anyway. I think it does make the most sense to put the name last, as then you can easily chop it off to make a more consistent network, as Talcos is doing now, and feed in unnamed cards to come up with names for them.
Of course you can trivially chop name off for the 'stable' format. You can also make an argument for supporting a name-first, 'intuitive' format, that allows seeding names to come up with whole cards, which is a pretty cool thing to be able to do:
Then there's the issue of the text. I'm thinking that some improvements can be made inside the text field by cutting it up into lines and reordering them in a better way. Each line is exactly one ability - if you have something like "flying, first strike" I'd identify the separate keywords and chop it into "flying" and "first strike". We can try to order the lines in a way that is convenient: put equip costs first, and kicker costs last, so the requirement is to add a simple kicker cost line rather than a complicated one with "if @ was kicked".
There are other games to play as well. It might be possible to abstract out "where X is" definitions into a separate sentence, after a full stop (or period, colloquially), since that's a pretty strong context switch, or even as a separate line. This mostly works as is, but you could try to unify with other forms of X, possibly even costs. I think this has been suggested before.
Whew. Apologies for being long-winded or restating anything obvious. Just wanted to get it out while it was on my mind. The custom batcher is coming along, latest version on mtg-rnn can randomize mana costs and fields. The 'exactly one card at a time' mode should be finished soon. My ground truth experiment for that finished training after more than 110 hours of GPU time, so in another week or so I'll be able to give some results.
I've finally finished the first releasable version of my batcher. It's available as part of mtg-rnn.
The latest update now provides Talcos' sample_hs_v3.lua, in addition to the modifications he made to the LSTM model. Development is ongoing.
Currently, the batcher only supports randomizing the order of cards in the training set. You can read about the details in the Readme. I believe the current code should be an all-around improvement in all cases, though I don't have any data to confirm that yet. Give it a try and let me know what happens!
I have plans to add the ability to randomize mana symbols and field order as well as just the order of the cards. I also figured out how to do the training on one card at a time thing, I think, but that isn't implemented yet either. I'm very curious to see how it works, but it will probably be horrendously slow. I'm currently working on a sort of control group experiment by training a network with -batch_size 1 and -seq_length 1000, and the total training time will be about 4.5 days, even on my GPU.
So I went ahead and wrote a tutorial. Should be all-encompassing and explain how to do everything.
Sweet thanks! I started yesterday but got stuck with some dependacirs for cuda drivers (which torch depends on) so I'll give it another try today or tomorrow.
Unfortunately directions for setting up CUDA are a bit beyond the scope of the tutorial. The process went pretty painlessly for me (GTX Titan on 64-bit Ubuntu 14.04). I got the installer packages here, under the tab for 'Linux x86', and frequently referred to the getting started guide.
You can use Torch without CUDA (you'll have to on a virtual machine), it's just much faster if you install CUDA so you can run on a GPU.
So I went ahead and wrote a tutorial. Should be all-encompassing and explain how to do everything.
I'd appreciate it if a few people would glance it over, or test drive it, so I can add anything that's missing or fix any typos and make the experience as smooth as possible.
Also, Talcos, you might want to link this from the original post.
Yep, start with this post. The first post of the thread has a sampling script you'll need too (or you can message Talcos for the updated v3 version). If you want to use the latest version of the network which Talcos developed only a day or so ago, see this post for the updated LSTM file and small modification instructions.
Just to be clear, the modified sampling script is NOT part of mtg-rnn (yet, it will be soon, or at least eventually) but the updated LSTM model IS part of mtg-rnn, complete with the small modifications. You should be able to drop Talcos' sampling script into mtg-rnn the same way as you can into char-rnn, but again, this is a minor inconvenience and will become unnecessary.
EDIT:
One more thing - if someone wants to write a guide / tutorial about how do make things work on Windows (either instructions for setting up a virtual machine, or full instructions for running things natively, if that's possible) I'd be glad to provide it on github. Same goes for anything Apple, or heaven forbid Android. Personally, I'm only willing to offer my own technical support for Ubuntu.
All of the formatting code is available from my github. If you just want the latest text input file, that's in data/output.txt. Should be simple to reverse it.
I also have a repo with all of the RNN code. Right now it's mostly a clone of this repo, with a few changes by Talcos. It also includes the standard input file we usually use. There should be instructions for installing and setting it up there in the README file.
I've been trying to collect installation and setup instructions for various thing in a centralized place for my own convenience, as I'd rather not go digging back through many pages of forum every few days. There's a bunch of stuff in here explaining how to set up all of the other utilities that work with mtgencode. I'll probably push out an update to the mtgencode main README tonight. In the future it would definitely make sense to have an end-to-end tutorial on how to set up all of the code and make it work together somewhere on github, probably in mtgencode or mtg-rnn. It's on my to-do list.
Pardon my near double post, but I just officially forked the char-rnn repo. So far my version is still identical, but it includes the standard output of mtgencode as the default example corpus, and it has Talcos's recent changes to the LSTM model.
Talcos, or anyone, if you want to try it out and tell me it I broke anything that would be great. I'm training something right now but it's rather crazy and not the best for sanity testing. Work on the batcher will continue soon.
EDIT:
As a quick reference for any knowledgeable people out there who'd like to audit the correctness of my port of Talcos's change, I noticed this issue on the master char-rnn repo, which is about implementing the forget gate bias.
I've pushed out another update to mtgencode, with more or less full support for MSE2 in conjunction with nice things. Any card that seems to be a split card will be formatted as a side-by-side split card, which looks really hilarious for things with transform (and Garruk Relentless) but is otherwise pretty serviceable. Additionally, all of the info you could possibly want is stuffed into the notes field of the card, including the raw encoded string (or the json if you put the card in that way), whatever text spoiler you wanted specified with the command line arguments, and the name / functionality creativity analysis against existing cards, if you asked for that.
Here's a little example set generated from a few random selections of the rfields dumps if you want to see an example. Download the whole archive, not the file inside it called 'set'. Most of the cards are totally borked, so you can get a pretty good idea of what the formatting will look like in a worst-case scenario with very messy output.
EDIT: So yeah, the only reason why the old networks do so well at defining the X without prompting is because of rote memorization. This new network has better predictive power all around and is much more creative. Today we achieved a pretty solid victory.
On a scale of 1 to awesome, 11/10, would play. This is incredible.
How significant are the changes? I'm planning on working on releasing a fork of karpathy's char-rnn repo this weekend, that has the special batcher in it (and hopefully the ability to do cool things like train on exactly one card at a time). I'm thinking it might make sense to pool these improvements together and release them from a common repo. It might make more sense if you owned that repo, I don't know. The batcher modifications are fairly large in terms of diff size because of the need for horrid lua string processing, though, so if it's just a few tweaks to set the biases then I could easily add that in and provide some options to control it. Delusional madness here we come!
On another note, it would be really cool to have set generation automatically check for "card named <CARDNAME>" and then send out a query to generate <CARDNAME>. So some cards that requested it would be allowed to have a +1.
Thanks to maplesmall, mtgencode now oficially supports Magic Set Editor as an output format. I've integrated most of the functionality into the main repo.
It should mostly work, though there are a few issues like the complete deletion of the other faces of split cards and such, and a general lack of text prettification to remove hideous ~s and = bullets. I'm also working on having it puke out lots of statistics / other info into MSE's comment field, which I think will be pretty useful if you want to know the gory details of what the RNN at the same time as you look at pretty cards. It;s a work in progress.
Work on the custom batcher / other modifications to the lua code has been placed on hold due to reasons. I intend to come back and finish that at some point.
I'm super intrigued by the developments in the network training techniques - I can't wait to see what happens next! Also, any progress on image generation? It would be really sweet to have a full-stack script that could create a full set from a skeleton, with images, and pack it up into an MSE set file, all with one command at the terminal.
Only thing missing is reach.
I'm considering a major overhaul to the format, so if you want to be a cutting-edge alpha tester you could wait until tonight, when hopefully I can push out a preliminary version. If you want to donate a lot of time, you could also help by testing out 'one card at a time' training mode, which is going to be massively slow.
These things are still being developed - I will post new changes as they happen.
I have to disagree with you here. I won't claim that separating the different kinds of types makes a difference one way or another, as the only way to get a satisfactory answer would be to generate some data. But, I think the cost of adding structure is very small. Separating into 3 fields only costs two more delimiters, and it has vastly simplified what the network needs to generate. Correctly finishing the type field is extremely predictable, rather that dragging in fairly open-ended creature subtypes.
The thing is the network is good at delimited fields. Extremely good. It can generate 65000 cards with only one or two mistakes due to badly delimited fields, and those might have been the incomplete cards on the end, I don't remember. And it's also pretty good at realizing that creatures have subtypes while other types of cards usually don't, except arcane, and other little things like that.
We certainly don't want to make planeswalkers look like other cards. If they're too different, then the best approach is just to get rid of them and admit that we can't generate them at all. But this isn't what we observe: the network is quite capable of generating working planeswalkers, it just trips up sometimes because it's reluctant to make them long enough to have 3 abilities. It usually gets the loyalty costs right.
TL;DR - Structure is a good things. Networks are good at learning structure, but bad at dealing with complex relationships in pure text, like kicker. So the more we can do to add predictable structure to the text of a card, the better.
I'd like to add my own observations / musings about the format.
The interesting thing about Magic cards is that some parts depend on others. A lot of these dependencies are pretty simple to detect and explain, which is encouraging.
There's (at least) two very different kinds of dependencies. First, some parts of a card might impose a constraint. For example, if the card has type 'creature', then it had better not have any lines of text like "draw a card", and it probably should have a subtype or two. Meanwhile if a card has type 'instant', it can have a line that just says "draw a card", but it shouldn't have activated abilities or effects that trigger on entering the battlefield.
Second, some parts of card might impose a requirement. The classic examples here are things like X, kicker, an being an equipment. If a card costs X, then it had better do something that involves X. If it doesn't, but it still uses X in an ability, then it needs to define what X is afterward. If a card is an equipment, then it really should say "equip <COST>" somewhere.
Constraints are about what a card can't have. Requirements are about what it needs to have.
There's a certain duality here: if the network is producing a card, and it has already made it a creature, then that sets constraints on the card's text; on the other hand if it's writing the text and gives it flying, then that's a requirement that the card be a creature. What's a constraint in one direction is often a requirement in the other. Note that since we control the order in which the network generates parts of a card, we have a great deal of control here, for better or for worse. Experimentally, throwing it away and allowing the fields to be in a random but labeled order makes it much harder for the network to generate well-formed cards.
Here's what I observe. Usually the networks are pretty good with constraints. We do still see mistakes where sorceries have activated abilities and so on, but there are plenty of reasonable cards, too. Requirements are a much, much bigger problem. For some of them the hit rate is near zero, like kicker and some situations involving X. Others are actually success stories though, like equip. By putting it first thing in the text, right after the types, most modern networks are pretty good at coming up with working equipment cards.
So, some requirements are much harder to get right than others. This actually makes perfect sense. To paraphrase jml, if you require something to happen right at the start of a line, or right immediately after some sort of context switch, the network does much better. For obvious reasons. It knows that it needs to insert something, so if there are convenient points available to start inserting it, then it will. Of all the characters it could start with after the context switch, the ones that make it start meeting the requirement are the most likely. If there's a lot of other stuff in the way, however, then the network has a very hard time. Imagine you're a network generating a card, and you stick X in the cost. There are only so many lines of text in a card, and it's very hard to tell from the first character if that line will use the X from the casting cost or not. It's not that you, the network, don't know that you should use the X, it's just that when you start making a line it's very hard to guarantee that that line will be able to use X, and once you've gotten a little ways into the text of the line there are much more immediate restrictions to the text you can generate. You can't begin with "uncast target" and then just throw X on, you're probably going to say something like "spell\".
The goal, then, is to come up with a format that minimizes the bad requirements. We want things that impose constraints to come first, and things that would impose bad requirements to come early enough that those requirements are either converted into constraints or are less bad.
So, the important bits of the format are probably:
|type|subtypes|p/t|text||cost|
Type first, because it imposes constraints on everything. I think it makes sense to put subtypes and p/t before anything else, as they are so strongly influenced by type. Next is the text, and finally the cost. If we put it at the end, hopefully X cards will be easier, because the requirement is a lot simpler that way. All the network has to do is stick an X somewhere in the cost, as opposed to fabricating an entire line of text.
You could put text immediately after types, which might help issues with cards having inappropriate abilities, but text is long and arbitrary so it would probably make subtypes and p/t less consistent. It would be interesting to see if a properly trained network could remember things over a large distance like that.
Supertypes should go near type. You could put it first, but I'm more inclined to put supertypes after type and let type be the primary decider of what kind of card it is. Loyalty should probably go near p/t, you could reuse the field but I think it's less confusing for the network if you don't, and the extra delimiter isn't a huge price.
Rarity and name can go anywhere. Both are extremely nebulous constraints; that makes me want to tack them on the end. Or, you could put rarity first, otherwise the distribution of rarities will be skewed toward whatever the network is more likely to create in the absence of rarity information. For set generation, this probably doesn't matter, as we can overgenerate and filter to get the distribution we want anyway. I think it does make the most sense to put the name last, as then you can easily chop it off to make a more consistent network, as Talcos is doing now, and feed in unnamed cards to come up with names for them.
So a new plan for a full format:
|type|supertypes|subtypes|p/t|loyalty|text|cost|rarity|name|
Of course you can trivially chop name off for the 'stable' format. You can also make an argument for supporting a name-first, 'intuitive' format, that allows seeding names to come up with whole cards, which is a pretty cool thing to be able to do:
|name|type|supertypes|subtypes|p/t|loyalty|text|cost|rarity|
Then there's the issue of the text. I'm thinking that some improvements can be made inside the text field by cutting it up into lines and reordering them in a better way. Each line is exactly one ability - if you have something like "flying, first strike" I'd identify the separate keywords and chop it into "flying" and "first strike". We can try to order the lines in a way that is convenient: put equip costs first, and kicker costs last, so the requirement is to add a simple kicker cost line rather than a complicated one with "if @ was kicked".
There are other games to play as well. It might be possible to abstract out "where X is" definitions into a separate sentence, after a full stop (or period, colloquially), since that's a pretty strong context switch, or even as a separate line. This mostly works as is, but you could try to unify with other forms of X, possibly even costs. I think this has been suggested before.
Whew. Apologies for being long-winded or restating anything obvious. Just wanted to get it out while it was on my mind. The custom batcher is coming along, latest version on mtg-rnn can randomize mana costs and fields. The 'exactly one card at a time' mode should be finished soon. My ground truth experiment for that finished training after more than 110 hours of GPU time, so in another week or so I'll be able to give some results.
The code to do this exists in mtgencode; all it needs is an option on the frontend.
The latest update now provides Talcos' sample_hs_v3.lua, in addition to the modifications he made to the LSTM model. Development is ongoing.
Currently, the batcher only supports randomizing the order of cards in the training set. You can read about the details in the Readme. I believe the current code should be an all-around improvement in all cases, though I don't have any data to confirm that yet. Give it a try and let me know what happens!
I have plans to add the ability to randomize mana symbols and field order as well as just the order of the cards. I also figured out how to do the training on one card at a time thing, I think, but that isn't implemented yet either. I'm very curious to see how it works, but it will probably be horrendously slow. I'm currently working on a sort of control group experiment by training a network with -batch_size 1 and -seq_length 1000, and the total training time will be about 4.5 days, even on my GPU.
Unfortunately directions for setting up CUDA are a bit beyond the scope of the tutorial. The process went pretty painlessly for me (GTX Titan on 64-bit Ubuntu 14.04). I got the installer packages here, under the tab for 'Linux x86', and frequently referred to the getting started guide.
You can use Torch without CUDA (you'll have to on a virtual machine), it's just much faster if you install CUDA so you can run on a GPU.
I'd appreciate it if a few people would glance it over, or test drive it, so I can add anything that's missing or fix any typos and make the experience as smooth as possible.
Also, Talcos, you might want to link this from the original post.
Just to be clear, the modified sampling script is NOT part of mtg-rnn (yet, it will be soon, or at least eventually) but the updated LSTM model IS part of mtg-rnn, complete with the small modifications. You should be able to drop Talcos' sampling script into mtg-rnn the same way as you can into char-rnn, but again, this is a minor inconvenience and will become unnecessary.
EDIT:
One more thing - if someone wants to write a guide / tutorial about how do make things work on Windows (either instructions for setting up a virtual machine, or full instructions for running things natively, if that's possible) I'd be glad to provide it on github. Same goes for anything Apple, or heaven forbid Android. Personally, I'm only willing to offer my own technical support for Ubuntu.
I also have a repo with all of the RNN code. Right now it's mostly a clone of this repo, with a few changes by Talcos. It also includes the standard input file we usually use. There should be instructions for installing and setting it up there in the README file.
I've been trying to collect installation and setup instructions for various thing in a centralized place for my own convenience, as I'd rather not go digging back through many pages of forum every few days. There's a bunch of stuff in here explaining how to set up all of the other utilities that work with mtgencode. I'll probably push out an update to the mtgencode main README tonight. In the future it would definitely make sense to have an end-to-end tutorial on how to set up all of the code and make it work together somewhere on github, probably in mtgencode or mtg-rnn. It's on my to-do list.
Talcos, or anyone, if you want to try it out and tell me it I broke anything that would be great. I'm training something right now but it's rather crazy and not the best for sanity testing. Work on the batcher will continue soon.
EDIT:
As a quick reference for any knowledgeable people out there who'd like to audit the correctness of my port of Talcos's change, I noticed this issue on the master char-rnn repo, which is about implementing the forget gate bias.
Here's a little example set generated from a few random selections of the rfields dumps if you want to see an example. Download the whole archive, not the file inside it called 'set'. Most of the cards are totally borked, so you can get a pretty good idea of what the formatting will look like in a worst-case scenario with very messy output.
On a scale of 1 to awesome, 11/10, would play. This is incredible.
How significant are the changes? I'm planning on working on releasing a fork of karpathy's char-rnn repo this weekend, that has the special batcher in it (and hopefully the ability to do cool things like train on exactly one card at a time). I'm thinking it might make sense to pool these improvements together and release them from a common repo. It might make more sense if you owned that repo, I don't know. The batcher modifications are fairly large in terms of diff size because of the need for horrid lua string processing, though, so if it's just a few tweaks to set the biases then I could easily add that in and provide some options to control it. Delusional madness here we come!
On another note, it would be really cool to have set generation automatically check for "card named <CARDNAME>" and then send out a query to generate <CARDNAME>. So some cards that requested it would be allowed to have a +1.
One of the most entertaining graphs produced by a machine learning researcher that I've ever seen. Five or six psychotically delusional thumbs up.
It should mostly work, though there are a few issues like the complete deletion of the other faces of split cards and such, and a general lack of text prettification to remove hideous ~s and = bullets. I'm also working on having it puke out lots of statistics / other info into MSE's comment field, which I think will be pretty useful if you want to know the gory details of what the RNN at the same time as you look at pretty cards. It;s a work in progress.
Work on the custom batcher / other modifications to the lua code has been placed on hold due to reasons. I intend to come back and finish that at some point.
I'm super intrigued by the developments in the network training techniques - I can't wait to see what happens next! Also, any progress on image generation? It would be really sweet to have a full-stack script that could create a full set from a skeleton, with images, and pack it up into an MSE set file, all with one command at the terminal.