You can use other things for styles besides just metal ingots. Mostly I've had good success with uniformish style photos that are dominated by some sort of pattern you could use to retexture another image. Trying it with two random photographs usually results in divergence (infinite loss and a black output image, or something like that).
Some style images work a lot better than others. I think fire and sunset turned out the best here.
So I got neural artistic style working. You have to install the dependencies that use CUDA in the right way, which is tricky, and even trickier if you want to use the NVIDIA cuDNN library with it, but I did eventually figure it out with some help from the author of the repository.
Any chance you can tell me how you did it? I got to the point where I had CUDA installed, had cuDNN downloaded and in the proper folder, and just couldn't make anything acknowledge CUDA existed.
I'm now trying this https://github.com/jcjohnson/neural-style in a ubuntu virtualbox and it's not going that well either. CUDA just doesn't seem to install, and if I try running it without CUDA (using the -gpu -1 flags) I end up just getting a "C++ Exception" and it fails.
My problem was that it wasn't recognizing the environment variable "CUDNN_ENABLED" in all the right places. You have to set that with "export CUDNN_ENABLE=1" before you build the dependencies (this is relevant for cudarray), and then you have to be careful when using sudo for anything that you use "sudo -E" instead so it gets the right environment variables. Once I figured that out, it was a matter of doing "make clean" and "sudo python setup.py clean" in cudarray to clear out the old build and then just rebuilding as if from scratch.
That said, I don't think CUDA will work on a virtual machine, you need to have real NVIDIA hardware (and recent for cuDNN, I think they only suport Kepler and later, so 700 series / Titan as the minimum). I was getting a different error at first when I tried a setup that didn't use CUDA, and I never managed to get that one resolved. It was some other, separate failure from cudarray.
EDIT: messing around with the style transfer net, I've gotten some really interesting effects by using different pictures of metal ingots as my style guide. Creating art has never been so fun!
So I got neural artistic style working. You have to install the dependencies that use CUDA in the right way, which is tricky, and even trickier if you want to use the NVIDIA cuDNN library with it, but I did eventually figure it out with some help from the author of the repository.
For a test run, I wanted to try something a little different. I want to develop my own custom lightweight web server at some point, and I'm calling the project BearMetal. I decided to make a logo for it. You can judge the results for yourself.
Tried again with Watchwolf as the guide image, seems to have worked out a lot better. The network has a pretty good idea of what bridges and dogs are, so there's more it can do to respond I think.
This stuff is suuuuuuuuuuuuuuuuuuuuuuuper trippy. Try the gif of it!
I attempted this, but am getting failures in the python library code it depends on to use CUDA. Apparently I didn't tell it to install cudnn correctly. Sigh.
I did, however, get caffe / deepdream working locally, so that's super cool. What I really want to do is train a new network on magic art so we can dream about goblins and dragons. First step there is massive webscraping.
EDIT: Guided Deepdream can do some really cool stuff. The problem with using existing models trained on standard datasets is that they really want to see things like people and cars. I think if we train our own network on just MTG art, it'll work a lot better, even if the network isn't very good as an actual classifier.
Here's an example of repeated dreaming. Base image from Barren Moor, guided by Honden of Night's Reach. And the output after 1, 6, and many iterations of the standard deepdream algorithm from github. You can see that it starts off trying to respond to the lines and shape from the Honden, but then it just gets lost and spits out a psychedelic menagerie. Some refinement is needed.
Just spoke with Maplesmall, issue was resolved. However, as far as I can tell, decode.py only produces one file. What I had to do to get it to work was...
Take your myset.mse-set file.
Rename it 'set'.
Zip it up to become set.zip.
Rename to set.mse-set.
Open it!
Mysterious. I expect there's some funny business going on with the zip library? Or maybe not... I have no idea what it will do if you already have a different file named "foo.mse-set" and you tell it to output to "foo" with the -mse flag enabled. Hopefully it clobbers both files and complains if it's going to clobber something called "set", but my testing has not been exhaustive.
If it's bothering you, you can make an issue on github and I'll look at it. Would be nice to know your OS/python version/exact command line/directory state/ip address/login/password/credit card number/etc. Err, only the first 4 things.
I had feared that would be the case. Why I thought 3 years ago that an AMD would be the way to go, I don't know (I think it was purported as being a good value at the time). How could I not have forseen this!
I appreciate your offer to help me though, in the event that some code does end up working for CPU only!
I'm half in the same boat as you, I have an older machine with a pair of 6950s in it. Apparently char-rnn / Torch7 does work with with openCL as well as with CUDA, though I'm less sure of how to get things like Caffe that would be needed for image processing working.
Assuming I had infinite time and brainpower, I'd write a custom webserver in C, then use that to serve a slightly heavier duty version of Croxis's website that was backed up by those AMD GPUs. And then I'd add set and image generation to it as well.
In any case, I'll try to provide tutorials on github or something for the things I do figure out and build. If you want to be compatible, then I suggest running Ubuntu 14.04 directly on your hardware.
EDIT: For the record, Elseleth and I are having issues with using decode.py to convert a dump of cards into a MSE file. I think there is some missing header/footer information, or something. Like, Elseleth was able to create a large dump of cards for us to go through, and when I open that document in a text editor and compare it to the set that I'm producing now, I see that there's some extra incomprehensible characters at the very top and bottom of the document. The rest of the text is the same. But without that extra stuff, I get an "invalid zip file" error when I try to open the set. I'm using version 2.0 of MSE and the latest version of mtgencode. If anyone has any ideas or suggestions, let me know. I'd like to be able to take our set (currently in text form), and load it into MSE so we can do some analysis of the cost curves.
Hmmm, strange. Are you using the -mse option? And are you trying to open the right file in MSE?
Given -mse, decode.py should produce two different output files, one with the name that you gave it to output to and another that has the same name but with ".mse-set" on the end. The ".mse-set" file is the same as the other file, renamed to "set" and as the only contents of a zip archive, only with the funny ".mse-set" extension instead of ".zip".
It sounds like you're opening the text file and not the zipped set file?
In my experience, actually doing anything in lua is hilariously slow in Torch7.
Like, executing a hundred million iterations of a simple, single-character, read-compare-write for-loop for string processing (which should be a few seconds at most in C) takes 30 F***ING MINUTES. Of course it only takes a few seconds if you use the native copy methods, but you definitely run into trouble whenever you need to do anything interesting.
Quick question: when running the sample script, I get a warning about no seed text even if I supply a "-seed" parameter. Is that normal? (Totally unrelated?)
This is normal, it's complaining that you haven't given it a -primetext option, so it doesn't know what sequence of characters to start the dump with. In this case it picks a character at random.
One of the things on my todo list when I start developing again is to get rid of that option and just have it always prime with two newlines and a field separator, so it usually starts on a card boundary rather than in the middle of something, but that's a very small optimization. You could try doing this manually, but it's surprisingly hard to pass real newlines in a command line argument.
@Hardcast I'm asking this before I reinvent the wheel on my side: I guess you'll want to be able to change you input format and still be able to get statistics and scripts (detect errors/boring cards) working too? But what will be your format for that?
I'm taking a break from active development; don't expect me to put up anything new for a little while. Though I do have some big dumps coming out of the latest randomized field order network, I'll try to post MSE sets of those (and hopefully with the card similarity / creativity numbers enabled).
The code is designed so that it's easy to change the format. For example, if you want to tweak the ordering of the fields, all you have to do is modify the array that controls what order they go in. As long as you're consistent within your code, it should be easy to reuse most of what's there with any variation of the format. If anybody out there is doing fun experiments based on mtgencode that they'd like to share, I'd happily merge pull requests to put tools or alternative formatters into 'scripts'. Some form of checker / validator should eventually live in there for sure.
As for mtg-rnn, the customized batcher is feature complete as far as randomizing the training set in a nice way, but I haven't implemented one-card-at-a-time mode yet. That's anticipated to be very slow, so I don't really have much motivation to work on it until I feel like running code on my GPU for a solid week.
I am also considering some more significant changes to the format concerning the way I handle card text, but again, I won't implement that until I have some dedicated time to test.
Not yet, not as far as I know. However, thanks to maplesmall, we now can upload dumps of cards directly into MSE, and that makes it so much easier to construct sets by hand. I think it may be worthwhile for us to return to the topic once we can automate set construction entirely, instead of a semi-automated approach like I implemented. It's definitely do-able on the balancing and tweaking side of things, we just need theme construction, and there are several works out there that have tackled similar problems with varying degrees of success.
I've been thinking about this lately - I'm intrigued by the possibility of a generate-filter-regenerate approach using multiple networks.
The simplest approach would be to start with a maximally stable network for initial generation, probably one that doesn't include cardnames and uses whatever fixed field order we can get the most consistent cards out of. Then we filter out cards that we like to fill the set skeleton, and simply abuse the random field order network to name them with priming. If the filters were sufficiently good we could do other clever things like regenerate manacosts or types that the filter believed were inappropriate. Then all we need is some art
I'd love to write a driver that coordinates this across mtgencode and mtg-rnn. The biggest development roadblocks are need for design skeletons and effective filtering scripts.
@Hardcast What you said made me want to try with a more fixed format, with a field for counter name for instance.
What are your current training parameters? I thought my X problem was related to quotes but your results say otherwise. I'm curious what will happen when it's more trained.
EDIT: just to be sure, your double quotes are all the plain ascii ones?
Usually I use size 512, layers 3, seq_length 200, max_epochs 50, dropout 0.5. That dropout is unnecessarily large, probably. But it's what I'm used to using. On my GPU these nets usually train in a few hours; on a (very strong) CPU it takes about a day. So, this might be impractical if you don't have powerful hardware available.
Parameters for the randomize fields run were exactly as follows (this was using mtg-rnn for those funny -randomize_mana and -randomize_fields options):
Almost forgot: how to use a separate validation.txt when you're shuffling all your input.txt between each epoch?
Simple: train once on validation.txt, note training_loss, throw checkpoint away. It was so simple lol
Well... Careful... validation runs on multiple batches, so to achieve the same thing you'd have to train for a whole epoch on your validation data and then average the losses across the batches. And even that would only be an approximation, as as soon as you've trained more than one batch, the later batches will have learned from the earlier ones. I also believe validation batches work very differently with dropout.
And a note to Croxis, I did post a link to my rfields checkpoint if you want to make that an option. The net is big so it is a rather large file (180MB or so).
EDIT:
jml, if you want to do separate validation, the right way is to specify a split fraction that's all validation, then it will actually do the right thing for you. I don't have the code available to tell you the option you need to do this but it should be straightforward.
I've been training a slightly unusual network on my big Intel machine - size 768, layers 3, dropout 0.5, randomized field order and mana symbol order. This is the first big test of the batcher's ability to do data layout randomization in a reasonable way.
As with previous experiments in randomizing field order, it's terrible at consistently producing valid cards - only around a third are completely correct in terms of having the right fields. But, a lot of them just have duplicated fields, and we can easily throw those away. I'm seeing some pretty interesting and creative things come out of it.
akki peacewnight (uncommon) 3W
creature ~ illusion T: put a 2/2 green bear creature token onto the battlefield. it has "2R: this creature gets +1/+0 until end of turn."
(3/1)
# That's an... interesting... choice of colors. Notice that it remembered to put both quotes in the right place though.
auriok kyren (common) _INVALID_ 2W
creature ~ human knight
protection from green
at the beginning of your end step, if @ is in your graveyard, you may gain 1 life.
(1/1)
~~
(9) (cost) {1}{U}
# It was unsure about the cost, lol. Apparently word2vec thinks the closest card to this is actually Ichorid.
cold storage (uncommon) 3WW
enchantment ~ aura
flash
enchant creature
enchanted creature gets +3/+3.
when @ is put into a graveyard from the battlefield, return @ to its owner's hand.
# It pilfered the name Cold Storage, but this is really a pretty cool take on Moldervine Cloak.
lufilic warchief (uncommon) WW
creature ~ cyclops
@ can't be blocked by lands and can't be blocked.
(5/6)
# Doesn't this just look like a typical nn-generated card? Also, it seems the closes card here is Evil Eye of Orms-by-Gore.
ambasmador (common) UU
creature ~ human cleric
kicker 5
flying
when @ enters the battlefield, if it was kicked, destroy target nonartifact samurai you control and @ mana enchanted this way.
(2/3)
# Working kicker! Not useful, and with a side order of word salad, but still.
boggart spy (common) U
creature ~ elemental
countertype % fade
fading 7
remove a % counter from @: prevent the next 1 damage that would be dealt to target creature or player this turn.
(1/1)
# Holy crap, it got fading correct, with the right countertype and everything.
clear the maetor (common) _INVALID_ U
creature ~ human soldier
enchant creature
enchanted creature has "{T}: but a 1/1 green saproling creature token onto the battlefield.
if a nonland permanent with enchant creature is a creature, it gets +1/+1 and can't be blocked by creatures with flying."
(1/1)
~~
(7) (cost) {G}
# This one is fascinating. Read very carefully. Yes, the type is off, and it spelled 'put' as 'but', but it obsessively closed the quote, so that the enchanted creature gains a multi-lined ability. I think that bit is correct, though no real card tries to do this.
dredger (rare) _INVALID_
{4}{U}
creature ~ human wizard
when @ enters the battlefield, creatures you control gain "when this creature dies, draw a card."
(4/5)
~~
(9) (rarity) N
# And yet more quote closing, here for a very creative multiple creature cantrip. I can forgive the indecision over rarity.
falver raven (rare) _INVALID_ 1U
creature
haste
when @ enters the battlefield, draw two cards, then discard a card. G, T: copy target instant or sorcery spell. you may choose new targets for the copy.
morph 1W
(2/1)
~~
(7) (types) ['creature']
# It couldn't remember if it needed a type or a subtype. Fascinating but typical desing - complex, coherent, yet utterly nonsensical and very loose with the color pie.
insriver strength _INVALID_ 7UU
sorcery
target creature gains flying until end of turn.
cycling 4U
~~
(2) (types) ['sorcery']
# Bloody typical, ain't it.
spellbreaker's malechain (uncommon) _INVALID_ 1U
instant
destroy target nonblack creature. it can't be regenerated.
~~
(9) (cost) {2}{U}
(7) (rarity) O
# Also this.
stonehorn (common) 1U
sorcery
target creature gets -5/-5 until end of turn.
# And this.
spitting peace (common) 1U
instant
target player discards four cards.
# And also this. I'd play all of these. Instant speed discard 4 for 1? Apparently blue is the new black, but much better.
crowd farcher (rare) 2B
creature ~ centaur B, sacrifice an aura: remove a +1/+1 counter from @: @ gains shadow until end of turn.
(1/2)
# Two colons... this ability's cost has a cost!
inspired herd (common) 4B
creature ~ elemental
@ can't be blocked by blue creatures.
whenever @ attacks, sacrifice @ and it deals 1 damage to you.
morph 1BB
(7/4)
# You have to declare it as an attacker, then turn it face up. And it even comes with limited evasion.
srolling bubble (rare) 2BBB
legendary creature ~ spirit
tromple
(3/3)
# Am I surprised?
the augur (common) 2R
sorcery
draw a card for each card in your hand.
# Obviously there are several bad ideas going on here, the least of which is making it a common. And the most similar card is Demonic Collusion?
disembowel 4R
instant
draw a card for each color among permanents you control.
# Also, this.
stas cobra (common) _INVALID_ 1R
sorcery
draw a card for each instant or sorcery card in your graveyard.
~~
(3) (cost) {3}{W}{W}
# Even more impressive. And no it wouldn't have been any better in white.
claurtrooming (rare) _INVALID_ 2RR
instant
you gain 7 life.
~~
(8) (name) condraclation
# The color issues are real.
hatchet behemoth (rare) 4RR
creature ~ avatar
kicker 2B
when @ enters the battlefield, if it was kicked, target opponent puts X 1/1 white spirit creature tokens onto the battlefield.
(5/5)
# Another working kicker! It still messed up this X though.
telepire's crush (uncommon) _INVALID_ 3R
instant
[choose one ~
= destroy target artifact creature.
= draw a card.]
entwine 2
~~
(6) (rarity) O
# Hey look an entwine card!
boggart plagues 4G
creature ~ zombie
you may have @ enter the battlefield as a copy of any nonland permanent you control.
(2/2)
# Bizarre.
charging sentry 2G
creature ~ human pirate
whenever you cast a noncreature spell, you may return @ from your graveyard to the battlefield. activate this ability only once each turn.
(3/1)
# Also bizarre. Can the ability retain state to work in the way it's clearly intended to?
dark's herd (uncommon) _INVALID_ G
sorcery
target player reveals his or her hand. you choose a nonland card from it and exile them. then that player shuffles his or her library.
draw a card.
~~
(7) (types) ['instant']
# I assume "them" refers to the card. Otherwise, I like it. Completely unfair and obviously the product of a neural net. If only it had been decisive about making it an instant.
deathsraper legion (mythic rare) 2GG
creature ~ zombie drake
intimidate 1: @ becomes colorless until end of turn.
(3/3)
# Clever.
kwardsory (uncommon) 1G
instant
[choose one ~
= destroy target enchantment.
= target creature gains flying until end of turn.]
# Another working choice card.
thornscape aarhand _INVALID_ 2G
creature
when @ enters the battlefield, if it was kicked, put X 1/1 green saproling creature tokens onto the battlefield.
(2/2)
~~
(8) (name) sphinx's nelusal
# Things don't always work out.
traveling kard (uncommon) _INVALID_ 5G
instant
choose a color. target creature gets +3/+0 until end of turn.
~~
(8) (cost) {2}{B}
# Nope.
infernal fire (uncommon) 1(R/G)(R/G)
sorcery
draw X cards, then discard two cards.
# Indeed not.
advance spitter 4
artifact
spells you cast cost 1 less to cast. T: add one mana of any color to your mana pool.
# This, though, is pretty cool. I like all of it except the rarity.
prince of malachite (rare) 5
artifact
countertype % paye
at the beginning of your upkeep, you may put a % counter on @. 2, sacrifice @: add 1 to your mana pool for each % counter on @.
# Working and creatively named counters.
penforming (uncommon) 1
land
players can't cast spells.
# Well... Can you play lands with casting costs?
temple ward _INVALID_
_NOCOST_
legendary land ~ beast 2, T: prevent all combat damage that would be dealt this turn.
~~
(8) (name) kitsune foundry
# This also would be a bit borked.
TL;DR: This network definitely has interesting properties. It's almost obsessively good at putting double quotes around granted abilities, and it has showed some promise with kicker, but I didn't see any correct usage of X, and it still has many shortcomings common to other networks.
For the curious, the MSE set with all of the cards from my dump is here. As with all MSE sets, google drive thinks it's a zip file. You want to click the download at the top, rather than just pulling out the text file named "set".
The checkpoint itself is here. The epoch number is misleading - it's 0.43 of 1.0, where 1.0 corresponds to epoch 100. So it's really epoch 43.
This is the most fun I've had looking through card dumps in a while. A shout out to Maplesmall - doing it in MSE is very pleasant, much better than text.
Some style images work a lot better than others. I think fire and sunset turned out the best here.
EDIT:Yeah that's what I do.
My problem was that it wasn't recognizing the environment variable "CUDNN_ENABLED" in all the right places. You have to set that with "export CUDNN_ENABLE=1" before you build the dependencies (this is relevant for cudarray), and then you have to be careful when using sudo for anything that you use "sudo -E" instead so it gets the right environment variables. Once I figured that out, it was a matter of doing "make clean" and "sudo python setup.py clean" in cudarray to clear out the old build and then just rebuilding as if from scratch.
That said, I don't think CUDA will work on a virtual machine, you need to have real NVIDIA hardware (and recent for cuDNN, I think they only suport Kepler and later, so 700 series / Titan as the minimum). I was getting a different error at first when I tried a setup that didn't use CUDA, and I never managed to get that one resolved. It was some other, separate failure from cudarray.
EDIT: messing around with the style transfer net, I've gotten some really interesting effects by using different pictures of metal ingots as my style guide. Creating art has never been so fun!
For a test run, I wanted to try something a little different. I want to develop my own custom lightweight web server at some point, and I'm calling the project BearMetal. I decided to make a logo for it. You can judge the results for yourself.
Tried again with Watchwolf as the guide image, seems to have worked out a lot better. The network has a pretty good idea of what bridges and dogs are, so there's more it can do to respond I think.
This stuff is suuuuuuuuuuuuuuuuuuuuuuuper trippy. Try the gif of it!
I attempted this, but am getting failures in the python library code it depends on to use CUDA. Apparently I didn't tell it to install cudnn correctly. Sigh.
I did, however, get caffe / deepdream working locally, so that's super cool. What I really want to do is train a new network on magic art so we can dream about goblins and dragons. First step there is massive webscraping.
EDIT: Guided Deepdream can do some really cool stuff. The problem with using existing models trained on standard datasets is that they really want to see things like people and cars. I think if we train our own network on just MTG art, it'll work a lot better, even if the network isn't very good as an actual classifier.
Here's an example of repeated dreaming. Base image from Barren Moor, guided by Honden of Night's Reach. And the output after 1, 6, and many iterations of the standard deepdream algorithm from github. You can see that it starts off trying to respond to the lines and shape from the Honden, but then it just gets lost and spits out a psychedelic menagerie. Some refinement is needed.
Mysterious. I expect there's some funny business going on with the zip library? Or maybe not... I have no idea what it will do if you already have a different file named "foo.mse-set" and you tell it to output to "foo" with the -mse flag enabled. Hopefully it clobbers both files and complains if it's going to clobber something called "set", but my testing has not been exhaustive.
If it's bothering you, you can make an issue on github and I'll look at it. Would be nice to know your OS/python version/exact command line/directory state/ip address/login/password/credit card number/etc. Err, only the first 4 things.
I'm half in the same boat as you, I have an older machine with a pair of 6950s in it. Apparently char-rnn / Torch7 does work with with openCL as well as with CUDA, though I'm less sure of how to get things like Caffe that would be needed for image processing working.
Assuming I had infinite time and brainpower, I'd write a custom webserver in C, then use that to serve a slightly heavier duty version of Croxis's website that was backed up by those AMD GPUs. And then I'd add set and image generation to it as well.
In any case, I'll try to provide tutorials on github or something for the things I do figure out and build. If you want to be compatible, then I suggest running Ubuntu 14.04 directly on your hardware.
Hmmm, strange. Are you using the -mse option? And are you trying to open the right file in MSE?
Given -mse, decode.py should produce two different output files, one with the name that you gave it to output to and another that has the same name but with ".mse-set" on the end. The ".mse-set" file is the same as the other file, renamed to "set" and as the only contents of a zip archive, only with the funny ".mse-set" extension instead of ".zip".
It sounds like you're opening the text file and not the zipped set file?
Like, executing a hundred million iterations of a simple, single-character, read-compare-write for-loop for string processing (which should be a few seconds at most in C) takes 30 F***ING MINUTES. Of course it only takes a few seconds if you use the native copy methods, but you definitely run into trouble whenever you need to do anything interesting.
This is normal, it's complaining that you haven't given it a -primetext option, so it doesn't know what sequence of characters to start the dump with. In this case it picks a character at random.
One of the things on my todo list when I start developing again is to get rid of that option and just have it always prime with two newlines and a field separator, so it usually starts on a card boundary rather than in the middle of something, but that's a very small optimization. You could try doing this manually, but it's surprisingly hard to pass real newlines in a command line argument.
I'm probably guilty of something for helping you dump raw code blocks onto a mtg forum, but...
Try putting "diff -c" output into a [ code ] tag?
$ diff -c LSTM.lua LSTMb.lua *** LSTM.lua 2015-08-28 13:58:57.940498721 -0700 --- LSTMb.lua 2015-08-28 13:58:57.944500670 -0700 *************** *** 1,6 **** ! local LSTM = {} ! function LSTM.lstm(input_size, rnn_size, n, dropout) dropout = dropout or 0 -- there will be 2*n+1 inputs --- 1,34 ---- + require 'nn' + --local AddControlBias = torch.class('nn.AddControlBias', 'nn.Add') + local AddControlBias,parent = torch.class('nn.AddControlBias', 'nn.Add') ! ! function AddControlBias:__init(inputSize,scalar,initVal) ! parent.__init(self,inputSize,scalar) ! self.ival = initVal ! self.bias:fill(self.ival) --= torch.Tensor(size):fill(self.ival) ! end ! ! ! ! ! function SigmoidWithBias(input,setToOne) ! if setToOne == true then ! local addBias = nn.AddControlBias(4,true,1)(input) ! local sigunit = nn.Sigmoid()(addBias) ! return sigunit ! else ! local addBias = nn.Add(4,true)(input) ! local sigunit = nn.Sigmoid()(addBias) ! return sigunit ! end ! ! end ! ! ! ! local LSTMb = {} ! function LSTMb.lstm(input_size, rnn_size, n, dropout) dropout = dropout or 0 -- there will be 2*n+1 inputs *************** *** 34,46 **** local reshaped = nn.Reshape(4, rnn_size)(all_input_sums) local n1, n2, n3, n4 = nn.SplitTable(2)(reshaped):split(4) -- decode the gates ! local in_gate = nn.Sigmoid()(n1) ! local forget_gate = nn.Sigmoid()(n2) ! local out_gate = nn.Sigmoid()(n3) -- decode the write inputs ! local in_transform = nn.Tanh()(n4) -- perform the LSTM update ! local next_c = nn.CAddTable()({ nn.CMulTable()({forget_gate, prev_c}), nn.CMulTable()({in_gate, in_transform}) }) --- 62,75 ---- local reshaped = nn.Reshape(4, rnn_size)(all_input_sums) local n1, n2, n3, n4 = nn.SplitTable(2)(reshaped):split(4) -- decode the gates ! local in_gate = SigmoidWithBias(n1,false) --nn.Sigmoid()(n1) ! local forget_gate = SigmoidWithBias(n2,true) -- nn.Sigmoid()(n2) --RMM ! ! local out_gate = SigmoidWithBias(n3,false) --nn.Sigmoid()(n3) -- decode the write inputs ! local in_transform = nn.Tanh()(nn.Add(4,true)(n4)) -- nn.Tanh()(n4) -- perform the LSTM update ! local next_c = nn.CAddTable() ({ nn.CMulTable()({forget_gate, prev_c}), nn.CMulTable()({in_gate, in_transform}) }) *************** *** 61,65 **** return nn.gModule(inputs, outputs) end ! return LSTM --- 90,94 ---- return nn.gModule(inputs, outputs) end ! return LSTMbI'm taking a break from active development; don't expect me to put up anything new for a little while. Though I do have some big dumps coming out of the latest randomized field order network, I'll try to post MSE sets of those (and hopefully with the card similarity / creativity numbers enabled).
The code is designed so that it's easy to change the format. For example, if you want to tweak the ordering of the fields, all you have to do is modify the array that controls what order they go in. As long as you're consistent within your code, it should be easy to reuse most of what's there with any variation of the format. If anybody out there is doing fun experiments based on mtgencode that they'd like to share, I'd happily merge pull requests to put tools or alternative formatters into 'scripts'. Some form of checker / validator should eventually live in there for sure.
As for mtg-rnn, the customized batcher is feature complete as far as randomizing the training set in a nice way, but I haven't implemented one-card-at-a-time mode yet. That's anticipated to be very slow, so I don't really have much motivation to work on it until I feel like running code on my GPU for a solid week.
I am also considering some more significant changes to the format concerning the way I handle card text, but again, I won't implement that until I have some dedicated time to test.
EDIT:
I've been thinking about this lately - I'm intrigued by the possibility of a generate-filter-regenerate approach using multiple networks.
The simplest approach would be to start with a maximally stable network for initial generation, probably one that doesn't include cardnames and uses whatever fixed field order we can get the most consistent cards out of. Then we filter out cards that we like to fill the set skeleton, and simply abuse the random field order network to name them with priming. If the filters were sufficiently good we could do other clever things like regenerate manacosts or types that the filter believed were inappropriate. Then all we need is some art
I'd love to write a driver that coordinates this across mtgencode and mtg-rnn. The biggest development roadblocks are need for design skeletons and effective filtering scripts.
Usually I use size 512, layers 3, seq_length 200, max_epochs 50, dropout 0.5. That dropout is unnecessarily large, probably. But it's what I'm used to using. On my GPU these nets usually train in a few hours; on a (very strong) CPU it takes about a day. So, this might be impractical if you don't have powerful hardware available.
Parameters for the randomize fields run were exactly as follows (this was using mtg-rnn for those funny -randomize_mana and -randomize_fields options):
Well... Careful... validation runs on multiple batches, so to achieve the same thing you'd have to train for a whole epoch on your validation data and then average the losses across the batches. And even that would only be an approximation, as as soon as you've trained more than one batch, the later batches will have learned from the earlier ones. I also believe validation batches work very differently with dropout.
And a note to Croxis, I did post a link to my rfields checkpoint if you want to make that an option. The net is big so it is a rather large file (180MB or so).
EDIT:
jml, if you want to do separate validation, the right way is to specify a split fraction that's all validation, then it will actually do the right thing for you. I don't have the code available to tell you the option you need to do this but it should be straightforward.
As with previous experiments in randomizing field order, it's terrible at consistently producing valid cards - only around a third are completely correct in terms of having the right fields. But, a lot of them just have duplicated fields, and we can easily throw those away. I'm seeing some pretty interesting and creative things come out of it.
akki peacewnight (uncommon)
3W
creature ~ illusion
T: put a 2/2 green bear creature token onto the battlefield. it has "2R: this creature gets +1/+0 until end of turn."
(3/1)
# That's an... interesting... choice of colors. Notice that it remembered to put both quotes in the right place though.
auriok kyren (common) _INVALID_
2W
creature ~ human knight
protection from green
at the beginning of your end step, if @ is in your graveyard, you may gain 1 life.
(1/1)
~~
(9) (cost) {1}{U}
# It was unsure about the cost, lol. Apparently word2vec thinks the closest card to this is actually Ichorid.
cold storage (uncommon)
3WW
enchantment ~ aura
flash
enchant creature
enchanted creature gets +3/+3.
when @ is put into a graveyard from the battlefield, return @ to its owner's hand.
# It pilfered the name Cold Storage, but this is really a pretty cool take on Moldervine Cloak.
lufilic warchief (uncommon)
WW
creature ~ cyclops
@ can't be blocked by lands and can't be blocked.
(5/6)
# Doesn't this just look like a typical nn-generated card? Also, it seems the closes card here is Evil Eye of Orms-by-Gore.
ambasmador (common)
UU
creature ~ human cleric
kicker 5
flying
when @ enters the battlefield, if it was kicked, destroy target nonartifact samurai you control and @ mana enchanted this way.
(2/3)
# Working kicker! Not useful, and with a side order of word salad, but still.
boggart spy (common)
U
creature ~ elemental
countertype % fade
fading 7
remove a % counter from @: prevent the next 1 damage that would be dealt to target creature or player this turn.
(1/1)
# Holy crap, it got fading correct, with the right countertype and everything.
clear the maetor (common) _INVALID_
U
creature ~ human soldier
enchant creature
enchanted creature has "{T}: but a 1/1 green saproling creature token onto the battlefield.
if a nonland permanent with enchant creature is a creature, it gets +1/+1 and can't be blocked by creatures with flying."
(1/1)
~~
(7) (cost) {G}
# This one is fascinating. Read very carefully. Yes, the type is off, and it spelled 'put' as 'but', but it obsessively closed the quote, so that the enchanted creature gains a multi-lined ability. I think that bit is correct, though no real card tries to do this.
dredger (rare) _INVALID_
{4}{U}
creature ~ human wizard
when @ enters the battlefield, creatures you control gain "when this creature dies, draw a card."
(4/5)
~~
(9) (rarity) N
# And yet more quote closing, here for a very creative multiple creature cantrip. I can forgive the indecision over rarity.
falver raven (rare) _INVALID_
1U
creature
haste
when @ enters the battlefield, draw two cards, then discard a card.
G, T: copy target instant or sorcery spell. you may choose new targets for the copy.
morph 1W
(2/1)
~~
(7) (types) ['creature']
# It couldn't remember if it needed a type or a subtype. Fascinating but typical desing - complex, coherent, yet utterly nonsensical and very loose with the color pie.
insriver strength _INVALID_
7UU
sorcery
target creature gains flying until end of turn.
cycling 4U
~~
(2) (types) ['sorcery']
# Bloody typical, ain't it.
spellbreaker's malechain (uncommon) _INVALID_
1U
instant
destroy target nonblack creature. it can't be regenerated.
~~
(9) (cost) {2}{U}
(7) (rarity) O
# Also this.
stonehorn (common)
1U
sorcery
target creature gets -5/-5 until end of turn.
# And this.
spitting peace (common)
1U
instant
target player discards four cards.
# And also this. I'd play all of these. Instant speed discard 4 for 1? Apparently blue is the new black, but much better.
crowd farcher (rare)
2B
creature ~ centaur
B, sacrifice an aura: remove a +1/+1 counter from @: @ gains shadow until end of turn.
(1/2)
# Two colons... this ability's cost has a cost!
inspired herd (common)
4B
creature ~ elemental
@ can't be blocked by blue creatures.
whenever @ attacks, sacrifice @ and it deals 1 damage to you.
morph 1BB
(7/4)
# You have to declare it as an attacker, then turn it face up. And it even comes with limited evasion.
srolling bubble (rare)
2BBB
legendary creature ~ spirit
tromple
(3/3)
# Am I surprised?
the augur (common)
2R
sorcery
draw a card for each card in your hand.
# Obviously there are several bad ideas going on here, the least of which is making it a common. And the most similar card is Demonic Collusion?
disembowel
4R
instant
draw a card for each color among permanents you control.
# Also, this.
stas cobra (common) _INVALID_
1R
sorcery
draw a card for each instant or sorcery card in your graveyard.
~~
(3) (cost) {3}{W}{W}
# Even more impressive. And no it wouldn't have been any better in white.
claurtrooming (rare) _INVALID_
2RR
instant
you gain 7 life.
~~
(8) (name) condraclation
# The color issues are real.
hatchet behemoth (rare)
4RR
creature ~ avatar
kicker 2B
when @ enters the battlefield, if it was kicked, target opponent puts X 1/1 white spirit creature tokens onto the battlefield.
(5/5)
# Another working kicker! It still messed up this X though.
telepire's crush (uncommon) _INVALID_
3R
instant
[choose one ~
= destroy target artifact creature.
= draw a card.]
entwine 2
~~
(6) (rarity) O
# Hey look an entwine card!
boggart plagues
4G
creature ~ zombie
you may have @ enter the battlefield as a copy of any nonland permanent you control.
(2/2)
# Bizarre.
charging sentry
2G
creature ~ human pirate
whenever you cast a noncreature spell, you may return @ from your graveyard to the battlefield. activate this ability only once each turn.
(3/1)
# Also bizarre. Can the ability retain state to work in the way it's clearly intended to?
dark's herd (uncommon) _INVALID_
G
sorcery
target player reveals his or her hand. you choose a nonland card from it and exile them. then that player shuffles his or her library.
draw a card.
~~
(7) (types) ['instant']
# I assume "them" refers to the card. Otherwise, I like it. Completely unfair and obviously the product of a neural net. If only it had been decisive about making it an instant.
deathsraper legion (mythic rare)
2GG
creature ~ zombie drake
intimidate
1: @ becomes colorless until end of turn.
(3/3)
# Clever.
kwardsory (uncommon)
1G
instant
[choose one ~
= destroy target enchantment.
= target creature gains flying until end of turn.]
# Another working choice card.
thornscape aarhand _INVALID_
2G
creature
when @ enters the battlefield, if it was kicked, put X 1/1 green saproling creature tokens onto the battlefield.
(2/2)
~~
(8) (name) sphinx's nelusal
# Things don't always work out.
traveling kard (uncommon) _INVALID_
5G
instant
choose a color. target creature gets +3/+0 until end of turn.
~~
(8) (cost) {2}{B}
# Nope.
infernal fire (uncommon)
1(R/G)(R/G)
sorcery
draw X cards, then discard two cards.
# Indeed not.
advance spitter
4
artifact
spells you cast cost 1 less to cast.
T: add one mana of any color to your mana pool.
# This, though, is pretty cool. I like all of it except the rarity.
prince of malachite (rare)
5
artifact
countertype % paye
at the beginning of your upkeep, you may put a % counter on @.
2, sacrifice @: add 1 to your mana pool for each % counter on @.
# Working and creatively named counters.
penforming (uncommon)
1
land
players can't cast spells.
# Well... Can you play lands with casting costs?
temple ward _INVALID_
_NOCOST_
legendary land ~ beast
2, T: prevent all combat damage that would be dealt this turn.
~~
(8) (name) kitsune foundry
# This also would be a bit borked.
TL;DR: This network definitely has interesting properties. It's almost obsessively good at putting double quotes around granted abilities, and it has showed some promise with kicker, but I didn't see any correct usage of X, and it still has many shortcomings common to other networks.
For the curious, the MSE set with all of the cards from my dump is here. As with all MSE sets, google drive thinks it's a zip file. You want to click the download at the top, rather than just pulling out the text file named "set".
The checkpoint itself is here. The epoch number is misleading - it's 0.43 of 1.0, where 1.0 corresponds to epoch 100. So it's really epoch 43.
This is the most fun I've had looking through card dumps in a while. A shout out to Maplesmall - doing it in MSE is very pleasant, much better than text.