I saw a bunch of cards coming out with "if you have &^^^^ or more cards in hand" as well, with no settings changed except to pick a different random seed. I think I used 2929 that time.
EDIT: Woot, pretty cards working again! I do think there's some sort of bleed happening across sessions; halfway through my latest generation, it started spitting out a bunch of cards named "Akaito" then "Yukkuri," which is not anything I requested (I clicked "generate" with everything at default, random seed was 124 I think).
I saw a bunch of cards coming out with "if you have &^^^^ or more cards in hand" as well, with no settings changed except to pick a different random seed. I think I used 2929 that time.
EDIT: Woot, pretty cards working again! I do think there's some sort of bleed happening across sessions; halfway through my latest generation, it started spitting out a bunch of cards named "Akaito" then "Yukkuri," which is not anything I requested (I clicked "generate" with everything at default, random seed was 124 I think).
Also, the progress bar goes well above 100%!
There IS bleeding happening. I don't know why. Either the different function calls, which are in their own processes, are somehow sharing the same pipe (which doesn't look like it should be the case based on python's documentation), or the websocket is firing to all browsers that happen to be in the generation step.
I need to get some more lesson planning done (going standards based grading is harder than I thought) so I wont be able to attack this again for a couple days.
Also the 100% is because I'm sneaking in about 100 more characters to help compensate for the nn debug output and incomplete cards. Sometimes it ends up below, sometimes it ends up above. It is also going above 100% because you are being sent cards being generated by other people. I didn't realize the site would be popular enough to have multiple simultaneous users
Private Mod Note
():
Rollback Post to RevisionRollBack
Proud to be saving the world since 1984 -- I also have an open source website to make AI generated magic cards. Source code
I saw a bunch of cards coming out with "if you have &^^^^ or more cards in hand" as well, with no settings changed except to pick a different random seed. I think I used 2929 that time.
EDIT: Woot, pretty cards working again! I do think there's some sort of bleed happening across sessions; halfway through my latest generation, it started spitting out a bunch of cards named "Akaito" then "Yukkuri," which is not anything I requested (I clicked "generate" with everything at default, random seed was 124 I think).
Also, the progress bar goes well above 100%!
There IS bleeding happening. I don't know why. Either the different function calls, which are in their own processes, are somehow sharing the same pipe (which doesn't look like it should be the case based on python's documentation), or the websocket is firing to all browsers that happen to be in the generation step.
I need to get some more lesson planning done (going standards based grading is harder than I thought) so I wont be able to attack this again for a couple days.
Also the 100% is because I'm sneaking in about 100 more characters to help compensate for the nn debug output and incomplete cards. Sometimes it ends up below, sometimes it ends up above. It is also going above 100% because you are being sent cards being generated by other people. I didn't realize the site would be popular enough to have multiple simultaneous users
It's failing in a consistent way for me.
I requested cards named 'unique ' and it gave me two, then stalled. Only the first actually rendered to card text. This seems to always be the case.
After waiting for a few minutes, on a different computer I requested cards named 'enthralling '. I got two, and only the first rendered to card text. Both of those cards and nine otheres (all starting with 'enthralling ') appeared on the first computer below the two with 'unique'. And all 11 rendered.
A few minutes later, while writing this, 16 more cards with no distinguishing features appeared on the second computer. None rendered. It still shows 47.52% a few minutes later.
The first seven of these nonspecific cards appeard on the first computer. And then the same seven again. And then the first thirteen (i.e the first seven plus the next six). None rendered to card text. Progress is 169.17%
Meanwhile back on the second computer, the first five nonspecific cards showed up a second time for that computer. So those five have been received a total of five times between the two computers. The secons computer has still only rendered the very first 'enthralling ' card as card text, and is now on 96.23%, remaining so after some minutes.
The same kind of thing happens when using two browser tabs on the same computer. Note that I've always waited several minutes between runs. I've never tried to queue up two at the same time.
[Edit] okay I tried starting a run with cards named 'one ' and then a couple of seconds later a run with cards named 'two ', in separate tabs on computer two.
The 'one ' tab produced 3 'one ' cards and rendered the first 2 (they were fairly short cards). It's stuck on 27.55%
The 'two ' tab produced 2 'two ' cards and rendered only the first. It's stuck on 17.85%
The original second-computer tab received and rendered 13 'one ' cards. Then it received and rendered 13 'two ' cards.
The original first-computer tab also received and rendered the same 13 'one ' cards and the 13 'two ' cards.
Both of those now say 100%.
So the best way to get all the cards you want is to do a dry run and then leave the tab open to catch the second run. But you might catch someone else's too. Interestingly, the cards I caught from my other computer and the same computer rendered where they were caught. But the cards I caught from someone else (some multiple times) were not rendered when caught.
I saw a bunch of cards coming out with "if you have &^^^^ or more cards in hand" as well, with no settings changed except to pick a different random seed. I think I used 2929 that time.
EDIT: Woot, pretty cards working again! I do think there's some sort of bleed happening across sessions; halfway through my latest generation, it started spitting out a bunch of cards named "Akaito" then "Yukkuri," which is not anything I requested (I clicked "generate" with everything at default, random seed was 124 I think).
Also, the progress bar goes well above 100%!
I might be the one that caused the "&^^^^ or more cards in hand" Since I feed those input earlier trying to coax some idea from the neural net. Weird thing though when I was using it, I only get 1 card that has that condition per run and the rest are a bunch of random cards.
@Talcos It's great that you replied to such length, I'm so sorry I hadn't been clear
* {X} in manacost I renamed Y. Works fine with a tag in repeater(50%-90% are OK, depends on net)
* X with "where X is" I renamed Z. Works fine (50-90% OK) "at next break, put a 'where Z is'"
* other X's are a big flat 0%. Not even 5%. I'm speaking of those. It's mostly "{X}:" or "Remove X counters:" then something involving an X should follow. You'd think after it has put a first X, it's the same situation as Y. The surprising thing is the contrast X vs Y and Z. In my current corpus, I have 348 Z, 231 Y, 148 X cards, X is not that far off down.
Other types of association are surprisingly difficult for the net to learn. 378 cards have quotes, yet it's quite confused! (Not a problem per se of course, easy to correct.) As you say, "...could go in virtually any direction. That creates noise that suggests that the network has more latitude than it actually does."
I'm currently trying the "more learning" route, that could work too. But if there's an easy tweak of a gradient parameter, it's worth trying.
There might be an easy tweak. I'm not quite sure what'll work best. But there are things we can look into. But yeah, let me know how additional training works out.
---
For the record, I'm working with Elseleth and tinkering with the set generation process. Generated new batches of cards from the most recent network. I'm very interested to see how things turn out.
The network comes up with some very interesting commons and uncommons, but it's difficult to say what kind of work is needed to balance or tweak things for limited purposes. Here are some examples (names mine).
Sometimes the network comes up with cards that are very interesting but its difficult to predict what kind of impact they'd actually have in a limited environment:
Sheltering Griffin 3W
Creature - Griffin (Uncommon)
Flying
If a source would deal damage to you, prevent 1 of that damage.
When Sheltering Griffin enters the battlefield, you may return target instant card from your graveyard to your hand.
2/3
Wandering Mage 2G
Creature - Human Wizard (Uncommon)
When Wandering Mage enters the battlefield, you may search your library for a Mountain or Forest card, reveal it, put it into your hand, then shuffle your library.
Evoke 2R
1/1
Othertimes the network comes up with things that are really ingenious but... not quite what we are looking for.
Trial of Purity 2W
Enchantment (Rare)
Whenever a creature enters the battlefield under your control, it deals damage to itself equal to its power.
Crystal Tutor 1BB
Enchantment (Uncommon)
When Crystal Tutor leaves the battlefield, you may search your library for a card and put that card into your hand. Then shuffle your library.
Dredge 4 So wondrous to behold, so delicate and finely crafted — and yet, such a pleasure to smash.
#Yeah, this card comes off as a broken version of Hatching Plans to me.
@Talcos It's great that you replied to such length, I'm so sorry I hadn't been clear
* {X} in manacost I renamed Y. Works fine with a tag in repeater(50%-90% are OK, depends on net)
* X with "where X is" I renamed Z. Works fine (50-90% OK) "at next break, put a 'where Z is'"
* other X's are a big flat 0%. Not even 5%. I'm speaking of those. It's mostly "{X}:" or "Remove X counters:" then something involving an X should follow. You'd think after it has put a first X, it's the same situation as Y. The surprising thing is the contrast X vs Y and Z. In my current corpus, I have 348 Z, 231 Y, 148 X cards, X is not that far off down.
Other types of association are surprisingly difficult for the net to learn. 378 cards have quotes, yet it's quite confused! (Not a problem per se of course, easy to correct.) As you say, "...could go in virtually any direction. That creates noise that suggests that the network has more latitude than it actually does."
I'm currently trying the "more learning" route, that could work too. But if there's an easy tweak of a gradient parameter, it's worth trying.
There might be an easy tweak. I'm not quite sure what'll work best. But there are things we can look into. But yeah, let me know how additional training works out.
---
For the record, I'm working with Elseleth and tinkering with the set generation process. Generated new batches of cards from the most recent network. I'm very interested to see how things turn out.
The network comes up with some very interesting commons and uncommons, but it's difficult to say what kind of work is needed to balance or tweak things for limited purposes. Here are some examples (names mine).
Sometimes the network comes up with cards that are very interesting but its difficult to predict what kind of impact they'd actually have in a limited environment:
Sheltering Griffin 3W
Creature - Griffin (Uncommon)
Flying
If a source would deal damage to you, prevent 1 of that damage.
When Sheltering Griffin enters the battlefield, you may return target instant card from your graveyard to your hand.
2/3
Wandering Mage 2G
Creature - Human Wizard (Uncommon)
When Wandering Mage enters the battlefield, you may search your library for a Mountain or Forest card, reveal it, put it into your hand, then shuffle your library.
Evoke 2R
1/1
Othertimes the network comes up with things that are really ingenious but... not quite what we are looking for.
Trial of Purity 2W
Enchantment (Rare)
Whenever a creature enters the battlefield under your control, it deals damage to itself equal to its power.
Crystal Tutor 1BB
Enchantment (Uncommon)
When Crystal Tutor leaves the battlefield, you may search your library for a card and put that card into your hand. Then shuffle your library.
Dredge 4 So wondrous to behold, so delicate and finely crafted — and yet, such a pleasure to smash.
#Yeah, this card comes off as a broken version of Hatching Plans to me.
I'll let y'all know how things go.
To be fair, Crystal Tutor wouldn't be that powerful in a Limited environment, where your access to reliable enchantment removal would be ... Well, limited. Especially if you commit heavily to black (double B in the cost).
But yeah, in constructed this thing is a game breaker.
I saw a bunch of cards coming out with "if you have &^^^^ or more cards in hand" as well, with no settings changed except to pick a different random seed. I think I used 2929 that time.
EDIT: Woot, pretty cards working again! I do think there's some sort of bleed happening across sessions; halfway through my latest generation, it started spitting out a bunch of cards named "Akaito" then "Yukkuri," which is not anything I requested (I clicked "generate" with everything at default, random seed was 124 I think).
Also, the progress bar goes well above 100%!
I might be the one that caused the "&^^^^ or more cards in hand" Since I feed those input earlier trying to coax some idea from the neural net. Weird thing though when I was using it, I only get 1 card that has that condition per run and the rest are a bunch of random cards.
Yes that is what was happening to me. It gets a card or two, then the rest are sent who-knows where.
I left one of my tabs from earlier open, and it's collected another thirty-something cards, a few of them twice.
Used croxis's web UI to generate some uncommons, then left the output page up for some 9 or 10 hours... came back just now to find that there were several hundred more cards than the initial fifteen or so, including the results of Mustard_Fountain's experiments with cards named "one" and "two".
Also:
war~crap rebound (uncommon)
{1}{R}
instant
War~Crap Rebound deals 2 damage to target creature or player. after this main phase, there is an additional combat phase. then if a creature died this turn, return War~Crap Rebound to the battlefield.
#Heh. 'War~crap.'
#Though, if returning a non-permanent card to the battlefield is to be interpreted as recasting the spell there may be some balance issues.
#Got a planeswalker out of the network without specifying for it to do so. The lack of subtype and starting loyalty is problematic:
|thalus of the spirit||planeswalker|&^^^^^^|kara|Y||{^^UUUU^}|+&^: look at the top two cards of your library. put one of those cards into your hand and the rest on the bottom of your library in any order.|
thalus of the spirit (mythic)
{3}{U}{U}
planeswalker
+1: look at the top two cards of your library. put one of those cards into your hand and the rest on the bottom of your library in any order.
Sorry, I don't know enough of the wiring of the net, it's just a general feeling that we observe outputs that are carried away by what was decided just before. Dashed text beginning always with the same words? Learnt perfectly even with a few cards. Something even short is there before the recurring pattern recurs? Needs hundreds of cards to be learnt partially. I'd put that next to X and quote observations. Something's wrong somewhere. Wouldn't low temp favor local effects? But why can't we go to temp 1?
Well maybe training for 500 epochs works too
You're fine!
I hypothesize that the problem with X costs (as a prime example of the problems we have with the network) is three-fold:
* First, X costs can occur in the mana cost, leading to situations where X doesn't need to be defined in the body text (e.g. Blaze). When the network generates a card with an undefined X cost, it may mistakenly think that it already tied X to the mana cost.
* Second, definitions of X almost always follow commas (e.g. Accelerated Mutation, and commas indicate topic shifts, where the network expects to have some leeway as to what comes next. That is misleading for the network.
* Third, the length of the dependency can be highly variable. Sometimes the distance between X and its definition is short (e.g. Aven Shrine), and sometimes it's longer (e.g. Aura Mutation).
That second and third problems are also related to why the network struggles with kicker costs. There's always some intervening text that, when viewed without regards to the kicker ability, could go in virtually any direction. That creates noise that suggests that the network has more latitude than it actually does.
There are trade-offs here, of course. The network wants to be the best it can at predicting the text of cards. We would like for it to be great at predicting cards with X-costs, but not at the expense of its ability to predict other kinds of cards. X-cost cards are particularly tricky because they represent a minority of cards and because they exhibit context-sensitive dependencies. We want the network to give us the best approximation it can of such cards without overfitting on them and without learning rules that bleed into the majority of cards that do not exhibit their features.
Don't think of it as a dumb mistake, think of it as a calculated sacrifice.
Now, there are ways of making life easier for the network, such as flattening the grammar, boiling things down to the essentials where needed. That's helpful. But we should be clear that for any non-trivial, complex language, there will always be some amount of "garbage" output due to strategic underapproximation (especially since we have such a severe input shortage). This "weakness" arises out of the greatest strength of stochastic models like our RNN, which, I might add, is one of the most powerful and sophisticated models that I have ever encountered.
As we've discussed, there are ways for us to specialize the architecture to better solve our problem. Better training regimes, better input preprocessing, etc. We could also add modules that refine the output, like a grammar checker or a semantics approximator. After all, the secret to good writing is rewriting.
I'd like to add my own observations / musings about the format.
The interesting thing about Magic cards is that some parts depend on others. A lot of these dependencies are pretty simple to detect and explain, which is encouraging.
There's (at least) two very different kinds of dependencies. First, some parts of a card might impose a constraint. For example, if the card has type 'creature', then it had better not have any lines of text like "draw a card", and it probably should have a subtype or two. Meanwhile if a card has type 'instant', it can have a line that just says "draw a card", but it shouldn't have activated abilities or effects that trigger on entering the battlefield.
Second, some parts of card might impose a requirement. The classic examples here are things like X, kicker, an being an equipment. If a card costs X, then it had better do something that involves X. If it doesn't, but it still uses X in an ability, then it needs to define what X is afterward. If a card is an equipment, then it really should say "equip <COST>" somewhere.
Constraints are about what a card can't have. Requirements are about what it needs to have.
There's a certain duality here: if the network is producing a card, and it has already made it a creature, then that sets constraints on the card's text; on the other hand if it's writing the text and gives it flying, then that's a requirement that the card be a creature. What's a constraint in one direction is often a requirement in the other. Note that since we control the order in which the network generates parts of a card, we have a great deal of control here, for better or for worse. Experimentally, throwing it away and allowing the fields to be in a random but labeled order makes it much harder for the network to generate well-formed cards.
Here's what I observe. Usually the networks are pretty good with constraints. We do still see mistakes where sorceries have activated abilities and so on, but there are plenty of reasonable cards, too. Requirements are a much, much bigger problem. For some of them the hit rate is near zero, like kicker and some situations involving X. Others are actually success stories though, like equip. By putting it first thing in the text, right after the types, most modern networks are pretty good at coming up with working equipment cards.
So, some requirements are much harder to get right than others. This actually makes perfect sense. To paraphrase jml, if you require something to happen right at the start of a line, or right immediately after some sort of context switch, the network does much better. For obvious reasons. It knows that it needs to insert something, so if there are convenient points available to start inserting it, then it will. Of all the characters it could start with after the context switch, the ones that make it start meeting the requirement are the most likely. If there's a lot of other stuff in the way, however, then the network has a very hard time. Imagine you're a network generating a card, and you stick X in the cost. There are only so many lines of text in a card, and it's very hard to tell from the first character if that line will use the X from the casting cost or not. It's not that you, the network, don't know that you should use the X, it's just that when you start making a line it's very hard to guarantee that that line will be able to use X, and once you've gotten a little ways into the text of the line there are much more immediate restrictions to the text you can generate. You can't begin with "uncast target" and then just throw X on, you're probably going to say something like "spell\".
The goal, then, is to come up with a format that minimizes the bad requirements. We want things that impose constraints to come first, and things that would impose bad requirements to come early enough that those requirements are either converted into constraints or are less bad.
So, the important bits of the format are probably:
|type|subtypes|p/t|text||cost|
Type first, because it imposes constraints on everything. I think it makes sense to put subtypes and p/t before anything else, as they are so strongly influenced by type. Next is the text, and finally the cost. If we put it at the end, hopefully X cards will be easier, because the requirement is a lot simpler that way. All the network has to do is stick an X somewhere in the cost, as opposed to fabricating an entire line of text.
You could put text immediately after types, which might help issues with cards having inappropriate abilities, but text is long and arbitrary so it would probably make subtypes and p/t less consistent. It would be interesting to see if a properly trained network could remember things over a large distance like that.
Supertypes should go near type. You could put it first, but I'm more inclined to put supertypes after type and let type be the primary decider of what kind of card it is. Loyalty should probably go near p/t, you could reuse the field but I think it's less confusing for the network if you don't, and the extra delimiter isn't a huge price.
Rarity and name can go anywhere. Both are extremely nebulous constraints; that makes me want to tack them on the end. Or, you could put rarity first, otherwise the distribution of rarities will be skewed toward whatever the network is more likely to create in the absence of rarity information. For set generation, this probably doesn't matter, as we can overgenerate and filter to get the distribution we want anyway. I think it does make the most sense to put the name last, as then you can easily chop it off to make a more consistent network, as Talcos is doing now, and feed in unnamed cards to come up with names for them.
Of course you can trivially chop name off for the 'stable' format. You can also make an argument for supporting a name-first, 'intuitive' format, that allows seeding names to come up with whole cards, which is a pretty cool thing to be able to do:
Then there's the issue of the text. I'm thinking that some improvements can be made inside the text field by cutting it up into lines and reordering them in a better way. Each line is exactly one ability - if you have something like "flying, first strike" I'd identify the separate keywords and chop it into "flying" and "first strike". We can try to order the lines in a way that is convenient: put equip costs first, and kicker costs last, so the requirement is to add a simple kicker cost line rather than a complicated one with "if @ was kicked".
There are other games to play as well. It might be possible to abstract out "where X is" definitions into a separate sentence, after a full stop (or period, colloquially), since that's a pretty strong context switch, or even as a separate line. This mostly works as is, but you could try to unify with other forms of X, possibly even costs. I think this has been suggested before.
Whew. Apologies for being long-winded or restating anything obvious. Just wanted to get it out while it was on my mind. The custom batcher is coming along, latest version on mtg-rnn can randomize mana costs and fields. The 'exactly one card at a time' mode should be finished soon. My ground truth experiment for that finished training after more than 110 hours of GPU time, so in another week or so I'll be able to give some results.
Wandering Mage 2G
Creature - Human Wizard (Uncommon)
When Wandering Mage enters the battlefield, you may search your library for a Mountain or Forest card, reveal it, put it into your hand, then shuffle your library.
Evoke 2R
1/1
I like the mechanics of this card, even if the flavour feels a little off. I mean, it's land searching in red, but you don't get a body like you would if you played green. If it said "basic Mountain or Forest", I think it'd be perfectly fine as a real card. Also should be a shaman and not a wizard, but let's not get too into details here.
If I were to redefine my format regarding types I'd completely let go of the distinction super/plain/sub, it is unimportant. You can rebuild the order of the magic card at postprocessing. What influences the card's contents the most? That's the focus.
Likewise for planeswalkers: even if you like them the net doesn't care specially for them. They're too rare so they must be squeezed in the general format. Not a problem if the initial loyalty looks like an equipment cost or whatever works.
I have to disagree with you here. I won't claim that separating the different kinds of types makes a difference one way or another, as the only way to get a satisfactory answer would be to generate some data. But, I think the cost of adding structure is very small. Separating into 3 fields only costs two more delimiters, and it has vastly simplified what the network needs to generate. Correctly finishing the type field is extremely predictable, rather that dragging in fairly open-ended creature subtypes.
The thing is the network is good at delimited fields. Extremely good. It can generate 65000 cards with only one or two mistakes due to badly delimited fields, and those might have been the incomplete cards on the end, I don't remember. And it's also pretty good at realizing that creatures have subtypes while other types of cards usually don't, except arcane, and other little things like that.
We certainly don't want to make planeswalkers look like other cards. If they're too different, then the best approach is just to get rid of them and admit that we can't generate them at all. But this isn't what we observe: the network is quite capable of generating working planeswalkers, it just trips up sometimes because it's reluctant to make them long enough to have 3 abilities. It usually gets the loyalty costs right.
TL;DR - Structure is a good things. Networks are good at learning structure, but bad at dealing with complex relationships in pure text, like kicker. So the more we can do to add predictable structure to the text of a card, the better.
Crystal Tutor 1BB
Enchantment (Uncommon)
When Crystal Tutor leaves the battlefield, you may search your library for a card and put that card into your hand. Then shuffle your library.
Dredge 4 So wondrous to behold, so delicate and finely crafted — and yet, such a pleasure to smash.
#Yeah, this card comes off as a broken version of Hatching Plans to me.
To be fair, Crystal Tutor wouldn't be that powerful in a Limited environment, where your access to reliable enchantment removal would be ... Well, limited. Especially if you commit heavily to black (double B in the cost).
But yeah, in constructed this thing is a game breaker.
Precisely. It's a nice card, but it'll never see constructed play, and it's not a very good fit for limited applications unless the network also generates a convenient way to sacrifice enchantments in black.
#Got a planeswalker out of the network without specifying for it to do so. The lack of subtype and starting loyalty is problematic:
|thalus of the spirit||planeswalker|&^^^^^^|kara|Y||{^^UUUU^}|+&^: look at the top two cards of your library. put one of those cards into your hand and the rest on the bottom of your library in any order.|
thalus of the spirit (mythic)
{3}{U}{U}
planeswalker
+1: look at the top two cards of your library. put one of those cards into your hand and the rest on the bottom of your library in any order.
Just so you know, the subtype is Kara, and the starting loyalty is 6.
Whew. Apologies for being long-winded or restating anything obvious. Just wanted to get it out while it was on my mind. The custom batcher is coming along, latest version on mtg-rnn can randomize mana costs and fields. The 'exactly one card at a time' mode should be finished soon. My ground truth experiment for that finished training after more than 110 hours of GPU time, so in another week or so I'll be able to give some results.
Excellent ideas! I will warn you that moving the cost to where it's after the rarity may cause some distortions of color. For example, with the current network, we can get a card with stats like
2W
Creature - Human Soldier
Vigilance 2R : @ gets +2/+0 until end of turn.
and that same card could have a mana cost of 2R if you take the text into consideration when generating the cost. Not that one outcome is better than the other, just something to note. Meanwhile, I agree with you completely about supertypes and so on, some reordering would definitely be for the best.
---
Honestly, if I had the time and energy to reimplement our neural architecture from scratch, here's what I'd try.
Train a network to generate everything but the body text, which can be handled by a separate network.
Feed the body text network the text character-by-character as we have been doing, but continuously repeat key information about the card like color and types. So the input to the network at each time step would look like:
[one-hot encoding of character, context]
Where context is
[WUBRG color vector (e.g. <1,0,0,0,1> = GW), CMC, field encoding vectors (supertype/type/subtype), power,toughness,loyalty]
So at each time step, as the network predicts each character, its knowledge of the context of the body text is refreshed. This liberates the network from having to keep track of that information while simultaneously tracking information about the body text. I'm very confident that if you do this, you'll see fewer color issues and fewer misplaced text issues. I was inspired by that paper about a neural network that generated images of chairs that I showed y'all before. If you see the attached image, you'll note that the network receives the class input (the kind of chair that we want), but is also separately fed information about the angle and the lighting for that chair. It'd be sort of like that.
From there you can feed the resulting output to any number of filter networks for syntactic/semantic refinement.
Now, I think that a general-purpose, unspecialized network like what we have would work just fine if we had more inputs for training, but our corpus is limited in size and we struggle to teach the network some things like color discipline. I mean, we'll stick with what we've got for now, but in the long run I have the feeling that segmenting the network in this way will deliver the best results.
Wandering Mage 2G
Creature - Human Wizard (Uncommon)
When Wandering Mage enters the battlefield, you may search your library for a Mountain or Forest card, reveal it, put it into your hand, then shuffle your library.
Evoke 2R
1/1
I like the mechanics of this card, even if the flavour feels a little off. I mean, it's land searching in red, but you don't get a body like you would if you played green. If it said "basic Mountain or Forest", I think it'd be perfectly fine as a real card. Also should be a shaman and not a wizard, but let's not get too into details here.
The subtype is probably my fault. What happened was that the network first decided that it wanted to make a Human Wizard, and then I forced it to make a green creature with evoke, and that's what we ended up with. If the network had planned on making it a green creature from the start, it probably would have been a druid or shaman.
OK, here's a kind of demented idea for calculating X costs.
Have the neurons divided in two "pages" - Page A and Page B.
First, both pages A and B parses this text:
Destroy target enchantment. Put X --
Then, whenever X appears on a created rules text, each page gets to write their own follow-up, and they're later merged.
That's not a bad idea, actually. It creates a marketplace of ideas. My concern, however, is that during training, if one of the pages makes a correct prediction and the other makes a wrong prediction, then backpropagation would reweight the wrong one over time to be more like the correct one. Unless you're careful with how you do your training, they'd end up in synchrony with each other. But I think that if you do the training right, you can avoid that sort of problem.
war~crap rebound (uncommon)
{1}{R}
instant
War~Crap Rebound deals 2 damage to target creature or player. after this main phase, there is an additional combat phase. then if a creature died this turn, return War~Crap Rebound to the battlefield.
#Heh. 'War~crap.'
#Though, if returning a non-permanent card to the battlefield is to be interpreted as recasting the spell there may be some balance issues.
Easy to fix so that it's interesting, though. Probably drop the damage off of it and have the "die" triggered on that combat phase. Also, make it a sorcery.
After this phase, there is an additional combat phase. Then, if a creature died during that combat phase, you may cast a copy of War~Crap Rebound again without paying it's mana cost.
Something like that. Also, cost is too low.
Private Mod Note
():
Rollback Post to RevisionRollBack
Why did I ever think a signature in comic sans was a good idea?
Just an update. I generated a batch of cards using the set constructor and passed that off to Elseleth, who has been wanting to organize a draft (and has already purchased sleeves and ink for printing). I engineered it such that the set constructor would provide up to three alternatives for each slot, but I made the garbage filtering fairly conservative, so most of the selections at this point will be for basic, staple kinds of cards. We may end up digging through a large dump later to find broken but fixable interesting cards to fill remaining slots in the set.
Then comes any necessary fine-tuning, balancing, playtesting, flavor/name/art stuff, etc. But we'll cross those bridges when we get to them.
Unless you're careful with how you do your training, they'd end up in synchrony with each other
At work we implemented a paper that has a penalty on the correlation between two sets of units in order to prevent them from coming into synchrony, and it worked wonderfully. I'm trying to dig up the paper, and I'll edit it into this post when I find it.
EDIT: I found the paper! It's here. The gist of it is that they wanted to predict facial expression and ignore identity. So they use this trick to make some hidden units really good at reconstructing the image and then made another group of units have a minimal cross-covariance with the reconstruct units. The idea being that these units would then be forced to learn something different but useful.
Fancy new ui feature! The card generation and image loading is almost parallel, meaning the page will load and start displaying cards as they generate*. This is available for all formats.
*So this works on my home development machine but is slow and unstable on the actual server. I'm going to have to do more testing. My hunch is that the neural net takes longer to warm up on my server and is blocking the websockets, causing them to time out. I am probably going to have to put the neural net behind python's multiprocess module to prevent GIL issues.
The choice of images this thing uses is frequently as hilarious as the older cards at the beginning of the thread. I mean, look at this:
Honestly, if I had the time and energy to reimplement our neural architecture from scratch, here's what I'd try[...]
Now that I've got my toolbox (mostly) up and running for RNN training, I can give that a try once I've got the time.
Feel free to try! But before I send you off on any wild goose chases I should warn you: everything I know about machine learning and neural networks came to me in the past four months or so. Any big, bold suggestions that I make are the products of my intuition and what I've learned from studying the relevant literature; I have little hands-on experience to confirm that what I'm suggesting will work. On the other hand, I'm a fast learner, and my intuitions rarely steer me in the wrong direction.
At work we implemented a paper that has a penalty on the correlation between two sets of units in order to prevent them from coming into synchrony, and it worked wonderfully. I'm trying to dig up the paper, and I'll edit it into this post when I find it.
Oh really? I'd love to see it. I figured that such a thing was possible but I wasn't sure whether anyone had made a documented attempt at it.
Hey all been following this post for a while now and i wanted to say this is all pretty amazing.
i had a few quick queations.
First up, is the link to the how to set up your own training network still current?
Are the linked sampling and sample files current?
If i were to sink a few days of time training up a new network is there any new things any of you would like me to try. (Basically donating a day or five of gpu time here)
This link should be an up-to-date tutorial, considering that I just wrote it a few days ago. I'd be very appreciative if you could test it out and provide feedback on any major shortcomings. I can try to provide some additional tech support if you're having trouble making GPUs work on a native Linux install.
I'm considering a major overhaul to the format, so if you want to be a cutting-edge alpha tester you could wait until tonight, when hopefully I can push out a preliminary version. If you want to donate a lot of time, you could also help by testing out 'one card at a time' training mode, which is going to be massively slow.
These things are still being developed - I will post new changes as they happen.
Feel free to try! But before I send you off on any wild goose chases I should warn you: everything I know about machine learning and neural networks came to me in the past four months or so. Any big, bold suggestions that I make are the products of my intuition and what I've learned from studying the relevant literature; I have little hands-on experience to confirm that what I'm suggesting will work. On the other hand, I'm a fast learner, and my intuitions rarely steer me in the wrong direction
This idea is actually very close to some things I was hoping to try anyway, and the architectural changes are trivial to implement. The only part that would take some time is getting the context data, and even that shouldn't take much time.
I guess I don't get the Croxis interface. I've tried to put "Kobolds you control have" in the prepend text a few times and I get no cards with that back from it. I also tried it with goblins.
Private Mod Note
():
Rollback Post to RevisionRollBack
Why did I ever think a signature in comic sans was a good idea?
I guess I don't get the Croxis interface. I've tried to put "Kobolds you control have" in the prepend text a few times and I get no cards with that back from it. I also tried it with goblins.
Try doing it sgain in a second tab while the first tab is still open.
Well I was going to post about an update, but now the neural net is throwing a hissy fit:
/home/croxis/torch/install/bin/luajit: bad argument #1 to '?' (empty tensor at /home/croxis/torch/pkg/torch/generic/Tensor.c:851)
stack traceback:
[C]: at 0x7f6782dc2cc0
[C]: in function '__index'
sample_hs_v3.1.lua:202: in main chunk
[C]: in function 'dofile'
...oxis/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:131: in main chunk
[C]: at 0x00405be0
Private Mod Note
():
Rollback Post to RevisionRollBack
Proud to be saving the world since 1984 -- I also have an open source website to make AI generated magic cards. Source code
To post a comment, please login or register a new account.
EDIT: Woot, pretty cards working again! I do think there's some sort of bleed happening across sessions; halfway through my latest generation, it started spitting out a bunch of cards named "Akaito" then "Yukkuri," which is not anything I requested (I clicked "generate" with everything at default, random seed was 124 I think).
Also, the progress bar goes well above 100%!
There IS bleeding happening. I don't know why. Either the different function calls, which are in their own processes, are somehow sharing the same pipe (which doesn't look like it should be the case based on python's documentation), or the websocket is firing to all browsers that happen to be in the generation step.
I need to get some more lesson planning done (going standards based grading is harder than I thought) so I wont be able to attack this again for a couple days.
Also the 100% is because I'm sneaking in about 100 more characters to help compensate for the nn debug output and incomplete cards. Sometimes it ends up below, sometimes it ends up above. It is also going above 100% because you are being sent cards being generated by other people. I didn't realize the site would be popular enough to have multiple simultaneous users
I requested cards named 'unique ' and it gave me two, then stalled. Only the first actually rendered to card text. This seems to always be the case.
After waiting for a few minutes, on a different computer I requested cards named 'enthralling '. I got two, and only the first rendered to card text. Both of those cards and nine otheres (all starting with 'enthralling ') appeared on the first computer below the two with 'unique'. And all 11 rendered.
A few minutes later, while writing this, 16 more cards with no distinguishing features appeared on the second computer. None rendered. It still shows 47.52% a few minutes later.
The first seven of these nonspecific cards appeard on the first computer. And then the same seven again. And then the first thirteen (i.e the first seven plus the next six). None rendered to card text. Progress is 169.17%
Meanwhile back on the second computer, the first five nonspecific cards showed up a second time for that computer. So those five have been received a total of five times between the two computers. The secons computer has still only rendered the very first 'enthralling ' card as card text, and is now on 96.23%, remaining so after some minutes.
The same kind of thing happens when using two browser tabs on the same computer. Note that I've always waited several minutes between runs. I've never tried to queue up two at the same time.
[Edit] okay I tried starting a run with cards named 'one ' and then a couple of seconds later a run with cards named 'two ', in separate tabs on computer two.
The 'one ' tab produced 3 'one ' cards and rendered the first 2 (they were fairly short cards). It's stuck on 27.55%
The 'two ' tab produced 2 'two ' cards and rendered only the first. It's stuck on 17.85%
The original second-computer tab received and rendered 13 'one ' cards. Then it received and rendered 13 'two ' cards.
The original first-computer tab also received and rendered the same 13 'one ' cards and the 13 'two ' cards.
Both of those now say 100%.
So the best way to get all the cards you want is to do a dry run and then leave the tab open to catch the second run. But you might catch someone else's too. Interestingly, the cards I caught from my other computer and the same computer rendered where they were caught. But the cards I caught from someone else (some multiple times) were not rendered when caught.
I might be the one that caused the "&^^^^ or more cards in hand" Since I feed those input earlier trying to coax some idea from the neural net. Weird thing though when I was using it, I only get 1 card that has that condition per run and the rest are a bunch of random cards.
There might be an easy tweak. I'm not quite sure what'll work best. But there are things we can look into. But yeah, let me know how additional training works out.
---
For the record, I'm working with Elseleth and tinkering with the set generation process. Generated new batches of cards from the most recent network. I'm very interested to see how things turn out.
The network comes up with some very interesting commons and uncommons, but it's difficult to say what kind of work is needed to balance or tweak things for limited purposes. Here are some examples (names mine).
Sometimes the network comes up with cards that are very interesting but its difficult to predict what kind of impact they'd actually have in a limited environment:
Sheltering Griffin
3W
Creature - Griffin (Uncommon)
Flying
If a source would deal damage to you, prevent 1 of that damage.
When Sheltering Griffin enters the battlefield, you may return target instant card from your graveyard to your hand.
2/3
Wandering Mage
2G
Creature - Human Wizard (Uncommon)
When Wandering Mage enters the battlefield, you may search your library for a Mountain or Forest card, reveal it, put it into your hand, then shuffle your library.
Evoke 2R
1/1
Othertimes the network comes up with things that are really ingenious but... not quite what we are looking for.
Trial of Purity
2W
Enchantment (Rare)
Whenever a creature enters the battlefield under your control, it deals damage to itself equal to its power.
Crystal Tutor
1BB
Enchantment (Uncommon)
When Crystal Tutor leaves the battlefield, you may search your library for a card and put that card into your hand. Then shuffle your library.
Dredge 4
So wondrous to behold, so delicate and finely crafted — and yet, such a pleasure to smash.
#Yeah, this card comes off as a broken version of Hatching Plans to me.
I'll let y'all know how things go.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
To be fair, Crystal Tutor wouldn't be that powerful in a Limited environment, where your access to reliable enchantment removal would be ... Well, limited. Especially if you commit heavily to black (double B in the cost).
But yeah, in constructed this thing is a game breaker.
I left one of my tabs from earlier open, and it's collected another thirty-something cards, a few of them twice.
Also:
war~crap rebound (uncommon)
{1}{R}
instant
War~Crap Rebound deals 2 damage to target creature or player. after this main phase, there is an additional combat phase. then if a creature died this turn, return War~Crap Rebound to the battlefield.
#Heh. 'War~crap.'
#Though, if returning a non-permanent card to the battlefield is to be interpreted as recasting the spell there may be some balance issues.
#Got a planeswalker out of the network without specifying for it to do so. The lack of subtype and starting loyalty is problematic:
|thalus of the spirit||planeswalker|&^^^^^^|kara|Y||{^^UUUU^}|+&^: look at the top two cards of your library. put one of those cards into your hand and the rest on the bottom of your library in any order.|
thalus of the spirit (mythic)
{3}{U}{U}
planeswalker
+1: look at the top two cards of your library. put one of those cards into your hand and the rest on the bottom of your library in any order.
Powered BUG Cube BUG (Thread)
Powered WUR Cube RWU
Powered RUG Cube URG
480 Unpowered shenanigans
EDH:
Prossh, Skyraider of Kher
Gahiji, Honored One
Daretti, Scrap Savant
For Autocarding: Æ û (Lim-Dûl's Vault)
I'd like to add my own observations / musings about the format.
The interesting thing about Magic cards is that some parts depend on others. A lot of these dependencies are pretty simple to detect and explain, which is encouraging.
There's (at least) two very different kinds of dependencies. First, some parts of a card might impose a constraint. For example, if the card has type 'creature', then it had better not have any lines of text like "draw a card", and it probably should have a subtype or two. Meanwhile if a card has type 'instant', it can have a line that just says "draw a card", but it shouldn't have activated abilities or effects that trigger on entering the battlefield.
Second, some parts of card might impose a requirement. The classic examples here are things like X, kicker, an being an equipment. If a card costs X, then it had better do something that involves X. If it doesn't, but it still uses X in an ability, then it needs to define what X is afterward. If a card is an equipment, then it really should say "equip <COST>" somewhere.
Constraints are about what a card can't have. Requirements are about what it needs to have.
There's a certain duality here: if the network is producing a card, and it has already made it a creature, then that sets constraints on the card's text; on the other hand if it's writing the text and gives it flying, then that's a requirement that the card be a creature. What's a constraint in one direction is often a requirement in the other. Note that since we control the order in which the network generates parts of a card, we have a great deal of control here, for better or for worse. Experimentally, throwing it away and allowing the fields to be in a random but labeled order makes it much harder for the network to generate well-formed cards.
Here's what I observe. Usually the networks are pretty good with constraints. We do still see mistakes where sorceries have activated abilities and so on, but there are plenty of reasonable cards, too. Requirements are a much, much bigger problem. For some of them the hit rate is near zero, like kicker and some situations involving X. Others are actually success stories though, like equip. By putting it first thing in the text, right after the types, most modern networks are pretty good at coming up with working equipment cards.
So, some requirements are much harder to get right than others. This actually makes perfect sense. To paraphrase jml, if you require something to happen right at the start of a line, or right immediately after some sort of context switch, the network does much better. For obvious reasons. It knows that it needs to insert something, so if there are convenient points available to start inserting it, then it will. Of all the characters it could start with after the context switch, the ones that make it start meeting the requirement are the most likely. If there's a lot of other stuff in the way, however, then the network has a very hard time. Imagine you're a network generating a card, and you stick X in the cost. There are only so many lines of text in a card, and it's very hard to tell from the first character if that line will use the X from the casting cost or not. It's not that you, the network, don't know that you should use the X, it's just that when you start making a line it's very hard to guarantee that that line will be able to use X, and once you've gotten a little ways into the text of the line there are much more immediate restrictions to the text you can generate. You can't begin with "uncast target" and then just throw X on, you're probably going to say something like "spell\".
The goal, then, is to come up with a format that minimizes the bad requirements. We want things that impose constraints to come first, and things that would impose bad requirements to come early enough that those requirements are either converted into constraints or are less bad.
So, the important bits of the format are probably:
|type|subtypes|p/t|text||cost|
Type first, because it imposes constraints on everything. I think it makes sense to put subtypes and p/t before anything else, as they are so strongly influenced by type. Next is the text, and finally the cost. If we put it at the end, hopefully X cards will be easier, because the requirement is a lot simpler that way. All the network has to do is stick an X somewhere in the cost, as opposed to fabricating an entire line of text.
You could put text immediately after types, which might help issues with cards having inappropriate abilities, but text is long and arbitrary so it would probably make subtypes and p/t less consistent. It would be interesting to see if a properly trained network could remember things over a large distance like that.
Supertypes should go near type. You could put it first, but I'm more inclined to put supertypes after type and let type be the primary decider of what kind of card it is. Loyalty should probably go near p/t, you could reuse the field but I think it's less confusing for the network if you don't, and the extra delimiter isn't a huge price.
Rarity and name can go anywhere. Both are extremely nebulous constraints; that makes me want to tack them on the end. Or, you could put rarity first, otherwise the distribution of rarities will be skewed toward whatever the network is more likely to create in the absence of rarity information. For set generation, this probably doesn't matter, as we can overgenerate and filter to get the distribution we want anyway. I think it does make the most sense to put the name last, as then you can easily chop it off to make a more consistent network, as Talcos is doing now, and feed in unnamed cards to come up with names for them.
So a new plan for a full format:
|type|supertypes|subtypes|p/t|loyalty|text|cost|rarity|name|
Of course you can trivially chop name off for the 'stable' format. You can also make an argument for supporting a name-first, 'intuitive' format, that allows seeding names to come up with whole cards, which is a pretty cool thing to be able to do:
|name|type|supertypes|subtypes|p/t|loyalty|text|cost|rarity|
Then there's the issue of the text. I'm thinking that some improvements can be made inside the text field by cutting it up into lines and reordering them in a better way. Each line is exactly one ability - if you have something like "flying, first strike" I'd identify the separate keywords and chop it into "flying" and "first strike". We can try to order the lines in a way that is convenient: put equip costs first, and kicker costs last, so the requirement is to add a simple kicker cost line rather than a complicated one with "if @ was kicked".
There are other games to play as well. It might be possible to abstract out "where X is" definitions into a separate sentence, after a full stop (or period, colloquially), since that's a pretty strong context switch, or even as a separate line. This mostly works as is, but you could try to unify with other forms of X, possibly even costs. I think this has been suggested before.
Whew. Apologies for being long-winded or restating anything obvious. Just wanted to get it out while it was on my mind. The custom batcher is coming along, latest version on mtg-rnn can randomize mana costs and fields. The 'exactly one card at a time' mode should be finished soon. My ground truth experiment for that finished training after more than 110 hours of GPU time, so in another week or so I'll be able to give some results.
I like the mechanics of this card, even if the flavour feels a little off. I mean, it's land searching in red, but you don't get a body like you would if you played green. If it said "basic Mountain or Forest", I think it'd be perfectly fine as a real card. Also should be a shaman and not a wizard, but let's not get too into details here.
Draft Link
Just a casual Legacy/Vintage/No-Bans player who mostly plays EDH.
I have to disagree with you here. I won't claim that separating the different kinds of types makes a difference one way or another, as the only way to get a satisfactory answer would be to generate some data. But, I think the cost of adding structure is very small. Separating into 3 fields only costs two more delimiters, and it has vastly simplified what the network needs to generate. Correctly finishing the type field is extremely predictable, rather that dragging in fairly open-ended creature subtypes.
The thing is the network is good at delimited fields. Extremely good. It can generate 65000 cards with only one or two mistakes due to badly delimited fields, and those might have been the incomplete cards on the end, I don't remember. And it's also pretty good at realizing that creatures have subtypes while other types of cards usually don't, except arcane, and other little things like that.
We certainly don't want to make planeswalkers look like other cards. If they're too different, then the best approach is just to get rid of them and admit that we can't generate them at all. But this isn't what we observe: the network is quite capable of generating working planeswalkers, it just trips up sometimes because it's reluctant to make them long enough to have 3 abilities. It usually gets the loyalty costs right.
TL;DR - Structure is a good things. Networks are good at learning structure, but bad at dealing with complex relationships in pure text, like kicker. So the more we can do to add predictable structure to the text of a card, the better.
Have the neurons divided in two "pages" - Page A and Page B.
First, both pages A and B parses this text:
Destroy target enchantment. Put X --
Then after that, each half of the network gets a different text.
Page A gets
Destroy target enchantment. Put X 1/1 green Saproling creature tokens onto the battlefield.
Page B gets
Destroy target enchantment. Put X where X is that enchantment's converted mana cost.
Then, whenever X appears on a created rules text, each page gets to write their own follow-up, and they're later merged.
Precisely. It's a nice card, but it'll never see constructed play, and it's not a very good fit for limited applications unless the network also generates a convenient way to sacrifice enchantments in black.
Just so you know, the subtype is Kara, and the starting loyalty is 6.
Excellent ideas! I will warn you that moving the cost to where it's after the rarity may cause some distortions of color. For example, with the current network, we can get a card with stats like
2W
Creature - Human Soldier
Vigilance
2R : @ gets +2/+0 until end of turn.
and that same card could have a mana cost of 2R if you take the text into consideration when generating the cost. Not that one outcome is better than the other, just something to note. Meanwhile, I agree with you completely about supertypes and so on, some reordering would definitely be for the best.
---
Honestly, if I had the time and energy to reimplement our neural architecture from scratch, here's what I'd try.
Train a network to generate everything but the body text, which can be handled by a separate network.
Feed the body text network the text character-by-character as we have been doing, but continuously repeat key information about the card like color and types. So the input to the network at each time step would look like:
[one-hot encoding of character, context]
Where context is
[WUBRG color vector (e.g. <1,0,0,0,1> = GW), CMC, field encoding vectors (supertype/type/subtype), power,toughness,loyalty]
So at each time step, as the network predicts each character, its knowledge of the context of the body text is refreshed. This liberates the network from having to keep track of that information while simultaneously tracking information about the body text. I'm very confident that if you do this, you'll see fewer color issues and fewer misplaced text issues. I was inspired by that paper about a neural network that generated images of chairs that I showed y'all before. If you see the attached image, you'll note that the network receives the class input (the kind of chair that we want), but is also separately fed information about the angle and the lighting for that chair. It'd be sort of like that.
From there you can feed the resulting output to any number of filter networks for syntactic/semantic refinement.
Now, I think that a general-purpose, unspecialized network like what we have would work just fine if we had more inputs for training, but our corpus is limited in size and we struggle to teach the network some things like color discipline. I mean, we'll stick with what we've got for now, but in the long run I have the feeling that segmenting the network in this way will deliver the best results.
The subtype is probably my fault. What happened was that the network first decided that it wanted to make a Human Wizard, and then I forced it to make a green creature with evoke, and that's what we ended up with. If the network had planned on making it a green creature from the start, it probably would have been a druid or shaman.
That's not a bad idea, actually. It creates a marketplace of ideas. My concern, however, is that during training, if one of the pages makes a correct prediction and the other makes a wrong prediction, then backpropagation would reweight the wrong one over time to be more like the correct one. Unless you're careful with how you do your training, they'd end up in synchrony with each other. But I think that if you do the training right, you can avoid that sort of problem.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
Easy to fix so that it's interesting, though. Probably drop the damage off of it and have the "die" triggered on that combat phase. Also, make it a sorcery.
After this phase, there is an additional combat phase. Then, if a creature died during that combat phase, you may cast a copy of War~Crap Rebound again without paying it's mana cost.
Something like that. Also, cost is too low.
Then comes any necessary fine-tuning, balancing, playtesting, flavor/name/art stuff, etc. But we'll cross those bridges when we get to them.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
At work we implemented a paper that has a penalty on the correlation between two sets of units in order to prevent them from coming into synchrony, and it worked wonderfully. I'm trying to dig up the paper, and I'll edit it into this post when I find it.
EDIT: I found the paper! It's here. The gist of it is that they wanted to predict facial expression and ignore identity. So they use this trick to make some hidden units really good at reconstructing the image and then made another group of units have a minimal cross-covariance with the reconstruct units. The idea being that these units would then be forced to learn something different but useful.
The choice of images this thing uses is frequently as hilarious as the older cards at the beginning of the thread. I mean, look at this:
Exciting!
Feel free to try! But before I send you off on any wild goose chases I should warn you: everything I know about machine learning and neural networks came to me in the past four months or so. Any big, bold suggestions that I make are the products of my intuition and what I've learned from studying the relevant literature; I have little hands-on experience to confirm that what I'm suggesting will work. On the other hand, I'm a fast learner, and my intuitions rarely steer me in the wrong direction.
Oh really? I'd love to see it. I figured that such a thing was possible but I wasn't sure whether anyone had made a documented attempt at it.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
i had a few quick queations.
First up, is the link to the how to set up your own training network still current?
Are the linked sampling and sample files current?
If i were to sink a few days of time training up a new network is there any new things any of you would like me to try. (Basically donating a day or five of gpu time here)
Thank you in advance.
I'm considering a major overhaul to the format, so if you want to be a cutting-edge alpha tester you could wait until tonight, when hopefully I can push out a preliminary version. If you want to donate a lot of time, you could also help by testing out 'one card at a time' training mode, which is going to be massively slow.
These things are still being developed - I will post new changes as they happen.
I edited it into my earlier post. It's really a good read and we've had a lot of fun and success implementing it into our work.
/home/croxis/torch/install/bin/luajit: bad argument #1 to '?' (empty tensor at /home/croxis/torch/pkg/torch/generic/Tensor.c:851)
stack traceback:
[C]: at 0x7f6782dc2cc0
[C]: in function '__index'
sample_hs_v3.1.lua:202: in main chunk
[C]: in function 'dofile'
...oxis/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:131: in main chunk
[C]: at 0x00405be0