Hi Talcos! Hey, that's great if I can help, this here is such a cool project. To be honest I thought you in particular would be facepalming your headdesk the moment you saw "Context Free Grammar"
Haha, naw. The thing is that while Magic English is not context-free, many subsets of the language are in fact context-free. So I'd imagine that a context-free approximation could deliver "good enough" results, and that's all that matters.
That's awesome stuff... will it be as quick when you try the StackRNN implementation? I'm guessing a 1080p Starry Night Jace is also within reach too?
As for the StackRNN implementation, yes. I mean, it'll be slower (relatively speaking), but still very fast.
As for the Starry Night Jace, I'm testing that now. Right now, with the VGG-19 model and LBFGS optimization, it looks like I can generate images that were twice as large as before (at a rate of about 2 seconds per iteration), but I'm still running into some limits. I'll try also with the Inception (DeepDream) model with SGD (what I used to generate the original Starry Night Jace), which would amount to a smaller memory footprint and would allow me to make the image larger than that. I might also be able to tweak some other parameters to allow the images to be larger.
Honestly, the implementation could probably be a lot more efficient. There's no reason in principle why I shouldn't be able to make images of an arbitrary size when you consider the fact that transformations in one region of the image have no effect on transformations on another, distant region. I should be able to parcel out chunks of the image more effectively. I'm sure that once the techniques we're using make their way into professional software like Photoshop and sites like Instagram (and believe me they will), their implementations will be much better.
But yeah, I've attached a sample. The parameters will need some tweaking so that I can get the images looking right. Right now I'm using the neural-style package rather than the neuralart one, and since they use different models and optimizations, they can come out looking different. I'll try again in just a bit, maybe with the neuralart package, just to see what I can get.
EDIT: Yes, I can make much larger images with the DeepDream model, as it's more compact. Testing with the neuralart package now. I'll post another version for the sake of comparison.
EDIT: For those of you discovering this thread for the first time, you might want to check out this article here. There's also another article here which does an excellent job of summarizing the story so far (if you don't feel like reading through the whole thread). Also, we've been hard at work improving the NN model, and we have been making progress by leaps and bounds. What you see here is only the beginning! :-D
Also, I strongly recommend using hardcast_sixdrop's input format for training, if you're looking to replicate our more recent results. A lot of work has gone into making the RNN produce more consistent results.
EDIT: You can download the most recent version of the sampling script for hardcast_sixdrop's format here.
-----
Hello! I'm a PhD candidate researcher in computer science at the University of Alabama at Birmingham who has been investigating the use of deep neural networks for classification and problem solving tasks. I saw a fun article about training a neural network on arbitrary data to generate novel sequences. For example, you force the network to read Shakespeare over and over and eventually it can write its own texts in the style of Shakespeare. I saw that and thought: hey, why not Magic cards instead of Shakespeare? So I downloaded the source code (here) and a json corpus of Magic cards (here). I decided to feed a deep neural network all of the Magic cards ever made in the hopes that it might be able to conjure up some new ones. The network was relatively simple (I can give you the details, if you'd like, but that gets technical). I would have done a more robust and complex network but it would have taken far too long to do the computations; I'm waiting on some new GPU hardware to come in to speed up the process.
Anyway, here are a few example cards produced by my network about two hours into the training process. The results produced by the recurrent neural network (RNN) early on were either verbose garbage or sensible-sounding cards that did absolutely nonsensical things. Keep in mind that the RNN has no prior knowledge of what Magic even is, let alone English, so it's interesting that the results were even vaguely intelligible.
--------
Amarogge Warfos 2U
Artifact Creature - Kavu Shaman
Morph B(B/B)(G/W) (You may cast this card from your graveyard to the battlefield, you may pay 1. If an enchantment card, then put the rest of target creature card from your graveyard for its flashback cost. If exile is you sacrifice it unless you pay 1G. If you do, put a 3/1 green Soldier creature token onto the battlefield. Put it into your graveyard.)
1/1
#I'm tickled by the absurd reminder text. The RNN knows that keyword abilities often come with reminder text, but it has no idea what "morph" means, so it just makes up stuff to put there. Oh, and the morph cost has a hybrid black/black mana symbol in it.
Slidshocking Krow U
Creature - Dragon
Tromple,Mointainspalk
4/2
#Slidshocking Krow is ridiculously overpowered. A 4/2 for 1? In blue? With Mointainspalk AND Tromple? I see power creep is alive and well.
Grirerer Knishing 4G
Instant - Arcane
Exile target creature you control.
#The price is a little steep on this one, but maybe it's worth it for the synergy with other Arcane spells...
Fransunn's Ent 1B
Sorcery
Counter target spell with five toughness 2 or greater.
#Almost a meaningful conditional counterspell. Almost, but not quite.
Adiswen Agenter 1U
Enchantment - Aura
Enchant creature
At the beginning of each player's upkeep, sacrifice a white Zombie creature.
#Although very bizarre, it is a "legal" card.
Skengi Hellldadietsn 1BU
Creature - Zombie
: Add G to your mana pool. If you do, put a -1/-1 counter on Skengi Hellldadietsn.
3/4
#Notice that it picked a creature type that actually matched the colors.
--------
I decided to let the training process continue over night. When I came back, I found that the text was starting to make a little more sense (not always, but more so than before). I noticed that the network, now more fully trained, could generate meaningful, novel cards. However, it also had a knack for generating profoundly useless cards. Here are a few snippets from the output:
* When $THIS enters the battlefield, each creature you control loses trample until end of turn.
* Whenever another creature enters the battlefield, you may tap two untapped Mountains you control.
* 3, : Add 2 to your mana pool.
* Legendary creatures can't attack unless its controller pays 2 for each Zombie you control.
Other times it would start with an idea, like giving a creature kicker, but then forget about having a "if kicked" clause, or it could have a card with X in the mana cost but then deciding to do nothing with the X. Also, the network gets planeswalkers confused with level up creatures (admittedly they do look very similar), which often results in messy combinations of the two. For planeswalkers, the problem is that, unlike run-of-the-mill creatures, they are few and far between, so there aren't many examples for the network to learn from. In any case, here are some of the typical examples I found the network churning out this morning:
--------
Tenjer Desineer 1
Artifact - Equipment
Equipped creature has fuseback.
Equip 1
#The RNN likes to make up new keywords. This one is a portmanteau of flashback and fuse. What it does for a creature, who knows? The RNN certainly has no idea.
Gravimite 1(G/W)(G/W)
Creature - Dryad 1(G/W): Regenerate $THIS.
When Gravimite enters the battlefield, draw a card.
2/3
#I think this is a reinterpretation of Carven Caryatid.
Light of the Bild 2WW
Creature - Spirit
Flying
Whenever Light of the Bild blocks, you may put a 1/1 green Angel creature token with flying onto the battlefield.
2/2
Horror B
Horror deals 3 damage to target creature or player.
#A colorshifted Lightning Bolt. I find the name to be simple and evocative!
Mided Hied Parira's Scepter 2
Artifact 3, T: Put a 1/1 white Advisor creature token onto the battlefield.
Shring the Artist 2BB
Legendary Creature - Cat
Flying
Whenever you cast a spell, you may return target creature card from your graveyard to your hand.
2/2
--------
In conclusion, I've learned quite a bit from this process. Originally, I designed the network to avoid overfitting because I feared it would generate cards that were mere carbon copies of the ones it had seen. However, I made the network too conservative and as a result it's unwilling to experiment with multi-part abilities like kicker. It's also worth exploring whether I can improve training on scarcely seen cards like planeswalkers, planes, schemes, etc. With any luck, I should be able to come up with a generative model for Magic cards that produces more robust and complex output.
Let me know what you think!
EDIT: Oh, this is too good not to mention. I coordinated with Mtgsalvation member pickfifteen who has created a twitter account that posts automatically generated renders of cards created with the RNN network. I look forward to seeing what it produces. Grin
EDIT: Kinje has provided helpful instructions on how to replicate what I've done here. Also, PM me if you want the modified data set, one that's easier to train on. Alternatively, you can use Onderzeeboot's version of the data set, which should work just as well if not better.
EDIT: Kinje has uploaded my reformatted data set to Google Drive. You can download it here (instead of having to PM me every time, though you can if you'd like to get the script I used to produce the file).
That's the eternal question, isn't it? I think that the network at one point defined it as the Slith mechanic, and that one seemed to make sense. But if I ask the network again, it's likely to give me a different answer.
[quote from="LASture »" url="http://www.mtgsalvation.com/forums/creativity/custom-card-creation/612057-generating-magic-cards-using-deep-recurrent-neural?comment=2006"]I've shifted over from "filling out a set skeleton" to "building a Greatest Hits cube".
This is kind of what I'm doing too. I'll generate a 150 thousand character sample and then cherry pick some of the most interesting ones. Eventually I think this could be helpful for all of us, because we might end up mixing and matching RNN cards from other people's drafts/sets.
The mana-cost thing is something that's kinda bugging me, because I'm noticing that I don't really seem to generate any land-ramp spells at all. However, I do run into quite a bit of cards with landfall, so I'm thinking putting a little more of a bias on the set having slow mana and big creatures, but that's just my preferences.
Oh and @Talcos, you may have missed it but Tromple appeared again in my set as "Trompling Giant", haha.
Oh and @Talcos, you may have missed it but Tromple appeared again in my set as "Trompling Giant", haha.
That's amazing, rofl. I think that there's something about the word "tromple" that makes it "make sense" in terms of the distribution of letters in words. Sort of like how I occasionally get words like necromancy spelled as necromancie.
---
By the way, the neuralart package when using the deepdream/sgd combo does not take well to large images. It leaves them mostly untouched. I'll have to look into that.
I'm doing some stress testing with the stack-rnn. I have a feeling that this implementation could be way faster than it is. The paper describes being able to support up to 60 stacks or more, and I'm doing like 3 seconds per a batch of ten on just four stacks. Something's not quite right, lol. But I'll figure that one out. I'm fairly certain the implementer of this code has done something unnecessary or silly. I'm using some of the most expensive hardware on the market, so if there's a slowdown, it's their fault, not mine, lol.
Anyway, I think the reason why your networks have problems with mana costs is that, well, there's nothing there to learn- there's not one pattern you can learn. Card and ability costs are not drawn from a single, neat distribution. That's my intuition anyway.
Like you've said, There are definitely outliers, everything is always changing, and there are lots of factors that go into the design that aren't immediately apparent or visible from reading the cards. But, just as Magic is not a context-free language but has context-free subsets, many pieces of the game can be understood and predicted.
For example, earlier in the thread I showed that you could reasonably predict features like converted mana cost based on the colors, keyworded abilities, and power/toughness stats of a card using fairly simple models like decision trees.
Meanwhile, distributions for things like power according to color are pretty consistent as well. Note on this graph...
that white is the leader of 2 power creatures, red loves 5 power dragons, blue leads in 0 and 1 power creatures, green is the leader for creatures with power >= 6, and black is always in the middle of the pack when it comes to creature power.
We also can see a lot of regularities when it comes to the color pie and keyworded abilities. Blue doesn't get haste (save for the one outlier, Bonded Fetch), white doesn't get deathtouch (unless you count the outlier Mardu Hateblade), etc. Meanwhile, elves are usually green, red likes damage spells, white likes damage prevention, etc.
Now, certain features are harder to deal with than others. A wordy activated ability, for instance, requires deeper comprehension, and we can be rather hit or miss with that. At the same time, when it comes to matching color and body text, a lot of the problems we're dealing with now could be ameliorated if we used better algorithms, like having permanent storage on the side, and that's something I'm wanting to investigate.
So I think maybe the best way to cost cards for a cube, like folks here want to do is to ignore the costs the networks produce and either just cost them by hand _or_ come up with a set of rules and write some code that will do just that: cost cards generated by the networks.
(... and, well, if you want a rule-based expert system you know what language you gonna call :D)
At this point your networks have done 90% of the job a human could do. I think there might be lots of fun to be had in finding a way to complete that result with auxiliary programs, like you think my algorithm could help with ungrammaticality, say.
I agree. I think that an ensemble approach would help us improve upon our current results.
Then again, doing it by hand might be even more fun still. It would certainly be a nice way to have something that people can contribute other than code. I'm thinking here of the way projects like M:tG Forge have engaged their community, in the case of Forge by having users script cards to add to the game engine. Though that sort of thing carries ... certain risks, I don't think it should be an issue in this case.
I'm all for engaging the community, but you must understand that I am pursuing a certain "blue-sky" vision with this project.
I want to automate the task of design. I want a human designer to be able to come up with the seed or kernel of an idea, and have an automated system cultivate and nurture that idea, to play with it and expand upon it. The human would the direct the action, with the machine helping to fill in the details to provide a richer, deeper experience. Think of the machine as a creative companion rather than a substitute. It would perform a very similar role to your idea of crowd-sourced wisdom.
EDIT: For the record, I'll be looking into the slowness of the stackRNN implementation. I think I can make things work the way I want them to.
Unrelated, but I found a card that actually used Y in addition to X, but sadly it only explained what X would be. It's also using it in an illegal manner (a non-creature giving itself +x/+y). And also a card where it just lost track of what it was doing.
Snip the +X/+Y sentence and that's quite an interesting card! I love that the Quest card picked up "from a height" via, presumably, Chaos Orb.
I don't have the resources to get my own network up and running, but if I can make a request - can someone run this cosplay image through the art generator mimicking the reference art? I can't find any solid fan art for Tamiyo, and am hoping to make a card for her. Of course, if it doesn't end up making a good art I understand, and would still be curious to see the outcome.
I don't have the resources to get my own network up and running, but if I can make a request - can someone run this cosplay image through the art generator mimicking the reference art? I can't find any solid fan art for Tamiyo, and am hoping to make a card for her. Of course, if it doesn't end up making a good art I understand, and would still be curious to see the outcome.
I'd be happy to give it a try.
EDIT: Running the generator now.
Sorry for not making much progress with the stackRNN as of yet, by the way. Once I have a good fix for the problems I'm facing, it won't take very long to run. But I'm currently deeply engaged with my dissertation work on software bug detection/prevention, and that eats up most of my waking hours.
I like to think of myself as a time-travelling detective. The deep neural network is an oracle who can predict that a crime is going to happen and where it will happen, but it can't tell me when or why. I have the power to move forwards and backwards through time, as well as across branching timelines, in order to piece together the evidence I need. However, there are two caveats:
* Every time I make a timeshift, I use up a certain amount of energy (well, space/time, but you get the idea), and I only have a finite budget that I can spend. I have to make the best use of the available evidence when making my decisions.
* When I've established the who/what/when/where/why/how, I have to prevent the crime from happening. But I can't do anything that will negatively interfere with future timelines. So pulling a little girl out of the way of a hit-and-run is too risky. The better course of action is to go back 20 years and petition the city council to have a crosswalk built on that street. I've done my job right when it looks like I didn't do anything at all.
The work I'm doing is exciting and I feel that it's vitally important, but it's also incredibly exhausting.
I can't formally describe why the oracle works, she is inscrutable. Sometimes she gives me leads that go cold, but it may be the case that she was right all along and I just didn't have the energy to search long enough. Other times she leads me straight to the crime. Lacking the theoretical framework to understand why she works, instead I'm having to establish some empirical bounds on the quality of her predictions.
The time-travelling part I can explain. It's all just formal logic and concrete facts. But without the oracle, I'd never know which way to go, there are too many directions, too many forking timelines. I once calculated that for a modest-sized problem, if a single point in the continuum occupied a physical region of one square meter, all the timelines together would equal 8 billion galaxies in size. I quickly reach a point where there aren't enough computational resources on Earth to search that space in a reasonable span of time.
Been playing a bit more and I've become interested in playing with neural nets in more areas than just magic. What would be a good place to start? I come from a physics background, where the most advanced "programming" we've done is matlab.
And is a more "longterm memory" a matter of merely training the network long enough/with big enough network size, or is this more inherent to the actual coding of the RNN? This would help with things like kicker, exploit, entwine and the like I imagine.
In principle, an LSTM cell can hold onto a piece of information indefinitely. It can shut out new inputs and prevent them from contaminating the data it's currently holding. In practice that's not quite how things work. Usually the cells will hold onto a piece of information as long as it is expedient to do so, but when new, tantalizing information comes along, they're quite content to flush the old in exchange for the new.
Overall what happens is that the memories decay over time, and this complicates the process of learning associations between text elements that are spaced far apart.
That's why I'm looking into allowing the network to make use of permanent data storage. In effect, it could write down things that it has to keep track of, and then it only has to retain the memory of where it wrote the information down rather than the information itself. That, from what I've read, would greatly improve upon the network's ability to overcome long-term memory limitations.
----
EDIT(3): So the first run was with the default weights and I got mixed results, depending on how you look at it. On one hand, we're asking the network to extract style from the card art rather than the content, and it does a decent job of that. The cosplay image takes on a more painterly style, the network delivered what it promised, with only a minimal amount of content bleed (e.g. the floor looks more blue and wavy than it did before, and there are some minor embellishments on the dress, like the pattern in the middle).
But really we would like for the network to be more liberal with what it transfers, so I readjusted the weights accordingly and I am re-running everything.
... and I over-shot it. There has to be a happy medium... Running it once more.
Not quite there yet either. Still some inappropriate content bleed. But if you merge versions of the image (as I've done), you get something that's halfway decent. I think the shadows are causing some problems, because it's interpreting those as if they were content boundaries. I think you'd probably get better results if the lighting were more even.
Oh, and I think that if you were to prime it with the original image, you might get more consistent results. I didn't think to do that, I used random priming, which contributes to the noise. So some combination of that and lighting normalization should get you a better image, and if I have time later this evening I can look into that for you.
Oh, while I was on my lunch break, I set the network running, and I mashed the Tamiyo cosplay up with some ancient Greek pottery art (see attached), and got some fun results with that. A light application of that art, merged with the card art, makes for an awesome alternate costume. If I turn things up too high, it eats the image, of course, but we knew that already.
In other words, I can more or less retexture her dress to be anything I want, lol.
So, on this- I met with my supervisor today and he clarified things for me (to be fair I have no excuse for not knowing the answer myself, but I just wanted to be sure I got it right). The grammar my algorithm learns is just a lexicalised CFG. That said, DCGs can represent the full power of a Turing machine, so there's no reason why I couldn't eventually stretch THELEMA out to make it learn a higher-order grammar like a context-sensitive grammar or a Turing machine... except of course, there's probably no reason to do that, since like you say, most of AT can be modelled as a CFG just fine.
This is all to say I'm still thinking about modelling the M:tG rules engine itself. I discussed before how I think that, theoretically, the rule engine, or the rule-machine shall we say, should be a superset of the AT language.
Of course, if I'm right about this then, given that this thread has already demonstrated that a network can learn the AT language, it should be possible to train a network to learn the rules of the game itself just from the cards
I'd say that the solution space is quite large. I'm certain that you could teach the network to play a Magic, but it will be challenging to teach it to play our Magic.
Sort of like how no one plays Monopoly correctly (i.e. "Wait, you mean the games aren't meant to last 10 hours?"). That is, unless you plan on incorporating some kind of reinforcement learning on top of whatever it picks up on its own.
Still, there's a lot of fun to be had in that direction.
With that you could do partial artworks in other artwork. Say You have Picture A with a vase in the background that shows Picture B. This way you could integrate some older stuff say old planeswalkers as "Mythology Gag". If you could partition the content of A so you can tell the network to run anorther instance of itself with new inputs for area x it could be possible.
This also could be useful to make better artwork. A real artist goes by various source materials, say clothing guides real objects etc. for one type of content while another draws say panoramic Photos for the background. Thus partition and priming a network with content matching examples might get better results.
That being said the Cosplay is Awesome but needs a much better background then plain grey walls. Maybe a japanese Temple?
Fun ideas! I agree. Partitioning is definitely possible.
And I could try updating the cosplay image later. Right now my machines are tied up diagnosing buggy code.
I want to automate the task of design. I want a human designer to be able to come up with the seed or kernel of an idea, and have an automated system cultivate and nurture that idea, to play with it and expand upon it. The human would the direct the action, with the machine helping to fill in the details to provide a richer, deeper experience. Think of the machine as a creative companion rather than a substitute. It would perform a very similar role to your idea of crowd-sourced wisdom.
That sounds... really nice, actually
Have you posted more about that in this thread, or elsewhere? I'd like to hear more about it- I'll be honest and say that I imagined my auto-generated parser as a component in a sort of open-source version of M:tG, where players could make their own cards, check their syntax locally, then validate their effects against some central server and finally use those validated cards in their decks. That was one use-case anyway.
Of course it never occurred to me that the design could be done in the way you say. With ANN-assisted design, I can imagine a whole marketplace with entire sets for every player, casual, competitive or whatever produced at a rate even teams of human designers couldn't hope to match. The central server could even house the networks, then the players could do what you say- seed an instance with an idea for a set then let the network churn out cards.
Well anyway- there's still some way to go before that sort of thing can be possible
Haha, sure. My primary research interest as a PhD candidate is in software reliability, that is, making sure that things don't break, fail, or try to kill us. But that's not my only area of research. During my undergraduate studies, I did research in computational aesthetics. That is, I sought to determine whether machines could arrive at an understanding of things like beauty and creativity. I feel that I'm much closer to an answer now than ever before, so this is all very exciting stuff for me.
But what is the end goal? What is our trajectory? I'm not looking to replace the human artist, because I believe that creative expression is an intrinsic part of the human experience. No, instead, I see opportunities for machines to act as co-creators.
Thanks to a combination of sheer computational power, limitless data (thanks to the Internet), and smart, adaptive algorithms, we now have access to an inexhaustible supply of creative potential. The human designer can set the stage and pick the direction, and the machine can come in help fill in the gaps and assist the creative process.
Like you say "There is a street, with many vendors..." and the machine comes in with "there's a hot-dog vendor... his name is Ollie (creates signage for Ollie's hot dog stand)... he lives alone... his wife died... of hot-dog poisoning! The memory haunts him." The machine just builds upon itself, a self-referential cycle producing layer upon layer of precious coral. The more you zoom in or zoom out, the more it creates - effortlessly, like in a dream. It's the same kind of extrapolation that makes our Magic cards or produces our pictures, just taken to its logical conclusion. The same process could be applied to music, to literature, to video games.. anything.
I see tons of avenues where man and machine can team up to create beautiful works, and I'm excited to see where things lead in the next decade or two.
---
EDIT: I spoke with the creator of the data-structure-augmented-network, and he told me that he plans on making updates and fixes to his code next week (hopefully), so I'll be keeping an eye out for that.
EDIT(2): As I promised some time ago, I'll do that H.R. Giger version of Damia when I get the chance. That's on the list.
Sorry for not making much progress with the stackRNN as of yet, by the way. Once I have a good fix for the problems I'm facing, it won't take very long to run. But I'm currently deeply engaged with my dissertation work on software bug detection/prevention, and that eats up most of my waking hours.
I like to think of myself as a time-travelling detective. The deep neural network is an oracle who can predict that a crime is going to happen and where it will happen, but it can't tell me when or why. I have the power to move forwards and backwards through time, as well as across branching timelines, in order to piece together the evidence I need. However, there are two caveats:
* Every time I make a timeshift, I use up a certain amount of energy (well, space/time, but you get the idea), and I only have a finite budget that I can spend. I have to make the best use of the available evidence when making my decisions.
* When I've established the who/what/when/where/why/how, I have to prevent the crime from happening. But I can't do anything that will negatively interfere with future timelines. So pulling a little girl out of the way of a hit-and-run is too risky. The better course of action is to go back 20 years and petition the city council to have a crosswalk built on that street. I've done my job right when it looks like I didn't do anything at all.
The work I'm doing is exciting and I feel that it's vitally important, but it's also incredibly exhausting.
I can't formally describe why the oracle works, she is inscrutable. Sometimes she gives me leads that go cold, but it may be the case that she was right all along and I just didn't have the energy to search long enough. Other times she leads me straight to the crime. Lacking the theoretical framework to understand why she works, instead I'm having to establish some empirical bounds on the quality of her predictions.
The time-travelling part I can explain. It's all just formal logic and concrete facts. But without the oracle, I'd never know which way to go, there are too many directions, too many forking timelines. I once calculated that for a modest-sized problem, if a single point in the continuum occupied a physical region of one square meter, all the timelines together would equal 8 billion galaxies in size. I quickly reach a point where there aren't enough computational resources on Earth to search that space in a reasonable span of time.
So yeah, that keeps me busy. lol.
Do you watch the show Person of Interest? Because much of your post is reminiscent of it.
Edit: and later you mentioned 'hotdog vendor' which is a term that has special meaning to me. In another part of the internet, it means 'game developer'. I'm not kidding.
Sorry for not making much progress with the stackRNN as of yet, by the way. Once I have a good fix for the problems I'm facing, it won't take very long to run. But I'm currently deeply engaged with my dissertation work on software bug detection/prevention, and that eats up most of my waking hours.
I like to think of myself as a time-travelling detective. The deep neural network is an oracle who can predict that a crime is going to happen and where it will happen, but it can't tell me when or why. I have the power to move forwards and backwards through time, as well as across branching timelines, in order to piece together the evidence I need. However, there are two caveats:
* Every time I make a timeshift, I use up a certain amount of energy (well, space/time, but you get the idea), and I only have a finite budget that I can spend. I have to make the best use of the available evidence when making my decisions.
* When I've established the who/what/when/where/why/how, I have to prevent the crime from happening. But I can't do anything that will negatively interfere with future timelines. So pulling a little girl out of the way of a hit-and-run is too risky. The better course of action is to go back 20 years and petition the city council to have a crosswalk built on that street. I've done my job right when it looks like I didn't do anything at all.
The work I'm doing is exciting and I feel that it's vitally important, but it's also incredibly exhausting.
I can't formally describe why the oracle works, she is inscrutable. Sometimes she gives me leads that go cold, but it may be the case that she was right all along and I just didn't have the energy to search long enough. Other times she leads me straight to the crime. Lacking the theoretical framework to understand why she works, instead I'm having to establish some empirical bounds on the quality of her predictions.
The time-travelling part I can explain. It's all just formal logic and concrete facts. But without the oracle, I'd never know which way to go, there are too many directions, too many forking timelines. I once calculated that for a modest-sized problem, if a single point in the continuum occupied a physical region of one square meter, all the timelines together would equal 8 billion galaxies in size. I quickly reach a point where there aren't enough computational resources on Earth to search that space in a reasonable span of time.
So yeah, that keeps me busy. lol.
Do you watch the show Person of Interest? Because much of your post is reminiscent of it.
Edit: and later you mentioned 'hotdog vendor' which is a term that has special meaning to me. In another part of the internet, it means 'game developer'. I'm not kidding.
Oh? I've never seen that show. I may have seen an ad for it a long time ago though as I recognize the name. But the trope of an all-knowing machine goes way back.
And I've never heard that term before either. That's funny, lol. :-D
Oh? I've never seen that show. I may have seen an ad for it a long time ago though as I recognize the name. But the trope of an all-knowing machine goes way back.
And I've never heard that term before either. That's funny, lol. :-D
The basic idea is that a computer (hidden somewhere) sends out (to the protagonists) the SSN of someone who it predicts (using all the surveillance feeds) will soon be the victim or perpetrator of a violent crime. But it's really a superhero (though not the comicbook sort) story with a small scifi element (i.e The Machine). And it's very well put together.
And that term is extremely niche. It all started with mustard, and Cookie Clicker, and Time...
Anyhow, it's always great to watch the progress. I really enjoyed the colour-sorted dump on the previous page. So many clever things going on, but I don't know how original they are, as I don't know the Magic corpus well enough to know what's ripped straight from real cards.
The basic idea is that a computer (hidden somewhere) sends out (to the protagonists) the SSN of someone who it predicts (using all the surveillance feeds) will soon be the victim or perpetrator of a violent crime. But it's really a superhero (though not the comicbook sort) story with a small scifi element (i.e The Machine). And it's very well put together.
Oh? That does sound interesting. I'll have to check that out when I can find time to watch television.
Anyhow, it's always great to watch the progress. I really enjoyed the colour-sorted dump on the previous page. So many clever things going on, but I don't know how original they are, as I don't know the Magic corpus well enough to know what's ripped straight from real cards.
Good question. There are many different ways for us to "prove" the originality of the cards produced. At the very least, you can use the vector trick I showed y'all to cheaply compare the semantics of the card with other, real cards.
I'd watch that show, Talcos.
Been following this thread for a long time (first time post),
and I love how this is actually almost a college course "Neural Networks 101: A Fun Introduction".
Keep up the good work! (And do tell us if you find other interesting papers)
Haha, I'm glad you've found all of this so entertaining.
And it's funny that you should describe it that way. Teaching is my passion, but lately I have not been teaching any courses because I've been focused on writing my dissertation. Aside from working with a lone undergraduate apprentice, and when I occasionally get called in to guest lecture, I'm locked up in my office without much contact with students. I suppose some of that pent-up energy has gotten redirected here. The difference is that normally the administration would charge y'all nearly a thousand dollars per person just to hear me pontificate, but here it's free, haha.
And I'll be sure to let you know if I read any other interesting papers. I go through about one or two papers a day, but a lot of them are just boring technical reports, that sort of thing.
---
By the way, here are a few interesting cards (just for fun)...
Turn the Endless 5BB
Sorcery (Rare)
As an additional cost to cast Turn the Endless, reveal an Elemental card from your hand.
Target player can't cast creature spells.
#I assume this effect lasts for the rest of the game.
Mana Breath 4G
Instant (Rare)
If a player would draw a card this turn, that player returns a creature card from his or her graveyard to the battlefield instead.
#Now that's interesting. Unfortunately, it doesn't draw any cards for you on its own, so you have to have some other card draw available, but the effect is interesting. Might be more more black than green though.
Skirmishisa 2G
Creature - Lizard (Uncommon)
Trample, haste
Cumulative Upkeep - Put a -1/-1 counter on Skirmishisa.
4/3
#I like that it's a lizard. That creature type does not get enough love.
And here are a few unusual ones...
Druid of Divility 6
Artifact Creature - Shapeshifter (Rare)
You may cast Druid of Divility from your graveyard if you control an Urza's River.
4/5
#The Urzatron trinity has gained a new combo piece, I guess. But what is that card? Survey says...
|urza's river||land|||N|||@ enters the battlefield tapped.\T: add {^} to your mana pool. when @ enters the battlefield, you may return target artifact card from your graveyard to your hand.|
|urza's river||land|||N|||@ enters the battlefield tapped.\T: add {^} to your mana pool.\{^} T: put a &/&^ blue barbarian creature token with haste onto the battlefield.|
|urza's river||land|||A|||T: add {^} to your mana pool.\T, pay &^ life, sacrifice @: search your library for a card named @, reveal it, and put it into your hand. then shuffle your library.|
Master of Familiar W
Creature - Human Soldier (Common)
Whenever Master of Familiar attacks, detrive three target cards from a single graveyard.
1/1
#Wait. Hold on. "detrive"? Does it mean "retrieve"? That's almost correct usage of a word... that's not used in Magic English, so no. I guess it's a mystery mechanic.
And here are a few not so good ones...
Skhhunter Dragon 3GRRR
Creature - Dragon (Timeshifted)
Flying
When Skhhunter Dragon dies, put a +1/+1 counter on target attacking creature if it's white.
5/6
#Now this is an example of a card that you don't often get shown. Sometimes the network comes up with perfectly legal cards that are so incredibly niche in terms of their uses that I don't bother mentioning them. Skhhunter Dragon's ability only matters if it dies during combat, and only if you play Naya colors, and only if you're attacking with a white creature. Oh, and if you have the misfortune of playing against a w/X aggro deck, they can kill this guy to pump their creature (again, only during combat).
Cover the Ground B
Instant (Rare)
You may shuffle your library.
#This has to have been inspired by One with Nothing. The name is funny though, it suggests "kick some dirt around".
---
EDIT: As promised, I've attached a version of Damia, Sage of Stone by H.R. Giger. I ran the network with the default settings. Right now it's mostly just a reskin, but I can turn up the Giger-ness and turn down the Argyle-ness if needed, and that would probably have the effect of fusing Damia with her chair in keeping with Giger-esque style.
Right now it's mostly just a reskin, but I can turn up the Giger-ness and turn down the Argyle-ness if needed, and that would probably have the effect of fusing Damia with her chair in keeping with Giger-esque style.
Thanks for the higher resolution version. A slightly Gigeresquer version would be nice, indeed, if it isn't a lot of work/trouble.
Right now it's mostly just a reskin, but I can turn up the Giger-ness and turn down the Argyle-ness if needed, and that would probably have the effect of fusing Damia with her chair in keeping with Giger-esque style.
Thanks for the higher resolution version. A slightly Gigeresquer version would be nice, indeed, if it isn't a lot of work/trouble.
Sure! I had time to make one more, though I'll have to put future requests on hold for now as I'm dedicating the machines to a series of experiments to be run over the weekend.
I turned up the Giger and turned down the Argyle, and got this result. She's more machine than she is Gorgon at this point, though I like that the transformation preserved her face. That's probably because the object recognition part of the network said "ooh, human face!" and fixated on it; it flattened her nose and mouth, but kept the glowing eyes. Her hair and arms we're not quite so lucky, they got merged into her throne. Meanwhile, the background details now have more gear/machine like features than they did before.
Is this to your liking?
EDIT: Oh, another breakthrough! Evidently MIT researchers have designed a deep neural network system that can learn to play text-based MUD-style games with no prior knowledge of language or of the game. Here's the paper.
The AI apparently develops a mental map of the game content and learns quest-solving strategies like "use sword on [threatening monster]". Obviously, it's not foolproof, but that it can actually learn to play the games is quite impressive. And the architecture they use is quite similar to the ones we're using right now.
With this and the suggestions that YeGoblynQueenne made yesterday, that gives me all kinds of ideas.
"Play cards.. turn them sideways. Play cards.. turn them sideways." lol. Maybe something a little more sophisticated than that.
It's funny. There are statements that I made on this very thread a few months ago about the limitations of training such a system to do long-term goal planning, and those statements already sound dated now.
EDIT(2): I attached a graph to give you some idea of what we're looking at. The red curve is the authors' approach, and the others are the best alternatives to their approach. The alternatives, like bag-of-words approaches (think like our card vectors, that's how they're estimating the semantic content of the words that appear on the screen), are mediocre when it comes to questing. The authors' approach, meanwhile, is uncannily good.
In order for the LSTM network to do so well, it has to have some kind of understanding of the consequences of its actions. That's what makes this interesting.
Any chance of getting another dump file... Or ten?
I don't care that the cards are malformed, or that some of them are un-interesting. I don't have the motivation to install the program and run the neural net / card generator myself, but I'm very interested in sifting through the output for items of interest! I feel like there may be others in the same boat.
Edit: Tried Croxis' web service, but I'm getting 504 Gateway Timeout every time. Any idea?
Wild nactl in the style of Kenket would be interresting to see since the artistic style is pretty close while we here often used stark contrasts in Style.
I guess it would help to have picture with similiar pose and content?
Not necessarily, but you're correct that there's an advantage. When you select two pictures that have the same level of detail (the scales match up), then the resulting image can look more coherent in terms of texture (as far as I I've seen).
Any chance of getting another dump file... Or ten?
I don't care that the cards are malformed, or that some of them are un-interesting. I don't have the motivation to install the program and run the neural net / card generator myself, but I'm very interested in sifting through the output for items of interest! I feel like there may be others in the same boat.
Sure, I can look into that.
Do you just want the raw text or what?
EDIT: I set the older machine running to generate a dump of cards. I'll post it later for you.
Edit: Tried Croxis' web service, but I'm getting 504 Gateway Timeout every time. Any idea?
It's been down, but Croxis has been very busy teaching his classes and other duties like that, so I don't think he's had the chance to bring it back online.
I'm on my windows machine where the dumps are all in MSE format, which is slightly less neat. But I guess I can satisfy your curiosity with some exported image dumps on imgur. Three checkpoints and a bunch of handpicked cards.
What we need now is to join our image generation process with something like Croxis' web service, haha. The problem is that the image rendering is computationally expensive, so there would be long turn-around times for card creation.
But honestly, there may be a way to make things cheaper if we had a limited artist selection. Like we have a specialized set up just for certain artists. But that only amortizes some of the cost of the style extraction.
There's a decent amount of overhead in the repeated use of the convnet, where we break down the image according to its content. See the attached image for an example of such a network tied to a caption-generating RNN (image taken from here, if you're interested). In short, we have to know what objects are in the image so that we can distinguish what constitutes the style of the image and the content of the image. The woman in the picture is the content, and the texture of her clothes, that we can treat as style. We can change the image as much as we'd like, style-wise, so long as we can still see the woman, racket, people in the background, etc. That restriction allows us do the style transfer without demolishing the original image.
Of course, there may be ways to optimize the computations involved, or even to skimp on some of them and still get good results, but that would require some in-depth analysis, lol.
---
Last night I went to a pre-release. Desolation Twin for the win.
For as much as I've talked about Magic, that was the first time I've gotten to play the game in ages, haha.
I can generate more for you on demand. I'll note that I kept the temperature fairly high so that I could give you a diverse range of results (as opposed to ho-hum limited filler quality cards). For the sake of comparison, here's a low temperature sample:
|spire shade||creature||shade|O|&^/&^|{BB^}|{BB}: @ gets +&^/+& until end of turn.|
|soul shaman||creature||human shaman|O|&^^/&^^|{^^WW}|when @ enters the battlefield, you may destroy target artifact or enchantment.|
|death's presence||enchantment|||A||{^^^^BB}|whenever a player casts a spell, that player discards a card.|
|burning star||instant|||O||{RR}|@ deals &^ damage to target creature or player.|
Perfectly legal, acceptable, and unadventurous. Even the names are boring and trite! At the same time, raising the temperature also means there's more room for mistakes to creep in. Many of the cards are just fine, but some leave you with more questions than answers. I've identified three rough categories of mistakes and oddities that you're likely to see, as exemplified by these cards:
1. Cards that are legal but onerous in their specificity.
Master of Dawn G
Creature - Human (Rare) 2G: Regenerate target creature. Activate this ability only during your upkeep.
1/1
#Hurray for debilitating restrictions! This looks like an attempt to make an ability on an underpriced creature "fair".
2. Cards that contain garbage.
Medicine Sphinx 6UUU
Creature - Sphinx (Rare)
Flying Linstipt2B
7/7
#I'm sure the high price of this creature is justified by whatever it is that "linstipt" does.
3. Cards that have disjoint rules text or that ask you to do impossible things.
Anarchize 1BB
Enchantment - Aura (Rare)
Enchant player
Whenever a creature deals combat damage to a player, tap target creature or player. otherwise, put one of them into your hand.
#Note: The ability triggers whenever any creature deals combat damage to any player, not just the enchanted player.
#Note: Players are considered tapped when laying on their side.
#Note: If you cannot reach across the table to hold another player in your hand, you may instead use a marker (such as a sticker) to indicate that a player is currently in your hand.
And then there are some other issues like misuse of abilities like kicker and X costs, as we've discussed before. On the bright side, you periodically get absolute gems.
EDIT: By the way, I like the user name. I have two ferrets myself.
EDIT(2): rofl at the name "Darwen, Martyr of Knowledge".
I'm on my windows machine where the dumps are all in MSE format, which is slightly less neat. But I guess I can satisfy your curiosity with some exported image dumps on imgur. Three checkpoints and a bunch of handpicked cards.
Any chance of getting another dump file... Or ten?
There are a few new 15MB MSE2 set files in this folder that I just recovered a few days ago after regaining access to one of my machines. They were generated with randomly ordered fields, so a lot of the cards will be a little messed up, but there's probably some interesting stuff in there as well.
Haha, naw. The thing is that while Magic English is not context-free, many subsets of the language are in fact context-free. So I'd imagine that a context-free approximation could deliver "good enough" results, and that's all that matters.
I wouldn't expect that, to be honest. My networks are composed of tons of wibbly-wobbly, time-wimey non-linearities.
I like the name!
As for the StackRNN implementation, yes. I mean, it'll be slower (relatively speaking), but still very fast.
As for the Starry Night Jace, I'm testing that now. Right now, with the VGG-19 model and LBFGS optimization, it looks like I can generate images that were twice as large as before (at a rate of about 2 seconds per iteration), but I'm still running into some limits. I'll try also with the Inception (DeepDream) model with SGD (what I used to generate the original Starry Night Jace), which would amount to a smaller memory footprint and would allow me to make the image larger than that. I might also be able to tweak some other parameters to allow the images to be larger.
Honestly, the implementation could probably be a lot more efficient. There's no reason in principle why I shouldn't be able to make images of an arbitrary size when you consider the fact that transformations in one region of the image have no effect on transformations on another, distant region. I should be able to parcel out chunks of the image more effectively. I'm sure that once the techniques we're using make their way into professional software like Photoshop and sites like Instagram (and believe me they will), their implementations will be much better.
But yeah, I've attached a sample. The parameters will need some tweaking so that I can get the images looking right. Right now I'm using the neural-style package rather than the neuralart one, and since they use different models and optimizations, they can come out looking different. I'll try again in just a bit, maybe with the neuralart package, just to see what I can get.
EDIT: Yes, I can make much larger images with the DeepDream model, as it's more compact. Testing with the neuralart package now. I'll post another version for the sake of comparison.
I'd imagine it's a bit of both.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
Whats tromple?
That's the eternal question, isn't it? I think that the network at one point defined it as the Slith mechanic, and that one seemed to make sense. But if I ask the network again, it's likely to give me a different answer.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
This is kind of what I'm doing too. I'll generate a 150 thousand character sample and then cherry pick some of the most interesting ones. Eventually I think this could be helpful for all of us, because we might end up mixing and matching RNN cards from other people's drafts/sets.
The mana-cost thing is something that's kinda bugging me, because I'm noticing that I don't really seem to generate any land-ramp spells at all. However, I do run into quite a bit of cards with landfall, so I'm thinking putting a little more of a bias on the set having slow mana and big creatures, but that's just my preferences.
Oh and @Talcos, you may have missed it but Tromple appeared again in my set as "Trompling Giant", haha.
That's amazing, rofl. I think that there's something about the word "tromple" that makes it "make sense" in terms of the distribution of letters in words. Sort of like how I occasionally get words like necromancy spelled as necromancie.
---
By the way, the neuralart package when using the deepdream/sgd combo does not take well to large images. It leaves them mostly untouched. I'll have to look into that.
I'm doing some stress testing with the stack-rnn. I have a feeling that this implementation could be way faster than it is. The paper describes being able to support up to 60 stacks or more, and I'm doing like 3 seconds per a batch of ten on just four stacks. Something's not quite right, lol. But I'll figure that one out. I'm fairly certain the implementer of this code has done something unnecessary or silly. I'm using some of the most expensive hardware on the market, so if there's a slowdown, it's their fault, not mine, lol.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
Like you've said, There are definitely outliers, everything is always changing, and there are lots of factors that go into the design that aren't immediately apparent or visible from reading the cards. But, just as Magic is not a context-free language but has context-free subsets, many pieces of the game can be understood and predicted.
For example, earlier in the thread I showed that you could reasonably predict features like converted mana cost based on the colors, keyworded abilities, and power/toughness stats of a card using fairly simple models like decision trees.
Meanwhile, distributions for things like power according to color are pretty consistent as well. Note on this graph...
https://plot.ly/~rmmilewi/800/dat-distribution-of-power-real-magic-cards/
that white is the leader of 2 power creatures, red loves 5 power dragons, blue leads in 0 and 1 power creatures, green is the leader for creatures with power >= 6, and black is always in the middle of the pack when it comes to creature power.
We also can see a lot of regularities when it comes to the color pie and keyworded abilities. Blue doesn't get haste (save for the one outlier, Bonded Fetch), white doesn't get deathtouch (unless you count the outlier Mardu Hateblade), etc. Meanwhile, elves are usually green, red likes damage spells, white likes damage prevention, etc.
Now, certain features are harder to deal with than others. A wordy activated ability, for instance, requires deeper comprehension, and we can be rather hit or miss with that. At the same time, when it comes to matching color and body text, a lot of the problems we're dealing with now could be ameliorated if we used better algorithms, like having permanent storage on the side, and that's something I'm wanting to investigate.
I agree. I think that an ensemble approach would help us improve upon our current results.
I'm all for engaging the community, but you must understand that I am pursuing a certain "blue-sky" vision with this project.
I want to automate the task of design. I want a human designer to be able to come up with the seed or kernel of an idea, and have an automated system cultivate and nurture that idea, to play with it and expand upon it. The human would the direct the action, with the machine helping to fill in the details to provide a richer, deeper experience. Think of the machine as a creative companion rather than a substitute. It would perform a very similar role to your idea of crowd-sourced wisdom.
EDIT: For the record, I'll be looking into the slowness of the stackRNN implementation. I think I can make things work the way I want them to.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
I'd be happy to give it a try.
EDIT: Running the generator now.
Sorry for not making much progress with the stackRNN as of yet, by the way. Once I have a good fix for the problems I'm facing, it won't take very long to run. But I'm currently deeply engaged with my dissertation work on software bug detection/prevention, and that eats up most of my waking hours.
I like to think of myself as a time-travelling detective. The deep neural network is an oracle who can predict that a crime is going to happen and where it will happen, but it can't tell me when or why. I have the power to move forwards and backwards through time, as well as across branching timelines, in order to piece together the evidence I need. However, there are two caveats:
* Every time I make a timeshift, I use up a certain amount of energy (well, space/time, but you get the idea), and I only have a finite budget that I can spend. I have to make the best use of the available evidence when making my decisions.
* When I've established the who/what/when/where/why/how, I have to prevent the crime from happening. But I can't do anything that will negatively interfere with future timelines. So pulling a little girl out of the way of a hit-and-run is too risky. The better course of action is to go back 20 years and petition the city council to have a crosswalk built on that street. I've done my job right when it looks like I didn't do anything at all.
The work I'm doing is exciting and I feel that it's vitally important, but it's also incredibly exhausting.
I can't formally describe why the oracle works, she is inscrutable. Sometimes she gives me leads that go cold, but it may be the case that she was right all along and I just didn't have the energy to search long enough. Other times she leads me straight to the crime. Lacking the theoretical framework to understand why she works, instead I'm having to establish some empirical bounds on the quality of her predictions.
The time-travelling part I can explain. It's all just formal logic and concrete facts. But without the oracle, I'd never know which way to go, there are too many directions, too many forking timelines. I once calculated that for a modest-sized problem, if a single point in the continuum occupied a physical region of one square meter, all the timelines together would equal 8 billion galaxies in size. I quickly reach a point where there aren't enough computational resources on Earth to search that space in a reasonable span of time.
So yeah, that keeps me busy. lol
EDIT(2):
I can try finding some resources for you later.
In principle, an LSTM cell can hold onto a piece of information indefinitely. It can shut out new inputs and prevent them from contaminating the data it's currently holding. In practice that's not quite how things work. Usually the cells will hold onto a piece of information as long as it is expedient to do so, but when new, tantalizing information comes along, they're quite content to flush the old in exchange for the new.
Overall what happens is that the memories decay over time, and this complicates the process of learning associations between text elements that are spaced far apart.
That's why I'm looking into allowing the network to make use of permanent data storage. In effect, it could write down things that it has to keep track of, and then it only has to retain the memory of where it wrote the information down rather than the information itself. That, from what I've read, would greatly improve upon the network's ability to overcome long-term memory limitations.
----
EDIT(3): So the first run was with the default weights and I got mixed results, depending on how you look at it. On one hand, we're asking the network to extract style from the card art rather than the content, and it does a decent job of that. The cosplay image takes on a more painterly style, the network delivered what it promised, with only a minimal amount of content bleed (e.g. the floor looks more blue and wavy than it did before, and there are some minor embellishments on the dress, like the pattern in the middle).
But really we would like for the network to be more liberal with what it transfers, so I readjusted the weights accordingly and I am re-running everything.
... and I over-shot it. There has to be a happy medium... Running it once more.
Not quite there yet either. Still some inappropriate content bleed. But if you merge versions of the image (as I've done), you get something that's halfway decent. I think the shadows are causing some problems, because it's interpreting those as if they were content boundaries. I think you'd probably get better results if the lighting were more even.
Oh, and I think that if you were to prime it with the original image, you might get more consistent results. I didn't think to do that, I used random priming, which contributes to the noise. So some combination of that and lighting normalization should get you a better image, and if I have time later this evening I can look into that for you.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
In other words, I can more or less retexture her dress to be anything I want, lol.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
I can definitely use some of these. What a neat way to get new art for specific characters.
No problem! I'm glad I was able to help you.
Ah, gotcha.
And how!
I'd say that the solution space is quite large. I'm certain that you could teach the network to play a Magic, but it will be challenging to teach it to play our Magic.
Sort of like how no one plays Monopoly correctly (i.e. "Wait, you mean the games aren't meant to last 10 hours?"). That is, unless you plan on incorporating some kind of reinforcement learning on top of whatever it picks up on its own.
Still, there's a lot of fun to be had in that direction.
Fun ideas! I agree. Partitioning is definitely possible.
And I could try updating the cosplay image later. Right now my machines are tied up diagnosing buggy code.
Haha, sure. My primary research interest as a PhD candidate is in software reliability, that is, making sure that things don't break, fail, or try to kill us. But that's not my only area of research. During my undergraduate studies, I did research in computational aesthetics. That is, I sought to determine whether machines could arrive at an understanding of things like beauty and creativity. I feel that I'm much closer to an answer now than ever before, so this is all very exciting stuff for me.
But what is the end goal? What is our trajectory? I'm not looking to replace the human artist, because I believe that creative expression is an intrinsic part of the human experience. No, instead, I see opportunities for machines to act as co-creators.
Thanks to a combination of sheer computational power, limitless data (thanks to the Internet), and smart, adaptive algorithms, we now have access to an inexhaustible supply of creative potential. The human designer can set the stage and pick the direction, and the machine can come in help fill in the gaps and assist the creative process.
Like you say "There is a street, with many vendors..." and the machine comes in with "there's a hot-dog vendor... his name is Ollie (creates signage for Ollie's hot dog stand)... he lives alone... his wife died... of hot-dog poisoning! The memory haunts him." The machine just builds upon itself, a self-referential cycle producing layer upon layer of precious coral. The more you zoom in or zoom out, the more it creates - effortlessly, like in a dream. It's the same kind of extrapolation that makes our Magic cards or produces our pictures, just taken to its logical conclusion. The same process could be applied to music, to literature, to video games.. anything.
I see tons of avenues where man and machine can team up to create beautiful works, and I'm excited to see where things lead in the next decade or two.
---
EDIT: I spoke with the creator of the data-structure-augmented-network, and he told me that he plans on making updates and fixes to his code next week (hopefully), so I'll be keeping an eye out for that.
EDIT(2): As I promised some time ago, I'll do that H.R. Giger version of Damia when I get the chance. That's on the list.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
Edit: and later you mentioned 'hotdog vendor' which is a term that has special meaning to me. In another part of the internet, it means 'game developer'. I'm not kidding.
Oh? I've never seen that show. I may have seen an ad for it a long time ago though as I recognize the name. But the trope of an all-knowing machine goes way back.
And I've never heard that term before either. That's funny, lol. :-D
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
And that term is extremely niche. It all started with mustard, and Cookie Clicker, and Time...
Anyhow, it's always great to watch the progress. I really enjoyed the colour-sorted dump on the previous page. So many clever things going on, but I don't know how original they are, as I don't know the Magic corpus well enough to know what's ripped straight from real cards.
Oh? That does sound interesting. I'll have to check that out when I can find time to watch television.
Good question. There are many different ways for us to "prove" the originality of the cards produced. At the very least, you can use the vector trick I showed y'all to cheaply compare the semantics of the card with other, real cards.
Haha, I'm glad you've found all of this so entertaining.
And it's funny that you should describe it that way. Teaching is my passion, but lately I have not been teaching any courses because I've been focused on writing my dissertation. Aside from working with a lone undergraduate apprentice, and when I occasionally get called in to guest lecture, I'm locked up in my office without much contact with students. I suppose some of that pent-up energy has gotten redirected here. The difference is that normally the administration would charge y'all nearly a thousand dollars per person just to hear me pontificate, but here it's free, haha.
And I'll be sure to let you know if I read any other interesting papers. I go through about one or two papers a day, but a lot of them are just boring technical reports, that sort of thing.
---
By the way, here are a few interesting cards (just for fun)...
Turn the Endless
5BB
Sorcery (Rare)
As an additional cost to cast Turn the Endless, reveal an Elemental card from your hand.
Target player can't cast creature spells.
#I assume this effect lasts for the rest of the game.
Mana Breath
4G
Instant (Rare)
If a player would draw a card this turn, that player returns a creature card from his or her graveyard to the battlefield instead.
#Now that's interesting. Unfortunately, it doesn't draw any cards for you on its own, so you have to have some other card draw available, but the effect is interesting. Might be more more black than green though.
Skirmishisa
2G
Creature - Lizard (Uncommon)
Trample, haste
Cumulative Upkeep - Put a -1/-1 counter on Skirmishisa.
4/3
#I like that it's a lizard. That creature type does not get enough love.
And here are a few unusual ones...
Druid of Divility
6
Artifact Creature - Shapeshifter (Rare)
You may cast Druid of Divility from your graveyard if you control an Urza's River.
4/5
#The Urzatron trinity has gained a new combo piece, I guess. But what is that card? Survey says...
|urza's river||land|||N|||@ enters the battlefield tapped.\T: add {^} to your mana pool. when @ enters the battlefield, you may return target artifact card from your graveyard to your hand.|
|urza's river||land|||N|||@ enters the battlefield tapped.\T: add {^} to your mana pool.\{^} T: put a &/&^ blue barbarian creature token with haste onto the battlefield.|
|urza's river||land|||A|||T: add {^} to your mana pool.\T, pay &^ life, sacrifice @: search your library for a card named @, reveal it, and put it into your hand. then shuffle your library.|
Master of Familiar
W
Creature - Human Soldier (Common)
Whenever Master of Familiar attacks, detrive three target cards from a single graveyard.
1/1
#Wait. Hold on. "detrive"? Does it mean "retrieve"? That's almost correct usage of a word... that's not used in Magic English, so no. I guess it's a mystery mechanic.
And here are a few not so good ones...
Skhhunter Dragon
3GRRR
Creature - Dragon (Timeshifted)
Flying
When Skhhunter Dragon dies, put a +1/+1 counter on target attacking creature if it's white.
5/6
#Now this is an example of a card that you don't often get shown. Sometimes the network comes up with perfectly legal cards that are so incredibly niche in terms of their uses that I don't bother mentioning them. Skhhunter Dragon's ability only matters if it dies during combat, and only if you play Naya colors, and only if you're attacking with a white creature. Oh, and if you have the misfortune of playing against a w/X aggro deck, they can kill this guy to pump their creature (again, only during combat).
Cover the Ground
B
Instant (Rare)
You may shuffle your library.
#This has to have been inspired by One with Nothing. The name is funny though, it suggests "kick some dirt around".
---
EDIT: As promised, I've attached a version of Damia, Sage of Stone by H.R. Giger. I ran the network with the default settings. Right now it's mostly just a reskin, but I can turn up the Giger-ness and turn down the Argyle-ness if needed, and that would probably have the effect of fusing Damia with her chair in keeping with Giger-esque style.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
Thanks for the higher resolution version. A slightly Gigeresquer version would be nice, indeed, if it isn't a lot of work/trouble.
Sure! I had time to make one more, though I'll have to put future requests on hold for now as I'm dedicating the machines to a series of experiments to be run over the weekend.
I turned up the Giger and turned down the Argyle, and got this result. She's more machine than she is Gorgon at this point, though I like that the transformation preserved her face. That's probably because the object recognition part of the network said "ooh, human face!" and fixated on it; it flattened her nose and mouth, but kept the glowing eyes. Her hair and arms we're not quite so lucky, they got merged into her throne. Meanwhile, the background details now have more gear/machine like features than they did before.
Is this to your liking?
EDIT: Oh, another breakthrough! Evidently MIT researchers have designed a deep neural network system that can learn to play text-based MUD-style games with no prior knowledge of language or of the game. Here's the paper.
The AI apparently develops a mental map of the game content and learns quest-solving strategies like "use sword on [threatening monster]". Obviously, it's not foolproof, but that it can actually learn to play the games is quite impressive. And the architecture they use is quite similar to the ones we're using right now.
With this and the suggestions that YeGoblynQueenne made yesterday, that gives me all kinds of ideas.
"Play cards.. turn them sideways. Play cards.. turn them sideways." lol. Maybe something a little more sophisticated than that.
It's funny. There are statements that I made on this very thread a few months ago about the limitations of training such a system to do long-term goal planning, and those statements already sound dated now.
EDIT(2): I attached a graph to give you some idea of what we're looking at. The red curve is the authors' approach, and the others are the best alternatives to their approach. The alternatives, like bag-of-words approaches (think like our card vectors, that's how they're estimating the semantic content of the words that appear on the screen), are mediocre when it comes to questing. The authors' approach, meanwhile, is uncannily good.
In order for the LSTM network to do so well, it has to have some kind of understanding of the consequences of its actions. That's what makes this interesting.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
Terrifyingly perfect, thank you very much!
I don't care that the cards are malformed, or that some of them are un-interesting. I don't have the motivation to install the program and run the neural net / card generator myself, but I'm very interested in sifting through the output for items of interest! I feel like there may be others in the same boat.
Edit: Tried Croxis' web service, but I'm getting 504 Gateway Timeout every time. Any idea?
Hah, good ideas.
Not necessarily, but you're correct that there's an advantage. When you select two pictures that have the same level of detail (the scales match up), then the resulting image can look more coherent in terms of texture (as far as I I've seen).
Well, timeshifted or "special" or something (e.g. Nalathni Dragon). The rarity symbol was a "P". I'll have to look into that, lol.
Sure, I can look into that.
Do you just want the raw text or what?
EDIT: I set the older machine running to generate a dump of cards. I'll post it later for you.
It's been down, but Croxis has been very busy teaching his classes and other duties like that, so I don't think he's had the chance to bring it back online.
What we need now is to join our image generation process with something like Croxis' web service, haha. The problem is that the image rendering is computationally expensive, so there would be long turn-around times for card creation.
But honestly, there may be a way to make things cheaper if we had a limited artist selection. Like we have a specialized set up just for certain artists. But that only amortizes some of the cost of the style extraction.
There's a decent amount of overhead in the repeated use of the convnet, where we break down the image according to its content. See the attached image for an example of such a network tied to a caption-generating RNN (image taken from here, if you're interested). In short, we have to know what objects are in the image so that we can distinguish what constitutes the style of the image and the content of the image. The woman in the picture is the content, and the texture of her clothes, that we can treat as style. We can change the image as much as we'd like, style-wise, so long as we can still see the woman, racket, people in the background, etc. That restriction allows us do the style transfer without demolishing the original image.
Of course, there may be ways to optimize the computations involved, or even to skimp on some of them and still get good results, but that would require some in-depth analysis, lol.
---
Last night I went to a pre-release. Desolation Twin for the win.
For as much as I've talked about Magic, that was the first time I've gotten to play the game in ages, haha.
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
Here's a link to a dump.
I can generate more for you on demand. I'll note that I kept the temperature fairly high so that I could give you a diverse range of results (as opposed to ho-hum limited filler quality cards). For the sake of comparison, here's a low temperature sample:
|spire shade||creature||shade|O|&^/&^|{BB^}|{BB}: @ gets +&^/+& until end of turn.|
|soul shaman||creature||human shaman|O|&^^/&^^|{^^WW}|when @ enters the battlefield, you may destroy target artifact or enchantment.|
|death's presence||enchantment|||A||{^^^^BB}|whenever a player casts a spell, that player discards a card.|
|burning star||instant|||O||{RR}|@ deals &^ damage to target creature or player.|
Perfectly legal, acceptable, and unadventurous. Even the names are boring and trite! At the same time, raising the temperature also means there's more room for mistakes to creep in. Many of the cards are just fine, but some leave you with more questions than answers. I've identified three rough categories of mistakes and oddities that you're likely to see, as exemplified by these cards:
1. Cards that are legal but onerous in their specificity.
Master of Dawn
G
Creature - Human (Rare)
2G: Regenerate target creature. Activate this ability only during your upkeep.
1/1
#Hurray for debilitating restrictions! This looks like an attempt to make an ability on an underpriced creature "fair".
2. Cards that contain garbage.
Medicine Sphinx
6UUU
Creature - Sphinx (Rare)
Flying
Linstipt 2B
7/7
#I'm sure the high price of this creature is justified by whatever it is that "linstipt" does.
3. Cards that have disjoint rules text or that ask you to do impossible things.
Anarchize
1BB
Enchantment - Aura (Rare)
Enchant player
Whenever a creature deals combat damage to a player, tap target creature or player. otherwise, put one of them into your hand.
#Note: The ability triggers whenever any creature deals combat damage to any player, not just the enchanted player.
#Note: Players are considered tapped when laying on their side.
#Note: If you cannot reach across the table to hold another player in your hand, you may instead use a marker (such as a sticker) to indicate that a player is currently in your hand.
And then there are some other issues like misuse of abilities like kicker and X costs, as we've discussed before. On the bright side, you periodically get absolute gems.
EDIT: By the way, I like the user name. I have two ferrets myself.
EDIT(2): rofl at the name "Darwen, Martyr of Knowledge".
My LinkedIn profile... thing (I have one of those now!).
My research team's webpage.
The mtg-rnn repo and the mtg-encode repo.
There are a few new 15MB MSE2 set files in this folder that I just recovered a few days ago after regaining access to one of my machines. They were generated with randomly ordered fields, so a lot of the cards will be a little messed up, but there's probably some interesting stuff in there as well.