I'm a CS student and I'm starting with a research in which I want to research the possibilities to apply augmented reality to (not in particular) MTG cards.
The biggest part of the research will consist of actually recognizing where cards are and what cards that are. But to visualize the recognition I want use a simple form of augmented reality (AR). This can be something as simple as sprite.
I've already done some research on card recognition and found that a lot of the tools are readily available. They just need to be connected in the right way.
Think of Tesseract OCR for text and libraries for algorithms like SIFT or something similar.
For the camera I want to use the Kinect. It allows for high resolution video stream and is easy to program with because the SDK already supplies ways to work with the different video streams.
I now need to find as much information as possible on this subject. I can't find any research adressing these kind of subjects, they only go as far as cards used with poker or blackjack.
My question to you is:
I you are in any way interested in this and have some information about existing research, software, iOS apps, Android apps or whatever that does card recognition in any form. Would you share this information with me, maybe even write up a small review about the performance of the recognition (rotation invariant, distance, speed of recognition, success rate). Anything will help me a lot as it narrows down where I need to look.
Research in general concerning card recognition is also very welcome as I can use the references and compare results with other papers.
If there are no such programs it will make my case to do this research better, because it means that there has yet to be done any research on the subject.
Have you looked at the Eye of Judgment game that was released for PS3 a few years ago? It sounds a lot like what you're looking at here, although I believe that game used special codes on the cards (similar, if not identical, to QR codes).
As for recgonizing a normal Magic card, that would be much tougher. You'd have to be able to account for at least three different card faces (Old, modern, and Time Spiral time-shifted).
Private Mod Note
():
Rollback Post to RevisionRollBack
I am no longer on MTGS staff, so please don't contact me asking me to do staff things. :|
You'd have to be able to account for at least three different card faces (Old, modern, and Time Spiral time-shifted).
Maybe not. If you try to recognize the card by OCR recognition of the card name, it could be easier. Then you'd really just need a database of all the cards, plus their foreign names if you wanted to recognize the foreign cards as well.
Reprints the one card that people point to when saying that art objectifies women.
Well done Wizards.
Liliana does not objectify women in any way at all. We have gotten to a point in our society that every single picture of a women must be objectifying a women in some negative way......blah blah blah.. That is not the case. (((Sarcasm)))Picture of a girl drinking a milk shake, must be sex related and putting women down, picture of girl sitting on a beach, picture of a girl driving a car, picture of a girl on the moon at a new space station.)))
You have a picture of an attractive strong power women who girls dress up as for anime conventions. What more do you want? The picture is fine, happy to see a reprint. Sick of of seeing people claim that everything in existence must be putting women down. Then all I have to do is replace the word "women" with anything else to get the same mentality; fish, cats, arabs, blacks, jews, men, environment, whites, chinese, old people, etc. It doesn't matter what word I put in. Stop sucking life out of everything man. That artwork of her is awesome. Stop putting stuff down man. Just stop. If the picture was really as negative as you claim she would totally nude, in a kitchen, making sandwiches and giving blow jobs. Her abilities would be horrible as well. +1 do nothing -2 do nothing -6 do nothing. Instead liliana of the veil is an amazing planeswalker comparable to jace, the mind sculpter with great art to appreciate.
My suggestion listen to some comedy radio for a while, pandora is free, youtube is free there is something out there for you. ***** go make fun of somebody. The whole world is so serious and campaigning for some cause, or someones rights, everything is a hate crime, racist, sexist. blah blah blah.
"O no mcdonalds must be slandering a hate crime against skinny people every time they make a big mac." hahaha jeeze You're just someone perpetuating another groups negative perspective that they've made you believe is correct. Look at the picture for a hour and tell me what's wrong with it? I don't see anything.
I have heard vague rumors of a moustache-dispensing vending machine in a distant laundromat, across the street from a tattoo parlor. However, this information is shaky, and time is of the essence.
Have you looked at the Eye of Judgment game that was released for PS3 a few years ago? It sounds a lot like what you're looking at here, although I believe that game used special codes on the cards (similar, if not identical, to QR codes).
I do know about the Eye of Judgment and have spend some time looking for useable material. I might try to contact the developer/Sony to see if there is anything related. You are also right that it uses special codes on the cards for recognition, not something I'll be able to do on any trading card.
As for recgonizing a normal Magic card, that would be much tougher. You'd have to be able to account for at least three different card faces (Old, modern, and Time Spiral time-shifted).
The Time-Spiral face will give me some trouble. Because it puts information on different places than the old and modern face. The last too are very similar in that they put information that I want in the same place.
Maybe not. If you try to recognize the card by OCR recognition of the card name, it could be easier. Then you'd really just need a database of all the cards, plus their foreign names if you wanted to recognize the foreign cards as well.
I thought of using a database for all the cards. Then I would only need to OCR and fetch all the data. But I try to be more generic and 'future-proof'. New sets would automatically not be recognized because the database holds no information. But if I pull everything from the card I would only need an visualization to be used.
I want to find a good balance between the information that I can read from a card and the amount of information that I pull from somewhere else.
For example: With Pokemon you have cards that have the same name, yet they do different things. The best situation would be to link the name of a card to a single visualization. And then use OCR and other methods to grab all the information from the card (Type, HP, attacks, etc). This makes the system much more flexible but also more complex.
While I work on the recognition I will just have to see how far I can take this.
Not to mention the fact that DFC's, planeswalkers & levellers have a slightly different frame again (DFC have the sun moon thing, walkers have loyalty slots, levellers have the levellers slots).
I probably won't be able to recognize every card ever printed. As long as I can reliably recognize 'standard' cards I'll be happy. If I have time to spare on the research I can put effort into recognizing these special cases. But otherwise I will put it in the Future Work section of the report.
I don't think I understand what augmented reality is...
If you looked at the Wikipedia article it might not be entirely clear how I want to work it into my research. But if you look up what the Eye of Judgment does it should become clear. You could also compare it to what Yu-Gi-Oh! does in the anime, but less sci-fi.
You really don't need to worry about scanning the text of the card other than the name. Aside from the technical challenges, one of the biggest reasons for this is the extensive errata present in the game. Old versions of cards sometimes scarcely resemble newer versions (check out the various versions of Clone). Sometimes they don't even work the same way from printing to printing (Gloom). Even newly printed cards are subject to terminology changes (Archon of Justice and anything else that now "dies"). Yet any version is legal to play with if the card is legal in the format.
Keep it simple and focus on scanning and recognizing the name. You can connect this to a database with any other information you need, including full text and your sprites, which I assume you have to link to anyway. I don't understand how you're going to "future proof" the project because you have to have animation for each card, right?
I don't think I understand what augmented reality is...
The short version if you would play over a video conference program which would identify where on the screen what cards were located and put some manner of animated graphic over them in real time.
Also, I suspect OCR programs would have problems with the card titles on the old card face. That font is probably not very machine-readable.
It very well could. I don't expect it will work out anything I throw at it. Besides, I'll be doing some filtering before passing the image to Tesseract. I will just have to see how much I can cover during my research. If I fail on the old face I will just focus on the modern face.
You really don't need to worry about scanning the text of the card other than the name. Aside from the technical challenges, one of the biggest reasons for this is the extensive errata present in the game. Old versions of cards sometimes scarcely resemble newer versions (check out the various versions of Clone). Sometimes they don't even work the same way from printing to printing (Gloom). Even newly printed cards are subject to terminology changes (Archon of Justice and anything else that now "dies"). Yet any version is legal to play with if the card is legal in the format.
You make an interesting point. Instead of using the rules text on the card I could look up the oracle text in a database.
I don't understand how you're going to "future proof" the project because you have to have animation for each card, right?
I did mention I would miss animations for new cards. But it's also something I don't really want to consider. But it's a way to make the whole package more generic, ignoring the missing animation.
--
Before I was thinking about using Pokemon cards for the project because of the existing sprites and the difficulty of having different cards with the same name.
But I think I'll switch to MTG instead. MTG doesn't have readily available animations for cards, but I would go for something that they use in the Magic 2011/2012 game where they represent certain abilities like flying and trample with different icons.
I will go for a database containing the oracle text. I would only need to recognize the name of the card. Anything else would be because-I-can rather than gaining something out of it (of course only when time allows it :)).
The iMTG developer has a fairly robust and accurate OCR card identifier built into his app. I'm not sure if he'd be willing to share code as he sells the OCR as a plugin for his free base app, but I do know it's completely possible to do what you're describing for a very wide variety of cards.
Most of the ones I've seen try to go after the names, but if it could get art, or even expansion symbols, you could narrow your scope down to the version you were scanning.
My only question with the kinnect (and I've never really explored this) is that you have to be so far away from it and that seems out of the range of a mtg card. Perhaps that is only to motion capture. I don't know.
Private Mod Note
():
Rollback Post to RevisionRollBack
Out of the blackness and stench of the engulfing swamp emerged a shimmering figure. Only the splattered armor and ichor-stained sword hinted at the unfathomable evil the knight had just laid waste.
The iMTG developer has a fairly robust and accurate OCR card identifier built into his app. I'm not sure if he'd be willing to share code as he sells the OCR as a plugin for his free base app, but I do know it's completely possible to do what you're describing for a very wide variety of cards.
Thanks. This will give me some comparison material to see what others have accomplished.
Most of the ones I've seen try to go after the names, but if it could get art, or even expansion symbols, you could narrow your scope down to the version you were scanning.
My only question with the kinnect (and I've never really explored this) is that you have to be so far away from it and that seems out of the range of a mtg card. Perhaps that is only to motion capture. I don't know.
Going for the name is obvious, as it is unique for each card. If I have time left I will be trying to get more information from cards, set symbol being one.
For motion tracking it requires a distance of about 2 meters. But you can get a videostream regardless. I won't use the motion tracking (maybe the depth stream if I make use of it somehow).
Not to mention the fact that DFC's, planeswalkers & levellers have a slightly different frame again (DFC have the sun moon thing, walkers have loyalty slots, levellers have the levellers slots).
All that information would be stored in the game, all it would need is to recognise the name and access the information it already owns.
Private Mod Note
():
Rollback Post to RevisionRollBack
I play D&D3.5e, 4e, and Magic.
If I'm bored enough I'll play DotPW(2012). Steam: ph34rless, (Screen Name:) ieatclowns
To echo a sentiment that has already been expressed here:
I don't think your computer should be trying to read anything on the cards except for cardname. You want to look up the rest of the info on an oracle database - kind of like how Cockatrice works.
To echo a sentiment that has already been expressed here:
I don't think your computer should be trying to read anything on the cards except for cardname. You want to look up the rest of the info on an oracle database - kind of like how Cockatrice works.
And I'm going to follow that advice.
I will recognize the name of the card, grab other information from a database and show icon like the ones used in 'Duel of the Planeswalkers'.
If time allows for it, I also want to recognize the set symbol in case of reprints. Although this only affects the art...
This is coming from a place of complete ignorance, but why is the scanning of the cards a coding issue (at least I assume you mean coding)? Couldn't you just use some off the shelf scanning eq and then just customize the image to text conversion? I know there are scanners that read printed text and formats it in to a document, so couldn't you just use similar equipment and instead code it to read the image title and then find it in a database to provide the image and rules?
I can, and will. Tesseract OCR lets me pass an image to it and will return the text on the image back.
The problem is that I don't know where cards are, or how they are placed (at what angle). This is something I will have to work on. Making my own character recognition software would be meaningless as there are tons of solutions that already do it for me.
Many scanners allow for OCR, but these assume certain conditions. Like pages can't be upside-down or placed sideways. In my case the recognition should work regardless of how the cards are positioned.
I also want to do the recognition with a video stream as source and perform certain operations before showing it to the user (Augemented Reality part). No off-the-shelf hardware can do that for me.
I can, and will. Tesseract OCR lets me pass an image to it and will return the text on the image back.
The problem is that I don't know where cards are, or how they are placed (at what angle). This is something I will have to work on.
Just establish fields by way of creating a play mat. By designating specific play zones you can associate rules associated with that zone, which should make things simpler. Then just designate what "tapped" means. Some people tap differently but most people tap by rotating the card 90° clockwise, so designated that position as "tapped". Create a brief visual tutorial that shows how to tap and where you have established the zones. Also it seems to me that you really do not need full motion range, you just need recognition of the established card orientation. There is no need for recognition of "half tapped" or any other orientation in between.
Oh...I got another idea. Take a page from motion capture technology and sleeve the cards with a special sleeve. Put a yellow (or whatever color) dot in the bottom left corner that the scanner can use to establish position and orientation of the card. It also will give your scanner a way to know where to look for the unique identifier.
Edit: Make that two dots of different colors so you will be able to use their relative positions to determine orientation better. Something like this:
Again, sorry if this is all stuff you have worked out or would not work, I am not trained in CS at all and frankly it baffles me.
One problem arises: tapped/untapped Kamigawa flip cards. Particularly since some people tap by rotating clockwise, while other people tap by rotating counter-clockwise.
One problem arises: tapped/untapped Kamigawa flip cards. Particularly since some people tap by rotating clockwise, while other people tap by rotating counter-clockwise.
This is why I think the player should learn how the software reads the game. If the tutorial dictates that tapping means turning a card 90 deg clockwise, then that is how players will do it.
Again, sorry if this is all stuff you have worked out or would not work, I am not trained in CS at all and frankly it baffles me.
That's oke. The thing is, I made this post to get views from other people. It has already helped me quite a lot by just making me think differently.
What you're doing is going a few step further than what I want to achieve with my research.
What you suggest is more in terms of playing the game. Seeing the difference between tapped and untapped cards and such. I don't view it that way. What I want is to just throw down a card and no matter how it ends up I must be able to recognize it.
When I have working software that does this, somebody else might pick it up and implement the rules of Magic with it. Then you would have tutorials about how to play, placing mana, zones, tapping etc.
You can see my research as just a small step towards playing paper Magic on computers. I'm only focusing on recognizing the cards, the AR part is a feature that IMO gives it a really nice touch.
One problem arises: tapped/untapped Kamigawa flip cards. Particularly since some people tap by rotating clockwise, while other people tap by rotating counter-clockwise.
Flip cards will be a pain anyway. Because there is no clear top or bottom because both sides have names. This can be solved in the database by storing both names and whether it's the flipped version.
What you suggest is more in terms of playing the game. Seeing the difference between tapped and untapped cards and such. I don't view it that way. What I want is to just throw down a card and no matter how it ends up I must be able to recognize it.
Ah, I see now. Seems like something like that would have applications outside of card games. Good luck.
I'm a CS student and I'm starting with a research in which I want to research the possibilities to apply augmented reality to (not in particular) MTG cards.
The biggest part of the research will consist of actually recognizing where cards are and what cards that are. But to visualize the recognition I want use a simple form of augmented reality (AR). This can be something as simple as sprite.
I've already done some research on card recognition and found that a lot of the tools are readily available. They just need to be connected in the right way.
Think of Tesseract OCR for text and libraries for algorithms like SIFT or something similar.
For the camera I want to use the Kinect. It allows for high resolution video stream and is easy to program with because the SDK already supplies ways to work with the different video streams.
I now need to find as much information as possible on this subject. I can't find any research adressing these kind of subjects, they only go as far as cards used with poker or blackjack.
My question to you is:
I you are in any way interested in this and have some information about existing research, software, iOS apps, Android apps or whatever that does card recognition in any form. Would you share this information with me, maybe even write up a small review about the performance of the recognition (rotation invariant, distance, speed of recognition, success rate). Anything will help me a lot as it narrows down where I need to look.
Research in general concerning card recognition is also very welcome as I can use the references and compare results with other papers.
If there are no such programs it will make my case to do this research better, because it means that there has yet to be done any research on the subject.
Thanks in advance,
devGod
As for recgonizing a normal Magic card, that would be much tougher. You'd have to be able to account for at least three different card faces (Old, modern, and Time Spiral time-shifted).
Two Score, Minus Two or: A Stargate Tail
(Image by totallynotabrony)
I do know about the Eye of Judgment and have spend some time looking for useable material. I might try to contact the developer/Sony to see if there is anything related. You are also right that it uses special codes on the cards for recognition, not something I'll be able to do on any trading card.
The Time-Spiral face will give me some trouble. Because it puts information on different places than the old and modern face. The last too are very similar in that they put information that I want in the same place.
I thought of using a database for all the cards. Then I would only need to OCR and fetch all the data. But I try to be more generic and 'future-proof'. New sets would automatically not be recognized because the database holds no information. But if I pull everything from the card I would only need an visualization to be used.
I want to find a good balance between the information that I can read from a card and the amount of information that I pull from somewhere else.
For example: With Pokemon you have cards that have the same name, yet they do different things. The best situation would be to link the name of a card to a single visualization. And then use OCR and other methods to grab all the information from the card (Type, HP, attacks, etc). This makes the system much more flexible but also more complex.
While I work on the recognition I will just have to see how far I can take this.
I probably won't be able to recognize every card ever printed. As long as I can reliably recognize 'standard' cards I'll be happy. If I have time to spare on the research I can put effort into recognizing these special cases. But otherwise I will put it in the Future Work section of the report.
If you looked at the Wikipedia article it might not be entirely clear how I want to work it into my research. But if you look up what the Eye of Judgment does it should become clear. You could also compare it to what Yu-Gi-Oh! does in the anime, but less sci-fi.
Keep it simple and focus on scanning and recognizing the name. You can connect this to a database with any other information you need, including full text and your sprites, which I assume you have to link to anyway. I don't understand how you're going to "future proof" the project because you have to have animation for each card, right?
The short version if you would play over a video conference program which would identify where on the screen what cards were located and put some manner of animated graphic over them in real time.
http://dsc.discovery.com/videos/curiosity-what-technology-makes-augmented-reality-possible.html
Currently looking to buy miscut Homelands, (my wife thinks I'm crazy too).
Semper Gumby (Always Flexible)
It very well could. I don't expect it will work out anything I throw at it. Besides, I'll be doing some filtering before passing the image to Tesseract. I will just have to see how much I can cover during my research. If I fail on the old face I will just focus on the modern face.
You make an interesting point. Instead of using the rules text on the card I could look up the oracle text in a database.
I did mention I would miss animations for new cards. But it's also something I don't really want to consider. But it's a way to make the whole package more generic, ignoring the missing animation.
--
Before I was thinking about using Pokemon cards for the project because of the existing sprites and the difficulty of having different cards with the same name.
But I think I'll switch to MTG instead. MTG doesn't have readily available animations for cards, but I would go for something that they use in the Magic 2011/2012 game where they represent certain abilities like flying and trample with different icons.
I will go for a database containing the oracle text. I would only need to recognize the name of the card. Anything else would be because-I-can rather than gaining something out of it (of course only when time allows it :)).
My only question with the kinnect (and I've never really explored this) is that you have to be so far away from it and that seems out of the range of a mtg card. Perhaps that is only to motion capture. I don't know.
Thanks. This will give me some comparison material to see what others have accomplished.
Going for the name is obvious, as it is unique for each card. If I have time left I will be trying to get more information from cards, set symbol being one.
For motion tracking it requires a distance of about 2 meters. But you can get a videostream regardless. I won't use the motion tracking (maybe the depth stream if I make use of it somehow).
All that information would be stored in the game, all it would need is to recognise the name and access the information it already owns.
If I'm bored enough I'll play DotPW(2012).
Steam: ph34rless, (Screen Name:) ieatclowns
I don't think your computer should be trying to read anything on the cards except for cardname. You want to look up the rest of the info on an oracle database - kind of like how Cockatrice works.
BRG Loam Control (Assault - Loam) BRG
W Mono White Control (Martyr - Proc) W
And I'm going to follow that advice.
I will recognize the name of the card, grab other information from a database and show icon like the ones used in 'Duel of the Planeswalkers'.
If time allows for it, I also want to recognize the set symbol in case of reprints. Although this only affects the art...
Reprint Opt for Modern!!
FREE DIG THOROUGH TIME!
PLAY MORE ROUGE DECKS!
The problem is that I don't know where cards are, or how they are placed (at what angle). This is something I will have to work on. Making my own character recognition software would be meaningless as there are tons of solutions that already do it for me.
Many scanners allow for OCR, but these assume certain conditions. Like pages can't be upside-down or placed sideways. In my case the recognition should work regardless of how the cards are positioned.
I also want to do the recognition with a video stream as source and perform certain operations before showing it to the user (Augemented Reality part). No off-the-shelf hardware can do that for me.
Oh...I got another idea. Take a page from motion capture technology and sleeve the cards with a special sleeve. Put a yellow (or whatever color) dot in the bottom left corner that the scanner can use to establish position and orientation of the card. It also will give your scanner a way to know where to look for the unique identifier.
Edit: Make that two dots of different colors so you will be able to use their relative positions to determine orientation better. Something like this:
Again, sorry if this is all stuff you have worked out or would not work, I am not trained in CS at all and frankly it baffles me.
Reprint Opt for Modern!!
FREE DIG THOROUGH TIME!
PLAY MORE ROUGE DECKS!
Two Score, Minus Two or: A Stargate Tail
(Image by totallynotabrony)
Reprint Opt for Modern!!
FREE DIG THOROUGH TIME!
PLAY MORE ROUGE DECKS!
That's oke. The thing is, I made this post to get views from other people. It has already helped me quite a lot by just making me think differently.
What you're doing is going a few step further than what I want to achieve with my research.
What you suggest is more in terms of playing the game. Seeing the difference between tapped and untapped cards and such. I don't view it that way. What I want is to just throw down a card and no matter how it ends up I must be able to recognize it.
When I have working software that does this, somebody else might pick it up and implement the rules of Magic with it. Then you would have tutorials about how to play, placing mana, zones, tapping etc.
You can see my research as just a small step towards playing paper Magic on computers. I'm only focusing on recognizing the cards, the AR part is a feature that IMO gives it a really nice touch.
Flip cards will be a pain anyway. Because there is no clear top or bottom because both sides have names. This can be solved in the database by storing both names and whether it's the flipped version.
Reprint Opt for Modern!!
FREE DIG THOROUGH TIME!
PLAY MORE ROUGE DECKS!
Misc. EDH Stuff: Commander Cube | Zombies (Horde)
Resources:Commander Rulings FAQ | Commander Deckbuilding Guide
Follow me on Twitter! @cryogen_mtg