After two days and a few online questions and answers, I got the thing to work. I've loaded two sets worth of cards into my DB so far. My biggest roadblock was solved when I was told to use the mysql_real_escape_string() to escape the slashes and such (the DB didn't like them at all).
I can provide my working PHP/MySQL code for anyone who wants it.
Private Mod Note
():
Rollback Post to RevisionRollBack
Play history: Revised through Mirage, then M14 to present.
mtgjson isn't intended to be an "API" necessarily. You probably should not be loading the json into your app clients straight from the site. Rather, the intention is to provide a comprehensive and consistent set of data so that you can build your own data set, specific to the needs of your app.
That being said, the only reason I can imagine you're getting Hearthstone sets is that there was a problem with your DNS and/or caching. Try clearing your browser cache, then figure out how to clear your DNS cache, and try again?
Using a JSON parser I don't have any trouble with the 3.3.5 version. If you're doing string parsing of the file I highly suggest you rework your parser to use some sort of JSON parser. Whatever language you're using there is probably a parser library already written for it.
I second Nis's comment. The order of the fields shouldn't really matter in well-formed JSON, and you should save yourself a lot of work by using a third-party parser!
@andarilhomtg: I've created a simple Java project to demonstrate using Jackson to deserialize the data. The example linked to in the documentation returned 404.
Another documentation issue: the sets field doesn't include any information about magicRaritiesCodes. Haven't ever used it so my sample parser stumbled across it.
I've noticed some have interest in obtaining this data through an api, rather than having to parse it and store it in their own database. I have a MTG API (magicthegathering.io) that builds its datasource primarily from mtgjson. I have quite a few regular consumers of the api now, but haven't really advertised it yet since I wanted to have it running for awhile first. This API really does rely on mtgjson staying up to date, so I just wanted to thank @Andarilhomtg and @Sembiance for the awesome work they have done.
influenced by the tips from you and others I finally managed to remove my simple json parser and included a thirdparty library (SuperEasyJson). Now everything is working again!
But SuperEasyJson needs 20 seconds to deserialize the json file, compared to 2 secs before
That's unfortunate. I'm not sure how SuperEasyJson handles parsing; some parsers are just more efficient. The manner in which you use the file can affect speed as well. Streaming the json, i.e. deserializing an object at a time, should be faster than deserializing the whole file into memory. Maybe compare SuperEasyJson to another parser like libjson. SEJ's own readme states that it's not designed for speed or memory efficiency but rather simplicity.
I saw you added color identity as a field in the card data. I've been calculating color identity based on MTGJson data for a while, so I figured I should double-check that they're the same, and I noticed a few issues:
1. Basic land types contribute to color identity. This goes for plain old basics as well as duals.
2. Symbols that are part of reminder text don't count. Most notably, this mean cards with Extort, though Charmed Pendant is another example. This regular expression matches symbols (anything within curly brackets) that are not contained within parentheses:
\{[^\}]*\}(?![^(]*\))
3. (Mostly stylistic) Having the color identity as a set of 1-char strings seems at odds with having the color as a set of full-word strings.
1. I'll fix the basic lands. Have you noticed any non-basic lands with errors?
1.1. I believe fetch lands don't have any color identity, am I right?
2. Awesome! I'll update the regex.
3. Yeah... I'll get back to you on this one.
1. I'll fix the basic lands. Have you noticed any non-basic lands with errors?
1.1. I believe fetch lands don't have any color identity, am I right?
2. Awesome! I'll update the regex.
3. Yeah... I'll get back to you on this one.
Thanks!
Fetchlands don't count. It's only nonbasics that have basic types, so Prairie Stream, Hallowed Fountain, and Tundra should have color identity of WU, for example, even though they don't have any (non-reminder text) symbols in their text box.
What do you think about adding rarities (or at least the most recent) to AllCards-x.json? This really is the last thing missing for the card info to make it complete.
That file is meant specifically for properties of the card not from any specific set. To add rarity you need to decide what you're considering most recent. For example, do you include online only sets like vintage masters? What about media promos? Supplemental printed projects? It's not a decision that can be made without knowing the use case, so I'd recommend not adding it to that file.
Well... last printed set was only a suggestion. If this particular file is meant for card properties not from any specific set, then why include info about sets a card was printed in in the first place? I mean is it really bad to have rarity information alongside already provided set info? It's just a suggestion which - in my personal opinion - would make the specific file slightly more usable.
What is Wizards' position on this? Is a card's official rarity overridden by its most recent printing, or is it a property that's intrinsically unique to a printing?
Private Mod Note
():
Rollback Post to RevisionRollBack
Minouris's Library - Collection manager and deck builder. It's nifty - Check it out!
Well... last printed set was only a suggestion. If this particular file is meant for card properties not from any specific set, then why include info about sets a card was printed in in the first place? I mean is it really bad to have rarity information alongside already provided set info? It's just a suggestion which - in my personal opinion - would make the specific file slightly more usable.
It would make that file more useful, yes, but where do you stop? The printings there are so you know where to find that card. Adding rarities would obviously be useful to people who need rarities, but what about artist, card number, etc? The idea is that AllCards is not a replacement for AllSets, but a purposefully light(er)weight file.
What is Wizards' position on this? Is a card's official rarity overridden by its most recent printing, or is it a property that's intrinsically unique to a printing?
I do not believe Wizards gives any "official" rarity to cards. It's intrinsic to a specific printing, though power-level and cost dictate similar rarities of reprints. If you look on Gatherer, you'll see that there is no "rarity" field, only colored expansion symbols in which they were printed.
Is there a nice .csv version of this table somewhere?
I have been hunting for a nice table I can manipulate in excel of all Magic Cards and their properties, and I cannot for the life of me find anything usable. No website seems to have that anywhere, any plaintext list is unparsable for one reason or another, and then I got to this and thought "Oh, I'll just use a json parser and convert this one" Wrong. Every converter I have tried has either failed, frozen, told me it was an improper json file, or tried to charge me for the privilege to convert one file.
Or if anyone knows where I can go to find a current database of all magic cards in csv/xls they'd be my hero.
I am completely flabbergasted that this doesn't exist somewhere.
Hey, I just realized there are several cards that contain double spaces in their text. Hopefully this isn't indicative of any larger problems. Daru Stinger is one of these cards, it has a double space between "each" and "Soldier".
Since mtgimage.com is no longer live, how you get images for the cards? At the moment I'm considering just pulling them off gatherer and storing them locally, but I'm wondering if there are any alternatives.
Just a heads up; the way double-faced cards appear in packs in SOI is different than in the original Innistrad block. Every pack has a common or uncommon DFC that replaces a common. 1 in 8 packs has a Rare or Mythic Rare DFC that replaces another common (in addition to the Common/Uncommon DFC). With that in mind, the "booster" field should look something like this:
I can provide my working PHP/MySQL code for anyone who wants it.
mtgjson isn't intended to be an "API" necessarily. You probably should not be loading the json into your app clients straight from the site. Rather, the intention is to provide a comprehensive and consistent set of data so that you can build your own data set, specific to the needs of your app.
That being said, the only reason I can imagine you're getting Hearthstone sets is that there was a problem with your DNS and/or caching. Try clearing your browser cache, then figure out how to clear your DNS cache, and try again?
[card=Jace Beleren]Jace[/card] = Jace
Magic CompRules
Scry Rollover Popups for Google Chrome
The first rule of Cursecatcher is, You do not talk about Cursecatcher.
I'm ordering the fields by name, so the "diffs" are more readable.
You should really use a parser that does not rely on the order of the fields...
Regards,
- Sergio Moura
Another documentation issue: the sets field doesn't include any information about magicRaritiesCodes. Haven't ever used it so my sample parser stumbled across it.
[card=Jace Beleren]Jace[/card] = Jace
Magic CompRules
Scry Rollover Popups for Google Chrome
The first rule of Cursecatcher is, You do not talk about Cursecatcher.
I'll check on the magicRaritiesCodes doc issue.
That's unfortunate. I'm not sure how SuperEasyJson handles parsing; some parsers are just more efficient. The manner in which you use the file can affect speed as well. Streaming the json, i.e. deserializing an object at a time, should be faster than deserializing the whole file into memory. Maybe compare SuperEasyJson to another parser like libjson. SEJ's own readme states that it's not designed for speed or memory efficiency but rather simplicity.
[card=Jace Beleren]Jace[/card] = Jace
Magic CompRules
Scry Rollover Popups for Google Chrome
The first rule of Cursecatcher is, You do not talk about Cursecatcher.
I saw you added color identity as a field in the card data. I've been calculating color identity based on MTGJson data for a while, so I figured I should double-check that they're the same, and I noticed a few issues:
1. Basic land types contribute to color identity. This goes for plain old basics as well as duals.
2. Symbols that are part of reminder text don't count. Most notably, this mean cards with Extort, though Charmed Pendant is another example. This regular expression matches symbols (anything within curly brackets) that are not contained within parentheses:
\{[^\}]*\}(?![^(]*\))3. (Mostly stylistic) Having the color identity as a set of 1-char strings seems at odds with having the color as a set of full-word strings.1. I'll fix the basic lands. Have you noticed any non-basic lands with errors?
1.1. I believe fetch lands don't have any color identity, am I right?
2. Awesome! I'll update the regex.
3. Yeah... I'll get back to you on this one.
Thanks!
Fetchlands don't count. It's only nonbasics that have basic types, so Prairie Stream, Hallowed Fountain, and Tundra should have color identity of WU, for example, even though they don't have any (non-reminder text) symbols in their text box.
That file is meant specifically for properties of the card not from any specific set. To add rarity you need to decide what you're considering most recent. For example, do you include online only sets like vintage masters? What about media promos? Supplemental printed projects? It's not a decision that can be made without knowing the use case, so I'd recommend not adding it to that file.
Hopefully it's just temporary! In the meantime Internet Archive has a copy, which I believe includes the JSON files: https://web.archive.org/web/20151124175319/http://mtgjson.com/
What is Wizards' position on this? Is a card's official rarity overridden by its most recent printing, or is it a property that's intrinsically unique to a printing?
Minouris's Library - Collection manager and deck builder. It's nifty - Check it out!
It would make that file more useful, yes, but where do you stop? The printings there are so you know where to find that card. Adding rarities would obviously be useful to people who need rarities, but what about artist, card number, etc? The idea is that AllCards is not a replacement for AllSets, but a purposefully light(er)weight file.
I do not believe Wizards gives any "official" rarity to cards. It's intrinsic to a specific printing, though power-level and cost dictate similar rarities of reprints. If you look on Gatherer, you'll see that there is no "rarity" field, only colored expansion symbols in which they were printed.
Thank you very much, man! I appreciate it
I have been hunting for a nice table I can manipulate in excel of all Magic Cards and their properties, and I cannot for the life of me find anything usable. No website seems to have that anywhere, any plaintext list is unparsable for one reason or another, and then I got to this and thought "Oh, I'll just use a json parser and convert this one" Wrong. Every converter I have tried has either failed, frozen, told me it was an improper json file, or tried to charge me for the privilege to convert one file.
Or if anyone knows where I can go to find a current database of all magic cards in csv/xls they'd be my hero.
I am completely flabbergasted that this doesn't exist somewhere.
[Remixes] - [The Brutal Cube - 360 Powered] - [My Cube Article] - ['Print-This' Wishlist]
That being said, if you get me the columns you would want, I can try to whip something up for you.
Small query about the set code, though - is that the official one? The set symbol image link in Gatherer (http://gatherer.wizards.com/Handlers/Image.ashx?type=symbol&set=DDQ&size=small&rarity=U) has it as DDQ, following the convention of the previous duel decks.
Minouris's Library - Collection manager and deck builder. It's nifty - Check it out!
{ "name" : "Shadows Over Innistrad", "booster" : [ ["land","checklist"] "marketing", "common", "common", "common", "common", "common", "common", "common", "common", ["common", "double faced rare", "double faced mythic rare"], ["double faced common", "double faced uncommon"], "uncommon", "uncommon", "uncommon", ["rare", "mythic rare"] ] }