SHA1 does generate duplicates in some cases. The unique id problem was solved a couple of months ago, though. I believe one has to use a string which contains (english card name + expansion name + image name) to generate an unique SHA1.
Aye, you are correct, it is indeed unique and the SHA1 card 'id' is made up of (setCode + cardName + cardImageName). It's mentioned in the mtgjson docs.
@cyclotis04 Basically, I did a bunch of database inserts using cardId as a primary key, and got duplcate primary key clashes.
Not sure what this 'cardId' you are referring to. The card objects in mtgjson have an 'id' field. Did you mean that?
Because on that field, there are no duplicate id's.
The mtgjson code checks for this every time the JSON files are generated and I also just tested them again just to be sure. The point of this field is to be unique and it is indeed, unique.
Unfortunately, SHA is a hash, not a UUID, so it can't guarantee uniqueness. The bigger the dataset, the bigger the chance of a duplicate
Yes, it is technically possible for there to be a collision with a SHA1 hash. However is is EXTREMELY unlikely. See: http://crypto.stackexchange.com/questions/2583/is-it-fair-to-assume-that-sha1-collisions-wont-occur-on-a-set-of-100k-strings
That URL talks about how hashing 100,000 strings and even over 1 million strings, produced no collisions. It goes on to talk about really big numbers before generating any collisions in theory. There are only 29,416 cards in the MTGJSON files. Even with 100 more years of MTG cards, it is unlikely to have a collision, but as I said, this is checked automatically when the JSON is generated to ensure there are no collisions, just in case.
I wanted to let everyone know that I am in the process of transferring mtgjson over to another individual. There shouldn't be any downtime at all and the website address will stay the same and everything. I'll update the the thread here once the process is complete.
I launched MTGJSON over 2 years ago. Since then my enthusiasm for it has diminished a lot.
I am open to someone else taking over the site, domain and code.
Do not fear, I am not planning on shutting it down, and I will continue to update it for the time being. However, if someone out there is up for taking it over who would continue to update it and run the site, I am open to that. Feel free to come forward and discuss here on the forum or you can e-mail me robert@cosmicrealms.com
Heya, I just noticed that the foreignNames field is no longer present in AllCards-X.json. How come? It was populated by the latest translation before and now it is completely missing. Can you please add it back to AllCards-X?
I just need to know the translated name of each card and don't care about in which set it was translated / not translated.
Assuming this is because the Expedition cards are only printed in English, even though they can appear in international boosters
You're right. It seems it's a new behavior since MTGJSON 3.0.0.
Well, I think it's disturbing for non-english users. For example, a search with the name 'Vue de la canopée' (the french translation of 'Canopy Vista') will retrieve only the 'Canopy Vista' of 'Battle for Zendikar' (because this set have a french translation), not the 'Zendikar Expedition' one. The user will miss a print of this card.
It should be easy enough for anyone who is importing the JSON data into their app/webpage to simply do what I was doing before 3.0.0 which is to copy the foreignNames found on any card in any set to every other instance of that card in all sets.
Sembiance, I noticed a problem in Shadowmoor: the Chinese Trad. fields contain the english names.
Ahh yes, this is messed up on Gatherer too, which is where the non-english language data comes from. It is also messed up on magiccards.info
The presence of 'Chinese Traditional' on Gatherer for Shadowmoor may signal that it was indeed printed in that language, but they just don't have the data for it. On the other hand it may be a gatherer bug entirely and maybe it wasn't printed in that language. Due to this uncertainty I am hesitant to remove it from MTGJSON since right now the presence of those fields denotes whether or not it was printed in those languages. Tricky situation
I've just added Battle for Zendikar (BFZ) and Zendikar Expedition (EXP). Note: I'm aware that everywhere but gatherer has 'Expedition' as plural 'Expeditions', but MTGJSON always uses exactly as it is on Gatherer, so.
I've tried without success to import Allsets-x.json into MySQL.
If anyone has managed it could they please advise what "tricks" are required?
Thanks in advance...
Someone on GitHub has already done this: https://github.com/laxika/MTGJsonConverter
However the repo seems to be gone, you may want to contact them to see if they would provide you the code.
Any possibility of getting the card numbers used by magiccards.info into the data set? I've noticed that certain card types the numbers don't quite match - Cunning Bandit, for example, is 99 in your data, whereas magiccards.info uses 99a. And of course, the early sets lack a number entirely
Edit - Strike that first bit - that's a fault in my code, not your data.
I don't want to introduce numbers onto cards that didn't have them originally. I know that some of the promo sets I import from magiccards.info have numbers but they are incorrect or invalid (see https://github.com/Sembiance/mtgjson/issues/43)
To 'number' the cards yourself on sets that don't have numbers, you just need to sort them a specific way.
Hi,
Do you add Origins Clash pack? It has some alternate art cards.
Thanks
Michael
The clash pack cards are in a single set, 'Clash Pack' (CPK). This comes from magiccards.info and they have not yet updated to include the Magic Origins cards. They don't update very often, but I'm not aware of a more reliable and up to date source that has full card info. http://www.magiclibrarities.net/859-rarities-clash-pack-box-set-cards-english-cards-index.html has lists of the cards, but not the details of each card, so parsing that for the full card data isn't an option.
I see in the Changelog that you re-added the Variations field... Should that be appearing on all sets? It doesn't seem to be present in FTV: Angels.
The variations field is present when there are multiple cards with the same name in the same set. Because From the Vault: Angels doesn't have any duplicate cards (different art usually) then it doesn't have any variations field.
It has lots of backwards incompatible changes, I figured I'd do them all at once to limit how much anyone may need to change their apps/sites going forward.
Feel free to take a look at the changelog to see more details. Just wanted to post a warning here about the changes.
Aye, you are correct, it is indeed unique and the SHA1 card 'id' is made up of (setCode + cardName + cardImageName). It's mentioned in the mtgjson docs.
Not sure what this 'cardId' you are referring to. The card objects in mtgjson have an 'id' field. Did you mean that?
Because on that field, there are no duplicate id's.
The mtgjson code checks for this every time the JSON files are generated and I also just tested them again just to be sure. The point of this field is to be unique and it is indeed, unique.
Yes, it is technically possible for there to be a collision with a SHA1 hash. However is is EXTREMELY unlikely. See: http://crypto.stackexchange.com/questions/2583/is-it-fair-to-assume-that-sha1-collisions-wont-occur-on-a-set-of-100k-strings
That URL talks about how hashing 100,000 strings and even over 1 million strings, produced no collisions. It goes on to talk about really big numbers before generating any collisions in theory. There are only 29,416 cards in the MTGJSON files. Even with 100 more years of MTG cards, it is unlikely to have a collision, but as I said, this is checked automatically when the JSON is generated to ensure there are no collisions, just in case.
AllCards takes data from multiple sets and creates a single card entry. So anything that is set specific (like multiverseid) is excluded.
Best to use AllSets if you need the multiverseid.
I am open to someone else taking over the site, domain and code.
Do not fear, I am not planning on shutting it down, and I will continue to update it for the time being. However, if someone out there is up for taking it over who would continue to update it and run the site, I am open to that. Feel free to come forward and discuss here on the forum or you can e-mail me robert@cosmicrealms.com
Whoops, sorry about that. I've added this issue https://github.com/Sembiance/mtgjson/issues/78 to github and will work on getting it added back in.
It should be easy enough for anyone who is importing the JSON data into their app/webpage to simply do what I was doing before 3.0.0 which is to copy the foreignNames found on any card in any set to every other instance of that card in all sets.
Ahh yes, this is messed up on Gatherer too, which is where the non-english language data comes from. It is also messed up on magiccards.info
The presence of 'Chinese Traditional' on Gatherer for Shadowmoor may signal that it was indeed printed in that language, but they just don't have the data for it. On the other hand it may be a gatherer bug entirely and maybe it wasn't printed in that language. Due to this uncertainty I am hesitant to remove it from MTGJSON since right now the presence of those fields denotes whether or not it was printed in those languages. Tricky situation
Someone asked this on Stack Overflow: http://stackoverflow.com/questions/18327093/converting-json-to-a-mysql-table-how-should-the-table-structure-look-like
Someone on GitHub has already done this: https://github.com/laxika/MTGJsonConverter
However the repo seems to be gone, you may want to contact them to see if they would provide you the code.
Someone else also seems to have done this on github: https://github.com/six7zero9/whoo/tree/master/dbloader
Having it here is just fine with me
That's some good work there
I don't want to introduce numbers onto cards that didn't have them originally. I know that some of the promo sets I import from magiccards.info have numbers but they are incorrect or invalid (see https://github.com/Sembiance/mtgjson/issues/43)
To 'number' the cards yourself on sets that don't have numbers, you just need to sort them a specific way.
Sort order: Blue, Black, Red, Green, White, Multicolor, Artifact, Special Land, Basic Lands
Within each category, sort by name.
The clash pack cards are in a single set, 'Clash Pack' (CPK). This comes from magiccards.info and they have not yet updated to include the Magic Origins cards. They don't update very often, but I'm not aware of a more reliable and up to date source that has full card info. http://www.magiclibrarities.net/859-rarities-clash-pack-box-set-cards-english-cards-index.html has lists of the cards, but not the details of each card, so parsing that for the full card data isn't an option.
The variations field is present when there are multiple cards with the same name in the same set. Because From the Vault: Angels doesn't have any duplicate cards (different art usually) then it doesn't have any variations field.
It has lots of backwards incompatible changes, I figured I'd do them all at once to limit how much anyone may need to change their apps/sites going forward.
Feel free to take a look at the changelog to see more details. Just wanted to post a warning here about the changes.