Here is my software "Gatherer Extractor", which can easily be downloaded at the following URL:
------------------------------ Version: 5.0f
Release Date: 11/02/2018
The software helps you to:
- Get a full or partial MtG Database. The software gathers information from gatherer.wizards.com and magiccards.info.
The card database can be saved as CSV or XML. I could see that many people are looking for MtG Database for MTG online shops, personal websites or applications. Here is a good tool to get it!
- Get a MWS Text Spoiler for home-made sets, as long as the original CSV file is correctly formated for Gatherer Extractor
- Get card scans in the language of your choice, and in the format of your choice (original, fullscan, cropped), and save them on your computer.
The source website can be chosen by the users, so that are accessible low quality scans from the Gatherer, and semi-HQ scans from magiccards.info (English only)
- Compile a Spoiler List text file for MagicWorkstation. Very useful to get a clean and bug-free spoiler text with correct formating.
- OS : Windows XP, Vista, Seven, 8, 10.
- Microsoft Framework .NET 4 If you don't have it installed on your computer, please download it for free at: http://www.microsoft.com/download/en/details.aspx?id=17718 - Screen resolution at least 1024x768
- Execution as Administrator for Windows Vista and Windows 7/8/10.
- Microsoft Office Web Components 11 (OWC11). This is mandatory if you want exportation to xlsx format to work properly. If you don't have it installed on your computer, please download it for free at: http://www.microsoft.com/download/en/details.aspx?id=22276
For those interested in, it is possible to add others supported languages. Translate directly EN.lang into a target foreign language and add the new .lang file into the "/data/language" folder!
Please let me know if you translate the software in another foreign language, so that I can add your translation to the further versions!
Enjoy, and feel free to report bugs, post comment, and to suggest improvement!
To be done :
- JSON exportation
- Correction for foreign extraction (linebreaks are missing)
Perhaps some sort of custom filters could be added? For example, the Color column will export as G, R, W, etc. Having a filter that says something like "convert all instances of G in Color to Green" would be useful.
There was not even thinking about UTF-8 ... edited in the "textpad"! Thanks for the correction!
I'm planning to put this program on my compilation "MWS EasyInstall" but I have a doubt. Would like all the program settings (folder options / selected and all the rest) be loaded from an external file for the program? (file ".ini" for example)
2/ Gatherer Extractor provides internal default settings. Nevertheless, you'd better add the .../Gatherer Extractor/data/Abreviations.cfg file. This file contains all information about MtG sets. Including it will prevent the user from keyboarding all set codes (internal and from MagicCards.info) and all released dates.
Note that when an additional set is available (from gatherer or magiccards.info), the program send you an alert to invite you to keyboard the corresponding codes.
3/ You may include Gatherer Extractor to you MWS EasyInstall project, but you'd better include the Gatherer Extractor Installer.exe to ensure a bug-free iinstallation
4/ The language you choose in the "General" Option Tab is for the displayed language for the software. The language you choose in the "Scan" Option Tab is the language used for scans. The replacement language is always English, when a scan is required in another foreign language but not available from gatherer or magiccards.info
Unfortunately, the magiccards.info & gatherer database are very incomplete. Only the foreign card name are available. I'm currently working on finding a way to extract from other sources an additional database in a given foreign language (French, German or Portuguese for example)
I opened your installation file so I had access to all command lines that you used for installation (inno setup).
This way I can compile into a single installer, without having any bug.
My idea is to make your program in the root folder MWS, so that the folder "pics" of his program and the MWS would be the same.
It also includes a shortcut to your start menu program, such as "extra " MWS. Obviously, your program will integrate the file of "special thanks" along with a link to this forum.
If you want some more information to be included (maybe email) send me a private message.
When to my questions, I understood everything perfectly! Once you have new updates let me know so I update the "extra " in EasyInstall.
I'm expecting just another confirmation (another feature of MWS) to upgrade to version 1.2 EasyInstall that already include your program
The acronyms of the issues change much from program to program, or even from site to site.
Would put a "filter " that took a letter "X"of the site and time to save save the hard drive as the letter "Y" (current filter)?
From the Vault: Exiled
In Gatherer = EXL;
In Magic Cards Info = FVE;
In Filter (and therefore what will be generated regardless of the source in HD) = 123
In fact, you can edit the code file (Menu : Edit > Edit Set Code) as you wish. The first column contains the full name of sets. The second column is not the set codes for the gatherer, but contains the codes you want! The third is the official magiccards.info codes.
The program doesn't need codes to download data and scans from the gatherer, as the ID is the only data required to get cards. Set codes (magiccards.info column) are however required for magiccards.info as cards can be retrieved on the website only with their set codes and card #.
The name of the folder created to store card scans is the code you decide in the 2nd column of the code editor.
It should be enough for a customizable definition of sets, or you still want a "filter" ?
Unfortunately, the magiccards.info & gatherer database are very incomplete. Only the foreign card name are available.
What do you mean? They are incomplete set wise, but for any card that you can get a foreign name, there is also the full text in Gatherer. It's very easy to miss, but if you set the 'printed' var to 'true' you get printed text, which for foreign cards mean native (translated) text.
So, you append the foreign card's ID (as you now, they have unique multiverse IDs) to this URL:
I know about foreign printed text. But see for example old cards, like Vindicate in French. You got this : http://gatherer.wizards.com/Pages/Card/Details.aspx?multiverseid=179210
Only the flavor text and name is translated. not the type and ability. This is in that way I say Gatherer is very incomplete for foreign cards. You may notice portuguese and spanish translation is also lacking form the Gatherer. The situation is even worst for set like Tempest or Urza, in which cards are available only in English despite many foreign issue (at least German, Italian, French, Spanish, Portuguese, Japanese, Chinese and Korean).
The situation is much better for more recent cards, as you mentioned.
The program gets access to foreign scans via foreign ID. I can add columns to store the foreign ID. But there is more work to store a complete database in a foreign language.
I get the foreign name from magiccards.info, whose foreign database is complete for foreign name. But many foreign types and abilities are lacking...
I'm not doubting you, but I see french Vindicate fine? Click on "Printed" in your link of just add '&printed=true'. What am I missing?
Maybe some cookie or wotc.com Language issue? You know, I spent four+ months thinking gatherer was down and I just needed to change Language to English when searching english card names (I'm spanish). Or something, I can't remember. Gatherer is so annoying sometimes...
Yeah, the new gatherer gave me headaches too about language cookies...
However, the fact that some language are missing (Spanish and Portuguese for Apocalypse for instance), and that some sets are not translated (Tempest block for example) is very redhibitory for an accurate extraction of a foreign database from the Gatherer. That's the main problem.
I noticed a little bug in spoiler text extraction from Duel Decks editions : some token cards are included in the Gatherer spoiler text, but should not be.
I added a token filter in the v2.2 to avoid tokens in extracted database.
— First, may you open the /data/Abreviations.cfg file with Notepad. This file is UTF-8 encoded. Can you see all the sets registered in the file with lines like "Arabian Nights // AN // AN // 12/1993" ?
Normally, that file should be OK.
— Is the connectivity to the gatherer and to magiccards.info OK?
I'm pretty sure you don't have connectivity. If not, 3 possibilites:
1/ Can you access to the gatherer or magiccards.info from your web browser? If not, you may have a problem with your Internet connexion on your computeur.
2/ Please check your firewall settings and make sure you allowed Gatherer Extractor to access the Internet.
3/ If you still have no connectivity to the gatherer or magiccards.info with Gatherer Extractor, you may have a malware or trojan running on your computer which prevent svhost.exe from authorizing applications you run to access the Internet (such as your firewall/antivirus upgrades etc.) Install Malwarebytes' anti-malware, run it and clean your computer from malwares.
Everything is installed fine and I have connectivity.
The file you mentioned abbreviations.cfg is completely empty!
Well it only has the following line:
" // // // "
There is nothing else in the file at all.
You probably tried to edit and save the Abreviations file from the "Edit" menu and you had no permission to write the file, resulting in a empty file.
1/ Please reinstall the program: it will overwrite the current file so that you'll get a safe Abreviations.cfg file.
2/ Make sure you run the program as administrator, so that you will have all permissions to write files.
Note that when new sets are available, you'll get an alert almost like the one you have to invite you to complete the abreviations.cfg thanks to the editor provided in the "Edit" menu.
In your case, you told me the Set listbox was empty. It was only disable which is not at all the same, for bug finding and fixing