@ user-18041209 : Added to the updated 6.5 release, together with Salvat 2011.
@ swallowtail23 : Indeed, it was missing. It is now added to the RLS set.
I propose a new release v.6.5 with up-to-date parameter files (set-code list, booster packs, legalities etc.).
In the AddOn_scryfall_promos.cfg file, you can find the corresponding Scryfall setcodes gathered in the 'Release Promos' set:
Release Promos // PTHB,PSLD,PELD,PM20,PMH1,PWAR,PPP1,PRNA,PGRN,PM19,PBBD,PDOM,PRIX,PUST,PXLN,PHOU,PAKH,PAER,PKLD,PEMN,PSOI,POGW,PBFZ,PORI,PDTK,PFRF,PKTK,PM15,PJOU,PBNG,PTHS,PM14,PDGM,PGTC,PRTR,PM13,PAVR,PDKA,PISD,PM12,PNPH,PMBS,PSOM,PM11,PROE,PWWK,PZEN,PM10,PLPA,P10E,PPRE,PREL
I used your program to make a data base for my own collection. I used almost all information exporting data via XLS. Now im trying to implement a API to check the prices that change in a period of time. Using Scyfall database, they don´t use the ID the gatherer show to me.
EX: ID-28857(gatherer) card name:Accelerate ed: TO
if I try in scryfall it will return ID:59e2d0ea-fec9-5bad-a8ac-46da0a0a1326 card name:Accelerate ed: TO
How can I convert the ID from gatherer extractor to Id from Scryfall?
Love the application. I've been looking for something like this for ages, so thank you!
However, when I export the data to SQL scripts, the syntax within those scripts isn't quite right.
For example, the 'Nsets' table is created with five fields, but the insert script contains 20 fields, so they fail.
Also, there are a number of cards that contain an apostrophe, which affects the syntax, which also causes the script to fail.
I would love to see these changes implemented in the next release.
I work with SQL on a daily basis, so if there's anything I can do to help with these improvements, please let me know.
@ Kendeaux: You're absolutely right. The boosterpack section is not well implemented in NSets. I will fix that in the next release.
Normally, cards with apostrophes in name/flavor/ability are supposed to be well managed by replacing ' by '' during the sql formatting. Do you have an example showing the error?
When trying to export price data for cards to an Excel spreadsheet, I've noticed that the program only exports 16,888 cards instead of the whole database. There seems to be whole sets missing from the export such as M11, despite the fact that Gather had downloaded pricing info for the set. Do I have a setting wrong?
Any help would be greatly appreciated.
Thank you!
P.S. I love the program. Thank you so much for creating it.
@ Airon: thanks for the report. The issue concerns all the split cards and equivalent (adventure etc.). It has been fixed. All data will be correctly extracted in the next release, incl. ability text etc.
@ BeardedGamer9000: I cannot reproduce the issue. From a complete database with default settings, I can create a 320MB excel file with all the cards. Do you have specific settings for Excel export ?
You're right, it does seem to be fixed. If I uncheck 'Booster Packaging' in Options > Export, I only get Nname, Ncode, Ndate, and Nis_promo.
With that optioned checked, I get all of those other fields.
I'll give the new script a spin and see how it goes.
If I run into any problems, or if I think of any other ideas, I'll be sure to let you know.
But, I have a question about the latest Ikoria set.
I can currently find the normal Ikoria in the setlist, together with the Commander variant.
Any idea where the collector version is (just like previous sets)?
I see the godzilla style cards are part of the original set list (with different variant number), but can't seem to find the other art styles from the collector boosters...
Hi @Chaudakh , thanks a lot for your work on this project.
Unfortunately, i'm on Linux Mint and i tried to run Gatherer Extractor 6.6 with Wine but its not working : (paste bin : https://textbin.net/3gEu8pK6r8 )
then, i also tried to run the ruby file, but its not worked (i'm not a programmer so i have no idea why this didnt work...)
@Chaudakh: Thx for reply. I downloaded the latest version, but when I download the info for CIKO, I just get a list of 7 cards... any idea what is going wrong? I'm mostly searching for a list with alt art.
@ JeVuY : Thanks for the feedback. They changed something on Scryfall with the "ispromo" tag. So I changed the filters for Collector Booster editions to "not isbooster". You should recover all the cards in Collector Booster editions (CIKO, etc.).
For those interested, sets extracted from Scryfall are listed in /data/AddOn_scryfall_promos.cfg. For each set, you find either a list of Scryfall set codes (in case the set is a collection of small Scryfall sets, like FNM etc.) or a filter, as for CIKO etc.
Please have a look at the Gatherer Extractor new release. The version number is unchanged as I did not modify the program itself. I just updated the AddOn_scryfall_promos.cfg file.
@ Hennel : Unfortunately, I don't have access to a Linux installation, so I cannot guarantee Gatherer Extrator smoothly operates on virtual Windows environments. Gatherer Extractor has been developed for a genuine Windows environment, as it massively relies on the Microsoft .NET framework. Gatherer Extractor is currently developed with Virtual Studio 2019.
However, your problem might come from :
the .NET framework (v4.5) in Wine. Are you sure the framework is well installed and well operates in the Wine environment?
What about the Office Web Component 11 (used to generate outputs in Excel format). Is it correctly installed in the Wine environment?
Hi. Great tool but how can i sort the sets by date? Everytime a new set appears i have to go look at the abbreviation of the set instead i would just like to filter by date so i can download the most recent ones.
@ Adrian: great suggestion. It is not possible at the moment by I can implement it in a next release. However, new sets usually don't have set codes attached to them, and they are highlighted in red in the set list.
I updated GEx. v6.7 is now released.
I added several features :
Sort set to proceed by name, set code or release date
New/updated cards in the extracted-card list are now highlithted (customizable)
Reworked option panel, with the addition of a set panel so as to customize the promo sets from Scryfall. I used to edit manually a related txt file. Now it is editable directly in the software.
Better organized scan panel
Fixed minor bugs. the set list to proceed is no more blinking
NB: I noticed Gatherer offers now regular and promo cards for M21. In order to avoid a mix between M21 and CM21, it is recommended to first extract the main data of the two sets, then extract all data. If you do so, the M21 altered cards (in fact CM21 cards) that are found from the gatherer when parsing the M21 cards will not be changed to M21. They will be updated when parsing the CM21 cards as normal.
@Chaudakh - This is a great tool. I am running across an error though.
I'm on v6.7, and when I click the button to count reprints and detect the original, I'm getting the error below. I am trying to process the full list, not a partial. Is there something I'm missing on my end, or is this truly a program issue? Thanks!
See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.
************** Exception Text **************
System.IndexOutOfRangeException: There is no row at position -1.
at System.Data.RBTree`1.GetNodeByIndex(Int32 userIndex)
at System.Data.DataRowCollection.get_Item(Int32 index)
at Gatherer_Extractor.FenetrePrincipale.Treatment_MAJ(Int32 nbre_tache, BackgroundWorker worker, DoWorkEventArgs e)
at Gatherer_Extractor.FenetrePrincipale.BWkMAJ_DoWork(Object sender, DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)
************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.
When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.
@ swallowtail23 : Indeed, it was missing. It is now added to the RLS set.
I propose a new release v.6.5 with up-to-date parameter files (set-code list, booster packs, legalities etc.).
In the AddOn_scryfall_promos.cfg file, you can find the corresponding Scryfall setcodes gathered in the 'Release Promos' set:
Release Promos // PTHB,PSLD,PELD,PM20,PMH1,PWAR,PPP1,PRNA,PGRN,PM19,PBBD,PDOM,PRIX,PUST,PXLN,PHOU,PAKH,PAER,PKLD,PEMN,PSOI,POGW,PBFZ,PORI,PDTK,PFRF,PKTK,PM15,PJOU,PBNG,PTHS,PM14,PDGM,PGTC,PRTR,PM13,PAVR,PDKA,PISD,PM12,PNPH,PMBS,PSOM,PM11,PROE,PWWK,PZEN,PM10,PLPA,P10E,PPRE,PREL
I used your program to make a data base for my own collection. I used almost all information exporting data via XLS. Now im trying to implement a API to check the prices that change in a period of time. Using Scyfall database, they don´t use the ID the gatherer show to me.
EX: ID-28857(gatherer) card name:Accelerate ed: TO
if I try in scryfall it will return ID:59e2d0ea-fec9-5bad-a8ac-46da0a0a1326 card name:Accelerate ed: TO
How can I convert the ID from gatherer extractor to Id from Scryfall?
thank you
However, when I export the data to SQL scripts, the syntax within those scripts isn't quite right.
For example, the 'Nsets' table is created with five fields, but the insert script contains 20 fields, so they fail.
Also, there are a number of cards that contain an apostrophe, which affects the syntax, which also causes the script to fail.
I would love to see these changes implemented in the next release.
I work with SQL on a daily basis, so if there's anything I can do to help with these improvements, please let me know.
Normally, cards with apostrophes in name/flavor/ability are supposed to be well managed by replacing ' by '' during the sql formatting. Do you have an example showing the error?
@ Accelerate [TO] multiverseID is 29857. To get all data from Scryfall API, get the JSON output of https://api.scryfall.com/cards/multiverse/29857
Then just read the properties you want.
http://prntscr.com/r5fy5c
When trying to export price data for cards to an Excel spreadsheet, I've noticed that the program only exports 16,888 cards instead of the whole database. There seems to be whole sets missing from the export such as M11, despite the fact that Gather had downloaded pricing info for the set. Do I have a setting wrong?
Any help would be greatly appreciated.
Thank you!
P.S. I love the program. Thank you so much for creating it.
@ BeardedGamer9000: I cannot reproduce the issue. From a complete database with default settings, I can create a 320MB excel file with all the cards. Do you have specific settings for Excel export ?
Sorry about the late reply. The current world situation ate into my computer time.
I am using specific settings. Under options and then export, I set the export settings for Excel to only export ID, Name, Set, and Price (USA)
When I do this I only get an export containing about 16,000 cards. An example of sets that it is not exporting are M11.
Thanks again for your help
@Kendeaux: did you use the last release ? This bug has been recently fixed. The error is raised because you have:
Nboosterpack text,
instead of :
Nboosterpack_nM text,
Nboosterpack_nR text,
Nboosterpack_nU text,
Nboosterpack_nC text,
Nboosterpack_nE text,
Nboosterpack_pM text,
Nboosterpack_pR text,
Nboosterpack_typeExtra1 text,
Nboosterpack_typeExtra2 text,
Nboosterpack_listExtra1 text,
Nboosterpack_listExtra2 text,
Nboosterpack_has_foil text,
Nboosterpack_pF text,
Nboosterpack_hasmasterpiece text,
Nboosterpack_pPM text,
Nboosterpack_listPMid text
With that optioned checked, I get all of those other fields.
I'll give the new script a spin and see how it goes.
If I run into any problems, or if I think of any other ideas, I'll be sure to let you know.
Thanks
-KD
Nice software !!! Great to create your own DB.
But, I have a question about the latest Ikoria set.
I can currently find the normal Ikoria in the setlist, together with the Commander variant.
Any idea where the collector version is (just like previous sets)?
I see the godzilla style cards are part of the original set list (with different variant number), but can't seem to find the other art styles from the collector boosters...
Thx in advance
For anyone interested, Its here: https://www.mtgsalvation.com/forums/magic-fundamentals/other-magic-products/third-party-products/817234-mws-functional-masterbase-up-to-ikoria-6-8-gb
Unfortunately, i'm on Linux Mint and i tried to run Gatherer Extractor 6.6 with Wine but its not working : (paste bin : https://textbin.net/3gEu8pK6r8 )
then, i also tried to run the ruby file, but its not worked (i'm not a programmer so i have no idea why this didnt work...)
Do you have a special tricks to fix that ?
Thank a lot for your time and for your answer.
Thx in advance
Jevuy
For those interested, sets extracted from Scryfall are listed in /data/AddOn_scryfall_promos.cfg. For each set, you find either a list of Scryfall set codes (in case the set is a collection of small Scryfall sets, like FNM etc.) or a filter, as for CIKO etc.
Please have a look at the Gatherer Extractor new release. The version number is unchanged as I did not modify the program itself. I just updated the AddOn_scryfall_promos.cfg file.
@ Hennel : Unfortunately, I don't have access to a Linux installation, so I cannot guarantee Gatherer Extrator smoothly operates on virtual Windows environments. Gatherer Extractor has been developed for a genuine Windows environment, as it massively relies on the Microsoft .NET framework. Gatherer Extractor is currently developed with Virtual Studio 2019.
However, your problem might come from :
I added several features :
I'm on v6.7, and when I click the button to count reprints and detect the original, I'm getting the error below. I am trying to process the full list, not a partial. Is there something I'm missing on my end, or is this truly a program issue? Thanks!
just-in-time (JIT) debugging instead of this dialog box.
************** Exception Text **************
System.IndexOutOfRangeException: There is no row at position -1.
at System.Data.RBTree`1.GetNodeByIndex(Int32 userIndex)
at System.Data.DataRowCollection.get_Item(Int32 index)
at Gatherer_Extractor.FenetrePrincipale.Treatment_MAJ(Int32 nbre_tache, BackgroundWorker worker, DoWorkEventArgs e)
at Gatherer_Extractor.FenetrePrincipale.BWkMAJ_DoWork(Object sender, DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.OnDoWork(DoWorkEventArgs e)
at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)
************** Loaded Assemblies **************
mscorlib
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4200.0 built by: NET48REL1LAST_C
CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll
----------------------------------------
Gatherer Extractor
Assembly Version: 3.8.0.0
Win32 Version: 3.8.0.0
CodeBase: file:///C:/Program%20Files%20(x86)/Gatherer%20Extractor/Gatherer%20Extractor.exe
----------------------------------------
Microsoft.VisualBasic
Assembly Version: 10.0.0.0
Win32 Version: 14.8.3752.0 built by: NET48REL1
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/Microsoft.VisualBasic/v4.0_10.0.0.0__b03f5f7f11d50a3a/Microsoft.VisualBasic.dll
----------------------------------------
System
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4200.0 built by: NET48REL1LAST_C
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Core
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4200.0 built by: NET48REL1LAST_C
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
----------------------------------------
System.Windows.Forms
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4200.0 built by: NET48REL1LAST_C
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System.Drawing
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3752.0 built by: NET48REL1
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Configuration
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4190.0 built by: NET48REL1LAST_B
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3752.0 built by: NET48REL1
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------
System.Runtime.Remoting
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3752.0 built by: NET48REL1
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Runtime.Remoting/v4.0_4.0.0.0__b77a5c561934e089/System.Runtime.Remoting.dll
----------------------------------------
System.Data
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4200.0 built by: NET48REL1LAST_C
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_32/System.Data/v4.0_4.0.0.0__b77a5c561934e089/System.Data.dll
----------------------------------------
System.Numerics
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3752.0 built by: NET48REL1
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Numerics/v4.0_4.0.0.0__b77a5c561934e089/System.Numerics.dll
----------------------------------------
Accessibility
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3752.0 built by: NET48REL1
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/Accessibility/v4.0_4.0.0.0__b03f5f7f11d50a3a/Accessibility.dll
----------------------------------------
System.Data.DataSetExtensions
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3752.0 built by: NET48REL1
CodeBase: file:///C:/WINDOWS/Microsoft.Net/assembly/GAC_MSIL/System.Data.DataSetExtensions/v4.0_4.0.0.0__b77a5c561934e089/System.Data.DataSetExtensions.dll
----------------------------------------
************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.
For example:
<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>
When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.