A long time ago, I decided we shouldn't worry about Playtesting and Development until we had at least 145 cards—in other words, until there were only 100 cards left to design. But I hear a lot of push for starting the process now. At the very least, we need a thread to discuss how we're going to go about it.

This is that thread. How are we going to playtest the set? How should we change existing cards?

I'll work on making a Visual Spoiler (sans art) like TPzombieW suggested, to help people digest the set so far more easily. But for now, here's the set map:

My dirty little secret is that I've never playtested and don't really know how to go about it. Magic Workstation should work though I think.

I think maintaining a Cockatrice plugin is the best option if you want to playtest. It works across all platforms and releasing a new update is as easy as posting up an XML file.

Well, if we want to playtest, we need to figure out what formats we're balancing it for. I'm going to assume this is balanced for limited and block constructed, so the way we test for them is by playing draft/sealed and building/testing block constructed decks. If it becomes apparent that one common is insane in draft, or that one deck is absurd in block constructed, we make changes.

I've made some kind of frankenstein file here. This is a replacement for "cards.xml". This file includes only starstill cards. It is intended to be placed in C:\Starstill\StarstillDev (7-23-14) which is a folder that would then be full of card images, but the latest version of cockatrice doesn't support custom card images yet (that i can figure). Since the card images don't work, put the file wherever is convenient for you. You will have to change your card-playing-program settings to point to the new xml file, at which point your program will crash, but if you restart it you will have a list of cards with text rules descriptions.

I fixed most of the power/toughness omissions (but still missed some).

The set abbreviation i used (SSI) is intended to be startstill+roman numeral so later versions will have different set abbreviations.

EDIT: I have not found a good resource for actually drafting cards. CCGDecks is the recommended resource, but will first require the mindbreaking task of manually entering every custom card. If no one has a better solution, I will undertake this mindbreaking task on Saturday.

EDIT: I am also trying and failing to have MSE export to lackey. Lackey could be used to draft and play, and would be easy to update with MSE exports.

EDIT: I have taken down the link in order to pursue using a different program

Note that you must uncheck "automatically download card images," at least on the most recent Mac build, otherwise Cockatrice will replace all the card images with Magic card backs.

An update; I am really liking LackeyCCG; it makes packs, you can draft in it; you can play in it, it supports images; Just have to figure out how to easily export cards from MSE to its format without manually entering the text. So this may be a program to check out if you wish to playtest at some future date. I am trying to locate/reverse engineer an mse-to-lackey export template.

P.S. If you have a mse-to-lackey export template (even one that doesn't work) please pm me.

P.S.S. Feel free to beat me to an alternate solution; some frameworks for playing/drafting to which to is easy to export the many iterations of our set.

I have modified the mtg plugin for LackeyCCG to be smaller and use different size images and pre-configured it for some starstill files and made some of the defaults draft-friendly.
I have created an MSE plugin to export files into Lackey-readable files.
Instructions are in the readme. Download the entire folder "magiccustomsets."

The plugin contains some of the set we have already made, but with no images. Follow readme instructions if you want images.
You may have problems if you try to play with someone who has a different set data file than you;
Both parties must start with the same MSE file; and output it with the same Set Code. Then make images if you want them.
I have not actually tested drafting or playing with another person online, although it appears to work on one end.
If you are interesting further testing proof-of-principle with me, install this and then pm me to set up a time.

Cool. I might steal this setup for the eventual NGA Constructed plugin.

Yeah, tech expertise for the win!

Whatever happened to stopping for development?

@storyteller: Playtesting is part of development. Playtesting helps identify issues.

I mean, weren't we supposed to pause card creation and go into development for a while?

No, I think the plan was always to get a good chunk of the set done, then start playtesting and developing as we finish up design. That said it might be a stretch to say we had a "plan".

Well, I suggested we pause for the first round of playtesting around when we got enough cards for a small set (~140), which we already have.
I really think we should.

What are we planning to develop? We can't playtest Limited because the commons are still majorly unfinished. We can't playtest Constructed because half the rares and mythics aren't done. I think there are just too many unknowns to start right now.

We might not be able to playtest but we can definitely develop anyway. Stuff like fixing the old cards to merge the instant/sorcery themes, or changing a couple cards to fix the incredibly low curve, or adding flying to some guys, or buffing some underpowered stuff.

Of course we need playtesting to figure out if cards might be too good, or balance the different limited archetypes against each other, or see if mastery is too easy or too hard or too unbreakable or something in actual gameplay. But there's nothing keeping us from, like, ditching Ethereal Form or nerfing Gravefall.

Or removing some cantrips/card draw, if you guys really don't want to support laboratory maniac for daft.

I don't think that we are? Every set has cantrips and card draw, and we don't even have that much that I can think of. Blue's got one cantrip and one draw-two at common. That's about what Theros and Gatecrash had, and less than Return to Ravnica.

We don't need a finished product to benefit from playtesting. There are a lot of unknowns we can answer by testing the cards. If the decks area little wonky it won't stop us from drawing conclusions like "archetype X needs more love" or "Y isn't very fun and needs something to spice it up". It will also let us see what sort of strategies have already emerged naturally and tell us what to focus on.