Login | Register


All times are UTC - 7 hours


It is currently 2019-Oct-14 11:06 pm




Post new topic Reply to topic  [ 128 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 9  Next
Author Message
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-17 4:07 pm 
User avatar

Joined: 2011-Jan-02 5:25 am
Age: Elder Dragon
Location: Costa La Haya, capital del ducado Holanda
paragon_deku wrote:
Uktabi_Kong wrote:
Primeval Titan is the most requested unban... I don't think there could be a better argument against the format being run democratically. As I've explained in a different thread, it's nearly impossible to overstate how bad that card was the format. And worst of all, it was in a subtle and seemingly innocuous way that's only obvious in hindsight.

This is a rather silly statement considering that only roughly a quarter of people want it back. That would get shot down in almost any voting system.


Unless you go with a runoff system with multiple voting rounds to determine a winner, or first past the post.

Edit: I agree with UK's point that PT showing up in a list is not a good indication. What I like of a poll like this is that it gets a prioritised list as a starting point of a discussion. By that I mean a poll is a discussion tool, not a referendum. But my issue with polling is that inevitably someone starts using the poll numbers as an argument ("the will of the people") while they are statistically irrelevant.

_________________
"Our words are backed with OBLIVION STONE!"


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-17 10:01 pm 

Joined: 2012-Mar-31 11:52 am
Age: Elder Dragon
Well I took a look at the results and it was quite predictable. Namely,

1. The community you sampled is all over the place in terms of their approach to the ban list.
2. People are trolls and anonymous data should always be taken with a grain of salt.

It is for this reason that we should be thankful the format is run by a small group of analytical thinkers that separate their own opinions from their management approach.

Interestingly, when you filter out the noise, some of your data mirrors my years of polling on MTGS. There, the largest consistencies have been that people want Gifts and Painters unbanned, and Sol Ring, Mana Crypt, and Iona banned (although I didnt see much mention of her oddly enough). I also asked direct questions regarding specific rules like infect, BaaC and tuck. The consensus there was to leave the status quo alone. I can only attribute the differences to the demographics of the sample population, since a forum full of members who regularly discuss the ban list with the RC themselves will generally be more informed than a random Facebook group.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-17 11:57 pm 
User avatar

Joined: 2006-Dec-31 12:26 pm
Age: Elder Dragon
I find it somewhat interesting that the lowest amount that people want unbanned is power + time vault.

Wonder if that's because people feel they don't need to play it, or feel they wouldn't be able to get a copy if it were legal so don't need to feel like they are underpowered if it's available, or something else.

I'm vaguely surprised at how many people feel the RC aren't managing the ban list properly (about 10% at 1-3). Makes me wonder if they want the same thing as the RC but have differing opinions on how to get there, or if the format "isn't for them" and they don't like it because they want the format to be different from what it is.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 2:39 am 

Joined: 2019-Jun-06 6:25 am
Age: Drake
cryogen wrote:
Well I took a look at the results and it was quite predictable. Namely,

1. The community you sampled is all over the place in terms of their approach to the ban list.
2. People are trolls and anonymous data should always be taken with a grain of salt.

It is for this reason that we should be thankful the format is run by a small group of analytical thinkers that separate their own opinions from their management approach.

Interestingly, when you filter out the noise, some of your data mirrors my years of polling on MTGS. There, the largest consistencies have been that people want Gifts and Painters unbanned, and Sol Ring, Mana Crypt, and Iona banned (although I didnt see much mention of her oddly enough). I also asked direct questions regarding specific rules like infect, BaaC and tuck. The consensus there was to leave the status quo alone. I can only attribute the differences to the demographics of the sample population, since a forum full of members who regularly discuss the ban list with the RC themselves will generally be more informed than a random Facebook group.

I feel like you didn't read the survey closely enough then, because this was sampling multiple different groups, including this very forum. None of which were "a random facebook group."

Carthain wrote:
I find it somewhat interesting that the lowest amount that people want unbanned is power + time vault.

Wonder if that's because people feel they don't need to play it, or feel they wouldn't be able to get a copy if it were legal so don't need to feel like they are underpowered if it's available, or something else.

I'm vaguely surprised at how many people feel the RC aren't managing the ban list properly (about 10% at 1-3). Makes me wonder if they want the same thing as the RC but have differing opinions on how to get there, or if the format "isn't for them" and they don't like it because they want the format to be different from what it is.

The most radical banlist opinion I saw shared was "everything should be fair game besides the power 9." I do wonder if that's just because of availability or if the overarching status quo of most of the non-vintage community is that those cards are a big taboo.

Overall, the mean approval of the RC and the format/banlist was a 6, so not bad considering the number of people I canvassed from different communities (and the fact the /tg/ community was involved, considering how vitriolic they are about the game)

I still considered some of the "troll" response though since they gave legitimate data besides the free response section, and honestly if I didn't want anything banned I might jokingly put something like "island" as well for a laugh.

_________________
ImageImage
ImageImage


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 2:50 am 

Joined: 2012-Mar-31 11:52 am
Age: Elder Dragon
A couple of Discord servers, Reddit, and local players. The point still stands, because it wasn't aiming to belittle your sample population, but to highlight the differences in our two data sets. Collectively it is all data, and the more data the RC has access to, the better informed their decisions can be.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 3:06 am 
User avatar

Joined: 2006-Dec-31 12:26 pm
Age: Elder Dragon
paragon_deku wrote:
I still considered some of the "troll" response though since they gave legitimate data besides the free response section, and honestly if I didn't want anything banned I might jokingly put something like "island" as well for a laugh.

That seems... unwise.

Take a look at political voting -- if you do anything to screw up the ballot, they discard the whole thing. There's a reason for it. If you're not taking part of it seriously, how can we be sure that the rest of the answers you put were serious?

I'm assuming you're trying to collect data to use as some sort of motivation/justification for changes. How can that be representative of what people want when you are intentionally including in some data from people who aren't taking it seriously. How do you know that the rest of the answers from that person are serious or not?

If people find fault with your methods, then the results you get are tainted as well. And when trying to convince people of something, you want your support to be as untainted as possible.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 3:41 am 

Joined: 2019-Jun-06 6:25 am
Age: Drake
Carthain wrote:
paragon_deku wrote:
I still considered some of the "troll" response though since they gave legitimate data besides the free response section, and honestly if I didn't want anything banned I might jokingly put something like "island" as well for a laugh.

That seems... unwise.

Take a look at political voting -- if you do anything to screw up the ballot, they discard the whole thing. There's a reason for it. If you're not taking part of it seriously, how can we be sure that the rest of the answers you put were serious?

I'm assuming you're trying to collect data to use as some sort of motivation/justification for changes. How can that be representative of what people want when you are intentionally including in some data from people who aren't taking it seriously. How do you know that the rest of the answers from that person are serious or not?

If people find fault with your methods, then the results you get are tainted as well. And when trying to convince people of something, you want your support to be as untainted as possible.

It would be more unwise to omit their responses considering that they were largely genuine. The only way for me to tell that a response is complete trolling is if they put a 1 for everything or vote to unban everything, but even then they could genuinely hold those beliefs.

_________________
ImageImage
ImageImage


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 3:44 am 
User avatar

Joined: 2016-Nov-27 2:39 pm
Age: Elder Dragon
Yeah I'd agree with discarding the troll response.

If I wanted to damage your survey the #1 thing I'd do is just give it bad data (check everything, check random stuff maybe) that does not actually represent me or anybody. That messes with your stats and makes those stats count for less. It is very normal to expunge bad statistical inputs for analysis and when someone is clearly responding in bad faith that's a bad statistical input. Expunge it.

(Were they genuine, or were they just checking random boxes?)

_________________
Decks: Chaos colored dragons, Mathas, the Instigator (politics and mayhem).
Beloved precons: Atraxa, Praetors' Voice; Saskia the Unyielding; Freyalise, Llanowar's Fury.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 3:48 am 
User avatar

Joined: 2006-Dec-31 12:26 pm
Age: Elder Dragon
paragon_deku wrote:
It would be more unwise to omit their responses considering that they were largely genuine.

But there is some data that is obviously never going to happen. Such as your example of listing 'Island' to ban. If something like that comes up, then you should toss out all results related to that submission.

I fully agree it can be hard to tell which ones may be trolling, and which may be just a rarely held opinion -- but sometimes, you can tell. And you should be sure to sanitize your data.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 5:06 am 

Joined: 2019-Jun-06 6:25 am
Age: Drake
Carthain wrote:
paragon_deku wrote:
It would be more unwise to omit their responses considering that they were largely genuine.

But there is some data that is obviously never going to happen. Such as your example of listing 'Island' to ban. If something like that comes up, then you should toss out all results related to that submission.

I fully agree it can be hard to tell which ones may be trolling, and which may be just a rarely held opinion -- but sometimes, you can tell. And you should be sure to sanitize your data.

I think we just had a miscommunication. I'm not going to count troll responses to the free response section, but I will still include the other information they gave since it shows an actual pattern to it and not just random selection (dissatisfied with the banlist AND asking for lots of unbans, satisfied with the banlist AND putting "none of the above"). But yeah, not gonna represent "two people wanted island banned" in the data proper.

_________________
ImageImage
ImageImage


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 5:15 am 

Joined: 2012-Mar-31 11:52 am
Age: Elder Dragon
But the point is that you should throw out the entire survey answer if you want to remain objective. Once a user has demonstrated that they don't take the survey seriously, younger imparting your own opinion by choosing to accept certain data but not other. But your poll, your decision.

The only suggestion I have, if you are trying to give the RC some data, is to give them the whole unfiltered data. That way they can take from it what they want. They may even figure out who "sheldor" is.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 5:21 am 
User avatar

Joined: 2006-Dec-31 12:26 pm
Age: Elder Dragon
cryogen wrote:
But the point is that you should throw out the entire survey answer if you want to remain objective. Once a user has demonstrated that they don't take the survey seriously, younger imparting your own opinion by choosing to accept certain data but not other.

This. This is what I'm saying.

But if instead you're picking and choosing which data to include (even potentially tainted responses) then you're not trying to sanitize your data properly. It looks a LOT more like you are just trying to pick & choose which data to look at and consider -- which means your results are likely skewed in some way which means your results are questionable at best.

Giving a link to the spreadsheet of results that it generated (as read-only of course) would be pretty awesome.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 5:32 am 
User avatar

Joined: 2012-Feb-07 4:15 pm
Age: Elder Dragon
paragon_deku wrote:
It would be more unwise to omit their responses considering that they were largely genuine.

I would like an explanation of how on earth you came to the bolded portion. Especially since these polls are largely anonymous and there are zero repercussions for trying to mess with the data. If someone's going to put an obvious joke or insult in the comments section, you have no way of knowing if anything else they said is a joke.

Since I don't have access to the data of which responses line up with which, we'll use a hypothetical. Imagine someone said that they want Erayo, Painter's Servant, and Primeval Titan unbanned, while also saying they wanted Islands banned. Based on that information alone, we are pretty certain that the banning Islands portion is a joke, and based on that information we are absolutely sure that they didn't take the poll entirely seriously. That fact calls into question whether they took any of it seriously.

They said "ban Islands" for a reason unrelated to whether or not it would improve the format, so there's reason to doubt the reasoning behind others as well. Maybe they want Painter's Servant unbanned because they have 30 of them and want the price to go up. Maybe they picked Erayo because they thought it would be funny to pick what they thought was the most banworthy card. Maybe they even picked Erayo because in their troll logic, Islands being banned means Erayo is no longer a problem.

The entire point of trying to gain data is to have some sort of factual basis for an argument or decision. If it's a known fact that some of the data is corrupted, then using that data for the final conclusion corrupts said conclusion.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 7:19 am 

Joined: 2012-Apr-11 7:17 am
Age: Elder Dragon
I did find it interesting OP skipped over how highly the RC rated on the poll, while previous to it they had said many people were telling him how bad a job they had been doing.

_________________
sir squab wrote:
My... history of buying Magic cards is probably a tapestry of bad financial decisions >_>
niheloim wrote:
No, I think he's right. I'm just all butt-hurt over prophet.


Top
 Offline Profile  
 
 Post subject: Re: A General EDH Survey
AgePosted: 2019-Jun-18 9:32 am 

Joined: 2014-Jul-26 11:35 am
Age: Elder Dragon
I mentioned it briefly in my last post, but I'm going to put it at the top here. I am a maths teacher at a high school and I teach units on statistics and surveying. Now that doesn't make me an expert but I do speak with some moderate level of expertise when I say you really should not pick parts of responses to include, no matter how well intentioned you may be in that.

Unconscious bias is a significant factor in any survey and even if you don't think you are imposing your personal beliefs on the data, by making a choice about what gets in and what doesn't you run the risk of biasing them.

Given that you either set rules for removing a data set, like in political elections, and follow them rigidly, or you accept all of your data and adjust your analysis with an understanding of where it all came from.

The reason I have for this is how do you decide what is and isn't a troll response? Is "Ban Island" a troll? is "Kill Sheldon" a troll? is "survey are bullshit", did you throw out the whole response when the respondent used the free comment space to abuse you and threaten you personally or did you keep their numerical data? If you set the line on what is and isn't trolling then it's hard to accept the results as unbiased anymore.

_________________
Favourite Deck:
Ghost Council of Orzhova

Playing Online:
Noyan Darr & Sedris Zombie Guy


Top
 Offline Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 128 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 9  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 23 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron