And Speaking of Sample Size Problems…

And Speaking of Sample Size Problems… June 26, 2012

Remember that study contrasting outcomes for children of gay parents with children in other kinds of families?  We discussed a couple methodological concerns here already, but I want to give them credit where it’s due.  The New Family Structures Survey put together a very nice interactive visualization for their data.  And, even more credit, their sample sizes and demographic breakdowns are available right on the first page of the visualization.

People have complained about how few children in the survey were in long-term stable gay families a la Dan Savage’s, but no one I’ve seen has broken down the numbers, even though NFSS gave us the tools to do it.  NFSS counted a child as raised in a gay household if either the mother or the father had ever been in a same-sex relationship while the child was a child.  Here are the subgroups they broke those relationships into:

  • Lived with the queer parent while the parent was in a same-sex relationship
  • Lived with the queer parent while parent was with a same-sex partner of more than four months
  • Lived with the queer parent while parent was with a same-sex partner of more than three years

If we’re talking about the impact of gay marriage, we’re most interested in that third category.  Let’s go to the table!

The number of respondents in each category are not whole numbers because, presumably, NFSS rounded the percentages

So we’ve got, at most, two kids raised by a long-term male-male couple, and up to 41 raised by two women who have stuck it out together.  These data say pretty much nothing about gay marriage for men, and anything about gay marriage for women is probably overshadowed by the 77% of the queer women sample who weren’t in >3 year relationships.  (Forty one people is probably too few to contrast this subgroup directly with long-term straight couples).

I can anticipate one question already: “What if there just aren’t many gay couple who stay together?  Isn’t it ok if the data reflects that?”

No, not really.  You’re really talking about two different studies.  If you want to give a demographic snapshot, it’s fine that some groups have really small showings — that’s what you’re trying to quantify.  If you want to contrast two groups, you have to oversample the rarer group so you have enough people to make a good comparison.  The NFSS people didn’t get enough two-parent, same-sex households to compare them to opposite-sex, not-separated families.  Next time round (and I do hope there will be a next time) they need to change their sampling frame.


Browse Our Archives