Choosing Studies: Eenie Meenie Or Something More?

I want to pick up on the last post, which you should probably read before going further.   Here we are with two studies, one published in Pediatrics and the other on the website of Family Scholars.    They appear to reach contradictory conclusions, with the first study finding that children of lesbian couples (who are children conceived with third-party sperm) do as well, if not better, than other children while the second study concludes that children of conceived with (only some of whom are children of lesbian couples) suffer harm.   

One thought is that it might be possible to harmonize the two studies, because they have studied different populations.    The first study focused on 17 year olds who were children of lesbian couples.    The second study focused on adults between the ages of 18 and 45.   Of the 485 adults studied, 39 of them (about 8% were children of lesbians).   Each of these distinctions is important, but perhaps the first point is that no single person could be the subject of both studies.   Thus, the researchers studied completely separate populations, which might mean that it is not so surprising they reached different conclusions.   

In addition, because the children of lesbians only consist of 8% of the groups studied by the Family Scholars, it’s not always possible to extract conclusions about that populations.   For some of the Family Scholars data, data reporting responses of children of lesbians are reported separately but in other instances, children of lesbians are included in the larger undifferentiated group.   In those instances, it’s really not possible to reach any conclusion about the responses of the 8% of the population who were children of lesbians.  Any variation in their response could be overwhelmed by the responses of 92% of the subjects who are not children of lesbians.

The age difference between the two populations studied may also be important.   Children who are 17 now came of age within the last decade.   The environment for children of lesbians and gay men in the 2000s is quite different from that a child born in 1965 (45 years ago) might have faced.   I cannot tell the age range of the children of lesbians (this is one of those things that isn’t broken out separately) but it does seem to me that social conditions have changed substantially within the last decade or so.  Thus, difficulties that may have afflicted children born in 1980 may not persist for those born in 1990.    What I mean is, it seems possible to me that an older sample would reflect more adverse impact than a younger sample. 

But truly, this is all speculation and I’m not an expert on the analysis of social science studies.  Perhaps the studies really are irreconcilable.  How then should one choose which one to credit?  

Let me start with the obvious.  It seems to me wrong to simply credit the study that happens to reinforce an existing belief.    It’s hard to resist the temptation perhaps, but I think it is worth the effort. 

One potential issue that must be of concern is bias on the part of the researcher.   But surely all these researchers have some interest in their topic, else they wouldn’t be doing the research.   The mere fact that a researcher has a point of view cannot disqualify them from doing research, or we’d be left with researchers who didn’t care about their topics–hardly my ideal researchers.   The crucial question, it seems to me, is whether bias influences the design, execution or reporting of the studies in a way that undermines our confidence in the results.     

It’s hard for me to make that assessment, since I’m not a social scientist.   But it seems to me this is a critical purpose of peer review.  Peer reviewers ensure that the design, execution and reporting of the study is not infected with bias.    And only one of these studies is peer reviewed–the one in Pediatrics.  

I’ve read the response on the Family Scholars blog about peer review–which I think is essentially an assurance that the authors have done something that is the equivalent of peer review.   (Mind you, I’m not sure exactly what it is they have done, because they do not say.) 

I find it impossible to accept this assurance.   Peer review isn’t self-administered.  It’s administered by an editor who does not share the author’s bias.  It seems to me that Mr. Blankenhorn, who is President of the Institute for American Values which commissioned the study, shares the viewpoint of the authors.  He is as interested as they are, and hence is not in a good position to design proper peer review.     

I must confess, there are other aspects of the presentation of the Family Scholars report that leads me to be somewhat suspicious of it.   Consider, for example, the name of the report.   It’s called “My Daddy’s Name is Donor.”   And what proportion of the people they surveyed called their donor “daddy?”  Seven (I think this is percent, but it could be seven people, I suppose.   This is on page 91 of the report.)    The most common referent is “donor” (55) followed by “seed giver” and “contributor of genetic material” (each with 32).    

Choosing a title that suggests that lots of donor conceived people think of the man who provided the sperm as “daddy” may make good press, but it’ s not a fair representation of the results of the study.  I don’t think any peer-reviewed scientific journal would allow it.  And it does give me pause.

Advertisements

6 responses to “Choosing Studies: Eenie Meenie Or Something More?

  1. I have not yet read the study in pediatrics, but share your impressions of bias of the “family scholar’s study” in that sometimes the statistical results do not match up with authors’ conclusions.

    However regarding the title, it was meant to be facetious.

    This does not mean, however, that their results are wrong.

    In fact, there may be more overlap in the study than appears at first glance:
    -The authors agree that 39 families is too small a sample to draw conclusions.
    -They too point to certain difference among lesbian families, in which the donor children are doing better than the donor children of the single mom’s or the hetero couples.

    Also, the study does not seem to differentiate between anonymous and ID release donors. To me this is a major flaw.

    • The anonymous v. identity release donor question interests me. I have come to put some weight on that distinction in my own thinking, in part because of conversations I’ve had here. But the longitudinal survey says it makes no difference and the Family Scholar survey doesn’t make the distinction in its reported data (that I can see).

      Having now read the Family Scholars publication more carefully, I do have some problems with it. The data is interesting but the report that is attached to the data is essentially and advocacy piece. There’s nothing wrong with advocacy–I’m a law professor and I train people to be advocates. But people conducting studies are not supposed to be advocates. I have no problem with advocates using studies to support their points of view, and perhaps that’s really how I should view this. But it is presented as though it were a research study, which bothers me.

      By contrast, the longitudinal study is pretty dry and academic. I am quite sure others will use it in their advocacy in time, but it seems to me a less argumentative presentation.

      There’s actually a lot of interesting data in the Family Scholars paper. I’m not sure what to make of it all. And there are comparisons I wish they’d done. For example, why only ask donor-conceived if they worry that their parents may have lied to them? I’m sure there are non-donor-conceived kids who also have this worry (though perhaps for different reasons) but it is the comparison that is important. And as to this question especially, I’d want to break out the kids who always knew they were donor conceived from those whose parents concealed it.

  2. Perhaps, most relevant to this blog, is the conclusion: that no difference was found between the lesbian couples who were still together and those who had separated, although I would of course have to find out more about it.

  3. Julie,

    You are gracious, sweet and generous towards the IAV.

    “It seems to me that Mr. Blankenhorn, who is President of the Institute for American Values which commissioned the study, shares the viewpoint of the authors. He is as interested as they are, and hence is not in a good position to design proper peer review.”

    I bet if you commented on their blog and asked Elizabeth to tease out more data on the lesbian couples, she’d do that for you.

    Did you notice that Mercator.net was the first news outlet to interview Elizabeth? Mercator is an Australian “Orthodox Christian” news outlet and they may harbor a little bit of bias.

  4. Hi Julie,

    Here’s the breakdown for age groups for the 39 offspring of lesbian couples in our study of adult sperm-donor offspring. (Readers, see “My Daddy’s Name is Donor” available at FamilyScholars.org )

    18-25 year olds, 14 of them
    26-35 year olds, 18 of them
    36-45 year olds, 7 of them

    Best,
    Elizabeth

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s