Yesterday, I posted my analysis of what I think are the critical findings (at least as far as they relate to the sea year
stand down cancellation) from the Academy’s SA/SH surveys (officially titled, “Service Academy Gender Relations sexual harassment and sexual assault surveys” or (“SAGR surveys”). As shown in that analysis, there are two key points:
- The percentage of sexual assaults on female midshipmen during “sea year” declined dramatically from 2011-12 to 2013-14 – yet MARAD is telling the opposite story.
- The assailants in almost all of the sexual assaults on female midshipmen are other midshipmen. This is true no matter where the sexual assault takes place. That means sea year is not the problem.
This post will explain how I distilled the facts from the 2011-12 and 2013-14 SAGR reports. It’s important to know how to interpret these studies to get past the misinformation about them that is being reported. I think it is easiest to do this in the form of a FAQ.
1. How is it possible to compare the two surveys?
It’s difficult, but not impossible. Part of the difficulty is because the two reports do not report the data in the same way. And, the 2011-12 report does not contain some of the analysis and tables regarding the survey responses that the 2013-14 report contains. Because the 2013-14 report gives most data strictly in terms of percentages rather than estimated numbers (as was done in the 2011-12 report), it was necessary for me to convert the data in the 2011-12 report to percentages to allow there to be an “apples-to-apples” comparison.
2. In table 10 of the 2012-13 SAGR study, the various sources of the assailants are listed but the percentages add up to 142%. Why is that? And, how are you able to say with such confidence that 88% of the assaults could not have been committed by a commercial mariner?
In the survey, each midshipmen who was assaulted was asked to identify the assailant by source. Some of the choices they were given overlapped. For example, a plebe who was assaulted by a midshipman officer would respond to the multiple choices in the survey that she was assaulted by a “Fellow academy student in a higher class year” AND by a “Fellow Academy student higher in the midshipman chain of command.” So that doubles the response rate for those instances – leading to percentages that total more than 100%.
To get to the 88% figure, it was necessary to think outside-of-the-box. I regret that I didn’t figure it out myself. But one of the people working in the background on all this stuff noticed it and pointed it out to me. (That person has to remain confidential, so this the best hat tip I can give to him/her.) Anyway, if you look at Table 10, there is one category: “A person not affiliated with DOD, DHS,DOT” that has no overlap with the other categories.** So 12% of the female midshipmen who reported a sexual assault in the survey responses identified a non-Academy, non-DOD/DOD/DHS offender. Armed with that knowledge, you can work backwards – that means that 88% of the assailants were either affiliated with the Academy (midshipmen or employees) or with DOD/DOT/DHS.
3. How did you get the percentages for sources of assailants in the 2011-12 SAGR report?
That was easy. The survey reported the estimated number of each “category” of assailant and the total number of sexual assaults. It is found in the text that introduces Table 7 of the 2011-12 SAGR report.
4. How did you arrive at the totals for number of sexual assaults committed?
Those come directly from each SAGR report. They are estimates. The 2011-12 estimate was 17 sexual assaults on women midshipmen. (The results regarding sexual assaults on male midshipmen were deemed unreliable in both surveys.) The 2013-14 estimate was 19-28 sexual assaults on women midshipmen.
5. How did you arrive at the totals for the number of sexual assaults committed during “sea year.”
In the 2011-12 report, that number (10) is directly reported. In the 2013-14 report, the number is reported as a percentage. I applied the percentage (1/3) to the estimated total sexual assaults to arrive at the range of 6-9 sexual assaults during “sea year.” Note that the SAGR report is inconsistent on this figure. Table 11 reports that 33% of the women who reported a sexual assault identified the location as occurring during “sea year.” In another location, it says 37%. I believe that is a mistake that was made by using responses that are reported in Table 12. However, Table 12’s reference to “during summer experience/training/sea duty” is based upon Question 12 which did not limit the question to off-campus assaults (and therefore it would include on-campus assaults that occurred during that time). By contrast, Table 11 is based upon Question 26, which limits the location to “Off Academy grounds during summer experience/training/sea duty.” Therefore, I conclude that 33%, rather than 37% should be used.
6. If the (admittedly estimated) number of sexual assaults went from 10 in 2011-12 to 19-28 in 2013-14, how can you possibly claim that the rate of sexual assault went down?
Because the overall number of female midshipmen at the Academy went from 123 in 2011-12 to 136 in 2013-14. So the rate of assault went down.
An example from baseball statistics helps explain this. Who is the better hitter: A batter who gets 100 hits in a season or a batter who gets 150 hits in the season? If the batter who got 100 hits only had 200 at bats, he’s a much better hitter than the batter who got 150 hits but did so after 1000 at bats.
7. Why do you keep putting sea year in quotation marks when referring to the SAGR report results?
Because the SAGR results that included sea year are not limited to sea year! The respondents were asked about off-campus sexual assaults during “summer experience/training/sea duty.” That includes things like the required summer internship; it could include a sexual assault on an upper class midshipmen during indoctrination that occurred off-campus; it could include a sexual assault during a team movement (such as a regimental band event) that occurred off-campus during the summer (including indoctrination). It also includes sexual assaults that have absolutely no connection to sea year, but which occur while the midshipman is assigned away from the Academy. For example, if a midshipman is between ships during sea year and goes home while awaiting a ship and is sexually assaulted while at home, that falls within the SAGR survey category that is being used as shorthand for “sea year.” It also includes a sexual assault while on shore leave that has absolutely nothing to do with the ship’s crew: If a midshipman during sea year goes to a bar and strikes up a conversation with someone who subsequently grabs her in an inappropriate manner, that’s a sexual assault that falls under the SAGR survey category that is being used as shorthand for “sea year.” Thus, it is possible that only a small number of the sexual assaults that are being categorized as during “sea year” have anything to do with the culture aboard commercial ships.
8. Do you have any other criticisms about the SAGR surveys?
I have a number of criticisms, some of which are discussed in my recommendations, below. But a major criticism of the 2013-14 report is that I believe that its estimate of the number of sexual assaults is flawed. Out of 136 female midshipmen at the Academy, 108 responded to the survey and 28 women either did not respond or did not give a complete enough response. We are not told how many sexual assaults the 108 women reported. That’s a known number with a low error rate (there may be some error because the respondent did not understand the definition of “sexual assault” or was reluctant to report the assault even in an anonymous survey). Regardless, the total estimate should build off of that specific number. And, unless there is a reason for believing that the 108 who responded overstated the number of sexual assaults, the number of assaults the 108 reported should be the floor. In other words, if 108 women reported X number of assaults, then a projection for all 136 women should be X plus [an estimate for the other 28]. It should not be X plus or minus [an estimate for the other 28].
The 2013-14 SAGR report estimated that there were 19-28 total sexual assaults against the 136 female midshipman attending the Academy. Per my explanation above, the “floor” is 19 and therefore the 108 women respondents presumably reported 19 sexual assaults (a rate of 17.6%). So the SAGR report with its estimate of 19-28 sexual assaults (presumably) is estimating that of the 28 women who did not respond to the survey, as many as 9 were sexually assaulted (a rate of 32% – far higher than the 17.6% I’ve arrived at for the 108 female midshipman and far higher than the 17.1% overall figure reported in the SAGR report). It seems absurd to assume such a high rate. For comparison sake, if we assume that the 28 women were assaulted at the same rate as the 108 women, the total estimate would be 23 (and it would properly be reported as an estimate of 19-23 [since it is possible that none of the 28 were assaulted or were assaulted at a lesser rate than the 108 respondents]). This is significantly lower than the 19-28 estimate in the SAGR report.
9. How can the SAGR surveys be made more helpful to addressing the issue of SASH at the Academy?
- Greater transparency is in order. The raw data totals for each answer (or subpart of an answer) should be reported in an appendix.
- The methodology for weighting the responses to account for midshipmen who did not respond to the survey should be fully disclosed in a footnote or appendix. We do not even know whether the same weighting formula was used in 2013-14 as was used in 2011-12.
- There should be a specific series of questions with respect to SA/SH during “sea year” (to exclude internships, time on the TV Kings Pointer, time off between ships, etc.). The questions should specifically focus on the location of the SA/SH (aboard ship, in a bar, in another public area [park, health club, etc.], while between ship assignments, etc.) and the source of the assault (midshipman, civilian crew member, DOD/DOT/DHS employee, or other) needs to be ascertained.
**Actually, the question on the survey (Q.28) is poorly worded and it really should have said, “not affiliated with DOD, DHS, DOT or the Academy”; however, it is clear from the small response rate (12%), that the survey respondents understood this question as I have interpreted it. Otherwise, the response rate would have had to have been much higher as it would include all of the responses above the line for “person affiliate with DOD, DHS, DOT”).