Two weeks ago, the Academy administered the 2015-16 Defense Manpower Data Center (“DMDC”) Sexual Assault and Gender Relations (“SAGR”) survey to the B-Split midshipmen who had recently returned from what the administration continues to euphemistically call “sea year.” (Midshipmen have taken to calling it “Pier Year.”) This survey is required by law and is administered every other year. At the Academy, the survey is administered in April and November of even numbered years; midshipmen who are away from the Academy on sea year in April are surveyed in November. (The 2013-14 survey was also administered in August of 2014.)
A fundamental requirement when giving a survey is to administer it identically to every person taking the survey. Everyone must be given the same instructions and same questions and should take it under similar conditions. A single word missing from the instructions or some other variation in the survey methodology can dramatically change the results of the survey. Persons administering a survey are typically given a written protocol to follow and are told that they must follow it verbatim. Failure to follow the protocol renders the survey invalid.
The protocol for administering the DMDC SAGR surveys is standardized and has not changed from year-to-year. Here is how the DMDC described the protocol for the administration of the 2013-14 survey:
A team of researchers from RSSC [a unit within the DMDC] administered the paper-and-pencil survey in group sessions. The 2014 SAGR was administered in this manner for maximum assurance of anonymity. Three administrations were required because approximately half of the sophomore and junior classes were away from campus on sea duty during the first administration in April. About half of those students were subsequently surveyed in August after they had returned to campus, with the other half surveyed in November. Separate sessions were held for female and male students at each Academy. After checking in, each student was handed a survey, an envelope, a pen, and an information sheet. This sheet included information about the survey and details on where students could obtain help if they became upset or distressed while taking the survey or afterwards. Students were briefed on the purpose and details of the survey, the importance of participation, and that completion of the survey itself was voluntary. If students did not wish to take the survey, they could leave the session at the completion of the mandatory briefing. Students returned completed or blank surveys (depending on whether they chose to participate) in sealed envelopes to a bin as they exited the session; this process was monitored by the survey proctors as an added measure for protecting students’ anonymity.
2013-14 DMDC SAGR Report (p.7 of the PDF) (emphasis added).
Unfortunately, the survey given to the B-splits earlier this month was not administered under the above conditions. To begin with, it was not administered to male and female midshipmen in separate sessions. Further, Academy leadership made participation in the survey mandatory, apparently in an effort to overcome this blog’s No Sea, No D[ata] recommendation. As relayed to us by a midshipman:
B-split midshipmen received the following order via email from the Regimental Commander:
“Reminder: report to Melville Hall tomorrow morning, Monday November 14, at 0700 for your DMDC Survey. The meeting is mandatory for all 2018B.”
When the midshipmen reported as ordered, their names were checked off and CDR McCarthy said to each midshipman at that time, “Take a survey and fill it out.” Surveys were in a box. No group instructions were given. No other instructions other than “Take a survey and fill it out” were given. M/N were not told that participation in the survey was voluntary. Surveys came in an envelope and the M/N could put the survey back in the envelope and submit it. Surveys were submitted to a DMDC representative. The survey was not administered to women separately from the men.
Under no circumstances can the participation in the survey be considered to have been voluntary. Midshipmen were given a direct order to fill out the survey and this order was never countermanded by an instruction that participation was voluntary. (Buried in the survey’s “Privacy Advisory” is a statement that participation is voluntary, but given a direct order from CDR McCarthy, that pre-printed statement is not sufficient to overcome a charge of failure to follow CDR McCarthy’s direct order.)
Academy leadership violated the survey protocols. The midshipmen should have been expressly informed that participation was voluntary; and, the survey should have been given to male midshipmen and female midshipmen in separate settings. The failure to follow the protocol is significant; and, it is made worse by the fact that it means that the November 2016 survey was not administered under the same conditions as the April 2016 survey. The results from April cannot be combined with the results from November because the surveys were administered under different conditions.
The breaches of the protocol cannot be shrugged off as mere technicalities. A mandatory survey will invariably yield different results than one that is voluntary (and that assumes that a mandatory survey can ever be valid). Further, there are obvious reasons why a survey on a sensitive topic such as SA/SH should be administered to males and females separately. DMDC certainly recognizes the importance of administering the survey to the sexes independently. The failure to follow the DMDC protocol renders the data invalid and the survey useless.
It would be easy to chalk this up as just one more example of incompetence by Academy leadership. But, we also have direct evidence that Academy leadership has lied about the results of SA/SH studies in the past; lied about what the Middle States Accreditation Commission said about sea year and SA/SH; and lied when it claimed that it had done everything it could to address SA/SH at USMMA. So one can’t help but wonder whether Academy leadership might want the 2015-16 DMDC SAGR survey results to be declared invalid — the last two DMDC SAGR surveys revealed Academy leadership’s failure to address SA/SH issues. Maybe the administration would prefer invalid results over more results showing its abject failures.