In 1997, the Air Force conducted a survey to gauge the career intentions of its members. The results provided ammunition in the battle to change the military retirement system, flight pay, and housing allowances. Within two years, Congress passed legislation incorporating those changes.
Service officials said the 1997 survey gets much of the credit for that victory.
Getting that legislation “had a lot to do with our ability to provide information from the troops regarding their financial needs,” said Charles Hamilton, chief of the Survey Branch at the Air Force Personnel Center.
The Air Force has conducted surveys of its members for more than 30 years. Most Air Force veterans would remember being polled at some point in their careers, but, unless they served recently, they would not recognize today’s survey process.
The common practice in the early days was to circulate printed questionnaires to the field, where members responded by checking boxes with No. 2 pencils. In 1995, however, USAF began transmitting its surveys electronically to field units and set up its first electronic database of results.
Four years later, USAF survey officials took the process to a new level by introducing the ability to respond via the Internet. And, in 2000, they employed their first targeted e-mail approach.
Faster, Better, Cheaper
While there are some researchers who question whether Internet surveys really are faster, better, cheaper, or easier to conduct, the Air Force says electronic polling has made responding easier and, consequently, brings in larger returns. It also provides more flexibility in the sampling process. In fact, a Rand study of Internet-based surveys found that USAF has a decided advantage. It is an organization tailor-made for electronic polling because of its standardized e-mail address system, information about its members, and widespread access to computers.
For instance, Rand noted that, in response to a Congressional inquiry, the USAF survey branch designed, implemented, analyzed, and reported an Air Force-wide survey in just 11 days using a combination of e-mail contacts and Web responses.
The ability to reach a wider audience can also be a disadvantage, however.
By using a Web-based poll, “we can survey as many people as we want to,” said Hamilton. “And survey them as often as we want to.”
That can lead to overkill. “Just because we can survey 100,000 people this week doesn’t mean we need to,” emphasized Hamilton, adding that there are more advantages than disadvantages to doing electronic polls, “but clearly you can overburden the people with surveys.”
One definite plus for electronic polling is that it enables the service to refine both its sample population and questions throughout the process.
“We can look at the data at any point in the collection period,” said Hamilton. “If we see, for example, we are short in the E-1, -2, or -3 grades, we can focus our follow-up effort on that group.” Since USAF no longer needs to shotgun surveys to get the right sampling, it can save time and money.
Random and Anonymous
Hamilton maintained, though, that the ability to target specific groups does not obviate the need to “do this randomly.”
Ensuring anonymity is another concern.
USAF employs “the most advanced information-masking software available,” USAF Chief of Staff Gen. John P. Jumper said in a message to Air Force personnel urging them to complete the 2002 Chief of Staff Quality of Life Survey.
According to directives, survey officials must “ensure individual responses are kept confidential” to remove the possibility that an individual could suffer “adverse actions.” Such assurance is vital to winning the cooperation of members, insists Hamilton.
“We never look at an individual’s data by name,” he said. “We have a lot of safeguards in place, and, as a practical matter, we don’t have the time to look up a particular person’s response. Anyway, we’re not interested in doing that.”
Hamilton’s office comprises only four full-time civilian specialists, but it can draw on the expertise of military behavioral scientists and research analysts. “We bring them in because that military influence lends a lot to our shop,” he said. “We need that.”
He added that even though he had worked with Air Force surveys for about 25 years, there is a military perspective “civilians might lack.” The AFPC survey branch does not handle all surveys conducted within the service, but it does control those that cover service policies and programs.
“All such surveys have to come into my office for approval,” said Hamilton. In fact, he said, it is “much easier” for his office to conduct large-scale surveys, so major commands usually defer to them for most of their command-specific information. He added, “Not many major commands are doing their own.”
At local bases, however, officials may poll their personnel about local subjects, specifically those areas that a base or unit commander has the authority to change. A couple of exceptions to that rule arise if a commander wants to survey civilian personnel. First, the commander must coordinate any survey with the base civilian personnel officer, and, second, if the survey includes questions such as satisfaction with pay or benefits, it must have USAF approval.
There are a lot of base-level surveys going on, “but they are mostly service customer-satisfaction kinds of things,” said Hamilton. “Those, we don’t get involved in because the commander can make changes based on any input he gets back.”
Taboo
Even with official sanction, some polling questions are necessarily taboo.
The Air Force forbids surveys that might harm mission accomplishment and those covering areas of possible intelligence value. Officials label as “potentially inappropriate” such topics as political views, personality assessments, measurement of knowledge or skill, opinions about specific individuals or their job performance, and any topic with responses categorized by ethnic group or sex.
To guard against bias, Air Force units must submit their questions to the survey office for approval.
“You would be amazed at the kinds of questions we sometimes get from customers,” said Hamilton. “Some are totally slanted and geared toward getting the answers they prefer.” His office’s job, he said, is to ensure objective collection of data.
“My staff spends a good deal of time writing questions,” he explained. “ We have been in this business long enough, too, that we have huge reference library of questions that have been used successfully.”
There are also questions that “haven’t worked,” Hamilton said.
Using electronic surveys means that “broken” questions can be fixed midstream. “ If you’ve got a question that isn’t working, or you are getting strange information back, you can go in and change that question,” he explained. “ You can also do that on the telephone, but with paper-and-pencil surveys, once you have put it out there, you have lost control. You don’t know your question is broken until you get it back.”
However, he conceded that, despite its obvious advantages, electronic polling is not always the best way to get information. “If we have a topic that may be sensitive, we use telephone surveys,” said Hamilton. For instance, if questions need explanation or, based on the response, need different follow-up questions, then his office would employ a telephone survey.
“It’s more sensitive in nature,” he said, adding, “we haven’t done one of these in a couple of years.”
Over the years, there’s been almost as great a change in the content of surveys as in the manner in which they are conducted. Early polls were relatively limited in scope. They simply took note of such things as the use of base facilities and career intentions.
The New Wave
Eventually, though, the Air Force began to probe further, searching for attitudes and opinions on an ever-wider range of issues. The new direction drew fire from some commanders, who felt the questions delved into matters of morale that were best handled internally.
Despite such reservations, the service persisted. Surveys now poll members about leadership and unit effectiveness as well as traditional quality of life issues.
What also helped win the critics over was the fact that the service was able to use survey data to back its bids for policy changes and legislative improvements. Rather than just plead for pay increases, for example, officials were able to show with some precision how financial problems affected retention rates.
Today, service leaders not only support the use of surveys but often request that polls be initiated on specific subjects, said Hamilton.
When the service faced rising recruiting and retention problems after the drawdown of the early 1990s, officials struggled to pinpoint the causes and devise remedies. A succession of quality of life surveys helped provide answers.
Respondents voiced typical complaints about pay and promotions but they also surfaced a new irritant: high operations tempo. A much smaller force was taking on more and more missions with frequent and, in some cases, prolonged deployments.
A 1995 quality of life poll found that 90 percent of officers and 64 percent of airmen had been away on temporary duty during the previous 12 months. Many said the absences caused family problems, delayed their training, and strained their budgets.
The optempo problem, according to that and subsequent surveys, was a major reason cited for leaving the service. Identification of that retention problem was one reason service leaders began to develop the expeditionary force concept as a means to reduce stress levels without compromising the mission. The aim was to spread deployments more evenly among members and make such movements more predictable.
The advent of electronic polling has enabled the Air Force to track trends for issues such as optempo more easily.
“The old surveys were on a piece of paper somewhere in a drawer,” said Hamilton. “ What we have tried to do with electronic surveys, particularly on the retention side, is go back a number of years and have a single, very comprehensive report.” In effect, electronic processing has allowed officials to replace the traditional “snapshot” of opinions at a specific time with a moving picture of members’ attitudes as they change and evolve.
The Air Force has found through trend tracking that its personnel generally do what they say they will do.
The survey branch tracked responses for 10 years and found that 73 percent of the company grade officers (lieutenants and captains) who said in 1989 that they planned to leave the service, actually did so by 1999. For first-term airmen, the number was even higher: 83 percent.
That was a sobering fact for officials when they reviewed career intent in the 1999 quality of life survey: some 75 percent of company grade pilots and first-term enlisted members said they did not plan a career in the service. The percentages were fairly dismal for other categories as well.
Normally, USAF would have conducted another QOL survey in 2001, but because of 9/11, it was delayed until late last year. The results, released publicly May 30, show a dramatic rise in those who say they plan to make the Air Force a career. (See “Views From 2002: Retention Up, Manpower Down,” p. 78. or pdf version)
Remarkably, too, despite the service’s continuing high operations tempo, participation was very high— “about a 45 percent response rate,” Hamilton said. “That’s better than some of our past pencil-and-paper surveys.”
Bruce D. Callander is a contributing editor of Air Force Magazine. He served tours of active duty during World War II and the Korean War and was editor of Air Force Times from 1972 to 1986. His most recent article for Air Force Magazine, “The Citizen Air Fleet,” appeared in the June issue.