Has Technology Changed Your Surveying Practices?
Printer Friendly Version
A review of discussions about technology and surveys
in CAE program courses has me convinced that a major upheaval
is occurring in member information gathering. Why is that? Back
in 1997, course discussions indicated that the typical association
undertook a major member survey once every five years. This was
usually the extent of formal information gathering conducted by
associations to help them understand member needs and expectations.
Seven years later it appears that major member surveys are occurring
more frequently, often within a two or three year time frame.
In addition to major member needs surveys, a lot of special surveys
are occurring - these are usually issue based and quite short.
More associations are also getting on the phone to reach out to
key segments of their membership to probe in-depth issues and
explore product and service possibilities. As we all know, many
members won't complete a paper or on-line survey – many
associations find their members are willing to express their needs
and wants on the phone to independent surveyors. Those that use
this method tout the quality of the responses they receive.
Why the shift in practice? Technology is making it possible to
support "just-in-time" member input and accountability.
Associations are reaping the benefits of reduced survey and administration
costs directly linked to continuing improvements in technology.
In the case of phone research, targeting calls to about five respondents
per member segment doesn't break the bank and probing beyond standard
questions provides valuable information.
Smaller associations, never able to afford independent or self-administered
surveys, are now using low cost online packages that for $500
or less will get them information from members they never had
before.
It appears members, through the use of online surveys, now have
more say in the direction and priorities of their association.
This raises a lot of questions. Which members are having a say?
Will increased frequency reduce response rates? Will a select
group that always responds move the association in a direction
not consistent with the silent majority? What impact is this having
on board and staff leadership initiative? Is the association becoming
more reactive? Perhaps it's necessary to step back and look at
the source, quality, and use of survey information.
Associations now realize they need to reach out beyond the ten
percent in their organization that are most likely to participate.
Finding out the real needs of the ninety percent which represent
other types of member segments will improve decision making and
help the association achieve its core purpose. One student noted
that information from the responses of the ten percent "most
likely to" is just information - input from the other ninety
percent will help develop real knowledge to support decision making.
There is no doubt that technology is enabling new approaches
to surveys. While lower costs are always welcome, it should not
sacrifice quality. Perhaps policy around survey types, timing,
intent, etc. is required to ensure a balanced and strategic approach.
I have a chance to review many completed member surveys, and
despite the increase in survey activity I have noticed the key
question about overall member satisfaction is often missed. This
is a key performance measure that needs to be obtained to allow
for internal and external benchmarking activity. In addition,
some of the other questions being asked could have benefited from
greater thought. If you have limited training in this area, seek
the advice of a research professional to comment on the quality
of the questions you are asking – this could prevent some
embarrassing moments and improve the quality of the information
you generate.
Is it possible to get a one hundred percent response rate? The
answer is yes. One association indicated they achieved 100% rate.
The good rate was attributed to member involvement with the strategic
plan. Other helpful suggestions include keeping things as short
as possible without impacting on quality and making it easy to
respond with minimal clicking.
Those who have embraced the online survey are quick to point
out the following advantages:
- Lower costs
- Easy to create
- Getting it out is a few clicks away
- The potential for response rates is high
- The crunching is done within minutes
- Distribution of results is easy
On the other hand there are some things to watch out for such
as:
- Open responses are limited as some people have concern about
security and confidentiality
- It is not always easy to customize a low cost package
- Some survey packages have limits in terms of responses –
a large response may incur a surcharge
- A current email address may not be available for all members
nor are all members able to access high speed connections that
make downloading the survey instrument easy
- The survey is so easy to create that not enough time is spent
on question development and the relationship of the questions
to the decisions that need to be made
Other notable observations and lessons learned:
- Even where membership is mandatory, members have opinions
and want to provide input - boards and staff need to understand
the satisfaction level
- Telephone interviews to a variety of member segments by an
independent produces high quality information to support future
decision making
- Whether your managing online based surveys in-house or outsourcing,
it is important to be sure security needs are met
- Being entered into a draw if you complete a survey is not
a major incentive
- Get expert advice on question development and keep the number
as short as possible without sacrificing the intent and quality
of the survey
- Share the analysis with others as you may be too close and
miss key trends or issues
- Let members know when it is coming, when it is available,
and what the results were
This information suggests we need to spend more time on our
strategy and policies around member strategic information gathering.
Cam Davis, CMRP a Past President and Instructor with the Professional
Marketing Research Society and Vice President of Pollara says,
"the quantum shift to both qualitative and quantitative market
research on the Internet has realized incredible time and cost
savings and has revolutionized the business".
Davis suggests associations will need to spend more time on strategy
and policies around member strategic information gathering. Given
the changes occurring, now is a good time to start making improvements
in this area.
This column features innovation and practical solutions applied
to challenges, trends, issue and opportunities for the association
community. Column editor Jim Pealow, MBA, CMA, CAE is a consultant
and the Association Management Education Program Lead Instructor/Coach
for CSAE. He can be reached at jim@amces.com
|
