More and more companies are starting to see the benefits of conducting their own surveys because it's cheaper and faster than previous methods. Doing surveys yourself costs less than hiring a market research firm, and results can be seen instantly. Most people today are still using a combination of paper, phone, and web based data collection methods, although internet based research is quickly on the rise. This article attempts to describe survey collection methods in general, with some discussion of research across multiple channels.

1. Keep the questionnaire as short as possible.

It is statistically proven that more people will complete a shorter questionnaire, regardless of the interviewing method. If a question is not necessary, do not include it.

2. Get off to a good start.

Start with a Title (e.g., Leisure Activities Survey). Always include a short introduction - who you are and why you are doing the survey. If you are seeking critical feedback about how your brand stacks up against the competition, it is often a good idea to give the name of the research company rather than the client (e.g., XYZ Research Agency rather than the manufacturer of the product/ service being surveyed). Many firms create a separate research company name (even if it is only a direct phone line to the research department) to disguise themselves. This is to avoid possible bias, since people rarely like to criticize someone to their face and are much more open to a third party.

In some cases, though, it may help to mention the client. If you are surveying members of an organization, the members may be more likely to respond if they think the organization is asking their opinions on how it can best meet their needs. The same could be true when you are surveying users of a particular service.

3. Reassure confidentiality of responses.

Reassure your respondent that his or her responses will not be revealed to your client, but only combined with many others to learn about overall attitudes. Mention this in the opening text of the survey and also include a privacy policy in the footer of the survey page.

4. Include a good cover letter /email invite text.

Include a cover letter with all mail surveys. A good cover letter or email invitation to take a Web page survey will increase the response rate. A bad cover letter or none at all, will reduce the response rate. Include the information in the preceding two paragraphs and mention the incentive (if any). Describe how to return the questionnaire. Include the name and telephone number of someone the respondent can call if they have any questions. Include instructions on how to complete the survey itself. The most effective cover letters and invitations include the following elements: Ask the recipient to take the survey. Explain why taking it will improve some aspect of the recipient's life (it will help improve a product, make an organization better meet their needs, make their opinions heard). Appeal to the recipient's sense of altruism ("please help"). Ask the recipient again to take the survey.

5. Always provide a space for their name. You may want to leave a space for the respondent to add their name and title. Some people will put in their names, making it possible for you to contact them for clarification or follow-up questions. Indicate that filling in their name is optional. If the questions are sensitive in nature, do not have a space for a name. Some people would become suspicious and not complete the survey. If you hand out questionnaires on your premises, you obviously cannot remain anonymous, but keep the bias problem in mind when you consider the answers.

6. Consider survey software that support security.

If the survey contains commercially sensitive material, ask a "security" question up front to find whether the respondent or any member of his family, household or any close friend works in the industry being surveyed. If so, terminate the interview immediately. They (or family or friends) may work for the company that commissioned the survey - or for a competitor. In either case, they are not representative and should be eliminated. If they work for a competitor, the nature of the questions may betray valuable secrets. The best way to ask security questions is in reverse (i.e., if you are surveying for a pharmaceutical product, phrase the question as "We want to interview people in certain industries - do you or any member of your household work in the pharmaceutical industry?). If the answer is "Yes" thank the respondent and terminate the interview. Similarly, it is best to eliminate people working in the advertising, market research or media industries, since they may work with competing companies.

7. Recommended flow for your next survey. After the security question, start with general questions. If you want to limit the survey to users of a particular product, you may want to disguise the qualifying product. As a rule, start from general attitudes to the class of products, through brand awareness, purchase patterns, specific product usage to questions on specific problems (i.e., work from "What types of coffee have you bought in the last three months" to "Do you recall seeing a special offer on your last purchase of Brand X coffee?"). If possible put the most important questions into the first half of the survey. If a person gives up half way through, at least you have the most important information. 8. Include all relevant alternatives.

Make sure you include all the relevant alternatives as answer choices. Leaving out a choice can give misleading results. For example, a number of recent polls that ask Americans if they support the death penalty "Yes" or "No" have found 70-75% of the respondents choosing "Yes." Polls that offer the choice between the death penalty and life in prison without the possibility of parole show support for the death penalty at about 50-60%. Polls that offer the alternatives of the death penalty or life in prison without the possibility of parole, with the inmates working in prison to pay restitution to their victims' families have found support for the death penalty closer to 30%.

So what is the true level of support for the death penalty? The lowest figure is probably truest, since it represents the percentage that favor that option regardless of the alternative offered. The need to include all relevant alternatives is not limited to political polls. You can get misleading data anytime you leave out alternatives.

10. Do not put two questions into one.

Avoid questions such as "Do you buy frozen meat and frozen fish?" A "Yes" answer can mean the respondent buys meat or fish or both. Similarly with a question such as "Have you ever bought Product X and, if so, did you like it?" A "No" answer can mean "never bought" or "bought and disliked." Be as specific as possible. "Do you ever buy pasta?" can include someone who once bought some in 1990. It does not tell you whether the pasta was dried, frozen or canned and may include someone who had pasta in a restaurant. It is better to say "Have you bought pasta (other than in a restaurant) in the last three months?" "If yes, was it frozen, canned or dried?" Few people can remember what they bought more than three months ago unless it was a major purchase such as an automobile or appliance.

11. Begin with the End in Mind.

The overriding consideration in questionnaire design is to make sure your questions can accurately tell you what you want to learn. The way you phrase a question can change the answers you get. Try to make sure the wording does not favor one answer choice over another.

12. Avoid biased words.

Avoid emotionally charged words or leading questions that point towards a certain answer. You will get different answers from asking "What do you think of the XYZ proposal?" than from "What do you think of the Republican XYZ proposal?" The word "Republican" in the second question would cause some people to favor or oppose the proposal based on their feelings about Republicans, rather than about the proposal itself. It is very easy to create bias in a questionnaire. This is another good reason to test it before going ahead.

13. Give products/services neutral names.

If you are comparing different products to find preferences, give each one a neutral name or reference. Do not call one "A" and the second one "B." This immediately brings images of A grades and B grades to mind, with the former being seen as superior to the latter. It is better to give each a "neutral" reference such "M" or "N" that do not have as strong a quality difference image.

14. Avoid acronyms and jargon.

Avoid technical terms and acronyms, unless you are absolutely sure that respondents know they mean. LAUTRO, AGI, GPA, EIEIO (Life Assurance and Unit Trust Regulatory Organization, Adjusted Gross Income, Grade Point Average and Engineering Information External Inquiries Officer) are all well-known acronyms to people in those particular fields, but very few people would understand all of them. If you must use an acronym, spell it out the first time it is used.

15. Providing all the answers - some examples.

Make sure your questions accept all the possible answers. A question like "Do you use regular or premium gas in your car?" does not cover all possible answers. The owner may alternate between both types. The question also ignores the possibility of diesel or electric-powered cars. A better way of asking this question would be "Which type(s) of fuel do you use in your cars?"

The responses allowed might be: _ Regular gasoline _ Premium gasoline _ Diesel _ Other _ Do not have a car

If you want only one answer from each person, ensure that the options are mutually exclusive. For example:

In which of the following do you live? _ A house _ An apartment _ The suburbs

This question ignores the possibility of someone living in a house or an apartment in the suburbs. Score or rating scale questions (e.g., "If '5' means very good and '1' means very poor how would rate this product?") are a particular problem. Researchers are very divided on this issue. Many surveys use a ten-point scale, but there is considerable evidence to suggest that anything over a five point scale is irrelevant. This depends partially on education. Among university graduates a ten point scale will work well. Among people with less than a high school education five points is sufficient. In third world countries, a three-point scale (good/acceptable/bad) may be all some respondents can understand. Another issue on which researchers differ is whether to use a scale with an odd or even number of points. Some like to force people to give an answer that is clearly positive or negative. This can make the analysis easier. Others feel it is important to offer a neutral, middle option. Your interviewing mode can make a difference here. A good interviewer can often get a answer, but in a self-administered interview, such as a Web page survey, a person who is frustrated by being unable to give a middle answer may leave a question blank or quit the survey altogether.

16. Be sure any rating scale labels are meaningful. For example:

What do you think about product X? __ It's the best on the market __ It's about average __ It's the worst on the market A question phrased like the one above will force most answers into the middle category, resulting in very little usable information.

17. Use similar scales/questions to accurately compare trends.

If you have used a particular scale before and need to compare results, use the same scale. Four on a five-point scale is not equivalent to eight on a ten-point scale. Someone who rates an item "4" on a five-point scale might rate that item anywhere between "6" and "9" on a ten-point scale.

18. Be aware of cultural factors.

In the third world, respondents have a strong tendency to exaggerate answers. Researchers may be perceived as being government agents, with the power to punish or reward according to the answer given. Accordingly they often give "correct" answers rather than what they really believe. Even when the questions are not overtly political and deal purely with commercial products or services, the desire not to disappoint important visitors with answers that may be considered negative may lead to exaggerated scores.

19. Always discount "favorable" answers by a significant factor.

The desire to please is not limited to the third world. Unfortunately, there is no hard and fast rule on how much to do this. It depends on the situation.

The desire to please translates into a tendency to pick agreeing answers on agreement scales. While logically the percentage that strongly agrees that "X is good" should exactly equal the percentage that strongly disagrees that "X is bad," in the real world, this is unlikely to be true. Experiments have shown that more people will agree than disagree. One way to eliminate this problem is to ask half your respondents if they agree that "X is good" and the other half if they agree that "X is bad." You could then reverse the answers given by the second group. This is extra work, but it may be worth it if it is important to get the most accurate percentage of people who really agree with something. People sometimes give answers they feel will reflect well on them. This is a constant problem for pre-election polls. More people say they will vote than actually will vote. More people say they go to museums or libraries than actually do. This problem is most significant when your respondents are talking directly to a person. People give more honest answers when answering questions on a computer. Mail surveys are in-between.

20. Provide a mixed sample.

In personal interviews it is vital for the Interviewer to have empathy with the Interviewee. In general, Interviewers should try to "blend" with respondents in terms of race, language, sex, age, etc. Choose your Interviewers according to the likely respondents.

21. Leave demographics until the end.

Leave your demographic questions (age, gender, income, education, etc.) until the end of the questionnaire. By then the interviewer should have built a rapport with the interviewee that will allow honest responses to such personal questions. Mail and Internet questionnaires should do the same, although the rapport must be built by good question design, rather than personality. Exceptions to this rule are any demographic questions that qualify someone to be included in the survey. For example, many researchers limit some surveys to people in certain age groups. These questions must come near the beginning.

Leave a space at the end of a questionnaire entitled "Other Comments." Sometimes respondents offer casual remarks that are worth their weight in gold and cover some area you did not think of, but which respondents consider critical. Many products have a wide range of secondary uses that the manufacturer knows nothing about but which could provide a valuable source of extra sales if approached properly. In one third world market, a major factor in the sale of candles was the ability to use the spent wax as floor polish - but the manufacturer only discovered this by a chance remark.

22. Always consider the layout of your questionnaire.

This is especially important on paper, computer, direct, and Internet surveys. You want to make it attractive, easy to understand and easy to complete. If you are creating a paper survey, you also want to make it easy for your data entry personnel.

23. Try to keep your answer spaces in a straight line, either horizontally or vertically.

A single answer choice on each line is best. Eye tracking studies show the best place to use for answer spaces is the right hand edge of the page. It is much easier for a field worker or respondent to follow a logical flow across or down a page. Using the right edge is also easiest for data entry. The Survey System lets you create a Questionnaire Form with the answer choices in two columns. Creating the form that way can save a lot of paper or screen space, but you should recognize doing so makes the questionnaire a little harder to complete. It also slows the data entry process when working with paper questionnaires.

24. Grid questions, while attractive and space saving, get annoying fast.

Questions and answer choice grids, as in the second of the following examples, are popular with many researchers. They can look attractive and save paper, or computer screen space. They also can avoid a long series of very repetitive question and answer choice lists. Unfortunately, they also are a bit harder than the repeated lists for some people to understand. As always, consider whom you are studying when you create your questionnaire.

Look at the following layouts and decide which you would prefer to use:

Do you agree, disagree or have no opinion that this company has: A good vacation policy - agree/not sure/disagree. Good management feedback - agree/not sure/disagree. Good medical insurance - agree/not sure/disagree. High wages - agree/not sure/disagree.

An alternative layout is:

Do you agree, disagree or are not sure that this company has:

Agree Not Sure Disagree

A good vacation policy 3 2 1

Good management feedback 3 2 1

Good medical insurance 3 2 1

High wages 3 2 1

The second example shows the answer choices in neat columns and has more space between the lines. It is easier to read. The numbers in the second example will also speed data entry, if you are using a paper questionnaire.

25. Keep trying and perfecting your surveys.

Surveys are a mixture of science and art, and a good researcher will save their cost many times over by knowing how to ask the correct questions.