Glossary

A

Activities: The actions carried out by a program to achieve the objectives.

Assessment: Often used as a synonym for evaluation.The term is sometimes recommended for processes that are focused on quantitative or testing approaches.

Attitudes: What people accept as true—their feelings, perceptions, or opinions.

Attributes: Who people are, their characteristics, and their environment.

Average: This usually refers to the arithmetic mean (the sum of all of the list divided by the number of items in the list), but also could be the mode, or median that summarizes or represents the general significance of a set of unequal values.

B

Behaviors: What people do (in the past, present, or future).

C

Central tendency: A representative or typical value for the dataset. It can be represented by the mean (or average), median (middle point that divides data set into two equal halves) or mode (most frequent value) depending on the data and your needs.

Closed-ended questions: A broad category of questions that provide a set of possible responses from which to choose. 

D

Data-collection instruments: The tools used to collect information. Examples of data-collection instruments include:

• Questionnaires
• observation checklists
• Interview protocols

Data analysis: The process by which meaning, themes, and useful information are extracted from raw quantitative or qualitative data.

Descriptive data: Information and findings expressed in words; unlike statistical data, which are expressed in numbers.

E

Evaluation: The systematic collection of information to answer important questions about activities, characteristics, and outcomes of a program. Steps included in the CDC Evaluation Framework are:

1. Engage stakeholders
2. Describe the program
3. Focus the evaluation design
4. Gather credible evidence
5. Justify conclusions
6. Ensure use and share lessons learned

Evidence: Information, both qualitative and quantitative, that an evaluator collects to reflect the process or outcome of a program.

F

Focus group: A group selected for its relevance to an evaluation that is engaged by a trained facilitator in a series of discussions designed for sharing insights, ideas, and observations on a topic of concern to the evaluation.

Frequency distributions: A tabulation of the possible values for a variable (e.g., levels of satisfaction) and the number or range of observations that fall into each of the possible value categories.

G

Goal: A long-range statement describing what your program is working toward; a goal describes the “big picture,” or the conditions that will result if your program is successful.

I

Impacts: Long-term outcomes may be called impacts. Impacts tend to be influenced by external factors as well as program activities, so are more difficult to directly connect to the program activities.

Indicators: Measurable elements that tell you, or indicate, that the program efforts are successful. Indicators help to define what information must be collected to answer evaluation questions.

Inputs: Resources, such as costs, materials, and personnel, required to carry out the program.

Instruments: see data-collection instruments

K

Knowledge: What people understand or know.

L

Logic model: A diagram that visually organizes the elements of a program and shows relationships between those elements.

O

Objectives: Interim measurable goals (think of them as markers or steps along the way to a goal). Though it may be difficult for you to know whether you have achieved your goals, you should be able to measure whether you have accomplished your objectives. Whereas goals are broad and achieved over one or more years, objectives are clear, measurable, and can be achieved in much shorter periods of time (typically within one year, or program cycle, or less). As you accomplish each objective, you will be closer to reaching your overall goal.

Observation: A method to gather information about things that can be observed. For example, by visiting a participant's workplace, you can directly collect information on the physical surroundings. By monitoring program activities or meetings, you can observe who shows up for meetings or the program, how many individuals actively participate in a meeting or activity, how people interact, whether participants can apply the skills that are being taught, etc.

Open-ended questions: Questions that stimulate free thought by asking people to write their answer in their own words rather than choosing from a predetermined set of response options. Open-ended questions allow for spontaneous, unstructured, descriptive responses.

Outcomes: Outcomes are the positive differences the program makes in the lives of people and communities. Outcomes are changes in beliefs, attitudes, knowledge, and action the program produces. Outcomes should flow directly from program goals. Outcomes may be short term, intermediate, or long term.

Outputs: What the program is intended to produce, such as the number of services provided or the number of people reached, and other results you can count, observe, or measure. Outputs are linked to and intended to result in outcomes.

P

Percentage: A part of a whole expressed in hundredths.

Pilot test: In the case of surveys or interviews, a pilot test can determine whether the instrument used is well suited to the intended respondents, if the instructions and questions are clear, the time required to complete the instrument, and how easily the instrument can be implemented.

Population: The total number of people who belong to the specific group about which you need information. The population is defined by specific characteristics such as:

• All adults who live in a specified neighborhood
• All children who are enrolled in the local school district
• All people who participate in your educational program

Q

Quantitative data: Information that can be counted or expressed numerically. Examples of quantitative data are age, personal income, and amount of time. Even traits that you do not think of as quantitative, such as opinions, can be collected using numbers if you create scales to measure them. Quantitative data are often collected using closed-ended questions, where users are given a limited set of possible responses to select from.

Qualitative data: Thoughts, observations, opinions, or other data expressed in words. Qualitative data typically come from asking open-ended questions to which the answers are not limited by a set of choices or a scale. Examples of qualitative data include answers to questions such as "How can the program be improved?" or "What did you like best about your experience?"—but only if the user is not restricted by a pre-selected set of answers. Qualitative data are best used to gain answers to questions that produce too many possible answers to list them all or for answers that you would like in the participant's own words.

Questionnaires: An instrument useful in gathering focused, limited information from a specific population. Questionnaires ask questions in a standardized format that allows consistency and the ability to aggregate responses. Information collected through questionnaires can include participants' characteristics as well as their self-assessment of attitudes, behaviors, beliefs, and activities. Responses are limited to what is asked in the questionnaire, although questionnaires may include both closed-ended and open-ended questions.  

R

Reliability: The consistency of measurement or the degree to which results obtained by an evaluation instrument can be reproduced. 

Response rate: The percentage of people who respond to a survey. For example, if 100 people receive a survey questionnaire and 73 people complete the questionnaire, the response rate is 73/100, or 73%.

S

Sample: A collection of units or observations from the larger population. For example, it is usually not possible to survey an entire population, but one can select a sample or portion of the population to survey.

Sample, simple random: Each member of a population has an equal chance of being chosen.

Sample, stratified: The population of interest is divided into subgroups based on characteristics (e.g., gender, age, geography, participant, eligible but not participating) and randomly selected from each strata. This method is used when you have specific subgroups that you want to balance, or when you want to compare data from different subgroups.

Sample, systematic: Names are chosen from a list by starting from some randomly selected point and selecting names at a standard interval.

Snowball sampling: A method of recruiting research or evaluation participants that involves identifying someone who meets the criteria for participating and asking him or her to suggest the names of other people who might participate.

Stakeholders: A stakeholder is someone who has a stake in an organization or program. Stakeholders either affect the organization/program or are affected by it. Stakeholders include:

• People who staff a program (e.g., management, staff)
• People who are affected by a program (e.g., clients, their families, and the community)
• People who contribute to a program in other ways (e.g., funders, volunteers, partner organizations, board members)
• People with a vested interest in the program (e.g., politicians, neighbors)

Survey: A method for gathering quantitative or qualitative information directly from a defined population. Surveys are used to get a general idea of a situation, to generalize about a population, or to get a total count of the aspects concerning a group of people. The information gathered is generally easy to analyze although limited and offers little or no explanation as to the reasons behind the results. Surveys are useful for evaluations that deal with issues other than the success of the program (for example, if an evaluation is in part to identify barriers to participating in the program, questions in a survey instrument may ask about access to transportation, childcare). Surveys may be administered as paper questionnaires, Web-based questionnaires, or by a second party in person or by telephone.

V

Validity: The degree of accuracy of a measurement. For survey instruments, validity refers to the degree to which the instrument measures what it is intended to measure.

W

Weighted average: A method of calculating the mean of a set of numbers in which some elements of the set carry more importance (or weight) than others.