2.1 Introduction to Sampling

Video Presentation

Learning Guide

Sampling is selecting a part of something with the intent of showing the quality, style, or nature of the whole. We sample because we want to learn about a population or an area of interest, such as a species of concern of a specific management unit. Why don’t we just do a complete census of the area or population that we are interested in? Conducting an accurate census is difficult, time consuming, and often impossible.

For example, assume we are interested in cheatgrass (Bromus tectorum), an invasive annual grass that is problematic on western rangelands (Figure 1). Because it is an annual plant, we are concerned about its reproductive potential and need to make some inferences about population size and the total number of seeds produced by the population each year. To conduct a complete census of all cheatgrass plants in the western United States, we would need to locate and measure every single cheatgrass plant in every single population.

Figure 1. A stand of cheatgrass (Bromus tectorum). Photo courtesy of Cassondra Skinner @ USDA-NRCS PLANTS Database.

How could we ensure that we quantified seed production throughout the entire growing season?  How much time and effort would this require? Even if we scaled our area of study back to a management unit such as a 500-hectare grazing allotment, and we had unlimited time and resources, obtaining an accurate and complete census of this variable would be impossible!  However, information about population size and reproductive potential is important to assist managers as they prioritize management actions and prevent cheatgrass invasions. How might we, go about providing this information?  We sample, and the results of sampling can be used to make inferences about the population as a whole.

Sampling Terminology

Population is a term that has two different meanings in natural resource management. In the statistical sense, a population is the group of individuals or area about which one wishes to make statistical inferences. In this context, the population may be a population of plants of animals, a community, or an area of land.  In essence, the population in the statistical sense represents the “sampling universe” from which we select our sample to measure.  In the biological sense, “population” refers to a biological population, which includes all individuals of a species that occupy a particular area, and are capable of exchanging genetic material.

To further illustrate these two populations, consider the following. To make generalizations about cheatgrass seed production, we might record the number of seeds found on plants in 100 plots (1-m²) in the McCone pasture (Figure 2).  We want to make inferences by sampling the cheatgrass population located within that pasture, and there are 10,000 possible 1-m² plots that we could measure in this 1-hactare (1 ha) pasture. The statistical population encompasses all the possible plots within that pasture boundary, our “sampling universe”.

Figure 2. Map showing a statistical population of cheatgrass (McCone Pasture) and a biological population of cheatgrass, the Canyon Creek Watershed.

Alternatively, the biological population of cheatgrass may be all of the cheatgrass located within the Canyon Creek watershed.  In this case, the statistical population (cheatgrass within the pasture) is a subset of a specified biological population (cheatgrass in the watershed). At other times, the statistical population may encompass the entire biological population or even multiple biological populations. Such would be the case if we wanted to sample and make statistical inferences about cheatgrass across the entire state of Montana.

A sample is a subset of sampling units selected from a population.  In sampling, we take measurements of a variable of interest, and use the results to make inferences about the population. In the cheatgrass example above, the 100 plots measured for seed production was the sample.

A sampling unit is a discrete entity or member of the population from which we take measurements.  Essentially, the population consists of all the possible sampling units that could be measured, and the sample represents the sub-sample of sampling units that were selected for measurement.  In the example above, each 1-m² plot was a sampling unit, and the measured variable was the number of seeds produced in each plot. The number of sampling units is considered the sample size (n).

The values that we obtained when we measured each sampling unit are the values that we use in our data analysis.  This is very straight-forward in the case when a single point, quadrat or plot is the sampling unit, because there is one measured value per sampling unit.  However, sometimes we collect data using multiple points positioned along a transect line, or we place quadrats along a transect line and take a measurement in each quadrat.

For example, if we estimate cover by measuring 25 points along each of 10 transects, is the number of sampling units 10 transects or 250 points?  In other words, do we analyze the data based on 250 individual point measurements, or do we calculate an average value of the 25 points per transect and base our analysis on the values of the 10 transects? In this case, the points were used to clearly describe conditions along the transect and the transect is the sampling unit.  Our choice of sampling unit depends on our objectives for sampling, and the sampling unit needs to be clearly defined in the monitoring or sampling protocol.

Examples of common sampling units used for vegetation measurements are:

• quadrat – plot of a known area and dimensions, with a distinct boundary
• transect – a line of known distance
• point –a quadrat of smallest possible dimensions, theoretically dimensionless, usually measured at the intersection of cross-hairs or the tip of a highly-sharpened steel pin
• individual – either the whole plant or specific plant parts such as stems or flowers

Sampling is the process of selecting a group of sampling units to gather information about variables of interest in such a way that you can make inferences about the total population. A variable is a characteristic whose value can vary from one sampling unit to another. Variables can be measured or estimated and can be quantitative as in the case of plant height measured in inches or qualitative as in the case of “heavy”, “moderate” or “light” used to describe levels of grass utilization.

Decisions about which variables to measure need to be carefully considered in the context of management objectives.  Collecting data is time consuming and expensive, so we need to ensure that the variables measured will provide necessary information, so that we can avoid measuring useless variables.  For example, we could measure the leaf length on cheatgrass plants, but this information would be of little use when developing a management plan for the population. Commonly measured vegetation attributes or variables are Density, Frequency, Cover, Biomass, Height, and Utilization.

Describing Populations and Samples

Population parameters are descriptive measures that characterize a population. Population parameters are assumed to be fixed, but their exact values are unknown because we cannot measure each member of the population. For example, we know that a cheatgrass population produces a certain number of seeds each year, but we do not know what that quantity is. If the population is growing, then it follows that cheatgrass seed production will increase in the next year, because the population changed. Population parameters change only if the population changes and are denoted with Greek letters, such as μ and ơ.

Sample statistics are descriptive measures that characterize a sample.  We sample to obtain an unbiased estimate of population parameters: remember, population parameters are fixed but unknown or unknowable!  We use sample statistics to estimate the population parameters.  We expect sample statistics to change if the population changes.  However, sample statistics may also change from sample to sample due to the nature of selecting a subset of sampling units from the population. Sample statistics are denoted with Roman letters such as x or s.

Common Parameters and Statistics

The most common parameters and statistics that we use to describe populations and samples are estimates of central tendency and estimates of variability.

The most common measures of central tendency are:

• Mean
• Median
• Mode

The mean is the most commonly reported measure of central tendency and is derived by summing the values obtained from each sampling unit and dividing by the total number of sampling units.

For example, if the number of seeds produced by 5 individual cheatgrass plants were 3, 4, 4, 7, and 13, the mean would be calculated as:

Note: Population means are denoted by the Greek letter µ, and sample means are denoted by .

Although means are commonly reported and are relatively easy to comprehend, the mean is sensitive to extreme values, or outliers.  As such, the mean may be somewhat misleading when the dataset includes outliers.  In the example above, if the last measured value was 39 seeds instead of 13, the mean would be 11.4 seeds per plant.  In this case, the mean value clearly is not representative of the sample.

The median is the middle measurement in a dataset when all measurements are ordered from smallest to largest.  Essentially, this means that an equal number of values are distributed above and below the median value. From the original example of seed counts, the median value of 3, 4, 4, 7, and 13 equals 4.  In general, the median is a more robust measure of central tendency than the mean in cases where the dataset includes extreme outliers.

The mode is the value that occurs most often in the sampling. In the above example, the mode equals 4.  Another way to consider the mode is that it reveals areas of concentrated values in the dataset.  For example, if we collected data on 20 sampling units with the following values:

1, 2, 3, 3, 3, 3, 4, 5, 5, 6, 8, 8, 8, 8, 8, 8, 8, 8, 9, 10

we could report the mode to reveal the concentration of values at 8.  In addition, this dataset appears to be bimodal, with measurement concentrations at 3 and 8.

We are also interested in describing the variability of the data.

Variability refers to the amount of spread that we observe in the measured values: are they clustered close together, or are they widely scattered around the mean?  Some variables exhibit relatively little variation in nature, whereas other variables may assume a wide range of values.  We use the following descriptive measures to describe the variability of populations and samples:

Variance (σ² for populations and s² for samples) is a measure of variability or dispersion that takes both the magnitude of the deviation, or distance of individual values from the mean, and the frequency of these deviations into consideration. Variance is not often reported as a sample statistic because the units of variance are squared, so they do not match the units of the mean.  For example, if the mean plant height for a pasture is 4 ft, the variance might be 0.25 ft².

Standard deviation (σ for populations and s for samples) is a measure of how much the individual measurements in the sample diverge from the mean.  The standard deviation is commonly used to report variability of samples because the units for standard deviation are the same as those of the mean.  For example, in the plant height example from above, the standard deviation would equal 0.5 ft.  Mathematically, the standard deviation is simply the square root of the variance.

Standard error (SE, also reported as s_x) is also a measure of deviation from the mean, but standard error takes into account the number of sampling units (sample size, n) that were measured. The larger the data set, the smaller the standard error will be.   Mathematically, the standard error is calculated as the standard deviation divided by the square root of the sample size: s/√n

A coefficient of variation (CV) is another way to describe the variability of a sample. It is simply the calculation of the standard deviation divided by the mean (s /  ); for ease of interpretation, the CV is often converted to percent by multiplying by 100. The advantage of coefficients of variation is that they do not have units, so they are not sensitive to the magnitude of measurements.  We often will use CV’s to compare variability of two or more samples for which the magnitudes of the means are very different.

Let’s consider a situation where the mean weight of two species (coyotes and mule deer) are quite different in magnitude, and we want to be able to describe how variable the measured weights of individuals in the samples were.   Based on sampling efforts, we determine the mean and standard deviation for weight of adult coyotes and mule deer (Table 1). If used the standard deviation to determine which mammal exhibits greater variability in adult weights, it would be easy to assume that the weight of adult mule deer is more variable than that of coyotes because 40 lbs is obviously larger than 5 lbs.  However, given that the mean weight of mule deer is around 10 times greater than coyotes, we would expect that the standard deviation of large measurements is going to be greater than that for smaller measurements.  By calculating the coefficient of variation for the two populations, we account for the difference in the magnitude of the mean values because the measure of variability (s) is relative to the mean ().  Smaller CV’s indicate less variability, and by comparing the CV’s we see that mule deer weights are actually less variable than coyote weights.

Table 1.  Comparison of the weight of individual adults of two mammal species, coyote (Canis latrans) and mule deer (Odocoileus hemionus).  The values given are mean (x), standard deviation (s), and coefficient of variation (CV).

In another example, we compare measurements of variability using data collected in 3 plots at 3 different sites.  In this case, we counted the number of cheatgrass seeds produced in each plot (Table 2).  Note that the mean number of seeds for the 3 sites is the same ( = 14 seeds/plot), but by comparing the different measures of variability, we can see that seed production is most variable at the Ridge Top Pasture.

Table 2. Cheatgrass seeds produced in 3 plots at 3 different sites on the Round Top Ranch.

Accuracy and Precision

Two important concepts in sampling are the accuracy and precision of the data.

• Accuracy is the closeness of a measured value to its true value. Remember that we cannot know the true mean value for a population, but through sampling we estimate the population mean with the sampling mean.
• Precision is the closeness of repeated measurements to each other.

A convenient way to visualize the concepts of accuracy and precision is to envision a target and target practice (Figure 3).  The bull’s-eye of the target represents the true population mean, or the true value, and each “x” represents a measurement from a sample, or in the target analogy, a shot at the target.  In the shot pattern from the target on the left, the shots are fairly evenly distributed around the bull’s-eye, and their average value would be close to the center: the shooting is accurate, but not very precise.  In the center target, the shots are closely grouped together, but they are not very close to the bull’s-eye, and if you averaged the values for those shots, it would be quite different from the true population mean. In this case, the shooting is precise, but not accurate.  In the target on the right, the shots are closely grouped and their average location is centered near the bull’s-eye, or true population mean.  In this case, the shooting is both precise and accurate.

Figure 3. A graphical illustration demonstrating the concepts of accuracy and precision.

The target analogy is convenient for learning about the concepts of accuracy and precision, but we don’t actually collect samples by shooting at them!  So let’s apply these concepts to the data collected on the Round Top Ranch (Table 2).  Let’s start by ranking the three datasets in terms of their precision.  When repeated measurements are similar in value, this is indicated by low estimates of variability.  The measurements from the East Hill Pasture were the most precise because this dataset has the smallest values for all of the estimates of variability, and the measurements from the Ridge Top Pasture were the least precise, or most variable.  We can use sample statistics such as standard deviation, standard error, and coefficient of variation to estimate the precision of our sampling.

Sources of Error in Sampling

Unfortunately, we cannot use statistics to determine whether these measurements were accurate.  Remember that the actual population parameters are fixed, but unknown. We cannot use statistics to tell how accurately we estimated the true population parameters.  In the cheatgrass example above (Table 2), we cannot be sure that true population mean for the number of cheatgrass seeds produced per plot was actually equal to 14.  In part this is due to the nature of sampling.  Because our sample is a randomly selected sub-set of the population, we could get an estimate for the mean that is quite different from the true population simply because the sample consisted of a group of “atypical” individuals or sampling units.  Therefore, when we sample, the estimate of variability from the sample reflects both the inherent variation that is characteristic of the population, and variation due to the random selection of sampling units in the sampling process.  The difference between the sample statistics and the true population parameters is called error.

There are two types of error associated with sampling:

• sampling error
• non-sampling error

Sampling error is due to chance. Since the sampling process involves chance selection of a sub-set of individuals or sampling units, some samples are “more” representative of the population, and some samples are “less” representative of the population. Therefore, the difference between the sample statistic and population parameter is not due to technique or procedure, but is different just by chance! We can estimate the amount of sampling error statistically because it is a measure of variation between all the samples. Sampling error affects the precision of our samples.  In general, sampling error can be reduced by increasing the sample size, n, and by ensuring that we use an efficient sampling design to increase the precision of our estimates.

Non-sampling error is measurement error that results from using faulty equipment, incorrect or inconsistent application of measurement techniques, or a variety of mistakes that can be attributed to “human error”.  Human errors can include misidentification of species, poor data recording, data entry errors, subjective selection of sampling units, etc.  Non-sampling errors affect the accuracy of our estimates because the measurements are faulty or suspect.  Non-sampling error may result in measurements or estimations that are unpredictable in the direction of the error; sometimes the values are high and sometimes they are low.  In many cases, non-sampling error results in bias, a consistent over- or under-estimation of a value. It is for this reason that non-sampling error is also called systematic error.

Unfortunately, since non-sampling error affects accuracy, it cannot be detected statistically.  Therefore we need to take actions to eliminate the potential for non-sampling errors.  Using well calibrated equipment, ensuring that observers are properly trained to employ the sampling technique and follow the sampling protocol are a few ways to reduce sampling error.  In addition, ensuring that the selection of sampling units is done randomly and not subjectively also reduces the potential for this type of error.  In general, we assume that in the absence of bias and non-sampling error, precision will lead to accuracy.