The Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) instrument on the polar orbiter Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) is an elastic backscatter lidar that produces a global uniformly-calibrated aerosol data set. Several Calibration/Validation (Cal/Val) studies for CALIOP conducted with ground-based lidars and CALIOP data showed large aerosol profile disagreements, both random and systematic. In an attempt to better understand these problems, we undertook a series of ground-based lidar measurements in Atlanta, Georgia, which did not provide better agreement with CALIOP data than the earlier efforts, but rather prompted us to investigate the statistical limitations of such comparisons. Meaningful Cal/Val requires intercomparison data sets with small enough uncertainties to provide a check on the maximum expected calibration error. For CALIOP total attenuated backscatter, reducing the noise to the required level requires averaging profiles along the ground track for distances of at least 1,500 km. Representative comparison profiles often cannot be acquired with ground-based lidars because spatial aerosol inhomogeneities introduce systematic error into the averages. These conclusions have implications for future satellite lidar Cal/Val efforts, because planned satellite lidars measuring aerosol backscatter, wind vector, and CO2 concentration profiles may all produce data requiring considerable along-track averaging for meaningful Cal/Val.