It would be useful to have regular benchmarking exercises performed on both commercial and open source GIS software on both the Windows and Linux 32-bit and 64-bit platforms. Such benchmarks would include serving up data via OGC web Services, http://www.opengeospatial.org/standards, such as WMS, WFS, WCS, SOS, etc. This exercise could be based on the techniques used at the 2009 FOSSG WMS Shootout.
Also, desktop GIS systems could be benchmarked on the processing of large data sets like a 400GB ASCII Lidar dataset, or merging and reprojecting 1 m NAIP data for an entire state.
The results of these benchmarks could be made publicly available, or if restrictions are placed on the publication of benchmark results by the software license, shared across the Department of Interior.