A turning point in the history of American advertising and marketing happened in mid-2006 when the Department of Commerce posted all data tables from its most important publication, The Statistical Abstract of the United States, in spreadsheet compatible format for free downloading on the DOC Website. The Statistical Abstract has been available online in Acrobat format for several years, but this is the first time the data has been made available online in its most useful format. In addition, many previous editions of the Statistical Abstract dating back to 1878 were made available in Acrobat format. This is a real bonanza for market research.

To understand how momentous this is, consider that when I first reviewed the Statistical Abstract for Colorado Media, Agencies and Client News nearly 20 years ago, the big news then was that an information firm named Slater-Hall had just published the first CD version of the Statistical Abstract with Excel files of each table, at $1500 a copy. The DOC is now making all tables of the current (and 125th edition) of the Statistical Abstract available in Excel format for free downloading. If you want to think of it as $1500 worth of data for free, you're not far off.

For those still not familiar with it, the Statistical Abstract is an annual publication with over 1300 data tables covering nearly every aspect of the nation's social, political, economic and demographic life. The data is currently divided into thirty sections, including Business Enterprise, Domestic Trade, Prices, Comparative International Statistics, Manufacturers, and Income, Expenditures & Wealth. No matter what your business is, chances are that there is information in the Statistical Abstract that you would find useful, even profitable.

For most of its 125 years, the Statistical Abstract was only available in book format, and analyzing its data was a cumbersome process. The Abstract's data became a magnitude more useful when it became available in digital format, initially on 2400 foot half inch wide tape reels commonly used by mainframe computers, priced at $175 a copy. However, this format meant that for decades, DOC information highly useful to all sized businesses was only accessible and affordable to the largest corporations.

About a decade ago the DOC started releasing the Statistical Abstract on CD to a wider market, initially for about $50 a copy. All tables were in both Acrobat and Excel files, so the data could be imported into spreadsheets for analysis, and presentation in charts for easier comprehension.

Since the release of the CD, the DOC has increasingly used the Internet instead of CDs to deliver its publications to businesses, and also to collect much of the information it publishes. This trend could lead to some badly needed changes.

Some of the most important DOC publications aren't published on a yearly basis. The Economic Census only comes out every five years (in years ending in a 2 or 7) and several other economic reports are only available once every ten years. They all produce incredibly useful information, the reason they aren't yearly is that when the schedules were originally set, in the days before mainframes and networked computers, it was prohibitively expensive to produce the reports more often. Now, it's feasible to produce these reports on a yearly basis, or even more frequently.

In a future we could have reasonably soon, all of the information the DOC currently collects and publishes would be freely available in real time on the Internet. The Statistical Abstract would be constantly and instantly updated. The City and County Business wouldn't be updated every five years, it would be updated every microsecond.

So, how would you make sense out of all this data? Well, it's going to take more than a spreadsheet, but the right tools are on the way. Data mining has been around for a long time now, it's just been a question of affordability and target market, and that's changing. A vast industry is about to be born that will specialize in data mining for specific types of small businesses, like hat shops and family restaurants, the kind of places that until recently an IBM or Microsoft salesman wouldn't waste a business card on, let alone a sales call. Now that the Fortune 500 companies are saturated with Internet based technology, the only growth areas left are with much smaller businesses. IBM, Microsoft and many other companies are developing products and services scaled for smaller businesses. Even more changes will be needed, though.

At this point in time, America, by way of the DOC, collects and publishes more information than any other country in the world. It's a national disgrace that so little of that information actually reaches the hands of small to mid-sized business owners who need it the most. Due to politics, the DOC is prohibited by law from advertising about its products and services in anything other than its own publications, so most small business owners never hear of most DOC data publications. This is not how it should be.

The Department of Commerce has done an excellent job of modernizing its data collection and distribution, and justly deserves our thanks for its efforts. Posting the Statistical Abstract online in Excel format is a major milestone, but the DOC can do far more than that, and we need to make sure that it does.

It's time to reform the restrictions and responsibilities of the Department of Commerce. The DOC needs to be able to advertise about its products and services like any other data publisher. It's also time to start moving the DOC data collection and publishing to a real time basis. It will be a tough political fight to make these changes happen, but it will be worth the effort.

From Advertising & Marketing Review, October, 2006.

Copyright © 1994 - 2006 by Glen Emerson Morris All Rights Reserved