Welcome to European Tribune. It's gone a bit quiet around here these days, but it's still going.

Inflation and Transparency

by Laurent GUERBY Mon Nov 20th, 2006 at 02:33:50 PM EST

A few monthes ago I asked various economists, including Bernard Salanié (see comments) why the administrations computing inflation didn't make their raw price data available since it would end conspiracy theories, endless debates about the magic "global" inflation number and provide quite useful data to taxpayers who all have different perspectives on inflation : yound, couple with or without children, old or retired face very different inflation, different regions face different prices level and inflation, etc...

The near unanimous answer (see the comments in the link above) was that data can't be released otherwise store owners would change their price behaviour (and that I didn't know anything about economics, etc... nothing unexpected).

After a while, encouraged by Max Sawicki who told me honnestly not knowing the reason and that Bureau of Labor Statistics (BLS) people were nice people, I decided to fire a few emails.

I first asked the BLS contact email listed in their inflation FAQ. Next morning USA time the answer came in, after an exchange of 7 emails during the day here is what I found out about the BLS way of measuring inflation:

  • BLS workers show immediately their official ID to the store owner and ask for permission before collecting price data. The reason is that the store owner could kick them out for suspicious behaviour otherwise (could be an unpleasant experience)

  • They come back to the same stores periodically, so store owner is so perfectly aware of the use of her prices, and owners do cooperate seriously with BLS

  • Raw data cannot be published since they need the agreement of the store owners, and publication is forbiden by this agreement

Of course, a widely different answer than the one from the so-called experts who-know-everything cited above!

For reference, I tried the same experience with the French INSEE but this went nowhere, to cut it short: data is not released because data is not released.

But in France, citizen can (and do) collect prices in stores (even without hiding it, as long as you don't do something foolish inside the store) and publish them.

In the end no excuse of any kind for not publishing the raw price data (and a few other data needed to build meaningful measures).

Side story: a big store chain "owner" (a bit more complicated) in France, Michel-Edouard Leclerc did try to show that Leclerc stores where cheaper than other store. In a first step he published on the web an index like INSEE and did not release raw data. The site was legally threatened and closed quickly.

But last friday  he did put up another site with raw data available for 355 stores all over France, 1536 products, 414469 prices collected: quiestlemoinscher.com.

I quickly wrote a  python script to collect all data in a spreadsheet compatible file and did a few experiment with it, eg: on the 30 most widely referenced product, basket prices vary 14% between cheapest and most expensive.

This little story IMHO tells a lot about "experts" and "citizens", hiding data while pretending to be a science ("economics"), and a good target for free our data-like grass-root campaigns.

How's the situation in your country?

Why do economists invent excuses to hide their data from the public scrutiny?

French version on my blog (a bit more data)

Update 20061124: Slightly different version in French accepted at Agoravox, a french citizen-journalist site

This is surely a prime candidate for the 'distributed processing' www model.

Get 100,000 people to spend 10 minutes a week entering their grocery bill data and you get something like 400 man years of work per annum.

You can't be me, I'm taken

by Sven Triloqvist on Mon Nov 20th, 2006 at 02:46:28 PM EST
Someone made the same comment on my blog.

I agree it can be done (just a scan or camera-phone picture of the bill), but since the taxpayer money already fund collection, I think it's better to start by convincing those who are supposed to represent the general interest :).

I wanted to know more about all this economist secrecy stuff too.

by Laurent GUERBY on Mon Nov 20th, 2006 at 02:53:22 PM EST
[ Parent ]
Yes, sure. It would also be interesting to push the large chains to release the information they collect under the guise of 'bonus cards'.

They not only can attach that data to individuals and thus to demographics, but it is also tracked by the hour.

You can't be me, I'm taken

by Sven Triloqvist on Mon Nov 20th, 2006 at 02:58:31 PM EST
[ Parent ]
I'm wondering how my local mart is reacting to the data that is pouring in now that we have Costco.  Many items are more than 20% cheaper at the new big box than at safeway (a slightly smaller box).  And the service is better so "loyalty" is out the window.

I do know our gas stations are starting to squeal.  At first with Costco at $2.79 they were still holding to $3.09.  Not anymore.  Down to 2.89 and fading.  6-8 big high speed fueling stations has to be really taking a big market share.

by HiD on Tue Nov 21st, 2006 at 04:22:09 AM EST
[ Parent ]
I think it is easier to bypass the arrogant economists at the INSEE into irrelevance than to get them to open their archives.

Those whom the Gods wish to destroy They first make mad. -- Euripides
by Carrie (migeru at eurotrib dot com) on Mon Nov 20th, 2006 at 03:05:45 PM EST
[ Parent ]
I don't know INSEE economists or internal politics so I can't comment.

But I have some (small) hope that if a certain candidate is elected in 2007, a lot of arrogant people in the administration will have to deal with what citizen want and change a bit their habits.

by Laurent GUERBY on Mon Nov 20th, 2006 at 03:20:43 PM EST
[ Parent ]
I just knew you were for Sarkozy. ;)

There's no reason why they shouldn't release price data. It doesn't have to indicate the names of the stores where it was collected. They release plenty of data about citizens from the census -- which (in France at least) is nominative, though no names are made public.

BTW, your experience with BLS and INSEE fits with my feelings about each, from attempting to collate data from their web sites. The BLS is fairly open, INSEE is fairly shut. I think INSEE has never been told to do otherwise. It's true it would be nice to see that change...

by afew (afew(a in a circle)eurotrib_dot_com) on Mon Nov 20th, 2006 at 03:55:50 PM EST
[ Parent ]

I don't see a reason to remove the store name, I don't ask them either to release their data on the spot. After say three monthes would be fine and would protect any hypothetical business interest while preserving usefulness for everyone.

I don't think INSEE is alone here, most (if not all) french administration have no culture of being friendly about releasing their data.

by Laurent GUERBY on Mon Nov 20th, 2006 at 04:17:55 PM EST
[ Parent ]
Fits with the tradition of enlightened despotism.

Those whom the Gods wish to destroy They first make mad. -- Euripides
by Carrie (migeru at eurotrib dot com) on Mon Nov 20th, 2006 at 04:31:40 PM EST
[ Parent ]
With the qualifier that INSEE's remit is actually to provide data... :-)
by afew (afew(a in a circle)eurotrib_dot_com) on Tue Nov 21st, 2006 at 12:03:37 PM EST
[ Parent ]
The fact is, in France, they used to (until they started reading the FT...)

In the long run, we're all dead. John Maynard Keynes
by Jerome a Paris (etg@eurotrib.com) on Mon Nov 20th, 2006 at 04:57:37 PM EST
[ Parent ]
They all want to work for banks or what :)
by Laurent GUERBY on Mon Nov 20th, 2006 at 05:13:40 PM EST
[ Parent ]
From a merchant point of view keeping their prices private puts them at an advantage over the customer.

A store like Walmart knows the price of every item in every store and usually similar things for competitors. They use this information to adjust their prices locally.

A shopper might know a few prices from personal experience, but not in depth. So the store features a low price on items that shoppers use as an index ("loss leaders") and then makes it up on other items.

The internet is putting an end to this game, at least for more expensive items like electronics. Online merchants can offer nothing special except price so many sites now provide price comparison databases. This is partly manipulated by the merchants who must agree to provide the data, but is much more transparent than before. Since the comparison sites usually get a referral fee it is sometimes cheaper to go to various merchants directly and look for a deal. This is especially true with airline tickets. The tradeoff is spending more time online.

As to measuring inflation, the issue is so politically sensitive that it is impossible to get real data from government departments. In the US, for example, the annual adjustment to Social Security depends on the inflation data, so a small change can amount to billions of additional payouts. There is also the issue of how to price improvements in the quality of an item.
If a new computer is 2GHz instead of 1GHz and costs the same is it really "cheaper"?

Policies not Politics
---- Daily Landscape

by rdf (robert.feinman@gmail.com) on Tue Nov 21st, 2006 at 10:58:15 AM EST
There was a commission to discuss inflation computation in the USA (Boskin), but publishing the raw data is the only way to progress on these issues otherwise the debate is totally sterile.

In particular for pensions, I'm pretty sure using overall inflation is meaningless for the standard of living of old people who face a very different inflation.

by Laurent GUERBY on Tue Nov 21st, 2006 at 12:03:45 PM EST
[ Parent ]
This discussion thread in another diary might be of some relevance here:



by Pierre on Tue Nov 21st, 2006 at 04:10:01 PM EST
by Laurent GUERBY on Tue Nov 21st, 2006 at 05:01:52 PM EST
[ Parent ]

Go to: [ European Tribune Homepage : Top of page : Top of comments ]