Exploring the use of scanner data in the Norwegian CPI for products with high churn
Statistics Norway has a clear strategy of making increasingly use of new data sources in official statistics. The national statistical institute (NSI) has a long history of using scanner data in price statistics and in the Consumer Price Index (CPI) in particular. With scanner data we mean aggregated transaction data that provides information on turnover and quantity sold by certain article codes or barcodes. By increasing the use of scanner data in the price index the use of more traditional data sources like web questionnaires filled out manually by retailers can be reduced equivalently, and by that lowering the response burden and increasing index quality.
The calculation method mostly applied in the Norwegian CPI on scanner data is a matched model approach at article code level aggregated by a monthly chained unweighted geometric mean index (Jevons index) – referred to as the “dynamic method” according to Eurostat’s practical guide on scanner data. The method works well on relatively stable article codes like for most supermarket data, but is not appropriate for products with high churn i.e. more frequent changes in article codes, like clothing and consumer electronics for instance. The aim of the present scanner data development work in Statistics Norway is to implement a more generic method that is able to handle frequent changes in article codes. Secondly, a new calculation method should also preferably use both prices and quantities without causing index bias.
New calculation methods like multilateral index methods presented internationally during the last couple of years, do not by themselves solve the problem with product replacements. This issue must therefore be addressed separately. One crucial step is product definition. Defining the product at article code level may in many cases be too detailed especially for products with high churn as the match between new and old article code is lacking. A more appropriate approach could be to apply a broader definition of a product, for instance to combine different article codes of similar attributes of which the consumers are indifferent to. A practical solution is to create these homogenous products (HPs) by clustering together homogenous article codes and calculate a unit value. By calculating a unit value across homogenous article codes we allow for comparisons of new codes entering and old ones disappearing from the market. This paper presents work done in an ongoing grant agreement project and presents challenges related to HP definition and formation as well as effects on different multilateral index formulas.
Reference:
IPS02-003
Session:
Big Data and Consumer Price statistics
Presenter/s:
Ragnhild Nygaard
Presentation type:
Oral presentation
Room:
GASP
Chair:
DJ Hoogerdijk, ESTAT, Luxembourg, (Email)
Date:
Tuesday, 12 March
Time:
16:00 - 17:00
Session times:
16:00 - 17:00