Expand AU Menu

Economics | Information and Entropy Econometrics

Information and Entropy Econometrics – Theory and Practice

Instructor: Amos Golan, American University

DATES: MAY 16-20, 2005

Objectives and Scope

Information and Entropy Econometrics (IEE) represents a class of methods (within econometrics and statistics) that directly or indirectly builds on the foundations of Information Theory (IT) and the principle of Maximum Entropy (ME). IEE includes research dealing with statistical inference of problems given incomplete knowledge or data, as well as research dealing with the analysis, diagnostics and statistical properties of information measures. A common thread connecting all the IEE estimation methods is the objective of trying to better understand the data, while abstracting away from distributional assumptions or assumptions on the likelihood function.

This class of methods includes the Bayesian Method of Moments (BMOM), the Empirical (or Generalized Empirical) Likelihood (EL/GEL), variations of the Generalized Method of Moments (GMM) and the Generalized ME (GME). All of these methods share the same basic objective of analyzing limited and noisy data using minimal assumptions.

Within the class of IEE methods, the GME is a robust estimation method that is used, primarily, for analyzing finite or limited data sets as well as data sets that are ill-conditioned or ill-behaved. Most economic data fall within these types of ill-behaved data. However, there are many other areas of scientific research where this approach proves to be very useful. Like other IEE methods, the GME uses minimum statistical (distributional) assumptions, performs well under a large class of distributions and is easy to apply and compute.

The primary purpose of this class is to provide the background for understanding both (i) the theory, and (ii) to develop the necessary theoretical and empirical tools for practicing the theory in a wide range of economic/econometric estimation problems. Throughout the course, entropy estimators will be compared with their traditional counterparts and the computational aspects will be discussed and practiced with artificial and real data (your ‘own’ dataset).

Preliminary time schedule

Monday May 16
09.00 - 12.00: Theory
1.00 - 4.30: Computer Lab

Tuesday May 17
09.00 - 12.00: Theory
1.00 - 4.30: Computer Lab

Wednesday May 18
09.00 - 12.00: Theory
1.00 - 4.30: Computer Lab

Thursday May 19
09.00 - 12.00: Theory
1.00 - 4.30: Computer Lab

Friday May 20
09.00 - 12.00: Theory and Guests’ Presentations
1.00 - 2:30: Computer Lab
2:30 - 4:30: Summary and Open Discussion

Content and Topics

Below is a tentative topical outline where the order of topics may change. Contingent on availability of time, and interest, other topics may be included and some topics may not be covered.

Morning sessions: Theory

1. Background, motivation and philosophy

  • The foundations of information theory; 
  • What is Entropy; 
  • The axiomatic and combinatorial derivations of the entropy measure; 
  • The basic problem; 
  • The classical Maximum Entropy (ME) principle and formulation; 
  • The dual (concentrated) formulation; 
  • Basic diagnostics and test-statistics; 
  • Comparison with the standard ML and other estimation methods.

2. Derivation of the basic Generalized ME (GME) method - A simple economic example:

Recovering the unknown coefficients of an Input-Output Table, or a Social Accounting Matrix (matrix balancing). A complete comparison with the traditional methods (e.g., ML) will be developed. Extensions that allow incorporating more variables (e.g., macro/policy) and accommodating for noisy data will be discussed in great detail.

3. The traditional linear statistical model:

  • The basic set-up of the problem; 
  • Basic derivation; 
  • Primal vs. dual formulations; 
  • Diagnostics.

4. A brief comparison of other information-theoretic methods with the GME.

5. Extensions of the linear model to non-scalar identity covariance matrix (e.g., autocorrelation, heteroskedasticity).

6. Presentation of different theoretical and empirical applications where the GME method is used.

7. Special additional topics (to be presented, formulated and discussed if requested by the participants. We can discuss all/some of these topics as well as other topics of interest):

  • Set of equations and simultaneous equations; 
  • Discrete choice models (ordered/unordered); 
  • Censored models; 
  • Model and variable selection; 
  • Linear and non-linear dynamic systems with control.

8. Summary and discussion of possible future directions

Afternoon sessions: Computer Lab and “Hands on Data”

Practical examples and open discussions will take place in the afternoons. In these sessions we:

1. will use the generalized maximum entropy method to evaluate and estimate real world economic problems;
2. exchange/develop relevant software; and
3. discuss philosophical, practical, technical issues.

The main software package will be GAMS, but some codes will be available in other software packages (SAS and LIMDEP). In addition, some suggested computational problems (and solutions) will be provided.

Target Group and Requirements

The course may be of interest to

1. PhD students interested in new methods of estimation. Students from American University and other universities are welcome.
2. Faculty, professional economists, researchers and econometricians who work in support of decision making in government agencies as well as the private market.

NOTE: Participants should have prior knowledge at the level of an introductory course econometrics or a course in statistical analysis (estimation techniques) at the PhD level.


The course can be taken for three credits or for no credit. To receive the full three credits, the participant needs to complete a research paper. Credits can only be obtained by writing the applied paper. Students can start working on the paper at the end of the course. The paper is due a number of weeks after the end of the sessions (to be submitted to Golan).


  • The main text is Golan, A., G.G. Judge, and D. Miller (1996), Maximum Entropy Econometrics: Robust Estimation with Limited Data, New York: John Wiley & Sons. 
  • A Reader composed of the main reading for the class will be provided to each participant. 
  • A detailed reading list will be provided with the Final Syllabus. Handouts for SAS (and LIMDEP) will be provided as well.


Three Credit costs for students.
Fixed fee (zero credits) for Researchers.


American University, Washington, D.C.


Please look at: http://www.american.edu/american/registrar/

About the instructor

Amos Golan is a professor in the Department of Economics at American University. He is an econometrician specialized in information and entropy econometrics with a special interest in processing and estimating incomplete, ill-behaved and/or non-experimental data. His work is both on the theoretical and applied level. He is the senior author of Maximum Entropy Econometrics, Guest Editor for the Journal of Econometrics volumes on Information and Entropy Econometrics and he is currently on the editorial boards of Econometric Reviews.