This package has been significantly re-designed as part of a work in progress with Giovanni Compiani and Adam Smith. I have removed the functionality for variable/model selection, but have significantly increased the performance of the package and improved the simplicity of use. The old code has been saved in the copy_old_code
branch, if for some reason that code is useful for you.
Please let me know if you have any suggestions for the package or find any bugs.
The package is not registered. To install, use
pkg> add https://github.com/jamesbrandecon/NPDemand.jl
There are three important functions included here so far: define_problem
, estimate!
, and price_elasticity
:
define_problem(df::DataFrame; exchange = [], index_vars = ["prices"], FE = [], constraints = [], bO = 2, obj_tol = 1e-5, constraint_tol = 1e-5)
: Constructs aproblem::NPDProblem
using the provided problem characteristics. Inputs:exchange
: A vector of (at most two) groups of products which are exchangeable. E.g., with 4 goods, if the first and second are exchangeable and so are the third and fourth, setexchange
= [[1 2], [3 4]].index_vars
: String array listing column names indf
which represent variables that enter the inverted index.FE
: String array listing column names indf
which should be included as fixed effects.obj_tol
: Tolerance specifying the value ofg_abstol
withinOptim.Options
during optimization.constraint_tol
: Tolerance specifying tightness of constraintsconstraints
: A list of symbols of accepted constraints. Currently supported constraints are:- :monotone: Demand is increasing in the index.
- :all_substitutes: All products are substitutes.
- :diagonal_dominance_group: Diagonal dominance (see Compiani, 2022) within exchangeable groups.
- :diagonal_dominance_all: Diagonal dominance across all products.
- :subs_in_group: All products within exchangeable groupings are substitutes.(Note: this constraint is the only available nonlinear constraint and will slow down estimation considerably)
estimate!(problem::NPDProblem; max_iterations=10000, show_trace = false)
: solvesproblem
subject to provided constraints, and replacesproblem.results
with the resulting parameter vector.max_iterations
is passed into Optim.Options for every optimization step.update_constraints!(problem::NPDProblem, new_constraints::Vector{Symbol})
: Replaces the constraints inproblem
withnew_constraints
. Because of the way:exchangeability
is enforced, this function cannot be used to change the structure of exchangeable groupings.price_elasticities!(problem::NPDProblem)
: Takes the solvedproblem
as first argument, aDataFrame
as the second argument, and evaluates all price elasticities in-sample. Currently does not calculate out-of-sample price elasticities. For this, use the functioncompute_demand_function!
.compute_demand_function!(problem, df; max_iter = 1000, show_trace = false)
: estimates the demand function/curve using NPD estimates calculated via estimate!. The function takes in an estimated problem::NPDProblem and a dataframe with counterfactual values of the covariates in the utility function. One must specify all fields that were used in estimation (including shares). The function will change the values of df[!,r"shares"] to take on the value of the estimated demand function. Options:max_iter
: controls the number of iterations for the nonlinear solver calculating market shares. Default is 1000 but well-estimated problems should converge faster.show_trace
: iftrue
, Optim will print the trace for each iteration of the nonlinear solver.
Two helpful additional functions are simulate_logit
and toDataFrame
, which allows one to easily simulate data from a logit demand system with endogenous prices (and instruments for those prices) to test the NPDemand
functions. Usage shown in /examples
.
As described above, begin with a DataFrame which takes the following form (with three products in this example):
julia> df
2000×16 DataFrame
Row │ shares0 prices0 x0 share_iv0 price_iv0 shares1 prices1 x1 share_iv1 price_iv1 shares2 prices2 x2 ⋯
│ Float64 Float64 Float64 Float64 Float64 Float64 Float64 Float64 Float64 Float64 Float64 Float64 Fl ⋯
──────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ 0.328229 0.66807 0.416167 0.416167 0.0717971 0.14538 1.89975 0.903872 0.903872 0.7109 0.236292 0.458123 0. ⋯
2 │ 0.0994119 1.50148 0.367425 0.367425 0.899897 0.25597 1.48679 0.8222 0.8222 0.636302 0.208002 0.834816 0.
3 │ 0.131559 1.60465 0.323835 0.323835 0.596641 0.320453 1.35984 0.84993 0.84993 0.429504 0.175882 0.87676 0.
⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱
1998 │ 0.252616 0.315564 0.456881 0.456881 0.319923 0.33775 0.65299 0.493729 0.493729 0.212751 0.0719814 1.96853 0.
1999 │ 0.193392 1.51716 0.928404 0.928404 0.664692 0.428216 -0.120323 0.501273 0.501273 0.0640289 0.0655636 1.52594 0. ⋯
2000 │ 0.373363 0.476147 0.850132 0.850132 0.398496 0.177178 1.27511 0.374716 0.374716 0.533122 0.0835981 1.69474 0.
4 columns and 1994 rows omitted
Then, we can define a problem
. In this package, a problem
is constructed by combining data (in df
) and constraints. A problem can be constructed simply using define_problem
:
bO = 2; # Order of each dimension of Berntstein polynomial
exchange = [[1 2 3]] # All products are exchangeable
index_vars = ["prices", "x"] # endogenous prices and exogenous x enter the index
constraints = [:exchangeability, :monotone]; # Impose exchangeability and that demand for each product is increasing in the index (decreasing in price)
npd_problem = define_problem(df;
exchange = exchange,
index_vars = index_vars,
constraints = constraints,
bO = bO);
show(npd_problem)
We can then estimate the problem and calculate in-sample price elasticities with two lines.
estimate!(problem)
elasticities, average_elast_mat, shares, all_own_elasts = price_elasticity(npd_problem, df; whichProducts = [1,1]);
Fixed effects are estimated as parameters, not absorbed from the data. So, be careful including fixed effects with too many values, as this may both slow down the optimization and require more memory.
To include fixed effects (categorical variables), use the option FE
to provide a vector of strings, where each element of the vector is a name of a column in the provided data df
. Note however that at present, variables that are included as fixed effect must be constant across products within a market. The only exception to this rule is "product"
which is a keyword which will produce product fixed-effects. There need not be a column named product
in the data, and in fact the code will ignore it if it's there.