InventoryManagement.jl:
Discrete-time simulation environment for Inventory Management in Supply Networks.
Table of Contents
- Overview
- Dependencies
- Installation
- Sequence of Events
- Model Assumptions
- Model Limitations
- Inventory replenishment policies
- Model Inputs
- Model Output: SupplyChainEnv
- Examples
- Contact
Overview
InventoryManagement.jl allows modeling a multi-period multi-product supply network. A supply network can be constructed using the following node types:
Producers
: Nodes where inventory transformation takes place (e.g., intermediates or final materials are produced). Reactive systems, including those with co-products, can be modelled using Bills of Materials (see Model Inputs section).Distributors
: Intermediate nodes where inventory is stored and distributed (e.g., distribution centers).Markets
: Nodes where end-customers place final product orders (i.e., retailer). These are the last (sink) nodes in the network.
The simplest network that can be modeled is one with a single market with one producer or distributor. However, more complex systems can be modelled as well.
When defining a supply network, a SupplyChainEnv
object is created based on system inputs and network structure. This object can then be used to execute a simulation of the inventory dynamics. During a simulation, stochastic demand at each of the markets can occur for each of the materials in each period. When product demand occurs at the market, sales are made based on available inventory. Any unfulfilled demand is either backlogged or considered a lost sale depending on the system definition. If no action is taken duirng the simulation, the inventory levels will eventually be depleted. To avoid this from happening, a decision-maker can interact with the system in each period by making inventory replenishment decisions (refered to as actions
). Lead times for in-transit inventory as well as production lead times are accounted for in the simulation. Transportation lead times can be modelled stochastically to account for lead time uncertainty. From a service time perspective, demand at market nodes has zero service time, whereas non-market nodes have service time equal to the production lead time + transportation lead time.
The SupplyChainEnv
can also potentially be used in conjunction with ReinforcementLearning.jl to train a Reinforcement Learning agent
.
This package generalizes and extends and the inventory management environment available in OR-Gym.
Dependencies
InventoryManagement.jl mainly relies on the following packages:
- MetaGraphs.jl: Define supply network structure and specify node- and edge-specific parameters.
- DataFrames.jl: Tabulate results.
- Distributions.jl: Define probability distributions for the lead times in between nodes and the demands at the market nodes.
Installation
The package can be installed with the Julia package manager. From the Julia REPL, type ]
to enter the Pkg
REPL mode and run:
pkg> add InventoryManagement
Sequence of Events
The following sequence of events occurs in each period of the simulation:
- Start period.
- Place inventory replenishment orders at each node. These are limited to the available production capacity (supplier is a producer) or available inventory. If
reallocate = true
, then any amount that cannot be satisfied is reallocated to the next priority supplier.- Distributors ship inventory.
- Producers attempt to satisfy the requested amount in the following order: 1) using any on-hand inventory, 2) using any uncommitted scheduled production (only relevant when there is co-production), and 3) manufacturing material. Materials scheduled for production have a production lead time. Production costs are incurred at the start of production.
- Producers send orders that have completed (after the production lead time).
- Receive inventory that has arrived at each node (after the lead time has transpired).
- Pay suppliers for inventory received.
- Pay shipper for inventory shipped.
- Market demand occurs after tossing a weighted coin with the probability of demand occurring defined by the
demand_frequency
. - Demand (including any backlog if
backlog = true
) is fulfilled up to available inventory at the markets. - Unfulfilled demand is penalized and backlogged (if
backlog = true
). - Each node pays a holding cost and a transportation cost for on-hand inventory and in-transit inventory at each period.
Model Assumptions
The following assumptions hold in the current implementation, but can be modified in future releases.
Producers
produce material on demand (make-to-order policy).Producers
can hold inventory. Downstream replenishment orders are fulfilled first with any on-hand inventory, and then via production only after there is no on-hand inventory left.- Replenishment orders can only be satisfied with current on-hand inventory or available production capacity.
- Commited production orders count towards the inventory position of the downstream node, even if they haven't yet shipped (due to production lead time).
- Production lead times are fixed and independent of the amount being produced.
- Backlogging is only allowed at the
Markets
, it is not allowed for inventory replenishment decisions. - Transportation costs are paid to a third party (not a node in the network).
Model Limitations
The following features are not currently supported:
- Inventory capacity limits at the nodes is not currently supported.
Producers
do not operate under a make-to-stock policy since any material produced gets shipped downstream. However, this can be accomodated by adding a dumby node downstream of theproducer
that holds inventory produced by theproducer
with zero lead time in between the nodes. Thus, using a proper reorder policy, theproducer
can act as amake-to-stock
system that pushes inventory to the inventory holding node.Producers
do not have market demand. However, this can be modelled by adding amarket
node downstream of theproducer
with zero lead time in between the nodes.- Alternate bills of materials (see Model Inputs) for the same material are not currently supported. This is particularly relevant for chemical systems. However, the following workarounds can be done:
- If the alternate reaction pathway has a byproduct, then the main product can be included as a co-product in the bill of materials of the byproduct. For example: A system with 5 materials (
:A - :E
) can have two ways to produce:A
,:B + :C -> :A
and:D -> :A + :E
. The column for material:A
can have the bill of material:[0 -1 -1 0 0]
. The column for material:E
can have the bill of materials:[1 0 0 -1 0]
. However,:A
will only be produced by the second pathway if a request for:E
is made. - Make a copy of the material to specify an alternate pathway. This will require specifying parameters for the copied material throughout the network.
- If the alternate reaction pathway has a byproduct, then the main product can be included as a co-product in the bill of materials of the byproduct. For example: A system with 5 materials (
- Capacity limitations on shared inventory among producer nodes (e.g., shared inventory tanks) cannot be enforced directly. This is because the shared inventory is its own node and feeds the inventory holding area in the producer node. Thus the total inventory is the inventory at the inventory node plus the inventory positions at the producers. Capacity limitations must be enforced manually via the reorder actions (see Kondili Example).
- If a
producer
can produce more than 1 material, it is possible to produce all materials it is capable of producing simultaneously. This does not account for resource constraints (e.g., single reactor can only do reaction 1 or reaction 2, but not both simultaneously). However, these can be enforced manually with the reorder actions (see Kondili Example).
Inventory replenishment policies
At each iteration in the simulation, an action
can be provided to the system, which consists of the replenishment orders placed on every link in the supply network. This action
must be of type Vector{Real}
and must be nonnegative
of the form: [Edge1_Material1, Edge1_Material2, ..., Edge1_MaterialM, Edge2_Material1, ..., Edge2_MaterialM, ..., EdgeE_Material1, ..., EdgeE_MaterialM]
, where the ordering in the edges is given by edges(env.network)
and the ordering in the materials by env.materials
.
An action
vector can be visualized as a DataFrame
using show_action(action, env::SupplyChainEnv)
.
The function reorder_policy
can be used to implement an inventory reorder policy. The two most common policies used in industry are the (s,S)
and (r,Q)
policies.
The reorder_policy
takes the following inputs and returns an action
vector.
env::SupplyChainEnv
: inventory management environmentparam1::Dict
: thes
orr
parameter in each node for each material in the system. Thekeys
are of the form(node, material)
.param2::Dict
: theS
orQ
parameter in each node for each material in the system. Thekeys
are of the form(node, material)
.level::Symbol
::position
if the policy is based on the node's inventory position, or:on_hand
if the policy is based on the node's on-hand inventory level.kind::Symbol
::rQ
for an(r,Q)
policy, or:sS
for an(s,S)
policysupplier_selection::Symbol
: evenly distribute reorder quantities among all suppliers if:random
; otherwise (if:priority
), assign reorder quantities based on supplier priority (e.g., if supplier 1 does not have enough capacity or inventory, then request as much as possible and then request any remaining amount from the next supplier, and so forth).review_period::Int
: number of periods between each inventory review (Default =1
for continuous review. Note: only relevant whenlevel = :on_hand
)
Model Inputs
The supply network topology must be mapped on a network graph using MetaGraphs.jl. The system parameters are stored in the network's metadata.
Node-specific
Producers
will have the following fields in their node metadata:
:initial_inventory::Dict
: initial inventory for each material (keys
):inventory_capacity::Dict
: maximum inventory for each material (keys
):holding_cost::Dict
: unit holding cost for each material (keys
):supplier_priority::Dict
: (only when the node has at least 1 supplier)Vector
of supplier priorities (from high to low) for each material (keys
). When a request cannot be fulfilled due to insufficient productio capacity or on-hand inventory, the system will try to reallocate it to the supplier that is next in line on the priority list (ifenv.reallocate == true
).:production_cost::Dict
: unit production cost for each material (keys
):production_capacity::Dict
: maximum production capacity for each material (keys
).:production_time::Dict
: production lead time for each material (keys
).
Distributors
will have the following fields in their node metadata:
:initial_inventory::Dict
: initial inventory for each material (keys
):inventory_capacity::Dict
: maximum inventory for each material (keys
):holding_cost::Dict
: unit holding cost for each material (keys
):supplier_priority::Dict
:Vector
of supplier priorities (from high to low) for each material (keys
). When a request cannot be fulfilled due to insufficient productio capacity or on-hand inventory, the system will try to reallocate it to the supplier that is next in line on the priority list (ifenv.reallocate == true
).
Markets
will have the following fields in their node metadata:
:initial_inventory::Dict
: initial inventory for each material (keys
):inventory_capacity::Dict
: maximum inventory for each material (keys
):holding_cost::Dict
: unit holding cost for each material (keys
):supplier_priority::Dict
:Vector
of supplier priorities (from high to low) for each material (keys
). When a request cannot be fulfilled due to insufficient productio capacity or on-hand inventory, the system will try to reallocate it to the supplier that is next in line on the priority list (ifenv.reallocate == true
).:demand_distribution::Dict
: probability distributions from Distributions.jl for the market demands for each material (keys
). For deterministic demand, instead of using a probability distribution, use[D]
whereD
is aNumber
.:demand_frequency::Dict
: probability that demand will occur (value between0.0
and1.0
) for each material (keys
):sales_price::Dict
: market sales price for each material (keys
):demand_penalty::Dict
: unit penalty for unsatisfied market demand for each material (keys
)
Edge-specific
All edges have the following fields in their metadata:
:sales_price::Dict
: unit sales price for inventory sent on that edge (from supplier to receiver) for each material (keys
):transportation_cost::Dict
: unit transportation cost per period for inventory in-transit for each material (keys
):lead_time::Distribution{Univariate, Discrete}
: probability distributions from Distributions.jl for the lead times for each material (keys
) on that edge. For deterministic lead times, instead of using a probability distribution, use[L]
whereL
is aNumber
.
General Network
The graph metadata should have the following fields in its metadata:
:materials::Vector
with a list of all materials in the system.:bill_of_materials::Matrix
: bill of materials indicating the production recipies for the materials in the system. The row numbers correspond to the input materials and the column numbers to the output materials. The numbering matches that of thematerials
vector. The magnitude of each element is proportional to the production of one unit of output material. Each element can have one of three types of values:zero
: input not involved in production of output.negative number
: input is consumed in the production of output.positive number
: input is a co-product of the output.
Model Output
A SupplyChainEnv
has the following fields:
network::MetaDiGraph
: Supply Chain Network (metagraph)markets::Array
: list of market nodesproducers::Array
: list of producer nodesdistributors::Array
: list of distribution nodes (excludes end distributors where markets exist)materials::Array
: list of all material (material) names (strings)bill_of_materials::Matrix
square matrix with BOM (rows = input materials, cols = output materials; indices follow materials list; positive value is a co-product, negative is a feedstock)inv_on_hand::DataFrame
: timeseries On Hand Inventory @ each node at the end of each periodinv_pipeline::DataFramet
: timeseries Pipeline Inventory on each edge at the end of each periodinv_position::DataFrame
: timeseries Inventory Position for each node at the end of each periodreplenishments::DataFrame
: timeseries Replenishment orders placed on each edge at the end of each periodshipments::DataFrame
: current shipments and time to arrival for each nodeproduction::DataFrame
: current material production committed to an edge and lead time to ship. Note: byproducts are scheduled to go to the producing noden
(edge(n,n)
).demand::DataFrame
: timeseries with realization of demand, sold units, unfulfilled demand, and backlog at each marketprofit::DataFrame
: timeseries with profit at each nodereward::Float64
: reward in the system (used for RL)period::Int
: period in the simulationnum_periods::Int
: number of periods in the simulationdiscount::Float64
: time discount factor (interest rate)backlog::Bool
: backlogging allowed iftrue
; otherwise, unfulfilled demand is lost salesreallocate::Bool
: the system try to reallocate requests if they cannot be satisfied iftrue
; otherwise, no reallocation is attempted.seed::Int
: random seed
Examples
Example 1
This example is for a 100 period simulation of a supply network with one plant (node 1) that supplies a retailer (node 3), with stochastic demand for product :A
. Node 3, has an alternate supplier, which is a distribution center (node 2). Node 3 prefers replenishing from the plant, which has a lower lead time. A (s,S)
reorder policy is used at the retailer. When the on-hand level for material :A
is depleted at the plant, the plant begins transforming raw material :B
into :A
. There is limited raw material supply at the plant. When the raw material stocks-out, node 3 switches to node 2 for its supply.
See code here.
Example 2
This example is for a 100 period simulation of a supply network with one warehouse (node 1) that supplies a retailer (node 2) with stochastic demand for product :A
. A (r,Q)
reorder policy is used at the retailer every 25 periods.
See code here.
Example 3
This example shows how the simulator can be used for a chemical production system with co-production and material recycle. The system modeled is the batch plant described in Kondili, et al. (1993), which can be modeled as the following supply network:
See code here.
Contact
Author: Hector D. Perez
Position: Ph. D. Candidate @ Carnegie Mellon University
Email: hdperez@cmu.edu
Year: 2021