Autonomous Vehicle Risk Assessment Framework

@techreport{moss2021autonomous,
  title={{Autonomous Vehicle Risk Assessment}},
  author={Robert J. Moss and Shubh Gupta and Robert Dyro and Karen Leung and Mykel J. Kochenderfer and Grace X. Gao and Marco Pavone and Edward Schmerling and Anthony Corso and Regina Madigan and Matei Stroila and Tim Gibson},
  institution={Stanford Center for AI Safety, Stanford University},
  year={2021}
}

Note: Julia v1.5+ is recommended for AutomotiveSimulator and POMDPStressTesting.

julia install.jl

using RiskSimulator

system = IntelligentDriverModel()
scenario = get_scenario(MERGING)
planner = setup_ast(sut=system, scenario=scenario)

search!(planner)

fail_metrics = failure_metrics(planner)
α = 0.2 # risk tolerance
risk_metrics = metrics(planner, α)
risk = overall_area(planner, α=α)

See adversarial CARLA environment instructions here.

using AVExperiments

config = ExperimentConfig(
    seed=0xC0FFEE,
    agent=WorldOnRails,
    N=100,
    dir="results_wor",
    use_tree_is=true,
    leaf_noise=true,
    resume=false,
)

results = run_carla_experiment(config)

• Adaptive stress testing:
  • https://github.com/sisl/POMDPStressTesting.jl
• Signal temporal logic computation graphs:
  • https://github.com/StanfordASL/stlcg
• Automotive simulator:
  • https://github.com/sisl/AutomotiveSimulator.jl (note AutomotiveDrivingModels.jl is deprecated)
  • Driving visualizations:
    • https://github.com/sisl/AutomotiveVisualization.jl (note AutoViz.jl is deprecated)
• Adversarial driving:
  • https://github.com/sisl/AdversarialDriving.jl
• Interpretability for validation:
  • https://github.com/sisl/InterpretableValidation.jl
  • Defining expression rules (for STL):
    • https://github.com/sisl/ExprRules.jl
• POMDPs framework:
  • https://github.com/JuliaPOMDP/POMDPs.jl

• See PUBLICATIONS.md

• See CARLAIntegration.

• See: https://github.com/invenia/BlueStyle

This error was observed on MacOS and appears to be related to Conda. One solution that worked was to run

using Conda
Conda.rm("mkl")
Conda.add("nomkl")

See JuliaPy/PyPlot.jl#315 for relevant discussions.

Some versions of Python e.g., 3.9 are incompatible with the framework as they do not support packages such as pytorch that are needed. It is possible to switch to a working version of Python as follows:

using Conda
Conda.add("python=3.7.5")

However, note that if you were using an incompatible of Python before, you might have installed Python packages of versions that can conflict with the new compatible version of Python as the packages remain under the Conda directory. You may see error messages like the following if this is the case:

Package enum34 conflicts for:
pyopenssl -> cryptography[version='>=2.8'] -> enum34
cryptography -> enum34
brotlipy -> enum34
urllib3 -> brotlipy[version='>=0.6.0'] -> enum34
pyqt -> enum34

If so, you may need to remove the Conda directory to remove the packages and resintall them after setting Conda to use a correct version of Python i.e.,

rm -R ~/.julia/conda/  # Make sure this is okay to do in your case

then,

using Conda
Conda.add("python=3.7.5")
using RiskSimulator

• Stanford Intelligent Systems Laboratory (SISL)
  • Robert Moss: mossr
  • Kyu-Young Kim: kykim0
• Navigation and Autonomous Vehicles Laboratory (NAV Lab)
  • Shubh Gupta: shubhg1996
• Stanford Autonomous Systems Laboratory (ASL)
  • Karen Leung: karenl7
  • Robert Dyro: rdyro