AutoExpe

Automates repetitive tasks in numerical experiments and the generation of result tables.

This package can be useful if you have:

• 1 problem (e.g., a multiple knapsack problem);
• at least 1 instance of this problem (e.g, weights and values of the objects that can be added in the knapsacks);
• at least 1 resolution method to solve this problem (e.g., these two knapsack methods);
• (optional) parameters of the problem (e.g., the number of knapsacks and the size of the knapsacks ).

and if you want to automate:

• the resolution of each instance by each method for each combination of the values of the parameters;
• the creation of latex result tables.

Here is an example of a table generated from the knapsack example:

To perform an experiment through this package, you must provide:

• the resolution methods;
• the path of the instances;
• the different values of each parameters;
• the format of each result latex table (optional).

The package does the rest which includes:

• apply all resolution methods on all instances for all combination of values of all parameters;
• manage the results: format, reading, writing;
• display logs: progress, errors, warnings, ...;
• interruption management: regular backups, resumption without unecessary recalculations;
• generation of latex result tables during and after the experiment.

The user must create a JSON file which describes the experiment. The experiment can then be executed using the command:

autoExpe("PATH/TO/MY_JSON_FILE.json")

The user describes his experiment in a JSON file which only contains one dictionary. This dictionary must at least contain the two following keys:

• "resolutionMethods" which represents the resolution methods; and
• "instancesPath" which represents the instances.

Here is the content of the JSON file used to describe the experiment of the knapsack example:

{
    "instancesPaths":"./data",
    "resolutionMethods": ["randomResolution", "ratioResolution"],
    "latexFormatPath":  ["./config/averageTable.json", "./config/expandedTable.json"],
    "latexOutputFile": "./results/result_tables.tex",
    "parametersToCombine": {"knapsacksCount": [1, 2, 3], "knapsacksSize": [10, 20, 30]}
}

More details on the format are available here.

The user describes a latex table in a JSON file which contains one array of dictionaries.

Example of the structure of such a file:

[{ TABLE_PARAMETERS },
 { ROW_PARAMETER1 },
 { ROW_PARAMETER2 },
 { RESULT_COLUMN1 },
 { GROUP_OF_COLUMNS }
]

Each dictionary in this array either represents:

1. the table parameters (e.g., caption, vertical lines, ...); or
2. a row parameter: an entry saved in the result files which value will vary across the rows of the table (e.g. : and in the table above);
3. a result column: describes the content of a column of the table which includes results (e.g. : each of the last five columns in the previous table);
4. a group of columns: several columns in the table that will appear under a common label (e.g., the groups Objective and Time in the previous table).

Here is a representation of the structure of the knapsack table:

The format of each of these entries is briefly described below. A more thorough presentation is available here.

As an example, the content of the JSON file used to describe the table above is available here.

This dictionary must contain an entry of key "caption" which represents the caption of the table. For example :

{"caption": "Caption of my table"}

This dictionary must contain a string "rowParameterName" which corresponds to the name of the parameter as it appears in the result files. For example:

{"rowParameterName": "knapsacksSize"}

This dictionary must contain the entries:

• "columnParameters" which specifies the result(s) from the saved files used to obtain the value in the column;
• "method" which specifies the resolution method used to obtain the result(s).

For example:

{"columnParameters": "time", "method": "myMethod"}

If the value in the column is obtained from several results in the result files, "columnParameters" must be an array. For example, if we want to compute the mean of two values "v1" and "v2" both returned by the method "myMethod", we would use:

{
  "columnParameters": ["v1", "v2"],
  "function": "mean",
  method": "myMethod"
}

This dictionary must contain the entries:

• "columnGroupName" which corresponds to the label displayed above the columns of the group;
• "column" which is an array of result columns as defined in the previous section.

For example:

{"columnGroupName": "Time", "column":
  {"columnParameters": "time", "method": "method1"},
  {"columnParameters": "time", "method": "method2"}
}

Currently resolution methods must be implemented in Julia but the package should eventually be able to execute any resolution method from any language. The julia resolution methods must:

• take a single argument of type Dictionary{String, Any}. This input will both provide a value for each parameter and the path of the current instance (which enables the method to access the data related to the instance);
• return a Dictionary{String, Any} which contains all the results to save.

Here is an example inspired from method ratioResolution from knapsack.jl:

function randomResolution(param::Dict{String, Any})

    # Get the value of the parameters from the input
    m = param["knapsacksCount"]
    K = param["knapsacksSize"]

    # Get the path of the instance from the input
    instancePath = param["instancePath"]

    # Read the instance file
    instance = KnapsackInstance(instancePath)

    # ... solve the problem ...

    # Create the output dictionary and fill it with all the results to save
    results = Dict{String, Any}()
    results["resolutionTime"] = resolutionTime
    results["objectiveValue"] = objectiveValue
    results["knapsacksWeight"] = knapsacksWeight
    results["knapsacksContent"] = knapsacksContent

    return results
end