Archived OpenModelica forums. Posting is disabled.

Alternative forums include GitHub discussions or StackOverflow (make sure to read the Stack Overflow rules; you need to have well-formed questions)


Forgot password? | Forgot username? | Register

Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Hello,
I'm trying to find a way to perform a parameter sweep on multiple parameters (like 20), and write a separate csv output file and log file for each simulation. I've looked at the following sample code for a single parameter sweep, but this script only varies 1 parameter.

https://openmodelica.org/doc/OpenModeli … eter-sweep

I'd like to read in a matrix of different input parameters as a test matrix (rows are the run number, columns of input parameters), update the 20 parameters I'm interested by over riding the parameters in the model file I load with an index in the matrix, and then write a separate csv file and log file for each run. I'm looking at about 140 runs and 20 input variables so my test matrix is a [140X20]

Anybody have experience with batch processing a large test matrix by performing a parameter sweep of multiple input parameters for a given run ? In the bounding ball example for a parameter sweep, the parameter e is varied in a for loop with a new results file written for each run iteration of e. I'd like to do this same basic concept, but over ride multiple parameters in each loop. Additionally, I'd like to write a results file and the LOG_NLS file. Thanks in advance for any help

system("./BouncingBall -override=e="+String(value)+" -r=BouncingBall" + String(i) + "_res.mat");

Edited by: jcrabtree - Aug-16-19 18:09:58

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

You would need to create one input file for each run . Of course you could have it as a matrix in a mos-script and create the 140 input files. Then use -overrideFile fileNameXXX where the file contains one line for each parameter on the form PARAMETER=VALUE. It is also possible with -override=p1=1,p2=2,p3=3,...,p20=20

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Hello,
Thanks for the help, but I was wondering if you could take a look at my script to see where I'm going wrong with the system() API script? There isn't great documentation on the API scripting web page for how to override multiple variables. In the attached file, I'm trying to load a file, build the model, read in a testMatrix, replace 20 variables in the model with a single row of the testMatrix (row 1 of test matrix), then simulate and the model using the simulation parameters built saves in the model file. 

I have had all sorts of trouble trying to do this. I've posted an example of the API scripts I've tried to enter into OM Shell to override 20 variables in the model, but I've tried many different combinations of API commands to do this (buildModel(), simulate(), system()). I've received a number of different errors and warnings in trying to do this. Here are some of the issues

1). Simulation output files write to a temp folder on my C drive instead of the working directory I've saved in my model file and specified again using the CD("") command. This happens whenever I loadModel(), and then simulate()
2).  When I run the commands in the attached file, there are no output files (system() returns code 1)


If you could upload a sample .mos file to load a model, build model, override 20 variables, simulate model, and write a csv file to the working directory defined in the model file Id appreciate it....



Override-Variables.txt


Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Thanks. I will look into these example files. I was having problems getting the result and log files to plots due to errors. I've traced this down to command line options I had in my annotation section of the model. When I delete outputFormat = "mat" from the command line options in the annotation section of the model code I get a results file.

Edited by: jcrabtree - Aug-21-19 17:20:51

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Sorry... One more question on this thread. Can anyone explain the difference between buildModel() then System() vs. simulate()? I am looking to do a parameter sweep
using the method shown in the example parameter sweep script below:

https://openmodelica.org/doc/OpenModeli … eter-sweep

I am looking for the results file as a csv, and am also looking for the LOG file at each iteration of the for loop. I am specifically looking for LOG_NLS_RES. When I simulate the model after I load it , I get the results file and the LOG files I'm looking for at each iteration. When I buildModel() and then System() similar the example above, I do not get the log files at each iteration step.

Here is the annotation section of my .mo file:


annotation(
    uses(Buildings(version = "5.1.0"), Modelica(version = "3.2.3")),
    Diagram(coordinateSystem(extent = {{-500, -300}, {500, 300}})),
    Icon(coordinateSystem(extent = {{-500, -300}, {500, 300}})),
    version = "",
    __OpenModelica_commandLineOptions = "--matchingAlgorithm=PFPlusExt --indexReductionMethod=dynamicStateSelection -d=initialization,NLSanalyticJacobian,newInst",
    __OpenModelica_simulationFlags(lv = "LOG_NLS_RES,LOG_STATS", s = "ida", cpu = "()", nls = "hybrid"),
    experiment(StartTime = 0, StopTime = 500, Tolerance = 1e-06, Interval = 1));

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Simulate first calls buildModel and then calls system using the simulation flags.

In your case you have to call system with the simulation flags e.g., `system("./model.exe -lv= \"LOG_NLS_RES,LOG_STATS\"")` otherwise the simulation flags are not used.

Adeel.

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Thanks for the help again. Unfortunately, I tried the following system call (below), and only got errors:

system("./ShoreBasedTestSystem.exe -lv= \"LOG_NLS_RES,LOG_STATS\"")


When I try to run the following for loop in OM Shell, it will create a new folder for every run, build the model, and run the simulation. I will get a result file in the new directory (in CSV format), but I still haven't been able to see any log files. I've tried many different combinations of log file simulation flags, but I have never seen a log file in the new directory I create

for i in 1:140 loop 
  i_str:= String(i);
  mkdir("C:\Users\jcrabtree\Desktop\ShoreBasedDemonstratorTraining\High Fidelity\Run"+i_str);
  cd("C:\Users\jcrabtree\Desktop\ShoreBasedDemonstratorTraining\High Fidelity\Run"+i_str);
  buildModel(ShoreBasedTestSystem,outputFormat="csv");
  system("ShoreBasedTestSystem.exe -nls=hybrid -lv=LOG_EVENTS,LOG_NLS_JAC");
  getErrorString()
end for;

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

I think the logs are send to /dev/null by default. You can specify a log file in the system command:
https://build.openmodelica.org/Document … ystem.html

Code:


system("ShoreBasedTestSystem.exe -nls=hybrid -lv=LOG_EVENTS,LOG_NLS_JAC", "LogFile.log");

Also, is strange that backward slash works (without the need to double it \\), please use forward slash instead for directories:
C:\Users\jcrabtree\Desktop\ShoreBasedDemonstratorTraining\High Fidelity\Run ->
C:/Users/jcrabtree/Desktop/ShoreBasedDemonstratorTraining/High Fidelity/Run

You can also just run build model once and then use -r to generate result files with new names.




Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Works! Thanks you very much for the help in getting me through that!

Re: Simulation Paramenter Sweep - Change Multiple Parameters and write csv

Sorry. Hopefully one final question. I am attempting to override a list of about 20 parameters using the for loop I wrote above. It appears that I'm able to override some parameters, but not all. I get a few different types of warnings when overriding system parameters. Here is the output from the log file:


[b]stdout            | warning | You are overriding CoolingWater.level_start with a small value or zero.
|                 | |       | This could lead to numerically dirty solutions or divisions by zero if not tearingStrictness=veryStrict.
stdout            | warning | You are overriding HeatTransPipeFlux.Q_flow with a small value or zero.
|                 | |       | This could lead to numerically dirty solutions or divisions by zero if not tearingStrictness=veryStrict.
stdout            | warning | You are overriding JacketWater.level_start with a small value or zero.
|                 | |       | This could lead to numerically dirty solutions or divisions by zero if not tearingStrictness=veryStrict.
stdout            | warning | It is not possible to override the following quantity: system.T_ambient
|                 | |       | It seems to be structural, final, protected or evaluated or has a non-constant binding.


I'm confused because none of the parameters I'm overriding are "small value or zero". Also, I'm wondering if I can override system initialization parameters (like system.T_ambient)? When I check my model code I don't believe that this is a protected parameter, but I could be wrong. Any help on what these warnings are all about? In checking the result file it doesn't appear the parameters like system.T_ambient with the warning regarding "structural, final, protected, or evaluated" parameters have been overridden by the values I've selected with my API script.

There are 0 guests and 0 other users also viewing this topic
You are here: