library(simplaceUtil)
We can read a solution xml file via getSolutionFromFile
and get its content as an xml_document
object. From the xml
object we can get the components, user variables and links between
components as dataframes.
<- system.file("solution","Complete.sol.xml",
solfile package="simplaceUtil")
<- getSolutionFromFile(solfile)
sol <- getComponents(sol)
components <- getUserVariables(sol)
variables <- getLinks(sol, components) links
As a shorthand we can use getElementsFromSolutionFile
,
which returns the xml object as well as the dataframes in a list.
<- getElementsFromSolutionFile(solfile)
el <- el$solution
sol <- el$components
components <- el$variables
variables <- el$links links
The components dataframe contains not only sim components, but also interfaces, resources and outputs.
c(1:3,30:31,42:44),c("id","type","subtype","nr")] components[
id | type | subtype | nr | |
---|---|---|---|---|
1 | CURRENT | var | var | 1 |
2 | variables | var | var | 2 |
3 | compare | interface | CSV | 3 |
30 | crop | resource | resource | 30 |
31 | ndemand | resource | resource | 31 |
42 | init_transform | resource | transform | 42 |
43 | DefaultManagement | simcomponent | mgm | 43 |
44 | EvapTranDemand | simcomponent | normal | 44 |
The links dataframe contains the links between resources, simcomponents, outputs, but also dependencies via key variables or via variables used in rules. Relationships between resources and transformers are included too.
c(1:2,23,300,313,322,325),] links[
from | name | to | rel | |
---|---|---|---|---|
1 | CURRENT | DOY | DefaultManagement | value |
2 | phenology | emergencedoy | DefaultManagement | value |
23 | DefaultManagement | DoHarvest | LintulPhenology | value |
300 | weatherfile | - | weather | storage |
313 | soil | - | soil_transform | data |
322 | variables | vSoiltype | soil | key |
325 | DefaultManagement | DoSow | crop | rule |
User variables show the values as defined in the solution. Notice that user variables can be changed when running a solution in project mode.
c(1:5,18:20),] variables[
id | value | unit | datatype | kind | |
---|---|---|---|---|---|
1 | vCLIMATEZONE | 00002 | NA | CHAR | var |
2 | vSoiltype | sandy loam | NA | CHAR | var |
3 | vTempLimit | 35 | NA | DOUBLE | var |
4 | vTempSlopeAboveLimit | 7 | NA | DOUBLE | var |
5 | vBaseLUE | 3.0 | NA | DOUBLE | var |
18 | vCO2StartYear | 2000 | NA | INT | dyn |
19 | startdate | 01.01.1991 | NA | DATE | dyn |
20 | enddate | 31.12.2004 | NA | DATE | dyn |
Filenames for interfaces may use placeholder variables like
_WORKDIR_
or user defined variables
(e.g. vFilename
)
head(components[,c("id","ref")])
id | ref |
---|---|
CURRENT | SYSTEM |
variables | User |
compare | ${WORKDIR}/gk/data/lintul/compare/compare_Complete.csv |
weatherfile | ${vFileName} |
phenologyfile | ${WORKDIR}/gk/data/lintul/LintulPhenologyObservations.csv |
cropfile | ${WORKDIR}/gk/data/lintul/CropPropertiesLintulWaterstress.xml |
With replaceVariablesWithValues
we can replace the
placeholder with the values from the user value dataframe. We can supply
additional values for system variables like _WORKDIR_
.
$ref <- replaceVariablesWithValues(
components$ref,
components
variables, additional=c("_WORKDIR_"="~"))
head(components[,c("id","ref")])
id | ref |
---|---|
CURRENT | SYSTEM |
variables | User |
compare | ~/gk/data/lintul/compare/compare_Complete.csv |
weatherfile | ~/gk/data/lintul/00002Weather.txt |
phenologyfile | ~/gk/data/lintul/LintulPhenologyObservations.csv |
cropfile | ~/gk/data/lintul/CropPropertiesLintulWaterstress.xml |
Subsetting the components dataframe to get only normal components
<- components[components$type=="simcomponent" & components$subtype=="normal",]
ncomp $id ncomp
## [1] "EvapTranDemand" "Co2InfluenceOnLUE"
## [3] "Co2InfluenceOnTranspiration" "LintulPhenology"
## [5] "VernalisationAndPhotoresponse" "SlimRoots"
## [7] "SlimWater" "SlimNitrogen"
## [9] "LintulWaterStress" "LintulPartitioning"
## [11] "LintulBiomass" "NitrogenDemand"
## [13] "SoilCN" "SnowCoverCalculator"
## [15] "STMPsimCalculator"
Getting only links between normal components
<- links[links$from %in% ncomp$id & links$to %in% ncomp$id,]
nlinks head(nlinks)
from | name | to | rel | |
---|---|---|---|---|
12 | LintulBiomass | sLAI | EvapTranDemand | value |
16 | SlimWater | ActualTranspiration | Co2InfluenceOnTranspiration | value |
21 | VernalisationAndPhotoresponse | RTSUM | LintulPhenology | value |
31 | LintulPhenology | sDevStage | VernalisationAndPhotoresponse | value |
44 | STMPsimCalculator | SoilTempArray | SlimRoots | value |
45 | SlimWater | DRYFAC | SlimRoots | value |
Counting the links between components
<- aggregate(name ~ from + to , data=nlinks, length)
nlinks <- nlinks[order(-nlinks$name),]
nlinks head(nlinks,10)
from | to | name | |
---|---|---|---|
23 | SlimWater | SlimNitrogen | 9 |
14 | LintulBiomass | NitrogenDemand | 5 |
3 | LintulPartitioning | LintulBiomass | 4 |
13 | SlimWater | LintulWaterStress | 2 |
22 | SlimRoots | SlimNitrogen | 2 |
30 | SlimRoots | SlimWater | 2 |
31 | LintulBiomass | SnowCoverCalculator | 2 |
34 | LintulBiomass | SoilCN | 2 |
1 | SlimWater | Co2InfluenceOnTranspiration | 1 |
2 | LintulBiomass | EvapTranDemand | 1 |
getSolutionInfoAsDataframe
gives information about the
solutions content (simcomponents) as well as the solution itself
(filename, modification date and possibly simulation experiment
information derived from folder structure).
<- system.file(package="simplaceUtil")
wd <- list.files(wd,pattern=".sol.xml",recursive=TRUE,full.names = TRUE)
solfiles
<- do.call(rbind,
df lapply(solfiles,getSolutionInfoAsDataframe,workdir=wd))
$file <- basename(df$file)
df$type=="simcomponent",c("id","file")] df[df
id | file | |
---|---|---|
43 | DefaultManagement | Complete.sol.xml |
44 | EvapTranDemand | Complete.sol.xml |
45 | Co2InfluenceOnLUE | Complete.sol.xml |
46 | Co2InfluenceOnTranspiration | Complete.sol.xml |
47 | LintulPhenology | Complete.sol.xml |
48 | VernalisationAndPhotoresponse | Complete.sol.xml |
49 | SlimRoots | Complete.sol.xml |
50 | SlimWater | Complete.sol.xml |
51 | SlimNitrogen | Complete.sol.xml |
52 | LintulWaterStress | Complete.sol.xml |
53 | LintulPartitioning | Complete.sol.xml |
54 | LintulBiomass | Complete.sol.xml |
55 | NitrogenDemand | Complete.sol.xml |
56 | SoilCN | Complete.sol.xml |
57 | SnowCoverCalculator | Complete.sol.xml |
58 | STMPsimCalculator | Complete.sol.xml |
78 | DefaultManagement | Yield.sol.xml |
79 | LintulPheno | Yield.sol.xml |
80 | VernalisationAndPhotoresponse | Yield.sol.xml |
81 | LintulPartitioning | Yield.sol.xml |
82 | LintulBiomass | Yield.sol.xml |
A graph showing components and links can be created from a solution file:
<- solutionToGraph(solfile) graph
::render_graph(graph) DiagrammeR
We can restrict the graph to specific component types and the neighborhood of selected components:
<- getNeighborhood(graph, c("SlimWater","SlimRoots"),
subgraph distance=1, type="simcomponent")
::render_graph(subgraph) DiagrammeR
A solution can be modified on the fly to meet requirements for sensitivity analysis or calibration by
<- system.file("solution","Yield.sol.xml",
solfile package="simplaceUtil")
<- getSolutionFromFile(solfile)
sol <- sol |>
newsol removeNonMemoryOutputs() |>
addMemoryOutput("Yields",frequence = "YEARLY") |>
addOutputVariable("Yields","Year","CURRENT.YEAR","INT") |>
addOutputVariable("Yields", "ClimateZone","vClimateZone","CHAR") |>
addOutputVariable("Yields", "Yield","LintulBiomass.sWSO","DOUBLE","MAX") |>
addUserVariable("vLUE",3.0,"DOUBLE") |>
addUserVariable("vSLA",0.02,"DOUBLE") |>
addUserVariable("vRGRL",0.009,"DOUBLE") |>
replaceVariable("crop.LUE","vLUE") |>
replaceVariable("crop.SLA","vSLA") |>
replaceVariable("crop.RGRL","vRGRL")
Let’s output the beginning of the solution as text:
<- getTextFromSolution(newsol) soltext
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE solution PUBLIC "-//SIMPLACE/DTD SOL 1.0//EN" "http://simplace.net/dtd/SimSolution_4_1.dtd">
<solution version="4.1" check="LAZY">
<description title="Yield" author="XXX_XXX" qualitylevel="I">WIKI_START
=== Purpose ===
Example solution for simplaceUtil package.
WIKI_END
* removed non-memory outputs for interfaces yieldout, sumyieldout
* added memory output Yields
* added output variable Year to output Yields
* added output variable ClimateZone to output Yields
* added output variable Yield to output Yields
* added user variable vLUE with value 3
* added user variable vSLA with value 0.02
* added user variable vRGRL with value 0.009
* replaced crop.LUE with vLUE
* replaced crop.SLA with vSLA
* replaced crop.RGRL with vRGRL</description>
<variables>
<var id="vTempLimit" datatype="DOUBLE">35</var>
<var id="vTempSlopeAboveLimit" datatype="DOUBLE">7</var>
<var id="vCropName" datatype="CHAR">winter wheat</var>
<var id="vBaseLUE" datatype="DOUBLE">3.0</var>
<var id="vBaseTempBeforeAnt" datatype="DOUBLE">3.0</var>
<var id="vBaseTempAfterAnt" datatype="DOUBLE">3.0</var>
<var id="startdate" datatype="DATE">01.01.1991</var>
<var id="enddate" datatype="DATE">31.12.2004</var>
<var id="vClimateZone" datatype="CHAR">C02</var>
<var id="vLUE" datatype="DOUBLE">3</var>
<var id="vSLA" datatype="DOUBLE">0.02</var>
<var id="vRGRL" datatype="DOUBLE">0.009</var>
</variables>
<interfaces defaul
Notice that the modifications are logged in the
description
section of the new solution.
The solution can be saved as xml file:
<- "simulation/solution/YieldModified.sol.xml"
solution_modified writeSolutionToFile(newsol, solution_modified)
As the order of SimComponents can have a considerable impact, one can generate random orders of components and run the reordered solutions to analyse the impact.
<- system.file("solution","Yield.sol.xml",
solfile package="simplaceUtil")
<- getSolutionFromFile(solfile)
sol
<- getComponents(sol)
cmp $type=="simcomponent",c("id","subtype","nr")] cmp[cmp
id | subtype | nr | |
---|---|---|---|
14 | DefaultManagement | mgm | 14 |
15 | LintulPheno | normal | 15 |
16 | VernalisationAndPhotoresponse | normal | 16 |
17 | LintulPartitioning | normal | 17 |
18 | LintulBiomass | normal | 18 |
We want to swap the components 2 to 5
<- 2:5
toswap <- sample(toswap, length(toswap))
swapped swapped
## [1] 5 3 4 2
<- swapComponents(sol,swapped)
newsol <- getComponents(newsol)
cmp $type=="simcomponent",c("id","subtype","nr")] cmp[cmp
id | subtype | nr | |
---|---|---|---|
14 | DefaultManagement | mgm | 14 |
15 | LintulBiomass | normal | 15 |
16 | VernalisationAndPhotoresponse | normal | 16 |
17 | LintulPartitioning | normal | 17 |
18 | LintulPheno | normal | 18 |
The order of the SimComponent is now different.
There is a special SimComponent that records the processing time of SimComponents.
To add timing to a solution, one has to
This can be done for every solution automatically via
addTimingSimComponent
.
<- system.file("solution","Yield.sol.xml",
solfile package="simplaceUtil")
<- getSolutionFromFile(solfile)
sol
<- addTimingSimComponent(sol, componentlist=c("LintulBiomass","LintulPheno"))
newsol <- getComponents(newsol)
cmp $type=="simcomponent" | grepl("timing",cmp$id),c("id","type","subtype","nr")] cmp[cmp
id | type | subtype | nr | |
---|---|---|---|---|
8 | automatic_timing_interface | interface | MEMORY | 8 |
15 | DefaultManagement | simcomponent | mgm | 15 |
16 | LintulPheno | simcomponent | normal | 16 |
17 | VernalisationAndPhotoresponse | simcomponent | normal | 17 |
18 | LintulPartitioning | simcomponent | normal | 18 |
19 | LintulBiomass | simcomponent | normal | 19 |
20 | AutomaticTiming | simcomponent | normal | 20 |
23 | automatic_timing_output | output | output | 23 |
The new solution has now an additional SimComponent AutomaticTiming as well as an additional output together with it’s interface.
With parseDate
the standard Date column of Simplace
output files is converted to Date.
<- system.file("output","water.csv",
outfile package="simplaceUtil")
<- read.csv(outfile, header=TRUE, sep=";")
water <- parseDate(water) water
By default the simplace outputs are in wide format, e.g. each layer
is in an own column. We can transform the dataframe from wide format to
long format, by putting values from all layers into one column and
distinguish them by an extra column layer
.
1:4,c(2,9:11,49)] water[
CURRENT.DATE | MobileWater_1 | MobileWater_2 | MobileWater_3 | RetainedWater_1 |
---|---|---|---|---|
1991-07-18 | 0 | 0.5214681 | 1.197492 | 3.402230 |
1991-07-19 | 0 | 0.0000000 | 0.000000 | 2.045055 |
1991-07-20 | 0 | 0.0000000 | 0.000000 | 1.584985 |
1991-07-21 | 0 | 0.0000000 | 0.000000 | 2.906192 |
<- transformLayeredData(water)
water_long c(1:3,41:43,81:83),c(2,9:11)] water_long[
CURRENT.DATE | layer | MobileWater | RetainedWater |
---|---|---|---|
1991-07-18 | 1 | 0.0000000 | 3.402230 |
1991-07-18 | 2 | 0.5214681 | 5.400000 |
1991-07-18 | 3 | 1.1974920 | 5.400000 |
1991-07-19 | 1 | 0.0000000 | 2.045055 |
1991-07-19 | 2 | 0.0000000 | 4.271450 |
1991-07-19 | 3 | 0.0000000 | 5.305210 |
1991-07-20 | 1 | 0.0000000 | 1.584985 |
1991-07-20 | 2 | 0.0000000 | 4.396188 |
1991-07-20 | 3 | 0.0000000 | 5.041876 |
Multiple scalar values can be plotted in a common diagram.
plotScalarOutput(water,"CURRENT.DATE",c("Evaporation","Transpiration"))
Default plot for layered values.
plotLayeredOutput(water,"RetainedWater")
The package includes a Shiny App that runs locally within a webbrowser.
runSimplaceGuiApp()
We can interactively explore solutions by filtering and displaying solution graphs.
We can also use a local Simplace installation to run the simulation defined by the solution and plot some of the output variables.
We can create the interface (location and format) and resource (structure) description for a solution as XML stubs from given input data.
Data can be given in CSV or XML format. The structure of the data has to be compatible with Simplace.
<- createCsvResourceStubs(
stubs filename = system.file("input","weather.csv", package="simplaceUtil"),
id = "weather",
sep =",",
keyvals = c("CURRENT.DATE" = "Date")
)cat(stubs$interface)
## <interface id="weather_file" type="CSV">
## <poolsize>100</poolsize>
## <divider>,</divider
## <filename>/tmp/Rtmp0ywPU1/Rinst1733189a2167a/simplaceUtil/input/weather.csv</filename>
## </interface>
cat(stubs$resource)
## <resource id="weather" interface="weather_file" frequence="DAILY">
## <res id="DOY" datatype="INT", unit=""/>
## <res id="Date" key="CURRENT.DATE" datatype="CHAR", unit=""/>
## <res id="Rain" datatype="DOUBLE", unit=""/>
## <res id="Radiation" datatype="DOUBLE", unit=""/>
## <res id="Tmean" datatype="DOUBLE", unit=""/>
## <res id="RelHum" datatype="DOUBLE", unit=""/>
## </resource>
The function tries to determine the datatype from the data. We have to check and possibly correct the inferred datatype (e.g. replacing ‘CHAR’ by ‘DATE’ or ‘INT’ by ‘DOUBLE’ or ‘BOOLEAN’).
When creating a new SimComponent, we can put all SimVariables (constants, inputs, states, rates, outputs) in a CSV table.
<- system.file("variables","translocation.csv",package="simplaceUtil")
varfile <- read.csv(varfile, header=TRUE, sep=";")
vartable c(4,8:9),] vartable[
contenttype | id | description | datatype | unit | min | max | default | |
---|---|---|---|---|---|---|---|---|
4 | constant | FruitingEfficiency | Number of grains per storage organs mass | Double | g^-1 | 0 | NA | .1 |
8 | state | WEA | Weight of ears | Double | g m^-2 | 0 | NA | 0 |
9 | rate | RWEA | Change rate of ears | Double | g m^-2 d^-1 | NA | NA | 0 |
With the function createCodeStubsForSimVariables
we can
generate so called boilerplate code for Java as well as XML
stubs for solutions.
<- createCodeStubsForSimVariables(varfile, "BMTransloc", sep=";")
stubs names(stubs)
## [1] "JavaFields" "JavaVariables" "JavaInit"
## [4] "ParametersXML" "ResourceParametersXML" "ResourceInputsXML"
## [7] "ComponentInputsXML" "OutputsXML"
When we specify an output folder, the stubs are written to files.
<- createCodeStubsForSimVariables(varfile, "BMTransloc", sep=";",
stubs outfolder="translocation_stubs")
Defining fields in Java (stubs$JavaFields
):
private FWSimVariable<Double> cFruitingEfficiency;
private FWSimVariable<Double> rRWEA;
private FWSimVariable<Double> sWEA;
Creating the SimVariables (stubs$JavaVariables
):
addVariable(FWSimVariable.createSimVariable("cFruitingEfficiency", "Number of grains per storage organs mass", DATA_TYPE.DOUBLE, CONTENT_TYPE.constant, "g^-1", 0d, null, .1d, this));
addVariable(FWSimVariable.createSimVariable("rRWEA", "Change rate of ears", DATA_TYPE.DOUBLE, CONTENT_TYPE.rate, "g m^-2 d^-1", null, null, 0d, this));
addVariable(FWSimVariable.createSimVariable("sWEA", "Weight of ears", DATA_TYPE.DOUBLE, CONTENT_TYPE.state, "g m^-2", 0d, null, 0d, this));
Default initialisation of states and rates (sic!)
(stubs$JavaInit
):
rRWEA.setDefaultValue();
sWEA.setDefaultValue();
XML stub for linking inputs
(stubs$ComponentInputsXML
):
<input id="cTranslocationFraction" source="bmtransloc_parameters.TranslocationFraction" />
<input id="cFruitingEfficiency" source="bmtransloc_parameters.FruitingEfficiency" />
<input id="iEarPartitioningReductionFactor" source="bmtransloc_inputs.EarPartitioningReductionFactor" />
XML stubs for parameter file (stubs$ParametersXML
) and
coresponding resource section in solution
(stubs$ResourceParametersXML
):
<parameter id="EarsPartitioningTableDVS" datatype="DOUBLEARRAY" unit="" description="DVS for fraction of above-ground dry matter to ears">
<value>0</value>
<value>1</value>
<value>2</value>
</parameter>
<parameter id="EarsPartitioningTableFraction" datatype="DOUBLEARRAY" unit="" description="Fraction of above-ground dry matter to ears as function of DVS">
<value>.0</value>
<value>.1</value>
<value>.0</value>
</parameter>
<parameter id="TranslocationFraction" datatype="DOUBLE" unit="" description="Fraction of stems and leaves biomass that is reserved for translocation at anthesis">0.2</parameter>
<parameter id="FruitingEfficiency" datatype="DOUBLE" unit="g^-1" description="Number of grains per storage organs mass">.1</parameter>
<res id="EarsPartitioningTableDVS" datatype="DOUBLEARRAY" unit="" description="DVS for fraction of above-ground dry matter to ears" />
<res id="EarsPartitioningTableFraction" datatype="DOUBLEARRAY" unit="" description="Fraction of above-ground dry matter to ears as function of DVS" />
<res id="TranslocationFraction" datatype="DOUBLE" unit="" description="Fraction of stems and leaves biomass that is reserved for translocation at anthesis" />
<res id="FruitingEfficiency" datatype="DOUBLE" unit="g^-1" description="Number of grains per storage organs mass" />