Top Navigation Bar

Article Request Page ASABE Journal Article

Analysis of Parameter Sensitivity and Identifiability of Root Zone Water Quality Model (RZWQM) for Dryland Sugarbeet Modeling

M. J. Anar, Z. Lin, L. Ma, P. N. S. Bartling, J. M. Teboh, M. Ostl


Published in Transactions of the ASABE 60(6): 1995-2010 (doi: 10.13031/trans.12313). 2017 American Society of Agricultural and Biological Engineers.


Submitted for review in February 2017 as manuscript number NRES 12313; approved for publication as part of the “Crop Modeling and Decision Support for Optimizing Use of Limited Water” collection by the Natural Resources & Environmental Systems Community of ASABE in June 2017.

Mention of company or trade names is for description only and does not imply endorsement by the USDA. The USDA is an equal opportunity provider and employer.

The authors are Mohammad J. Anar, ASABE Member, Graduate Student, and Zhulu Lin, ASABE Member, Assistant Professor, Department of Agricultural and Biosystems Engineering, North Dakota State University, Fargo, North Dakota; Liwang Ma, Research Soil Scientist, and Patricia N. S. Bartling, Agronomist, USDA-ARS Rangeland Resources and Systems Research Unit, Fort Collins, Colorado; Jasper M. Teboh, Soil Scientist, and Michael Ostlie, Agronomist, Carrington Research Extension Center, Carrington, North Dakota. Corresponding author: Zhulu Lin, North Dakota State University, Dept. 7620, P.O. Box 6050, Fargo, ND 58108-6050; phone: 701-231-7118; e-mail: zhulu.lin @ndsu.edu.

Abstract.  Sugarbeet is being considered as one of the most viable feedstock alternatives to corn for biofuel production since herbicide-resistant energy beets were deregulated by the USDA in 2012. Growing sugarbeets for biofuel production may have significant impacts on soil health and water quality in the north-central regions of the U.S., where 50% of the nation’s total sugarbeets were produced in 2015. Almost all the current sugarbeet models simulate only plant growth and yield but have no capability to simulate the effects of sugarbeet production on soil and water quality. The Root Zone Water Quality Model (RZWQM) is a widely used model that simulates crop yield, water flow, and transport of salts and nitrogen in crop fields. RZWQM is currently linked to 23 specific crop models in the Decision Support System for Agrotechnology Transfer (DSSAT) version 4.0, not including a sugarbeet model. In this study, the Crop and Environment REsource Synthesis (CERES) in RZWQM was adapted for sugarbeet simulation to model the soil and water quality impact of sugarbeet for biofuel production. The Beet model was then evaluated against dryland sugarbeet production at the Carrington Research and Extension Station (North Dakota) in 2014 and 2015. The PEST (Parameter ESTimation) tool in RZWQM was used for parameter estimation and sensitivity and identifiability analysis. The model did reasonably well in both 2014 (d-statistic = 0.709 to 0.992; rRMSE = 0.066 to 1.211) and 2015 (d-statistic = 0.733 to 0.990; rRMSE = 0.043 to 0.930) in terms of simulating leaf area index, top weight, root weight, soil water content, and soil nitrates. Under dry conditions, the most sensitive soil parameters were soil bulk densities and saturated hydraulic conductivities in different layers. Identifiability analysis also showed that three to five model parameters may be identifiable by calibration datasets. RZWQM enhanced with a sugarbeet module and its parameter analysis can be used for water use optimization under dryland conditions.

Keywords.Biofuels, CERES, DSSAT, RZWQM, Sugarbeet.

Biofuel is defined as any fuel source that is derived from biomass and can be used to produce heat, electricity, or transportation fuel (Wang et al., 2011). Based on their potential to reduce net greenhouse gas (GHG) emission, the Energy Independence and Security Act (EISA) of 2007 classified biofuels into three categories called conventional, advanced, and cellulosic biofuels, offering 20%, 50%, and 60% reduction in GHG emission, respectively. Currently, 97% of biofuels produced in the U.S. are corn-based ethanol, which may offer up to 40% reduction in GHG emission when compared with gasoline on an equivalent energy basis (Hettinga et al., 2009; Wang et al., 2011; Canter et al., 2016; Flugge et al., 2017). Two crops, sugarbeet and sugarcane, are currently considered to be uniquely qualified as “advanced biofuels” under the EISA of 2007 (Jessen, 2011). In addition, compared to corn, the use of non-food grade sugarbeets (or “energy beets”) for biofuel production has less impact on the food supply (Maung and Gustafson, 2011; Nahar and Pryor, 2013; Vargas-Ramirez et al., 2013).

Sugarbeet is grown in a wide range of temperate climatic conditions and in a wide variety of soils, ranging from sandy to clay, silty clay, or silty clay loam with high organic matter and/or high clay content (Cattanach, 1991). In the U.S., sugarbeet is grown in eleven states spread across four regions: Michigan in the Great Lakes region; Minnesota and North Dakota in the Upper Midwest region; Colorado, Montana, Nebraska, and Wyoming in the Great Plains region; and California, Idaho, Oregon, and Washington in the Far West region (USDA-ERS, 2016). In 2015, about 50% of the nation’s total sugarbeets were produced in the Red River Valley of western Minnesota and eastern North Dakota and its vicinity, while another 34% were harvested in Idaho and Michigan (USDA-ERS, 2016).

It is reported that sugarbeet is the most utilized sucrose-containing feedstock for commercial biofuel production in European countries (Grahovac et al., 2011; Nahar and Pryor, 2013; Vargas-Ramirez et al., 2013). In contrast, there is no history of biofuel production from sugarbeets in the U.S. Hence, tremendous opportunities exist to expand sugarbeet production into nontraditional sugarbeet planting areas in the U.S., and models can be very useful in understanding sugarbeet growth processes in nontraditional planting areas and their effect on soil health and water quality.

A number of crop models have been developed to describe sugarbeet’s growth and yield production. Models based on empirical relationships include PIEteR (Biemond et al., 1989; Smit et al., 1993), LUTIL (Spitters et al., 1989, 1990), and the model developed by Modig (1992). Examples of process-based models include SUBGRO (Fick et al., 1971), SUBGOL (Hunt, 1974), SUCROS (Spitters et al., 1989), CERES-Beet (Leviel, 2000), Broom’s Barn (Qi et al., 2005), Green Lab (Vos et al., 2007), Pilote (Taky, 2008), and the model developed by Webb et al. (1997), etc. Excellent reviews of sugarbeet models were provided by Vandendriessche and Ittersum (1995) and Baey et al. (2014). However, all these models, except SIMBEET (Lee, 1983), simulate only plant growth and yield of sugarbeet and do not simulate agricultural management effects on soil and water quality (see also Ma et al., 2012). One approach is to incorporate a sugarbeet growth model into the Root Zone Water Quality Model (RZWQM) to study the plant-soil-water interactions in sugarbeet fields.

RZWQM, developed by the USDA-ARS, is a process-based, one-dimensional, simulation model based on the knowledge acquired of the physical, chemical, and biological processes in the root zone. It has been widely used for simulating agricultural management effects on crop production and soil health and water quality (Jaynes and Miller, 1999; Ahuja et al., 2000; Saseendran et al., 2007; Malone et al., 2010; Ma et al., 2012). RZWQM2 is a significant upgrade from the earlier version of RZWQM (Ma et al., 2012). It incorporates surface energy balance from the SHAW (Simultaneous Heat and Water) model (Flerchinger et al., 2012) and the crop-specific growth modules from DSSAT (Decision Support System for Agrotechnology Transfer) (Jones et al., 2003). However, it currently does not include a sugarbeet module. A new CSM (Crop System Module)-CERES-Beet model has been recently incorporated into DSSAT by adopting CERES-Beet (Leviel, 2000; Anar et al., 2015), which can be readily linked to RZWQM2 (Anar and Lin, 2016). Therefore, the objectives of this study are (1) to calibrate and validate RZWQM2 for modeling crop growth and soil water and nitrate contents in dryland sugarbeet fields, and (2) to evaluate the sensitivity and identifiability of RZWQM2 model parameters related to sugarbeet modeling using the Parameter ESTimation (PEST) software (Doherty, 2005, 2010, 2016a, 2016b; Necpálová et al., 2015).

Materials and Methods

Two approaches may be taken to develop a new crop growth module in RZWQM2 for sugarbeet. One is to parameterize the generic plant growth module in RZWQM2 for sugarbeet. The other is to develop a crop-specific plant growth module for sugarbeet in DSSAT, which, in turn, may be linked to the plant growth module of RZWQM2. Because no model was available in DSSAT for sugarbeet, Anar et al. (2015) modified and incorporated CERES-Beet (Leviel, 2000) into DSSAT 4.6.1, and the resultant sugarbeet model is termed CSM-CERES-Beet. Baey et al. (2014) showed that CERES-Beet provided overall good predictions of plant growth and yield for sugarbeets after comparing CERES-Beet with four other sugarbeet models: GreenLab (Vos et al., 2007), LNAS (Cournede et al., 2013), STICS (Brisson et al., 1998), and Pilote (Taky, 2008).

CSM-CERES-Beet is a daily time step, process-based model that simulates a number of processes, such as phenological development; growth of leaves, stems, and roots; biomass accumulation and partitioning; soil water and nitrogen transformations; and nitrogen uptake and partitioning among plant components (Leviel, 2000). CSM-CERES-Beet considers sugarbeet as an annual crop for beet production purposes and classifies the phenology into four events: sowing, germination, emergence, and harvest. In CSM-CERES-Beet, crop growth stages are distinguished based on degree-day threshold parameters (i.e., P1, P5, PHINT) with a base temperature of 3°C. During the early growth stages, 15% to 40% of the daily dry matter produced is partitioned into root. After the full canopy development, 85% of the daily dry matter produced is partitioned into root tuber formation (Milford et al., 1988; Leviel, 2000). Final marketable sugarbeet yield is computed from total root dry matter, assuming that 95% of the root is harvested and that roots have 82% moisture content (Leviel, 2000). CSM-CERES-Beet was calibrated and validated using PEST (Doherty, 2016a, 2016b) against two sets of plant growth data collected for different sugarbeet varieties grown in two different regions: one in Romania and the other in North Dakota (see also Anar et al., 2015). CSM-CERES-Beet was then readily linked to RZWQM2 (ver. 4.0) to model sugarbeet production and its impact on soil and water quality.

Field Experiment

Field experiments for dryland sugarbeet cultivation were conducted at the Carrington Research Extension Center (CREC) in Carrington, North Dakota, (47.510° N, -99.123° W). CREC is located outside of the Red River Valley and is considered a nontraditional sugarbeet planting area. A specific cultivar of sugarbeet bred for biofuel purposes was cultivated in rotation with wheat, corn, and soybeans (not shown) in a randomized complete block design with four replicates for testing the effects of crop rotation and tillage on soil health and water quality. A total of twelve plots with dimensions of 12.19 m × 15.24 m (40 ft × 50 ft) were cultivated for dryland sugarbeet production. Soils of the experimental plots were loamy with an average pH of 7.0. Plots that were used for sugarbeet cultivation are shown in figure 1. Plots planted with sugarbeet in 2014 are shown with upward slanted fill, whereas those in 2015 are shown with downward slanted fill. Plots with horizontal fill were planted with sugarbeet in both 2014 and 2015. Field management data and soil profiles are provided in tables 1 and 2. Soil texture was determined using the hydrometer method, while soil organic matter (OM) content was determined by loss of weight on ignition at 360°C, and the salts were determined with conductivity meter in a 1:1 soil:water suspension. All lab analyses were conducted at the Agvise Laboratories (Northwood, N.D.). The weather data required to run RZWQM2 were collected from a North Dakota Agricultural Weather Network (NDAWN) station located at Carrington, North Dakota (47.509° N, -99.132° W). The required weather files (.met, .brk, and .sno) were then generated using RZWQM2’s weather generation wizard.

Figure 1. Schematic of field experimental plots planted with sugarbeet in 2014 (upward slanted fill), 2015 (downward slanted fill), and both 2014 and 2015 (horizontal fill).

Each year, six of the twelve plots were randomly selected to collect plant growth data. In each plot, one sugarbeet plant was harvested for sampling of leaves, stems, and roots periodically. Both fresh weight and dry weight of the samples were measured. Leaf area index (LAI) was measured using the indirect ground-based measurement method based on radiative transfer theory (Breda, 2003). Soil water content (SWC) and soil nitrate concentration data were also collected from a number of different plots. Five plots in 2014 and eight plots in 2015 were selected for SWC and soil nitrate data collection. Soil water contents in four different soil layers (0-15, 15-30, 30-45, and 45-60 cm) were measured using in situ neutron probes (Troxler Electronic Laboratories, Research Triangle Park, N.C.). Soil samples were also analyzed in the laboratory periodically for soil profile nitrate concentrations in the four different layers.

Parameter Estimation

Nonlinear regression methods as implemented in PEST were used to estimate model parameters of RZWQM2 for sugarbeet modeling. The nonlinear regression method involves estimation of model parameters by minimizing an objective function using iterative optimization. The process ends when the objective function reaches a minimum value (Doherty, 2010). The objective function is expressed in general form as (Doherty, 2010):

(1)

Table 1. Field management for sugarbeet experimental plots at the Carrington Research Extension Center, North Dakota.
Field Management20142015
Planting date27 May1 June
Planting density98,842 seeds ha-1
(40,000 seeds ac-1)
122, 932 seeds ha-1
(49,749 seeds ha-1)
FertilizerN112.08 kg ha-1
(100 lb ac-1)
112.08 kg ha-1
(100 lb ac-1)
P22.42 kg ha-1
(20 lb ac-1)
22.42 kg ha-1
(20 lb ac-1)
S11.21 kg ha-1
(10 lb ac-1)
11.21 kg ha-1
(10 lb ac-1)
Fertilizer application date26 May31 May
Harvest date17 Oct.17 Oct.
Table 2. Average soil characteristics of the experimental plots at the Carrington Research Extension Center, North Dakota.
Depth
(cm)
Sand
(%)
Silt
(%)
Clay
(%)
Soil TypeOM
(%)
Salts
(mmhos cm-1)
0-15453421Loam4.00.16
15-30473617Loam3.60.25
30-45492823Loam--
45-60532819Sandy loam--
60-120652510Sandy loam--

where Q is a weight matrix, y is a vector of observations, y'(b) is a vector of simulated values produced by the model based on parameter vector b, and T indicates matrix transpose. Both y and y'(b) should have the same dimension. Parameters that minimize this equation are attained by solving the normal equations using the Gauss-Marquardt-Levenberg (GML) gradient search algorithm (Doherty, 2010).

Table 4. RZWQM2 parameters adjusted by PEST for sugarbeet modeling.
ParameterDefinitionUnitDepth
(cm)
Initial
Value
Lower
Bound
Upper
Bound
Estimated
Value
BD1Bulk densityg cm-30-151.531121.438
BD2Bulk densityg cm-315-301.084121.091
BD3Bulk densityg cm-330-451.102121.106
BD4Bulk densityg cm-345-601.000121.00
BD5Bulk densityg cm-360-901.793121.873
Ks1Saturated hydraulic conductivitycm h-10-1511201.18
Ks2Saturated hydraulic conductivitycm h-115-3011201.04
Ks3Saturated hydraulic conductivitycm h-130-4521203
Ks4Saturated hydraulic conductivitycm h-145-6021203
Ks5Saturated hydraulic conductivitycm h-160-9021203
P1Growing degree days (GDD) from seedling emergence
to end of juvenile phase
°C d-9509501100970
P2Photoperiod sensitivity--0.0010.000.010.001
P5Thermal time from panicle initiation to physiological maturity°C d-700660900700
G2Maximum possible seed growth number--9007001000900
G3Seed filling rate during the linear vegetative filling stagemg seed-1 d-1-5.511005.5
PHINTPhyllochron interval, the interval in thermal time
between successive leaf tip appearances
°C d-38.9384938.9
RUERadiation use efficiencyg MJ-1-2.82.84.23.3
PARSRPhotosynthetically active solar radiationMJ m-2 d-1-0.480.460.520.52
SDSZMaximum potential seed sizemg seed-1-0.2750.250.300.275
RSGRRelative seed growth rate below which plant may mature earlymg d-1-0.100.100.200.10
RSGRTNumber of consecutive days relative seed growth rate
is below RSGR that triggers early maturity
d-10.0121
CARBOTNumber of consecutive days CARBO is less <0.001 before plant
matures due to temperature, water, or nitrogen stress
d-7587
DSGTMaximum days from sowing to germination before seed diesd-40354540
DGETGrowing degree days between germination and emergence
after which the seed dies due to drought
°C d-150140160150
SWCGMinimum available soil water required for seed germinationcm3 cm-3-0.020.010.040.02
PORMMinimum porosity required for supplying oxygen to roots
for optimum growth
--0.050.010.100.04
RLWRRoot length to weight ratio--0.820.821.820.84

The calibration dataset comprises 65 observations divided into five observation groups: LAI (4), top weight (4), root weight (5), soil water content (32), and soil profile NO3--N content (20) (table 3). On a given sampling date, the field observation for each group was taken as the average of the data collected. Each observation group formed a component of the objective function (eq. 1). An intergroup weighting strategy was defined using the PEST utility PWTADJ1 (Doherty and Welter, 2010) such that all the groups contributed equally to the objective function at the beginning of the estimation process, irrespective of the number of observations per group, units of measurement, and other confounding factors.

Table 3. Field observations included in RZWQM2 calibration and validation.
Observation Group
(unit)
Data SourceNo. of Observations
20142015
Leaf area index
(unitless)
Ground-based measurement48
Top weight
(kg ha-1)
Harvested top plant parts48
Root weight
(kg ha-1)
Harvested root58
Soil water content
(cm3 cm-3)
Neutron probe readings3224
Soil profile nitrate
(µg g-1)
Laboratory analysis2020

The 27 parameters to be adjusted by PEST were selected based on prior sensitivity analyses of the model (table 4). These adjustable parameters are anticipated to affect sugar-beet growth, soil water content, and nitrate concentrations in soils. For these 27 adjustable parameters, initial values as well as lower and upper bounds were specified based on a literature review. All the adjustable parameters were log-transformed to strengthen the linear relationships between parameters and model-simulated values (Doherty and Hunt, 2010). The truncated singular value decomposition (SVD) regularization method was used to ensure numerical stability, and the level of truncation was calculated automatically based on a stability criterion of maximum likelihood combination of the parameters consistent with observations (Aster et al., 2005; Moore and Doherty, 2005; Tonkin and Doherty, 2005; Nolan et al., 2011).

Parameter Correlation, Sensitivity, and Identifiability

Pre-calibration parameter correlations were obtained from the correlation coefficient matrix by employing a standard GML parameter estimation method implemented in PEST. The relative composite sensitivity of each parameter with respect to each observation group and the entire calibration dataset at the beginning of the parameter estimation process was calculated based on the magnitude of the column of the Jacobian matrix corresponding to the ith parameter with each entry in that column multiplied by the squared weight associated with that observation group and the absolute value of that parameter using equation 2:

(2)

where J is the Jacobian matrix, Q is the diagonal matrix whose elements are comprised of the squared weights of the observation, and |vi| is the absolute value of the parameter. The sensitivities of a parameter represent the amount of change in the model-simulated values per unit change in a parameter’s value (Poeter and Hill, 1997).

Parameter identifiability represents the calibration dataset’s ability to constrain model parameters (Doherty and Hunt, 2009), and it is usually obtained through SVD of the weighted Jacobian matrix calculated based on initial parameter values (Necpálová et al., 2015). The premise is that the parameter space of a model can be properly decomposed into an orthogonal calibration solution space and calibration null space (Moore and Doherty, 2005). The calibration solution space is a subset of the parameter space comprising combinations of parameters that can be estimated uniquely by the calibration dataset, whereas the calibration null space can be thought of as combinations of parameters that cannot be estimated by the calibration dataset (Doherty and Hunt, 2009).

Therefore, the identifiability of a parameter describes the degree to which that parameter can be determined uniquely by relating the contributions of each adjustable parameter to any of the eigenvectors spanning the calibration solution space. Because eigenvectors are normalized, the largest value of a parameter’s contribution to an eigenvector is 1.0. Parameters with low identifiability cannot be estimated because they have a large projection onto the calibration null space, due to correlation with other parameters or low sensitivity to all observations. In contrast, parameters with an identifiability value of 1.0 can be uniquely estimated because they are entirely projected onto the calibration solution space. The identifiability of the ith parameter is calculated as the sum of the squared ith components of all eigenvectors spanning the calibration solution space (Doherty, 2010).

In this study, the boundary between the calibration solution and null spaces was set at a specific singular value calculated using the SUPCALC utility (Doherty and Hunt, 2009; Doherty, 2016b). PEST utility IDENTPAR was then used to compute the parameter identifiability for each of the observations (Doherty, 2016b). The number of singular vectors used to compute identifiability differed between the observation groups from 4 to 11 by means of the different number of field observations. Parameters with identifiability greater than 0.7 were considered to be identifiable with the available calibration dataset (Nolan et al., 2011; Necpálová et al., 2015).

Model Evaluation

Best parameters obtained from inverse modeling using the 2014 dataset were validated with data from 2015. We calculated both relative root mean square error (rRMSE) and index of agreement (d) as indicators of goodness of fit. The rRMSE is the root mean square error normalized to the mean of the observed values:

(3)

where m is the number of observations, is the mean of the observed values, yi' is the model-simulated value, and yi is the observed value. The index of agreement is estimated using the following equation:

(4)

The index of agreement is more sensitive than traditional correlation measures to differences between observed and simulated means and variances. The value of d varies between 0 and 1, with higher values indicating better fit (Legates and McCabe, 1999).

Results and Discussion

The sugarbeet module in RZWQM2 was calibrated using 2014 field data collected at CREC. The PEST optimization required nine optimization iterations and 433 model calls to minimize the objective functions. The number of singular values used in SVD ranged from 9 to 20 on an iteration-by-iteration basis, based on a stability criterion. The total objective function was decreased by 34.2%. Table 5 summarizes the measures of the goodness of model prediction to the observations of crop growth, SWC, and soil nitrate content in 2014 and 2015. In general, the model did very well in terms of both d-statistic and rRMSE. The d-statistic ranged from 0.709 to 0.992 for model calibration and from 0.733 to 0.990 for model validation, while the rRMSE took values of 0.066 to 1.211 for model calibration and 0.043 to 0.930 for model validation.

Figure 3. Model-simulated and observed values of (a) leaf area index, (b) top weight, and (c) root weight for model validation in 2015. Observed values are plotted in boxplots with medians shown as lines within the boxes, 25th and 75th percentiles as the tops and bottoms of the boxes, 5% and 95% percentiles as whiskers below and above the boxes, and plus signs (+) as outliers.
Table 5. RZWQM2 calibration and validation results for individual observation groups.
Observation GroupIndex of Agreement (d)Relative Root Mean Square Error (rRMSE)
Model Calibration
2014
Model Validation
2015
Model Calibration
2014
Model Validation
2015
Leaf area index0.9600.8910.3450.464
Top weight0.9770.8770.2390.507
Root weight0.9330.8850.2040.735
SWC (0-15 cm)0.8940.8630.1390.193
SWC (15-30 cm)0.9740.9450.0660.043
SWC (30-45 cm)0.8260.9890.1320.044
SWC (45-60 cm)0.7090.9250.1790.088
Soil profile nitrate0.9920.9900.2030.214
Soil nitrate (0-15 cm)0.8810.8950.4570.573
Soil nitrate (15-30 cm)0.9260.9030.8560.930
Soil nitrate (30-45 cm)0.9250.8020.6500.715
Soil nitrate (45-60 cm)0.8250.7331.2110.710

Plant Growth

Simulation of plant growth in a water quality model is important because it affects the hydrology and chemical uptake in a plant-soil-water system. For this reason, plant growth variables (LAI, top weight, and root weight) were first calibrated against 2014 field observations and then validated against 2015 field observations. In 2014, the model showed a good fit for LAI, top weight, and root weight by closely tracking the medians of the observed values (fig. 2). In 2015, the model’s performance was less than ideal when compared with the observed values (fig. 3). The model consistently overpredicted the observed values between the 67th and 100th days after planting for LAI, top weight, and root weight (fig. 3). This might be due to a strong wind gust (~22.5 m s-1) occurring around the 65th day after planting (28-29 July 2015). This less than satisfactory model performance in 2015 was also reflected in the model evaluation statistics (table 5). CSM-CERES-Beet is not designed to simulate the damage caused by unexpected events, such as strong wind gusts in the early plant development stage or freezing temperature close to harvesting. These limitations were also discussed by Leviel (2000).

The calibrated model was also used to simulate the average sugarbeet root yields in 2014 and 2015 (fig. 4). In both years, the average observed yields along with their standard deviations were computed from the yields from all twelve sugarbeet plots (fig. 1). For the model-simulated root yields, the dry weights output by the model were converted to fresh yields assuming 82% moisture content in the beets. Figure 4 shows that RZWQM2 did very well in simulating the average observed yields of sugarbeet in both years.

Figure 2. Model-simulated and observed values of (a) leaf area index, (b) top weight, and (c) root weight for model calibration in 2014. Observed values are plotted in boxplots with medians shown as lines within the boxes, 25th and 75th percentiles as the tops and bottoms of the boxes, 5% and 95% percentiles as whiskers below and above the boxes, and plus signs (+) as outliers.
Figure 4. Model-simulated and average observed root yields of sugarbeet planted at the Carrington Research and Extension Center, North Dakota. Short vertical lines above the average observed yields are standard deviations.
Figure 5. Soil water content at different soil depths of (a) 0-15 cm, (b) 15-30 cm, (c) 30-45 cm, and (d) 45-60 cm in 2014. Observed values are plotted in boxplots with medians shown as lines within the boxes, 25th and 75th percentiles as the tops and bottoms of the boxes, and 5% and 95% percentiles as whiskers below and above the boxes.

Soil Water Content and Soil Nitrate

Observed and simulated soil water content at four soil depths down to 60 cm were plotted for 2014 (fig. 5) and 2015 (fig. 6). In 2014, soil water content was measured for five plots on the first sampling date and for two plots at other times. In 2015, soil water content readings were taken from eight plots throughout the year.

Figure 5 shows that the model-simulated soil water con-tent in the top two soil layers (0-15 and 15-30 cm) followed the trends of observed soil water content very well over the entire 2014 growing season (figs. 5a and 5b). The d-statistics for these two layers were 0.894 and 0.974 and the rRMSE values were 0.139 and 0.066, respectively (table 5). However, in the deeper two layers (30-45 and 45-60 cm), RZWQM2 overpredicted SWC in the early part of the growing season and underpredicted SWC in the latter part of the growing season. The degree of over- or underprediction of the model-simulated SWC increased with depth (figs. 5c and 5d). According to the soil water balance provided by RZWQM2 (table 6), the total transpiration in the sugarbeet plots at CREC accounted for ~60% of the total water losses in 2014 and 2015, and the water loss through plant transpiration was about 5 times larger than that through soil evaporation. Landa et al. (1999) and Martin and Watts (1999) argued that inaccurate simulation of ET and/or LAI might have contributed to over- or underpredictions of SWC (see also Malone et al., 2010). In the past, Jaynes and Miller (1999) observed that RZWQM underpredicted ET mostly during dry conditions in September in a four-year corn-soybean rotation field with clarion loam soil. However, Farahani et al. (1996) observed that RZWQM provided reasonable ET predictions, although it tended to underpredict ET at smaller LAI values (<0.5) and overpredict ET at greater LAI values.

Figure 6 shows that, during the 2015 growing season, RZWQM2 was able to simulate the SWCs in all four layers well up to about the 70th day after planting. After that, the model started to underpredict the SWCs in the top layer (0-15 cm; fig. 6a) and the bottom layer (45-60 cm; fig. 6d) while maintaining good simulations for the middle two layers (15-30 and 30-45 cm; figs. 6b and 6c). Processes that affect soil water content include soil evaporation, crop transpiration, surface runoff, snowmelt, deep drainage, rooting depth, and tile flow. Root development to deeper soils along with changes in LAI may alter ET in an ecosystem (Tanaka et al., 2004). Deep-rooted crops can maintain ET by absorbing water from deeper soils and maintain ET throughout the year (Jackson et al., 2000). ET from vegetation depends on LAI and leaf physiological characteristics (i.e., carboxylation rate) in conjunction with hydrological and meteorological variables (e.g., precipitation, radiation, temperature, wind speed, etc.).

When a crop is small, actual evapotranspiration is also low. As the growing season progresses, the LAI and rooting depth of the crop increase, allowing more evaporative surfaces and deeper areas for extracting water for ET (Jackson et al., 2000; Tanaka et al., 2004). The ability of the soil to transmit water to plant roots and the evaporative demand from the environment together determine actual crop evapotranspiration. Inaccurate simulation of LAI during the period of 70 to 100 days after planting may have contributed to the underpredictions of SWCs in the top and bottom layers.

Figure 6. Soil water content at different soil depths of (a) 0-15 cm, (b) 15-30 cm, (c) 30-45 cm, and (d) 45-60 cm in 2015. Observed values are plotted in boxplots with medians shown as lines within the boxes, 25th and 75th percentiles as the tops and bottoms of the boxes, and 5% and 95% percentiles as the whiskers below and above the boxes.

As for soil nitrate, the RZWQM2-simulated nitrate concentration in the entire soil profile is plotted with the observed soil nitrate in figures 7a and 7b, and nitrate concentrations at different soil depths are plotted in figures 8 and 9. Overall, the model did very well in simulating the soil profile nitrate throughout the growing seasons of 2014 (fig. 7a) and 2015 (fig. 7b), especially during the early plant growth stage. The d-statistics for model calibration (2014) and model validation (2015) were 0.992 and 0.990, respectively (table 5). For soil nitrate contents at different depths, the model did reasonably well considering that it was calibrated with the observed soil profile nitrates only. The d-statistics were 0.733 to 0.926 and the rRMSE values were 0.457 to 1.211, including both the calibration and validation periods (table 5). In 2014, the model underpredicted the nitrate content in the top layer (0-15 cm) and overpredicted the nitrate contents in the other layers (15-30, 30-45, and 45-60 cm) during the early growing stage (fig. 8). In 2015, the model generally underpredicted the soil nitrate content in all but one layer (fig. 9). In addition, nitrogen mass balance in the soil profile (table 7) shows that 63% to 66% of soil nitrogen input was from fertilization and 31% to 35% was from net mineralization of soil organic matters in 2014 and 2015.

Estimated Parameter Values and Correlations

Estimated parameter values obtained from model calibration are shown in table 3. Through the SVD-based regularization procedure, PEST changed the values of 14 of the 27 parameters. The parameter values changed most were BD1, BD5, Ks1, Ks3, Ks4, Ks5, P1, and RUE. Except P1 and RUE, these parameters are bulk densities and saturated hy-draulic conductivities of different soil layers that affect water and nutrient contents in the soil profile. P1 is related to the length of the sugarbeet growth cycle from seedling emergence to the end of the juvenile phase, while RUE determines the radiation use efficiency of the crop, which may vary from 2.8 to 4.2 g plant dry matter MJ-1 for sugarbeet (Leviel, 2000).

(a)(b)
Figure 7. Total soil profile nitrate-N for (a) model calibration (2014) and (b) model validation (2015)

Table 8 displays the correlation coefficient matrix of RZWQM2 parameters for sugarbeet. A careful examination of table 8 identified six parameter correlations with absolute values of correlation coefficient |r| = 0.8 (shown in bold in table 8). Among these six large parameter correlation coefficients, only the correlation coefficient between BD2 and Ks2 exceeded 0.95, which is an indication of highly correlated parameters that may not be uniquely estimated in the inverse modeling process (Poeter and Hill, 1997).

Incidentally, all the large parameter correlation coefficients (|r| = 0.8) identified in this study were negative, indicating that the strongly correlated parameters have opposite effects on each other. Among these six strongly or highly correlated parameter pairs, three pairs included soil parameters related to soil water movement (i.e., BD2 vs. Ks2, BD5 vs. Ks5, and BD5 vs. Ks1). It is not surprising that the bulk densities and hydraulic conductivities of the same soil layers (e.g., BD2 vs. Ks2 and BD5 vs. Ks5) were strongly but negatively correlated to each other. This suggests that, when calibrating RZWQM2, it makes more sense to fix soil bulk density values while allowing the inverse modeling software, such as PEST, to automatically adjust soil hydraulic conductivities. Another two strongly correlated parameter pairs included plant parameters (i.e., P1 vs. PHINT and RUE vs. PARSA). Only one strongly correlated parameter pair included a soil parameter and a plant parameter (i.e., BD5 vs. PHINT).

(a)(b)
(c)(d)
Figure 9. Soil nitrate content at soil depths of (a) 0-15 cm, (b) 15-30 cm, (c) 30-45 cm, and (d) 45-60 cm in 2015.

Parameter Sensitivity and Identifiability

The relative sensitivities of parameters with respect to the five observation groups and all observations are plotted in figure 10. A close inspection of the six plots in figure 10 reveals that almost all the sensitive parameters, except for PHINT, with respect to any observation groups including plant growth variables (i.e., LAI, top weight, and root weight) were soil bulk densities and saturated hydraulic conductivities of soils in different layers. This implies that the soil water content was a strong limiting factor for sugarbeet growth at CREC. This should not be surprising because it was a dryland sugarbeet system without any use of irrigation water. The most sensitive parameters may be different when modeling irrigated conditions, under which soil water content may not be as strong a limiting factor for plant growth as under dry conditions. We should also note that the local sensitivity analysis conducted in this study was based on initial parameter values. The diagnosis results may be different if calibrated parameter values are used or if global sensitivity analysis methods are used (Ferreira et al., 1995; Ma et al., 2000).

Table 8. Correlation coefficient matrix of RZWQM2 parameters for sugarbeet modeling. Strong correlations (|r| > 0.8) are shown in bold.
BD1BD2BD3BD4BD5Ks1Ks2Ks3Ks4Ks5P1P2P5G2G3
BD11.000.000.010.040.49-0.480.01-0.040.25-0.29-0.05-0.02-0.09-0.01-0.02
BD2-1.00-0.630.62-0.380.21-0.980.280.750.48-0.51-0.17-0.29-0.17-0.11
BD3--1.00-0.720.41-0.330.60-0.74-0.16-0.490.280.050.070.050.03
BD4---1.00-0.240.06-0.570.610.270.51-0.66-0.11-0.19-0.11-0.11
BD5----1.00-0.880.45-0.160.11-0.820.210.190.310.190.13
Ks1-----1.00-0.300.14-0.270.74-0.06-0.03-0.18-0.03-0.02
Ks2------1.00-0.24-0.70-0.550.480.210.300.210.11
Ks3-------1.00-0.160.29-0.15-0.010.03-0.01-0.01
Ks4--------1.00-0.11-0.33-0.01-0.20-0.01-0.01
Ks5---------1.00-0.50-0.18-0.30-0.18-0.15
P1----------1.000.120.730.120.08
P2-----------1.000.110.110.13
P5------------1.000.020.16
G2-------------1.000.04
G3--------------1.00
PHINTRUEPARSRSDSZRSGRRSGRTCARBOTDSGTDGETSWCGPORMRLWR
BD1-0.47-0.36-0.22-0.01-0.02-0.03-0.01-0.07-0.06-0.02-0.07-0.05
BD20.530.310.19-0.07-0.10-0.13-0.09-0.11-0.10-0.11-0.11-0.15
BD3-0.39-0.23-0.210.020.010.040.030.060.050.030.030.09
BD40.340.180.15-0.08-0.11-0.17-0.06-0.11-0.09-0.11-0.08-0.13
BD5-0.82-0.51-0.330.090.090.190.130.160.130.130.160.21
Ks10.590.370.18-0.02-0.02-0.04-0.02-0.06-0.02-0.02-0.05-0.07
Ks2-0.57-0.27-0.170.110.080.130.110.130.110.110.160.23
Ks30.270.130.09-0.01-0.01-0.03-0.01-0.02-0.01-0.01-0.02-0.03
Ks40.100.050.05-0.01-0.01-0.03-0.01-0.03-0.01-0.01-0.01-0.04
Ks50.710.430.32-0.11-0.13-0.15-0.15-0.17-0.15-0.150.110.13
P1-0.800.150.160.060.080.110.080.090.080.08-0.310.52
P2-0.210.050.110.110.100.130.140.090.130.13-0.080.06
P5-0.43-0.23-0.090.120.110.160.160.160.140.160.36-0.70
G20.070.040.040.020.030.040.040.040.040.040.040.03
G30.080.070.050.010.01-0.02-0.02-0.11-0.060.030.090.17
PHINT1.00-0.290.090.050.080.17-0.110.120.100.130.11-0.79
RUE-1.00-0.890.020.050.210.030.050.080.17-0.150.71
PARSR--1.000.030.040.05-0.110.110.060.03-0.160.53
SDSZ---1.000.010.110.130.090.010.030.030.03
RSGR----1.000.23-0.020.060.090.110.010.01
RSGRT-----1.000.120.080.030.080.080.08
CARBOT------1.000.070.050.030.050.05
DSGT-------1.000.110.090.040.04
DGET--------1.000.050.010.01
SWCG---------1.000.17-0.15
PORM----------1.000.27
RLWR-----------1.00

Figure 10 shows that, in addition to soil property parameters, LAI and top weight were very sensitive to PHINT, a plant phenological parameter (figs. 10a and 10b). PHINT, defined as phyllochron interval (table 4), is the thermal time interval (or the sum of the degree days) required to grow the phytomer units of successive leaves. Therefore, PHINT is directly related to the growth of phytomer units, a basic unit for the phenological development and vegetative growth of a crop. The development and growth of sugarbeet are actually characterized by the repeated formation, expansion, and subsequent senescence of phytomer units (Wilhelm and McMaster, 1995). Figure 10 also shows that root weight and soil nitrate were less sensitive to PHINT (figs. 10c and 10e), while soil water content were not sensitive to PHINT at all (fig. 10d). Overall, the most sensitive parameters for sugarbeet under dry conditions include the bulk densities (BD1-5), saturated hydraulic conductivities (Ks1-5), and PHINT (fig. 10f).

The parameter identifiability with respect to the five observation groups and all observations is plotted in figure 11. Parameter identifiability represents the observation group’s ability to constrain the model parameters. The height of the vertical bars measures the parameter’s identifiability, and the different colors correspond to the individual contributions from each of the eigenvectors spanning the calibration solution space. Figures 11a through 11c show that the information contained in the observations of LAI, top weight, and

(a)(b)
(c)(d)
(e)(f)
Figure 10. Bar plot of RZWQM2 relative composite sensitivities with respect to individual observations and to the entire calibration dataset based on their initial values. Parameter definitions are shown in table 4.

root weight was sufficient to estimate three RZWQM2 pa-rameters (BD1 or BD2, Ks1, and PHINT), of which two were soil parameters and the other was a plant parameter. Four soil parameters (BD1 to BD3 and Ks1) may be estimated by the observed soil water contents in the four soil layers (fig. 11d), while only three soil parameters (BD3, BD4, and Ks1) may be estimated by the observed soil nitrate concentrations (fig. 11e). Overall, five RZWQM2 parameters may be identifiable by the entire calibration dataset (fig. 11f). It is worth mentioning that Ks1 was identifiable by any of the observation groups (figs. 11a through 11f), while PHINT was identifiable by any of the observations of plant variables, such as LAI, top weight, and root weight (figs. 11a through 11c).

(a)(b)
(c)(d)
(e)(f)
Figure 11. Bar plot of RZWQM2 parameter identifiability at the beginning of the inverse modeling by selected observation groups.

Conclusions

The CSM-CERES-Beet model was incorporated into RZWQM2 through a linkage to DSSAT. The RZWQM2 model was then applied to model dryland sugarbeets planted at the Carrington Research and Extension Center, North Dakota, in 2014 and 2015. The model did reasonably well in both 2014 and 2015 in terms of simulating LAI, top weight, root weight, SWC, and soil nitrates. The d-statistic ranged from 0.709 to 0.992 in 2014 for model calibration and from 0.733 to 0.990 in 2015 for model validation. The corresponding ranges for rRMSE were 0.066 to 1.211 and 0.043 to 0.930, respectively.

Soil water balance analysis shows that the water loss through plant transpiration was about 5 times larger than that through soil evaporation in the sugarbeet plots in 2014 and 2015, and the total transpiration accounted for ~60% of the total water losses in both years. Soil nitrogen mass balance analysis shows that more than 60% of soil nitrogen input was from fertilization, and 31% to 35% was from net mineralization of soil organic matter.

Sensitivity analysis using PEST shows that, under dry conditions when soil available water becomes a strong limiting factor for sugarbeet growth, the most sensitive parameters were soil bulk densities and saturated hydraulic conductivities in different layers. The only sensitive plant parameter was PHINT, which determines the thermal time needed for leaf appearance. Identifiability analysis shows that three to five model parameters may be identifiable by the calibration datasets that include observations of LAI, top weight, and root weight, as well as SWCs and soil nitrate concentrations in four different soil layers. More interestingly, the saturated hydraulic conductivity in the top layer (Ks1) was identifiable by all of the observation groups, while PHINT was identifiable by all of the observations of plant variables, such as LAI, top weight, and root weight. Our study demonstrated that the sensitivity analysis methods of PEST, which are based on linear theory, can be computed with modest computational burden and can readily accommodate parameter correlations.

In the future, the developed model will be applied to simulate sugarbeet production under different management scenarios for different soils and under different climatic conditions in the Red River Valley. As sugarbeet production may be expanded into nontraditional planting areas in the region due to potential demand for biofuel production, RZWQM2 enhanced with a sugarbeet module can be used to assess the associated environmental impacts.

Acknowledgements

The funding for this research is provided by the USDA National Institute of Food and Agriculture Foundational Program through Project No. 2013-67020-21366. The authors would like to thank Mrs. Szilvia Yuja and undergraduate researchers for their assistance in field data collection. The authors also gratefully acknowledge Dr. Gerrit Hoogenboom and Dr. Vakhtang Sheila from the University of Florida for their assistance with incorporation of CSM-CERES-Beet into DSSAT.

References

Ahuja, L. R., Rojas, K. W., Hanson, J. D., Shaffer, M. J., & Ma, L. (2000). Root Zone Water Quality Model: Modeling management effects on water quality and crop production. Littleton, CO: Water Resources Publications.

Anar, M. J., & Lin, Z. (2016). Modeling the impacts of sugarbeet for biofuel production on soil and water quality using the Root Zone Water Quality Model (RZWQM). ASABE Paper No. 162459803. St. Joseph, MI: ASABE. https://doi.org/10.13031/aim.20162459803

Anar, M. J., Lin, Z., Teboh, J., & Ostlie, M. (2015). Modeling energy beets using the Decision Support System for Agrotechnology Transfer. ASABE Paper No. RRV15-048. St. Joseph, MI: ASABE. https://doi.org/10.13031/RRV15-048

Aster, R. C., Borchers, B., & Thurber, C. H. (2005). Parameter estimation and inverse problems. Amsterdam, The Netherlands: Elsevier Academic.

Baey, C., Didier, A., Lemaire, S., & Maupas, F. (2014). Parametrization of five classical plant growth models applied to sugar beet and comparison of their predictive capacity on root yield and total biomass. Ecol. Model., 290, 11-20. http://dx.doi.org/10.1016/j.ecolmodel.2013.11.003

Biemond, T., Greve, H. J., Schiphouwer, T., & Verhage, A. J. (1989). PIEteR: Semi green-box produktiemodel suikerbieten. Wageningen, The Netherlands: Vakgroep Agrarische Bedrijfseconomie.

Breda, N. J. J. (2003). Ground-based measurements of leaf area index: A review of methods, instruments, and current controversies. J. Exp. Bot., 54(392), 2403-2417. https://doi.org/10.1093/jxb/erg263

Brisson, N., Mary, B., Ripoche, D., Jeuffroy, M. H., Ruget, F., Nicoullaud, B., ... Delécolle , R. (1998). STICS: A generic model for the simulation of crops and their water and nitrogen balances: I. Theory and parameterization applied to wheat and corn. Agronomie, 18(5-6), 36. https://doi.org/10.1051/agro:19980501

Canter, C. E., Dunn, J. B., Han, J., Wang, Z., & Wang, M. (2016). Policy implications of allocation methods in the life cycle analysis of integrated corn and corn stover ethanol production. Bioenergy Res., 9(1), 77-87. https://doi.org/10.1007/s12155-015-9664-4

Cattanach, A. W., Dexter, A. G., & Oplinger, E. S. (1991). Sugarbeets. Alternative field crops manual. Madison, WI: University of Wisconsin Extension. Retrieved from http://www.hort.purdue.edu/newcrop/afcm/sugarbeet.html

Cournede, P.-H., Chen, Y., Wu, Q., Baey, C., & Bayol, B. (2013). Development and evaluation of plant growth models: Methodology and implementation in the PyGMAlion platform. Math. Model. Nat. Phenom., 8(4), 112-130. https://doi.org/10.1051/mmnp/20138407

Doherty, J. E. (2005). PEST: Model-independent parameter estimation user manual (5th Ed.). Brisbane, Australia: Watermark Numerical Computing.

Doherty, J. E. (2010). PEST: Model-independent parameter estimation user manual (5th Ed. with sligh additions). Brisbane, Australia: Watermark Numerical Computing.

Doherty, J. E. (2016a). PEST: Model-independent parameter estimation user manual part I: PEST utility support software (6th Ed.). Brisbane, Australia: Watermark Numerical Computing.

Doherty, J. E. (2016b). PEST: Model-independent parameter estimation user manual part II: PEST utility support software (6th Ed.). Brisbane, Australia: Watermark Numerical Computing.

Doherty, J. E., & Hunt, R. J. (2009). Two statistics for evaluating parameter identifiability and error reduction. J. Hydrol., 366, 119-127. http://dx.doi.org/10.1016/j.jhydrol.2008.12.018

Doherty, J. E., & Hunt, R. J. (2010). Approaches to highly parameterized inversion guide to using PEST for groundwater-model calibration. USGS Scientific Investigations Report 2010-5169. Reston, VA: U.S. Geological Survey.

Doherty, J. E., & Welter, D. (2010). A short exploration of structural noise. Water Resour. Res., 46(5), W05525. https://doi.org/10.1029/2009WR008377

Farahani, H. J., Bausch, W. C., Aiken, R. M., & Ahuja, L. R. (1996). Evapotranspiration in system-wide agricultural models. In C. R. Camp, E. J. Sadler, & E. Yoder. R. (Eds.), Proc. Intl. Conf. Evapotranspiration and Irrigation Scheduling (pp. 673-678). St. Joseph, MI: ASAE.

Ferreira, V. A., Weesies, G. A., Yoder, D. C., Foster, G. R., & Renard, K. G. (1995). The site and condition specific nature of sensitivity analysis. J. Soil Water Cons., 50(5), 493-497.

Fick, G. W. (1971). Analysis and simulation of the growth of sugar beet (Beta vulgaris L.). PhD diss. Davis, CA: University of California.

Flerchinger, G. N., Caldwell, T. G., Cho, J., & Hardegree, S. P. (2012). Simultaneous heat and water model: Model use, calibration, and validation. Trans. ASABE, 55(4), 1395-1411. https://doi.org/10.13031/2013.42250

Flugge, M., Lewandrowski, J., Rosenfeld, J., Boland, C., Hendrickson, T., Jaglo, K., ... Pape, D. (2017). A life-cycle analysis of the greenhouse gas emissions of corn-based ethanol. Washington, DC: USDA Climate Change Program Office. Retrieved from https://www.usda.gov/oce/climate_change/mitigation_technologies/USDAEthanolReport_20170107.pdf

Grahovac, J. A., Dodic, J. M., Dodic, S. N., Popov, S. D., Jokic, A. I., & Zavargo, Z. Z. (2011). Optimization of bioethanol production from intermediates of sugar beet processing by response surface methodology. Biomass Bioenergy, 35(10), 4290-4296. http://dx.doi.org/10.1016/j.biombioe.2011.07.016

Hettinga, W. G., Junginger, H. M., Dekker, S. C., Hoogwijk, M., McAloon, A. J., & Hicks, K. B. (2009). Understanding the reductions in U.S. corn ethanol production costs: An experience curve approach. Energy Policy, 37(1), 190-203. http://dx.doi.org/10.1016/j.enpol.2008.08.002

Hunt, W. F. (1974). Respiratory control and its prediction by a dynamic model of sugar beet growth. PhD diss. Davis, CA: University of California.

Jackson, R. B., Schenk, H. J., Jobbágy, E. G., Canadell, J., Colello, G. D., Dickinson, R. E., ... Sykes, M. T. (2000). Belowground consequences of vegetation change and their treatment in models. Ecol. Appl., 10(2), 470-483. https://doi.org/10.1890/1051-0761(2000)010[0470:BCOVCA]2.0.CO;2

Jaynes, D. B., & Miller, J. G. (1999). Evaluation of the Root Zone Water Quality Model using field-measured data from the Iowa MSEA. Agron. J., 91(2), 192-200. https://doi.org/10.2134/agronj1999.00021962009100020005x

Jessen, H. (2011). RFS volume requirements released by EPA. Ethanol Producer Magazine (28 Dec. 2011). Retrieved from http://ethanolproducer.com/articles/8444/

Jones, J. W., Hoogenboom, G., Porter, C. H., Boote, K. J., Batchelor, W. D., Hunt, L. A., ... Ritchie, J. T. (2003). The DSSAT cropping system model. European J. Agron., 18(3-4), 235-265. http://dx.doi.org/10.1016/S1161-0301(02)00107-7

Landa, F. M., Fausey, N. R., Nokes, S. E., & Hanson, J. D. (1999). Plant production model evaluation for the Root Zone Water Quality Model (RZWQM 3.2) in Ohio. Agron. J., 91(2), 220-227. http://dx.doi.org/10.2134/agronj1999.00021962009100020008x

Lee, G. S. (1983). Conceptual development of a sugarbeet crop growth model. PhD diss. Fort Collins, CO: Colorado State University.

Legates, D. R., & McCabe, G. J. (1999). Evaluating the use of “goodness-of-fit” measures in hydrologic and hydroclimatic model validation. Water Resour. Res., 35(1), 233-241. https://doi.org/10.1029/1998WR900018

Leviel, B. (2000). Evaluation des risques et maîtrise des flux d’azote au niveau d’uneparcelle agricole dans la plaine roumaine et bulgare. Application aux cultures demais, blé, colza et betterave. PhD diss. Toulouse, France: Institut National Polytechnique de Toulouse.

Ma, L., Ahuja, L. R., Nolan, B. T., Malone, Trout, T. J., & Qi, Z. (2012). Root Zone Water Quality Model (RZWQM2): Model use, calibration, and validation. Trans. ASABE, 55(4), 1425-1446. https://doi.org/10.13031/2013.42252

Ma, L., Ascough II, J. C., Ahuja, L. R., Shaffer, M. J., Hanson, J. D., & Rojas, K. W. (2000). Root Zone Water Quality Model sensitivity analysis using Monte Carlo simulation. Trans. ASAE, 43(4), 883-895. https://doi.org/10.13031/2013.2984

Malone, R. W., Jaynes, D. B., Ma, L., Nolan, B. T., Meek, D. W., & Karlen, D. L. (2010). Soil-test N recommendations augmented with PEST-optimized RZWQM simulations. J. Environ. Qual., 39(5), 1711-1723. https://doi.org/10.2134/jeq2009.0425

Martin, D. L., & Watts, D. G. (1999). Evaluation of the Root Zone Water Quality Model for conditions in central Nebraska. Agron. J., 91, 201-211. https://doi.org/10.2134/agronj1999.00021962009100020006x

Maung, T. A., & Gustafson, C. R. (2011). The economic feasibility of sugar beet biofuel production in central North Dakota. Biomass Bioenergy, 35, 3737-3747. http://dx.doi.org/10.1016/j.biombioe.2011.05.022

Milford, G. F. J., Travis, K. Z., Pocock, T. O., Jaggard, K. W., & Day, W. (1988). Growth and dry matter partitioning in sugar beet. J. Agric. Sci., 110(2), 301-308. https://doi.org/10.1017/S0021859600081326

Modig, S. A. (1992). Swedish forecasts of sugar beet yields: Some regression models. Proc. IIRB 55th Winter Congress (pp. 189-210). Brussels, Belgium: International Institute for Beet Research.

Moore, C., & Doherty, J. E. (2005). Role of the calibration process in reducing model predictive error. Water Resour. Res., 41(5), W05020. https://doi.org/10.1029/2004WR003501

Nahar, N., & Pryor, S. W. (2013). Enzymatic hydrolysis and fermentation of crushed whole sugar beets. Biomass Bioenergy, 59, 512-519. http://dx.doi.org/10.1016/j.biombioe.2013.10.012

Necpalova, M., Anex, R. P., Fienen, M. N., Del Grosso, S. J., Castellano, M. J., Sawyer, J. E., ... Barker, D. W. (2015). Understanding the DayCent model: Calibration, sensitivity, and identifiability through inverse modeling. Environ. Model. Software, 66, 110-130. http://dx.doi.org/10.1016/j.envsoft.2014.12.011

Nolan, B. T., Malone, R. W., Ma, L., Green, C. T., Fienen, M. N., & Jaynes, D. B. (2011). Inverse modeling with RZWQM2 to predict water quality. In L. R. Ahuja & L. Ma (Eds.), Methods of introducing system models in agricultural research (pp. 327-363). Madison, WI: ASA, CSSA, SSSA. https://doi.org/10.2134/advagricsystmodel2.c12

Poeter, E. P., & Hill, M. C. (1997). Inverse models: A necessary next step in ground-water modeling. Ground Water, 35(2), 250-260. https://doi.org/10.1111/j.1745-6584.1997.tb00082.x

Qi, A., Kenter, C., Hoffmann, C., & Jaggard, K. W. (2005). The Broom’s Barn sugar beet growth model and its adaptation to soils with varied available water content. European J. Agron., 23(2), 108-122. http://dx.doi.org/10.1016/j.eja.2004.09.007

Saseendran, S. A., Ma, L., Malone, R., Heilman, P., Ahuja, L. R., Kanwar, R. S., ... Hoogenboom, G. (2007). Simulating management effects on crop production, tile drainage, and water quality using RZWQM-DSSAT. Geoderma, 140(3), 297-309. http://dx.doi.org/10.1016/j.geoderma.2007.04.013

Smit, A. B., van Niejenhuis, J. H., & Struik, P. C. (1993). Introduction to a bio-economic production model for sugar beet growing. Proc. Plant Production on the Threshold of New Century (pp. 115-122). Wageningen, The Netherlands: Springer.

Spitters, C. J. T., Kiewiet, B., & Schiphouwer, T. (1990). A weather-based yield-forcasting model for sugar beet. Netherlands J. Agric. Sci., 38(4), 731-735.

Spitters, C. J. T., van Keulen, H., & van Kraalingen, D. W. G. (1989). A simple and universal crop growth simulator: SUCROS87. In R. Rabbinge, S. A. Ward, & H. H. van Laar (Eds.), Simulation and systems management in crop protection (pp. 147-181). Wageningen, The Netherlands: Pudoc.

Taky, A. (2008). Maîtrise des excès d’eau hivernaux et de Iirrigation et de leurs con-séquences sur la productivité de la betterave sucrière dans le périmètre irriguédu Gharb (Maroc). Analyse expérimentale et modélisation. PhD diss. Paris, France: AgroParisTech.

Tanaka, K., Takizawa, H., Kume, T., Xu, J., Tantasirin, C., & Suzuki, M. (2004). Impact of rooting depth and soil hydraulic properties on the transpiration peak of an evergreen forest in northern Thailand in the late dry season. J. Geophys. Res.: Atmos., 109, D23107. https://doi.org/10.1029/2004JD004865

Tonkin, M. J., & Doherty, J. E. (2005). A hybrid regularized inversion methodology for highly parameterized environmental models. Water Resour. Res., 41(10), W10412. https://doi.org/10.1029/2005WR003995

USDA-ERS. (2016). Table 14: U.S. sugar beet crops: Area planted, acres harvested, yield per acre, and production, by state and region. Washington, DC: USDA Economic Research Service. Retrieved from http://www.ers.usda.gov/data-products/sugar-and-sweeteners-yearbook-tables.aspx

Vandendriessche, H. J., & van Ittersum, M. K. (1995). Crop models and decision support systems for yield forecasting and management of the sugar beet crop. European J. Agron., 4(3), 269-279. http://dx.doi.org/10.1016/S1161-0301(14)80027-0

Vargas-Ramirez, J. M., Haagenson, D. M., Pryor, S. W., & Wiesenborn, D. P. (2013). Determination of suitable storage conditions to preserve fermentable sugars in raw thick beet juice for ethanol production. Biomass Bioenergy, 59, 362-369. http://dx.doi.org/10.1016/j.biombioe.2013.07.014

Vos, J., Marcelis, L., De Visser, P., Struik, P., & Evers, J. (2007). Functional-structural plant modelling in crop production (Chapter 1). Dordrecht, The Netherlands: Springer.

Wang, M. Q., Han, J., Haq, Z., Tyner, W. E., Wu, M., & Elgowainy, A. (2011). Energy and greenhouse gas emission effects of corn and cellulosic ethanol with technology improvements and land use changes. Biomass Bioenergy, 35(5), 1885-1896. http://dx.doi.org/10.1016/j.biombioe.2011.01.028

Webb, C. R., Werker, A. R., & Gilligan, C. A. (1997). Modelling the dynamical components of the sugar beet crop. Ann. Bot., 80(4), 427-436. https://doi.org/10.1006/anbo.1997.0461

Wilhelm, W. W., & McMaster, G. S. (1995). Importance of the phyllochron in studying development and growth in grasses. Crop Sci., 35(1), 1-3. https://doi.org/10.2135/cropsci1995.0011183X003500010001x