How much energy can optimal control of domestic water heating save?

Scheduled control of domestic electric water heaters, designed to cut energy use while minimising the impact on users’ comfort and convenience, has been fairly common for some time in a number of countries. The aim is usually load-shifting (by heating water at oﬀ-peak times) and/or maximising time-of-use pricing beneﬁts for users. The scheduling tends not to be linked to actual hot water usage and depends largely on stored thermal energy. Heat losses therefore tend to be greater than if the heater ran without a break. The eﬀect of such a control strategy is thus to worsen the energy loss and in most cases increase greenhouse gas emissions. Many developing countries have ﬂat-pricing (no time-of-use incentives) and rely heavily on energy from fossil fuels, making these considerations even more pressing. We explore three strategies for optimal control of domestic water heating that do not use thermostat control: matching the delivery temperature in the hot water, matching the energy delivered in the hot water, and a variation of the second strategy which provides for Legionella sterilisation. For each of these strategies we examine the energy used in heating, the energy delivered at the tank outlet, and issues of convenience to the user. The study diﬀers from most previous work in that it uses real daily hot-water usage proﬁles, ensures like-for-like comparison in delivered energy at the point of use, and includes a daily Legionella avoidance strategy. We tackled this as an optimal control problem using dynamic programming. Our results demonstrate a median energy saving of between 8% and 18% for the three strategies. Even more savings would be realised if intended and unintended usage events are correctly classiﬁed, and the optimal control only plans for intended usage events.


Introduction
Scheduled control of domestic water heating has been touted as a viable way to reduce electrical energy usage. But the savings that are achievable have not been purposefully and methodically analysed and quantified to find an optimal control strategy (including schedule and temperature) that takes into account actual hot water usage patterns and personal comfort and convenience.
Electricity generators, especially those in developing countries, are struggling to meet ever-increasing demand. The problem is expected to worsen with the depletion of natural resources, the trend to move away from coal-based generation, initiatives to combat climate change (necessitating a reduction in the burning of fossil fuels), and the increasing use of electric vehicles (Serra, 2012;Monigatti et al., 2014). Developing countries rely heavily on fossil fuels for electricity generation, which directly links electrical energy usage to greenhouse gas emissions (BloombergNEF, 2018). According to the Global Energy Statistical yearbook, the global domestic electrical energy usage in 2017 was just over 22 000 TWh -a 67 % increase in usage from the year 2000 (Enerdata, 2018). In Africa, it was 663 TWh -a 75 % increase from the year 2000.
A large contributor to domestic electrical energy usage in developing countries is water heating (Hohne et al., 2019). In South Africa, for example, the residential sector uses an estimated 17 % of the total electricity generated, with domestic water heaters being responsible for a combined 38.5 GWh/day (Forlee, 1998;Beute and Delport, 2006;Eskom, 2013). Given the region's fossil-fuel-dependent electricity generation, this translates as the release of approximately 37 700 kg/day of CO 2 (Nersa, 2018).
Water heaters are well suited for demand response to manage peak load because they can store thermal energy for prolonged periods without significant heat loss, and their water, and consequently energy, consumption patterns are cyclical (Ericson, 2009). They can thus act as a buffer to store energy in the form of heat for delayed use when peak-shifting power scheduling schemes are applied (Du and Lu, 2011;Diduch et al., 2012;Shaad et al., 2012). Scheduling schemes need to account for the device's thermal behaviour, water draw patterns and customer comfort and convenience (Gholizadeh and Aravinthan, 2016;Roux et al., 2018). Thermal models for water heaters, and algorithms for their control, are described extensively in the literature for smart grid applications (Goh and Apt, 2004;Nehrir et al., 2007;Du and Lu, 2011;Lu et al., 2011;Diao et al., 2012;Diduch et al., 2012;Booysen et al., 2013;Boudreaux et al., 2014;Nel et al., 2016a;Kepplinger et al., 2015;Gholizadeh and Aravinthan, 2016;Zuniga et al., 2017;Ahmed et al., 2018;Hohne et al., 2018;Jack et al., 2018;Kapsalis et al., 2018;Lunacek et al., 2018;Kepplinger et al., 2019;Gerber et al., 2019). However, only rarely are the proposed models explicitly considered as a means to reduce the overall energy used for water heating, and the resulting greenhouse gas emissions. Mostly they are considered as a means of peak load management for the benefit of the generator, or to benefit the user through time-of-use cost optimisation.
The relevant literature and major remaining challenges for energy saving through scheduled control of storage-based water heaters are listed in Table 1 and summarised in the following section. In developing countries, where the user typically pays a time-independent flat fee per kWh, and not a tariff based on time-of-use or congestion, poorer users resort to schedule control simply to reduce their monthly bill, and possibly their environmental footprint (Nel et al., 2016b;Hohne et al., 2019), and not to shift peaks or to avoid congestion charges. In this situation it is the user who bears the burden of the increased energy usage that may result from any demand management schemes (Roux et al., 2018). And given the coal-intensive energy generation typical of these countries, any change in energy used usually implies a proportional change in greenhouse gasses emitted. Nel et al. (2018) evaluated typical strategies that people use in developing countries to save on domestic water heating, such as insulating the heater with a blanket, using less hot water, lowering the set-point temperature, and applying an on-off schedule. They simulated various hot water usage profiles (e.g. one shower per day versus two baths per day) using a one-node (lumped-mass) model for the water heater. They found that scheduled control had the biggest impact, with an expected energy saving of 9 to 18%.
In this paper, we focus on overall energy savings achieved through optimal control of temperature and scheduling, rather than cost savings achieved through scheduling to avoid congestion charges, or to enable load shifting.
The paper establishes the extent to which electrical energy used for water heating can be reduced for the case of known water draw patterns. We propose a novel optimal water heater control strategy that minimises the electrical energy used while satisfying the user's hot water demand profile and limiting the growth of Legionella bacteria, and takes into account time-varying external disturbances, such as ambient temperature and cold water inlet temperature, and input constraints, such as scheduled supply-side interruption of the electricity supply. We formulate the water heater control problem as an optimal control problem and then solve it using a dynamic programming algorithm to find the optimal switching schedule for the heating element. To make the optimal control problem tractable to be solved with dynamic programming, the electric water heater (EWH) thermal dynamics is modelled with a one-node lumped-parameter model. We envisage the use of a smart water heater remote controller to manage the temperature and heating schedule using a minute-based control cycle, such as the one developed by Brown (2015) and used by Roux et al. (2018). We evaluate the performance of the proposed algorithm against the unscheduled thermostat-controlled case using one-minute simulation step time. The simulations executed to evaluate the performance also use the one-node lumped-parameter model to model the thermal dynamics, which admittedly means that the stratification in the water heater is not taken into account. We evaluate two implementations of the optimised schedule to ensure fair comparisons: one that aims to deliver water at the same temperature for each draw event, and one that aims to deliver the same energy for each draw event. We compare the strategies by using field-measured hot water usage patterns that were sampled every minute from 30 water heaters over a period of 20 days using a simulated thermal model. The water heaters were part of a research project in which users from South Africa's Western Cape, Gauteng, and Mpumalanga volunteered to trial smart water heater controllers Reported energy savings (%) Fanney and Dougherty (1996) 4-6 Goh and Apt (2004) 5-6 Booysen et al. (2013) 14-17 Kepplinger et al. (2014) 5-13 Kepplinger et al. (2015) 10-12 Kepplinger et al. (2016) 12 Booysen and Cloete (2016) 29-34 Gholizadeh and Aravinthan (2016) 6 Cloete (2016) 6 Nel et al. (2018) 9-18 Matos et al. (2019) 39 1 Field-measured hot water usage used in simulation to determine savings. 2 Temperature-matched hot water used in situation with purported savings. 3 Energy-matched hot water used in situation with purported savings 4 Electrical energy savings split into reduced thermal losses and reduced alternative losses. (Roux et al., 2018;Booysen et al., 2019). We add an additional constraint to evaluate the energy impact of ensuring daily Legionella sterilisation in the energy-matched strategy, which operates at a lower temperature. Besides evaluating standing losses, we introduce the concept, and evaluate the impact, of the usage losses that result when hot water is drawn unintentionally. Table 1 summarises the challenges that remain in the literature and establishes the strategies to be explored in the paper. Fanney and Dougherty (1996) evaluated the thermal efficiencies of electrical water heaters under six combinations of usage patterns and heating schedules. The metric they used was thermal efficiency, which is problematic as a stand-alone metric of savings: a water heater that is not switched on has a thermal efficiency of 100 %, and it will have different efficiencies for high volume and low volume use. The simulation results do not state the real energy savings explicitly, but they do predict savings of between 4 % and 6 %, depending on the usage patterns and heating schedules. Goh and Apt (2004) reported savings of 5 to 8% with schedule control, and Gholizadeh and Aravinthan (2016) reported savings of 5.9 to 6.4% with a mixture of scheduled and temperature control. However, the results of both of these studies were from simulations that did not take into account the outlet temperature and useful energy used, and only with predicted (non-real) consumption patterns. Kepplinger et al. (2014) proposed an optimisation method that uses dynamic programming to optimise scheduling for cost and/or energy usage. The method involved an hourly optimisation and implemented an hourly control of water heaters. They used synthesised usage patterns from Jordan et al. (2001), and reported energy savings ranging from 4.5 to 13.3%. Their subsequent work (Kepplinger et al., 2015) proposed an autoscheduling mechanism with reported energy savings ranging from 10.5 to 12.4%, based on the simulation of a stratified thermal model, similar to the work presented here in our paper. In the case of cold events, however, the state constraint approach could not be satisfied. Moreover, the approach ensured equal delivery of matched energy, but did not consider the requirement of matched temperatures, so that the required temperature at the start of each usage event matched the temperature achieved by thermostat control at the start of the same event. They later extended these results (Kepplinger et al., 2016), and also ran a field trial, for which 12.3% savings were achieved.

Challenges in the literature
By implementing scheduled water heating control which was hot-water-usage driven, Booysen et al. (2013) estimated a reduction in energy used for water heating of between 14 % and 17 %. Although they did a small experiment to confirm the results, they did not take into account the effects of the possibly reduced temperature (i.e. reduced energy) of the water drawn from the tank. Moreover, their estimates were based on a simple lumped-mass analytical physics model (i.e. not step-wise simulated) with an assumed usage pattern for only one water heater.
Subsequently, Nel et al. (2016a) presented a more accurate model with the explicit purpose of modelling energy used for water heating under schedule control. Their model makes provision for horizontally mounted water heaters (the most common orientation found in South Africa) under thermostat-based and scheduledbased heating control. Booysen and Cloete (2016) presented preliminary work in which the same model was used to compare savings, achieved by scheduling, in a controlled field trial of four water heaters. The results were augmented by a controlled laboratory experiment with one heater. They reported savings of 29 % due to scheduling. However, the study failed to determine the savings in the scenario where the output energy (and temperature) were matched to the baseline condition. This probably invalidated the conclusion, as the utility extracted from the water heater was not comparable before and after the intervention that resulted in the savings. More importantly, the metric used to measure savings, namely the change in energy per litre, did not represent the savings fairly, since it compared the electrical energy (kWh) used per litre of hot water delivered under the two conditions, but failed to take into account the temperature of (and therefore the utility extracted from) that volume of hot water. Cloete (2016) subsequently conducted a controlled study in which the experiment was repeated in a laboratory with longer, fixed heating times, and obtained just 16 % savings. He then used an iterative adjustment of the set-point to match the output temperature retrospectively for a fairer comparison, which reduced the predicted savings to only 6 %.
In recent studies, Matos et al. (2017Matos et al. ( , 2019 evaluated the impact of flow reducers and reduced temperature on energy used and the resulting CO 2 emissions. The focus of the study was more on flow reducers than on scheduling, and limited specifically to water used for baths. Their results indicated a reduction of 39 % in energy when the temperature was decreased from 75 • C to 60 • C. They described in detail the observed differences, but did not analyse in detail the energy changes from the perspective of the heater supplying the water. Further, they did not differentiate between savings achieved from standing losses and reduced unintentional hot water usage. Finally, they did not evaluate the impact of scheduled control. In addition to these remaining challenges only one of the studies listed in Table 1, Kepplinger et al. (2016), implemented weekly temperature control standards to limit Legionella growth, and none of the studies evaluated losses other than thermal losses. As far as we could determine, none of the studies used high-frequency (minutes, rather than hours) sampled water usage data in their simulations.

Heating control strategies
In our study we took thermostat control as the baseline heater control strategy and evaluated three alternative strategies, the third being a variant of the second, to determine which gives the best electrical energy savings.
0. (Baseline) Thermostat control (TC): This is the mode in which water heaters are designed to be used and how most people use them. The thermostat strives to maintain the water at a target temperature, normally set between 65 • C and 75 • C, with a small hysteresis band around the set temperature. This heating strategy is wasteful, as it maintains a high temperature between draw events that may be far apart. Of the three strategies, this one loses the most energy to the environment. For this heating strategy and set temperature, water is drawn from the water heater at a temperature that is higher than people require for most uses. To achieve a convenient temperature, the user normally has to regulate the temperature by mixing with cold water to achieve a nominal temperature of approximately 40 • C (Armstrong et al., 2014;Jacobs et al., 2018;Kepplinger et al., 2019).
1. Scheduled control with temperature matching (TM): In an attempt to save on bills and reduce loss of energy, financially sensitive and environmentally aware users resort to turning off their heaters for extensive periods between the times when they need hot water (Nel et al., 2016b). The timing applied in this strategy is individually motivated and could vary significantly between users, but the optimal scenario (for a water heater with energy storage) is to switch the heater on just sufficiently in advance of the time the hot water will be needed (Booysen et al., 2013;Nel et al., 2016b). When applying optimisation techniques, as we have done in this study, optimisation constraints can be set to ensure that the same volume of water is drawn at the same temperatures as under thermostat control, but the thermal losses to the environment are minimised. This approach assumes that the user requires water at these high temperatures, intending to mix the water with cold water to achieve a desired temperature. For this control strategy, the heater will deliver the same amount of useful output energy as in the baseline thermostat control scenario.
2. Scheduled control with energy matching (EM): An alternative approach to the optimised schedule control with temperature matching is to assume that the user does not require the water at the high temperatures, but rather is satisfied with a lower, more directly usable temperature. In this case, there is less need, or no need, to add cold water, as the water is already at or just above the desired temperature for use. A lower target temperature during water draw-offs, of say 38 • C, could be used. For our study we increased the volume drawn from the heater, to ensure that the same amount of energy was delivered in the water drawn as under thermostat control (Armstrong et al., 2014;Jacobs et al., 2018;Roux et al., 2018).
3. Scheduled control with energy matching plus Legionella prevention (EML): Although optimising the heating schedule has energy benefits, there are health risks when the water is maintained and delivered at low temperatures. Legionella pneumophila thrives at temperatures between 32 • C and 42 • C and has been found in water heaters (Armstrong et al., 2014;Stone et al., 2019). According to Stout et al. (1986), to sterilise the bacteria, the heater must spend 11 min at 60 • C or 3 min at 70 • C, at least once a day.
We therefore define a Legionella-driven control strategy that ensures that the heater temperature reaches 60 • C for at least 11 min at least once before the largest water usage event of the day.

Electric water heater (EWH) dynamics
The EWH acts as a closed-loop system with a thermostat providing the feedback. Figure 1 shows an overview of the energy flow in this system and the function of the feedback loop. The temperature of the water inside the tank T tank is fed in as input to the thermostat. The thermostat is directly connected to the heating element, which provides electrical power P elec to increase the thermal energy inside the tank, E tank . This energy depends on the rated power of the heating element, P rated . When water is drawn from the tank at a higher temperature than the inlet temperature, the net effect is a reduction in thermal energy at rate of P draw . When heat is lost to the environment because of the temperature difference between the tank and the environment, the net effect is a reduction in thermal energy at a rate of, P loss .
We model the EWH thermal dynamics with the following one-node lumped-parameter model.
where E tank is the thermal energy of the water inside the EWH, P elec is the power input delivered by the heating element, P draw (t) is the power output due to hot water draw (hot water leaving the EWH and being 5 RIP Neef Jan du Plessis replaced by cold water), and P loss is the power output due to thermal losses to the environment. The heating element is either off or on, which means that the power input P elec delivered by the element is either zero or its rated power P rated The electrical energy supplied by P rated is given by The power output due to hot water draw P draw is given by where ρ is the density of the water, Q draw (t) is the hot water outlet volumetric flow rate, andĥ inlet andĥ outlet are the specific enthalpy entering and leaving the water heater, respectively. Under conditions of constant pressure and constant specific heat capacity, this can be approximated by where c P is the constant pressure-specific heat capacity of the water, T outlet is the hot water outlet temperature, and T inlet is the cold water inlet temperature.
The power output due to thermal losses P loss (t) is given by where R T H is the thermal resistance of the EWH wall, T tank is the water temperature inside the EWH, and T amb is the ambient temperature. Note that for the one-node lumped-parameter model, the tank temperature and the hot water outlet temperature are assumed to be equal.
The relationship between the EWH energy E tank and the EWH water temperature T tank relative to a reference temperature where we define energy to be zero, is given by where V tank is the volume of the EWH, c V is the constant volume specific heat capacity, which is approximately equal to c P for water, and henceforth denoted as c.
The hot water outlet flow rate Q draw (t) is the superposition of the flow rate due to intentional usage Q usage (t) and the flow rate due to unintentional usage Q unintentional (t), defined below.
The power output due to hot water draw P draw (t) is therefore also the superposition of the power output due to intentional usage P usage (t) and the power output due to unintentional usage P unintentional (t) Similarly, the energy output due to hot water draw E draw (t) is the superposition of the energy output due to intentional usage E usage (t) and the energy output due to unintentional usage E unintentional (t) The distinction between usage and unintentional usage is important, because we give the usage profile to the optimal control algorithm as an objective to satisfy, while we treat the unintentional usage as a disturbance and an energy loss.
The optimal control problem formulation In this section we briefly explain optimal control theory, using Kirk (2012) as our primary source, and then formulate the EWH control as an optimal control problem.

Optimal control theory
Optimal control theory is concerned with finding a control law for a given system so that a specified optimality criterion is achieved. This criterion is typically specified as a cost function to be minimised. The problem is therefore to find a control law that produces the optimal control input signals and resulting optimal state trajectories that together minimise a given cost function, subject to a set of dynamic constraints, terminal state constraints, and state variable and control input inequality constraints. This can be expressed mathematically as follows u * (t) = arg min = arg min subject to the dynamic constraintẋ the terminal state constraint and the state space and input space constraints In these equations, J is the cost function, x(t) is a candidate state trajectory, u(t) is a candidate control input signal, t is time, f () represents the set of nonlinear, time-variant differential equations of the system, g() is the state transition cost function, h() is the terminal state cost function, X f is the set of admissible final states, and X and U are the sets of admissible states and admissible inputs. x * (t) and u * (t) are the optimal state trajectory and optimal control input signal that minimise the cost function subject to the constraints.
Formulation of EWH thermal control as an optimal control problem Optimal control problem: Given a hot water usage profile in terms of flow rate Q usage (t) and desired minimum hot water temperature T usage (t) as a function of time t, the cold water inlet temperature T in (t) and the ambient temperature T amb (t) as time-varying disturbance signals, and scheduled supply-side interruption of the electricity supply to the EWH P max (t), we wish to determine the optimal EWH control signal P * elec (t) that will satisfy the hot water usage profile, while minimising the total energy used.
System dynamics: The system dynamics are defined as the nonlinear differential equations describing the EWH thermal dynamics as described above, and specifically by equations (1), (2), (5), and (6).
System state: The state variable for the system is the thermal energy E of the water inside the EWH Control input: The control input for the system is the power input P elec delivered by the heating element State constraints: The physical limitations on the thermal energy inside the EWH are specified by defining the following set of admissible states where the minimum energy E min and the maximum energy E max correspond to the minimum temperature T min and the maximum temperature T max specified for the EWH Input constraints: The control input constraints are specified by defining the following set of admissible inputs The heating element is either off or on, which means that the power input P elec delivered by the element is either zero or its rated power P rated .
Terminal state constraints: No special terminal state constraints are imposed, and the terminal state constraints are the same as the state constraints.
Cost function: The objective to minimise the total energy used is represented by the following cost function where t i is the initial time, and t f is the final time. Note that this a specific implementation of the general cost function with the state transition cost defined as and the termination cost defined as h(x f (t)) = 0 (29) Temperature profile constraints (usage and Legionella prevention): The objective of satisfying the hot water usage profile is represented by the following time-varying state inequality constraint on the thermal energy of the water inside the EWH The need to increase the temperature once per day to prevent the growth of Legionella bacteria can also be included in this time-varying state inequality constraint. The profile temperature T profile (t) is set to the desired usage temperature T usage for times that correspond to intentional draw, to the minimum Legionella prevention temperature T Legionella for a set duration once per day, and to the minimum EWH temperature T min for all other times.
Electricity supply constraints The constraint represented by the scheduled supply-side interruption of the electricity supply to the EWH is represented by the following time-varying input constraint The maximum power input is set to P rated when the supply-side electricity is available, and to zero when the supply-side electricity is interrupted, with P max (t) = 0 when the electricity is interrupted P rated when the electricity is available (33) Constructing the temperature profile constraint: We construct the temperature profile constraint T profile (t) differently for temperature matching, energy matching, and energy matching with Legionella prevention. In constructing the constraint we take into account "unreasonable" hot water usage profiles where it is impossible to deliver hot water at the minimum desired temperature, even if the heating element is switched on permanently. For example, drawing all of the hot water from the tank and then expecting more hot water shortly thereafter is considered "unreasonable" hot water usage behaviour. Optimal temperature matching (TM): We construct the temperature profile constraint for temperature matching so that the required EWH temperature at the start of each usage event matches the EWH temperature achieved by thermostat control at the start of the same event.
Optimal energy matching (EM): We construct the temperature profile constraint for energy matching so that the EWH temperature remains above 40 • C for the duration of each usage event. However, we increase the outlet flow rate so that the energy in the volume of hot water delivered at 40 • C matches the energy in the volume of hot water delivered at the thermostat control temperature.
Optimal energy matching with Legionella prevention (EML): We construct the temperature profile constraint for energy matching with Legionella prevention in exactly the same way as for energy matching (EM), except that once per day we increase the EWH temperature to 60 • C for 11 min to prevent the growth of Legionella. We schedule this to be just before the largest usage event for the day.
"Unreasonable" hot water usage profiles: To accommodate these profiles, we run a forward simulation for the entire usage pattern, assuming that the heating element is always switched on, to determine the best temperatures the EWH can deliver for a given usage profile. We then modify the temperature profile constraint T profile (t) to be the minimum of the temperature profile constructed to satisfy the hot water usage and the Legionella prevention and the achievable temperatures that were determined from the forward simulation.

The dynamic programming solution
Dynamic programming is a search algorithm that models an optimal control problem as a multi-stage decision process and uses the principle of optimality to find the optimal state trajectories and control inputs to minimise the cost function from all initial states. It uses the principle of optimality to drastically reduce the number of calculations required to determine the optimal control law. For systems with a large number of dimensions, the memory requirements of the dynamic programming algorithm becomes prohibitive. This is called "the curse of dimensionality". Fortunately, the EWH system can be modelled as a first-order system, which makes dynamic programming a very suitable approach to solving the optimal control problem.

Discretisation
To apply dynamic programming to an optimal control problem, the problem has to be discretised in time to represent different decision stages, and discretised in state to represent a finite number of decisions to be made at each decision stage. The dynamic programming algorithm starts at the terminal stage and works backward in time through intermediate stages until it finds the optimal admissible path from the initial state to a terminal state.
Discrete-time dynamic model: The continuous-time differential equations describing the system dynamics are discretised to produce discrete-time difference equations that describe the state transition from one discrete time instant to the next.
where ∆t is the sampling period of the discrete time step.
Quantised state vector array: The continuous set of admissible states X is discretised to create a finite set of state values for the dynamic programming algorithm. We therefore create an array X q of quantised state vector values x 0 , x 1 , . . . , x n that span the set of admissible states X .
Control inputs The continuous set of admissible inputs U must be discretised to create a finite set of input values. We therefore create an input array U q of quantised input vector values u 0 , u 1 , . . . , u n that span the set of admissible inputs U.
Incremental state transition cost function: The continuous-time cost function is discretised by expressing the total path cost J ij from the current state x i (k) via the next state x j (k + 1) as the sum of the incremental cost ∆J ij of transitioning from the current state to the next state and the total path cost of the next state J j to a final state.
The incremental state transition cost ∆J ij is obtained by discretising the state transition cost function g(x(t), u(t), t) as follows where u ij is the admissible control input that transitions the system from state x i at time k to state x j at time k + 1.
The incremental transition cost ∆J is therefore obtained as ∆J ij (x i (k), x j (k + 1)) = P elec (k)∆t (39) Algorithm Initialisation 1. Create tables to store the quantised states X q , and the optimal path cost J * from a given state to a final state, the optimal next state index j * , and the optimal control input U * , as follows: where i is the quantisation index of the current state j is the quantisation index of the next state, and k is the index of the current time instant. 2. Populate the final column of the optimal cost table with zeroes to assign termination costs of zero to all final states.
3. Populate the rest of the optimal cost table with infinity values so that if a finite cost is calculated for a state, it will be lower than infinity and will replace the initial cost as the new lowest path cost.
4. Populate the final column of the optimal next state table so that the optimal next state (index j * ) for all final states is the state itself (index i).
Execution 1. Set the index j of the next state x j (k) to the first index of the state vector array 2. For the given index j of the next state x j (k + 1), calculate the previous state x(k) for all admissible values of the control input u(k).

For each
and P elec (k) ≤ P max (k) calculate where and the inlet water temperature T in (k) and the ambient temperature T ambient (k) are known external disturbances at time instant k.
3. Check whether the previous state x(k) is an admissible state.
If the previous state is an admissible state, continue to step 4. Else, continue iterating through the possible values of the control input. 4. Find the index i of the quantised previous state x i (k) in the state vector array that is closest to the calculated previous state x(k). 5. Calculate the cost from the current state x i (k) to a final state through this next state x j (k + 1) where ∆J ij = P elec (k)∆t (56) 6. If the new cost is lower than the lowest cost so far, it becomes the new lowest cost. Also, the control input u i (k) and the next state index j become the new best control input u * i (k) and the new best next state j * i for the current state x i (k). In other words, if J i < J * i then If the next state index j was not the last index, increment the next state index j and return to step 2.
Else, if the next state index j was the last index, and the time instant was not the first time instant, then step one time instant k backward and return to step 1.
Else, terminate the execution.

Lookup table navigation
The dynamic programming algorithm produces a lookup table of the optimal state trajectories and optimal control sequences for EWH control from all time instants and admissible initial states. Given a time instant k and an initial state x i (k), the optimal state trajectory {x * (k) : k = k, k + 1, . . . , N } and the optimal control sequence {u * (k) : k = k, k + 1, . . . , N } can be obtained by starting at the column index k and the row index i of the initial state in the lookup table and iteratively navigating through the lookup table by following the optimal next state indexes j * .

Simulation parameters
The hot water usage data, the software implementation in Jupyter Notebook, and the simulation output are available at http://bit.ly/EWHSavingsESDDataset. Table 2 lists the dataset properties, parameters and constants used in the optimisation and simulation.

Metrics
Water draw is aggregated into what we term usage events. An event starts when a tap is opened and stops when it is closed or, put differently, an event starts when a non-zero volume is sampled one sample after a zero volume sample, and stops when the flow returns to zero after the non-zero sample. The event definition quantifies the water usage into bundles, providing a more convenient way of referring to sections of water  usage patterns, and providing a metric for counting the number of times a user experiences an undesired temperature. Given the plumbing between the water heater outlet and the point of use (i.e. tap), events with a volume of less than 2 L are unlikely to result in the hot water reaching the point of use. This can be shown to be true with the normal pipe diameter of 22 mm and a conservative pipe length of 5 m, which results in a volume of 1.9 L hot water drawn into the piping before the point of use. These events that draw hot water that does not reach the point-of-use are therefore considered unintended events, for which our algorithm does not instruct the element to heat, and which are excluded when counting the number of cold events. These events are likely to occur when the user uses a mixer tap with a position between hot and cold, or unwittingly opens the hot tap for a quick cold draw. Table 3 lists the metrics we use to assess the performance of the various strategies. The electrical energy, effective energy in the drawn water, energy lost to the environment, and energy lost due to unintentional usage comprise the energy metrics. We use the change in electrical energy as a percentage reduction and as an absolute change to determine the achieved savings. We also report the average event temperatures, with the number of cold events, to establish user comfort and convenience.
The distributions of the electrical energy used per day, thermal energy drawn per day, average outlet temperature during usage events, and thermal energy losses per day over all EWHs are shown in Figure 3 (a) to (d). The average electrical energy used per day by an individual EWH is calculated using where P elec|h is the average electrical energy used by heater h per day, P elec|h (k) is the instantaneous electrical power used by heater h at sampling instant k, ∆t is the sampling period, N h is the total number of samples, and D is the total number of days in the data set. The average thermal energy drawn per day P draw|h and the average thermal energy losses per day P loss|h for an individual EWH are calculated using The distributions of the electrical energy savings per day for each strategy (TM, EM, and EML), expressed both as a reduction in kWh per day and as a percentage reduction, are shown in Figure 3 (e) and (f). We calculate the distribution of the energy savings for a particular strategy by first calculating the individual savings for each EWH, and then plotting the distribution of the individual savings over all EWHs. For example, we calculated the energy savings per day for the EM strategy compared to the baseline TC strategy for each individual EWH using the following formulas, and then plotted the distribution of savings for the EM strategy over all EWHs: ∆P elec|h,EM (kWh/d) = P elec|h,TC − P elec|h,EM kWh/day ∆P elec|h,EM (%) = P elec|h,TC − P elec|h,EM P elec|h,TC × 100%

Results and discussion
Simulation results for a single EWH Figure 2 shows example simulation results for a single EWH, comparing thermostat control (TC), optimal temperature matching (TM), optimal energy matching (EM), and optimal energy matching with Legionella prevention (EML). For each control technique, the EWH temperature, outlet flow rate, and heating element control signal are plotted as a function of time for a period of two days (48 hours). The results are shown for the same hot water draw profile, with all simulations starting from the same initial EWH temperature of 70 • C.
Thermostat control (TC): The thermostat control keeps the temperature at about a set point of 68.5 • C (allowing for 1.5 • C hysteresis). When the temperature drops below 67 • C, the heating element switches on; when the temperature rises above 70 • C, it switches off. During usage events, when the outlet flow rate is non-zero, the temperature drops significantly. We observed a single cold event at time t = 9 hours, when the temperature dropped below 40 • C. This cold event was caused by an unusually large usage event, and could not be prevented even with the temperature at maximum at the start of the event, and with the heating element switched on for the full duration of the event. The electrical power consumption for thermostat control was 23.85 kWh over the two days.
Optimal temperature matching (TM): The temperature matching control ensures that the temperature matches the corresponding temperature for thermostat control, but only during usage events. Between usage events, the TM control allows the temperature to drop, and the EWH only starts heating again just before the next usage event. The temperature between the usage events is therefore lower for TM than for TC. We observed the same cold event at time t = 9 hours, due to the unusually large usage event. Since TC was not able to prevent the cold event, starting at maximum temperature and with the heating element switched fully on, we did not expect that TM would be able to prevent the cold event either. The electrical power consumption for optimal temperature matching was 16.7 kWh over the two days.   Note: The distributions are reported as 25 th percentile, median, 75 th percentile * All these cold events were generated by six heavy users out of the 30, which means the median and 75 th percentile EWHs had zero cold events. The number of cold events did not increase from that of TC.
Optimal energy matching (EM): The energy matching control ensures that the temperature remains above 40 • C during usage events. We increased the outlet flow rate to represent the fact that all the water that reached the user was drawn from the EWH, and that the hot water from the EWH was not mixed with cold water before reaching the user. Between usage events, the EM control allows the temperature to drop, and only starts heating again just before the next usage event. The temperature both during and between the usage events is therefore lower for EM than for TC and TM. The same cold event is observed at time t = 9 hours, due to the overlarge usage event. The electrical power consumption for optimal energy matching was 16.55 kWh over the two days.
Optimal energy matching with Legionella prevention (EML): The results for the energy matching control with Legionella prevention look almost exactly the same as the results for energy matching control without this prevention, except that we increased the EWH temperature to the Legionella prevention threshold of 60 • C at time t = 41 hours and then held it there for 11 min. This increase in temperature was scheduled to be just before the largest usage event for the day, excluding cold events. We observed the same cold event at time t = 9 hours, due to the unusually large usage event. The temperature both during and between usage events was therefore almost the same for EM and EML, except that the latter had a higher temperature during the scheduled Legionella prevention heating. We therefore expected that EML would have a slightly higher power consumption. The electrical power consumption for optimal energy matching with Legionella prevention was 16.05 kWh over the two days.
The large difference of 7.15 kWh between the power consumption for TC (23.85 kWh) and the power consumption for TM (16.7 kWh) may seem surprising, given that the standing loss for an EWH is typically only 2 kWh per day. However, it should be noted that TC immediately replenishes the usage energy of the last usage event of the day, while TM, EM and EML wait until just before the next usage event, which occurs only the next day.

Distribution of results over all EWHs
The effects of the different control strategies on energy and temperature are summarised in Table 4 and shown in Figure 3. Figure 3 (a) shows that all the other strategies used less electrical energy than the baseline TC strategy. Since hot water was not needed every day, the minimums were zero for the TM and EM strategies, but not zero for the EML method because of the Legionella control.

Temperature-matched optimisation
The median electrical energy used for TM was 7.9 kWh/day, 0.3 kWh/day (3.7 %) less than the 8.2 kWh/day median for TC. These reductions are presented as distributions in Figures 3 (e) and (f), as daily energy (kWh/day) reduction and percentage reduction, respectively.
Despite the reduction in electrical energy used to heat the water, Figures 3 (b) and (c) show that the thermal energy drawn and the outlet temperature during events, respectively, were the same for TM as for TC. The median event temperatures for TC and TM were both 65 • C. The number of cold events, a metric of negative user satisfaction, did not increase for the TM control, and actually decreased by 3 from 36 out of (a) depicts electrical energy used per EWH per day, (b) depicts thermal energy drawn per EWH per day, (c) depicts outlet temperatures during usage events, (d) depicts thermal losses per EWH per day, and (e) and (f) depict the savings achieved in electrical energy per EWH as a reduction in kWh per day and percentage of total used, respectively. a total of 4486 events. From the user's perspective, the TC and TM strategies will therefore have no adverse effect on perceived temperatures, despite the energy savings.
Looking at the distribution of savings for the water heaters, we found the percentage reduction for TM, given as 25 th percentile, median, 75 th percentile, was 0.4, 0.6, 0.8 kWh/day, or 3.6, 7.9, 11.2 %. This result is in line with the savings reported by Fanney and Dougherty (1996); Goh and Apt (2004), and Cloete (2016), indicating that their approaches employed heating schedules that provided sufficient heating time to effectively achieve results that resemble temperature matching. Figure 3 (d) shows that the median thermal losses decreased from 2.4 kWh/day to 2.0 kWh/day -a 0.4 kWh/day reduction.

Energy-matched optimisation
The average outlet temperature for EM is significantly lower than for TC and TM, as intended, with a median temperature of 42 • C. The implication is that the user will have to mix in less or no cold water to reach the desired temperature of 40 • C or below. Despite this lower temperature, the thermal energy drawn was higher than for TC -5.7 kWh/day vs. 5.9 kWh/day -owing to a larger volume draw of 142 L vs. 229 L. The number of cold events for the EM strategy stayed at 0.06, with two fewer cold events than for TC out of 4486 events. Figures 3 (a) and (b) show that the electrical energy used in the EM strategy was substantially less than either in TC and TM, despite its slightly higher thermal energy delivery. The median electrical energy used for EM was 6.8 kWh/day, 1.4 kWh/day (17 %) less than the 8.2 kWh/day median for TC. The reduction is shown in Figures 3 (e) and (f), with respective energy and percentage reductions from the TC baseline of 1.4, 1.5, 1.7 kWh/day and 12.8, 17.8, 26.3 %. These results are in line with the simulated and field-measured and/or energy-matched results reported by Booysen et al. (2013); Kepplinger et al. (2015Kepplinger et al. ( , 2016 and Nel et al. (2018). The median standing loss for this strategy was a mere 1.0 kWh/day, a reduction of 1.4 kWh/day (58 %) from TC's 2.4 kWh/day.

Legionella control
The results for the Legionella strategy lie between the TM and EM strategies' results, as expected. The median electrical energy usage for Legionella control was 7.2 kWh/day, 12 %less than the median for TC. The results in Table 4 shows that the respective percentage and absolute reductions from TC are 9.1, 13.1, 17.0 % and 0.8, 1.0, 1.2 kWh/day. The median standing losses for this strategy was 1.3 kWh/day, a reduction of 1.1 kWh/day from that of TC. The median event temperature for this strategy was 44 • C, while the median number of cold events was the same as for EM.

Unintentional usage loss
The unintentional usage loss, albeit very small, was 0.06 kWh/day for TC and 0.05 kWh/day for TM, indicating that TM did not entirely counter the unintended usage losses, and that most of the savings were due to a reduction in standing losses as a result of the temperature-matched optimised scheduling.
The unintentional energy usage for EM was only 0.02 kWh/day, compared to the 0.06 kWh/day for TC. This result is important, because it demonstrates that EM control limits the detrimental effect of unintentional usage on the electrical energy used by essentially applying a lower temperature control. This result also makes intuitive sense -the amount of energy lost due to unintentional usage loss will be less when the water is at a lower temperature, which is also true for simply operating at a lower set point temperature. It further shows that more savings can be achieved by optimal control than by merely reducing standing losses. Although these numbers are quite small, it stands to reason that the unintended usage is underestimated, which may explain the significantly higher savings reported by some recent field trials (Cloete, 2016;Booysen and Cloete, 2016;Matos et al., 2019). Consider, for example, the warm water that remains in the piping network between the heater and the point of use after an event -the energy in this water also constitutes usage losses, since that water will cool down if not drawn shortly after the initial event. Assuming a conservative volume of 2 L in the piping between the heater and the point of use, each event will in fact comprise legitimate usage and 2 L unintentional usage loss. If the residual warm water in the piping is at a lower temperature than the TC set point, the net losses will be significantly less: the average volume used per user was 142 L/day with an average of 7.5 events/day, which means as much as 10 % of usage would have resulted in usage losses. As an example of unintended use, we have anecdotal evidence of users of a smart EWH controller who became aware that their casual workers were unnecessarily washing cars or cleaning floors with hot water -behaviour that was quickly remedied (Roux and Booysen, 2017;Booysen et al., 2019). It may be argued that this control is essentially the same as setting the thermostat to a lower temperature, but the EM control does better than just supplying a lower temperature on occasion; it ensures that hot water is delivered when needed. However, these savings will be highly dependent on the specifics of each user's behaviour, each installation's plumbing setup, including the spouts, and warrants further investigation.

Conclusion
Heating water is a substantial contributor to total domestic electrical energy usage in developing countries. Cost is a concern for users in these countries. The tend to pay a flat fee per kWh, which makes them sensitive to how much energy they use rather than when they use it. A broader concern is that electricity in these countries is generated mainly by burning fossil fuels, which leads to emission of greenhouse gasses. To investigate the potential electrical energy savings that can be achieved by applying optimal (schedule and temperature) control to energy-storing electric water heaters, we examined 30 water heaters over a period of 20 days. We used thermostat control (always on), which is the default mode of water heaters, as our baseline, and compared this strategy with three optimal control strategies. Our three strategies, using dynamic programming to ensure optimal heating, ensure comparable delivery at the outlet of the water heater. The paper describes our problem formulation and our algorithm for applying dynamic programming to the water heater. The first strategy ensures a temperature-matched output with equal volume, the second provides an energy-matched output with decreased but still-hot temperature and increased volume, and the third adapts the second method to ensure that Legionella is sterilised despite the lower temperatures. We found that temperature-matching gave a median saving of 7.9 %, without adversely affecting the temperature at which water is delivered; the energy-matching method gave a median saving of 17.8 %, with a reduction in energy lost to the environment as well as a reduction in energy lost due to unintentional use; and the energymatching method with daily Legionella sterilisation gave a median saving of 13.1 %. For all three strategies, the number of cold events did not increase from the baseline strategy. By using real world, not synthesised, hot water usage profiles, we determined the absolute best energy savings, with the fewest cold events, that can be achieved with scheduling and temperature control while causing the minimum of inconvenience to the consumer. However, it should be noted that the results are based on simulations performed with a one-node lumped-parameter model for the EWH thermal dynamics, which means that thermal stratification in the water heater was not taken into account. Further work is therefore needed to evaluate the effect of stratification on the energy savings and number of cold events.