## Python identifier conventions for alignment with the Powell Unified Framework

### Background

Dr. Warren Powell’s unified framework shows great promise for unifying the formalisms of at least a dozen different fields. Using his framework enables easier access to thinking patterns in these other fields that might be beneficial and informative to the sequential decision problem at hand. Traditionally, this kind of problem would be approached from the *reinforcement learning* perspective. However, using Dr. Powell’s wider and more comprehensive perspective almost certainly provides additional value.

Here is information on Dr. Powell’s perspective on Sequential Decision Analytics.

Dr. Powell places strong emphasis on having a consistent and well thought out notation - to him it is akin to a language that, once you understand it, unlocks all the potential benefits of sequential decision analytics. It is best to standardize this language among the communities. He works through a complete example in chapter 9 of his impressive textbook Reinforcement Learning and Stochastic Optimization: A Unified Framework for Sequential Decisions. **Section 9.2 Notational Style**, in particular, lays down some valuable conventions.

### Motivation

In order for me to have a strong mapping between Python code that follows Dr. Powell’s Unified Framework (PUF) and the mathematics from the unified framework, I find it useful to follow the conventions mentioned below. In the software community it is a best practice to use descriptive Python identifier (i.e. variable) names. However, I tend to lean towards the more ‘cryptic’ (i.e. mathematical) way of naming things in order to gain a less ‘noisy’ mapping between the mathematics and the code.

I experimented with making use of Greek symbols as well as super/sub scripts in notebooks but, until now, have not had consistent success with that, in particular when I tried to do this in Google Colab notebooks. Maybe things will be much easier in this regard with the *Julia* language.

### Mapping

It is helpful to consider my mapping in the context of chapter 9, and in particular, *Section 9.2 Notational Style* of Dr. Powells book. Here is a summary of my mapping between the mathematics and the corresponding variable names in the Python code:

- Superscripts
- variable names have a
*double*underscore to indicate a superscript - \(X^{\pi}\): has code
`X__pi`

, is read ‘X pi’

- variable names have a
- Subscripts
- variable names have a
*single*underscore to indicate a subscript - \(S_t\): has code
`S_t`

, is read ‘S at t’ - \(M^{Spend}_t\) has code
`M__Spend_t`

which is read: ‘M Spend at t’

- variable names have a
- Arguments
- collection variable names may have argument information added
- \(X^{\pi}(S_t)\): has code
`X__piIS_tI`

, is read ‘X pi in S at t’ - the surrounding
`I`

’s are used to imitate the parentheses around the argument

- Next time/iteration
- variable names that indicate one step in the future are quite common
- \(R_{t+1}\): has code
`R_tt1`

, is read ‘R at t+1’ - \(R^{n+1}\): has code
`R__nt1`

, is read ‘R at n+1’

- Rewards
- State-independent terminal reward and cumulative reward
- \(F\): has code
`F`

for terminal reward - \(\sum_{n}F\): has code
`cumF`

for cumulative reward

- \(F\): has code
- State-dependent terminal reward and cumulative reward
- \(C\): has code
`C`

for terminal reward - \(\sum_{t}C\): has code
`cumC`

for cumulative reward

- \(C\): has code

- State-independent terminal reward and cumulative reward
- Vectors where components use different names
- \(S_t(R_t, p_t)\): has code
`S_t.R_t`

and`S_t.p_t`

, is read ‘S at t in R at t, and, S at t in p at t’ - the code implementation is by means of a named tuple
`self.State = namedtuple('State', SNames)`

for the ‘class’ of the vector`self.S_t`

for the ‘instance’ of the vector

- \(S_t(R_t, p_t)\): has code
- Vectors where components reuse names
- \(x_t(x_{t,GB}, x_{t,BL})\): has code
`x_t.x_t_GB`

and`x_t.x_t_BL`

, is read ‘x at t in x at t for GB, and, x at t in x at t for BL’ - the code implementation is by means of a named tuple
`self.Decision = namedtuple('Decision', xNames)`

for the ‘class’ of the vector`self.x_t`

for the ‘instance’ of the vector

- \(x_t(x_{t,GB}, x_{t,BL})\): has code
- Use of mixed-case variable names
- to reduce confusion, sometimes the use of
*mixed-case*variable names are preferred (even though it is not a best practice in the Python community), reserving the use of underscores and double underscores for math-related variables

- to reduce confusion, sometimes the use of

### Pronunciation

I find it very helpful to read the mathematical variables by pronouncing them. It is important (to me) to do this in a well-defined way and consistently. Here are a few conventions in this regard:

- Flavors
- \(M^{Spend}\) has code
`M__Spend`

which is read: ‘M Spend’ - Note that the ‘flavor’ is orally combined with the main name, i.e. it is read ‘M Spend’ and not ‘M super Spend’

- \(M^{Spend}\) has code
- Time/Iteration indices
- \(R_{t+1}\): has code
`R_tt1`

, is read ‘R at t+1’ - \(R^{n+1}\): has code
`R__nt1`

, is read ‘R at n+1’ - Note the preposition
**at**which always precedes the mention of the index

- \(R_{t+1}\): has code
- Function arguments
- \(S_t(R_t)\): is read ‘S at t in R at t’
- Note the preposition
**in**which always precedes the mention of the argument(s)

- Vectors where components reuse names
- \((x_{t,GB}, x_{t,BL})\): is read ‘x at t for GB, and, x at t for BL’
- Note the preposition
**for**for the different components of the vector

### Conclusion

Admittedly, the above scheme, if you are not comfortable with the PUF, makes it harder to read and understand the code. However, once you have mastered the conventions in the PUF, it should make the code much *easier* to digest. In the future, we may be able to reliably make use of natural superscripts and subscripts in notebooks as well as Greek symbols which would make the mapping even less noisy.