Last year I wrote about a concept called the Modeling Lifecycle (Modeling Lifecycle). In that blog, I spent time addressing the many steps that are necessary for a predictive modeling project to be a success. Obviously, one of those critical steps is actual implementation of the model itself. Without that, you only have a fancy formula that doesn’t do much of anything for you.

The model implementation topic continues to garner increasing interest. In fact, I was scheduled to speak about the topic at sessions at the iCAS Community of Practice Event and the CAS Ratemaking, Product and Modeling Seminar in New Orleans. (The in-person event was cancelled due to COVID-19.) 

While insurers continue to devote more and more time and resources to predictive analytics, it would also benefit them to make sure they are devoting sufficient attention to model implementation.

Model implementation is a topic that can cover a wide range of aspects including both business and technical. However, one of the most important considerations should be to “begin with the end in mind.” That is, when starting a predictive modeling project, it is wise to think about questions such as:

• Who is going to be using the model?
• How is the model going to be used?
• Does the model need to be filed?
• How is the model going to be implemented?

Let’s dig deeper into each one of these questions.

Who is going to be using the model?

While seeming rather straightforward, this question can actually be quite complex and have multiple answers (which lead to multiple actions). At heart, this question tries to get at how sophisticated end users of the model results are, and how much training these users will need. Just as important, depending on the end users’ familiarity with modeling and the reasons for it, a considerable amount of effort may need to be devoted to education and change management. 

For example, an insurer may want to implement a claims triage model to help route (and eventually handle) simpler auto claims without involving a claims adjuster in the process. This would allow claims adjusters to spend more time on complex cases. The intent is to provide value to the customer as well as job enrichment to claims adjusters. But it may also be seen as “taking something away” or a threat to the job security of the claims adjusters. Thus, a potentially significant amount of ongoing effort may be needed to gain leadership support for the new model and to secure employee buy-in.

How is the model going to be used?

The answer to this question can differ depending on the line of business (commercial versus personal, e.g.) or the function the model applies to (pricing, underwriting, claims, etc.). This is generally getting at the concept of “decision” vs. “recommendation.” Often, a model used for personal lines pricing is a decision model. The result of the model may be the final price or a component of the price that the customer is charged. 

An underwriting model might instead provide more of a recommendation that helps guide a human operator to arrive at a final action. In the first case, there is no latitude to what the model output says. In the second case, there can be a wide range of latitude in how the model output is used. The physical system (IT process), education and support needed in implementation can look very different depending on how the model is used.

Does the model need to be filed?

The answer to this question, again, often depends on the line of business and function of the model, as well as the jurisdiction in which the model will be used. As actuaries, it is always important to adhere to professional standards and follow all applicable laws and regulations. However, the methods or data used can differ depending on whether or not a model needs to be filed. A modeler may revert to a more understandable methodology (generalized linear model, possibly) for a project if a high degree of transparency in a filing is required, for example. If not, then a more black box method such as a neural network might be selected. Or, a particularly new or sophisticated variable interaction or data element might not be utilized in the model if it has to be (publicly) filed in a state.

How is the model going to be implemented?

This question balances technical and business considerations. Depending on how integrated the model is into the insurer’s IT processes, certain data may not be available to the model at the time it is invoked. Or, the model may only be run with certain frequency, which can cause some of the variables in the model or the results to become stale. From a technical standpoint, if the model has to be recoded into an insurer’s IT framework, considerable additional work may be needed to accomplish that or perhaps, even a different methodology may need to be selected. That is opposed to when an API is able to be built to access the model in its native format.

In the end, there are many implementation considerations when a model is being constructed. And, while implementation appears toward the end of the modeling lifecycle, it must be considered early on in the process, since answers to the above key questions (and others) can greatly shape what is used in a model and how it is used. 

With multiple decades of experience in predictive modeling, the consulting actuaries at Pinnacle would welcome the opportunity to speak with you about your upcoming analytics projects to help position you for success.