The actuarial profession is organized into societies with high educational and professional standards. This makes them institutionally conservative about adopting new modeling techniques. New methods from data science and AI present new opportunities for actuaries. They haven't yet been absorbed into actuarial practices largely because of training and compliance barriers. We propose solutions for bridging this gap between actuaries and data science.
While rudimentary forms of insurance date back to 1750 BC, modern mathematical actuarial practice began in the 1600’s with new forms of shipping, fire, and life insurance (Hagan, 2024). Today, a large insurance industry has at its heart a mature quantitative discipline that adheres to standards set by its own member-led societies, as well as state-mandated regulations. Actuarial culture is, in these respects, disciplined and responsible compared to many emerging fields of data science, machine learning, and AI. Waves of technical innovation from less regulated sectors have potential value for actuaries, and there are many in the insurance industry that would like to make the best use of these tools and techniques. Thus, the confrontation between new techniques from computer science – and the “tech” industry – and traditional actuarial mindsets has created both opportunity and confusion.
The barriers for actuaries adopting new data science methods has little to do with the algorithms themselves. Rather, it is due to high standards of methodological rigor used by actuaries, who are often professionally required to document their decisions, to be explicit about their assumptions and limitations, and to have clear structural models. Ultimately, actuaries evaluate their work by these standards of professional practice, not the accomplishment of a specific technical performance metric. When new techniques are not well-understood by the actuarial profession, they can appear to be inscrutable “black boxes”.
The goals of this article are threefold. First, we aim to support knowledge transfer between actuaries and data scientists, helping each culture understand the other, and facilitate cooperation without suspicion. Second, we aim to ease or guide adoption of new technologies into actuarial practice. We ultimately would like actuaries to have an expanded range of modeling options. But we do not want actuaries to compromise the integrity that is the hallmark of their discipline. Thus, our third goal is to support new binding standards, developed by actuarial professional organizations, that rise to the needs of evolving practice.
This paper is organized as follows. In Section II, we discuss how insurance governs itself through professional societies, and how this shapes their practice. In Section III, we discuss how actuaries approach modeling, and what makes them different from data scientists more generally. In Section IV, we discuss how the actuarial profession, and insurance industry more broadly, can better adapt to new techniques from data science, without compromising high professional standards.
Insurance is, at its core, about measuring and managing risk. As there are many kinds of risk, there are many kinds of insurance. In the United States, the major branches of insurance risk are Pensions, Health, Life, & P&C coverages. Risks can be covered by three types of providers - the governments (Social Security, Medicare and public pension plans being the most well known), a corporation (whether a pension or health or other group coverages) or a company selling to individuals (for life, health or and property and casualty (P&C) coverages). Life insurance is a contract between the insurer and policyholder guaranteeing a certain payout of funds in the event of the policyholder’s death. P&C includes a variety of similar contracts that are triggered by other kinds of events or calamities, such as automobile accidents, natural disasters, and injuries. Beyond covering different risks (to property and business, and to individual life), insurance can vary widely in the time horizon of the coverage. P&C insurance typically renews each year by mutual agreement of the company and the individual, while a life insurance policy may obligate a company over many decades.
In the US, actuarial practice is rigorously, and somewhat uniquely, self-regulated by the Actuarial Standards Board (ASB) and Actuarial Board for Counseling and Discipline (ABCD). The initial certification to become an actuary is governed by two professional societies, the Society of Actuaries (SOA), representing all areas of insurance, and the Casualty Actuarial Society (CAS), which represents primarily P&C insurance. Members are admitted to these societies only after a rigorous program of education and testing, which are reinforced by ongoing continuing education requirements. In 1964, these and other actuarial societies approved the creation of an umbrella organization, the American Academy of Actuaries, to represent the entire US actuarial profession. Associates or Fellows of the constitutive societies (such as SOA and CAS) who meet stated experience requirements can become members of the Academy. All three organizations have also agreed to a common “Code of Conduct” requirement for their members.
The Actuarial Standards Board (ASB) is responsible for developing professional standards for all actuaries, via Actuarial Standards of Practice (ASoPs). The board has 9 members who serve three year terms and are selected by representatives of the Academy, CAS and SOA. The board accomplishes its work through committees focused on specific areas of practice. The ASOPs describe the procedures an actuary should follow when performing actuarial services and identify what the actuary should disclose when communicating the results of those services. In addition there is a separate body, the Actuarial Board for Counseling and Discipline (ABCD) that monitors, via the actuarial professional membership, compliance with these standards.
Actuarial methods vary widely across different kinds of actuarial work. One popular tool in ratemaking for P&C policies is Generalized Linear Models, a generalization of linear regression. These are taught in actuarial textbooks (Anderson et al., 2007; Goldburd et al., 2016; Casualty Actuarial Society, 1990) and form the heart of the actuarial training curriculum. GLMs have many benefits. The mathematical assumptions are well understood and documented. The relationships between variables are clear. They are interpretable by design. Related techniques include Method-of-Moments or Maximum Likelihood distribution fitting (i.e. to find the best fit Poisson frequency distribution and Gamma severity distribution). GLMs are a technique that aligns very well with established actuarial principles and standards.
Using other techniques is not prohibited for actuaries, and indeed actuarial continuing education requirements lead professionals to discover new techniques that go beyond their initial training. However, when a new tool or method is not standardized or part of the educational curriculum, there are other barriers to its adoption. The ASOPs require that for any actuarial model, its users must be informed about the method by which it was derived, its limitations, and the rationale for its use. Regulatory boards will likely want more in-depth discussion as to why a new method was used instead of a more traditional approach.
An example of a data science method that has only recently become adopted by some actuaries is boosted decision trees, which is popularized by open source libraries like XGBoost. A decision tree is a machine-learnt model that implements a classification or regression task by walking through a ‘tree’ of simple decisions based on the input datum. Individual trees are highly interpretable: it’s clear exactly why the model predicts one outcome or another, because the decisions at each node are explicit. With boosted tree methods, many ‘weak’ trees are composed into an ‘ensemble’ of models that are trained together to reduce overall error. The resulting ensemble often outperforms simpler models, but at the expense of interpretability. The model decision is a weighted sum of other decisions.
Through more flexible techniques like boosted trees, data science takes the approach of letting the data speak for itself. Data mining tools reveal correlations and develop a model whose primary purpose is to fit the data. This is counter to a typical actuarial practice, in which the structural relationships driving the data are presumed to be understood, and ‘learning’ is used only to fit the coefficients of a model. In a data science regression, the number of free parameters may be far higher than in a GLM; this can improve predictive accuracy, but at the expense of interpretability.
Actuaries are, in this sense, much more like academic social scientists than computer scientists. Economists, for example, build models to help understand the structure of the phenomenon they are observing. Theoretical understanding is needed to guide and justify the use of policy instruments, such as changes to the inflation rate. For this reason, model clarity is often as critical as forecasting performance. Similarly, bankers need to provide to someone who is denied credit an account of the reasons for the rejection and recommended remedies. Thus, insurance is not the only sector in which data science methods that prioritize predictive power over interpretability are uncommon.
Actuarial practice can learn from data science. In principle, any algorithm can be used by actuaries, as long as it is used in a way that adheres to professional standards. Actuaries, via continuing education opportunities, will be able to incorporate them into responsible practices. Open source tools, such as XGBoost, are percolating into practice through some of the more technically ambitious groups. They can bring additional statistical power to bear on the core problem of managing risk. Likewise, exploratory analysis and data mining can play a role in the actuarial process in discovering new modeling ideas, as long as more robust methods of structural inference are used to make decisions about policies. Actuaries will be more innovative and effective if they can become “T-shaped modelers”, who have a wide breadth with depth in one area of modeling (Molnar, 2024).
The problem that remains before us is this: actuarial practice is, for good reason, conservative in its approaches, operating with a high standard of professionalism, regulation, and standardization. This has made it relatively slow to adopt new methods from the wider technical fields of data science and AI. How can actuarial practice capture the benefits of these new techniques without sacrificing the rigor and transparency at the heart of actuarial discipline?
First, we note that there are no hard regulator barriers to adoption. Indeed, machine learning methods have in several cases already received approval in P&C rate filings (Mason, 2024). Filing for approval is done state by state, and filings must include the defense of new practices. Some of the burden of progress lands on actuaries themselves as they innovate and justify their work using established practices.
Second, there is engagement with standardization. The Actuarial Standards Board convenes committees to offer standards on topics that are either motivated by new regulations, or by professional concerns. The focus of the standards is on the principles and transparency needed to be applied to a specific area of practice. Thus, the most relevant current applicable standards that apply to those using data science techniques include ASOP’s 1, 12, 23, 41,& 56. However, as the states continue to have concerns about the governance of the use of AI, there may be a need for a standard to address a possible company/professional certification needed to address regulatory requirements and concerns. This would allow the states to avoid the laborious and complicated process to keep updating laws across 50+ jurisdictions as practice and innovation evolve. As ASoPs are focused on the principles that define what practitioners must do, and what they should consider when undertaking a course of action, and the required documentation, they are an effective companion to allow both oversight and innovation to occur via state regulatory requirements. Those seeking to introduce new data science methods to actuaries would be well-advised to map new techniques to those principles.
Finally, better clarity about roles and responsibilities will ease the introduction of data science into actuarial practice. A clearer delineation of actuarial work that is for internal purposes (for example, establishment of underwriting criteria, or something that doesn't necessarily have to be filed/approved with states), and something that is subject to regulation and therefore needs to be entirely explainable/defensible, can create a space for actuaries to experiment with new methods before facing the regulatory requirements of justify their use for a critical function. Being an actuary is a career, and data science is a skillset. One way or another, actuaries will increasingly use data science in their work, and will encounter non-actuarial data scientists in other parts of their own industry. Indeed, data scientists have much to learn from actuaries as well, as ASoP standards are generally good standards for the “responsible” and “trustworthy” practices which are so desired across the technology sector. Communication and professional respect, and clarity about the division of labor, will enable data scientists and actuaries to work better together.
Anderson, D., Feldblum, S., Modlin, C., Schirmacher, D., Schirmacher, E. and Thandi, N., 2007. A Practitioner’s Guide to Generalized Linear Models—a foundation for theory, interpretation and application. Towers Watson.
Casualty Actuarial Society, 1990. Foundations of Casualty Actuarial Science. Casualty Actuarial Society.
Goldburd, M., Khare, A., Tevet, D. and Guller, D., 2016. Generalized linear models for insurance rating. Casualty Actuarial Society, CAS Monographs Series, 5, p.77.
Hagan, K. The History of Actuarial Science, April 2024.
Mason, T. (2024, August 27). Machine learning methods getting approval in P&C rate filings. S&P Global Market Intelligence.
Molnar, C. Modeling Mindsets, May 2024.
