Why AI Foundation Models Matter

Main Idea: Imagine a chef trained in French, Spanish and Italian cuisines being asked to cook Indian food. With no additional training, would the chef be able to cook a good Chicken Tikka Masala? Replace "chef" with "AI models" and "cooking" with "predictions" and you get the intuition behind transfer learning in AI. Foundation models or large language models hold the promise of transferring existing knowledge to new tasks without substantial additional training and are a key trend to observe for future AI applications across multiple industries.

What makes it possible: The innovative transformer architecture, massive scale in cloud and parallel computing and large troves of data are the key drivers for foundation models. Homogenization- convergence to just a few AI models that suffice for multiple applications, minimizes need for domain specific model development. Novel training approaches such as multimodal learning enable foundational models to learn across multiple data sources such as pictures, text and video as opposed to current siloed approaches like using only text data for training language models.

Representative Healthcare Use Cases: Consider the potential of a foundational model that gets trained on multimodal healthcare data such as patient EHRs, medical research publications, population health data and MRI images and its utilization for different use cases such as clinical decision support, patient care or clinical trials instead of bespoke model development for each use case.

Challenges, Risks and Costs: Results from current foundation models for many applications are barely passable. Adaptation, a step of fine tuning a foundation model to generate application specific results, can prove to be complex. Training models can run into millions of dollars. Convergence to just a few foundation models can introduce the risk from a single point of failure and perpetuate bias across multiple applications.

Three things to do:

1. Experiment with beginner friendly models using OpenAI's APIs

2. Check out Microsoft's commercially available large language API

3. Learn about the ethical and social risks posed by large language models

Home