Distillation

In knowledge distillation, a powerful model generates many answers that are used to train a smaller model. The student thus takes on part of the capabilities but is faster and cheaper to run.

Many compact models that run on laptops or smartphones are created this way. Providers often specify in their terms of use whether their models may be used as teachers.