Scaling Instruction-Finetuned Language Models

What's new?

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

This paper explores the benefits scaling instruction finetuning (opens in a new tab) and how it improves performance on a variety of models (PaLM, T5), prompting setups (zero-shot, few-shot, CoT), and benchmarks (MMLU, TyDiQA). This is explored with the following aspects: scaling the number of tasks (1.8K tasks), scaling model size, and finetuning on chain-of-thought data (9 datasets used).

Finetuning procedure:

• 1.8K tasks were phrased as instructions and used to finetune the model
• Uses both with and without exemplars, and with and without CoT

Finetuning tasks and held out tasks shown below:

Capabilities & Key Results

• Instruction finetuning scales well with the number of tasks and the size of the model; this suggests the need for scaling number of tasks and size of model further
• Adding CoT datasets into the finetuning enables good performance on reasoning tasks
• Flan-PaLM has improved multilingual abilities; 14.9% improvement on one-shot TyDiQA; 8.1% improvement on arithmetic reasoning in under-represented languages
• Plan-PaLM also performs well on open-ended generation questions, which is a good indicator for improved usability
• Improves performance across responsible AI (RAI) benchmarks
• Flan-T5 instruction tuned models demonstrate strong few-shot capabilities and outperforms public checkpoint such as T5

The results when scaling number of finetuning tasks and model size: scaling both the size of the model and the number of finetuning tasks is expected to continue improving performance, although scaling the number of tasks has diminished returns.

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

The results when finetuning with non-CoT and CoT data: Jointly finetuning on non-CoT and CoT data improves performance on both evaluations, compared to finetuning on just one or the other.

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

In addition, self-consistency combined with CoT achieves SoTA results on several benchmarks. CoT + self-consistency also significantly improves results on benchmarks involving math problems (e.g., MGSM, GSM8K).

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

CoT finetuning unlocks zero-shot reasoning, activated by the phrase "let's think step-by-step", on BIG-Bench tasks. In general, zero-shot CoT Flan-PaLM outperforms zero-shot CoT PaLM without finetuning.

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

Below are some demonstrations of zero-shot CoT for PaLM and Flan-PaLM in unseen tasks.

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

Below are more examples for zero-shot prompting. It shows how the PaLM model struggles with repetitions and not replying to instructions in the zero-shot setting where the Flan-PaLM is able to perform well. Few-shot exemplars can mitigate these errors.

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

Below are some examples demonstrating more zero-shot capabilities of the Flan-PALM model on several different types of challenging open-ended questions:

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

Image Source: Scaling Instruction-Finetuned Language Models (opens in a new tab)

You can try Flan-T5 models on the Hugging Face Hub (opens in a new tab).