WeWorkBuddy

Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

arXiv:2401.00788v2 Announce Type: replace Abstract: The high cost of full-parameter fine-tuning (FFT) of Large Language Models (LLMs) has led to a series of parameter-efficient fine-tuning (PEFT) methods. However, it remains unclear which methods provide the best cost-performance trade-off at different model scales. We introduce Astraios, a suite of 28 instruction-tuned OctoCoder models using 7 tuning methods and 4 model sizes up to 16 billion parameters. Through investigations across 5 tasks an

Read original source
Source reference for Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models. Screenshot not yet available.
Source record · preview pending · arxiv.orgOpen the source for the full page.

Source record

Project analysis

Project analysis is not yet available. See the source record above.