Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models
arXiv:2401.00788v2 Announce Type: replace Abstract: The high cost of full-parameter fine-tuning (FFT) of Large Language Models (LLMs) has led to a series of parameter-efficient fine-tuning (PEFT) methods. However, it remains unclear which methods provide the best cost-performance trade-off at different model scales. We introduce Astraios, a suite of 28 instruction-tuned OctoCoder models using 7 tuning methods and 4 model sizes up to 16 billion parameters. Through investigations across 5 tasks an
Read original sourceSource record
Project analysis
Project analysis is not yet available. See the source record above.