Aran Komatsuzaki   @arankomatsuzaki   6/11/2021       

The efficiency of fine-tuning on GPUs depends on a lot of variables, and nobody tried benchmarking yet. There are multiple groups trying to port it to HF, so if this means you can use PyTorch + GPU, I believe it would become much easier to do fine-tuning with GPUs then :)

Posted by Aran Komatsuzaki