Thinkings about the hugging face document

Single GPU Optimization Resource: Hugging Face Doc Method/tool Improves training speed Optimizes memory utilization Batch size choice Yes Yes Gradient accumulation No Yes Gradient checkpointing No Yes Mixed precision training Yes (No) Optimizer choice Yes Yes Data preloading Yes No DeepSpeed Zero No Yes torch.compile Yes No Parameter-Efficient Fine Tuning (PEFT) No Yes FP16 If your model doesn’t work well with mixed precision, for example if it wasn’t pretrained in mixed precision, you may encounter overflow or underflow issues which can cause NaN loss....

2024-03-11    2024-03-14    1100 words    6 min    Yuesir


からかい上手の高木さん Before The third season of Takagi-san (Japanese: からかい上手の高木さん) anime TV ended today (3.26), and there may not be a fourth season. Purposely went to catch up at 1 a.m. and felt a lot of emotions. Before learning that the theatrical version will be released in Japan in June,...

2022-03-26    102 words    1 min    Yuesir

Python Notebook

Life is not easy, exams too. Python Programming from Beginning to Practice is Python 3.5, while in Python 3.6 the internal algorithm of dict was rewritten, hence 3.6 dict is ordered, prior to this version it was unordered. Reference Links String formatting f'Results of the {event}'

2021-10-05    46 words    1 min    Yuesir