Indlela yokusebenzisa ama-GPU e-NVIDIA ekuQeqesheni nge-AI [Ividiyo kanye neMibuzo]

Impendulo emfushane: Sebenzisa ama-GPU e-NVIDIA ekuqeqeshweni kwe-AI ngokuqinisekisa kuqala ukuthi umshayeli kanye ne-GPU ziyabonakala nge -nvidia-smi, bese ufaka i-framework/CUDA stack ehambisanayo bese usebenzisa isivivinyo esincane esithi “model + batch on cuda”. Uma ushaya i-out of memory, yehlisa usayizi we-batch bese usebenzisa ukunemba okuxubile, ngenkathi uqapha ukusetshenziswa, imemori, kanye namazinga okushisa.

Izinto ezibalulekile okufanele uzicabangele:

Ukuhlolwa kwesisekelo: Qala nge -nvidia-smi; lungisa ukubonakala komshayeli ngaphambi kokufaka amafreyimu.

Ukuhambisana kwe-stack: Gcina umshayeli, isikhathi sokusebenza se-CUDA, kanye nezinguqulo zohlaka zihambisana ukuze kuvinjelwe ukuphahlazeka nokufakwa okubuthakathaka.

Impumelelo encane: Qinisekisa ukuthi i-CUDA ithola i-pass eyodwa ngaphambi kokuthi uthuthukise izivivinyo.

Isiyalo se-VRAM: Thembela ekuqondeni okuxubile, ukuqongelela kwe-gradient, kanye nokukhomba ukuze kulingane namamodeli amakhulu.

Umkhuba wokuqapha: Landelela ukusetshenziswa, amaphethini enkumbulo, amandla, kanye nezinga lokushisa ukuze ubone izithiyo kusenesikhathi.

Izihloko ongase uthande ukuzifunda ngemva kwalesi:

🔗 Indlela yokwakha i-ejenti ye-AI
Yakha ukuhamba komsebenzi kwe-ejenti yakho, amathuluzi, inkumbulo, kanye nezivikelo zokuphepha.

🔗 Indlela yokusebenzisa amamodeli e-AI
Setha izindawo, amamodeli ephakheji, bese uthumela ekukhiqizweni ngokuthembekile.

🔗 Ungakala kanjani ukusebenza kwe-AI
Khetha izilinganiso, sebenzisa ukuhlolwa, bese ulandelela ukusebenza ngokuhamba kwesikhathi.

🔗 Indlela yokwenza imisebenzi ngokuzenzakalelayo nge-AI
Yenza umsebenzi ophindaphindwayo ube ngokuzenzakalelayo ngezikhuthazo, imisebenzi yokusebenza, kanye nokuhlanganiswa.

1) Isithombe esikhulu - lokho okwenzayo uma "uqeqesha nge-GPU" 🧠⚡

Uma uqeqesha amamodeli e-AI, ikakhulukazi wenza intaba yezibalo ze-matrix. Ama-GPU akhelwe lolo hlobo lomsebenzi ohambisanayo, ngakho-ke izinhlaka ezifana ne-PyTorch, i-TensorFlow, ne-JAX zingalayisha ukuphakamisa okunzima ku-GPU. (PyTorch CUDA docs, TensorFlow install (pip), JAX Quickstart)

Empeleni, "ukusebenzisa ama-GPU e-NVIDIA ekuqeqesheni" kuvame ukusho ukuthi:

• Amapharamitha akho emodeli abukhoma (ikakhulukazi) ku-GPU VRAM

• Amaqoqo akho asuswa ku-RAM aye ku-VRAM isinyathelo ngasinye

• I-forward pass yakho kanye ne-backprop yakho isebenza kuma-CUDA kernels (Umhlahlandlela Wokuhlela we-CUDA)

• Izibuyekezo zakho ze-optimizer zenzeka ku-GPU (ngokufanele)

• Uqapha amazinga okushisa, inkumbulo, ukusetshenziswa ukuze ungapheki lutho 🔥 (NVIDIA nvidia-smi docs)

Uma lokho kuzwakala sengathi kuningi, ungakhathazeki. Ngokuvamile kuba uhlu lokuhlola kanye nemikhuba embalwa oyakhayo ngokuhamba kwesikhathi.

2) Yini eyenza inguqulo enhle yokusetha ukuqeqeshwa kwe-NVIDIA GPU AI 🤌

Lesi isigaba esithi “ungakhi indlu ngejeli”. Ukusetha okuhle kokuthi Ungayisebenzisa kanjani i-NVIDIA GPU yoQeqesho lwe-AI kungenye yezindlela eziphansi kakhulu. Izindlela eziphansi kakhulu zizinzile. Ukuqina kuyashesha. Ukushesha kuyashesha…kahle, kuyashesha 😄

Isethaphu sokuqeqeshwa okuqinile ngokuvamile sinalokhu:

• I-VRAM eyanele yosayizi we-batch yakho + imodeli + izimo ze-optimizer

  • I-VRAM ifana nendawo yesutikheyisi. Ungapakisha ngobuhlakani, kodwa awukwazi ukupakisha okungenamkhawulo.

• I-software stack efanisiwe (umshayeli + isikhathi sokusebenza se-CUDA + ukuhambisana kohlaka) (PyTorch Get Started (CUDA selector), TensorFlow install (pip))

• Isitoreji esisheshayo (i-NVMe isiza kakhulu kumasethi wedatha amakhulu)

• I-CPU ehloniphekile + i-RAM ukuze ukulayisha idatha kungalambisi i-GPU (PyTorch Performance Tuning Guide)

• Ukupholisa kanye negumbi lokuvula amandla (elilinganiselwe kancane kuze kube yilapho lingasekho 😬)

• Indawo ephindaphindwayo (i-venv/conda noma izitsha) ukuze ukuthuthukiswa kungabi yisiphithiphithi (ukubuka konke kwe-NVIDIA Container Toolkit)

Futhi into eyodwa abantu abayiqayo:

• Umkhuba wokuqapha - uhlola inkumbulo ye-GPU kanye nokusetshenziswa kwayo njengoba uhlola izibuko ngenkathi ushayela. (amadokhumenti e-NVIDIA nvidia-smi)

3) Ithebula Lokuqhathanisa - izindlela ezidumile zokuqeqesha ngama-GPU e-NVIDIA (ngezimpawu ezingavamile) 📊

Ngezansi kunephepha lokukhohlisa elisheshayo elithi “yiliphi elifanelana?”. Amanani ayinto engajwayelekile (ngoba iqiniso liyahlukahluka), futhi yebo elinye lala maseli liyinkimbinkimbi kancane, ngamabomu.

4) Isinyathelo sokuqala - qinisekisa ukuthi i-GPU yakho ibonwe kahle 🕵️♂️

Ngaphambi kokufaka izinto eziyishumi nambili, qinisekisa izisekelo.

Izinto ofuna ukuba yiqiniso ngazo:

• Umshini ubona i-GPU

• Umshayeli we-NVIDIA ufakwe kahle

• I-GPU ayibambeki ekwenzeni okunye

• Ungabuza ngokuthembekile

Isheke lakudala yileli:

• nvidia-smi (amadokhumenti e-NVIDIA nvidia-smi)

Okufunayo:

• Igama le-GPU (isb., i-RTX, uchungechunge lwe-A, njll.)

• Inguqulo yomshayeli

• Ukusetshenziswa kwememori

• Izinqubo zokuqalisa (NVIDIA nvidia-smi docs)

Uma i-nvidia-smi yehluleka, yima lapho. Ungafaki amafreyimu okwamanje. Kufana nokuzama ukubhaka isinkwa lapho ihhavini lakho lingaxhunywanga. (NVIDIA System Management Interface (NSVMI))

Inothi elincane lomuntu: ngezinye izikhathi i-nvidia-smi iyasebenza kodwa ukuqeqeshwa kwakho kusahluleka ngoba isikhathi sokusebenza se-CUDA esisetshenziswa uhlaka lwakho asihambisani nokulindelwe ngumshayeli. Lokho akusikho ukuthi uyisiwula. Yilokho nje... kunjalo 😭 (PyTorch Get Started (CUDA selector), TensorFlow install (pip))

5) Yakha i-software stack - abashayeli, i-CUDA, i-cuDNN, kanye "nomdanso wokuhambisana" 💃

Yilapho abantu belahlekelwa khona amahora. Icebo liwukuthi: khetha indlela bese unamathela kuyo.

Inketho A: I-CUDA ehlanganiswe ngohlaka (ngokuvamile elula)

Ama-PyTorch amaningi akha athunyelwa ngesikhathi sawo sokusebenza se-CUDA, okusho ukuthi awudingi ithuluzi eligcwele le-CUDA elifakwe ohlelweni lonke. Udinga nje umshayeli we-NVIDIA ohambisanayo. (I-PyTorch Get Started (CUDA selector), Izinguqulo Zangaphambilini ze-PyTorch (amasondo e-CUDA))

Izinzuzo:

• Izingxenye ezimbalwa ezihambayo

• Ukufakwa okulula

• Kungaphindeka kakhudlwana ngokwemvelo ngayinye

Ububi:

• Uma uxuba izindawo ngokunganaki, ungadideka

Inketho B: Ithuluzi le-CUDA lesistimu (ukulawula okwengeziwe)

Ufaka i-CUDA toolkit ohlelweni bese uvumelanisa konke nalo. (Amadokhumenti e-CUDA Toolkit)

Izinzuzo:

• Ukulawula okwengeziwe kokwakhiwa ngokwezifiso, amathuluzi athile akhethekile

• Ilungele ukuhlanganisa imisebenzi ethile

Ububi:

• Izindlela ezengeziwe zokungafani izinguqulo bese ukhala buthule

i-cuDNN kanye ne-NCCL, ngokwemibono yabantu

• -cuDNN isheshisa ama-primitive okufunda okujulile (ama-convolutions, ama-RNN bits, njll.) (amadokhumenti e-NVIDIA cuDNN)

• I-NCCL iyilabhulali esheshayo "yokuxhumana kwe-GPU-to-GPU" yokuqeqeshwa kwe-multi-GPU (ukubuka konke kwe-NCCL)

Uma wenza ukuqeqeshwa kwe-multi-GPU, i-NCCL ingumngane wakho omkhulu - futhi, ngezinye izikhathi, umuntu ohlala naye othanda isimo sengqondo esibi. (Isifinyezo se-NCCL)

6) Ukuqeqeshwa kwakho kokuqala kwe-GPU (isibonelo sengqondo se-PyTorch) ✅🔥

Ukuze ulandele Indlela yokusebenzisa ama-GPU e-NVIDIA ekuQeqesheni kwe-AI, awudingi iphrojekthi enkulu kuqala. Udinga impumelelo encane.

Imibono eyinhloko:

• Thola idivayisi

• Hambisa imodeli ku-GPU

• Thutha ama-tensor ku-GPU

• Qinisekisa ukuthi ama-forward pass agijima lapho (PyTorch CUDA docs)

Izinto engihlala ngizihlola kusenesikhathi:

• i-torch.cuda.is_available() ibuyisela i-True (torch.cuda.is_available)

• next(model.parameters()).device ibonisa i-cuda (Ingxoxo ye-PyTorch: hlola imodeli ku-CUDA)

• Iphasi elilodwa eliya phambili aliphazamisi

• Imemori ye-GPU iyakhuphuka uma uqala ukuqeqeshwa (uphawu oluhle!) (NVIDIA nvidia-smi docs)

Okuvamile ukuthi “kungani kuhamba kancane?”

• I-dataloader yakho ihamba kancane kakhulu (i-GPU ayilindeki) (PyTorch Performance Tuning Guide)

• Ukhohliwe ukuhambisa idatha ku-GPU (oops)

• Usayizi weqembu mncane (i-GPU ayisetshenziswa kahle)

• Wenza ukucubungula kwe-CPU okunzima esinyathelweni sokuqeqeshwa

Futhi, yebo, i-GPU yakho izovame ukubonakala “ingamatasa kangako” uma inkinga iwukuba nedatha. Kufana nokuqasha umshayeli wemoto yomjaho bese umenza alinde uphethiloli njalo uma efika.

7) Umdlalo we-VRAM - usayizi we-batch, ukunemba okuxubile, futhi awuqhumi 💥🧳

Izinkinga eziningi zokuqeqeshwa okusebenzayo zigcina engqondweni. Uma ufunda ikhono elilodwa, funda ukuphathwa kwe-VRAM.

Izindlela ezisheshayo zokunciphisa ukusetshenziswa kwememori

• Ukunemba okuxubile (FP16/BF16)

  • Ngokuvamile i-boost enkulu yesivinini nayo. Ukunqoba-ukunqoba-ukunqoba 😌 (amadokhumenti e-PyTorch AMP, umhlahlandlela wokunemba okuxubile we-TensorFlow)

• Ukuqongelela kwe-gradient

  • Lingisa usayizi omkhulu we-batch ngokuqongelela ama-gradients ngezinyathelo eziningi (amadokhumenti okuqeqesha e-Transformers (i-gradient accumulation, fp16))

• Ubude obuncane bokulandelana / usayizi wokunqampuna

  • Kunonya kodwa kuyasebenza

• Ukuhlola kokusebenza

  • Shintshanisa ukubala ngememori (ukubuyisela ukusebenza ngesikhathi sokubuyela emuva) (torch.utils.checkpoint)

• Sebenzisa i-lighter optimizer

  • Amanye ama-optimizer agcina izimo ezengeziwe ezihlafuna i-VRAM

Umzuzu othi “kungani i-VRAM isagcwele ngemva kokuyeka?”

Amafreyimu avame ukugcina imemori yokusebenza. Lokhu kuyinto evamile. Kubukeka kuyesabeka kodwa akuhlali kuvuza. Ufunda ukufunda amaphethini. (PyTorch CUDA semantics: caching allocator)

Umkhuba osebenzayo:

• Iwashi elibekiwe vs imemori egciniwe (ngokuqondene nohlaka) (PyTorch CUDA semantics: caching allocator)

• Ungesabi ngenombolo yokuqala eyesabekayo 😅

8) Yenza i-GPU isebenze ngempela - ukulungiswa kokusebenza okufanele isikhathi sakho 🏎️

Ukuthola "ukuqeqeshwa kwe-GPU kusebenze" kuyisinyathelo sokuqala. Ukukuthola ngokushesha kuyisinyathelo sesibili.

Ukulungiswa okunomthelela omkhulu

• Khulisa usayizi we-batch (kuze kube yilapho kubuhlungu, bese uhlehla kancane)

• Sebenzisa imemori ephiniwe kuma-dataloaders (amakhophi asheshayo okusingatha kudivayisi) (PyTorch Performance Tuning Guide, PyTorch pin_memory/non_blocking tutorial)

• Khulisa abasebenzi be-dataloader (qaphela, abaningi kakhulu bangaba nemiphumela emibi) (PyTorch Performance Tuning Guide)

• Landa ama-batch kusengaphambili ukuze i-GPU ingahlali phansi

• Sebenzisa ama-ops ahlanganisiwe / ama-kernels alungiselelwe uma etholakala

• Sebenzisa ukunemba okuxubile (futhi, kuhle kakhulu) (PyTorch AMP docs)

Inkinga enganakwa kakhulu

Ipayipi lakho lokugcina kanye nokucubungula kusengaphambili. Uma isethi yedatha yakho inkulu futhi igcinwe kudiski ehamba kancane, i-GPU yakho iba yi-space heater ebizayo. I-space heater ethuthukisiwe kakhulu, ekhazimulayo kakhulu.

Futhi, ukuvuma okuncane: "Ngilungise" imodeli ihora lonke kodwa ngaqaphela ukuthi ukubhala phansi kwakuyimbangela yenkinga. Ukuphrinta kakhulu kunganciphisa ukuqeqeshwa. Yebo, kungakwenza.

9) Ukuqeqeshwa kwe-Multi-GPU - i-DDP, i-NCCL, kanye nokukhulisa ngaphandle kwesiphithiphithi 🧩🤝

Uma usufuna amamodeli anesivinini esikhulu noma amakhulu, uthola ama-GPU amaningi. Yilapho izinto ziba zimbi khona.

Izindlela ezivamile

• Idatha Ehambisanayo (i-DDP)

  • Hlukanisa amaqoqo kuwo wonke ama-GPU, vumelanisa ama-gradients

  • Ngokuvamile inketho "enhle" ezenzakalelayo (PyTorch DDP docs)

• Imodeli Ehambisanayo / Ehambisanayo Ye-Tensor

  • Hlukanisa imodeli phakathi kwama-GPU (kumamodeli amakhulu kakhulu)

• Iphayiphi Elihambisanayo

  • Hlukanisa izendlalelo zemodeli zibe yizigaba (njengomugqa wokuhlanganisa, kodwa kuma-tensor)

Uma uqala, ukuqeqeshwa kwesitayela se-DDP kuyindlela enhle. (Isifundo se-PyTorch DDP)

Amathiphu awusizo e-multi-GPU

• Qiniseka ukuthi ama-GPU anekhono elifanayo (ukuxuba ithini eliyimbangela yenkinga)

• Ukuxhumeka kwewashi: I-NVLink vs i-PCIe ibalulekile emisebenzini enzima yokuvumelanisa (Isifinyezo se-NVIDIA NVLink, amadokhumenti e-NVIDIA NVLink)

• Gcina osayizi be-batch ngayinye ye-GPU belinganisiwe

• Ungayinaki i-CPU kanye nesitoreji - ama-GPU amaningi angakhulisa izithiyo zedatha

Futhi yebo, amaphutha e-NCCL angazwakala njengempicabadala egoqwe ngemfihlakalo egoqwe ngokuthi “kungani manje”. Awuqalekisiwe. Mhlawumbe. (Isifinyezo se-NCCL)

10) Ukuqapha kanye nokubhala iphrofayili - izinto ezingathandeki ezikongela amahora 📈🧯

Awudingi amadeshibhodi amahle ukuze uqale. Udinga ukuqaphela uma kukhona okungasebenzi.

Izimpawu ezibalulekile okufanele uzibuke

• Ukusetshenziswa kwe-GPU: ingabe ihlala iphakeme noma inameva?

• Ukusetshenziswa kwememori: okuzinzile, okukhuphukayo, noma okungajwayelekile?

• Ukudonsa amandla: ukwehla okungavamile kungasho ukusetshenziswa okungaphansi

• Izinga lokushisa: amazinga okushisa aphezulu ahlala isikhathi eside angathuthukisa ukusebenza kahle

• Ukusetshenziswa kwe-CPU: izinkinga zombhobho wedatha zivela lapha (Umhlahlandlela Wokulungisa Ukusebenza kwe-PyTorch)

Indlela yokucabanga ngephrofayili (inguqulo elula)

• Uma i-GPU ingasetshenziswa kahle - idatha noma i-CPU ibopha

• Uma i-GPU iphezulu kodwa ihamba kancane - ukungasebenzi kahle kwe-kernel, ukunemba, noma ukwakheka kwemodeli

• Uma isivinini sokuqeqesha sehla ngokungahleliwe - ukucindezelwa kokushisa, izinqubo zangemuva, ukuphazamiseka kwe-I/O

Ngiyazi, ukuqapha kuzwakala kungathandeki. Kodwa kufana nokufaka i-floss. Kuyacasula, bese impilo yakho ithuthuka ngokuzumayo.

11) Ukuxazulula izinkinga - okusolwayo okuvamile (kanye nalezo ezingavamile) 🧰😵💫

Lesi sigaba ngokuyisisekelo: "izinkinga ezinhlanu ezifanayo, kuze kube phakade."

Inkinga: I-CUDA ayisekho enkumbulo

Ukulungiswa:

• nciphisa usayizi weqembu

• sebenzisa ukunemba okuxubile (amadokhumenti e-PyTorch AMP, umhlahlandlela wokunemba okuxubile we-TensorFlow)

• ukuqongelela kwe-gradient (amadokhumenti okuqeqesha ama-Transformers (ukuqongelela kwe-gradient, fp16))

• ukusebenza kwe-checkpoint (torch.utils.checkpoint)

• vala ezinye izinqubo ze-GPU

Inkinga: Ukuqeqeshwa kusebenza ngengozi ku-CPU

Ukulungiswa:

• qinisekisa ukuthi imodeli ithuthelwe e -cuda

• qinisekisa ukuthi ama-tensor athuthelwe ku -cuda

• hlola ukulungiselelwa kwedivayisi yohlaka (amadokhumenti e-PyTorch CUDA)

Inkinga: Ukuphahlazeka okungavamile noma ukufinyelela imemori okungekho emthethweni

Ukulungiswa:

• qinisekisa ukuhambisana komshayeli + nesikhathi sokusebenza (i-PyTorch Get Started (CUDA selector), ukufakwa kwe-TensorFlow (pip))

• zama i-env ehlanzekile

• nciphisa ukusebenza ngokwezifiso

• sebenzisa kabusha ngezilungiselelo ze-deterministic-ish ukuze ukhiqize kabusha

Inkinga: Kuhamba kancane kunokulindelekile

Ukulungiswa:

• hlola i-dataloader throughput (Umhlahlandlela Wokulungisa Ukusebenza kwe-PyTorch)

• khuphula usayizi weqembu

• nciphisa ukuqopha

• vumela ukunemba okuxubile (amadokhumenti e-PyTorch AMP)

• ukuhlukaniswa kwesikhathi sesinyathelo sephrofayela

Inkinga: I-Multi-GPU iyalenga

Ukulungiswa:

• qinisekisa izilungiselelo ezilungile ze-backend (amadokhumenti asakazwa yi-PyTorch)

• hlola izilungiselelo zemvelo ze-NCCL (qaphela) (ukubuka konke kwe-NCCL)

• hlola i-GPU eyodwa kuqala

• qinisekisa ukuthi inethiwekhi/ukuxhumana kuphilile

Inothi elincane lokubuyela emuva: ngezinye izikhathi ukulungiswa kuwukuqala kabusha. Kuzwakala kuyisiwula. Kuyasebenza. Amakhompyutha anjalo.

12) Izindleko kanye nokusebenza kahle - ukukhetha i-NVIDIA GPU efanele kanye nokusetha ngaphandle kokucabanga ngokweqile 💸🧠

Akuwona wonke amaphrojekthi adinga i-GPU enkulu kakhulu. Ngezinye izikhathi udinga eyanele .

Uma ulungisa kahle amamodeli aphakathi

• Beka phambili i-VRAM kanye nokuzinza

• Ukunemba okuxubile kusiza kakhulu (amadokhumenti e-PyTorch AMP, umhlahlandlela wokunemba okuxubile we-TensorFlow)

• Ungakwazi ukuphumelela nge-GPU eyodwa enamandla

Uma uqeqesha amamodeli amakhulu kusukela ekuqaleni

• Uzodinga ama-GPU amaningi noma i-VRAM enkulu kakhulu

• Uzokhathalela i-NVLink kanye nesivinini sokuxhumana (i-NVIDIA NVLink overview, i-NCCL overview)

• Cishe uzosebenzisa ama-memory optimizer (ZeRO, offload, njll.) (DeepSpeed ​​ZeRO docs, Microsoft Research: ZeRO/DeepSpeed)

Uma wenza ukuhlola

• Ufuna ukuphindaphinda okusheshayo

• Ungasebenzisi yonke imali yakho ku-GPU bese ulahla isitoreji ne-RAM

• Uhlelo olulinganiselayo lunqoba oluphambene (ezinsukwini eziningi)

Futhi eqinisweni, ungachitha amasonto uphishekela ukukhetha kwehadiwe "okuphelele". Yakha into esebenzayo, ulinganise, bese ulungisa. Isitha sangempela ukungabi nohlelo lokuphendula.

Amanothi okuvala - Indlela yokusebenzisa ama-GPU e-NVIDIA ekuQeqesheni i-AI ngaphandle kokulahlekelwa yingqondo 😌✅

Uma ungathathi lutho olunye kulo mhlahlandlela wokuthi Ungayisebenzisa Kanjani i-NVIDIA GPU Yokuqeqeshwa Kwe-AI, thatha lokhu:

• Qiniseka ukuthi i-nvidia-smi isebenza kuqala (ama-NVIDIA nvidia-smi docs)

• Khetha indlela yesofthiwe ehlanzekile (i-CUDA ehlanganiswe nohlaka ivame ukuba lula kakhulu) (PyTorch Get Started (CUDA selector))

• Qinisekisa ukugijima kokuqeqeshwa kwe-GPU okuncane ngaphambi kokukhulisa (torch.cuda.is_available)

• Phatha i-VRAM sengathi iyishelufu elilinganiselwe le-pantry

• Sebenzisa ukunemba okuxubile kusenesikhathi - akuzona nje "izinto ezithuthukisiwe" (amadokhumenti e-PyTorch AMP, umhlahlandlela wokunemba okuxubile we-TensorFlow)

• Uma kuhamba kancane, sola i-dataloader kanye ne-I/O ngaphambi kokusola i-GPU (PyTorch Performance Tuning Guide)

• I-Multi-GPU inamandla kodwa yengeza ubunzima - yenza kancane kancane (PyTorch DDP docs, NCCL overview)

• Gada ukusetshenziswa kanye namazinga okushisa ukuze izinkinga zivele kusenesikhathi (amadokhumenti e-NVIDIA nvidia-smi)

Ukuqeqeshwa kuma-NVIDIA GPU kungenye yamakhono azwakala esabisa, bese kuthi ngokuzumayo kube yinto evamile. Njengokufunda ukushayela. Ekuqaleni konke kuba nomsindo futhi kuyadida bese ubamba isondo kanzima kakhulu. Bese ngolunye usuku uzobe uhamba ngesikebhe, uphuza ikhofi, futhi ulungisa inkinga yobukhulu bebhetshi ngokunganaki sengathi akuyona into enkulu.

Isibonelo sangempela: Ukuqeqesha isihlukanisi sesithombe esincane ku-NVIDIA GPU eyodwa 🧪🖼️

Isimo

Cabanga ngethimba elincane lezohwebo nge-inthanethi lifuna ukuqeqesha umuntu ohlukanisa izithombe ngezigaba ezihlanu: izicathulo, izikhwama, amajakethi, amawashi, kanye nezesekeli.

Abaqeqeshi imodeli enkulu kusukela ekuqaleni. Balungisa imodeli yombono eqeqeshwe kusengaphambili ku-NVIDIA GPU eyodwa, ukuze ithimba likwazi ukuhlola ngokushesha ukuthi lo mbono uyakufanelekela yini ukukhuliswa.

Umgomo ulula: qinisekisa ukuthi ukusethwa kwe-GPU kuyasebenza, gwema isiphithiphithi se-CUDA, futhi wakhe iluphu yokuqeqesha ephindaphindwayo ngaphambi kokuchitha imali kuhadiwe enkulu noma ku-cloud runs.

Okudingwa ukusetha

Kulolu hlobo lokuhlolwa, uzodinga:

Umshini one-NVIDIA GPU eyodwa kanye ne-VRAM eyanele yosayizi we-batch

Umshayeli we-NVIDIA osebenzayo uqinisekisiwe nge-nvidia-smi

Indawo ehlanzekile yePython yePyTorch, TensorFlow, noma i-JAX

Isethi yedatha yesithombe encane enelebula, ehlukaniswe kahle ngamafolda esitimela, okuqinisekisa, kanye nokuhlola

Isikhathi sokuqala se-CPU sokuqhathanisa

Ishidi lokubhalisa elilula elinesikhathi sesinyathelo, inkumbulo ye-GPU, ukusetshenziswa kwe-GPU, izinga lokushisa, kanye nokunemba kokuqinisekisa

Ngaphambi kokuqeqesha kahle, ithimba kufanele lenze ukuhlolwa okuncane kwentuthu ye-CUDA: layisha ibhetshi elilodwa, lihambise imodeli kanye nebhetshi ku-cuda, lisebenzise iphasi elilodwa eliya phambili, bese liqinisekisa ukwanda kwememori ye-GPU ku-nvidia-smi.

Isibonelo semiyalelo

Imiyalelo yephrojekthi esebenzayo ingabukeka kanje:

Qeqesha isihlukanisi sesithombe somkhiqizo omncane usebenzisa imodeli yesitayela se-ResNet eqeqeshwe kusengaphambili. Okokuqala qinisekisa ukuthi i-nvidia-smi ingayibona i-GPU. Bese usebenzisa ukuhlolwa kwe-CUDA yeqembu elilodwa ngaphambi kokuqeqeshwa okugcwele. Sebenzisa ukunemba okuxubile uma kusekelwa. Qala ngosayizi weqembu 32, wandise kuphela uma imemori ye-GPU ihlala izinzile, bese ubhala isikhathi sesinyathelo, ukusetshenziswa kwememori ye-GPU, ukusetshenziswa kwe-GPU, izinga lokushisa, kanye nokunemba kokuqinisekisa ngemva kokugijima ngakunye. Uma kuvela i-CUDA engaphandle kwememori, nciphisa usayizi weqembu ngaphambi kokushintsha imodeli.

Indlela yokuyihlola

Uhlelo lokuhlola olunengqondo lungaba:

1. Sebenzisa i-nvidia-smi bese uqopha igama le-GPU, inguqulo yomshayeli, ukusetshenziswa kwememori engasebenzi, kanye nezinga lokushisa.

2. Sebenzisa ukuhlolwa kwe-CPU kweqembu elilodwa ukuqinisekisa isethi yedatha kanye nomsebenzi wekhodi yemodeli.

3. Sebenzisa isivivinyo esifanayo se-batch eyodwa ku-cuda.

4. Qeqesha izinyathelo ezingu-200 ezinosayizi weqembu ongu-32.

5. Phinda ngokunemba okuxubile okuvuliwe.

6. Zama usayizi we-batch 64 kuphela uma ukugijima kokuqala kushiya isikhala esanele se-VRAM.

7. Qhathanisa ukunemba kokuqinisekisa, isikhathi esimaphakathi sesinyathelo, i-VRAM ephezulu, kanye nokushisa kwe-GPU.

Umphumela omuhle awugcini nje ngokuthi “uqeqeshwe”. Umphumela omuhle ukuthi “uqeqeshwe ku-GPU, isivinini sithuthukisiwe, inkumbulo yahlala izinzile, futhi ukugijima kungaphindeka kusasa ngaphandle kokufaka konke kabusha”.

Umphumela

Umphumela obonisayo, osekelwe ekubaleni isikhathi sokuhlolwa okuthathu okuncane kwezinyathelo ezingu-200 ngaphambi nangemva kokushintsha ukuqeqeshwa kusuka ku-CPU kuya ku-NVIDIA GPU eyodwa:

Isisekelo se-CPU kuphela: imizuzwana engu-3.4 ngesinyathelo ngasinye sokuqeqesha

I-GPU ene-FP32: imizuzwana engu-0.42 ngesinyathelo ngasinye sokuqeqesha

I-GPU enokunemba okuxubile: imizuzwana engu-0.28 ngesinyathelo ngasinye sokuqeqesha

Imemori ye-GPU ephezulu enosayizi we-batch ongu-32: 5.8 GB

Imemori ye-GPU ephezulu enosayizi we-batch 64: 10.9 GB

Usayizi weqembu 96: wehlulekile nge-CUDA ephelelwe yinkumbulo

Ukusetshenziswa kwe-GPU ngesikhathi sokusebenza okuzinzile: 76% kuya ku-91%

Izinga lokushisa ngesikhathi sokusebenza okuzinzile: 67°C kuya ku-73°C

Ukunemba kokuqinisekisa ngemva kokuhlolwa okufushane: 82% nge-FP32, 82.4% ngokunemba okuxubile

Kulesi sibonelo sokulinganisa, ukunemba okuxubile kunciphisa isikhathi sesinyathelo cishe ngo-33% uma kuqhathaniswa nokusebenza kwe-FP32 GPU, kuyilapho kugcinwa ukunemba kokuqinisekisa cishe kufana. Ithimba lingaqinisekisa lezi zinombolo ngokubeka isikhathi sesinyathelo ngasinye sokuqeqesha, ukuhlola i-nvidia-smi ngesikhathi sokugijima, nokulondoloza ukunemba kokuqinisekisa ngemva kokuhlolwa ngakunye.

Yini engase ihambe kabi

Iphutha elivame kakhulu ukukala kusenesikhathi kakhulu. Uma ukuhlolwa kwe-CUDA kweqembu elilodwa kwehluleka, ukuqeqeshwa okuphelele ngeke kulungise ngomlingo.

Ezinye izingibe ezilula:

Ukufaka izinguqulo eziningi ze-CUDA futhi ungazi ukuthi yiluphi uhlaka olusebenzisayo

Ukuhambisa imodeli ku-cuda kodwa kushiya amaqoqo ku-CPU

Ukukhetha usayizi weqembu ofanelana kanye kodwa uphahlazeke ngemva kwezinyathelo eziningana

Ukungazinaki ezinye izinqubo ezisebenzisa i-VRAM kakade

Ukusola i-GPU uma i-dataloader ihamba kancane kakhulu

Ukuqhathanisa ukusebenza kwe-CPU ne-GPU ngaphandle kokusebenzisa isethi yedatha efanayo, usayizi we-batch, kanye nemodeli

Umuntu kufanele abukeze nezibikezelo ezimbalwa zokuqala. Ukuqeqeshwa okusheshayo akunanzuzo kangako uma amalebula enomsindo, amakilasi engalingani, noma imodeli ifunda izinqamuleli njengombala wangemuva esikhundleni sohlobo lomkhiqizo.

Ukudla okuwusizo

Umsebenzi wokuqeqeshwa kwe-NVIDIA GPU othembekile uqala kancane: qinisekisa ukuthi umshayeli uyasebenza, qinisekisa ukuthi i-CUDA iyasebenza, qinisekisa ukuthi i-batch eyodwa isebenza kahle, bese ukala usayizi we-batch kanye nobude bokuqeqeshwa kancane kancane. Ukusetha okusheshayo akusikho okune-GPU ehlaba umxhwele kakhulu ephepheni - yikona okukunikeza ukusebenza okuzinzile nokulinganisekayo ngaphandle kokuchitha amahora amaningi kunguqulo egwenywayo, i-VRAM, kanye nezinkinga zokulayisha idatha.

Imibuzo Evame Ukubuzwa

Kusho ukuthini ukuqeqesha imodeli ye-AI ku-NVIDIA GPU

Ukuqeqeshwa ku-NVIDIA GPU kusho ukuthi amapharamitha akho emodeli kanye namaqoqo okuqeqesha ahlala ku-GPU VRAM, futhi izibalo ezisindayo (i-forward pass, i-backprop, izinyathelo ze-optimizer) zisebenza ngama-CUDA kernels. Empeleni, lokhu kuvame ukwenzeka ekuqinisekiseni ukuthi imodeli nama-tensor ahlala ku- cuda, bese kugcinwa iso kumemori, ukusetshenziswa, kanye namazinga okushisa ukuze i-throughput ihlale ihambisana.

Indlela yokuqinisekisa ukuthi i-NVIDIA GPU iyasebenza ngaphambi kokufaka noma yini enye

Qala nge -nvidia-smi. Kufanele ibonise igama le-GPU, inguqulo yomshayeli, ukusetshenziswa kwememori yamanje, kanye nanoma yiziphi izinqubo ezisebenzayo. Uma i-nvidia-smi yehluleka, yima i-PyTorch/TensorFlow/JAX - lungisa ukubonakala komshayeli kuqala. Kuyisisekelo "ukuhlola ukuthi i-oven ixhunyiwe" kokuqeqeshwa kwe-GPU.

Ukukhetha phakathi kwe-CUDA yohlelo kanye ne-CUDA ehlanganiswe ne-PyTorch

Indlela evamile ukusebenzisa i-CUDA enezinqwaba ze-framework (njengamasondo amaningi e-PyTorch) ngoba inciphisa izingxenye ezihambayo - udinga kakhulu umshayeli we-NVIDIA ohambisanayo. Ukufaka i-CUDA toolkit yesistimu ephelele kunikeza ukulawula okwengeziwe (ukwakha ngokwezifiso, ukuhlanganisa ama-ops), kodwa futhi kwethula amathuba amaningi okungafani kwenguqulo kanye namaphutha esikhathi sokusebenza adidayo.

Kungani ukuqeqeshwa kusengaba kancane ngisho noma usebenzisa i-NVIDIA GPU

Ngokuvamile, i-GPU ayitholi amandla ngenxa yepayipi lokufaka. Ama-dataloader ahlala isikhathi eside, i-CPU esindayo icutshungulwa ngaphambi kwesikhathi sokuqeqesha, osayizi abancane be-batch, noma isitoreji esihamba kancane konke kungenza i-GPU enamandla iziphathe njengesifudumezi sesikhala esingasebenzi. Ukwandisa abasebenzi be-dataloader, ukuvumela inkumbulo ephiniwe, ukwengeza ukulanda kusengaphambili, kanye nokunciphisa ukuqopha kuyizinyathelo zokuqala ezivamile ngaphambi kokusola imodeli.

Indlela yokuvimbela amaphutha "e-CUDA aphelelwe yinkumbulo" ngesikhathi sokuqeqeshwa kwe-NVIDIA GPU

Ukulungiswa okuningi kungamasu e-VRAM: ukunciphisa usayizi we-batch, vumela ukunemba okuxubile (FP16/BF16), sebenzisa ukuqongelela kwe-gradient, ukufinyeza ubude be-sequence/usayizi we-crop, noma sebenzisa i-activation checkpointing. Hlola futhi ezinye izinqubo ze-GPU ezisebenzisa imemori. Okunye ukuzama namaphutha kuvamile - isabelomali se-VRAM siba umkhuba oyinhloko ekuqeqeshweni kwe-GPU okusebenzayo.

Kungani i-VRAM isabonakala igcwele ngemva kokuphela kweskripthi sokuqeqesha

Amafreyimu avame ukugcina imemori ye-GPU ukuze athole isivinini, ngakho-ke imemori egciniwe ingahlala iphezulu ngisho noma imemori ebekelwe yona yehla. Ingafana nokuvuza, kodwa ngokuvamile i-caching allocator isebenza njengoba iklanyelwe. Umkhuba osebenzayo ukulandelela iphethini ngokuhamba kwesikhathi bese uqhathanisa "ebekelwe vs ebekelwe" kunokugxila esithombeni esisodwa esesabekayo.

Indlela yokuqinisekisa ukuthi imodeli ayiqeqeshwa buthule ku-CPU

Hlola ingqondo kusenesikhathi: qinisekisa ukuthi i-torch.cuda.is_available() ibuyisela i-True, qinisekisa ukuthi i-next(model.parameters()).device ibonisa i-cuda, bese usebenzisa i-forward pass eyodwa ngaphandle kwamaphutha. Uma ukusebenza kuzwakala kuhamba kancane ngendlela exakile, qinisekisa nokuthi ama-batch akho ayiswa ku-GPU. Kuvamile ukuhambisa imodeli bese ushiya idatha ngephutha.

Indlela elula yokuqeqeshwa kwe-multi-GPU

I-Data Parallel (ukuqeqeshwa kwesitayela se-DDP) ivame ukuba yisinyathelo sokuqala esihle kakhulu: ukuhlukanisa ama-batch kuwo wonke ama-GPU nokuvumelanisa ama-gradients. Amathuluzi afana ne-Accelerate angenza i-multi-GPU ibe buhlungu kancane ngaphandle kokubhala kabusha okugcwele. Lindela iziguquguquko ezengeziwe - ukuxhumana kwe-NCCL, umehluko wokuxhumana (NVLink vs PCIe), kanye nokwanda kwezithiyo zedatha - ngakho-ke ukukala kancane kancane ngemva kokusebenza kwe-single-GPU eqinile kuvame ukuhamba kangcono.

Okufanele ukuqaphele ngesikhathi sokuqeqeshwa kwe-NVIDIA GPU ukuze kutholakale izinkinga kusenesikhathi

Bukela ukusetshenziswa kwe-GPU, ukusetshenziswa kwememori (okuzinzile vs ukukhuphuka), ukudonsa kwamandla, kanye namazinga okushisa - ukugoqa kunganciphisa isivinini buthule. Qaphela nokusetshenziswa kwe-CPU, njengoba inkinga yepayipi ledatha ivame ukuvela lapho kuqala. Uma ukusetshenziswa kunciphile noma kuphansi, sola i-I/O noma ama-dataloaders; uma kuphezulu kodwa isikhathi sesinyathelo sisahamba kancane, ama-kernel ephrofayili, imodi yokucacisa, kanye nokuhlukaniswa kwesikhathi sesinyathelo.

Izinkomba

1. NVIDIA - NVIDIA nvidia-smi - docs.nvidia.com

2. I-NVIDIA - I-NVIDIA System Management Interface (NSVMI) - developer.nvidia.com

3. NVIDIA - NVIDIA NVLink - nvidia.com

4. I-PyTorch - I-PyTorch Qala (isikhethi se-CUDA) - pytorch.org

5. I-PyTorch - PyTorch Amadokhumenti e-CUDA - docs.pytorch.org

6. -TensorFlow - TensorFlow (ipayipi) - tensorflow.org

7. I-JAX - Isiqalo Esisheshayo se-JAX - docs.jax.dev

8. Ubuso Obugonayo - Amadokhumenti Omqeqeshi - huggingface.co

9. I-Lightning AI - Amadokhumenti e-Lightning - lightning.ai

10. e-DeepSpeed ​​- ZeRO - deepspeed.readthedocs.io

11. Ucwaningo lweMicrosoft - Ucwaningo lweMicrosoft: ZeRO/DeepSpeed ​​- microsoft.com

12. Izinkundla zePyTorch - Inkundla yePyTorch: hlola imodeli ku-CUDA - discuss.pytorch.org

Thola i-AI Yakamuva Esitolo Esisemthethweni Somsizi we-AI

Mayelana NATHI