Indlela yokusebenzisa ama-GPU e-NVIDIA ekuQeqesheni i-AI

Indlela yokusebenzisa ama-GPU e-NVIDIA ekuQeqesheni i-AI

Impendulo emfushane: Sebenzisa ama-GPU e-NVIDIA ekuqeqeshweni kwe-AI ngokuqinisekisa kuqala ukuthi umshayeli kanye ne-GPU ziyabonakala nge -nvidia-smi , bese ufaka i-framework/CUDA stack ehambisanayo bese usebenzisa isivivinyo esincane esithi “model + batch on cuda”. Uma ushaya i-out of memory, yehlisa usayizi we-batch bese usebenzisa ukunemba okuxubile, ngenkathi uqapha ukusetshenziswa, imemori, kanye namazinga okushisa.

Izinto ezibalulekile okufanele uzicabangele:

Ukuhlolwa kwesisekelo : Qala nge -nvidia-smi ; lungisa ukubonakala komshayeli ngaphambi kokufaka amafreyimu.

Ukuhambisana kwe-stack : Gcina umshayeli, isikhathi sokusebenza se-CUDA, kanye nezinguqulo zohlaka zihambisana ukuze kuvinjelwe ukuphahlazeka nokufakwa okubuthakathaka.

Impumelelo encane : Qinisekisa ukuthi i-CUDA ithola i-pass eyodwa ngaphambi kokuthi uthuthukise izivivinyo.

Isiyalo se-VRAM : Thembela ekuqondeni okuxubile, ukuqongelela kwe-gradient, kanye nokukhomba ukuze kulingane namamodeli amakhulu.

Umkhuba wokuqapha : Landelela ukusetshenziswa, amaphethini enkumbulo, amandla, kanye nezinga lokushisa ukuze ubone izithiyo kusenesikhathi.

Izihloko ongase uthande ukuzifunda ngemva kwalesi:

🔗 Indlela yokwakha i-ejenti ye-AI
Yakha ukuhamba komsebenzi kwe-ejenti yakho, amathuluzi, inkumbulo, kanye nezivikelo zokuphepha.

🔗 Indlela yokusebenzisa amamodeli e-AI
Setha izindawo, amamodeli ephakheji, bese uthumela ekukhiqizweni ngokuthembekile.

🔗 Ungakala kanjani ukusebenza kwe-AI
Khetha izilinganiso, sebenzisa ukuhlolwa, bese ulandelela ukusebenza ngokuhamba kwesikhathi.

🔗 Indlela yokwenza imisebenzi ngokuzenzakalelayo nge-AI
Yenza umsebenzi ophindaphindwayo ube ngokuzenzakalelayo ngezikhuthazo, imisebenzi yokusebenza, kanye nokuhlanganiswa.

1) Isithombe esikhulu - lokho okwenzayo uma "uqeqesha nge-GPU" 🧠⚡

Uma uqeqesha amamodeli e-AI, ikakhulukazi wenza intaba yezibalo ze-matrix. Ama-GPU akhelwe lolo hlobo lomsebenzi ohambisanayo, ngakho-ke izinhlaka ezifana ne-PyTorch, i-TensorFlow, ne-JAX zingalayisha ukuphakamisa okunzima ku-GPU. ( PyTorch CUDA docs , TensorFlow install (pip) , JAX Quickstart )

Empeleni, "ukusebenzisa ama-GPU e-NVIDIA ekuqeqesheni" kuvame ukusho ukuthi:

  • Amapharamitha akho emodeli abukhoma (ikakhulukazi) ku-GPU VRAM

  • Amaqoqo akho asuswa ku-RAM aye ku-VRAM isinyathelo ngasinye

  • I-forward pass yakho kanye ne-backprop yakho isebenza kuma-CUDA kernels ( Umhlahlandlela Wokuhlela we-CUDA )

  • Izibuyekezo zakho ze-optimizer zenzeka ku-GPU (ngokufanele)

  • Uqapha amazinga okushisa, inkumbulo, ukusetshenziswa ukuze ungapheki lutho 🔥 ( NVIDIA nvidia-smi docs )

Uma lokho kuzwakala sengathi kuningi, ungakhathazeki. Ngokuvamile kuba uhlu lokuhlola kanye nemikhuba embalwa oyakhayo ngokuhamba kwesikhathi.

2) Yini eyenza inguqulo enhle yokusetha ukuqeqeshwa kwe-NVIDIA GPU AI 🤌

Lesi isigaba esithi “ungakhi indlu ngejeli”. Ukusetha okuhle kokuthi Ungayisebenzisa kanjani i-NVIDIA GPU yoQeqesho lwe-AI kungenye yezindlela eziphansi kakhulu. Izindlela eziphansi kakhulu zizinzile. Ukuqina kuyashesha. Ukushesha kuyashesha…kahle, kuyashesha 😄

Isethaphu sokuqeqeshwa okuqinile ngokuvamile sinalokhu:

  • I-VRAM eyanele yosayizi we-batch yakho + imodeli + izimo ze-optimizer

    • I-VRAM ifana nendawo yesutikheyisi. Ungapakisha ngobuhlakani, kodwa awukwazi ukupakisha okungenamkhawulo.

  • I-software stack efanisiwe (umshayeli + isikhathi sokusebenza se-CUDA + ukuhambisana kohlaka) ( PyTorch Get Started (CUDA selector) , TensorFlow install (pip) )

  • Isitoreji esisheshayo (i-NVMe isiza kakhulu kumasethi wedatha amakhulu)

  • I-CPU ehloniphekile + i-RAM ukuze ukulayisha idatha kungalambisi i-GPU ( PyTorch Performance Tuning Guide )

  • Ukupholisa kanye negumbi lokuvula amandla (elilinganiselwe kancane kuze kube yilapho lingasekho 😬)

  • Indawo ephindaphindwayo (i-venv/conda noma izitsha) ukuze ukuthuthukiswa kungabi yisiphithiphithi ( ukubuka konke kwe-NVIDIA Container Toolkit )

Futhi into eyodwa abantu abayiqayo:

3) Ithebula Lokuqhathanisa - izindlela ezidumile zokuqeqesha ngama-GPU e-NVIDIA (ngezimpawu ezingavamile) 📊

Ngezansi kunephepha lokukhohlisa elisheshayo elithi “yiliphi elifanelana?”. Amanani ayinto engajwayelekile (ngoba iqiniso liyahlukahluka), futhi yebo elinye lala maseli liyinkimbinkimbi kancane, ngamabomu.

Ithuluzi / Indlela Kuhle kakhulu Intengo Kungani kusebenza (ikakhulukazi)
I-PyTorch (i-vanilla) I-PyTorch abantu abaningi, amaphrojekthi amaningi Mahhala I-ecosystem eguquguqukayo, enkulu, ukulungisa amaphutha okulula - futhi wonke umuntu unemibono
Amadokhumenti e -PyTorch Lightning amaqembu, ukuqeqeshwa okuhlelekile Mahhala Kunciphisa i-boilerplate, izihibe ezihlanzekile; ngezinye izikhathi kuzwakala sengathi “kungumlingo”, kuze kube yilapho kungenzeki
Ama-Transformer obuso obugonwayo + amadokhumenti omqeqeshi womqeqeshi Ukulungiswa kwe-NLP + LLM Mahhala Ukuqeqeshwa okufakwe amabhethri, okuzenzakalelayo okuhle, ukuwina okusheshayo 👍
Sheshisa Sheshisa amadokhumenti i-multi-GPU ngaphandle kobuhlungu Mahhala Kwenza i-DDP ingacasuli kakhulu, ilungele ukwandisa ngaphandle kokubhala kabusha konke
e-DeepSpeed ​​​​ZeRO amamodeli amakhulu, amaqhinga okukhumbula Mahhala I-ZeRO, ukulayisha, ukukala - kungaba yinto exakile kodwa eyanelisayo uma icindezela
-TensorFlow + Keras TF amapayipi ahambisana nokukhiqiza Mahhala Amathuluzi aqinile, indaba enhle yokusetshenziswa; abanye abantu bayayithanda, abanye abayithandi buthule
Amadokhumenti e-JAX + Flax JAX Quickstart / Flax ucwaningo + ama-speed nerds Mahhala Ukuhlanganiswa kwe-XLA kungashesha kakhulu, kodwa ukulungisa amaphutha kungazwakala sengathi... akucaci
kwe-NVIDIA NeMo NeMo imisebenzi yokukhuluma + ye-LLM Mahhala Isitaki esenziwe ngcono yi-NVIDIA, izindlela zokupheka ezinhle - kuzwakala njengokupheka ngehhavini elihle 🍳
Ukubuka konke kwe-Docker + NVIDIA Container Toolkit Toolkit izindawo ezingaphinde zikhiqizwe Mahhala “Isebenza emshinini wami” iba “isebenza emishinini yethu” (ikakhulukazi, futhi)

4) Isinyathelo sokuqala - qinisekisa ukuthi i-GPU yakho ibonwe kahle 🕵️♂️

Ngaphambi kokufaka izinto eziyishumi nambili, qinisekisa izisekelo.

Izinto ofuna ukuba yiqiniso ngazo:

  • Umshini ubona i-GPU

  • Umshayeli we-NVIDIA ufakwe kahle

  • I-GPU ayibambeki ekwenzeni okunye

  • Ungabuza ngokuthembekile

Isheke lakudala yileli:

Okufunayo:

  • Igama le-GPU (isb., i-RTX, uchungechunge lwe-A, njll.)

  • Inguqulo yomshayeli

  • Ukusetshenziswa kwememori

  • Izinqubo zokuqalisa ( NVIDIA nvidia-smi docs )

Uma i-nvidia-smi yehluleka, yima lapho. Ungafaki amafreyimu okwamanje. Kufana nokuzama ukubhaka isinkwa lapho ihhavini lakho lingaxhunywanga. ( NVIDIA System Management Interface (NSVMI) )

Inothi elincane lomuntu: ngezinye izikhathi i-nvidia-smi iyasebenza kodwa ukuqeqeshwa kwakho kusahluleka ngoba isikhathi sokusebenza se-CUDA esisetshenziswa uhlaka lwakho asihambisani nokulindelwe ngumshayeli. Lokho akusikho ukuthi uyisiwula. Yilokho nje... kunjalo 😭 ( PyTorch Get Started (CUDA selector) , TensorFlow install (pip) )

5) Yakha i-software stack - abashayeli, i-CUDA, i-cuDNN, kanye "nomdanso wokuhambisana" 💃

Yilapho abantu belahlekelwa khona amahora. Icebo liwukuthi: khetha indlela bese unamathela kuyo .

Inketho A: I-CUDA ehlanganiswe ngohlaka (ngokuvamile elula)

Ama-PyTorch amaningi akha athunyelwa ngesikhathi sawo sokusebenza se-CUDA, okusho ukuthi awudingi ithuluzi eligcwele le-CUDA elifakwe ohlelweni lonke. Udinga nje umshayeli we-NVIDIA ohambisanayo. ( I-PyTorch Get Started (CUDA selector) , Izinguqulo Zangaphambilini ze-PyTorch (amasondo e-CUDA) )

Izinzuzo:

  • Izingxenye ezimbalwa ezihambayo

  • Ukufakwa okulula

  • Kungaphindeka kakhudlwana ngokwemvelo ngayinye

Ububi:

  • Uma uxuba izindawo ngokunganaki, ungadideka

Inketho B: Ithuluzi le-CUDA lesistimu (ukulawula okwengeziwe)

Ufaka i-CUDA toolkit ohlelweni bese uvumelanisa konke nalo. ( Amadokhumenti e-CUDA Toolkit )

Izinzuzo:

  • Ukulawula okwengeziwe kokwakhiwa ngokwezifiso, amathuluzi athile akhethekile

  • Ilungele ukuhlanganisa imisebenzi ethile

Ububi:

  • Izindlela ezengeziwe zokungafani izinguqulo bese ukhala buthule

i-cuDNN kanye ne-NCCL, ngokwemibono yabantu

Uma wenza ukuqeqeshwa kwe-multi-GPU, i-NCCL ingumngane wakho omkhulu - futhi, ngezinye izikhathi, umuntu ohlala naye othanda isimo sengqondo esibi. ( Isifinyezo se-NCCL )

6) Ukuqeqeshwa kwakho kokuqala kwe-GPU (isibonelo sengqondo se-PyTorch) ✅🔥

Ukuze ulandele Indlela yokusebenzisa ama-GPU e-NVIDIA ekuQeqesheni kwe-AI , awudingi iphrojekthi enkulu kuqala. Udinga impumelelo encane.

Imibono eyinhloko:

  • Thola idivayisi

  • Hambisa imodeli ku-GPU

  • Thutha ama-tensor ku-GPU

  • Qinisekisa ukuthi ama-forward pass agijima lapho ( PyTorch CUDA docs )

Izinto engihlala ngizihlola kusenesikhathi:

Okuvamile ukuthi “kungani kuhamba kancane?”

  • I-dataloader yakho ihamba kancane kakhulu (i-GPU ayilindeki) ( PyTorch Performance Tuning Guide )

  • Ukhohliwe ukuhambisa idatha ku-GPU (oops)

  • Usayizi weqembu mncane (i-GPU ayisetshenziswa kahle)

  • Wenza ukucubungula kwe-CPU okunzima esinyathelweni sokuqeqeshwa

Futhi, yebo, i-GPU yakho izovame ukubonakala “ingamatasa kangako” uma inkinga iwukuba nedatha. Kufana nokuqasha umshayeli wemoto yomjaho bese umenza alinde uphethiloli njalo uma efika.

7) Umdlalo we-VRAM - usayizi we-batch, ukunemba okuxubile, futhi awuqhumi 💥🧳

Izinkinga eziningi zokuqeqeshwa okusebenzayo zigcina engqondweni. Uma ufunda ikhono elilodwa, funda ukuphathwa kwe-VRAM.

Izindlela ezisheshayo zokunciphisa ukusetshenziswa kwememori

Umzuzu othi “kungani i-VRAM isagcwele ngemva kokuyeka?”

Amafreyimu avame ukugcina imemori yokusebenza. Lokhu kuyinto evamile. Kubukeka kuyesabeka kodwa akuhlali kuvuza. Ufunda ukufunda amaphethini. ( PyTorch CUDA semantics: caching allocator )

Umkhuba osebenzayo:

8) Yenza i-GPU isebenze ngempela - ukulungiswa kokusebenza okufanele isikhathi sakho 🏎️

Ukuthola "ukuqeqeshwa kwe-GPU kusebenze" kuyisinyathelo sokuqala. Ukukuthola ngokushesha kuyisinyathelo sesibili.

Ukulungiswa okunomthelela omkhulu

Inkinga enganakwa kakhulu

Ipayipi lakho lokugcina kanye nokucubungula kusengaphambili. Uma isethi yedatha yakho inkulu futhi igcinwe kudiski ehamba kancane, i-GPU yakho iba yi-space heater ebizayo. I-space heater ethuthukisiwe kakhulu, ekhazimulayo kakhulu.

Futhi, ukuvuma okuncane: "Ngilungise" imodeli ihora lonke kodwa ngaqaphela ukuthi ukubhala phansi kwakuyimbangela yenkinga. Ukuphrinta kakhulu kunganciphisa ukuqeqeshwa. Yebo, kungakwenza.

9) Ukuqeqeshwa kwe-Multi-GPU - i-DDP, i-NCCL, kanye nokukhulisa ngaphandle kwesiphithiphithi 🧩🤝

Uma usufuna amamodeli anesivinini esikhulu noma amakhulu, uthola ama-GPU amaningi. Yilapho izinto ziba zimbi khona.

Izindlela ezivamile

  • Idatha Ehambisanayo (i-DDP)

    • Hlukanisa amaqoqo kuwo wonke ama-GPU, vumelanisa ama-gradients

    • Ngokuvamile inketho "enhle" ezenzakalelayo ( PyTorch DDP docs )

  • Imodeli Ehambisanayo / Ehambisanayo Ye-Tensor

    • Hlukanisa imodeli phakathi kwama-GPU (kumamodeli amakhulu kakhulu)

  • Iphayiphi Elihambisanayo

    • Hlukanisa izendlalelo zemodeli zibe yizigaba (njengomugqa wokuhlanganisa, kodwa kuma-tensor)

Uma uqala, ukuqeqeshwa kwesitayela se-DDP kuyindlela enhle. ( Isifundo se-PyTorch DDP )

Amathiphu awusizo e-multi-GPU

  • Qiniseka ukuthi ama-GPU anekhono elifanayo (ukuxuba ithini eliyimbangela yenkinga)

  • Ukuxhumeka kwewashi: I-NVLink vs i-PCIe ibalulekile emisebenzini enzima yokuvumelanisa ( Isifinyezo se-NVIDIA NVLink , amadokhumenti e-NVIDIA NVLink )

  • Gcina osayizi be-batch ngayinye ye-GPU belinganisiwe

  • Ungayinaki i-CPU kanye nesitoreji - ama-GPU amaningi angakhulisa izithiyo zedatha

Futhi yebo, amaphutha e-NCCL angazwakala njengempicabadala egoqwe ngemfihlakalo egoqwe ngokuthi “kungani manje”. Awuqalekisiwe. Mhlawumbe. ( Isifinyezo se-NCCL )

10) Ukuqapha kanye nokubhala iphrofayili - izinto ezingathandeki ezikongela amahora 📈🧯

Awudingi amadeshibhodi amahle ukuze uqale. Udinga ukuqaphela uma kukhona okungasebenzi.

Izimpawu ezibalulekile okufanele uzibuke

  • Ukusetshenziswa kwe-GPU : ingabe ihlala iphakeme noma inameva?

  • Ukusetshenziswa kwememori : okuzinzile, okukhuphukayo, noma okungajwayelekile?

  • Ukudonsa amandla : ukwehla okungavamile kungasho ukusetshenziswa okungaphansi

  • Izinga lokushisa : amazinga okushisa aphezulu ahlala isikhathi eside angathuthukisa ukusebenza kahle

  • Ukusetshenziswa kwe-CPU : izinkinga zombhobho wedatha zivela lapha ( Umhlahlandlela Wokulungisa Ukusebenza kwe-PyTorch )

Indlela yokucabanga ngephrofayili (inguqulo elula)

  • Uma i-GPU ingasetshenziswa kahle - idatha noma i-CPU ibopha

  • Uma i-GPU iphezulu kodwa ihamba kancane - ukungasebenzi kahle kwe-kernel, ukunemba, noma ukwakheka kwemodeli

  • Uma isivinini sokuqeqesha sehla ngokungahleliwe - ukucindezelwa kokushisa, izinqubo zangemuva, ukuphazamiseka kwe-I/O

Ngiyazi, ukuqapha kuzwakala kungathandeki. Kodwa kufana nokufaka i-floss. Kuyacasula, bese impilo yakho ithuthuka ngokuzumayo.

11) Ukuxazulula izinkinga - okusolwayo okuvamile (kanye nalezo ezingavamile) 🧰😵💫

Lesi sigaba ngokuyisisekelo: "izinkinga ezinhlanu ezifanayo, kuze kube phakade."

Inkinga: I-CUDA ayisekho enkumbulo

Ukulungiswa:

Inkinga: Ukuqeqeshwa kusebenza ngengozi ku-CPU

Ukulungiswa:

  • qinisekisa ukuthi imodeli ithuthelwe e -cuda

  • qinisekisa ukuthi ama-tensor athuthelwe ku -cuda

  • hlola ukulungiselelwa kwedivayisi yohlaka ( amadokhumenti e-PyTorch CUDA )

Inkinga: Ukuphahlazeka okungavamile noma ukufinyelela imemori okungekho emthethweni

Ukulungiswa:

Inkinga: Kuhamba kancane kunokulindelekile

Ukulungiswa:

Inkinga: I-Multi-GPU iyalenga

Ukulungiswa:

Inothi elincane lokubuyela emuva: ngezinye izikhathi ukulungiswa kuwukuqala kabusha. Kuzwakala kuyisiwula. Kuyasebenza. Amakhompyutha anjalo.

12) Izindleko kanye nokusebenza kahle - ukukhetha i-NVIDIA GPU efanele kanye nokusetha ngaphandle kokucabanga ngokweqile 💸🧠

Akuwona wonke amaphrojekthi adinga i-GPU enkulu kakhulu. Ngezinye izikhathi udinga eyanele .

Uma ulungisa kahle amamodeli aphakathi

Uma uqeqesha amamodeli amakhulu kusukela ekuqaleni

Uma wenza ukuhlola

  • Ufuna ukuphindaphinda okusheshayo

  • Ungasebenzisi yonke imali yakho ku-GPU bese ulahla isitoreji ne-RAM

  • Uhlelo olulinganiselayo lunqoba oluphambene (ezinsukwini eziningi)

Futhi eqinisweni, ungachitha amasonto uphishekela ukukhetha kwehadiwe "okuphelele". Yakha into esebenzayo, ulinganise, bese ulungisa. Isitha sangempela ukungabi nohlelo lokuphendula.

Amanothi okuvala - Indlela yokusebenzisa ama-GPU e-NVIDIA ekuQeqesheni i-AI ngaphandle kokulahlekelwa yingqondo 😌✅

Uma ungathathi lutho olunye kulo mhlahlandlela wokuthi Ungayisebenzisa Kanjani i-NVIDIA GPU Yokuqeqeshwa Kwe-AI , thatha lokhu:

Ukuqeqeshwa kuma-GPU e-NVIDIA kungenye yalawo makhono azwakala esabisa, bese kuthi ngokuzumayo kube yinto evamile. Njengokufunda ukushayela. Ekuqaleni konke kuba nomsindo futhi kuyadida bese ubamba isondo kanzima kakhulu. Bese ngolunye usuku uzobe uhamba ngesikebhe, uphuza ikhofi, futhi uxazulula inkinga yosayizi webhetshi ngokunganaki sengathi akuyona into enkulu ☕😄

Imibuzo Evame Ukubuzwa

Kusho ukuthini ukuqeqesha imodeli ye-AI ku-NVIDIA GPU

Ukuqeqeshwa ku-NVIDIA GPU kusho ukuthi amapharamitha akho emodeli kanye namaqoqo okuqeqesha ahlala ku-GPU VRAM, futhi izibalo ezisindayo (i-forward pass, i-backprop, izinyathelo ze-optimizer) zisebenza ngama-CUDA kernels. Empeleni, lokhu kuvame ukwenzeka ekuqinisekiseni ukuthi imodeli nama-tensor ahlala ku- cuda , bese kugcinwa iso kumemori, ukusetshenziswa, kanye namazinga okushisa ukuze i-throughput ihlale ihambisana.

Indlela yokuqinisekisa ukuthi i-NVIDIA GPU iyasebenza ngaphambi kokufaka noma yini enye

Qala nge -nvidia-smi . Kufanele ibonise igama le-GPU, inguqulo yomshayeli, ukusetshenziswa kwememori yamanje, kanye nanoma yiziphi izinqubo ezisebenzayo. Uma i-nvidia-smi yehluleka, yima i-PyTorch/TensorFlow/JAX - lungisa ukubonakala komshayeli kuqala. Kuyisisekelo "ukuhlola ukuthi i-oven ixhunyiwe" kokuqeqeshwa kwe-GPU.

Ukukhetha phakathi kwe-CUDA yohlelo kanye ne-CUDA ehlanganiswe ne-PyTorch

Indlela evamile ukusebenzisa i-CUDA enezinqwaba ze-framework (njengamasondo amaningi e-PyTorch) ngoba inciphisa izingxenye ezihambayo - udinga kakhulu umshayeli we-NVIDIA ohambisanayo. Ukufaka i-CUDA toolkit yesistimu ephelele kunikeza ukulawula okwengeziwe (ukwakha ngokwezifiso, ukuhlanganisa ama-ops), kodwa futhi kwethula amathuba amaningi okungafani kwenguqulo kanye namaphutha esikhathi sokusebenza adidayo.

Kungani ukuqeqeshwa kusengaba kancane ngisho noma usebenzisa i-NVIDIA GPU

Ngokuvamile, i-GPU ayitholi amandla ngenxa yepayipi lokufaka. Ama-dataloader ahlala isikhathi eside, i-CPU esindayo icutshungulwa ngaphambi kwesikhathi sokuqeqesha, osayizi abancane be-batch, noma isitoreji esihamba kancane konke kungenza i-GPU enamandla iziphathe njengesifudumezi sesikhala esingasebenzi. Ukwandisa abasebenzi be-dataloader, ukuvumela inkumbulo ephiniwe, ukwengeza ukulanda kusengaphambili, kanye nokunciphisa ukuqopha kuyizinyathelo zokuqala ezivamile ngaphambi kokusola imodeli.

Indlela yokuvimbela amaphutha "e-CUDA aphelelwe yinkumbulo" ngesikhathi sokuqeqeshwa kwe-NVIDIA GPU

Ukulungiswa okuningi kungamasu e-VRAM: ukunciphisa usayizi we-batch, vumela ukunemba okuxubile (FP16/BF16), sebenzisa ukuqongelela kwe-gradient, ukufinyeza ubude be-sequence/usayizi we-crop, noma sebenzisa i-activation checkpointing. Hlola futhi ezinye izinqubo ze-GPU ezisebenzisa imemori. Okunye ukuzama namaphutha kuvamile - isabelomali se-VRAM siba umkhuba oyinhloko ekuqeqeshweni kwe-GPU okusebenzayo.

Kungani i-VRAM isabonakala igcwele ngemva kokuphela kweskripthi sokuqeqesha

Amafreyimu avame ukugcina imemori ye-GPU ukuze athole isivinini, ngakho-ke imemori egciniwe ingahlala iphezulu ngisho noma imemori ebekelwe yona yehla. Ingafana nokuvuza, kodwa ngokuvamile i-caching allocator isebenza njengoba iklanyelwe. Umkhuba osebenzayo ukulandelela iphethini ngokuhamba kwesikhathi bese uqhathanisa "ebekelwe vs ebekelwe" kunokugxila esithombeni esisodwa esesabekayo.

Indlela yokuqinisekisa ukuthi imodeli ayiqeqeshwa buthule ku-CPU

Hlola ingqondo kusenesikhathi: qinisekisa ukuthi i-torch.cuda.is_available() ibuyisela i-True , qinisekisa ukuthi i-next(model.parameters()).device ibonisa i-cuda , bese usebenzisa i-forward pass eyodwa ngaphandle kwamaphutha. Uma ukusebenza kuzwakala kuhamba kancane ngendlela exakile, qinisekisa nokuthi ama-batch akho ayiswa ku-GPU. Kuvamile ukuhambisa imodeli bese ushiya idatha ngephutha.

Indlela elula yokuqeqeshwa kwe-multi-GPU

I-Data Parallel (ukuqeqeshwa kwesitayela se-DDP) ivame ukuba yisinyathelo sokuqala esihle kakhulu: ukuhlukanisa ama-batch kuwo wonke ama-GPU nokuvumelanisa ama-gradients. Amathuluzi afana ne-Accelerate angenza i-multi-GPU ibe buhlungu kancane ngaphandle kokubhala kabusha okugcwele. Lindela iziguquguquko ezengeziwe - ukuxhumana kwe-NCCL, umehluko wokuxhumana (NVLink vs PCIe), kanye nokwanda kwezithiyo zedatha - ngakho-ke ukukala kancane kancane ngemva kokusebenza kwe-single-GPU eqinile kuvame ukuhamba kangcono.

Okufanele ukuqaphele ngesikhathi sokuqeqeshwa kwe-NVIDIA GPU ukuze kutholakale izinkinga kusenesikhathi

Bukela ukusetshenziswa kwe-GPU, ukusetshenziswa kwememori (okuzinzile vs ukukhuphuka), ukudonsa kwamandla, kanye namazinga okushisa - ukugoqa kunganciphisa isivinini buthule. Qaphela nokusetshenziswa kwe-CPU, njengoba inkinga yepayipi ledatha ivame ukuvela lapho kuqala. Uma ukusetshenziswa kunciphile noma kuphansi, sola i-I/O noma ama-dataloaders; uma kuphezulu kodwa isikhathi sesinyathelo sisahamba kancane, ama-kernel ephrofayili, imodi yokucacisa, kanye nokuhlukaniswa kwesikhathi sesinyathelo.

Izinkomba

  1. NVIDIA - NVIDIA nvidia-smi - docs.nvidia.com

  2. I-NVIDIA - I-NVIDIA System Management Interface (NSVMI) - developer.nvidia.com

  3. NVIDIA - NVIDIA NVLink - nvidia.com

  4. I-PyTorch - I-PyTorch Qala (isikhethi se-CUDA) - pytorch.org

  5. I-PyTorch - PyTorch Amadokhumenti e-CUDA - docs.pytorch.org

  6. -TensorFlow - TensorFlow (ipayipi) - tensorflow.org

  7. I-JAX - Isiqalo Esisheshayo se-JAX - docs.jax.dev

  8. Ubuso Obugonayo - Amadokhumenti Omqeqeshi - huggingface.co

  9. I-Lightning AI - Amadokhumenti e-Lightning - lightning.ai

  10. e-DeepSpeed ​​- ZeRO - deepspeed.readthedocs.io

  11. Ucwaningo lweMicrosoft - Ucwaningo lweMicrosoft: ZeRO/DeepSpeed ​​- microsoft.com

  12. Izinkundla zePyTorch - Inkundla yePyTorch: hlola imodeli ku-CUDA - discuss.pytorch.org

Thola i-AI Yakamuva Esitolo Esisemthethweni Somsizi we-AI

Mayelana NATHI

Buyela kubhulogi