Impendulo emfushane: Chaza ukuthi "okuhle" kubukeka kanjani esimweni sakho sokusetshenziswa, bese uhlola ngezikhuthazo ezimele, eziguquliwe kanye namacala onqenqema. Hlanganisa amamethrikhi azenzakalelayo nama-rubric scoring abantu, kanye nokuhlolwa kokuphepha okuphikisayo kanye nokuhlolwa kokufakwa ngokushesha. Uma imikhawulo yezindleko noma yokubambezeleka iba yisibopho, qhathanisa amamodeli ngempumelelo yomsebenzi ngephawundi elichithwe kanye nezikhathi zokuphendula ze-p95/p99.
Izinto ezibalulekile okufanele uzicabangele:
Ukuziphendulela : Nika abanikazi abacacile, gcina ama-log enguqulo, bese uphinda usebenzise ama-eval ngemva kwanoma yikuphi ukushintsha okusheshayo noma imodeli.
Ukucaca : Bhala phansi izindlela zokuphumelela, imingcele, kanye nezindleko zokwehluleka ngaphambi kokuthi uqale ukuqoqa amaphuzu.
Ukuhlolwa : Gcina ama-test suite aphindaphindwayo, amasethi edatha anelebula, kanye nezilinganiso ze-latency ze-p95/p99 ezilandelelwayo.
Ukuncintisana : Sebenzisa amarubrikhi okubuyekezwa ngabantu kanye nendlela echaziwe yezikhalazo zemiphumela ephikiswanayo.
Ukumelana nokusetshenziswa kabi : Ukufakwa ngokushesha kweqembu elibomvu, izihloko ezibucayi, kanye nokwenqaba ngokweqile ukuvikela abasebenzisi.
Uma ukhetha imodeli yomkhiqizo, iphrojekthi yocwaningo, noma ithuluzi langaphakathi, awukwazi nje ukuthi “kuzwakala kuhlakaniphile” bese ulithumela (bheka umhlahlandlela we-OpenAI evals kanye ne- NIST AI RMF 1.0 ). Yileyo ndlela ogcina ngayo une-chatbot echaza ngokuzethemba ukuthi ungayisebenzisa kanjani ifoloko kuma-microwave. 😬
Izihloko ongase uthande ukuzifunda ngemva kwalesi:
🔗 Ikusasa le-AI: izitayela ezibumba iminyaka eyishumi ezayo
Izinto ezintsha ezibalulekile, umthelela wemisebenzi, kanye nokuziphatha okufanele ukubheke phambili.
🔗 Amamodeli esisekelo ku-AI yokukhiqiza achazelwe abaqalayo
Funda ukuthi ayini, aqeqeshwe kanjani, nokuthi kungani ebalulekile.
🔗 Indlela i-AI ethinta ngayo imvelo nokusetshenziswa kwamandla
Hlola ukukhishwa kwegesi, isidingo sikagesi, kanye nezindlela zokunciphisa ukutholakala kwezindawo ezizungezile.
🔗 Indlela i-AI ekhuliswa ngayo izithombe ezibukhali namuhla
Bona ukuthi amamodeli anezela kanjani imininingwane, asusa kanjani umsindo, futhi akhulise ngokuhlanzekile.
1) Ukuchaza "okuhle" (kuya ngokuthi, futhi lokho kulungile) 🎯
Ngaphambi kokuthi wenze noma yikuphi ukuhlola, nquma ukuthi impumelelo ibukeka kanjani. Ngaphandle kwalokho uzolinganisa konke futhi ungafundi lutho. Kufana nokuletha i-tape measure ukuze wahlulele umncintiswano wekhekhe. Impela, uzothola izinombolo, kodwa ngeke zikutshele okuningi 😅
Cacisa:
-
Umgomo womsebenzisi : ukufingqa, ukusesha, ukubhala, ukucabanga, ukukhipha amaqiniso
-
Izindleko zokwehluleka : isincomo sefilimu esingalungile siyahlekisa; imiyalelo yezokwelapha engalungile ayihlekisi (ukubeka engcupheni: NIST AI RMF 1.0 ).
-
Indawo yokusebenza : kudivayisi, efwini, ngemuva kwe-firewall, endaweni elawulwayo
-
Imikhawulo eyinhloko : ukubambezeleka, izindleko ngesicelo ngasinye, ubumfihlo, ukuchazeka, ukwesekwa kwezilimi eziningi, ukulawula ithoni
Imodeli "engcono kakhulu" emsebenzini owodwa ingaba yinhlekelele komunye. Lokho akuyona into ephikisanayo, kuyiqiniso. 🙂
2) Yeka ukuthi uhlaka lokuhlola imodeli ye-AI oluqinile lubukeka kanjani 🧰
Yebo, lena yingxenye abantu abayigwemayo. Babamba i-benchmark, bayisebenzise kanye, bese beyibiza ngosuku. Uhlaka lokuhlola oluqinile lunezici ezimbalwa ezihambisanayo (izibonelo zamathuluzi asebenzayo: i-OpenAI Evals / umhlahlandlela we-OpenAI evals ):
-
Kungaphindwa - ungayiphinda futhi ngesonto elizayo bese uthemba ukuqhathanisa
-
Ummeleli - ubonisa abasebenzisi bakho bangempela kanye nemisebenzi (hhayi nje imibuzo engavamile)
-
I-Multi-layered - ihlanganisa ama-metric azenzakalelayo + isibuyekezo somuntu + ukuhlolwa okuphikisayo
-
Iyasebenza - imiphumela ikutshela ukuthi yini okufanele uyilungise, hhayi nje ukuthi "imiphumela yehle"
-
Ukumelana nokuphazamiseka - kugwema "ukufundisa ukuhlolwa" noma ukuvuza ngengozi
-
Ukuqaphela izindleko - ukuhlolwa ngokwakho akufanele kukuqede imali (ngaphandle kokuthi uthanda ubuhlungu)
Uma ukuhlola kwakho kungaphumeleli lapho umuntu ozakwenu engabazi ethi “Kulungile, kodwa hlanganisa lokhu nomkhiqizo,” khona-ke akukapheli. Yilokho okuhlola isimo.
3) Indlela Yokuhlola Amamodeli E-AI Ngokuqala Ngezingcezu Zokusetshenziswa 🍰
Nasi icebo elisindisa isikhathi esiningi: hlukanisa icala lokusebenzisa libe yizicucu .
Esikhundleni sokuthi “hlola imodeli,” yenza lokhu:
-
Ukuqonda inhloso (ingabe kuyakuthola lokho umsebenzisi akufunayo)
-
Ukusetshenziswa kokubuyisa noma komongo (ingabe kusebenzisa ulwazi olunikeziwe ngendlela efanele)
-
Ukubonisana / imisebenzi enezinyathelo eziningi (ingabe ihlala ihambisana kuzo zonke izinyathelo)
-
Ukufometha kanye nesakhiwo (ingabe kuyalandela imiyalelo)
-
Ukuhambisana kokuphepha kanye nenqubomgomo (ingabe kuyakugwema okuqukethwe okungaphephile; bheka i-NIST AI RMF 1.0 )
-
Ithoni kanye nezwi lomkhiqizo (ingabe kuzwakala sengathi ufuna kuzwakale)
Lokhu kwenza "Indlela Yokuhlola Amamodeli E-AI" kuzwakale sengathi akuyona nje inselelo enkulu kodwa kuzwakala sengathi yisethi yemibuzo eqondiwe. Imibuzo iyacasula, kodwa iyalawuleka. 😄
4) Izisekelo zokuhlola ezingaxhunyiwe ku-inthanethi - amasethi okuhlola, amalebula, kanye nemininingwane engathandeki ebalulekile 📦
I-eval engaxhunyiwe ku-inthanethi yilapho wenza khona izivivinyo ezilawulwayo ngaphambi kokuba abasebenzisi bathinte noma yini (amaphethini okuhamba komsebenzi: i-OpenAI Evals ).
Yakha noma uqoqe isethi yokuhlola engeyakho ngempela
Isethi enhle yokuhlola ivame ukufaka:
-
Izibonelo zegolide : imiphumela emihle ongayithumela ngokuziqhenya
-
Amacala e-Edge : izikhuthazo ezingacacile, okokufaka okungahlelekile, ukufometha okungalindelekile
-
Ama-probe emodi yokwehluleka : izimpendulo ezilinga ukubona izinto ezingekho noma izimpendulo ezingaphephile (ukuhlelwa kokuhlolwa kwengozi: NIST AI RMF 1.0 )
-
Ukumbozwa kokwehlukahluka : amazinga ahlukene amakhono abasebenzisi, izilimi, izilimi, izizinda
Uma uhlola kuphela ngemiyalelo "ehlanzekile", imodeli izobukeka imangalisa. Ngemuva kwalokho abasebenzisi bakho bafika benamaphutha okubhala, imisho engama-half, kanye namandla okuchofoza intukuthelo. Siyakwamukela ku-reality.
Izinketho zokulebula (okwaziwa nangokuthi amazinga okuqina)
Ungalebula imiphumela ngokuthi:
-
Binary : dlula/hluleka (okusheshayo, okunokhahlo)
-
Okujwayelekile : amaphuzu ekhwalithi angu-1-5 (okuhlukile, okusekelwe kumuntu ngamunye)
-
Izimfanelo eziningi : ukunemba, ukuphelela, ithoni, ukusetshenziswa kwengcaphuno, njll. (okungcono kakhulu, okuhamba kancane)
I-Multi-attribute iyindawo emnandi yamaqembu amaningi. Kufana nokunambitha ukudla nokwahlulela usawoti ngokwehlukana nokuthungwa. Ngaphandle kwalokho umane uthi “kuhle” bese unikina amahlombe.
5) Izilinganiso ezingaqambi amanga - kanye nezilinganiso eziqamba amanga 📊😅
Izilinganiso ziwusizo… kodwa zingaba yibhomu elikhazimulayo. Ziyakhazimula, yonke indawo, futhi kunzima ukuzihlanza.
Imindeni ejwayelekile ye-metric
-
Ukunemba / ukufana okuqondile : kuhle kakhulu ekukhipheni, ekuhlukaniseni, emisebenzini ehlelekile
-
F1 / ukunemba / ukukhumbula : kuyasiza uma uphuthelwe okuthile kubi kakhulu kunomsindo owengeziwe (izincazelo: scikit-learn ukunemba/ukukhumbula/i-F-score )
-
Isitayela se-BLEU/ROUGE sidlulana : kulungile ngemisebenzi efana nokufingqa, evame ukudukisa (izilinganiso zokuqala: BLEU kanye ne -ROUGE )
-
Ukushumeka ukufana : kuyasiza ekufanisweni kwe-semantic, kungavuza izimpendulo ezingalungile kodwa ezifanayo
-
Izinga lempumelelo yomsebenzi : "ingabe umsebenzisi ukutholile abekudinga" indinganiso yegolide uma ichazwe kahle
-
Ukuthobela imithetho yemingcele : kulandela ifomethi, ubude, ukusebenza kwe-JSON, ukunamathela kweskimu
Iphuzu eliyinhloko
Uma umsebenzi wakho uvulekile (ukubhala, ukucabanga, ingxoxo yokusekela), izilinganiso zenombolo eyodwa zingaba… ezintengantengayo. Akusizi ngalutho, ezintengantengayo nje. Ukulinganisa ubuhlakani ngerula kungenzeka, kodwa uzozizwa uyisiwula ukukwenza. (Futhi uzokhipha iso lakho, mhlawumbe.)
Ngakho-ke: sebenzisa izindlela zokulinganisa, kodwa ziqinise ekubuyekezweni kwabantu kanye nemiphumela yangempela yomsebenzi (isibonelo esisodwa sengxoxo yokuhlola esekelwe ku-LLM + izixwayiso: G-Eval ).
6) Ithebula Lokuqhathanisa - izinketho zokuhlola eziphezulu (ezinezimpawu ezingavamile, ngoba impilo inezimpawu ezingavamile) 🧾✨
Nansi imenyu ewusizo yezindlela zokuhlola. Xuba bese ufanisa. Amaqembu amaningi ayakwenza lokho.
| Ithuluzi / Indlela | Izithameli | Intengo | Kungani kusebenza |
|---|---|---|---|
| I-suite yokuhlola okusheshayo eyakhelwe ngesandla | Umkhiqizo + i-eng | $ | Iqondiswe kakhulu, ibamba ukuhlehla ngokushesha - kodwa kufanele uyigcine kuze kube phakade 🙃 (ithuluzi lokuqala: i-OpenAI Evals ) |
| Iphaneli yokubeka amaphuzu yerubric yabantu | Amaqembu angasindisa ababuyekezi | $$ | Kungcono kakhulu ngethoni, ukuguquguquka, "ingabe umuntu angakwamukela lokhu", isiphithiphithi esincane kuye ngababuyekezi |
| I-LLM-njengomahluleli (eneziqondiso) | Ama-loop okuphindaphinda okusheshayo | $-$$ | Iyashesha futhi ingakhula, kodwa ingazuza ubandlululo futhi ngezinye izikhathi amamaki angabonisi amaqiniso (ucwaningo + izinkinga zobandlululo ezaziwayo: G-Eval ) |
| Ukushwibeka kweqembu elibomvu eliphikisanayo | Ukuphepha + ukuthobela imithetho | $$ | Ithola izindlela zokwehluleka okubabayo, ikakhulukazi ukujova ngokushesha - kuzwakala njengokuhlolwa kokucindezeleka ejimini (ukubuka konke kokusongela: I-OWASP LLM01 Injection Prompt / I-OWASP Top 10 yezinhlelo zokusebenza ze-LLM ) |
| Ukwenziwa kokuhlolwa kokwenziwa | Amaqembu okukhanya kwedatha | $ | Ukumbozwa okuhle kakhulu, kodwa izeluleko zokwenziwa zingaba zihle kakhulu, zihlonipheke kakhulu... abasebenzisi abananhlonipho |
| Ukuhlolwa kwe-A/B ngabasebenzisi bangempela | Imikhiqizo yabantu abadala | $$$ | Isibonakaliso esicacile - futhi sicindezela kakhulu ngokomzwelo lapho izibalo zishintsha (umhlahlandlela osebenzayo wakudala: Kohavi et al., “Ukuhlolwa okulawulwayo kuwebhu” ) |
| I-eval esuselwe phansi (ukuhlolwa kwe-RAG) | Sesha + izinhlelo zokusebenza ze-QA | $$ | Izilinganiso "zisebenzisa umongo ngendlela efanele," kunciphisa ukukhuphuka kwamanani okubona izinto ezingekho (ukubuka konke kwe-RAG: Ukuhlolwa kwe-RAG: Ucwaningo ) |
| Ukuqapha + ukutholwa kokukhukhuleka | Izinhlelo zokukhiqiza | $$-$$$ | Ibamba ukuwohloka ngokuhamba kwesikhathi - ayikhanyi kuze kube usuku ekusindisa ngalo 😬 (ukubuka konke kokukhukhuleka: Ucwaningo lokukhukhuleka komqondo (PMC) ) |
Qaphela ukuthi amanani aphansi ngamabomu. Ancike esikalini, kumathuluzi, kanye nenani lemihlangano oyiqala ngengozi.
7) Ukuhlolwa kwabantu - isikhali esiyimfihlo abantu abasixhasa ngemali encane 👀🧑⚖️
Uma wenza ukuhlola okuzenzakalelayo kuphela, uzophuthelwa:
-
Ukungafani kwethoni (“kungani kuhlekisa kangaka”)
-
Amaphutha angabonakali angokoqobo abonakala ecacile
-
Imiphumela elimazayo, imibono engafani, noma ukubekwa kwamagama okungajwayelekile (ukubeka engcupheni + ubandlululo: NIST AI RMF 1.0 )
-
Ukwehluleka okulandela imiyalelo okusazwakala “kuhlakaniphile”
Yenza amarubrikhi abe yinto eqondile (noma ababuyekezi bazoyi-freestyle)
Isihloko esibi: “Usizo”
Isihloko esingcono:
-
Ukunemba : kunembile ngokweqiniso uma kubhekwa ukushesha + umongo
-
Ukuphelela : kuhlanganisa amaphuzu adingekayo ngaphandle kokukhuluma ngokunganaki
-
Ukucaca : okufundekayo, okuhlelekile, okuphansi kokudideka
-
Inqubomgomo / ukuphepha : igwema okuqukethwe okuvinjelwe, iphatha ukwenqatshwa kahle (uhlaka lokuphepha: NIST AI RMF 1.0 )
-
Isitayela : kuhambisana nezwi, ithoni, izinga lokufunda
-
Ukwethembeka : akusunguli imithombo noma izimangalo ezingasekelwa
Futhi, hlola abalinganisi bezilinganiso ngezinye izikhathi. Uma ababuyekezi ababili bengavumelani njalo, akuyona "inkinga yabantu," kuyinkinga yama-rubric. Ngokuvamile (izisekelo zokuthembeka kwabalinganisi bezilinganiso: uMcHugh ku-kappa kaCohen ).
8) Indlela Yokuhlola Amamodeli E-AI Ukuphepha, Ukuqina, kanye "no-awu, basebenzisi" 🧯🧪
Lena yingxenye oyenzayo ngaphambi kokwethula - bese uqhubeka nokwenza, ngoba i-inthanethi ayilali.
Ukuhlolwa kokuqina kufanele kufakwe
-
Ukuthayipha, isitsotsi, uhlelo lolimi oluphukile
-
Izimpendulo ezinde kakhulu nezifushane kakhulu
-
Imiyalelo ephikisanayo ("yiba mfushane kodwa ufake yonke imininingwane")
-
Izingxoxo ezishintshashintshayo lapho abasebenzisi beshintsha khona imigomo
-
Imizamo yokujova ngokushesha (“unganaki imithetho yangaphambilini…”) (imininingwane yokusongela: OWASP LLM01 Prompt Injection )
-
Izihloko ezibucayi ezidinga ukwenqatshwa ngokucophelela (ukubeka engcupheni/ukuphepha: NIST AI RMF 1.0 )
Ukuhlolwa kokuphepha akukhona nje ukuthi “kuyala”
Imodeli enhle kufanele:
-
Yenqaba izicelo ezingaphephile ngokucacile nangokuzola (uhlaka lwesiqondiso: NIST AI RMF 1.0 )
-
Nikeza ezinye izindlela eziphephile uma kufaneleka
-
Gwema ukwenqaba ngokweqile imibuzo engenangozi (imibono engalungile)
-
Phatha izicelo ezingacacile ngemibuzo ecacile (uma kuvunyelwe)
Ukwenqaba ngokweqile kuyinkinga yangempela yomkhiqizo. Abasebenzisi abakuthandi ukuphathwa njengama-goblin asolisayo. 🧌 (Ngisho noma bengama-goblin asolisayo.)
9) Izindleko, ukubambezeleka, kanye neqiniso lokusebenza - ukuhlolwa okukhohlwa yiwo wonke umuntu 💸⏱️
Imodeli ingaba “emangalisayo” kodwa ibe nephutha kuwe uma ihamba kancane, ibiza kakhulu, noma isebenza kalula.
Hlola:
-
Ukusatshalaliswa kwe-latency (hhayi nje kuphela isilinganiso - p95 kanye ne-p99 matter) (kungani ama-percentile ebalulekile: Incwadi Yokusebenzela ye-Google SRE mayelana nokuqapha )
-
Izindleko ngomsebenzi ngamunye ophumelelayo (hhayi izindleko ngethokheni ngayinye ezihlukanisiwe)
-
Ukuzinza ngaphansi komthwalo (izikhathi zokuvala, imikhawulo yesilinganiso, ukukhuphuka okungavamile)
-
Ukuthembeka kokubiza ithuluzi (uma lisebenzisa imisebenzi, ingabe liyaziphethe kahle)
-
Ukuthambekela kobude bokukhipha (amanye amamodeli ayadlala, futhi ukudlala kubiza imali)
Imodeli embi kancane eshesha kabili inganqoba ekusebenzeni. Lokho kuzwakala kusobala, kodwa abantu bayakushalazela. Njengokuthenga imoto yezemidlalo ukuze ugibele ukudla, bese ukhononda ngendawo yebhulukwe.
10) Indlela elula yokusebenza kusukela ekuqaleni kuze kube sekupheleni ongayikopisha (futhi uyilungise) 🔁✅
Nansi indlela ewusizo yokuhlola amamodeli e-AI ngaphandle kokubhajwa ezivivinyweni ezingapheli:
-
Chaza impumelelo : umsebenzi, imingcele, izindleko zokwehluleka
-
Dala isethi encane yokuhlola "eyisisekelo" : izibonelo ezingama-50-200 ezibonisa ukusetshenziswa kwangempela
-
Engeza amasethi onqenqema kanye nokuphikisana : imizamo yokujova, izixwayiso ezingacacile, ama-probe okuphepha (isigaba sokujova esisheshayo: OWASP LLM01 )
-
Sebenzisa ukuhlola okuzenzakalelayo : ukufometha, ukufaneleka kwe-JSON, ukunemba okuyisisekelo lapho kungenzeka khona
-
Qalisa isibuyekezo somuntu : imiphumela yesampula kuzo zonke izigaba, thola amaphuzu nge-rubric
-
Qhathanisa ukuhwebelana : ikhwalithi vs izindleko vs ukubambezeleka vs ukuphepha
-
Ukuhlolwa kokuhlolwa okulinganiselwe : Ukuhlolwa kwe-A/B noma ukukhishwa okuhleliwe (umhlahlandlela wokuhlola i-A/B: Kohavi et al. )
-
Ukuqapha ekukhiqizweni : ukuzulazula, ukuhlehla, ama-loop empendulo yomsebenzisi (ukubuka konke kokuzulazula: Ucwaningo lokuzulazula komqondo (i-PMC) )
-
Iterate : buyekeza izixwayiso, ukubuyisa, ukulungisa kahle, ukuvikela, bese usebenzisa kabusha i-eval (amaphethini okuphindaphinda eval: Umhlahlandlela we-OpenAI evals )
Gcina ama-log aguquliwe. Hhayi ngoba kumnandi, kodwa ngoba ikusasa - uzokubonga ngenkathi uphethe ikhofi futhi ukhononda ngokuthi "yini eshintshile ..." ☕🙂
11) Izingibe ezivamile (okwaziwa nangokuthi: izindlela abantu abazikhohlisa ngazo ngengozi) 🪤
-
Ukuqeqeshwa kokuhlolwa : wenza ngcono izikhuthazo kuze kube yilapho ibhentshi libukeka lihle, kodwa abasebenzisi bayahlupheka
-
Idatha yokuhlola evuzayo : izixwayiso zokuhlola zivela ekuqeqeshweni noma ekulungiseni idatha (hawu)
-
Ukukhulekela okukodwa kwe-metric : ukuphishekela amaphuzu owodwa angabonisi inani lomsebenzisi
-
Ukunganaki ukushintsha kokusabalalisa : izinguquko zokuziphatha komsebenzisi kanye nemodeli yakho yonakala buthule (ukwakheka kwengozi yokukhiqiza: Ucwaningo lwe-Concept drift (PMC) )
-
Ukubheka ngokweqile “ukuhlakanipha” : ukucabanga okuhlakaniphile akunandaba ukuthi kuyaphula ifomethi noma kusungula amaqiniso
-
Ukungahloli ikhwalithi yokwenqaba : “Cha” kungaba yiqiniso kodwa kusalokhu kuyi-UX embi kakhulu
Futhi, qaphela ama-demo. Ama-demo afana nama-trailer ama-movie. Abonisa izinto ezivelele, afihla izingxenye ezihamba kancane, futhi ngezinye izikhathi alale ngomculo odlalwayo. 🎬
12) Isifinyezo sokuphetha sendlela yokuhlola amamodeli e-AI 🧠✨
Ukuhlola amamodeli e-AI akuyona into eyodwa, kungukudla okulinganiselayo. Udinga amaprotheni (ukunemba), imifino (ukuphepha), ama-carbohydrate (isivinini kanye nezindleko), futhi yebo, ngezinye izikhathi i-dessert (ithoni kanye nenjabulo) 🍲🍰 (ukubeka ingozi: NIST AI RMF 1.0 )
Uma ungakhumbuli lutho olunye:
-
Chaza ukuthi kusho ukuthini ukuthi “okuhle” esimweni sakho sokusetshenziswa
-
Sebenzisa amasethi okuhlola amele, hhayi nje amabhentshimakhi adumile
-
Hlanganisa amamethrikhi azenzakalelayo nokubuyekezwa kwerubrikhi yomuntu
-
Ukuqina kokuhlolwa kanye nokuphepha njengabasebenzisi kuyaphikisana (ngoba ngezinye izikhathi... bayaphikisana) (isigaba sokufaka ngokushesha: OWASP LLM01 )
-
Faka izindleko kanye nokubambezeleka ekuhlolweni, hhayi njengokucabanga kamuva (kungani ama-percentiles ebalulekile: Incwadi Yokusebenzela ye-Google SRE )
-
Ukuqapha ngemva kokuqaliswa - amamodeli ayakhukhuleka, izinhlelo zokusebenza ziyathuthuka, abantu baba nobuciko (ukubuka konke kokukhukhuleka: Ucwaningo lokukhukhuleka komqondo (PMC) )
Yileyo Indlela Yokuhlola Amamodeli E-AI ngendlela ehlala isikhathi eside lapho umkhiqizo wakho ukhona futhi abantu beqala ukwenza izinto ezingalindelekile kubantu. Okuhlala kunjalo. 🙂
Imibuzo Evame Ukubuzwa
Yisiphi isinyathelo sokuqala sokuhlola amamodeli e-AI ngomkhiqizo wangempela?
Qala ngokuchaza ukuthi kusho ukuthini "okuhle" esimweni sakho esithile sokusetshenziswa. Chaza umgomo womsebenzisi, ukuthi yikuphi ukwehluleka okukubize (izindleko eziphansi vs izindleko eziphezulu), nokuthi imodeli izosebenza kuphi (ifu, kudivayisi, indawo elawulwayo). Bese ubhala imikhawulo eqinile njengokulibaziseka, izindleko, ubumfihlo, kanye nokulawula ithoni. Ngaphandle kwalesi sisekelo, uzolinganisa okuningi futhi usazokwenza isinqumo esibi.
Ngingayakha kanjani isethi yokuhlola ebonakalisa ngempela abasebenzisi bami?
Yakha isethi yokuhlola engeyakho ngempela, hhayi nje ibhentshi lomphakathi. Faka izibonelo zegolide ongazithumela ngokuziqhenya, kanye neziphakamiso ezinomsindo, ezingavamile ezinamaphutha okubhala, imisho eyingxenye, kanye nezicelo ezingacacile. Engeza izimo ezinqenqemeni kanye nezindlela zokuhlola zemodi yokwehluleka ezilinga ukubona izinto ezingekho noma izimpendulo ezingaphephile. Mboza ukuhlukahluka kwezinga lamakhono, izilimi, kanye nezindawo ukuze imiphumela ingawi ekukhiqizweni.
Yiziphi izindlela zokulinganisa okufanele ngizisebenzise, futhi yiziphi ezingadukisa?
Qondanisa amamethrikhi nohlobo lomsebenzi. Ukufana okuqondile kanye nokunemba kusebenza kahle ekukhipheni nasekuphumeni okuhlelekile, kuyilapho ukunemba/ukukhumbula kanye nokusiza kwe-F1 lapho uphuthelwa okuthile kubi kakhulu kunomsindo owengeziwe. Amamethrikhi ahambisanayo njenge-BLEU/ROUGE angadukisa imisebenzi evulekile, futhi ukushumeka ukufana kungavuza izimpendulo "ezingalungile kodwa ezifanayo". Ngokubhala, ukusekela, noma ukucabanga, hlanganisa amamethrikhi nokubuyekezwa kwabantu kanye namazinga empumelelo yomsebenzi.
Ngingahlela kanjani ukuhlolwa ukuze kuphindeke futhi kube yizinga lokukhiqiza?
Uhlaka lokuhlola oluqinile luyaphindaphindeka, lumelela, lunezingqimba eziningi, futhi luyasebenza. Hlanganisa ukuhlola okuzenzakalelayo (ifomethi, ukufaneleka kwe-JSON, ukunemba okuyisisekelo) nama-rubric scoring abantu kanye nokuhlolwa okuphikisayo. Kwenze kungaphazamiseki ngokugwema ukuvuza kanye "nokufundisa isivivinyo." Gcina uqaphela izindleko zokuhlola ukuze ukwazi ukukuphinda ukwenze njalo, hhayi kanye nje ngaphambi kokuqaliswa.
Iyiphi indlela engcono kakhulu yokwenza ukuhlolwa komuntu ngaphandle kokuthi kuphenduke isiphithiphithi?
Sebenzisa irubrikhi eqondile ukuze ababuyekezi bangasebenzisi isitayela esikhululekile. Beka amaphuzu ezicini ezifana nokunemba, ukuphelela, ukucaca, ukuphepha/ukuphathwa kwenqubomgomo, isitayela/ukufaniswa kwezwi, kanye nokwethembeka (hhayi ukusungula izimangalo noma imithombo). Hlola njalo isivumelwano sabalinganisi; uma ababuyekezi bengavumelani njalo, irubrikhi cishe idinga ukulungiswa. Ukubuyekezwa kwabantu kubaluleke kakhulu ekungahambelani kwethoni, amaphutha amaqiniso angabonakali, kanye nokwehluleka okulandela imiyalelo.
Ngingakuhlola kanjani ukuphepha, ukuqina, kanye nezingozi zokujova ngokushesha?
Hlola ngokufakwayo kokuthi “ugh, abasebenzisi”: amaphutha okubhala, isitsotsi, imiyalelo ephikisanayo, izikhuthazo ezinde kakhulu noma ezimfushane kakhulu, kanye nokushintsha imigomo eminingi. Faka imizamo yokufaka ngokushesha njengokuthi “ukungazinaki imithetho yangaphambilini” kanye nezihloko ezibucayi ezidinga ukwenqatshwa ngokucophelela. Ukusebenza kahle kokuphepha akukhona nje ukwenqaba - ukwenqaba ngokucacile, ukunikeza ezinye izindlela eziphephile lapho kufaneleka, kanye nokugwema ukwenqaba ngokweqile imibuzo engenangozi elimaza i-UX.
Ngingazihlola kanjani izindleko kanye nokubambezeleka ngendlela efanelana neqiniso?
Ungalingi nje kuphela isilinganiso - landelela ukusatshalaliswa kwe-latency, ikakhulukazi i-p95 kanye ne-p99. Hlola izindleko ngomsebenzi ngamunye ophumelelayo, hhayi izindleko ngethokheni ngayinye ngokuhlukana, ngoba ukuzama kabusha kanye nemiphumela yokuzulazula kungasula ukonga. Hlola ukuzinza ngaphansi komthwalo (ukuphela kwesikhathi, imikhawulo yesilinganiso, ama-spikes) kanye nokuthembeka kokubiza ithuluzi/umsebenzi. Imodeli embi kancane esheshayo kabili noma eqinile kakhulu ingaba ukukhetha komkhiqizo okungcono.
Yimuphi umsebenzi olula wokuhlola amamodeli e-AI?
Chaza izindlela zokuphumelela kanye nemikhawulo, bese udala isethi encane yokuhlola eyinhloko (cishe izibonelo ezingu-50-200) ekhombisa ukusetshenziswa kwangempela. Engeza amasethi onqenqema kanye nokuphikisana ukuze uzame ukuphepha kanye nokufaka. Sebenzisa ukuhlola okuzenzakalelayo, bese wenza isampula yemiphumela yokulinganisa i-rubric yomuntu. Qhathanisa ikhwalithi vs izindleko vs ukubambezeleka vs ukuphepha, i-pilot enokukhishwa okulinganiselwe noma ukuhlolwa kwe-A/B, bese uqapha ekukhiqizweni ukuze uthole ukukhukhuleka kanye nokubuyela emuva.
Yiziphi izindlela ezivame kakhulu amaqembu azikhohlisa ngazo ngengozi ekuhlolweni kwemodeli?
Izingibe ezivamile zifaka phakathi ukwenza ngcono izikhuthazo ukuze zifinyelele izinga elibekiwe ngenkathi abasebenzisi behlupheka, ukuvuza izikhuthazo zokuhlola kudatha yokuqeqeshwa noma yokulungisa kahle, kanye nokukhulekela i-metric eyodwa engabonakalisi inani lomsebenzisi. Amaqembu futhi awakunaki ukushintsha kokusabalalisa, ukubheka ngokweqile "ukuhlakanipha" esikhundleni sokuthobela ifomethi nokwethembeka, kanye nokweqa ukuhlolwa kwekhwalithi yokwenqaba. Ama-demo angafihla lezi zinkinga, ngakho-ke athembele kuma-eval ahlelekile, hhayi kuma-reels agqamisayo.
Izinkomba
-
I-OpenAI - Umhlahlandlela we-OpenAI evals - platform.openai.com
-
Isikhungo Sikazwelonke Sezindinganiso Nobuchwepheshe (i-NIST) - Uhlaka Lokuphathwa Kwengozi ye-AI (i-AI RMF 1.0) - nist.gov
-
I-OpenAI - i-openai/evals (indawo yokugcina i-GitHub) - github.com
-
ukufunda kwe-scikit - ukwesekwa kwe-precision_recall_fscore_score - scikit-learn.org
-
Inhlangano Yezilimi Zokusebenzisa Ikhompyutha (i-ACL Anthology) - BLEU - aclanthology.org
-
Inhlangano Yezilimi Zokusebenzisa Ikhompyutha (i-ACL Anthology) - ROUGE - aclanthology.org
-
arXiv - G-Eval - arxiv.org
-
I-OWASP - LLM01: Ukujova Okusheshayo - owasp.org
-
I-OWASP - I-OWASP Eziyi-10 Eziphezulu Zezinhlelo Zokusebenza Zemodeli Yolimi Olukhulu - owasp.org
-
I-Stanford University - Kohavi et al., “Ukuhlolwa okulawulwayo kuwebhu” - stanford.edu
-
arXiv - Ukuhlolwa kwe-RAG: Ucwaningo - arxiv.org
-
I-PubMed Central (PMC) - Ucwaningo lwe-Concept drift (PMC) - nih.gov
-
I-PubMed Central (PMC) - McHugh mayelana ne-kappa kaCohen - nih.gov
-
I-Google - Incwadi Yokusebenzela ye-SRE yokuqapha - google.workbook