Ukutholwa kwe-Anomaly kuyiqhawe elithule lokusebenza kwedatha - i-alamu yentuthu ehleba ngaphambi kokuba izinto ziqale umlilo.
Ngamagama alula: I-AI ifunda ukuthi "i-normal-ish" ibukeka kanjani, inike imicimbi emisha amaphuzu angajwayelekile , bese inquma ukuthi kufanele ifake umuntu ekhasini (noma ivimbe ngokuzenzakalelayo into) ngokusekelwe kumkhawulo . UDeveli usendleleni ochaza ngayo "normal-ish" lapho idatha yakho ingeyesizini, ingcolile, izulazula, futhi ngezinye izikhathi ikuqamba amanga. [1]
Izihloko ongase uthande ukuzifunda ngemva kwalesi:
🔗 Kungani i-AI ingaba yingozi emphakathini.
Ihlola izingozi zokuziphatha, ezomnotho, kanye nezenhlalo zokwamukelwa kwe-AI okusabalele.
🔗 Inani lamanzi asetshenziswa izinhlelo ze-AI empeleni.
Kuchaza ukupholisa isikhungo sedatha, izidingo zokuqeqeshwa, kanye nomthelela wamanzi emvelweni.
🔗 Iyini isethi yedatha ye-AI nokuthi kungani ibalulekile
Ichaza amasethi edatha, ukulebula, imithombo, kanye nendima yawo ekusebenzeni kwemodeli.
🔗 Indlela i-AI ebikezela ngayo izitayela ezivela kudatha eyinkimbinkimbi
Ihlanganisa ukuqashelwa kwamaphethini, amamodeli okufunda komshini, kanye nokusetshenziswa kokubikezela emhlabeni wangempela.
"I-AI Ithola Kanjani Izinto Ezingavamile?"
Impendulo enhle kufanele yenze okungaphezu kokubhala uhlu lwama-algorithms. Kufanele ichaze indlela esebenza ngayo nokuthi ibukeka kanjani uma uyisebenzisa kudatha yangempela, engaphelele. Izincazelo ezinhle kakhulu:
-
Khombisa izithako eziyisisekelo: izici , izisekelo , amaphuzu , kanye nemingcele . [1]
-
Qhathanisa imindeni engokoqobo: ibanga, ubuningi, isigaba esisodwa, ukuhlukaniswa, amathuba, ukwakha kabusha. [1]
-
Phatha izinkinga zochungechunge lwesikhathi: "okuvamile" kuncike esikhathini sosuku, usuku lwesonto, ukukhishwa, kanye namaholide. [1]
-
Phatha ukuhlolwa njengesithiyo sangempela: izixwayiso ezingamanga azicasuli nje kuphela - zishisa ukwethembana. [4]
-
Faka phakathi ukuhunyushwa + ukuhunyushwa komuntu ngaphakathi, ngoba "kuyaxaka" akuyona imbangela eyinhloko. [5]
Izindlela Eziyinhloko: Izisekelo, Amaphuzu, Iziphetho 🧠
Izinhlelo eziningi ezingavamile, kungaba zinhle noma cha, zihlukaniswe zibe izingxenye ezintathu ezinyakazayo:
okubonwa yimodeli )
Izimpawu ezingavuthiwe azivami ukwanela. Unjiniyela izici (izibalo ezigoqayo, izilinganiso, ukulibaziseka, ama-delta esizini) noma ufunda izethulo (ukushumeka, izikhala ezincane, ukwakhiwa kabusha). [1]
2) Ukuthola amaphuzu (okwaziwa nangokuthi: "kuxakile" kangakanani lokhu?)
Imibono evamile yokuthola amaphuzu ihlanganisa:
-
Okususelwe kude : kude nomakhelwane = okusolisayo. [1]
-
Okusekelwe ekuminyaneni : ukuminyana kwendawo okuphansi = okusolisayo (i-LOF iyingane yephosta). [1]
-
Imingcele yekilasi elilodwa : funda "okuvamile," phawula okuwela ngaphandle. [1]
-
Okungenzeka : amathuba aphansi ngaphansi kwemodeli efakiwe = okusolisayo. [1]
-
Iphutha lokwakha kabusha : uma imodeli eqeqeshwe ku-normal ingakwazi ukuyakha kabusha, cishe ivaliwe. [1]
3) Ukuvala intambo (okwaziwa nangokuthi: nini ukushaya insimbi)
Imingcele ingaba eqinile, esekelwe ku-quantile, ngesigaba ngasinye, noma ebucayi ezindleko - kodwa kufanele ilinganiswe ngokwesabelomali sokuxwayisa kanye nezindleko ezingezansi, hhayi ama-vibes. [4]
Umniningwane owodwa owusizo kakhulu: ama-scikit-learn's outlier/novelty detectors adalula amaphuzu angakalungiswa bese esebenzisa umkhawulo (ovame ukulawulwa ngokuqagela kwesitayela sokungcola) ukuguqula amaphuzu abe izinqumo ezingaphakathi/ezingaphandle. [2]
Izincazelo Ezisheshayo Ezivimbela Ubuhlungu Kamuva 🧯
Umehluko obili okusindisa emaphutheni angabonakali:
-
Ukutholwa kwe-Outlier : idatha yakho yokuqeqesha ingase ifake kakade ama-outliers; i-algorithm izama ukulingisa "isifunda esivamile esiminyene" noma kunjalo.
-
Ukutholwa kwentsha : idatha yokuqeqeshwa ithathwa njengehlanzekile; uyahlulela ukuthi okusha kuyahambisana yini nephethini evamile efundiwe. [2]
Futhi: ukutholwa kwezinto ezintsha kuvame ukuhlelwa njengokwahlukaniswa kwesigaba esisodwa - ukulingisa okujwayelekile ngoba izibonelo ezingavamile azivamile noma azichazwanga. [1]
Ama-Worhorse Angaqashiwe Ozowasebenzisa Ngempela 🧰
Uma amalebula enqabile (okuyinto ehlala ikhona), lawa amathuluzi avela emipayipini yangempela:
-
Ihlathi Lokuhlukaniswa : iphutha eliqinile ezimweni eziningi zethebula, elisetshenziswa kabanzi ekusebenzeni futhi lisetshenziswa ku-scikit-learn. [2]
-
I-SVM Yekilasi Elilodwa : ingasebenza kahle kodwa iyazwela ekulungiseni nasekucabangeni; i-scikit-learn iveza ngokusobala isidingo sokulungisa ngokucophelela i-hyperparameter. [2]
-
I-Local Outlier Factor (LOF) : amaphuzu asekelwe ku-density classic; kuhle kakhulu uma "okuvamile" kungeyona into ecacile. [1]
Amaqembu e-gotcha asebenzayo aphinde atholakale masonto onke: I-LOF iziphatha ngendlela ehlukile kuye ngokuthi wenza yini ukutholwa kwe-outlier kusethi yokuqeqesha vs. ukutholwa kwe-novelty kudatha entsha - i-scikit-learn idinga ngisho ne-novelty=True ukuthola amaphuzu angabonwa ngokuphephile. [2]
Isisekelo Esiqinile Esisasebenza Lapho Idatha Ingasebenzi Kahle 🪓
Uma usemodini ethi “sidinga nje into engasifaki ekukhohlweni”, izibalo eziqinile azinakwa kakhulu.
I -z-score eguquliwe isebenzisa i- median kanye ne -MAD (i-median absolute deviation) ukunciphisa ukuzwela kumanani adlulele. Incwadi yesandla ye-NIST ye-EDA ibhala ifomu le-z-score eguquliwe futhi iphawula umthetho ovame ukusetshenziswa "ongaphandle kwamandla" ngenani eliphelele elingaphezu kuka -3.5 . [3]
Lokhu ngeke kuxazulule yonke inkinga engavamile - kodwa ngokuvamile kuyindlela yokuqala yokuzivikela eqinile, ikakhulukazi ngezilinganiso ezinomsindo kanye nokuqapha kwesigaba sokuqala. [3]
Iqiniso Lochungechunge Lwesikhathi: “Okuvamile” Kuncike Ekuthini ⏱️📈
Ukungalingani kochungechunge lwesikhathi kuyaxaka ngoba umongo yiwona oyinhloko: ukunyuka okukhulu emini kungalindelwa; ukunyuka okufanayo ngo-3 ekuseni kungasho ukuthi kukhona okushisayo. Ngakho-ke izinhlelo eziningi ezisebenzayo zilinganisa ukujwayelekile zisebenzisa izici eziqaphela isikhathi (ukulibaziseka, ama-delta esizini, amafasitela ajikelezayo) kanye nokuphambuka kwamaphuzu maqondana nephethini elindelekile. [1]
Uma ukhumbula umthetho owodwa kuphela: hlukanisa isisekelo sakho (ihora/usuku/isifunda/izinga lesevisi) ngaphambi kokuthi umemezele ukuthi ingxenye yethrafikhi yakho “iyinqaba.” [1]
Ukuhlolwa: Ugibe Lwezehlakalo Ezingavamile 🧪
Ukutholwa kwe-Anomaly kuvame ukuba “inaliti enqwabeni yotshani,” okwenza ukuhlolwa kube yinto exakile:
-
Ama-ROC curve angabonakala elungile ngendlela ekhohlisayo lapho okuhle kungavamile.
-
Ukubukwa kokukhumbula kahle kuvame ukuba nolwazi oluthe xaxa ngezilungiselelo ezingalingani ngoba zigxila ekusebenzeni ekilasini elihle. [4]
-
Ngomsebenzi, udinga nesabelomali sesexwayiso : zingaki izexwayiso ngehora abantu abangazihlola ngempela ngaphandle kokuyeka ukucasuka? [4]
Ukuhlola emuva kumafasitela ajikelezayo kukusiza ukuthi ubambe imodi yokuhluleka yakudala: "isebenza kahle kakhulu... ekusakazweni kwenyanga edlule." [1]
Ukuhunyushwa kanye Nembangela Eyinhloko: Bonisa Umsebenzi Wakho 🪄
Ukuxwayisa ngaphandle kwencazelo kufana nokuthola ikhadi leposi eliyimfihlo. Kuwusizo, kodwa kuyacasula.
Amathuluzi okuhumusha angasiza ngokukhomba ukuthi yiziphi izici ezinegalelo elikhulu kumaphuzu angajwayelekile, noma ngokunikeza izincazelo zesitayela esithi “yini okungadingeka ishintshe ukuze lokhu kubukeke kujwayelekile?”. I-Interpretable Machine Learning iyisiqondiso esiqinile nesibalulekile sezindlela ezivamile (kufaka phakathi izici zesitayela se-SHAP) kanye nokulinganiselwa kwazo. [5]
Umgomo awukona nje ukududuza abathintekayo - ukushesha kokuhlolwa kanye nezigameko ezimbalwa eziphindaphindayo.
Ukuthunyelwa, Ukushayela, kanye Nezimo Zokuphendula 🚀
Amamodeli awahlali kumaslayidi. Ahlala emapayipini.
Indaba evamile "yenyanga yokuqala ekukhiqizweni": umtholi uvame ukufaka amafulegi, imisebenzi eminingi, kanye nedatha engekho... okusawusizo ngoba kukuphoqa ukuthi uhlukanise "izigameko zekhwalithi yedatha" "nezimo ezingavamile zebhizinisi."
Empeleni:
-
Gada ukuzulazula bese uqeqesha kabusha/ulinganise kabusha njengoba ukuziphatha kushintsha. [1]
-
Okokufaka amaphuzu elogi + inguqulo yemodeli ukuze ukwazi ukuphinda ubhale ukuthi kungani okuthile kufakwe ekhasini. [5]
-
Thwebula impendulo yabantu (izixwayiso eziwusizo uma kuqhathaniswa nezinomsindo) ukuze ulungise imingcele kanye nezingxenye ngokuhamba kwesikhathi. [4]
I-Engela Yokuphepha: I-IDS kanye ne-Behavioral Analytics 🛡️
Amaqembu ezokuphepha avame ukuhlanganisa imibono engavamile nokutholwa okusekelwe emithethweni: izisekelo "zokuziphatha okuvamile komsingathi," kanye nokusayina kanye nezinqubomgomo zamaphethini amabi aziwayo. I-SP 800-94 (Yokugcina) ye-NIST isalokhu iyindlela ekhulunywa kabanzi yokuthola ukungena kanye nokucabangela uhlelo lokuvimbela; iphawula nokuthi uhlaka luka-2012 oluthi "Rev. 1" aluzange lube ngolokugcina futhi kamuva lwasuswa emsebenzini. [3]
Ukuhumusha: sebenzisa i-ML lapho kusiza khona, kodwa ungayilahli imithetho eyisicefe - iyacasula ngoba iyasebenza.
Ithebula Lokuqhathanisa: Izindlela Ezidumile Ngokushesha 📊
| Ithuluzi / Indlela | Okuhle Kakhulu Kwaba | Kungani kusebenza (empeleni) |
|---|---|---|
| Izikolo ze-z eziqinile / eziguquliwe | Izilinganiso ezilula, izisekelo ezisheshayo | Ukudlula kokuqala okunamandla uma udinga "okwanele" kanye nezixwayiso zamanga ezimbalwa. [3] |
| Ihlathi Lokuzihlukanisa | Izici ezixubile zethebula | Ukuqaliswa okuzenzakalelayo okuqinile futhi kusetshenziswa kabanzi ekusebenzeni. [2] |
| I-SVM Yekilasi Elilodwa | Izindawo ezincane "ezivamile" | Ukutholwa kwezinto ezintsha ezisekelwe emngceleni; ukulungisa kubaluleke kakhulu. [2] |
| Isici Esingaphandle Kwendawo | Izindlela ezivamile eziningi | Umehluko phakathi kobuningi babantu uma kuqhathaniswa nomakhelwane ubamba izinto ezingavamile endaweni. [1] |
| Iphutha lokwakha kabusha (isb., isitayela se-autoencoder) | Amaphethini anobukhulu obuphezulu | Isitimela sisebenza ngendlela evamile; amaphutha amakhulu okwakha kabusha angabonisa ukuphambuka. [1] |
Ikhodi yokukhohlisa: qala ngezindlela eziyisisekelo eziqinile + indlela eyisicefe engagadiwe, bese wengeza ubunzima kuphela lapho ikhokhela khona irenti.
Incwadi Yokudlala Encane: Kusukela Ku-Zero Kuya Ku-Alerts 🧭
-
Chaza "okungajwayelekile" ngokusebenza (ukubambezeleka, ingozi yokukhwabanisa, ukuchithwa kwe-CPU, ingozi yempahla).
-
Qala ngesisekelo (izibalo eziqinile noma imikhawulo ehlukanisiwe). [3]
-
Khetha imodeli eyodwa engagadiwe njengephasi lokuqala (Isolation Forest / LOF / One-Class SVM). [2]
-
Beka imikhawulo ngesabelomali esiqaphile , bese uhlola ngokucabanga kwe-PR uma okuhle kungavamile. [4]
-
Engeza izincazelo + ukuloga ukuze isexwayiso ngasinye siphinde sikhiqizwe futhi silungiswe. [5]
-
Ukuhlolwa kwangemuva, ukuthunyelwa, ukufunda, ukulinganisa kabusha - ukuzulazula kuvamile. [1]
Ungakwenza lokhu ngeviki elilodwa… uma ucabanga ukuthi izitembu zakho zesikhathi azibanjwanga ndawonye nge-duct tape kanye nethemba. 😅
Amazwi Okugcina - Amade Kakhulu, Angizange Ngiwafunde🧾
I-AI ithola ukungalingani ngokufunda isithombe esisebenzayo "sokujwayelekile," ukulinganisa ukuphambuka, nokumaka ukuthi yini ewela umkhawulo. Izinhlelo ezinhle kakhulu aziphumeleli ngokuba zikhazimulayo, kodwa ngokulinganiswa : izisekelo ezihlukaniswe, isabelomali sezexwayiso, imiphumela ehunyushwayo, kanye nomjikelezo wempendulo oguqula ama-alamu anomsindo abe isignali ethembekile. [1]
Izinkomba
-
UPimentel et al. (2014) - Isibuyekezo sokutholwa kwezinto ezintsha (PDF, University of Oxford) funda kabanzi
-
Imibhalo ye-scikit-learn - Ukutholwa Okusha Nokungaphandle funda kabanzi
-
I-e-Handbook ye-NIST/SEMATECH - Ukutholwa Kwabangaphandle funda kabanzi kanye ne-NIST CSRC - SP 800-94 (Yokugcina): Umhlahlandlela Wezinhlelo Zokuthola Nokuvimbela Ukungena (IDPS) funda kabanzi
-
USaito noRehmsmeier (2015) - I-Precision-Recall Plot Inolwazi Olukhulu Kune-ROC Plot Lapho Kuhlolwa Ama-Binary Classifiers Kuma-Dataset Angalingani (PLOS ONE) funda kabanzi
-
I-Molnar - Ukufunda Komshini Okungahunyushwa (incwadi yewebhu) funda kabanzi