Lapho abantu abaningi bezwa "ubuhlakani bokwenziwa," bacabanga ngamanethi e-neural, ama-algorithms amahle, noma mhlawumbe lawo marobhothi angavamile kancane. Okungavamile ukukhulunywa ngawo kusengaphambili yilokhu: I-AI idla isitoreji cishe ngendlela edla kakhulu njengoba ibala. Futhi akuyona nje noma yisiphi isitoreji sezinto zokugcina esihlala buthule ngemuva, senza umsebenzi ongemuhle kodwa obaluleke kakhulu wokunikeza amamodeli idatha ayidingayo.
Ake sichaze ukuthi yini eyenza ukugcina izinto kubaluleke kangaka kwi-AI, ukuthi kuhluke kanjani “ekuqapheni okudala” kwezinhlelo zokugcina izinto, nokuthi kungani kuphelela kungenye yezindlela ezibalulekile zokukhulisa nokusebenza.
Izihloko ongase uthande ukuzifunda ngemva kwalesi:
🔗 Yimaphi ubuchwepheshe okumele bube khona ukuze kusetshenziswe i-AI ekhiqizayo enkulu ebhizinisini
Amabhizinisi adinga ubuchwepheshe obubalulekile ukuze andise i-AI ekhiqizayo ngempumelelo.
🔗 Ukuphathwa kwedatha yamathuluzi e-AI okufanele uwabheke
Izindlela ezinhle kakhulu zokuphatha idatha ukuze kuthuthukiswe ukusebenza kwe-AI.
🔗 Imiphumela yobuhlakani bokwenziwa kwesu lebhizinisi
Indlela i-AI ethinta ngayo amasu ebhizinisi kanye nokwenza izinqumo zesikhathi eside.
Yini Eyenza Isitoreji Sezinto Sibe Uphawu Lwe-AI? 🌟
Umqondo omkhulu: ukugcina izinto akukhathazi ngamafolda noma izakhiwo zamabhulokhi aqinile. Kuhlukanisa idatha ibe “izinto,” ngayinye ene-tag ene-metadata. Leyo metadata ingaba izinto zezinga lesistimu (usayizi, izitembu zesikhathi, isigaba sesitoreji) kanye namathegi achazwe ngumsebenzisi: inani [1]. Cabanga ngayo njengefayela ngalinye eliphethe inqwaba yamanothi anamathelayo akutshela kahle ukuthi iyini, yadalwa kanjani, nokuthi ingena kuphi emgudwini wakho.
Kumaqembu e-AI, lokho kuguquguquka kuyashintsha umdlalo:
-
Isikali esingenamakhanda abuhlungu - Amachibi edatha anwebeka abe ama-petabytes, futhi izitolo zezinto ziwaphatha kalula. Aklanyelwe ukukhula okungenamkhawulo kanye nokuqina kwe-multi-AZ (i-Amazon S3 iqhosha "ngama-nines angu-11" kanye nokuphindaphindwa kwezindawo eziwela ngokuzenzakalelayo) [2].
-
Ukuceba kwe-metadata - Ukusesha okusheshayo, izihlungi ezihlanzekile, kanye nezindlela ezihlakaniphile njengoba umongo uhamba nento ngayinye [1].
-
nge-Cloud - Idatha ingena nge-HTTP(S), okusho ukuthi ungadonsa ngendlela efanayo futhi uqhubeke nokuqeqeshwa okusatshalaliswayo.
-
Ukuqina okubangelwa - Uma uziqeqesha izinsuku eziningi, awukwazi ukufaka engcupheni i-epoch 12 yokubulala i-shard eyonakele. Indawo yokugcina izinto iyakugwema lokho ngokuklama [2].
Empeleni isikhwama semali esingenasisekelo: mhlawumbe singcolile ngaphakathi, kodwa konke kusatholakala uma usithatha.
Ithebula Lokuqhathanisa Okusheshayo Lesitoreji Sezinto Ze-AI 🗂️
| Ithuluzi / Isevisi | Okuhle Kakhulu (Izithameli) | Ibanga Lentengo | Kungani Kusebenza (Amanothi Aseceleni) |
|---|---|---|---|
| I-Amazon S3 | Amaqembu e-Amabhizinisi + Amafu-first | Khokha njengoba uhamba | Ihlala isikhathi eside kakhulu, iyakwazi ukumelana nesifunda [2] |
| Isitoreji Samafu se-Google | Ososayensi bedatha kanye nabathuthukisi be-ML | Izinga eliguquguqukayo | Ukuhlanganiswa okuqinile kwe-ML, okusekelwe efwini ngokuphelele |
| Isitoreji se-Azure Blob | Izitolo ezithengisa izinto eziningi ze-Microsoft | Izigaba (ezishisayo/ezibandayo) | Akunazihibe ngedatha ye-Azure + amathuluzi e-ML |
| I-MinIO | Ukusetha komthombo ovulekile / kwe-DIY | Mahhala/ozisingathayo | Iyahambisana ne-S3, ilula, isetshenziswa noma kuphi 🚀 |
| Ifu Elishisayo LaseWasabi | Izinhlangano ezikhathalela izindleko | Intengo ephansi ephansi $ | Akukho zindleko zokuphuma noma zesicelo se-API (ngenqubomgomo ngayinye) [3] |
| Isitoreji Sezinto Zamafu se-IBM | Amabhizinisi amakhulu | Kuyahlukahluka | Isitaki esivuthiwe esinezinketho zokuphepha eziqinile zebhizinisi |
Hlola njalo amanani ngendlela osebenzisa ngayo izinto zangempela—ikakhulukazi ukuphuma, ivolumu yesicelo, kanye nengxube yekilasi lokugcina izinto.
Kungani Ukuqeqeshwa Kwe-AI Kuthanda Isitoreji Sezinto 🧠
Ukuqeqeshwa akusikho “inqwaba yamafayela.” Kuyizigidi ngezigidi zamarekhodi aphulwe ngokulingana. Izinhlelo zamafayela ezilandelanayo zibopha ngaphansi kwesilinganiso esinzima semali. Isitoreji sezinto siyakugwema lokho ngezikhala zamagama ezisicaba nama-API ahlanzekile. Yonke into inokhiye oyingqayizivele; abasebenzi bayashanela futhi balande ngokulingana. Amasethi edatha aqoshiwe + i-I/O ehambisanayo = ama-GPU ahlala ematasa esikhundleni sokulinda.
Ithiphu elivela emiseleni: gcina ama-shards ashisayo eduze kweqoqo le-compute (isifunda noma indawo efanayo), bese ugcina ngokuqapha ku-SSD. Uma udinga ama-feed aseduze ngqo kuma-GPU, i-NVIDIA GPUDirect Storage ikufanele ukubhekisiswa - inciphisa ama-buffer e-CPU bounce, inciphise ukubambezeleka, futhi inyuse i-bandwidth iqonde kuma-accelerator [4].
I-Metadata: Amandla Aphezulu Angalinganiswa Kahle 🪄
Nakhu lapho isitoreji sezinto sikhanya khona ngezindlela ezingacacile. Ekulayisheni, unganamathisela imethadatha yangokwezifiso (njenge -x-amz-meta-… ye-S3). Isethi yedatha yombono, isibonelo, ingamaka izithombe ngokuthi lighting=low noma blur=high. Lokho kuvumela amapayipi ukuthi ahlunge, alinganisele, noma ahlukanise ngaphandle kokuskena kabusha amafayela aluhlaza [1].
Bese kuba khona ukuhumusha. Izitolo eziningi zezinto zigcina izinguqulo eziningi zezinto eceleni - zilungele ukuhlolwa okuphindaphindwayo noma izinqubomgomo zokuphatha ezidinga ukuphindiselwa emuva [5].
Into vs Ibhulokhi vs Isitoreji Sefayela ⚔️
-
Isitoreji Sebhulokhi: Kuhle kakhulu kuma-database okuthengiselana - okusheshayo nokunembile - kodwa kubiza kakhulu kudatha engahlelekile yesikali se-petabyte.
-
Isitoreji Sefayela: Sijwayelekile, sinobungani ne-POSIX, kodwa ama-directory aminyaniswe yimithwalo eminingi efanayo.
-
Isitoreji Sezinto: Sakhiwe kusukela phansi ukuze sifinyeleleke ngezinga, ukuhambisana, kanye nokufinyelela okuqhutshwa yi-metadata [1].
Uma ufuna isingathekiso esingaqondakali: isitoreji samabhulokhi siyikhabhinethi yokufaka amafayela, isitoreji samafayela siyifolda yedeskithophu, kanti isitoreji sezinto... siyimgodi ongenamkhawulo onezinothi ezinamathelayo ezenza kube lula ukuyisebenzisa.
Ukugeleza Komsebenzi we-AI we-Hybrid 🔀
Akuhlali kungamafu kuphela. Ingxube evamile ibukeka kanje:
-
Isitoreji sezinto esise-prem (i-MinIO, i-Dell ECS) sedatha ebucayi noma elawulwayo.
-
Isitoreji sezinto zefu semisebenzi eminingi, izivivinyo, noma ukubambisana.
Lokhu kulinganisela kufinyelela ezindlekweni, ekuthobeleni imithetho, kanye nasekusebenzeni kalula. Ngibone amaqembu elahla ama-terabyte ngobusuku obubodwa ebhakedeni le-S3 ukuze akhanyise i-GPU cluster yesikhashana - bese ewushaya wonke uma i-sprint isiphelile. Ukuze uthole isabelomali esiqinile, imodeli ye-Wasabi ye-flat-rate/no-egress [3] yenza impilo ibe lula ukuyibikezela.
Ingxenye Okungekho Ozigqaja Ngayo 😅
Ukuhlola amaqiniso: akulona iphutha.
-
Ukubambezeleka - Beka ukubala kanye nesitoreji kude kakhulu bese ama-GPU akho ekhasa. I-GDS iyasiza, kodwa ukwakheka kusabalulekile [4].
-
Izimanga zezindleko - Izindleko zokuphuma kanye nezicelo ze-API zingena kubantu. Abanye abahlinzeki bayawayeka (i-Wasabi iyawayeka; abanye abawenzi) [3].
-
Isiphithiphithi se-metadata esikalini - Ubani ochaza "iqiniso" kumathegi nasezinguqulweni? Uzodinga izinkontileka, izinqubomgomo, kanye nemisipha ethile yokuphatha [5].
Ukugcina izinto kuyingqalasizinda yokufakelwa kwamapayipi: kubalulekile, kodwa akukuhle.
Lapho Kuya Khona 🚀
-
Isitoreji esihlakaniphile, esiqaphela i-AI esibeka amathegi ngokuzenzakalelayo futhi siveze idatha ngezingqimba zombuzo ezifana ne-SQL [1].
-
Ukuhlanganiswa kwehadiwe okuseduze (izindlela ze-DMA, ukulayishwa kwe-NIC) ukuze ama-GPU angadli i-I/O [4].
-
Intengo esobala, ebikezelwayo (amamodeli alula, izimali zokuphuma ezingakhokhiwe) [3].
Abantu bakhuluma ngokubala njengekusasa le-AI. Kodwa empeleni? Inkinga imayelana nokufaka idatha kumamodeli ngokushesha ngaphandle kokusebenzisa isabelomali esiningi. Yingakho indima yokugcina izinto ikhula kuphela.
Isiphetho 📝
Indawo yokugcina izinto ayikhazimuli, kodwa iyisisekelo. Ngaphandle kwendawo yokugcina izinto ekwaziyo ukukhulisa, eqaphela imethadatha, neqinile, ukuqeqesha amamodeli amakhulu kuzwakala sengathi kugijima umjaho omkhulu ogqoke izimbadada.
Ngakho-ke yebo-ama-GPU abalulekile, uhlaka lubalulekile. Kodwa uma uzimisele nge-AI, ungashayi indiva ukuthi idatha yakho ihlala kuphi. Amathuba akhona ukuthi, isitoreji sezinto sesivele sibambe yonke imisebenzi buthule.
Izinkomba
[1] I-AWS S3 – I-metadata yezinto - isistimu kanye ne-metadata eyenziwe ngokwezifiso
https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html
[2] I-AWS S3 – Amakilasi okugcina - ukuqina (“amaminithi angu-11”) + ukuqina
https://aws.amazon.com/s3/storage-classes/
[3] I-Wasabi Hot Cloud – Intengo - izinga elilinganiselwe, akukho zindleko zokuphuma/ze-API
https://wasabi.com/pricing
[4] Isitoreji Esiqondile se-NVIDIA GPU – Amadokhumenti - Izindlela ze-DMA eziya kuma-GPU
https://docs.nvidia.com/gpudirect-storage/
[5] I-AWS S3 – Ukwenziwa Kwezinguqulo - izinguqulo eziningi zokuphatha/ukuphinda kukhiqizwe
https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html