éåŠç¿ãšã¯ïŒåå ã»èŠæ¥µãæ¹ã»å®è·µç察çãŸã§åŸ¹åºè§£èª¬
ãAIã®ç²ŸåºŠãé«ãããã®ã«å®çšå Žé¢ã§ããŸãæ©èœããªãããã®åå ãéåŠç¿ïŒOverfittingïŒãããããŸããã
éåŠç¿ãšã¯ãæ©æ¢°åŠç¿ã¢ãã«ãèšç·ŽããŒã¿ã«å¯ŸããŠåŠã³ãããŠããŸããæ°ããããŒã¿ã«å¯Ÿå¿ã§ããªããªãçŸè±¡ã®ããšã§ãã
ãã£ãŒãã©ãŒãã³ã°ãLoRAãªã©é«åºŠãªAIææ³ãå°å ¥ããäŒæ¥ãå¢ããäžæ¹ã§ããã¢ãã«ãæ¬åœã«æ±çšæ§ãæã£ãŠããããã®èŠæ¥µãã¯æ¥åãšãªã£ãŠããŸãã
ãã®èšäºã§ã¯ãéåŠç¿ã®åºç€ç¥èããåå ãèŠåãæ¹ãåŠç¿æ²ç·ãã°ã©ãã䜿ã£ããã§ãã¯æ³ãæ£ååãããããã¢ãŠããšãã£ãå®è·µçãªå¯Ÿçæ¹æ³ãŸã§ããããããã解説ããŸãã
éçºçŸå Žã§çŽé¢ããã粟床ã¯é«ãã®ã«å®çšã§ããªãããšãã課é¡ãè§£æ¶ãããã³ããèŠã€ãããŸãã
ãŸããåŒç€Ÿã§ã¯ãããã³ãŒãŒãGAFAåºèº«ã®AIãšãã¹ããŒããAIå°å
¥ã«é¢ããç¡æçžè«ãæ¿ã£ãŠãããŸãã
ç¡æçžè«ã¯å
ç20瀟æ§éå®ã§ã貎瀟ã®AI掻çšäœå°åæã¬ããŒãããç¡åã§ãæäŸãããã£ã³ããŒã³ã宿œäžã§ãã
ãèå³ããæã¡ã®æ¹ã¯ã以äžã®ãªã³ã¯ãããé£çµ¡ãã ãã:
AIå°å
¥ã«é¢ããç¡æçžè«ã¯ãã¡ã
è³æè«æ±ã¯ãã¡ã
éåŠç¿ãšã¯ïŒ
éåŠç¿ãšã¯ãAIãåŠç¿ããŒã¿ãèŠããããŠãæ°ããããŒã¿ãžã®å¯Ÿå¿åïŒæ±åæ§èœïŒã倱ãç¶æ ãæããŸãã
æ¬ç« ã§ã¯ããéåŠç¿ã®æå³ããšãæªåŠç¿ãšã®éããã«ã€ããŠè©³ãã解説ããŸãããŸããèŠåãæ¹ã®åºæ¬ã泚æãã¹ããã€ã³ãã玹ä»ããŸãã
éåŠç¿ã®æå³
éåŠç¿ãšã¯ãAIãæ©æ¢°åŠç¿ã¢ãã«ãèšç·ŽããŒã¿ã«å¯ŸããŠéå°ã«é©å¿ããããŠããŸããæ±çšçãªäºæž¬ãã§ããªããªãçŸè±¡ãæããŸãã
ããã§ã®ãåŠç¿ãã¯ãäžè¬çãªç¥èç¿åŸã§ã¯ãªããäžããããããŒã¿ããæ³åããã¿ãŒã³ãèŠã€ãåºãã³ã³ãã¥ãŒã¿ãŒã®åŠçãæå³ããŸãã
éåŠç¿ã®ç¶æ ã«ãªããšãåŠç¿æã®ç²ŸåºŠã¯é«ããŠããæªç¥ã®ããŒã¿ã«å¯ŸããŠã¯æ£ç¢ºãªäºæž¬ãã§ãããå®çšæ§ã倧ããæãªãããŸãã
æªåŠç¿ãšã®éã
éåŠç¿ãšæªåŠç¿ã®éãã¯ãåŠç¿ã®ãããããããè¶³ããªããããšããç¹ã«ãããŸãã
éåŠç¿ã¯ãèšç·ŽããŒã¿ãèŠããããŠããŸããæ°ããããŒã¿ãžã®å¯Ÿå¿åïŒæ±åæ§èœïŒã倱ã£ãç¶æ ã§ãã
æªåŠç¿ã¯ãããããèšç·ŽããŒã¿ãååã«åŠç¿ã§ããŠããªããããèšç·ŽããŒã¿ã§ãæªç¥ããŒã¿ã§ã粟床ãäœãç¶æ ã§ãã
å ·äœäŸããçè§£ããéåŠç¿
éåŠç¿ãå ·äœäŸã§çè§£ããã«ã¯ã身è¿ãªåŠç¿è¡åãã€ã¡ãŒãžããã®ã广çã§ãã
ããšãã°ãåŠçãç¹å®ã®æš¡è©Šã ããç¹°ãè¿ãæèšããæ¬çªã§å°ãã§ãåºé¡åœ¢åŒãå€ãããšè§£ããªããªããŸãããã®ããã«ãèŠããããŠå¿çšãå¹ããªããç¶æ ããéåŠç¿ã®å žåçãªäŸã§ãã
AIã§ãåæ§ã«ãèšç·ŽããŒã¿ã«ã ãé床ã«é©å¿ããŠããŸããšãæ°ããããŒã¿ãžã®æ±çšæ§ã倱ããå®çšæ§ãäœäžããŸããããããæ©æ¢°åŠç¿ã«ãããéåŠç¿ã®åé¡ã§ãã
éåŠç¿ãåŒãèµ·ãã3ã€ã®åé¡ç¹
æ¬ç« ã§ã¯ããæ±åæ§èœãäœäžããããæ°èŠããŒã¿ã«å¯Ÿå¿ã§ããªããªããã誀ã£ãæææ±ºå®ãå¢ããããšãã£ããéåŠç¿ãåŒãèµ·ãã3ã€ã®åé¡ç¹ã«ã€ããŠè©³ãã解説ããŸãã
æ±åæ§èœãäœäžãã
éåŠç¿ãèµ·ãããšãèšç·ŽããŒã¿ã«ãã察å¿ã§ããªãç¶æ ã«ãªããæ±åæ§èœãäœäžããŸãã
æ±åæ§èœãšã¯ãæªç¥ã®ããŒã¿ã«å¯ŸããŠãé©åã«äºæž¬ã§ããåã®ããšã§ãã
ã¢ãã«ãåŠç¿ããŒã¿ã«äŸåãããããšãæ°ããå ¥åã«å¯ŸããŠæ£ç¢ºãªå€æãã§ãããå®çšæ§ã倧ããæãªãããŸãã
æ°èŠããŒã¿ã«å¯Ÿå¿ã§ããªããªã
æ°èŠããŒã¿ã«å¯Ÿå¿ã§ããªããªãã®ã¯ãéåŠç¿ã«ããã¢ãã«ãèšç·ŽããŒã¿ã«éå°ã«é©å¿ããŠããŸãããã§ãã
å®éã®æ¥åã§ã¯ãæ¥ã å€åããããŒã¿ãæªç¥ã®ãã¿ãŒã³ã«æè»ã«å¯Ÿå¿ããåãæ±ããããŸããã
éåŠç¿ç¶æ ã®AIã¯åŠç¿æç¹ã«ååšããªãäºäŸãžã®å¯Ÿå¿åãä¹ãããäºæž¬ãã¹ã粟床äœäžã«ã€ãªãããŸãã
誀ã£ãæææ±ºå®ãå¢ãã
誀ã£ãæææ±ºå®ãå¢ããã®ã¯ãéåŠç¿ã«ãã£ãŠAIã¢ãã«ãåã£ãããŒã¿ããã€ãºã«åŒã£åŒµãããããã§ãã
èšç·ŽããŒã¿ã«é床ã«é©åããã¢ãã«ã¯ãæ¬æ¥äžèŠãªç¹åŸŽãŸã§éèŠããŠããŸãã
å®åäžã®å€æãäºæž¬ã«èª€ãããããããªã¹ã¯ãé«ãŸããŸããç¹ã«èªååãããæ¥åããã»ã¹ã§ã¯ã
å°ããªå€æãã¹ã倧ããªæå€±ã«ã€ãªããå¯èœæ§ãããããæ³šæãå¿ èŠã§ãã
éåŠç¿ãèµ·ããäž»ãªåå 3éž
éåŠç¿ã¯ãã¢ãã«ã®èšèšãåŠç¿ã«çšããããŒã¿ã®ç¶æ³ã«ãã£ãŠåŒãèµ·ããããŸãã
æ¬ç« ã§ã¯ããåŠç¿ããŒã¿ã®äžè¶³ããã¢ãã«ãè€éãããããããŒã¿ã«åããããããšãã£ã éåŠç¿ã®ä»£è¡šçãªåå ãå ·äœçã«èŠãŠãããŸãããã
åŠç¿ããŒã¿ã®äžè¶³
åŠç¿ããŒã¿ã®äžè¶³ã¯ãã¢ãã«ãååãªãã¿ãŒã³ãåŠã¹ããèšç·ŽããŒã¿ã«é床ã«äŸåããŠããŸãåå ã§ãããã®çµæãæªç¥ã®ããŒã¿ã«å¯ŸããŠæè»ã«å¯Ÿå¿ã§ãããæ±åæ§èœãäœäžãããããªããŸãã éåŠç¿ãé²ãããã«ã¯ãéã ãã§ãªã倿§æ§ã®ããããŒã¿ãçšæããããšãéèŠã§ãã
ã¢ãã«ãè€éããã
ã¢ãã«ãè€éããããšãèšç·ŽããŒã¿ã«å«ãŸãããã€ãºãäŸå€ãŸã§åŠç¿ããŠããŸããããéåŠç¿ã®åå ã«ãªããŸãã
ãã©ã¡ãŒã¿ã®å€ã深局ã¢ãã«ã§ã¯ãå¿ èŠä»¥äžã«çްããªç¹åŸŽãæããããæ°ããããŒã¿ã«å¯ŸããŠæè»ã«å¯Ÿå¿ã§ããªããªããŸãã
ãã®çµæãæ¬çªç°å¢ã§ã®äºæž¬ç²ŸåºŠãäœäžãããªã¹ã¯ãé«ãŸããŸãã
ããŒã¿ã«åãããã
ããŒã¿ã«åãããããšãAIã¯ç¹å®ã®ãã¿ãŒã³ã«éå°é©å¿ãããã以å€ã®ç¶æ³ã«å¯Ÿå¿ã§ããªããªããããéåŠç¿ã®åå ã«ãªããŸãã
åã£ãããŒã¿æ§æã§ã¯ãçŸå®ã®å€æ§ãªã±ãŒã¹ãåŠç¿ã§ãããçµæãšããŠæ°èŠããŒã¿ã«åŒ±ãã¢ãã«ãã§ãããããŸãã
éåŠç¿ãé²ãããã®å¹æçãª6ã€ã®å¯Ÿç
éåŠç¿ã¯ãã¢ãã«ã®æ±åæ§èœãæãªã倧ããªãªã¹ã¯ã§ãã
æ¬ç« ã§ã¯ãããããã¢ãŠããå©çšããããæ£ååãå°å ¥ããããåŠç¿ããŒã¿ãå¢ãããããã€ããŒãã©ã¡ãŒã¿ïŒåŠç¿çïŒã調æŽããããã¢ã³ãµã³ãã«ã¢ãã«ã掻çšããããªã©6ã€ã®å¯Ÿçã玹ä»ããŸãã
ããããã¢ãŠããå©çšãã
ç¹å®ã®ããŒããžã®äŸåãé¿ããéåŠç¿ãé²ãã«ã¯ããããã¢ãŠããå©çšãã察çã广çã§ãã
ããããã¢ãŠãã¯ãåŠç¿äžã«ã©ã³ãã ã§äžéšã®ããŒããç¡å¹åããä»çµã¿ã§ãã¢ãã«ã®éå°ãªé©åãé²ããŸããããã«ãããããæ±çšæ§ã®é«ãã¢ãã«ãè²ãŠãããšãã§ããŸãã
æ£ååãå°å ¥ãã
ã¢ãã«ã®éã¿ã極端ã«åãã®ãæããéåŠç¿ãé²ãã«ã¯æ£ååãå°å ¥ãã察çã广çã§ãã
æ£ååãšã¯ãéã¿ã«å¯ŸããŠå¶çŽããããããšã§ãã¢ãã«ãèšç·ŽããŒã¿ã«é床ã«é©åããããªãããã«ããææ³ã§ãã
L1æ£ååïŒLassoïŒãL2æ£ååïŒRidgeïŒãªã©ã代衚çã§ãããã«ããã·ã³ãã«ã§æ±çšæ§ã®é«ãã¢ãã«æ§ç¯ãå¯èœã«ãªããŸãã
åŠç¿ããŒã¿ãå¢ãã
ããŒã¿ãå°ãªãç¶æ ã§ã¯ã¢ãã«ãéããããã¿ãŒã³ãèšæ¶ããŠããŸããããæ°ããããŒã¿ãžã®å¯Ÿå¿åïŒæ±åæ§èœïŒãé«ããã«ã¯ãåŠç¿ããŒã¿ãå¢ãã察çã广çã§ãã
倿§ãªããŒã¿ãåãå ¥ããããšã§ãã¢ãã«ãããæ±çšçãªç¹åŸŽãåŠç¿ããç¹å®ã®ããŒã¿ã«åãããšãªãæè»ãªäºæž¬ãå¯èœã«ãªããŸãã
ãã€ããŒãã©ã¡ãŒã¿ïŒåŠç¿çïŒã調æŽãã
åŠç¿çãé«ããããšåæãããäœããããšç¹å®ããŒã¿ã«åãããããªãããããã©ã³ã¹ãèŠæ¥µããã«ã¯ããã€ããŒãã©ã¡ãŒã¿ïŒåŠç¿çïŒã調æŽãã察çã广çã§ãã
åŠç¿çã¯ã¢ãã«ã®åŠç¿é床ã粟床ã«å€§ãã圱é¿ãããããéåŠç¿ãé²ãã«ã¯æ®µéçã«èª¿æŽããæé©ãªå€ãèŠã€ããããšãéèŠã§ããç¹ã«è€éãªã¢ãã«ã»ã©ãç¹çްãªãã¥ãŒãã³ã°ãæ±ããããŸãã
ã¢ã³ãµã³ãã«ã¢ãã«ã掻çšãã
è€æ°ã®ã¢ãã«ã®äºæž¬çµæãçµ±åããããšã§ãäºæž¬ç²ŸåºŠã®å®å®ãšæ±åæ§èœã®åäžãå³ãã«ã¯ãã¢ã³ãµã³ãã«ã¢ãã«ã掻çšãã察çã广çã§ãã
ç°ãªãç¹åŸŽãæã€è€æ°ã®ã¢ãã«ãçµã¿åãããããšã§ãäžã€ã®ã¢ãã«ã§ã¯æŸããããªããã¿ãŒã³ãè£å®ã§ããéåŠç¿ã®ãªã¹ã¯ãæãã€ã€å šäœã®æ§èœãåºäžãã§ããŸãã
LoRAã䜿çšãã
ãã©ã¡ãŒã¿æŽæ°ãéå®ããããšã§åŠç¿è² è·ãæããç¹å®ã¿ã¹ã¯ãžã®éå°é©åãé²ãã«ã¯LoRAã䜿çšãã察çã广çã§ãã
LoRAïŒLow-Rank AdaptationïŒã¯ãå€§èŠæš¡ã¢ãã«ã®äžéšãã©ã¡ãŒã¿ã«çµã£ãŠå¹ççã«åŠç¿ãè¡ãæè¡ã§ããã¢ãã«å šäœãååŠç¿ããå¿ èŠããªããããéåŠç¿ã®ãªã¹ã¯ãæãã€ã€ãã¿ã¹ã¯ããšã®åŸ®èª¿æŽãå¯èœã«ãªããŸããç¹ã«åŠç¿ããŒã¿ãéãããå Žé¢ã§å¹æãçºæ®ããŸãã
åŠç¿ãèŠåãã4ã€ã®æ¹æ³
éåŠç¿ãæªåŠç¿ã®ç¶æ ãèŠæ¥µããããšã¯ãã¢ãã«ã®ä¿¡é Œæ§ãä¿ã€ããã«æ¬ ãããŸããã
æ¬ç« ã§ã¯ããåŠç¿æ²ç·ã®ã°ã©ãã§ç¢ºèªããããèšç·ŽããŒã¿ã»æ€èšŒããŒã¿ã§ç²ŸåºŠå·®ã確èªããããããŒã«ãã¢ãŠãæ³ã§æ€èšŒãããã亀差æ€èšŒæ³ã䜿ã£ãŠç²ŸåºŠã確èªãããç¶æ ãèŠåãã4ã€ã®æ¹æ³ã«ã€ããŠè§£èª¬ããŸãã
åŠç¿æ²ç·ã®ã°ã©ãã§ç¢ºèªãã
èšç·Žãšæ€èšŒã®èª€å·®ã®å·®ããéåŠç¿ã»æªåŠç¿ãèŠåããã®ããåŠç¿æ²ç·ã®ã°ã©ãã§ç¢ºèªããæ¹æ³ã§ãã
èšç·ŽããŒã¿ã§ã¯èª€å·®ãäžãã£ãŠããã®ã«æ€èšŒããŒã¿ã§ã¯èª€å·®ãé«ããŸãŸã®å ŽåãéåŠç¿ã®å¯èœæ§ãé«ãŸããŸãã
Â äžæ¹ãäž¡æ¹ã®èª€å·®ã髿¢ãŸãããŠããå Žåã¯æªåŠç¿ã瀺ããŠããŸãã ãã®ã°ã©ãã確èªããããšã§ãåŠç¿ç¶æ³ãèŠèŠçã«ææ¡ã§ããæ©æã®ä¿®æ£ã«ã€ãªãããããªããŸãã
èšç·ŽããŒã¿ã»æ€èšŒããŒã¿ã§ç²ŸåºŠå·®ã確èªãã
ã¢ãã«ãèšç·ŽããŒã¿ã§ã¯é«ãããã©ãŒãã³ã¹ã瀺ãäžæ¹ãæ€èšŒããŒã¿ã§æ¥µç«¯ã«ç²ŸåºŠãèœã¡ãå Žåã¯éåŠç¿ã®å¯èœæ§ãããããããé²ãã«ã¯èšç·ŽããŒã¿ãšæ€èšŒããŒã¿ã®ç²ŸåºŠå·®ã宿çã«ç¢ºèªããæ¹æ³ã广çã§ãã
粟床差ã倧ããå Žåã¯ãã¢ãã«ãèšç·ŽããŒã¿ã«éå°é©åããŠãããµã€ã³ãšæãããã©ã¡ãŒã¿èª¿æŽãããŒã¿éã®èŠçŽããªã©æ©æå¯ŸçãéèŠã§ãã
ããŒã«ãã¢ãŠãæ³ã§æ€èšŒãã
ããŒã¿ãäžåºŠã ãèšç·Žçšã»æ€èšŒçšã«åããŠæ¯èŒããæ€èšŒããŒã¿ã§èãã粟床ãäžããå Žåã¯ãããŒã«ãã¢ãŠãæ³ã§æ€èšŒããæ¹æ³ã广çã§ãã
ã¢ãã«ã®éåŠç¿åŸåãæè»œã«ç¢ºèªã§ããåææ®µéã®è©äŸ¡ããªãœãŒã¹ã®éããããããžã§ã¯ãã§ã掻çšããããã®ãç¹åŸŽã§ããã
亀差æ€èšŒæ³ã䜿ã£ãŠç²ŸåºŠã確èªãã
ããŒã¿ã»ãããè€æ°ã«åããåŠç¿ãšæ€èšŒãç¹°ãè¿ãããšã§è©äŸ¡ã®ã°ãã€ããé²ãã«ã¯ã亀差æ€èšŒæ³ã䜿ã£ãŠç²ŸåºŠã確èªããæ¹æ³ã广çã§ãã
亀差æ€èšŒæ³ïŒã¯ãã¹ããªããŒã·ã§ã³ïŒã¯ãããŒã¿ãè€æ°ã®ã°ã«ãŒãã«åå²ããŠé çªã«åŠç¿ãšæ€èšŒãè¡ãããšã§ãåãã®å°ãªãå®å®ãã粟床è©äŸ¡ãå¯èœã«ãªããŸããç¹ã«ããŒã¿éãéãããŠããå Žåã§ããä¿¡é Œæ§ã®é«ãã¢ãã«è©äŸ¡ãè¡ããã®ãç¹åŸŽã§ãã
ãŸãšã
éåŠç¿ãšã¯ãAIãåŠç¿ããŒã¿ãèŠããããŠæ°ããããŒã¿ã«å¯Ÿå¿ã§ããªããªãç¶æ ãæããŸãããã®èšäºã§ã¯ãéåŠç¿ã®æå³ãåå ãåŠç¿æ²ç·ãã°ã©ãã«ããèŠåãæ¹ãããã«ã¯ããããã¢ãŠããæ£ååãLoRAãªã©ã®å®è·µçãªå¯ŸçãŸã§ãäŒæ¥ã®AIå°å ¥æã«æ¬ ãããªãç¥èããããããã解説ããŸããã
AIã®å°å ¥ãã¢ãã«éçºã§ããªã粟床ãäžããã®ããã«æ©ãã§ããæ¹ã¯ããã®èšäºã§èª²é¡ã®æ¬è³ªãšãã®è§£æ±ºçãèŠã€ãããŸãã