{"id":479385,"date":"2023-08-09T10:35:54","date_gmt":"2023-08-09T10:35:54","guid":{"rendered":""},"modified":"2023-09-05T11:18:41","modified_gmt":"2023-09-05T11:18:41","slug":"transformers-in-natural-language-processing","status":"publish","type":"wiki","link":"https:\/\/oneproxy.pro\/kr\/wiki\/transformers-in-natural-language-processing\/","title":{"rendered":"\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc758 \ubcc0\ud658\uae30"},"content":{"rendered":"<p>Transformers\ub294 \uc790\uc5f0\uc5b4 \ucc98\ub9ac(NLP) \ubd84\uc57c\uc5d0\uc11c \uc0ac\uc6a9\ub418\ub294 \ub525\ub7ec\ub2dd \ubaa8\ub378 \ud074\ub798\uc2a4\uc785\ub2c8\ub2e4. \uadf8\ub4e4\uc740 \uae30\uacc4 \ubc88\uc5ed, \ud14d\uc2a4\ud2b8 \uc0dd\uc131, \uac10\uc815 \ubd84\uc11d \ub4f1\uacfc \uac19\uc740 \ub2e4\uc591\ud55c \uc5b8\uc5b4 \uc791\uc5c5\uc5d0\uc11c \uc0c8\ub85c\uc6b4 \ud45c\uc900\uc744 \uc124\uc815\ud588\uc2b5\ub2c8\ub2e4. Transformer\uc758 \uad6c\uc870\ub294 \uc2dc\ud000\uc2a4\uc758 \ubcd1\ub82c \ucc98\ub9ac\ub97c \uac00\ub2a5\ud558\uac8c \ud558\uc5ec \ub192\uc740 \ud6a8\uc728\uc131\uacfc \ud655\uc7a5\uc131\uc758 \uc7a5\uc810\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc5d0\uc11c \ud2b8\ub79c\uc2a4\ud3ec\uba38\uc758 \uc720\ub798\uc640 \ucd5c\ucd08 \uc5b8\uae09\uc758 \uc5ed\uc0ac<\/h2>\n<p>Transformer \uc544\ud0a4\ud14d\ucc98\ub294 2017\ub144 Ashish Vaswani\uc640 \uadf8\uc758 \ub3d9\ub8cc\ub4e4\uc774 &quot;Attention is All You Need&quot;\ub77c\ub294 \uc81c\ubaa9\uc758 \ub17c\ubb38\uc5d0\uc11c \ucc98\uc74c \uc18c\uac1c\ud588\uc2b5\ub2c8\ub2e4. \uc774 \ud68d\uae30\uc801\uc778 \ubaa8\ub378\uc740 \ubaa8\ub378\uc774 \uc785\ub825\uc758 \uc77c\ubd80\uc5d0 \uc120\ud0dd\uc801\uc73c\ub85c \uc9d1\uc911\ud560 \uc218 \uc788\ub3c4\ub85d \ud558\ub294 &quot;attention&quot;\uc774\ub77c\ub294 \uc0c8\ub85c\uc6b4 \uba54\ucee4\ub2c8\uc998\uc744 \uc81c\uc2dc\ud588\uc2b5\ub2c8\ub2e4. \ucd9c\ub825\uc744 \uc0dd\uc131\ud569\ub2c8\ub2e4. \uc774 \ub17c\ubb38\uc740 \uc804\ud1b5\uc801\uc778 \uc21c\ud658 \uc2e0\uacbd\ub9dd(RNN)\uacfc \uc7a5\ub2e8\uae30 \uae30\uc5b5(LSTM) \ub124\ud2b8\uc6cc\ud06c\uc5d0\uc11c \ubc97\uc5b4\ub098 NLP\uc758 \uc0c8\ub85c\uc6b4 \uc2dc\ub300\ub97c \uc5f4\uc5c8\uc2b5\ub2c8\ub2e4.<\/p>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc758 \ubcc0\ud658\uae30\uc5d0 \ub300\ud55c \uc790\uc138\ud55c \uc815\ubcf4<\/h2>\n<p>Transformer\ub294 \ubcd1\ub82c \ucc98\ub9ac \ubc0f \ud14d\uc2a4\ud2b8\uc758 \uc7a5\uac70\ub9ac \uc885\uc18d\uc131\uc744 \ucc98\ub9ac\ud558\ub294 \ud6a8\uc728\uc131\uc73c\ub85c \uc778\ud574 \ud604\ub300 NLP\uc758 \uae30\ubc18\uc774 \ub418\uc5c8\uc2b5\ub2c8\ub2e4. \uc778\ucf54\ub354\uc640 \ub514\ucf54\ub354\ub85c \uad6c\uc131\ub418\uc5b4 \uc788\uc73c\uba70 \uac01\uac01\uc5d0\ub294 \uc5ec\ub7ec \ub808\uc774\uc5b4\uc758 self-attention \uba54\ucee4\ub2c8\uc998\uc774 \ud3ec\ud568\ub418\uc5b4 \uc788\uc5b4 \ubb38\uc7a5 \ub0b4 \uc704\uce58\uc5d0 \uad00\uacc4\uc5c6\uc774 \ub2e8\uc5b4 \uac04\uc758 \uad00\uacc4\ub97c \ucea1\ucc98\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<h3>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc5d0\uc11c \ubcc0\ud658\uae30\uc758 \uc8fc\uc81c \ud655\uc7a5<\/h3>\n<ul>\n<li><strong>\uc790\uae30 \uc8fc\uc758 \uba54\ucee4\ub2c8\uc998<\/strong>: \ubaa8\ub378\uc774 \uc785\ub825\uc758 \ub2e4\uc591\ud55c \ubd80\ubd84\uc5d0 \ub2e4\ub974\uac8c \uac00\uc911\uce58\ub97c \ubd80\uc5ec\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<li><strong>\uc704\uce58 \uc778\ucf54\ub529<\/strong>: \uc2dc\ud000\uc2a4 \ub0b4\uc758 \ub2e8\uc5b4 \uc704\uce58\ub97c \uc778\ucf54\ub529\ud558\uc5ec \ub2e8\uc5b4 \uc21c\uc11c\uc5d0 \ub300\ud55c \uc815\ubcf4\ub97c \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>\ud655\uc7a5\uc131<\/strong>: \ub300\uaddc\ubaa8 \ub370\uc774\ud130 \uc138\ud2b8\uc640 \uae34 \uc2dc\ud000\uc2a4\ub97c \ud6a8\uc728\uc801\uc73c\ub85c \ucc98\ub9ac\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>\uc751\uc6a9<\/strong>: \ud14d\uc2a4\ud2b8 \uc694\uc57d, \ubc88\uc5ed, \uc9c8\ubb38 \ub2f5\ubcc0 \ub4f1 \ub2e4\uc591\ud55c NLP \uc791\uc5c5\uc5d0 \uc0ac\uc6a9\ub429\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc5d0\uc11c \ubcc0\ud658\uae30\uc758 \ub0b4\ubd80 \uad6c\uc870<\/h2>\n<p>Transformer\ub294 \uc778\ucf54\ub354\uc640 \ub514\ucf54\ub354\ub85c \uad6c\uc131\ub418\uba70 \ub458 \ub2e4 \uc5ec\ub7ec \ub808\uc774\uc5b4\ub85c \uad6c\uc131\ub429\ub2c8\ub2e4.<\/p>\n<ul>\n<li><strong>\uc778\ucf54\ub354<\/strong>: self-attention \ub808\uc774\uc5b4, \ud53c\ub4dc\ud3ec\uc6cc\ub4dc \uc2e0\uacbd\ub9dd \ubc0f \uc815\uaddc\ud654\ub85c \uad6c\uc131\ub429\ub2c8\ub2e4.<\/li>\n<li><strong>\ub514\ucf54\ub354<\/strong>: \uc778\ucf54\ub354\uc640 \uc720\uc0ac\ud558\uc9c0\ub9cc \uc778\ucf54\ub354\uc758 \ucd9c\ub825\uc5d0 \uc8fc\uc758\ub97c \uae30\uc6b8\uc774\uae30 \uc704\ud55c \ucd94\uac00 \uad50\ucc28 \uc8fc\uc758 \ub808\uc774\uc5b4\uac00 \ud3ec\ud568\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc5d0\uc11c \ubcc0\ud658\uae30\uc758 \uc8fc\uc694 \ud2b9\uc9d5 \ubd84\uc11d<\/h2>\n<p>Transformer\ub294 \ud6a8\uc728\uc131, \ubcd1\ub82c \ucc98\ub9ac, \uc801\uc751\uc131 \ubc0f \ud574\uc11d \uac00\ub2a5\uc131\uc73c\ub85c \uc798 \uc54c\ub824\uc838 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<ul>\n<li><strong>\ub2a5\ub960<\/strong>: \ubcd1\ub82c \ucc98\ub9ac\ub85c \uc778\ud574 \uae30\uc874 RNN\ubcf4\ub2e4 \ud6a8\uc728\uc801\uc785\ub2c8\ub2e4.<\/li>\n<li><strong>\ud574\uc11d \uac00\ub2a5\uc131<\/strong>: \uc8fc\uc758 \uba54\ucee4\ub2c8\uc998\uc740 \ubaa8\ub378\uc774 \uc2dc\ud000\uc2a4\ub97c \ucc98\ub9ac\ud558\ub294 \ubc29\ubc95\uc5d0 \ub300\ud55c \ud1b5\ucc30\ub825\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>\uc801\uc751\uc131<\/strong>: \ub2e4\uc591\ud55c NLP \uc791\uc5c5\uc5d0 \ub9de\uac8c \ubbf8\uc138 \uc870\uc815\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc758 \ubcc0\ud658\uae30 \uc720\ud615<\/h2>\n<table>\n<thead>\n<tr>\n<th>\ubaa8\ub378<\/th>\n<th>\uc124\uba85<\/th>\n<th>\uc0ac\uc6a9 \uc0ac\ub840<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>\ubc84\ud2b8<\/td>\n<td>\ubcc0\ud658\uae30\uc758 \uc591\ubc29\ud5a5 \uc778\ucf54\ub354 \ud45c\ud604<\/td>\n<td>\uc0ac\uc804 \ud6c8\ub828<\/td>\n<\/tr>\n<tr>\n<td>GPT<\/td>\n<td>\uc0dd\uc131\uc801 \uc0ac\uc804 \ud6c8\ub828\ub41c \ubcc0\ud658\uae30<\/td>\n<td>\ud14d\uc2a4\ud2b8 \uc0dd\uc131<\/td>\n<\/tr>\n<tr>\n<td>T5<\/td>\n<td>\ud14d\uc2a4\ud2b8-\ud14d\uc2a4\ud2b8 \uc804\uc1a1 \ubcc0\ud658\uae30<\/td>\n<td>\uba40\ud2f0\ud0dc\uc2a4\ud0b9<\/td>\n<\/tr>\n<tr>\n<td>\ub514\uc2a4\ud2f8\ubc84\ud2b8<\/td>\n<td>BERT\uc758 \uc99d\ub958\ub41c \ubc84\uc804<\/td>\n<td>\uc790\uc6d0 \ud6a8\uc728\uc801\uc778 \ubaa8\ub378\ub9c1<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac, \ubb38\uc81c \ubc0f \ud574\uacb0 \ubc29\ubc95\uc5d0\uc11c Transformer\ub97c \uc0ac\uc6a9\ud558\ub294 \ubc29\ubc95<\/h2>\n<p>Transformer\ub294 \ub2e4\uc591\ud55c NLP \uc560\ud50c\ub9ac\ucf00\uc774\uc158\uc5d0 \uc0ac\uc6a9\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4. \ubb38\uc81c\uc5d0\ub294 \uacc4\uc0b0 \ub9ac\uc18c\uc2a4, \ubcf5\uc7a1\uc131 \ubc0f \ud574\uc11d \uac00\ub2a5\uc131\uc774 \ud3ec\ud568\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<ul>\n<li><strong>\uc0ac\uc6a9<\/strong>: \ubc88\uc5ed, \uc694\uc57d, \uc9c8\uc758\uc751\ub2f5.<\/li>\n<li><strong>\ubb38\uc81c<\/strong>: \uacc4\uc0b0 \ube44\uc6a9\uc774 \ub192\uace0 \uad6c\ud604\uc774 \ubcf5\uc7a1\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>\uc194\ub8e8\uc158<\/strong>: \uc99d\ub958, \uac00\uc9c0\uce58\uae30, \ucd5c\uc801\ud654\ub41c \ud558\ub4dc\uc6e8\uc5b4.<\/li>\n<\/ul>\n<h2>\uc8fc\uc694 \ud2b9\uc9d5 \ubc0f \uae30\ud0c0 \uc720\uc0ac \uc6a9\uc5b4\uc640\uc758 \ube44\uad50<\/h2>\n<ul>\n<li><strong>\ud2b8\ub79c\uc2a4\ud3ec\uba38 vs RNN<\/strong>: Transformer\ub294 \ubcd1\ub82c \ucc98\ub9ac\ub97c \uc81c\uacf5\ud558\ub294 \ubc18\uba74 RNN\uc740 \uc21c\ucc28\uc801\uc73c\ub85c \ucc98\ub9ac\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>\ud2b8\ub79c\uc2a4\ud3ec\uba38\uc640 LSTM<\/strong>: Transformer\ub294 \uc7a5\uac70\ub9ac \uc885\uc18d\uc131\uc744 \ub354 \uc798 \ucc98\ub9ac\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc758 \ubcc0\ud658\uae30\uc5d0 \uad00\ud55c \ubbf8\ub798\uc758 \uad00\uc810\uacfc \uae30\uc220<\/h2>\n<p>Transformers\uc758 \ubbf8\ub798\ub294 \ub2e4\uc74c\uacfc \uac19\uc740 \ubd84\uc57c\uc5d0\uc11c \uc9c0\uc18d\uc801\uc778 \uc5f0\uad6c\ub97c \ud1b5\ud574 \uc720\ub9dd\ud569\ub2c8\ub2e4.<\/p>\n<ul>\n<li><strong>\ud6a8\uc728\uc131 \ucd5c\uc801\ud654<\/strong>: \ubaa8\ub378\uc744 \ub354\uc6b1 \uc790\uc6d0 \ud6a8\uc728\uc801\uc73c\ub85c \ub9cc\ub4ed\ub2c8\ub2e4.<\/li>\n<li><strong>\ub2e4\uc911 \ubaa8\ub4dc \ud559\uc2b5<\/strong>: \uc774\ubbf8\uc9c0, \uc0ac\uc6b4\ub4dc \ub4f1 \ub2e4\ub978 \ub370\uc774\ud130 \uc720\ud615\uacfc \ud1b5\ud569\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>\uc724\ub9ac\uc640 \ud3b8\uacac<\/strong>: \uacf5\uc815\ud558\uace0 \ud3b8\uacac \uc5c6\ub294 \ubaa8\ub378\uc744 \uac1c\ubc1c\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h2>\uc790\uc5f0\uc5b4 \ucc98\ub9ac\uc5d0\uc11c \ud504\ub85d\uc2dc \uc11c\ubc84\ub97c \ubcc0\ud658\uae30\uc640 \uc0ac\uc6a9\ud558\uac70\ub098 \uc5f0\uacb0\ud558\ub294 \ubc29\ubc95<\/h2>\n<p>OneProxy\uc640 \uac19\uc740 \ud504\ub85d\uc2dc \uc11c\ubc84\ub294 \ub2e4\uc74c\uacfc \uac19\uc740 \uc5ed\ud560\uc744 \ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<ul>\n<li><strong>\ub370\uc774\ud130 \uc218\uc9d1<\/strong>: Transformer \ud6c8\ub828\uc744 \uc704\ud574 \ub300\uaddc\ubaa8 \ub370\uc774\ud130 \uc138\ud2b8\ub97c \uc548\uc804\ud558\uac8c \uc218\uc9d1\ud569\ub2c8\ub2e4.<\/li>\n<li><strong>\ubd84\uc0b0 \uad50\uc721<\/strong>: \ub2e4\uc591\ud55c \uc704\uce58\uc5d0 \uac78\uccd0 \ubaa8\ub378\uc744 \ud6a8\uc728\uc801\uc73c\ub85c \ubcd1\ub82c \uad50\uc721\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/li>\n<li><strong>\uac15\ud654\ub41c \ubcf4\uc548<\/strong>: \ub370\uc774\ud130\uc640 \ubaa8\ub378\uc758 \ubb34\uacb0\uc131\uacfc \uac1c\uc778\uc815\ubcf4\ub97c \ubcf4\ud638\ud569\ub2c8\ub2e4.<\/li>\n<\/ul>\n<h2>\uad00\ub828\ub41c \ub9c1\ud06c\ub4e4<\/h2>\n<ul>\n<li><a href=\"https:\/\/arxiv.org\/abs\/1706.03762\" target=\"_new\" rel=\"noopener nofollow\">\uc6d0\ub798 \ubcc0\uc555\uae30 \uc885\uc774<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/google-research\/bert\" target=\"_new\" rel=\"noopener nofollow\">BERT GitHub \ub9ac\ud3ec\uc9c0\ud1a0\ub9ac<\/a><\/li>\n<li><a href=\"https:\/\/openai.com\/models\" target=\"_new\" rel=\"noopener nofollow\">OpenAI GPT \ubaa8\ub378<\/a><\/li>\n<li><a href=\"https:\/\/oneproxy.pro\/kr\/\" target=\"_new\" rel=\"noopener\">OneProxy \uc6f9\uc0ac\uc774\ud2b8<\/a><\/li>\n<\/ul>\n<p>NLP\uc758 Transformer\uc5d0 \ub300\ud55c \uc774 \ud3ec\uad04\uc801\uc778 \ubcf4\uae30\ub294 \uad6c\uc870, \uc720\ud615, \uc560\ud50c\ub9ac\ucf00\uc774\uc158 \ubc0f \ud5a5\ud6c4 \ubc29\ud5a5\uc5d0 \ub300\ud55c \ud1b5\ucc30\ub825\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4. OneProxy\uc640 \uac19\uc740 \ud504\ub85d\uc2dc \uc11c\ubc84\uc640\uc758 \uc5f0\uacb0\uc740 \uae30\ub2a5\uc744 \ud655\uc7a5\ud558\uace0 \uc2e4\uc81c \ubb38\uc81c\uc5d0 \ub300\ud55c \ud601\uc2e0\uc801\uc778 \uc194\ub8e8\uc158\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>","protected":false},"featured_media":470727,"menu_order":0,"template":"","meta":{"_acf_changed":false,"content-type":"","inline_featured_image":false,"footnotes":""},"class_list":["post-479385","wiki","type-wiki","status-publish","has-post-thumbnail","hentry"],"acf":{"faq_title":"Frequently Asked Questions about <mark>Transformers in Natural Language Processing<\/mark>","faq_items":null},"_links":{"self":[{"href":"https:\/\/oneproxy.pro\/kr\/wp-json\/wp\/v2\/wiki\/479385","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oneproxy.pro\/kr\/wp-json\/wp\/v2\/wiki"}],"about":[{"href":"https:\/\/oneproxy.pro\/kr\/wp-json\/wp\/v2\/types\/wiki"}],"version-history":[{"count":0,"href":"https:\/\/oneproxy.pro\/kr\/wp-json\/wp\/v2\/wiki\/479385\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/oneproxy.pro\/kr\/wp-json\/wp\/v2\/media\/470727"}],"wp:attachment":[{"href":"https:\/\/oneproxy.pro\/kr\/wp-json\/wp\/v2\/media?parent=479385"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}