{"id":2201,"date":"2020-07-02T22:32:07","date_gmt":"2020-07-02T13:32:07","guid":{"rendered":"http:\/\/141.164.34.82\/?p=2201"},"modified":"2020-07-02T22:53:27","modified_gmt":"2020-07-02T13:53:27","slug":"softmax%ec%97%90-%eb%8c%80%ed%95%9c-%ec%a7%81%ea%b4%80%ec%a0%81-%ec%9d%b4%ed%95%b4","status":"publish","type":"post","link":"http:\/\/ds.sumeun.org\/?p=2201","title":{"rendered":"Softmax\uc5d0 \ub300\ud55c \uc9c1\uad00\uc801 \uc774\ud574"},"content":{"rendered":"

Go Wiki!<\/h2>\n

\uc601\ubb38 \uc704\ud0a4\ub97c \ubcf4\uc790.<\/p>\n\n

The following function:<\/p>\n

\\[\\textrm{softmax}(k, x_1, \\cdots, x_n) = \\frac{e^{x_k}}{\\sum_{i=1}^n e^{x_i}}\\]<\/p>\n

is referred to as the softmax function. The reason is that the effect of exponentiating the values \\(x1, \\cdots, x_n\\) is to exaggerate the differences between them. As a result, \\(\\textrm{softmax}(k, x_1, \\cdots, x_n)\\) will return a value close to 0 whenever \\(x_k\\) is significantly less than the maximum of all the values, and will return a value close to 1 when applied to the maximum value, unless it is extremely close to the next-largest value. Thus, the softmax function can be used to construct a weighted average that behaves as a smooth function(which can be conveniently differentiated, etc.) and which approximates the indicator function.<\/p>\n\n

argmax<\/code><\/h2>\n

argmax \ud568\uc218\ub294 \uc8fc\uc5b4\uc9c4 \uac12\uc5d0\uc11c \ucd5c\ub300\uac12\uc744 \ucc3e\uc544\ub0b8\ub2e4. \uc608\ub97c \ub4e4\uc5b4,<\/p>\n

\\(\\textrm{argmax}_{1 \\leq k \\leq 10} (k^3 – 21k^2+98k)\\) \ub294 \\(3\\) \uc774\ub2e4.<\/p>\n

\\(x_k = k^3 – 21k^2+98k\\) \ub85c \ub193\uace0, \\(k=1, 2, \\cdots, 10\\) \uc77c\ub54c, \\(x_k\\) \ub97c \uad6c\ud574\ubcf4\uba74 \ub2e4\uc74c\uacfc \uac19\ub2e4.<\/p>\n

\\[x_1 = 78, x_2=120, x_3=132, x_4=120, x_5=90, x_6=48, x_7=0, \\cdots\\]<\/p>\n

\ucd5c\ub300\uac12\uc740 \\(132\\) , \uadf8\ub54c \\(k=3\\) \uc774\ub2e4.<\/p>\n

\uc774 \\(k=3\\) \uc744 \uc18c\uc704 dummy-coding\uc73c\ub85c \uc801\uc73c\uba74 \\((0,0,1,0,0,0,0,0,0,0)\\) \uc774 \ub41c\ub2e4.<\/p>\n

\uc774\uc81c \\(\\textrm{argmax}\\) \uac00 \uc544\ub2c8\ub77c \\(\\textrm{softmax}\\) \ub97c \ucde8\ud574\ubcf8\ub2e4.<\/p>\n

softmax = function(x) {\n  exp(x)\/sum(exp(x))\n}\n\nf = function(x) {x^3-21*x^2+98*x}\n\nsoftmax(f(1:10))\n<\/code><\/pre>\n
##  [1]  3.532585e-24  6.144137e-06  9.999877e-01  6.144137e-06  5.749452e-19  3.305660e-37  4.711108e-58\n##  [8]  6.714102e-79  3.860288e-97 3.612312e-110\n<\/pre>\n
\uc6b0\uc640.. \uc218\ub97c \uc77d\uae30 \uc27d\uc9c0\uac00 \uc54a\uc73c\ub2c8…<\/p>\n
round(softmax(f(1:10)), 2)\n<\/code><\/pre>\n
##  [1] 0 0 1 0 0 0 0 0 0 0\n<\/pre>\n
\uc789?<\/p>\n
round(softmax(f(1:10)), 5)\n<\/code><\/pre>\n
##  [1] 0.00000 0.00001 0.99999 0.00001 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000\n<\/pre>\n
\ub530\ub77c\uc11c \\(\\textrm{argmax}\\) \uac00 \uc544\ub2c8\ub77c \\(\\textrm{softmax}\\) \ub97c \ube44\uad50\ud574\ubcf8\ub2e4\uba74,<\/p>\n
dummy coding(argmax(x_k)) = (0.00000, 0.00000, 1.00000, 0.00000, 0.00000, 0.00000, 0.00000, 0.00000, 0.00000, 0.00000)\n             softmax(x_k) = (0.00000, 0.00001, 0.99999, 0.00001, 0.00000, 0.00000, 0.00000, 0.00000, 0.00000, 0.00000)\n<\/pre>\n
\uad49\uc7a5\ud788 \ube44\uc2b7\ud558\uc9c0 \uc54a\uc740\uac00? <\/p>\n
\uadf8\ub7ec\ub2c8\uae4c \ub0b4 \ub9d0\uc740, softmax\ub294 soft(dummycoding(argmax)) \uc815\ub3c4\ub85c \uc774\ud574\ud560 \uc218 \uc788\ub2e4\ub294 \uc18c\ub9ac\ub2e4.<\/p>\n
\uadf8\ubc16\uc758 \uc131\uc9c8<\/h2>\n
\\[\\textrm{softmax}(k, x_1, \\cdots, x_n) = \\frac{e^{x_k}}{\\sum_{i=1}^n e^{x_i}}\\]<\/p>\n
\uc704\uc758 \uc2dd\uc5d0\uc11c \ubd84\ubaa8\uc640 \ubd84\uc790\ub97c \ubaa8\ub450 \\(e^{x_k}\\) \ub85c \ub098\ub220\uc8fc\uba74 \ub2e4\uc74c\uacfc \uac19\ub2e4.<\/p>\n
\\[\\textrm{softmax}(k, x_1, \\cdots, x_n) = \\frac{1}{\\sum_{i=1}^n e^{x_i-x_k}}\\]<\/p>\n
\uacb0\uad6d \\(\\textrm{softmax}\\) \uac12\uc740 \\(x_1-x_k, x_2 – x_k, \\cdots\\) \uc5d0 \uc758\ud574 \uc88c\uc6b0\ub420 \ubfd0, \\(x_1, x_2, \\cdots\\) \uc758 \ud06c\uae30\uac00 \uc911\uc694\ud558\uc9c0 \uc54a\ub2e4. \ub2e4\uc2dc \ub9d0\ud574 \\(x_1, x_2, \\cdots\\) \uc5d0 \uc77c\uc815\ud55c \uc0c1\uc218\ub97c \ub354\ud558\uac70\ub098 \ube7c\uc918\ub3c4 \\(\\textrm{softmax}\\) \uac12\uc740 \ubcc0\ud558\uc9c0 \uc54a\ub294\ub2e4.<\/p>\n
\ub2e4\uc74c\uc740 \uac00\uc7a5 \ud070 \\(x_k=0\\) \uc774\uace0, \ub098\uba38\uc9c0 \\(x_i(i \\neq k)\\) \uac00 \ubaa8\ub450 \\(-1\\) \ub610\ub294 \\(-2\\) , …\uc77c \ub54c, \\(\\textrm{softmax}(x_k)\\) \uc758 \ubcc0\ud654\ub97c \ubcf4\uc5ec\uc900\ub2e4. <\/p>\n
library(dplyr)\nlibrary(ggplot2)\ncond = expand.grid(n=2:10, diffx = 1:10)\ndat = cond %>% mutate(softmax = 1\/(exp(0)+(n-1)*exp(-diffx)))\nggplot(dat, aes(x=n, col=factor(diffx), y=softmax)) + \n  geom_point() + \n  geom_line() + ylim(0,1) + \n  geom_hline(yintercept= 0.9, linetype='dotted') + \n  scale_x_continuous(breaks=1:10)\n<\/code><\/pre>\n
\"plot<\/p>\n
\ub9cc\uc57d \uc120\ud0dd \uac00\uc9c0 \uc218\uac00 5\uac1c \uc77c\ub54c, \ucd5c\ub300 softmax \uac12\uc774 0.9\ub97c \ub118\uc5b4\uac00\ub824\uba74, \uadf8 \\(x_k\\) \uc758 \ucc28\uc774\uac00 \ubaa8\ub450 3~4 \uc774\uc0c1\uc774\uba74 \ub41c\ub2e4\ub294 \uac83\uc744 \ubcf4\uc5ec\uc900\ub2e4. \\(x_k\\) \uc758 \ucc28\uc774\uac00 \ubaa8\ub450 5\ub97c \ub118\uc5b4\uac00\uba74, \uc120\ud0dd\uac00\uc9c0\uc218\uac00 10\uac1c\ub77c\ub3c4 \ucd5c\ub300 softmax\ub294 0.9 \uc774\uc0c1\uc774 \ub428\uc744 \ud655\uc778\ud560 \uc218 \uc788\ub2e4. <\/p>\n","protected":false},"excerpt":{"rendered":"
Go Wiki! \uc601\ubb38 \uc704\ud0a4\ub97c \ubcf4\uc790. The following function: \\[\\textrm{softmax}(k, x_1, \\cdots, x_n) = \\frac{e^{x_k}}{\\sum_{i=1}^n e^{x_i}}\\] is referred to as the softmax function. The reason is that the effect of exponentiating the values \\(x1, \\cdots, x_n\\) is to exaggerate the differences between them. As a result, \\(\\textrm{softmax}(k, x_1, \\cdots, x_n)\\) will return a value close to […]<\/p>\n","protected":false},"author":1,"featured_media":2204,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[390,28],"tags":[463,437,464,318],"jetpack_featured_media_url":"http:\/\/ds.sumeun.org\/wp-content\/uploads\/2020\/07\/softmax.png","_links":{"self":[{"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=\/wp\/v2\/posts\/2201"}],"collection":[{"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2201"}],"version-history":[{"count":4,"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=\/wp\/v2\/posts\/2201\/revisions"}],"predecessor-version":[{"id":2206,"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=\/wp\/v2\/posts\/2201\/revisions\/2206"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=\/wp\/v2\/media\/2204"}],"wp:attachment":[{"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2201"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2201"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/ds.sumeun.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2201"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}