{"id":475880,"date":"2023-08-09T07:24:43","date_gmt":"2023-08-09T07:24:43","guid":{"rendered":""},"modified":"2023-09-05T11:11:30","modified_gmt":"2023-09-05T11:11:30","slug":"apache-spark","status":"publish","type":"wiki","link":"https:\/\/oneproxy.pro\/jp\/wiki\/apache-spark\/","title":{"rendered":"\u30a2\u30d1\u30c3\u30c1\u30b9\u30d1\u30fc\u30af"},"content":{"rendered":"<p>Apache Spark \u306f\u3001\u30d3\u30c3\u30b0 \u30c7\u30fc\u30bf\u306e\u51e6\u7406\u3068\u5206\u6790\u306e\u305f\u3081\u306b\u8a2d\u8a08\u3055\u308c\u305f\u30aa\u30fc\u30d7\u30f3\u30bd\u30fc\u30b9\u306e\u5206\u6563\u30b3\u30f3\u30d4\u30e5\u30fc\u30c6\u30a3\u30f3\u30b0 \u30b7\u30b9\u30c6\u30e0\u3067\u3059\u3002\u6700\u521d\u306f 2009 \u5e74\u306b\u30ab\u30ea\u30d5\u30a9\u30eb\u30cb\u30a2\u5927\u5b66\u30d0\u30fc\u30af\u30ec\u30fc\u6821\u306e AMPLab \u3067\u958b\u767a\u3055\u308c\u3001\u305d\u306e\u5f8c Apache Software Foundation \u306b\u5bc4\u4ed8\u3055\u308c\u30012010 \u5e74\u306b Apache \u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u306b\u306a\u308a\u307e\u3057\u305f\u3002\u305d\u308c\u4ee5\u6765\u3001Apache Spark \u306f\u30d3\u30c3\u30b0 \u30c7\u30fc\u30bf \u30b3\u30df\u30e5\u30cb\u30c6\u30a3\u3067\u5e83\u304f\u4eba\u6c17\u3092\u96c6\u3081\u3066\u304d\u307e\u3057\u305f\u3002\u30b9\u30d4\u30fc\u30c9\u3001\u4f7f\u3044\u3084\u3059\u3055\u3001\u591a\u7528\u9014\u6027\u3002<\/p>\n<h2>Apache Spark \u306e\u8d77\u6e90\u3068\u305d\u306e\u6700\u521d\u306e\u8a00\u53ca\u306e\u6b74\u53f2<\/h2>\n<p>Apache Spark \u306f\u3001\u958b\u767a\u8005\u304c Hadoop MapReduce \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3068\u4f7f\u3044\u3084\u3059\u3055\u306e\u9650\u754c\u306b\u76f4\u9762\u3057\u3066\u3044\u305f AMPLab \u3067\u306e\u7814\u7a76\u6d3b\u52d5\u304b\u3089\u751f\u307e\u308c\u307e\u3057\u305f\u3002 Apache Spark \u306b\u3064\u3044\u3066\u521d\u3081\u3066\u8a00\u53ca\u3057\u305f\u306e\u306f\u30012012 \u5e74\u306b Matei Zaharia \u3089\u304c\u767a\u8868\u3057\u305f\u300cResilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing\u300d\u3068\u3044\u3046\u30bf\u30a4\u30c8\u30eb\u306e\u7814\u7a76\u8ad6\u6587\u3067\u3057\u305f\u3002\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001Resilient Distributed Datasets (RDD) \u306e\u6982\u5ff5\u304c\u7d39\u4ecb\u3055\u308c\u307e\u3057\u305f\u3002 )\u3001Spark \u306e\u57fa\u672c\u7684\u306a\u30c7\u30fc\u30bf\u69cb\u9020\u3002<\/p>\n<h2>Apache Spark \u306e\u8a73\u7d30\u60c5\u5831: \u30c8\u30d4\u30c3\u30af\u306e\u5c55\u958b<\/h2>\n<p>Apache Spark \u306f\u3001\u5927\u898f\u6a21\u306a\u30c7\u30fc\u30bf\u3092\u51e6\u7406\u3059\u308b\u305f\u3081\u306e\u52b9\u7387\u7684\u304b\u3064\u67d4\u8edf\u306a\u65b9\u6cd5\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002\u30a4\u30f3\u30e1\u30e2\u30ea\u51e6\u7406\u3092\u63d0\u4f9b\u3059\u308b\u305f\u3081\u3001Hadoop MapReduce \u306a\u3069\u306e\u5f93\u6765\u306e\u30c7\u30a3\u30b9\u30af\u30d9\u30fc\u30b9\u306e\u51e6\u7406\u30b7\u30b9\u30c6\u30e0\u3068\u6bd4\u8f03\u3057\u3066\u3001\u30c7\u30fc\u30bf\u51e6\u7406\u30bf\u30b9\u30af\u304c\u5927\u5e45\u306b\u9ad8\u901f\u5316\u3055\u308c\u307e\u3059\u3002 Spark \u3092\u4f7f\u7528\u3059\u308b\u3068\u3001\u958b\u767a\u8005\u306f Scala\u3001Java\u3001Python\u3001R \u306a\u3069\u306e\u3055\u307e\u3056\u307e\u306a\u8a00\u8a9e\u3067\u30c7\u30fc\u30bf\u51e6\u7406\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u3092\u4f5c\u6210\u3067\u304d\u308b\u305f\u3081\u3001\u3088\u308a\u5e45\u5e83\u3044\u30e6\u30fc\u30b6\u30fc\u304c\u30a2\u30af\u30bb\u30b9\u3067\u304d\u308b\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n<h2>Apache Spark \u306e\u5185\u90e8\u69cb\u9020: Apache Spark \u306e\u4ed5\u7d44\u307f<\/h2>\n<p>Apache Spark \u306e\u4e2d\u6838\u3068\u306a\u308b\u306e\u306f\u3001\u4e26\u5217\u51e6\u7406\u53ef\u80fd\u306a\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u306e\u4e0d\u5909\u5206\u6563\u30b3\u30ec\u30af\u30b7\u30e7\u30f3\u3067\u3042\u308b Resilient Distributed Dataset (RDD) \u3067\u3059\u3002 RDD \u306f\u30d5\u30a9\u30fc\u30eb\u30c8 \u30c8\u30ec\u30e9\u30f3\u30c8\u3067\u3059\u3002\u3064\u307e\u308a\u3001\u30ce\u30fc\u30c9\u969c\u5bb3\u304c\u767a\u751f\u3057\u305f\u5834\u5408\u306b\u5931\u308f\u308c\u305f\u30c7\u30fc\u30bf\u3092\u56de\u5fa9\u3067\u304d\u307e\u3059\u3002 Spark \u306e DAG (\u6709\u5411\u975e\u5de1\u56de\u30b0\u30e9\u30d5) \u30a8\u30f3\u30b8\u30f3\u306f\u3001RDD \u64cd\u4f5c\u3092\u6700\u9069\u5316\u304a\u3088\u3073\u30b9\u30b1\u30b8\u30e5\u30fc\u30eb\u3057\u3066\u3001\u6700\u5927\u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3092\u5b9f\u73fe\u3057\u307e\u3059\u3002<\/p>\n<p>Spark \u30a8\u30b3\u30b7\u30b9\u30c6\u30e0\u306f\u3001\u3044\u304f\u3064\u304b\u306e\u9ad8\u30ec\u30d9\u30eb\u306e\u30b3\u30f3\u30dd\u30fc\u30cd\u30f3\u30c8\u3067\u69cb\u6210\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n<ol>\n<li>Spark Core: \u57fa\u672c\u6a5f\u80fd\u3068 RDD \u62bd\u8c61\u5316\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002<\/li>\n<li>Spark SQL: \u69cb\u9020\u5316\u30c7\u30fc\u30bf\u51e6\u7406\u306e\u305f\u3081\u306e SQL \u306e\u3088\u3046\u306a\u30af\u30a8\u30ea\u3092\u6709\u52b9\u306b\u3057\u307e\u3059\u3002<\/li>\n<li>Spark Streaming: \u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u306e\u30c7\u30fc\u30bf\u51e6\u7406\u3092\u53ef\u80fd\u306b\u3057\u307e\u3059\u3002<\/li>\n<li>MLlib (\u6a5f\u68b0\u5b66\u7fd2\u30e9\u30a4\u30d6\u30e9\u30ea): \u5e45\u5e83\u3044\u6a5f\u68b0\u5b66\u7fd2\u30a2\u30eb\u30b4\u30ea\u30ba\u30e0\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002<\/li>\n<li>GraphX: \u30b0\u30e9\u30d5\u306e\u51e6\u7406\u3068\u5206\u6790\u3092\u53ef\u80fd\u306b\u3057\u307e\u3059\u3002<\/li>\n<\/ol>\n<h2>Apache Spark \u306e\u4e3b\u306a\u6a5f\u80fd\u306e\u5206\u6790<\/h2>\n<p>Apache Spark \u306e\u4e3b\u306a\u6a5f\u80fd\u306b\u3088\u308a\u3001Apache Spark \u306f\u30d3\u30c3\u30b0 \u30c7\u30fc\u30bf\u306e\u51e6\u7406\u3068\u5206\u6790\u306b\u4eba\u6c17\u306e\u9078\u629e\u80a2\u3068\u306a\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n<ol>\n<li>\u30a4\u30f3\u30e1\u30e2\u30ea\u51e6\u7406: Spark \u306e\u30e1\u30e2\u30ea\u5185\u306b\u30c7\u30fc\u30bf\u3092\u4fdd\u5b58\u3059\u308b\u6a5f\u80fd\u306b\u3088\u308a\u3001\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u304c\u5927\u5e45\u306b\u5411\u4e0a\u3057\u3001\u30c7\u30a3\u30b9\u30af\u306e\u8aad\u307f\u53d6\u308a\/\u66f8\u304d\u8fbc\u307f\u64cd\u4f5c\u3092\u7e70\u308a\u8fd4\u3059\u5fc5\u8981\u6027\u304c\u8efd\u6e1b\u3055\u308c\u307e\u3059\u3002<\/li>\n<li>\u30d5\u30a9\u30fc\u30eb\u30c8 \u30c8\u30ec\u30e9\u30f3\u30b9: RDD \u306f\u30d5\u30a9\u30fc\u30eb\u30c8 \u30c8\u30ec\u30e9\u30f3\u30b9\u3092\u63d0\u4f9b\u3057\u3001\u30ce\u30fc\u30c9\u969c\u5bb3\u304c\u767a\u751f\u3057\u305f\u5834\u5408\u3067\u3082\u30c7\u30fc\u30bf\u306e\u4e00\u8cab\u6027\u3092\u4fdd\u8a3c\u3057\u307e\u3059\u3002<\/li>\n<li>\u4f7f\u3044\u3084\u3059\u3055: Spark \u306e API \u306f\u30e6\u30fc\u30b6\u30fc\u30d5\u30ec\u30f3\u30c9\u30ea\u30fc\u3067\u3001\u8907\u6570\u306e\u30d7\u30ed\u30b0\u30e9\u30df\u30f3\u30b0\u8a00\u8a9e\u3092\u30b5\u30dd\u30fc\u30c8\u3057\u3001\u958b\u767a\u30d7\u30ed\u30bb\u30b9\u3092\u7c21\u7d20\u5316\u3057\u307e\u3059\u3002<\/li>\n<li>\u6c4e\u7528\u6027: Spark \u306f\u3001\u30d0\u30c3\u30c1\u51e6\u7406\u3001\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406\u3001\u6a5f\u68b0\u5b66\u7fd2\u3001\u30b0\u30e9\u30d5\u51e6\u7406\u7528\u306e\u5e45\u5e83\u3044\u30e9\u30a4\u30d6\u30e9\u30ea\u3092\u63d0\u4f9b\u3057\u3001\u6c4e\u7528\u6027\u306e\u9ad8\u3044\u30d7\u30e9\u30c3\u30c8\u30d5\u30a9\u30fc\u30e0\u306b\u3057\u3066\u3044\u307e\u3059\u3002<\/li>\n<li>\u901f\u5ea6: Spark \u306e\u30a4\u30f3\u30e1\u30e2\u30ea\u51e6\u7406\u3068\u6700\u9069\u5316\u3055\u308c\u305f\u5b9f\u884c\u30a8\u30f3\u30b8\u30f3\u304c\u3001\u512a\u308c\u305f\u901f\u5ea6\u306b\u8ca2\u732e\u3057\u307e\u3059\u3002<\/li>\n<\/ol>\n<h2>Apache Spark \u306e\u7a2e\u985e<\/h2>\n<p>Apache Spark \u306f\u3001\u305d\u306e\u7528\u9014\u3068\u6a5f\u80fd\u306b\u57fa\u3065\u3044\u3066\u3055\u307e\u3056\u307e\u306a\u30bf\u30a4\u30d7\u306b\u5206\u985e\u3067\u304d\u307e\u3059\u3002<\/p>\n<table>\n<thead>\n<tr>\n<th>\u30bf\u30a4\u30d7<\/th>\n<th>\u8aac\u660e<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>\u30d0\u30c3\u30c1\u51e6\u7406<\/td>\n<td>\u5927\u91cf\u306e\u30c7\u30fc\u30bf\u3092\u4e00\u5ea6\u306b\u5206\u6790\u304a\u3088\u3073\u51e6\u7406\u3057\u307e\u3059\u3002<\/td>\n<\/tr>\n<tr>\n<td>\u30b9\u30c8\u30ea\u30fc\u30e0\u51e6\u7406<\/td>\n<td>\u30c7\u30fc\u30bf \u30b9\u30c8\u30ea\u30fc\u30e0\u304c\u5230\u7740\u3059\u308b\u3068\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u306b\u51e6\u7406\u3057\u307e\u3059\u3002<\/td>\n<\/tr>\n<tr>\n<td>\u6a5f\u68b0\u5b66\u7fd2<\/td>\n<td>\u6a5f\u68b0\u5b66\u7fd2\u30a2\u30eb\u30b4\u30ea\u30ba\u30e0\u306e\u5b9f\u88c5\u306b\u306f Spark \u306e MLlib \u3092\u5229\u7528\u3057\u307e\u3059\u3002<\/td>\n<\/tr>\n<tr>\n<td>\u30b0\u30e9\u30d5\u51e6\u7406<\/td>\n<td>\u30b0\u30e9\u30d5\u3068\u8907\u96d1\u306a\u30c7\u30fc\u30bf\u69cb\u9020\u3092\u5206\u6790\u304a\u3088\u3073\u51e6\u7406\u3057\u307e\u3059\u3002<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2>Apache Spark\u306e\u4f7f\u3044\u65b9\uff1a\u4f7f\u7528\u306b\u95a2\u3059\u308b\u554f\u984c\u3068\u89e3\u6c7a\u7b56<\/h2>\n<p>Apache Spark \u306f\u3001\u30c7\u30fc\u30bf\u5206\u6790\u3001\u6a5f\u68b0\u5b66\u7fd2\u3001\u63a8\u5968\u30b7\u30b9\u30c6\u30e0\u3001\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0 \u30a4\u30d9\u30f3\u30c8\u51e6\u7406\u306a\u3069\u3001\u3055\u307e\u3056\u307e\u306a\u30c9\u30e1\u30a4\u30f3\u306e\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u3092\u898b\u3064\u3051\u307e\u3059\u3002\u305f\u3060\u3057\u3001Apache Spark \u306e\u4f7f\u7528\u4e2d\u306b\u3001\u3044\u304f\u3064\u304b\u306e\u4e00\u822c\u7684\u306a\u8ab2\u984c\u304c\u767a\u751f\u3059\u308b\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n<ol>\n<li>\n<p><strong>\u30e1\u30e2\u30ea\u7ba1\u7406<\/strong>: Spark \u306f\u30e1\u30e2\u30ea\u5185\u51e6\u7406\u306b\u5927\u304d\u304f\u4f9d\u5b58\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u30e1\u30e2\u30ea\u4e0d\u8db3\u30a8\u30e9\u30fc\u3092\u56de\u907f\u3059\u308b\u306b\u306f\u52b9\u7387\u7684\u306a\u30e1\u30e2\u30ea\u7ba1\u7406\u304c\u91cd\u8981\u3067\u3059\u3002<\/p>\n<ul>\n<li>\u89e3\u6c7a\u7b56: \u30c7\u30fc\u30bf \u30b9\u30c8\u30ec\u30fc\u30b8\u3092\u6700\u9069\u5316\u3057\u3001\u30ad\u30e3\u30c3\u30b7\u30e5\u3092\u614e\u91cd\u306b\u4f7f\u7528\u3057\u3001\u30e1\u30e2\u30ea\u4f7f\u7528\u91cf\u3092\u76e3\u8996\u3057\u307e\u3059\u3002<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong>\u30c7\u30fc\u30bf\u30b9\u30ad\u30e5\u30fc<\/strong>: \u30d1\u30fc\u30c6\u30a3\u30b7\u30e7\u30f3\u9593\u3067\u30c7\u30fc\u30bf\u304c\u4e0d\u5747\u4e00\u306b\u5206\u6563\u3055\u308c\u3066\u3044\u308b\u3068\u3001\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u306e\u30dc\u30c8\u30eb\u30cd\u30c3\u30af\u304c\u767a\u751f\u3059\u308b\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n<ul>\n<li>\u89e3\u6c7a\u7b56: \u30c7\u30fc\u30bf\u518d\u5206\u5272\u6280\u8853\u3092\u4f7f\u7528\u3057\u3066\u3001\u30c7\u30fc\u30bf\u3092\u5747\u7b49\u306b\u5206\u6563\u3057\u307e\u3059\u3002<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong>\u30af\u30e9\u30b9\u30bf\u30fc\u306e\u30b5\u30a4\u30b8\u30f3\u30b0<\/strong>: \u30af\u30e9\u30b9\u30bf\u306e\u30b5\u30a4\u30b8\u30f3\u30b0\u304c\u6b63\u3057\u304f\u306a\u3044\u3068\u3001\u30ea\u30bd\u30fc\u30b9\u304c\u5341\u5206\u306b\u6d3b\u7528\u3055\u308c\u306a\u304b\u3063\u305f\u308a\u3001\u30ea\u30bd\u30fc\u30b9\u304c\u904e\u8ca0\u8377\u306b\u306a\u3063\u305f\u308a\u3059\u308b\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n<ul>\n<li>\u89e3\u6c7a\u7b56: \u30af\u30e9\u30b9\u30bf\u30fc\u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3092\u5b9a\u671f\u7684\u306b\u76e3\u8996\u3057\u3001\u305d\u308c\u306b\u5fdc\u3058\u3066\u30ea\u30bd\u30fc\u30b9\u3092\u8abf\u6574\u3057\u307e\u3059\u3002<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong>\u30c7\u30fc\u30bf\u306e\u30b7\u30ea\u30a2\u30eb\u5316<\/strong>: \u975e\u52b9\u7387\u7684\u306a\u30c7\u30fc\u30bf\u306e\u30b7\u30ea\u30a2\u30eb\u5316\u306f\u3001\u30c7\u30fc\u30bf\u8ee2\u9001\u4e2d\u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u306b\u5f71\u97ff\u3092\u4e0e\u3048\u308b\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n<ul>\n<li>\u89e3\u6c7a\u7b56: \u9069\u5207\u306a\u30b7\u30ea\u30a2\u30eb\u5316\u5f62\u5f0f\u3092\u9078\u629e\u3057\u3001\u5fc5\u8981\u306b\u5fdc\u3058\u3066\u30c7\u30fc\u30bf\u3092\u5727\u7e2e\u3057\u307e\u3059\u3002<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<h2>\u4e3b\u306a\u7279\u5fb4\u3068\u985e\u4f3c\u7528\u8a9e\u3068\u306e\u6bd4\u8f03<\/h2>\n<table>\n<thead>\n<tr>\n<th>\u7279\u6027<\/th>\n<th>\u30a2\u30d1\u30c3\u30c1\u30b9\u30d1\u30fc\u30af<\/th>\n<th>Hadoop MapReduce<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>\u51e6\u7406\u30d1\u30e9\u30c0\u30a4\u30e0<\/td>\n<td>\u30a4\u30f3\u30e1\u30e2\u30ea\u51e6\u7406\u3068\u53cd\u5fa9\u51e6\u7406<\/td>\n<td>\u30c7\u30a3\u30b9\u30af\u30d9\u30fc\u30b9\u306e\u30d0\u30c3\u30c1\u51e6\u7406<\/td>\n<\/tr>\n<tr>\n<td>\u60c5\u5831\u51e6\u7406<\/td>\n<td>\u30d0\u30c3\u30c1\u51e6\u7406\u3068\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u51e6\u7406<\/td>\n<td>\u30d0\u30c3\u30c1\u51e6\u7406\u306e\u307f<\/td>\n<\/tr>\n<tr>\n<td>\u30d5\u30a9\u30fc\u30eb\u30c8\u30c8\u30ec\u30e9\u30f3\u30b9<\/td>\n<td>\u306f\u3044\uff08RDD\u7d4c\u7531\uff09<\/td>\n<td>\u306f\u3044 (\u30ec\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u7d4c\u7531)<\/td>\n<\/tr>\n<tr>\n<td>\u30c7\u30fc\u30bf\u30b9\u30c8\u30ec\u30fc\u30b8<\/td>\n<td>\u30a4\u30f3\u30e1\u30e2\u30ea\u304a\u3088\u3073\u30c7\u30a3\u30b9\u30af\u30d9\u30fc\u30b9<\/td>\n<td>\u30c7\u30a3\u30b9\u30af\u30d9\u30fc\u30b9<\/td>\n<\/tr>\n<tr>\n<td>\u751f\u614b\u7cfb<\/td>\n<td>\u591a\u69d8\u306a\u30e9\u30a4\u30d6\u30e9\u30ea\u30bb\u30c3\u30c8 (Spark SQL\u3001Spark Streaming\u3001MLlib\u3001GraphX \u306a\u3069)<\/td>\n<td>\u9650\u3089\u308c\u305f\u30a8\u30b3\u30b7\u30b9\u30c6\u30e0<\/td>\n<\/tr>\n<tr>\n<td>\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9<\/td>\n<td>\u30a4\u30f3\u30e1\u30e2\u30ea\u51e6\u7406\u306b\u3088\u308a\u9ad8\u901f\u5316<\/td>\n<td>\u30c7\u30a3\u30b9\u30af\u306e\u8aad\u307f\u53d6\u308a\/\u66f8\u304d\u8fbc\u307f\u306b\u3088\u308a\u901f\u5ea6\u304c\u4f4e\u4e0b\u3059\u308b<\/td>\n<\/tr>\n<tr>\n<td>\u4f7f\u3044\u3084\u3059\u3055<\/td>\n<td>\u30e6\u30fc\u30b6\u30fc\u30d5\u30ec\u30f3\u30c9\u30ea\u30fc\u306a API \u3068\u8907\u6570\u8a00\u8a9e\u306e\u30b5\u30dd\u30fc\u30c8<\/td>\n<td>\u5b66\u7fd2\u66f2\u7dda\u304c\u6025\u3067Java\u30d9\u30fc\u30b9<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2>Apache Spark \u306b\u95a2\u9023\u3059\u308b\u5c06\u6765\u306e\u5c55\u671b\u3068\u30c6\u30af\u30ce\u30ed\u30b8\u30fc<\/h2>\n<p>\u30d3\u30c3\u30b0\u30c7\u30fc\u30bf\u306f\u5f15\u304d\u7d9a\u304d\u3055\u307e\u3056\u307e\u306a\u696d\u754c\u306b\u3068\u3063\u3066\u91cd\u8981\u306a\u5074\u9762\u3067\u3042\u308b\u305f\u3081\u3001Apache Spark \u306e\u5c06\u6765\u306f\u6709\u671b\u306b\u898b\u3048\u307e\u3059\u3002 Apache Spark \u306e\u5c06\u6765\u306b\u95a2\u9023\u3059\u308b\u91cd\u8981\u306a\u8996\u70b9\u3068\u30c6\u30af\u30ce\u30ed\u30b8\u30fc\u306b\u306f\u6b21\u306e\u3088\u3046\u306a\u3082\u306e\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n<ol>\n<li><strong>\u6700\u9069\u5316<\/strong>: Spark \u306e\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3068\u30ea\u30bd\u30fc\u30b9\u4f7f\u7528\u7387\u3092\u5411\u4e0a\u3055\u305b\u308b\u305f\u3081\u306e\u7d99\u7d9a\u7684\u306a\u53d6\u308a\u7d44\u307f\u306b\u3088\u308a\u3001\u51e6\u7406\u304c\u3055\u3089\u306b\u9ad8\u901f\u5316\u3055\u308c\u3001\u30e1\u30e2\u30ea \u30aa\u30fc\u30d0\u30fc\u30d8\u30c3\u30c9\u304c\u524a\u6e1b\u3055\u308c\u308b\u53ef\u80fd\u6027\u304c\u3042\u308a\u307e\u3059\u3002<\/li>\n<li><strong>AI\u3068\u306e\u7d71\u5408<\/strong>\uff1aApache Spark \u306f\u3001\u4eba\u5de5\u77e5\u80fd\u304a\u3088\u3073\u6a5f\u68b0\u5b66\u7fd2\u30d5\u30ec\u30fc\u30e0\u30ef\u30fc\u30af\u3068\u3088\u308a\u6df1\u304f\u7d71\u5408\u3055\u308c\u308b\u53ef\u80fd\u6027\u304c\u9ad8\u304f\u3001AI \u3092\u6d3b\u7528\u3057\u305f\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u306b\u3068\u3063\u3066\u983c\u308a\u306b\u306a\u308b\u9078\u629e\u80a2\u306b\u306a\u308a\u307e\u3059\u3002<\/li>\n<li><strong>\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u5206\u6790<\/strong>\uff1aSpark \u306e\u30b9\u30c8\u30ea\u30fc\u30df\u30f3\u30b0\u6a5f\u80fd\u306f\u4eca\u5f8c\u3082\u9032\u5316\u3057\u3001\u3088\u308a\u30b7\u30fc\u30e0\u30ec\u30b9\u306a\u30ea\u30a2\u30eb\u30bf\u30a4\u30e0\u5206\u6790\u304c\u53ef\u80fd\u306b\u306a\u308a\u3001\u77ac\u6642\u306e\u6d1e\u5bdf\u3068\u610f\u601d\u6c7a\u5b9a\u304c\u53ef\u80fd\u306b\u306a\u308b\u3068\u8003\u3048\u3089\u308c\u307e\u3059\u3002<\/li>\n<\/ol>\n<h2>\u30d7\u30ed\u30ad\u30b7\u30b5\u30fc\u30d0\u30fc\u3092 Apache Spark \u3067\u4f7f\u7528\u3059\u308b\u65b9\u6cd5\u307e\u305f\u306f\u95a2\u9023\u4ed8\u3051\u308b\u65b9\u6cd5<\/h2>\n<p>\u30d7\u30ed\u30ad\u30b7 \u30b5\u30fc\u30d0\u30fc\u306f\u3001Apache Spark \u30c7\u30d7\u30ed\u30a4\u30e1\u30f3\u30c8\u306e\u30bb\u30ad\u30e5\u30ea\u30c6\u30a3\u3068\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u3092\u5f37\u5316\u3059\u308b\u4e0a\u3067\u91cd\u8981\u306a\u5f79\u5272\u3092\u679c\u305f\u3057\u307e\u3059\u3002\u30d7\u30ed\u30ad\u30b7 \u30b5\u30fc\u30d0\u30fc\u3092 Apache Spark \u3067\u4f7f\u7528\u3059\u308b\u65b9\u6cd5\u3084\u3001Apache Spark \u306b\u95a2\u9023\u4ed8\u3051\u308b\u65b9\u6cd5\u306b\u306f\u3001\u6b21\u306e\u3088\u3046\u306a\u3082\u306e\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n<ol>\n<li><strong>\u30ed\u30fc\u30c9\u30d0\u30e9\u30f3\u30b7\u30f3\u30b0<\/strong>: \u30d7\u30ed\u30ad\u30b7 \u30b5\u30fc\u30d0\u30fc\u306f\u3001\u53d7\u4fe1\u30ea\u30af\u30a8\u30b9\u30c8\u3092\u8907\u6570\u306e Spark \u30ce\u30fc\u30c9\u306b\u5206\u6563\u3057\u3001\u30ea\u30bd\u30fc\u30b9\u306e\u5747\u7b49\u306a\u4f7f\u7528\u3068\u30d1\u30d5\u30a9\u30fc\u30de\u30f3\u30b9\u306e\u5411\u4e0a\u3092\u5b9f\u73fe\u3057\u307e\u3059\u3002<\/li>\n<li><strong>\u5b89\u5168<\/strong>: \u30d7\u30ed\u30ad\u30b7 \u30b5\u30fc\u30d0\u30fc\u306f\u30e6\u30fc\u30b6\u30fc\u3068 Spark \u30af\u30e9\u30b9\u30bf\u30fc\u306e\u9593\u306e\u4ef2\u4ecb\u8005\u3068\u3057\u3066\u6a5f\u80fd\u3057\u3001\u8ffd\u52a0\u306e\u30bb\u30ad\u30e5\u30ea\u30c6\u30a3\u5c64\u3092\u63d0\u4f9b\u3057\u3001\u6f5c\u5728\u7684\u306a\u653b\u6483\u304b\u3089\u4fdd\u8b77\u3057\u307e\u3059\u3002<\/li>\n<li><strong>\u30ad\u30e3\u30c3\u30b7\u30f3\u30b0<\/strong>: \u30d7\u30ed\u30ad\u30b7 \u30b5\u30fc\u30d0\u30fc\u306f\u983b\u7e41\u306b\u8981\u6c42\u3055\u308c\u308b\u30c7\u30fc\u30bf\u3092\u30ad\u30e3\u30c3\u30b7\u30e5\u3067\u304d\u308b\u305f\u3081\u3001Spark \u30af\u30e9\u30b9\u30bf\u30fc\u306e\u8ca0\u8377\u304c\u8efd\u6e1b\u3055\u308c\u3001\u5fdc\u7b54\u6642\u9593\u304c\u5411\u4e0a\u3057\u307e\u3059\u3002<\/li>\n<\/ol>\n<h2>\u95a2\u9023\u30ea\u30f3\u30af<\/h2>\n<p>Apache Spark \u306e\u8a73\u7d30\u306b\u3064\u3044\u3066\u306f\u3001\u6b21\u306e\u30ea\u30bd\u30fc\u30b9\u3092\u53c2\u7167\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n<ol>\n<li><a href=\"https:\/\/spark.apache.org\/\" target=\"_new\" rel=\"noopener nofollow\">Apache Spark \u516c\u5f0f\u30a6\u30a7\u30d6\u30b5\u30a4\u30c8<\/a><\/li>\n<li><a href=\"https:\/\/spark.apache.org\/documentation.html\" target=\"_new\" rel=\"noopener nofollow\">Apache Spark \u30c9\u30ad\u30e5\u30e1\u30f3\u30c8<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/apache\/spark\" target=\"_new\" rel=\"noopener nofollow\">Apache Spark GitHub \u30ea\u30dd\u30b8\u30c8\u30ea<\/a><\/li>\n<li><a href=\"https:\/\/databricks.com\/spark\/about\" target=\"_new\" rel=\"noopener nofollow\">\u30c7\u30fc\u30bf\u30d6\u30ea\u30c3\u30af \u2013 Apache Spark<\/a><\/li>\n<\/ol>\n<p>Apache Spark \u306f\u9032\u5316\u3092\u7d9a\u3051\u3001\u30d3\u30c3\u30b0\u30c7\u30fc\u30bf\u74b0\u5883\u306b\u9769\u547d\u3092\u3082\u305f\u3089\u3057\u3001\u7d44\u7e54\u304c\u30c7\u30fc\u30bf\u304b\u3089\u8cb4\u91cd\u306a\u6d1e\u5bdf\u3092\u8fc5\u901f\u304b\u3064\u52b9\u7387\u7684\u306b\u5f15\u304d\u51fa\u3059\u3053\u3068\u304c\u3067\u304d\u308b\u3088\u3046\u306b\u3057\u307e\u3059\u3002\u30c7\u30fc\u30bf \u30b5\u30a4\u30a8\u30f3\u30c6\u30a3\u30b9\u30c8\u3001\u30a8\u30f3\u30b8\u30cb\u30a2\u3001\u30d3\u30b8\u30cd\u30b9 \u30a2\u30ca\u30ea\u30b9\u30c8\u306e\u3044\u305a\u308c\u3067\u3042\u3063\u3066\u3082\u3001Apache Spark \u306f\u30d3\u30c3\u30b0 \u30c7\u30fc\u30bf\u306e\u51e6\u7406\u3068\u5206\u6790\u306e\u305f\u3081\u306e\u5f37\u529b\u3067\u67d4\u8edf\u306a\u30d7\u30e9\u30c3\u30c8\u30d5\u30a9\u30fc\u30e0\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002<\/p>","protected":false},"featured_media":467620,"menu_order":0,"template":"","meta":{"_acf_changed":false,"content-type":"","inline_featured_image":false,"footnotes":""},"class_list":["post-475880","wiki","type-wiki","status-publish","has-post-thumbnail","hentry"],"acf":{"faq_title":"Frequently Asked Questions about <mark>Apache Spark: A Comprehensive Guide<\/mark>","faq_items":[{"question":"What is Apache Spark?","answer":"<p>Apache Spark is an open-source distributed computing system designed for big data processing and analytics. It provides fast in-memory processing, fault tolerance, and supports multiple programming languages for data processing applications.<\/p>"},{"question":"How did Apache Spark originate?","answer":"<p>Apache Spark originated from research efforts at the AMPLab, University of California, Berkeley, and was first mentioned in a research paper titled \"Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing\" in 2012.<\/p>"},{"question":"What is the internal structure of Apache Spark?","answer":"<p>At the core of Apache Spark is the concept of Resilient Distributed Datasets (RDDs), which are immutable distributed collections of objects processed in parallel. Spark's ecosystem includes Spark Core, Spark SQL, Spark Streaming, MLlib, and GraphX.<\/p>"},{"question":"What are the key features of Apache Spark?","answer":"<p>The key features of Apache Spark include in-memory processing, fault tolerance, ease of use with various APIs, versatility with multiple libraries, and superior processing speed.<\/p>"},{"question":"What are the types of Apache Spark?","answer":"<p>Apache Spark can be categorized into batch processing, stream processing, machine learning, and graph processing.<\/p>"},{"question":"What are the ways to use Apache Spark?","answer":"<p>Apache Spark finds applications in data analytics, machine learning, recommendation systems, and real-time event processing. Some common challenges include memory management, data skew, and cluster sizing.<\/p>"},{"question":"How does Apache Spark compare to Hadoop MapReduce?","answer":"<p>Apache Spark excels in in-memory and iterative processing, supports real-time analytics, offers a more diverse ecosystem, and is user-friendly compared to Hadoop MapReduce's disk-based batch processing and limited ecosystem.<\/p>"},{"question":"What are the future perspectives for Apache Spark?","answer":"<p>The future of Apache Spark looks promising with ongoing optimizations, deeper integration with AI, and advancements in real-time analytics.<\/p>"},{"question":"How can proxy servers be associated with Apache Spark?","answer":"<p>Proxy servers can enhance Apache Spark's security and performance by providing load balancing, caching, and acting as intermediaries between users and Spark clusters.<\/p>"}]},"_links":{"self":[{"href":"https:\/\/oneproxy.pro\/jp\/wp-json\/wp\/v2\/wiki\/475880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oneproxy.pro\/jp\/wp-json\/wp\/v2\/wiki"}],"about":[{"href":"https:\/\/oneproxy.pro\/jp\/wp-json\/wp\/v2\/types\/wiki"}],"version-history":[{"count":0,"href":"https:\/\/oneproxy.pro\/jp\/wp-json\/wp\/v2\/wiki\/475880\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/oneproxy.pro\/jp\/wp-json\/wp\/v2\/media\/467620"}],"wp:attachment":[{"href":"https:\/\/oneproxy.pro\/jp\/wp-json\/wp\/v2\/media?parent=475880"}],"curies":[{"name":"\u3046\u30fc\u3093","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}