与服务器一起的日子

  • mysql
  • linux
  • 高可用
  • nginx
与服务器一起的日子
冰冷的机器也熄不灭火热的心
  1. 首页
  2. 大数据
  3. hadoop
  4. 正文

hadoop技术探秘05——hadoop源码简析

2017年12月5日 666点热度 0人点赞 0条评论

在我们编译源代码的时候,当我们构建成功的时候会输出一些信息

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hadoop Main ................................. SUCCESS [02:33 min]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [01:35 min]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 51.731 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 36.272 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.128 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [01:05 min]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 44.651 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [13:35 min]
[INFO] Apache Hadoop Auth ................................. SUCCESS [09:11 min]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  9.510 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [11:00 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [  1.408 s]
[INFO] Apache Hadoop KMS .................................. SUCCESS [01:47 min]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.045 s]
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [05:49 min]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [01:12 min]
[INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [  0.970 s]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 18.891 s]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [ 54.711 s]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  1.455 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.038 s]
[INFO] Apache Hadoop YARN ................................. SUCCESS [  0.038 s]
[INFO] Apache Hadoop YARN API ............................. SUCCESS [  8.119 s]
[INFO] Apache Hadoop YARN Common .......................... SUCCESS [07:25 min]
[INFO] Apache Hadoop YARN Server .......................... SUCCESS [  0.036 s]
[INFO] Apache Hadoop YARN Server Common ................... SUCCESS [01:23 min]
[INFO] Apache Hadoop YARN Registry ........................ SUCCESS [  1.004 s]
[INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [  4.236 s]
[INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [  0.631 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [ 36.280 s]
[INFO] Apache Hadoop YARN Timeline Service ................ SUCCESS [ 35.511 s]
[INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [  9.729 s]
[INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [  1.010 s]
[INFO] Apache Hadoop YARN Client .......................... SUCCESS [  2.120 s]
[INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [  0.803 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [  0.979 s]
[INFO] Apache Hadoop YARN Router .......................... SUCCESS [  1.107 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [08:02 min]
[INFO] Apache Hadoop YARN Timeline Service HBase tests .... SUCCESS [04:13 min]
[INFO] Apache Hadoop YARN Applications .................... SUCCESS [  0.032 s]
[INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [  0.853 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [  0.385 s]
[INFO] Apache Hadoop YARN Site ............................ SUCCESS [  0.030 s]
[INFO] Apache Hadoop YARN UI .............................. SUCCESS [  0.033 s]
[INFO] Apache Hadoop YARN Project ......................... SUCCESS [  8.972 s]
[INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [  0.117 s]
[INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [  4.210 s]
[INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [  2.598 s]
[INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [  0.686 s]
[INFO] Apache Hadoop MapReduce App ........................ SUCCESS [  2.710 s]
[INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [  1.744 s]
[INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [  3.607 s]
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [  0.472 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  1.160 s]
[INFO] Apache Hadoop MapReduce ............................ SUCCESS [  3.514 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 21.513 s]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [  3.392 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [  0.346 s]
[INFO] Apache Hadoop Archive Logs ......................... SUCCESS [  0.460 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [  1.026 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  0.892 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [  0.389 s]
[INFO] Apache Hadoop Ant Tasks ............................ SUCCESS [  0.156 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [  0.510 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [  0.029 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  0.643 s]
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [19:13 min]
[INFO] Apache Hadoop Azure support ........................ SUCCESS [ 27.780 s]
[INFO] Apache Hadoop Client ............................... SUCCESS [  6.349 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  0.860 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [  1.738 s]
[INFO] Apache Hadoop Resource Estimator Service ........... SUCCESS [ 16.888 s]
[INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [ 50.514 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 12.594 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.025 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [ 54.300 s]
[INFO] Apache Hadoop Cloud Storage ........................ SUCCESS [  0.591 s]
[INFO] Apache Hadoop Cloud Storage Project ................ SUCCESS [  0.024 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:38 h
[INFO] Finished at: 2017-12-06T00:01:59+08:00
[INFO] Final Memory: 196M/876M
[INFO] ------------------------------------------------------------------------

 

这些输出信息就是我们构建的组件,说白了就是我们的hadoop使用到的模块。

通过上面的模块,我们可以一一解读我们阅读源码的位置和顺序

通过上面的输出,我们知道hadoop的组件主要有:

  • Main
  • Build Tools
  • Project POM
  • Annotations
  • Assemblies
  • Project Dist POM
  • Maven Plugins
  • MiniKDC
  • Auth
  • Auth Examples
  • Common
  • NFS
  • KMS
  • Common Project
  • HDFS Client
  • HDFS
  • HDFS Native Client
  • HttpFS
  • HDFS BookKeeper Journal
  • HDFS-NFS
  • HDFS Project
  • YARN
  • YARN API
  • YARN Common
  • YARN Server
  • YARN Server Common
  • YARN Registry
  • YARN NodeManager
  • YARN Web Proxy
  • YARN ApplicationHistoryService
  • YARN Timeline Service
  • YARN ResourceManager
  • YARN Server Tests
  • YARN Client
  • YARN SharedCacheManager
  • YARN Timeline Plugin Storage
  • YARN Router
  • YARN TimelineService HBase Backend
  • YARN Timeline Service HBase tests
  • YARN Applications
  • YARN DistributedShell
  • YARN Unmanaged Am Launcher
  • YARN Site
  • YARN UI
  • YARN Project
  • MapReduce Client
  • MapReduce Core
  • MapReduce Common
  • MapReduce Shuffle
  • MapReduce App
  • MapReduce HistoryServer
  • MapReduce JobClient
  • MapReduce HistoryServer Plugins
  • MapReduce Examples
  • MapReduce
  • MapReduce Streaming
  • Distributed Copy
  • Archives
  • Archive Logs
  • Rumen
  • Gridmix
  • Data Join
  • Ant Tasks
  • Extras
  • Pipes
  • OpenStack support
  • Amazon Web Services support
  • Azure support
  • Client
  • Mini-Cluster
  • Scheduler Load Simulator
  • Resource Estimator Service
  • Azure Data Lake support
  • Tools Dist
  • Tools
  • Distribution
  • Cloud Storage
  • Cloud Storage Project

在后面的文章我们按照这个顺序来解读我们的hadoop源代码。

标签: 源代码 源码
最后更新:2017年12月5日

jhin

这个人很懒,什么都没留下

点赞
< 上一篇
下一篇 >

文章评论

razz evil exclaim smile redface biggrin eek confused idea lol mad twisted rolleyes wink cool arrow neutral cry mrgreen drooling persevering
取消回复

COPYRIGHT © 2024 与服务器一起的日子. ALL RIGHTS RESERVED.

Theme Kratos Made By Seaton Jiang