Spark源码剖析(一):如何将spark源码导入到IDEA中

由于近期准备深入研究一下Spark的核心源码,所以开了这一系列用来记录自己研究spark源码的过程!

想要读源码,那么第一步肯定导入spark源码啦(笔者使用的是IntelliJ IDEA),在网上找了一圈,尝试了好几种方法都没有成功,最终通过自己摸索出了一种非常简单的方式( 只需要两步即可!

环境要求

  1. IntelliJ IDEA(Community版本即可)
  2. maven(当然jdk是不可少的)

具体信息如下:

C:UsersAdministrator>mvn -version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: D:javaapache-maven-3.3.9bin..
Java version: 1.8.0_151, vendor: Oracle Corporation
Java home: D:javajdk-1.8u151jre
Default locale: zh_CN, platform encoding: GBK
OS name: "windows 7", version: "6.1", arch: "amd64", family: "dos"

顺便贴一下maven的settings.xml







  <!-- localRepository
   | The path to the local repository maven will use to store artifacts.
   |
   | Default: ${user.home}/.m2/repository
  D:Mavenworkspacem2repository
  -->
D:javamavenRepository
  <!-- interactiveMode
   | This will determine whether maven prompts you when it needs input. If set to false,
   | maven will use a sensible default value, perhaps based on some other setting, for
   | the parameter in question.
   |
   | Default: true
  true
  -->

  <!-- offline
   | Determines whether maven should attempt to connect to the network when executing a build.
   | This will have an effect on artifact downloads, artifact deployment, and others.
   |
   | Default: false
  false
  -->

  
  
    <!-- pluginGroup
     | Specifies a further group identifier to use for plugin lookup.
    com.your.plugins
    -->
  

  
  
    <!-- proxy
     | Specification for one proxy, to be used in connecting to the network.
     |
    
      optional
      true
      http
      proxyuser
      proxypass
      proxy.host.net
      80
      local.net|some.host.com
    
    -->
  

  
  
    <!-- server
     | Specifies the authentication information to use when connecting to a particular server, identified by
     | a unique name within the system (referred to by the 'id' attribute below).
     |
     | NOTE: You should either specify username/password OR privateKey/passphrase, since these pairings are
     |       used together.
     |
    
      deploymentRepo
      repouser
      repopwd
    
    -->

    <!-- Another sample, using keys to authenticate.
    
      siteServer
      /path/to/private/key
      optional; leave empty if not used.
    
    -->
  

  
  
    <!-- mirror
     | Specifies a repository mirror site to use instead of a given repository. The repository that
     | this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
     | for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
     |
    
      mirrorId
      repositoryId
      Human Readable Name for this Mirror.
      http://my.repository.com/repo/path
    
     -->
     
        nexus-osc
        central
        Nexus osc
        http://maven.aliyun.com/nexus/content/groups/public/
    

    
    
        osc_thirdparty
        thirdparty
        http://maven.aliyun.com/nexus/content/repositories/central
    
  

  
  
    <!-- profile
     | Specifies a set of introductions to the build process, to be activated using one or more of the
     | mechanisms described above. For inheritance purposes, and to activate profiles via 
     | or the command line, profiles have to have an ID that is unique.
     |
     | An encouraged best practice for profile identification is to use a consistent naming convention
     | for profiles, such as 'env-dev', 'env-test', 'env-production', 'user-jdcasey', 'user-brett', etc.
     | This will make it more intuitive to understand what the set of introduced profiles is attempting
     | to accomplish, particularly when you only have a list of profile id's for debug.
     |
     | This profile example uses the JDK version to trigger activation, and provides a JDK-specific repo.
    
      jdk-1.4

      
        1.4
      

      
        
          jdk14
          Repository for JDK 1.4 builds
          http://www.myhost.com/maven/jdk14
          default
          always
        
      
    
    -->

               
        jdk-1.8
      
        true
        1.8
      
      
        1.8
        1.8
        1.8
      
        
        

            
                nexus
                local private nexus
                http://maven.aliyun.com/nexus/content/repositories/central
                
                    true
                
                
                    false
                
            
                   
        
        
        

            
                    nexus
                    local private nexus
                    http://maven.aliyun.com/nexus/content/repositories/central
                    
                        true
                    
                    
                        false
                    
            
        
        
    
    
             
        osc
            
                true
            
            
                    
        
            
                osc
                http://maven.aliyun.com/nexus/content/repositories/central
            
            
            
                osc_thirdparty
                http://maven.aliyun.com/nexus/content/repositories/central
            
            
        
        
        
        
            
                osc
                http://maven.aliyun.com/nexus/content/repositories/central
            

        
    
      
      
    
    
    <!--
     | Here is another profile, activated by the system property 'target-env' with a value of 'dev',
     | which provides a specific path to the Tomcat instance. To use this, your plugin configuration
     | might hypothetically look like:
     |
     | ...
     | 
     |   org.myco.myplugins
     |   myplugin
     |
     |   
     |     ${tomcatPath}
     |   
     | 
     | ...
     |
     | NOTE: If you just wanted to inject this configuration whenever someone set 'target-env' to
     |       anything, you could just leave off the  inside the activation-property.
     |
    
      env-dev

      
        
          target-env
          dev
        
      

      
        /path/to/tomcat/instance
      
    
    -->
  

  <!-- activeProfiles
   | List of profiles that are active for all builds.
   |
  
    alwaysActiveProfile
    anotherAlwaysActiveProfile
  
  -->


settings.xml

好了,一旦环境准备就绪,那就速战速决吧!

第一步:从github上下载源代码

先选择你想要阅读的spark版本,笔者这里选择的是spark1.3版本

接着直接下载zip包到本地解压(当然也可以使用git拉下来啦)

第二步:使用IDEA导入spark源码

打开你的IntelliJ IDEA ,File -> Open 选中你源码解压后的文件夹即可! ( 不需要使用Import

到这里基本已经大功告成! 接下来只需要等待maven解决各种依赖即可(大概需要半个小时,大家耐心一点)

成功后的界面如下(提示: 可以使用ctrl + N 搜索你想要阅读的类文件 ):

博客园精华区稿源:博客园精华区 (源链) | 关于 | 阅读提示

本站遵循[CC BY-NC-SA 4.0]。如您有版权、意见投诉等问题,请通过eMail联系我们处理。
酷辣虫 » 综合编程 » Spark源码剖析(一):如何将spark源码导入到IDEA中

喜欢 (0)or分享给?

专业 x 专注 x 聚合 x 分享 CC BY-NC-SA 4.0

使用声明 | 英豪名录