sigs.k8s.io controller-runtime——GetConfigOrDie()方法詳解

阿新 • • 發佈：2022-04-04

1、概述

在使用controller-runtime框架進行kubernetes二次開發的時候，程式中通常都會使用GetConfigOrDie()方法獲取叢集配置以便與kubernetes叢集進行連線，示例如下：

opts := ctrl.Options{
	Scheme: scheme,
	MetricsBindAddress: metricsAddr,
	LeaderElection: enableLeaderElection,
	Port: 9443,
}

mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), opts)

本文主要通過剖析GetConfigOrDie()方法來進行講解controller-runtime框架的獲取kubernetes叢集認證配置檔案的載入順序。

2、 GetConfigOrDie()原始碼剖析

原始碼檔案位置：sigs.k8s.io/controller-runtime/pkg/client/config/config.go

注意：本文中不特殊說明原始碼檔案位置的程式碼片段都是sigs.k8s.io/controller-runtime/pkg/client/config/config.go檔案。

// GetConfigOrDie creates a *rest.Config for talking to a Kubernetes apiserver.
// If --kubeconfig is set, will use the kubeconfig file at that location.  Otherwise will assume running
// in cluster and use the cluster provided kubeconfig.
//
// Will log an error and exit if there is an error creating the rest.Config.
func GetConfigOrDie() *rest.Config {
	config, err := GetConfig()
	if err != nil {
		log.Error(err, "unable to get kubeconfig")
		os.Exit(1)
	}
	return config
}

可以看到GetConfigOrDie()方法如同它的名字，假設獲取*rest.Config失敗便會以異常狀態退出程式。

func GetConfig() (*rest.Config, error) {
	return GetConfigWithContext("")
}

func GetConfigWithContext(context string) (*rest.Config, error) {
	cfg, err := loadConfig(context)
	if err != nil {
		return nil, err
	}

	if cfg.QPS == 0.0 {
		cfg.QPS = 20.0
		cfg.Burst = 30.0
	}

	return cfg, nil
}

以上方法可以配置*rest.Config.QPS，訪問kubernetes叢集時的認證配置檔案的載入順序核心邏輯方法為loadConfig方法，邏輯如下：

var (
	kubeconfig string
	log        = logf.RuntimeLog.WithName("client").WithName("config")
)

func init() {
	// 啟動程式時客戶端傳參kubeconfig
	flag.StringVar(&kubeconfig, "kubeconfig", "",
		"Paths to a kubeconfig. Only required if out-of-cluster.")
}

// loadConfig loads a REST Config as per the rules specified in GetConfig.
func loadConfig(context string) (*rest.Config, error) {
	// If a flag is specified with the config location, use that
	// 1、如果flag初始化了kubeconfig，則從kubeconfig中讀取叢集配置
	if len(kubeconfig) > 0 {
		return loadConfigWithContext("", &clientcmd.ClientConfigLoadingRules{ExplicitPath: kubeconfig}, context)
	}

	// If the recommended kubeconfig env variable is not specified,
	// try the in-cluster config.
	// 2、 否則從環境變數KUBECONFIG讀取，若沒有則從叢集內部讀取，這種場景適用於部署到kubernetes中的場景，
	kubeconfigPath := os.Getenv(clientcmd.RecommendedConfigPathEnvVar)
	if len(kubeconfigPath) == 0 {
		if c, err := loadInClusterConfig(); err == nil {
			return c, nil
		}
	}

	// If the recommended kubeconfig env variable is set, or there
	// is no in-cluster config, try the default recommended locations.
	//
	// NOTE: For default config file locations, upstream only checks
	// $HOME for the user's home directory, but we can also try
	// os/user.HomeDir when $HOME is unset.
	//
	// TODO(jlanford): could this be done upstream?
	loadingRules := clientcmd.NewDefaultClientConfigLoadingRules()
	// 是否存在環境變數HOME
	if _, ok := os.LookupEnv("HOME"); !ok {
		u, err := user.Current()
		if err != nil {
			return nil, fmt.Errorf("could not get current user: %v", err)
		}
		// 3、如果通過KUBECONFIG環境變數和叢集內部這兩個方式都沒有找到（包括從叢集內部讀取叢集配置報錯），則會讀取預設配置。即：user.HomeDir/.kube/config
		loadingRules.Precedence = append(loadingRules.Precedence, path.Join(u.HomeDir, clientcmd.RecommendedHomeDir, clientcmd.RecommendedFileName))
	}

	return loadConfigWithContext("", loadingRules, context)
}

func loadConfigWithContext(apiServerURL string, loader clientcmd.ClientConfigLoader, context string) (*rest.Config, error) {
	return clientcmd.NewNonInteractiveDeferredLoadingClientConfig(
		loader,
		&clientcmd.ConfigOverrides{
			ClusterInfo: clientcmdapi.Cluster{
				Server: apiServerURL,
			},
			CurrentContext: context,
		}).ClientConfig()
}

接下來再看下從叢集內部和通過環境變數（包括讀取預設配置）獲取叢集配置的程式碼邏輯。

1）叢集內部獲取叢集配置：

InClusterConfig方法,邏輯很簡單：在Pod容器內，通過serviceaccout提供的token值和容器內部叢集的環境變數生成*rest.Config物件。

InClusterConfig方法原始碼檔案位置：k8s.io/client-go/rest/config.go

// InClusterConfig returns a config object which uses the service account
// kubernetes gives to pods. It's intended for clients that expect to be
// running inside a pod running on kubernetes. It will return ErrNotInCluster
// if called from a process not running in a kubernetes environment.
func InClusterConfig() (*Config, error) {
	const (
		tokenFile  = "/var/run/secrets/kubernetes.io/serviceaccount/token"
		rootCAFile = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
	)
	host, port := os.Getenv("KUBERNETES_SERVICE_HOST"), os.Getenv("KUBERNETES_SERVICE_PORT")
	if len(host) == 0 || len(port) == 0 {
		return nil, ErrNotInCluster
	}

	token, err := ioutil.ReadFile(tokenFile)
	if err != nil {
		return nil, err
	}

	tlsClientConfig := TLSClientConfig{}

	if _, err := certutil.NewPool(rootCAFile); err != nil {
		klog.Errorf("Expected to load root CA config from %s, but got err: %v", rootCAFile, err)
	} else {
		tlsClientConfig.CAFile = rootCAFile
	}

	return &Config{
		// TODO: switch to using cluster DNS.
		Host:            "https://" + net.JoinHostPort(host, port),
		TLSClientConfig: tlsClientConfig,
		BearerToken:     string(token),
		BearerTokenFile: tokenFile,
	}, nil
}

2）通過環境變數（包括讀取預設配置）獲取叢集配置：

不管是flag初始化了kubeconfig，還是從環境變數KUBECONFIG讀取，或者是讀取預設配置（$HOME/.kube/config) 都會例項化ClientConfigLoadingRules結構體物件。

本小節原始碼檔案位置：k8s.io/client-go/tools/clientcmd/loader.go

type ClientConfigLoadingRules struct {
	ExplicitPath string
	Precedence   []string

	// MigrationRules is a map of destination files to source files.  If a destination file is not present, then the source file is checked.
	// If the source file is present, then it is copied to the destination file BEFORE any further loading happens.
	MigrationRules map[string]string

	// DoNotResolvePaths indicates whether or not to resolve paths with respect to the originating files.  This is phrased as a negative so
	// that a default object that doesn't set this will usually get the behavior it wants.
	DoNotResolvePaths bool

	// DefaultClientConfig is an optional field indicating what rules to use to calculate a default configuration.
	// This should match the overrides passed in to ClientConfig loader.
	DefaultClientConfig ClientConfig

	// WarnIfAllMissing indicates whether the configuration files pointed by KUBECONFIG environment variable are present or not.
	// In case of missing files, it warns the user about the missing files.
	WarnIfAllMissing bool
}

其中：

ExplicitPath: 表示kubeconfig的路徑，優先順序低於Precedence；
Precedence: 表示從KUBECONFIG等環境變數中獲取到的路徑；
MigrationRules: 表示舊路徑到新路徑的對映關係；
DoNotResolvePaths: 表示是否需要把相對路徑轉換成絕對路徑；
DefaultClientConfig: 表示預設的配置。

建立ClientConfigLoadingRules結構體物件方法NewDefaultClientConfigLoadingRules()

const (
	RecommendedConfigPathFlag   = "kubeconfig"
	RecommendedConfigPathEnvVar = "KUBECONFIG"
	RecommendedHomeDir          = ".kube"
	RecommendedFileName         = "config"
	RecommendedSchemaName       = "schema"
)

var (
	RecommendedConfigDir  = filepath.Join(homedir.HomeDir(), RecommendedHomeDir)
	RecommendedHomeFile   = filepath.Join(RecommendedConfigDir, RecommendedFileName)
	RecommendedSchemaFile = filepath.Join(RecommendedConfigDir, RecommendedSchemaName)
)

// NewDefaultClientConfigLoadingRules returns a ClientConfigLoadingRules object with default fields filled in.  You are not required to
// use this constructor
func NewDefaultClientConfigLoadingRules() *ClientConfigLoadingRules {
	chain := []string{}
	warnIfAllMissing := false

	envVarFiles := os.Getenv(RecommendedConfigPathEnvVar)
	if len(envVarFiles) != 0 {
		fileList := filepath.SplitList(envVarFiles)
		// prevent the same path load multiple times
		chain = append(chain, deduplicate(fileList)...)
		warnIfAllMissing = true

	} else {
		chain = append(chain, RecommendedHomeFile)
	}

	return &ClientConfigLoadingRules{
		Precedence:       chain,
		MigrationRules:   currentMigrationRules(),
		WarnIfAllMissing: warnIfAllMissing,
	}
}

3、總結

controller-runtime框架通過GetConfigOrDie()方法獲取kubernetes叢集認證配置檔案的載入順序如下：

1、如果flag初始化了kubeconfig，則從kubeconfig中讀取叢集配置；
2、否則從環境變數KUBECONFIG讀取，若沒有則從叢集內部讀取，這種場景適用於部署到kubernetes中的場景；
3、如果通過KUBECONFIG環境變數和叢集內部這兩個方式都沒有找到（包括從叢集內部讀取叢集配置報錯），則會讀取預設配置。即：user.HomeDir/.kube/config。