《Migrating to Cloud-Native Application Architectures》學習筆記之Chapter 2. Changes Needed
Cultural Change 文化變革
A great deal of the changes necessary for enterprise IT shops to adopt cloud-native architectures will not be technical at all. They will be cultural and organizational changes that revolve around eliminating structures, processes, and activities that create waste.
企業IT團隊採用雲原生架構所需變革的並不是技術上的。而是圍繞造成產生浪費的結構、過程和活動的文化和組織變革。
From Silos to DevOps 從獨立組織到DevOps
企業IT通常被組織成以下許多獨立組織:
- 軟體開發
- 質量保證
- 資料庫管理
- 系統管理
- IT操作
- 釋出管理
- 專案管理
These silos were created in order to allow those that understand a given specialty to manage and direct those that perform the work of that specialty. These silos often have different management hierarchies, toolsets,communication styles, vocabularies, and incentive structures.
這些獨立的組織的建立是為了讓那些瞭解特定專業的人能夠管理和指導執行該專業工作的人。這些組織通常具有不同的管理層次結構、工具集、溝通方式、
Collaboration, communication, and simple handoff of work product becomes tedious and painful at best, and absolutely chaotic (even dangerous) at worst. Enterprise IT often tries to “fix” the situation by creating heavyweight processes driven by ticket-based systems and committee meetings. And the enterprise IT value stream slows to a crawl under the weight of all of the nonvalue-adding waste.
低效地協作、溝通和簡單的工作產品交接方式,往好了說是乏味和痛苦的,往壞了說是絕對混亂的(甚至是危險的)。企業IT經常試圖通過建立由投票方式和委員會會議驅動的重量級流程來“修復”改變這種情況。企業IT價值流在所有非增值浪費的重壓下步履瞞珊。
At its heart, DevOps represents the idea of tearing down these silos and building shared toolsets, vocabularies, and communication structures in service of a culture focused on a single goal: delivering value rapidly and safely. Incentive structures are then created that reinforce and award behaviors that lead the organization in the direction of that goal. Bureaucracy and process are replaced by trust and accountability.
DevOps的核心思想是拆除獨立組織,構建共享的工具集、詞彙表和通訊結構,以服務於一個專注於單一目標的文化:快速安全地交付價值。然後建立激勵結構,以強化和獎勵那些引導組織朝著目標前進的行為。官僚主義和流程被信任和責任所取代。
From Punctuated Equilibrium to Continuous Delivery 從間斷均衡到持續交付
Rather than each iteration resulting in value delivered to the customer and valuable feedback pouring back into the development team, we continue a “punctuated equilibrium” style of delivery. Punctuated equilibrium actually short-circuits two of the key benefits of agile delivery:
- ustomers will likely go several weeks without seeing new value in the software. They perceive that this new agile way of working is just “business as usual,” and do not develop the promised increased trust relationship with the development team.
- Teams may go several weeks without real feedback.That feedback provides valuable course corrections that enable teams to “build the right thing.” By delaying this feedback, the likelihood that the wrong thing gets built only increases, along with the associated costly rework.
我們不是每次迭代都向客戶交付價值,並將有價值的反饋反饋回開發團隊,而是繼續“間斷均衡”交付風格。間斷的均衡實際上會縮短敏捷交付的兩個關鍵好處:
- 客戶可能會在數週內看不到該軟體的新價值。他們認為這種新的敏捷的工作方式只是“照常工作”,並沒有與開發團隊建立承諾的信任關係。
- 團隊可能幾周都沒有真正的反饋。這種反饋提供了有價值的課程修正,使團隊能夠“構建正確的東西”。通過延遲這種反饋,錯誤的東西被構建的可能性只會增加,隨之而來的是代價高昂的返工。
Gaining the benefits of cloud-native application architectures requires a shift to continuous delivery. Rather than punctuated equilibrium driven by a water scrumfall organization, we embrace the principles of value from end to end.
獲得雲原生應用程式架構的好處需要轉向持續交付。我們不是由一個水瀑布組織驅動的間斷均衡,而是從頭到尾擁抱價值原則。
The idea of “Concept to Cash”considers all of the activities necessary to carry a business idea from its conception to the point where it generates profit, and constructs a value stream aligning people and process toward the optimal achievement of that goal.
“從概念到現金”的概念考慮了將業務概念從概念到產生利潤點所必需的所有活動,並構建了一個價值流,將人員和流程調整到實現該目標的最佳狀態。
We technically support this way of working with the engineering practices of continuous delivery, where every iteration (in fact, every source code commit!) is proven to be deployable in an automated fashion.We construct deployment pipelines which automate every test which would prevent a production deployment should that test fail.
我們在技術上支援這種與持續交付的工程實踐一起工作的方式,在這種情況下,每個迭代(實際上,每個原始碼提交!)都被證明可以以自動化的方式部署。我們構建了自動化每個測試的部署管道,這將防止在測試失敗時進行生產部署。
Centralized Governance to Decentralized Autonomy 集中治理到分散自治
"瀑布文化"已經成為阻礙cloud-native發展的障礙。
Enterprises normally adopt centralized governance structures around application architecture and data management, with committees responsible for maintaining guidelines and standards, as well as approving individual designs and changes. Centralized governance is intended to help with a few issues:
- It can prevent widespread inconsistencies in technology stacks,decreasing the overall maintenance burden for the organization.
- It can prevent widespread inconsistencies in architectural choices, allowing for a common view of application development across the organization.
- Cross-cutting concerns like regulatory compliance can be handled in a consistent way for the entire organization
- Ownership of data can be determined by those who have a broad view of all organizational concerns
企業通常採用圍繞應用架構和資料管理的集中治理結構,委員會負責維護指導方針和標準,以及批准單獨的設計和更改。集中式治理旨在幫助解決以下幾個問題:
- 它可以防止技術棧大範圍出現不一致,減少組織的整體維護負擔。
- 它可以防止架構選型出現大範圍不一致,從而在整個組織中形成應用程式開發的公共觀點。
- 對於整個組織,可以一致地處理跨部門關切。
- 資料所有權可由具有全域性視野的人來決定
These structures are created with the belief that they will result in higher quality, lower costs, or both. However, these structures rarely result in the quality improvements or cost savings desired, and further prevent the speed of delivery sought from cloud-native application architectures.
之前說提到的那些措施,是因為我們相信它們將有助於提高質量、降低成本或兩者兼而有之。然而,這些結構很少能帶來所需的質量改進或成本節約,並且進一步阻礙了從雲原生應用程式架構中尋求的交付速度。
Adoption of cloud-native application architectures is almost always coupled with a move to decentralized governance. The teams building cloud-native applications own all facets of the capability they’re charged with delivering.They own and govern the data, the technology stack, the application architecture, the design of individual components, and the API contract delivered to the remainder of the organization. If a decision needs to be made, it’s made and executed upon autonomously by the team.
採用雲原生應用程式架構通常伴隨著向分散治理的遷移。構建雲原生應用程式的團隊擁有他們負責交付的功能的所有方面。它們擁有並管理資料、技術堆疊、應用架構、每個元件的設計以及交付給組織的API介面。如果需要做出決策,則由團隊自主制定並執行。
The decentralization and autonomy of individual teams is balanced by minimal, lightweight structures that are imposed on the integration patterns used between independently developed and deployed services (e.g., they prefer HTTP REST JSON APIs rather than many different styles of RPC).
獨立開發和部署的服務之間使用的整合模式(例如,他們更喜歡HTTP REST JSON api,而不是許多不同風格的RPC)上使用輕量級互動模式平衡各個團隊的分散和自治需要。
Organizational Change 組織變革
設計系統的組織,最終產生的設計等同於組織之內、之間的溝通結構。
—Melvyn Conway
利用『康威定律』進行組織變革。
Our solution is to create a team combining staff with many disciplines around each long-term product,instead of segregating staff that have a single discipline in each own team, such as testing.
我們的解決方案是在長週期的產品開發中,建立一個包含了各方面專業員工的團隊,而不是將他們分離在單一的團隊中,例如測試人員。
Business Capability Teams 業務能力團隊
Each of these tiers spans multiple identifiable business capabilities,making it very difficult to innovate and deploy features related to one business capability independently from the others.Companies seeking to move to cloud-native architectures like microservices segregated by business capability have often employed what Thoughtworks has called the Inverse Conway Maneuver.
Rather than building an architecture that matches their org chart, they determine the architecture they want and restructure their organization to match that architecture. If you do that, according to Conway, the architecture that you desire will eventually emerge.
每一層都跨越多個業務能力領域,因此很難獨立於其他業務功能創新和部署新功能。尋求遷移到雲本地架構(如按業務能力劃分的微服務)的公司,常常採用Thoughtworks所稱的逆康威策略。
他們沒有建立一個與其組織結構圖相匹配的架構,而是決定了他們想要的架構,並重組組織以匹配該架構。按照Conway的說法,如果你這樣做,你想要的架構最終會出現。
So, as part of the shift to a DevOps culture, teams are organized as crosfunctional, business capability teams that develop products rather than projects. Products are long-lived efforts that continue until they no longer provide value to the business.
因此,作為轉向DevOps文化的一部分,我們組織了跨職能、業務能力的團隊,開發的是產品而不再是專案。產品是長期存在的工作,直到它們不再為業務提供價值為止。
All of the roles necessary to build, test, deliver, and operate the service delivering a business capability are present on a team, which doesn’t hand off code to other parts of the organization.
構建、測試、交付和操作交付業務功能的服務所需的所有角色都存在於團隊中,團隊不會將程式碼傳遞給組織的其他部分。
Business capability teams own the entire development-to-operations lifecycle for their applications.
業務能力團隊掌握應用程式從開發到運營的整個生命週期。
The Platform Operations Team 平臺運營團隊
業務能力團隊需要依賴於自助敏捷基礎設施。
we can express a special business capability defined as “the ability to develop, deploy, and operate business capabilities.” This capability is owned by the platform operations team.
我們可以將一種特殊的業務能力定義為“開發、部署和運營的能力”。這個功能屬於平臺運營團隊。
The platform operations team operates the self-service agile infrastructure platform leveraged by the business capability teams. This team typically includes the traditional system, network, and storage administrator roles. If the company is operating the cloud platform on premises, this team also either owns or collaborates closely with teams managing the data centers themselves, and understands the hardware capabilities necessary to provide an infrastructure platform.
平臺運營團隊操作業務能力團隊利用的自助敏捷基礎架構平臺。這個團隊通常包括傳統的系統、網路和儲存管理員角色。如果公司在本地運行雲平臺,那麼這個團隊還擁有或與管理資料中心本身的團隊緊密合作,並且瞭解提供基礎設施平臺所需的硬體功能。
Just as the business capability teams collaborate with one another around defined API contracts, the platform operations team presents an API contract for the platform. Rather than queuing up requests for application environments and data services to be provisioned, business capability teams are able to take the leaner approach of building automated release pipelines that provision environments and services on-demand.
正如業務能力團隊圍繞已定義的API介面彼此協作一樣,平臺操作團隊也為平臺提供API介面。業務能力團隊能夠採用更精簡的方法來構建自動釋出管道,以按需提供環境和服務,而業務能力團隊不需要排隊等待應用程式環境和資料服務的服務。
Technical Change 技術變革
Decomposing Monoliths 拆分單體應用
Traditional n-tier, monolithic enterprise applications rarely operate well when deployed to cloud infrastructure, as they often make unsupportable assumptions about their deployment environment that cloud infrastructures simply cannot provide.
傳統的n層單體企業架構應用程式在部署到雲基礎設施時很少能很好地執行,因為它們常常對部署環境做出雲基礎設施無法提供的不可支援的需求。
Most of these assumptions are coupled with the fact that monoliths are typically deployed to long-lived infrastructure.
單體架構應用通常部署在壽命較長的基礎設施上並與之緊密結合。
But let’s assume that we could build a monolith that does not make any of these assumptions. We still have trouble:
- Monoliths couple change cycles together such that independent business capabilities cannot be deployed as required, preventing speed of innovation.
- Services embedded in monoliths cannot be scaled independently of other services, so load is far more difficult to account for efficiently.
- Developers new to the organization must acclimate to a new team, often learn a new business domain, and become familiar with an extremely large codebase all at once. This only adds to the typical 3–6 month ramp up time before achieving real productivity.
- Attempting to scale the development organization by adding more people further crowds the sandbox, adding expensive coordination and communication overhead.
- Technical stacks are committed to for the long term. Introducing new technology is considered too risky, as it can adversely affect the entire monolith.
單體架構應用面臨的問題:
- 整體架構將變更週期耦合在一起,這樣獨立的業務功能就無法按需部署,從而阻礙了創新的速度。
- 嵌入到單體應用中的服務不能獨立於其他服務進行伸縮,因此要有效地計算負載要困難得多。
- 新加入組織的開發人員必須適應新的團隊,經常要學習新的業務領域,並且一次熟悉一個非常大的程式碼庫。這隻會增加通常3-6個月的加速時間,然後才能實現真正的生產力。
- 試圖通過增加更多的人員來擴充套件開發組織會進一步擁擠沙箱,增加昂貴的協調和通訊開銷。
- 技術棧是長期致力於此的。引進新技術被認為風險太大,因為它會對整個整體產生不利影響。
The decomposition of the organization into business capability teams also requires that we decompose applications into microservices. Only then can we harness the maximum benefit from our move to cloud infrastructure.
將組織分解為業務能力團隊還需要將應用程式分解為微服務。只有這樣,我們才能最大地享受到向雲基礎設施遷移的好處。
Decomposing Data 拆分資料
前面拆分了單體應用,但這還遠遠不夠,資料模型也必須解耦。
For a domain model to be effective, it must also be internally consistent—we should not find terms or concepts with inconsistent definitions within the same model.
要使域模型有效,它還必須具有內部一致性——我們不應該在同一個模型中發現定義不一致的術語或概念。
It is incredibly difficult and costly (and arguably impossible) to create a federated domain model that does not suffer from such inconsistencies. Evans refers to internally consistent subsets of the overall domain model of the business as bounded contexts.
想要建立一個不受不一致性影響的領域模型是極其困難的(可以說是不可能的)。Evans將業務的整個域模型的內部一致子集稱為有界上下文。
Bounded contexts allow you to keep inconsistent definitions of a single concept across the organization, as long as they are defined consistently within the contexts themselves.
有界上下文允許您在整個組織中保持單個概念的不一致定義,只要它們在上下文中定義一致即可。
So we begin by identifying the segments of the domain model that can be made internally consistent. We’re then able to align business capability teams with these contexts, and those teams build microservices providing those capabilities.
首先確定可以在內部保持一致的域模型的各個部分,將業務能力團隊與領域驅動中上下文聯絡起來,這些團隊構建提供這些功能的微服務。
This definition of microservice provides a useful rubric for defining what your twelve-factor apps ought to be. Twelve-factor is primarily a technical specification, whereas microservices are primarily a business specification. We define our bounded contexts, assign them a set of business capabilities, commission capability teams to own those business capabilities, and have them build twelve-factor applications. The fact that these applications are independently deployable provides business capability teams with a useful set of technical tools for operation.
微服務的這一定義為定義12個因素的應用程式提供了一個有用的準則。12因素主要是技術規範,而微服務主要是業務規範。我們定義我們的邊界上下文,為它們分配一組業務功能,委託功能團隊擁有這些業務功能,並讓它們構建12個因素的應用程式。這些應用程式是可獨立部署的,這一事實為業務能力團隊提供了一組有用的操作技術工具。
We couple bounded contexts with the database per service pattern,where each microservice encapsulates, governs, and protects its own domain model and persistent store. In the database per service pattern, only one application service is allowed to gain access to a logical data store, which could exist as a single schema within a multitenant cluster or a dedicated physical database. Any external access to the concepts is made through a well-defined business contract implemented as an API (often REST but potentially any protocol).
我們將有界上下文與每個服務模式的資料庫結合起來,其中每個微服務封裝、管理和保護自己的域模型和持久儲存。在每個服務模式的資料庫中,只允許一個應用程式服務訪問邏輯資料儲存,邏輯資料儲存可以作為多租戶叢集或專用物理資料庫中的單個模式存在。對概念的任何外部訪問都是通過定義良好的業務契約實現的(通常是REST,但可能是任何協議)。
This decomposition allows for the application of polyglot persistence, or choosing different data stores based on data shape and read/write access patterns. However, data must often be recomposed via event-driven techniques in order to ask cross-context questions. Techniques such as command query responsibility segregation (CQRS) and event sourcing, beyond the scope of this report, are often helpful when synchronizing similar concepts across contexts.
這種分解允許應用多語言永續性,或者根據資料形狀和讀寫訪問模式選擇不同的資料儲存。然而,為了提出跨上下文的問題,通常必須通過事件驅動技術重新組合資料。在跨上下文同步類似概念時,命令查詢責任隔離(command query responsibility離析,CQRS)和事件來源等超出本報告範圍的技術通常很有用。
Containerization 容器化
Containers leverage modern Linux kernel primitives such as control groups (cgroups) and namespaces to provide similar resource allocation and isolation features as those provided by virtual machines with much less overhead and much greater portability. Application developers will need to become comfortable packaging applications as container images to take full advantage of the features of modern cloud infrastructure.
容器利用現代Linux核心原語,如控制組(control groups, cgroups)和名稱空間,提供與虛擬機器提供的資源分配和隔離特性類似的資源分配和隔離特性,開銷小得多,可移植性強得多。應用程式開發人員需要將應用程式輕鬆地打包為容器映像,以便充分利用現代雲基礎設施的特性。
From Orchestration to Choreography 從編制到編排
Not only must service delivery, data modeling, and governance be decentralized, but also service integration. Enterprise integration of services has traditionally been accomplished via the enterprise service bus (ESB). The ESB becomes the owner of all routing, transformation, policy, security, and other decisions governing the interaction between services. We call this orchestration, analogous to the conductor who determines the course of the music performed by an orchestra during its performance. ESBs and orchestration make for very simple and pleasing architecture diagrams, but their simplicity is deceiving. Often hiding within the ESB is a tangled web of complexity. Managing this complexity becomes a full-time occupation,
and working with it becomes a continual bottleneck for the application development team. Just as we saw with a federated data model,a federated integration solution like the ESB becomes a monolithic hazard to speed.
不僅服務交付、資料建模和治理必須分散,而且服務整合也必須拆解。傳統上,服務的企業整合是通過企業服務匯流排(ESB)完成的。ESB成為控制服務之間互動的所有路由、轉換、策略、安全性和其他決策的所有者。我們稱之為管絃樂編排,類似於指揮家在管絃樂隊的演奏中決定音樂的程序。ESB和業務流程可以生成非常簡單體系結構圖,但它們的簡單性具有欺騙性。通常隱藏在ESB中的是一個錯綜複雜的web。管理這種複雜性成為一項全職工作,使用它成為應用程式開發團隊的持續瓶頸。正如我們在領域資料模型中看到的那樣,ESB這樣的整合解決方案會對速度造成巨大的危害。
Cloud-native architectures, such as microservices, tend to prefer choreography, akin to dancers in a ballet. Rather than placing the smarts in the integration mechanism, they are placed in the endpoints, akin to the dumb pipes and smart filters of the Unix architecture. When circumstances on stage differ from the original plan,there’s no conductor present to tell the dancers what to do. Instead,they simply adapt. In the same way, services adapt to changing circumstances in their environment via patterns such as client-side load balancing and circuit breakers .
雲原生架構,如微服務,更傾向於編排,類似於芭蕾舞中的舞者。它們不是將智慧放在整合機制中,而是放在端點中,類似於Unix體系結構中的管道和過濾器。當舞臺上的情況與最初的計劃不同時,沒有指揮在場告訴舞者該做什麼。相反,他們只是適應。同樣,服務通過客戶端負載均衡和斷路器等模式來適應環境中的變化。
Choreography simply acknowledges and exposes the essential complexity of the system. Once again this shift is in support of the autonomy required to enable the speed sought from cloud-native architectures. Teams are able to adapt to their changing circumstances without the difficult overhead of coordinating with other teams, and avoid the overhead of coordinating changes with a centrally-managed ESB.
編排只是承認並公開了系統的本質複雜性。這種轉變再次支援實現從雲原生架構中尋求的速度所需的自主性。團隊能夠適應其不斷變化的環境,而無需與其他團隊協調的困難開銷,並避免使用集中管理的ESB協調更改的開銷。
總結
Culturally the overall theme is one of decentralization and autonomy:
- DevOps
- Decentralization of skill sets into cross-functional teams.
- Continuous delivery
- Decentralization of the release schedule and process.
- Autonomy
- Decentralization of decision making.
文化的主題是權力下放和自治:
- DevOps
- 將技能集分散到跨職能的團隊中。
- 持續交付
- 釋出計劃和過程的分散化。
- 自治
- 決策的分散化。
We codify this decentralization into two primary team structures:
- Business capability teams
Cross-functional teams that make their own decisions about design, process, and release schedule
- Platform operations teams
Teams that provide the cross-functional teams with the platform they need to operate.
我們將這種分散化整理成兩個主要的團隊結構:
- 業務能力的團隊
跨功能團隊對設計、流程和釋出計劃做出自己的決定
- 平臺運營團隊
為跨職能團隊提供他們需要操作的平臺的團隊。
And technically, we also decentralize control:
- Monoliths to microservices
Control of individual business capabilities is distributed to individual autonomous services.
- Bounded contexts
Control of internally consistent subsets of the business domain model is distributed to microservices.
- Containerization
Control of application packaging is distributed to business capability teams.
- Choreography
Control of service integration is distributed to the service endpoints.
從技術上講,我們也分散控制:
- 拆分單體應用為microservices
對單個業務功能的控制被分發到各個自治服務。
- 界定上下文
業務域模型內部一致子集的控制被分發給微服務。
- 容器化
應用程式打包的控制被分發給業務能力團隊。
- 編排
服務整合的控制被分發到服