Our constant endeavour at Hotstar, has been to deliver amazing experiences digitally, to millions of customers in the most frictionless way possible. These are those experiences that are not explained by words but are felt, when you are binge watching a show or when you are downloading a content while travelling or when you are watching live cricket in a low network zone or when you are watching Avengers on your living room home theatre setup. (Lot of “ORs” right. No more. But I think you get the idea)

我們在Hotstar的不懈努力，就是以最輕鬆的方式以數字方式為數百萬客戶提供出色的體驗。 這些是當您狂暴地觀看節目或旅行時下載內容，在低網路區域觀看現場板球或在自己的地方觀看復仇者時無法用言語解釋的體驗客廳家庭影院設定。 (很多“ OR”對。沒有。但是我想你明白了。)

To a customer, an experience is a sum of content and the application that delivers it with all its bells and whistles. To ensure good quality quality, we have to get it right, everytime, across platforms and technologies, that span, a diverse ecosystem of mobile, web, Smart TVs and set top boxes.

對客戶而言，體驗是內容和應用程式的總和，它可以帶給您無盡的歡樂。 為了確保高質量，我們必須跨平臺，跨技術，跨移動，Web，智慧電視和機頂盒的多樣化生態系統，始終做到最好。

We need to be able to consistently test, repeatedly, 24/7, pushing our code to boundaries so that it mimics user behavior, contexts and application states that a user experiences. To do this right and with confidence, we wanted to invest in automation technologies which are scalable, reliable and provide depth in how they can exploit the application under test as we take it through the paces.

我們需要能夠始終如一地反覆測試24/7，將我們的程式碼推向邊界，以便它模仿使用者所體驗的使用者行為，上下文和應用程式狀態。 為了正確而有信心地做到這一點，我們希望投資自動化技術，這些技術具有可伸縮性，可靠性，並隨著我們的發展逐步深入地介紹了它們如何利用被測應用程式。

While Mickey, Aladdin, and the Avengers took care of high quality content part of the problem (aren’t they just awesome!), we focussed on experimenting with different automation frameworks to get quality thoroughly analysed for the applications.

當米奇，阿拉丁和復仇者聯盟解決了問題的高品質內容時(不是太棒了！)，我們專注於嘗試使用不同的自動化框架來對應用程式進行全面的質量分析。

This blog talks about our journey in choosing XCUITest as the Mjölnir (Hammer of Thor, if you were still wondering) of choice.

該部落格討論了我們選擇XCUITest作為Mjölnir ( 雷神之錘，如果您仍然想知道的話 )的過程。

學會取消學習 (Learn to Unlearn)

Like most others in the industry, we evaluated Appium as a tool of choice. We quickly found issues with automation-at-scale, going deeper than the surface tests and manipulating the Application Under Test (AUT). Of course, Appium works handsomely on blackbox kind of testing but then, we weren’t looking to stop at surface tests.

與行業中的大多數其他公司一樣，我們將Appium視為首選工具。 我們很快發現了大規模自動化的問題，它比表面測試要深入得多，並且要處理被測應用程式(AUT)。 當然，Appium在黑盒測試方面做得不錯，但是，我們並不想停止表面測試。

Some details on why Appium didn’t work for us:

有關為何Appium對我們不起作用的一些詳細資訊：

• Complex Application Design + Flakier at Scale: Our application deals with multi level layered architecture of UI and it was difficult to recognise elements on Appium. Further we found ourselves restricted in simulating application states and the faster we wanted the test to run, the flakier it became.

  複雜的應用程式設計+ Flakier的規模：我們的應用程式處理UI的多層體系結構，很難識別Appium上的元素。 此外，我們發現自己在模擬應用程式狀態方面受到限制，並且我們希望測試執行得越快，它變得越發脆弱。

• Inability to mock unnecessary steps: Since Appium works as black box, it is not easy to reduce the use of redundant flows when the objective of the test is something else.

  無法模擬不必要的步驟：由於Appium充當黑匣子，因此當測試的目標是其他目的時，減少冗餘流的使用並不容易。

• Dynamic control on features: Appium can not control the behaviour of a feature to create the test scenario dynamically on runtime.

  動態控制功能： Appium無法控制功能的行為以在執行時動態建立測試方案。

• Tests are slow: Most importantly, Appium is built to support multiple platforms and languages, hence tests are slower in general.

  測試速度很慢：最重要的是，Appium構建為支援多種平臺和語言，因此測試速度通常較慢。

• Onboarding Curve: We needed a framework that was frictionless in onboarding our team, so that everyone could contribute to the effort. XCUITest was relatively easier to onboard our team.

  入職曲線：我們需要一個在加入我們的團隊時保持平穩的框架，以便每個人都可以為這項工作做出貢獻。 XCUITest相對容易加入我們的團隊。

快速失敗，快速學習！ (Fail Fast, Learn Fast!)

Automation is not a need, but a habit, so we started to channelise our habits in the correct direction.

自動化不是一種需要，而是一種習慣，因此我們開始朝著正確的方向引導我們的習慣。

解碼迷宮： (Decode the labyrinth:)

With the intent of moving fast and releasing often, it was needed that our tests are focussed and minimal enough to give us the required confidence in the product or feature.

為了快速移動和經常釋出，我們的測試重點是集中且最小化，以使我們對產品或功能具有所需的信心。

We wanted a medium to achieve our aspirations of:

我們想要一種媒介來實現我們的願望：

• Functional tests for focussed validation
  針對重點驗證的功能測試
• Mocking of components to deeply test interactions
  模擬元件以深入測試互動
• Easy integration with CI to get continuous feedback
  易於與CI整合以獲得持續的反饋
• Powerful enough to handle complexity
  功能強大，足以應付複雜性
• Stable enough to reduce flakiness
  足夠穩定以減少脫皮

切換到蘋果的XCTest框架： (Switch to Apple’s XCTest framework:)

We started using Apple’s XCUITest framework to write functional test cases. The strategy was to push testing deep into the process and system so that we fail early and reduce the turnaround time of a bug fix.

我們開始使用Apple的XCUITest框架編寫功能測試用例。 該策略是將測試深入到流程和系統中，以便我們儘早失敗並減少錯誤修復的週轉時間。

I know, Thanos made sense most of the time.

我知道，Thanos大多數時候都有意義。

With this decision, we invested in building the skill set of writing Swift code within the team and as we realised the full potential of the framework over the period of time, we started reaping the benefits of XCTest being native.

有了這個決定，我們投資了團隊內部的技能來編寫Swift程式碼。隨著我們認識到框架在一段時間內的全​​部潛力，我們開始從XCTest的本機中受益。

Some of them were:

其中一些是：

• Code is maintained inside the project itself, hence the maintenance is pretty easy with Xcode.
  程式碼在專案本身內部維護，因此使用Xcode維護非常容易。
• Developers get a sense of what is being tested and how, which helps them write test driven code.
  開發人員可以瞭解正在測試的內容以及測試的方式，這有助於他們編寫測試驅動的程式碼。
• Tests have great control over the features and they have the ability to tweak the environment as per the requirements.
  測試可以很好地控制這些功能，並且可以根據要求調整環境。
• Tests are much faster and reliable.
  測試更快，更可靠。
• Tests are more targeted to a functional flow
  測試更針對功能流程
• It is easier for any developer to run a subset of test while developing a feature so the quality gate is put on the first entry.
  任何開發人員在開發功能時都更容易執行測試的子集，因此將質量門放在第一位。

Here is a diagram that explains the architecture of our suite:

以下圖表說明了我們套件的體系結構：

Okay. Enough talking.

好的。 聊夠了。

我們是怎麼做的？ (How did we do it?)

問題陳述： (Problem statement:)

Let’s take a sample test:

讓我們進行樣本測試：

We wanted to run this test in multiple geographies and on different customer subscriptions to validate that our news clips are working fine.

我們希望在多個地區和不同的客戶訂閱中執行此測試，以驗證我們的新聞剪輯是否工作正常。

To achieve that, we designed a test suite that can process different configurations of the application and be smart enough to adjust accordingly.

為此，我們設計了一個測試套件，該套件可以處理應用程式的不同配置，並且足夠聰明以進行相應的調整。

Few dynamic configurations needed were:

所需的動態配置很少：

• Geographical regions: US, UK, Canada etc
  地理區域：美國，英國，加拿大等
• Environment: Production, Pre-production, QA etc
  環境：生產，預生產，質量檢查等
• User subscription type: Free, VIP, Premium
  使用者訂閱型別：免費，VIP，高階

And the hero in us nailed it right.

我們中的英雄把它釘得對。

啟動引數和環境變數： (Launch arguments and environment variables:)

XCUITest lets us pass string arguments or key,value pairs as launch arguments and environment variables respectively to the application under test.

XCUITest允許我們分別將字串引數或鍵，值對作為啟動引數和環境變數傳遞給被測應用程式。

Using this approach, we gained greater control over the application as we could make the application behave as we want as per our requirements at run time.

使用這種方法，我們可以對應用程式進行更大的控制，因為我們可以使應用程式在執行時按照要求執行。

測試計劃： (Test Plans:)

Another great utility that XCUITest provides for efficient management and configuration of test runs is XCTestPlan.

XCUITest提供的另一個有效的實用工具是XCTestPlan，它可以有效地管理和配置測試執行。

As shown above, using XCTestPlan, We created different suites of tests. With each suite, we added different configurations that we wanted to run the tests on.

如上所示，我們使用XCTestPlan建立了不同的測試套件。 對於每個套件，我們添加了要對其執行測試的不同配置。

For eg: We identified a collection of tests and added them to a new XCTestPlan called smoke test. Later we added configurations for each region based on the user type i.e. Smoke-US-PremiumUser and Smoke-UK-VIPUser.

例如：我們確定了測試的集合，並將它們新增到稱為煙霧測試的新XCTestPlan中。 後來，我們根據使用者型別(即Smoke-US-PremiumUser和Smoke-UK-VIPUser)為每個區域添加了配置。

The image above shows how we are able to run one test on different configurations with minimal effort.

上圖顯示了我們如何能夠以最小的努力對不同的配置執行一項測試。

結果 (Outcomes)

With the above plan and execution strategy, we saw great results in fast tracking our quality cycles and hence building the confidence in the product to release often.

通過上述計劃和執行策略，我們在快速跟蹤質量週期並因此建立了對經常釋出產品的信心方面取得了巨大成果。

分散式測試和並行執行： (Distributed tests and parallel runs:)

Using XCUITest’s parallel simulator runs, we are able to do painless distribution of tests across simulators and hence reducing the run time of the suite.

使用XCUITest的並行模擬器執行，我們可以在模擬器之間進行無痛的測試分配，從而減少套件的執行時間。

作為Github檢查執行測試： (Test runs as Github check:)

We identified the subset of tests that can give us enough confidence to merge new code and let it run on simulators. This reduced the occurrence of compilation errors and dead builds reaching the QA cycles.

我們確定了測試的子集，這些子集可以使我們有足夠的信心來合併新程式碼，並使其在模擬器上執行。 這減少了編譯錯誤的發生和達到QA週期的不可靠構建。

One-size-fits-all:

一應俱全：

With a proven framework built on iOS it was a cake walk to replicate our learnings and strategies to other Apple platforms like iPad and Apple TV.

藉助在iOS上構建的可靠框架，將我們的學習和策略複製到其他Apple平臺(如iPad和Apple TV)非常容易。

We built a similar solution for our Apple TV application and et voilà !

我們為Apple TV應用程式構建了類似的解決方案，等等！

As you say Iron man, let’s wrap up.

正如您說的鋼鐵俠，讓我們結束吧。

簡而言之 (In a Nutshell)

All these components come together for a single goal, raise the quality bar of our releases to infinity and beyond. In this conquest, XCUITest has been a paramount contributor to deliver a complete viewing experience on Apple platforms.

所有這些元件都彙集在一起​​以實現一個目標，將我們發行版的質量標準提高到了無限甚至更高。 在這一征服中，XCUITest一直是在Apple平臺上提供完整觀看體驗的最重要貢獻者。

With the belief of “Release fast, Release often”, we make use of these smaller and effective quality gates to make the release process completely automated, but that’s a topic for another blog.

秉持“快速釋出，經常釋出”的信念，我們利用這些較小且有效的質量門使釋出過程完全自動化，但這是另一個部落格的主題。

For now, keep binging for more awesomeness.

就目前而言，繼續加油以獲得更多真棒。

Till next time…

直到下一次…