Running Unit Test on Top of Serverless Service

Increase your parallelization by 50x

Adinata Thayib

Case Study Multi-Processing Public Cloud (AWS/Google/...) Test Libraries (pytest/nose/...) Tooling

See in schedule

Goals:
Attendees will learn how to utilize serverless architecture for a less common scenario - unit testing. They'll also learn on cost-benefit analysis related to increasing developer productivity.

Abstract:

I. Introduction (5 minutes)
A. Running > 2500 Python unit tests in Pocket Gems
B. How it dreads engineers and issues that happened because of it.

II. Serverless Introduction (7 minutes)
A. Example of serverless architecture usage: short-lived event-driven function, webserver with API gateway
B. Limitation of serverless architecture: no-system access level, 300s timeout limit, limited source size, frozen context

III. Case Study: Running Unit Tests using serverless (10 minutes)
A. Sharding Unit Tests: Created a nosetest plugin that accepts two additional parameters, the shard id and the total number of shards.
B. Test code packaging and upload: handling binary packaging, limiting source file size.
C. Concurrent execution and user-friendly output: spawning hundreds of cheap threads to invoke hundreds of processes, handling the returned result in a user-friendly way.
D. Limitation: network bandwidth, user concurrent usage.

IV. Cost-Benefit Analysis: (5 minutes)
A. 1-year usages report: being used by over 70 different engineers, with roughly 30 invocations/day.
B. Cost-benefit analysis: saved 12.5 hours/day of engineering time on average, which roughly equates to saving $10k/month, while costing less than $100/month.

V. Q&A (3 minutes)

Type: Talk (30 mins); Python level: Intermediate; Domain level: Intermediate


Adinata Thayib

Pocket Gems

Worked in Pocket Gems, a mobile game company.
We used Python for our backend architecture, data pipeline, and many of our tooling.