best practices: software engineering

Full Development Cycle

• backlog refinement meeting and estimations
• planning meeting
• business requirements epic and tickets (Happy Path, Unhappy Path)
• FE & BE contract: REST API Specs, GraphQL Schema (both mutation and resolver)
• Design documentation
• Infrastructure As Code Setup
• feature flags
• code documentation
• code logics implementation
  • clarify requirements in ticket
  • consider unhappy paths such as exception handling (ask business to requirement on unhappy path)
• lints, unittests, code reviews
• developer manual e2e tests
• QA / functional testings (deploy to test env, setup test accounts)
• deployment and rollbacks
• e2e tests in production to check production configuration
• monitoring
  • engineering: errors, performance, retrospective debugging logs,
  • operation: CPU, Memory, number of invokes, latency, throughputs,
• iteration on released feature
• software support, BA warranty period

Regular TODO

• prod hotfix with security patches
• monthly upgrade dependencies
• maintain documentation + index for documentation

Database Performance and Concurrency

• consider transaction
  • multiple save
  • select + save, submit endpoint to avoid introducing concurrent overwritten issue? consider select_for_update!
• care about DB Performance!
  • use self.assertNumQueries
  • print(queryset.query) or CaptureQueriesContext
  • optimize queries with select_related and prefetch_related
• express relative performance gain or decrease by Time old/Time new or Time before/Time after (CS:APP P.59)

Concurrency Problem

• submission endpoints especially submitting to third party API, must handle concurrent requests by using select_for_update, e.g. user can easily trigger multiple submission by clicking button multiple times
  • see notes Django Concurrency Problems

Python Exception Design

Each feature should have a base class with specific errors inherited from this base class, so that caller only needs to catch 1 error, the base error.

For example, base exception OneFeeCalculatorError, inherited by exceptions UnexpectedCalculatorDataError.