Django Concurrency Problems

Assumptions for readers

Two Real Concurrency Issues Encountered In Real World

1. Dependent Service calls two REST APIs both write to the same record

# comment: Client Completed ATR

--- Change at: 2025-08-04 14:38:35.409104+00:00 by admin ---
date_updated: 2025-06-24 07:45:29.248338+00:00 → 2025-08-04 14:38:35.406381+00:00
value: None → 3
date_submitted: None → 2025-08-04 14:38:34+00:00
pending: True → False


# comment: Data Reverted

--- Change at: 2025-08-04 14:38:38.187071+00:00 by None ---
date_updated: 2025-08-04 14:38:37.144342+00:00 → 2025-08-04 14:38:36.952026+00:00
value: 3 → None
date_submitted: 2025-08-04 14:38:34+00:00 → None
pending: False → True

# comment: handle_put Curo Sync Happened

--- Change at: 2025-08-04 14:38:38.386802+00:00 by None ---
date_updated: 2025-08-04 14:38:36.952026+00:00 → 2025-08-04 14:38:38.384281+00:00
value: None → 3
date_submitted: None → 2025-08-04 00:00:00+00:00

Note this line date_updated: 2025-08-04 14:38:37.144342+00:00 → 2025-08-04 14:38:36.952026+00:00 where the date_updated is changed from later date or older date.

2. Use clicked submission button twice within a short amount of time

User clicked submission button twice within few seconds to trigger two call to submission mutation endpoints

This leads to handover being submitted twice, hence resulted in duplicated records.

Breaking Down Concurrency Problems Encountered

These two Concurrency problems boil down to there are two http requests both doing select, then update, there is no coordination between two requests.

Consider this simple example, A and B stand for request A and B

A: review = Review.objects.get(id=review_id)
B: review = Review.objects.get(id=review_id)

A: call third party api to submit review
B: call third party api to submit review
A: review.third_party_id = 'AAAA'
B: review.third_party_id = 'BBBB'

A: review.save()
B: review.save()

When Request A and B runs concurrently, this review might have third_party_id sets with either AAAA or BBBB.

What we really want is to correctly handle two concurrency requests modifying the same database entity, so the review.third_party_id is ONLY set once to either be AAAA or BBBB!

What we really really want is to correctly handle two concurrency requests to submission, to ensure there only 1 call to third party API endpoints. So we need to find a way to ensure submission logics (database write or third party API call) only ever happens once!

Wrong Solution

Naively check review.third_party_id cannot be None!

review = Review.objects.get(id=review_id)
if review.third_party_id:
    raise ValueError('This review is already submitted')
review.third_party_id = 'AAAA'
review.save()

# Request B
review = Review.objects.get(id=review_id)
if review.third_party_id:
    raise ValueError('This review is already submitted')
review.third_party_id = 'BBBB'
review.save()

So when Request A and B runs concurrently, this review might have third_party_id sets with either AAAA or BBBB.

Counter Example: On concurrent requests A and B, The third_party_id check can happen before the write.

A: review = Review.objects.get(id=review_id)
A: call third party api to submit review

B: review = Review.objects.get(id=review_id)
# Note that review.third_party_id is still None
B: call third party api to submit review

A: review.third_party_id = 'AAAA'
A: review.save()
B: review.third_party_id = 'BBBB'
B: review.save()

Wrong Solution

wrap two requests in a transaction.atomic block, and check for third_party_id

assert review.third_party_id == None
# Request A
with transaction.atomic():
    review = Review.objects.get(id=review_id)
    if review.third_party_id:
        raise ValueError('This review is already submitted')
    review.third_party_id = 'AAAA'
    review.save()

# Request B
with transaction.atomic():
    review = Review.objects.get(id=review_id)
    if review.third_party_id:
        raise ValueError('This review is already submitted')
    review.third_party_id = 'BBBB'
    review.save()

A transaction only ensures atomicity. i.e. atomicity If the block of code is successfully completed, the changes are committed to the database. If there is an exception, the changes are rolled back.

A transaction doesn’t handle concurrent requests by default!

So an counter example is exactly same as previous failure case, as follow.

A: review = Review.objects.get(id=review_id)
A: call third party api to submit review

B: review = Review.objects.get(id=review_id)
# Note that review.third_party_id is still None
B: call third party api to submit review

A: review.third_party_id = 'AAAA'
A: review.save()
B: review.third_party_id = 'BBBB'
B: review.save()

Correct Solution

What we really need is a way to ensure business logics from two concurrent requests only happen once!

i.e. in concurrent computing language, the business logics are critical section such that this critical section cannot be entered by two or more processes at the same time.

We need some sort of LOCK!

Django’s select_for_update acquires a database row-level lock which blocks another transaction trying to acquire the same row-level lock for this particular object.

FOR UPDATE causes the rows retrieved by the SELECT statement to be locked as though for update. This prevents them from being locked, modified or deleted by other transactions until the current transaction ends. That is, other transactions that attempt UPDATE, DELETE, SELECT FOR UPDATE, SELECT FOR NO KEY UPDATE, SELECT FOR SHARE or SELECT FOR KEY SHARE of these rows will be blocked until the current transaction ends; conversely, SELECT FOR UPDATE will wait for a concurrent transaction that has run any of those commands on the same row, and will then lock and return the updated row (or no row, if the row was deleted)

In our example, when the first request (either A or B) locks review with id = review_id, which will BLOCK the other concurrent request, it will block until the first request exit the transaction, then the second request will run.

# request A
with transaction.atomic():
    review = review.objects.select_for_update().get(id=review_id)
    if review.third_party_id:
        raise ValueError('This review is already submitted')
    review.third_party_id = 'AAAA'
    review.save()

# request B
with transaction.atomic():
    review = review.objects.select_for_update().get(id=review_id)
    if review.third_party_id:
        raise ValueError('This review is already submitted')
    review.third_party_id = 'BBBB'
    review.save()

IMPORTANT: You must use select_for_update is within a transaction with transaction.atomic()!

Due to the write lock by using select_for_update, oneview backend ensures there are only 2 possible ways of execution! So third_party_id can only be set to AAAA or BBBB! and critical business logics is only happened once!

# request A
review = review.objects.select_for_update().get(id=review_id)
if review.third_party_id: # review.third_party_id is None, skipping the Raise
    raise ValueError('This review is already submitted')
# call third party api to submit review
review.third_party_id = 'AAAA'
review.save()

# request B
review = review.objects.select_for_update().get(id=review_id)
# Note: this review has latest data with third_party_id not None!
if review.third_party_id:
    # return from here
    raise ValueError('This review is already submitted')
# DOES NOT call third party api to submit review
# request B
review = review.objects.select_for_update().get(id=review_id)
if review.third_party_id: # review.third_party_id is None, skipping the Raise
    raise ValueError('This review is already submitted')
# call third party api to submit review
review.third_party_id = 'BBBB'
review.save()

# request A
review = review.objects.select_for_update().get(id=review_id)
if review.third_party_id:
    raise ValueError('This review is already submitted')
# DOES NOT call third party api to submit review

To understand how exactly is select_for_update work, See more Official Postgres Documentation on FOR UPDATE row level lock

Recipe to guard against duplicate write

In a transaction,

  1. Use select_for_update to avoid concurrency write
  2. then do an idempotent check to ensure this action hasn’t happen
  3. use try except Exception to avoid unexpected transaction rollback? (unsure about this last step)

Why do we need step3 a try except??

The messy real world: not just 1 submission of review, rather many others submission so transaction takes longer, need to be careful to not raise errors in submission functions due to atomicity of transaction! Consider a solution is to have a wide try except Exception to catch all errors y one safe.

Is there alternative to select_for_update?

See also

2026 public praise from Katie Thorn

tags:reference, work,

I’d like to give a huge thank you to Tiger for consistently going above and beyond to make life easier for the Product team. Over the past few months, we’ve been dealing with an issue where the risk score for a secondary contact wasn’t pulling through to OneView. Whilst the underlying issue has now been resolved, we knew there would still be a handful of historic reviews affected. Rather than leaving us to raise bug tickets every time one cropped up, Tiger worked his magic and gave us the ability to self-serve those remaining cases. It means we can now fix the impacted reviews ourselves in a matter of moments instead of waiting for developer intervention. It might sound like a small change, but it will save us a huge amount of time and makes the whole process so much smoother. This is just one example of the way Tiger works. His investigation notes and root cause analyses are always incredibly detailed, making it so much easier for our team to understand not just what went wrong, but why. That level of detail really helps us communicate issues to the business, spot trends and improve our own knowledge. Thank you, Tiger, for always thinking beyond the immediate fix and looking for ways to make life easier for everyone else. Your technical expertise is invaluable, but it’s your willingness to share your knowledge and improve the wider process that really stands out.