use set(), it takes either model instances or primary keys.
set()
For many-to-many relationships set() accepts a list of either model instances or field values, normally primary keys, as the objs argument.
See also django doc for set()