That is all good and well against simple actors, but does not defend against the company being bought out and the new owner(s) deciding to decrypt and sell the entire database to some shady data broker. Or against the FBI/your local equivalent coming over for a friendly visit and confiscating both the hardware token and the database.