It might be difficult to impossible to currently implement, but perhaps if you have access to :///* you should lose access to pretty much every other permission.
That’s not how permissions work in Chrome extensions. You pretty much can’t do anything with the URL matching permission alone, however broad your pattern (including <all_urls>).
That wouldn't help much. For example, with read/write access to your banking website a malicious extension could inject a script tag that exfiltrated data to their server.
