They have a ‘secure environment’ but the testing app ‘Bluebook’ is also available for download for practice tests and even to the general public. They have a chromeOS PWA, a Mac Universal app, and a Windows version.
I can also personally attest that the app will run perfectly happy inside a Windows virtual machine, even inside the ‘secure testing mode’, as this is how I personally took the 11th grade PSAT a few months ago.
So what you're saying is someone could record their screen the entire time, with no one the wiser? It would be tricky to use any external programs because the Bluebook app might be able to detect that it's not in focus, and other students/proctors could see something else on the screen.
Yes. A virtual machine specifically bypasses any on-device protections, protections which the terms of service indicate they have. However, I think the primary deterrent to cheating here would be as you mentioned the proctor and the other students.
To get around this, I theorize a more subtle overlay could be fashioned (imagine some sort of screen reader that speaks chatGPT-generated answers through the provided earbuds). However, at that point you could spend your time studying for the test in the first place.
If this is (and remains) true, I would guess that someone will build it and sell it to others. There are enough people out there who will never get above a 1200 — with any amount of studying/coaching — and who would happily pay $2k for a guaranteed 1500+. You wouldn't even need an LLM, really. Just a cellular-enabled laptop that can take screenshots every few seconds and transmit them to a confederate who whispers in your ear. Or to be even less detectable, just faintly displays a letter at the bottom of the screen once the problem has been solved.
I can also personally attest that the app will run perfectly happy inside a Windows virtual machine, even inside the ‘secure testing mode’, as this is how I personally took the 11th grade PSAT a few months ago.