The first listed risk is why I shy away from solutions that depend on pinning threads to logical processors:
Several processes can decide to schedule threads on the same NUMA node. If each process has only one runnable goroutine, the NUMA node will be over-subscribed, while other nodes will be idle. To partially alleviate the problem, we can randomize node numbering within each process. Then the starting NODE0 refers to different physical nodes across [Go] processes.
Basically, your particular runtime system is probably not going to be the only thing running on a host. And even if it is, the kernel itself may choose to run things on particular logical processors, and it may not take into account what pinning you have done. For that reason, I find these approaches brittle. If your users know exactly how they're going to deploy applications (not you, since you're implementing a runtime system for user code), they can squeeze out some more performance, but all it can take is one extra process running on that host to mess it all up.
That's the difficulty with implementing runtime systems, and not applications: your runtime system has to work for (usually) arbitrary user code on (usually) arbitrary systems. If you're writing a single application, and you know exactly how and where it will run, thread-pin away. But when implementing a runtime system, you don't have that kind of luxury. You often have to leave performance on the floor for a small number of cases so that you don't hose it for most cases.
In principle, I think this kind of scheduling should be handled by the operating system itself. If the kernel does not have enough information to do it properly, then we can identify what information it would need, and devise an API to inform it. But the kernel is the only entity that always has global knowledge of everything running, and controls all of the resources. I find that a much more promising direction.
As some minor support, consider the recent paper "The Linux Scheduler: a Decade of Wasted Cores", https://news.ycombinator.com/item?id=11501493. My intuition is that runtime systems which perform thread pinning like this will tend to make such problems worse, since it constrains the kernel scheduler even more.
> Basically, your particular runtime system is probably not going to be the only thing running on a host.
I'm running Erlang, not Go, but basically the runtime is the only real thing running on our systems[1], so it's good for the runtime to pin its os-threads to specific logical processors. On the systems where this isn't the case (for example, when using a separate TLS termination daemon), it's easy to unpin the threads and let the OS manage where to run things.
[1] there's also monitoring, ntpd, sshd, getty, and network/disk processing in the kernel
I'm unfamiliar with Erlang's runtime, so please forgive some basic questions.
Is Erlang's runtime doing the thread pinning without any input from you? Or are you, at the application level, explicitly telling the Erlang runtime how to pin threads?
edit: Did some googling, looks like it's the latter: http://erlang.org/doc/man/erl.html#+sbt. There are a bunch of policy options where the user picks what behavior they think will work best with their application, on the current system. Key to my point, though is: The runtime system will by default not bind schedulers to logical processors.
Providing options where users opt-in to such behavior is good. But the Go proposal, as far as I read, was unilaterally proposing that is how the runtime would work, always. That's not good, for the reasons I stated.
Erlang/Go/Java VMs could probably benefit from some kind of "appliance mode" where they take over the whole machine and reconfigure the kernel for maximum performance, but I wouldn't want that mode to be the default.
Several processes can decide to schedule threads on the same NUMA node. If each process has only one runnable goroutine, the NUMA node will be over-subscribed, while other nodes will be idle. To partially alleviate the problem, we can randomize node numbering within each process. Then the starting NODE0 refers to different physical nodes across [Go] processes.
Basically, your particular runtime system is probably not going to be the only thing running on a host. And even if it is, the kernel itself may choose to run things on particular logical processors, and it may not take into account what pinning you have done. For that reason, I find these approaches brittle. If your users know exactly how they're going to deploy applications (not you, since you're implementing a runtime system for user code), they can squeeze out some more performance, but all it can take is one extra process running on that host to mess it all up.
That's the difficulty with implementing runtime systems, and not applications: your runtime system has to work for (usually) arbitrary user code on (usually) arbitrary systems. If you're writing a single application, and you know exactly how and where it will run, thread-pin away. But when implementing a runtime system, you don't have that kind of luxury. You often have to leave performance on the floor for a small number of cases so that you don't hose it for most cases.
In principle, I think this kind of scheduling should be handled by the operating system itself. If the kernel does not have enough information to do it properly, then we can identify what information it would need, and devise an API to inform it. But the kernel is the only entity that always has global knowledge of everything running, and controls all of the resources. I find that a much more promising direction.
As some minor support, consider the recent paper "The Linux Scheduler: a Decade of Wasted Cores", https://news.ycombinator.com/item?id=11501493. My intuition is that runtime systems which perform thread pinning like this will tend to make such problems worse, since it constrains the kernel scheduler even more.