What's your ram usage for this? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

qumpis on March 12, 2023 | parent | context | favorite | on: Large language models are having their Stable Diff...

What's your ram usage for this?

JoeMattie on March 12, 2023 | [–]

I've got the (4-bit quantized) 65B param model running at somewhat acceptable speed on an i9-7900. It uses around 55GB of RAM.

ddren on March 12, 2023 | [–]

The (quantized) 13B model is 7.6 GB on disk and the program uses around 8 GB to run. It runs without hitting the swap with just 9 GB assigned to WSL2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact