Building Flash Attention from Source
Notes from a few rounds of compiling Flash Attention on an A800 server. I walk through checking that ninja is actually working, limiting the build to the GPU architecture I need, and balancing parallel jobs against RAM usage. If your build takes forever or ends with a “killed” error, these are the settings and pitfalls I wish I had checked first.