Anyone not doing distributed training across thousands of GPU, pytorch is the best option. Others have an infra team to work on it.
ansk 8 hours ago [-]
This kind of sounds like just giving up on making the compiler better.
ezyang 30 minutes ago [-]
One of the reasons why I have been thinking and writing this topic is because there is indeed a malaise among the team, which is along the lines of "AI agents are blowing up coding everywhere, what place does a compiler have in this brave new world?" If compilers truly are pointless, then I would like to update on this and save people time and grief!
But I also don't think compilers are pointless. In this post, which looks at the maximalist AI situation, the compiler resurrects itself as the verifier: the two types of systems do very similar things! You can also defend compilers on more banal grounds: not everyone has a ton of kernel engineers to hand optimize your kernels; token efficient ways of getting good baselines is useful; there's more to ML than transformer models and compiler leverage is really useful there. There's also a historical point which is that torch.compile... by all objective metrics, has really been quite successful!
I do agree it's hard to tell the future. These days, I ask myself, "Do I think this is likely to happen in six months." I don't think compilers die in six months. Ask me again in six months :)
pjmlp 12 minutes ago [-]
The folks submitting papers to SIGPLAN and PLDI seem to be the opinion use of AI technology in compilers will only increase merging both approaches, of natural language as frontend, and backends.
I think many that don't follow compilers haven't yet grasp how much there is already there, from machine learning based optimisation passes, JIT compiler heuristics, and so on, which never have been deterministic to start with.
antonvs 6 hours ago [-]
The point is there are limits on how much better the compilers can get, without modifying the language to be closer to the target environments. And there are multiple target environments, so a one-size-fits-all solution is not really possible. You have to compromise somewhere - you can’t have a single high level language that automatically produces maximally efficient code for every target, because the targets are not general-purpose computing environments, they have significant constraints.
If what you’re looking for is a way to avoid having to get into the operational details of the targets, and just provide a high level description that somehow gets figured out, then an LLM-based solution is almost certainly going to be your best bet.
madhu_ghalame 8 hours ago [-]
The idea makes sense because researchers already think in PyTorch. The tooling should adapt to them, not the other way around.
But I also don't think compilers are pointless. In this post, which looks at the maximalist AI situation, the compiler resurrects itself as the verifier: the two types of systems do very similar things! You can also defend compilers on more banal grounds: not everyone has a ton of kernel engineers to hand optimize your kernels; token efficient ways of getting good baselines is useful; there's more to ML than transformer models and compiler leverage is really useful there. There's also a historical point which is that torch.compile... by all objective metrics, has really been quite successful!
I do agree it's hard to tell the future. These days, I ask myself, "Do I think this is likely to happen in six months." I don't think compilers die in six months. Ask me again in six months :)
I think many that don't follow compilers haven't yet grasp how much there is already there, from machine learning based optimisation passes, JIT compiler heuristics, and so on, which never have been deterministic to start with.
If what you’re looking for is a way to avoid having to get into the operational details of the targets, and just provide a high level description that somehow gets figured out, then an LLM-based solution is almost certainly going to be your best bet.