I've been always wondering why do we even need specialized build tools and why does the community around every new programming language or framework feel obliged to invent their own, implemented in their native language. Why can't we just write a script in Python or any other scripting language which would fetch the deps, invoke the compilers, run the tests and spit out the package?
I can't answer myself because I only have basic theoretical idea of what build tools do, I have always mostly been using IDEs like VisualStudio which do everything behind the scenes in a press of a button.
You're right, there could be a single tool that everyone agrees to use, and that would make programming much nicer as you would be able to learn one build tool, and then use whatever language you want with it.
What a build system needs to offer, IMHO as someone who also wrote one, is the following:
* tasks (incl. figuring out the graph of dependencies between them and caching results).
* dependencies. Each language has its own repository so it needs to be pluggable. It also may need to resolve conflicts (e.g. Java can't handle multiple versions of the same lib, which you can get due to transitive deps).
* custom options for each language. Native projects may choose -static or -dynlib, Java projects which bytecode version to target and so on.
* tests, documentation generation, linting etc.
Of course there's more to it, but this is the minimum you need. To accomplish all of these, you'll probably need a real programming language or something very close to it - and that's where the problem lies: which language??
XMake chose Lua, which is a very good choice for this (it's a small language but high level and powerful enough to do everything you may need). But if you're writing Java or JS, do you really want to write builds in Lua? Or just use the language-specific build system? I don't think the answer should be obvious, but reality says we chose the latter.
I would love if one day, something like XMake (basically, a DSL to build stuff that's based on a simple lingua-franca, like Lua) became the standard for all communities, but that's not going to happen unless something changes completely in the industry.
> I've been always wondering why do we even need specialized build tools and why does the community around every new programming language or framework feel obliged to invent their own, implemented in their native language. Why can't we just write a script in Python or any other scripting language which would fetch the deps, invoke the compilers, run the tests and spit out the package?
Not-invented-here and developers want to do stupid things in build systems. Not normally intentionally or maliciously, it's just that if you can do something and the build system does not complain about it, it's in.
Then you get 'build systems a la carte' - that are designed based on rigor and experience such as Bazel and buck2 and you start seeing that you can no longer do those things for very good reasons such as guaranteed reproducibility and hermetic builds.
On Minecraft (Bedrock, so C++) for years we had a janky python script that used various platform specific project templates to crudely generate a franken-solution.
Eventually we got enough of an effort to do the long task of adopting CMake for every platform. It's soo much nicer and more consistent than what we had and handles most of our platforms out of the box with little custom tweaking for project file generation.
I do like the idea of it using a common language though. Lua, JS, etc. would all be a nice choice. I am not a fan of the custom language just for building.
Acquiring dependencies is a hard problem. There are generic solutions to at least parts of this, stuff like nix and bazel, but most devs choose an “easier” tool that’s tightly integrated with their language ecosystem rather than one that brings a lot of seemingly unneeded upfront complexity.
It could all be python, but you would need many customization points to serve all the use cases needed to build natively compiled code in all its forms.
Language ecosystems like Rust and Go, and even Haskell shrink the problem space to a smaller universe to grab dependencies. You can see this by how awkward it would be to build a program containing those three languages. They all have different package repositories and build workflows.
C, Fortran, C++ have very rough tools, but the community wants, and often needs, a wide range of dependency management options. Like mixing languages. Or shipping to PYPI. Or integrating with exotic embedded build environments.
You are spot on actually, despite - or because? - of your very high level perspective.
I also agree with the why to invet yet another tool for the same thing kind of note. Could be that fresh graduates think they know much better than those worked this field for long and revolutionise/disrupt it with cool new fresh (all the usual adjectives here) utility? Make life better for everyone at last, finding the Saint Grail that all the makers of brand new cool fresh (other adjectives) utilities of the way past - one comment here told half a decade is too old - could not? I don't know, but it is interesting seeing all the rage of new tools in thick row. Yet, still they are all bad so need to make yet another one!
Which make? I’m guessing you mean gnu make, well there’s several incompatible variants as bsd make and nmake. Also bash is not universally used, not windows, not current macos.
For new languages writing build tools in the language gives you convenient access to meta programming capabilities and removes a dependency (like python in your example)
I can't answer myself because I only have basic theoretical idea of what build tools do, I have always mostly been using IDEs like VisualStudio which do everything behind the scenes in a press of a button.